Skip to main content

Release Management

TODO

This article is a work in progress.

Release Management Pipeline​

The following diagram illustrates the release management pipeline as a sequence of events where different acting teams, disclipines, components or audiences interact.

See Terminology & Acronyms and please excuse the brevity (lack of real estate).

Terminology & Acronyms​

Development​

Development includes all work on source code, and encapsulates upstream AOSP, Chipset Manufacturer, ODM and Graphene changes, as well as Apostrophy original source code.

The diagram specifically does not include actors and interaction lines for many development interactions since this is a release management pipeline diagram and we can only zoom in to dense diagrams to an extent.

NEXT​

NEXT reflects the name for the branch name used in our platform manifest, which in turn describes the exact source code revisions to use for the ~2'000 contributory constituent repositories.

Inclusion in to the NEXT branch means the resolution to a topic is going to be included in the sub- sequent weekly development release.

QA​

Stands for Quality Assurance. Various participants in this discipline double in development, and teams outside of the scope of this diagram such as Support and Customer Care.

Quality Assurance will typically involve a variety of forms of testing against both Engineering Sample devices/releases as well as production devices/releases.

note

Quality assurance includes acceptance testing, with resulting feedback ranging from "resolution OK" to "resolution provided, but also (...)" and "resolution not provided".

QA also includes regression testing, which intends to ensure there's no unintended consequences to the changes applied in development.

It is devices in the hands of the Quality Assurance team members that get assigned OTA updates before any OTA update progresses to become published.

MASTER​

MASTER reflects the (current) name for the branch name used in our platform manifest, which in turn describes the exact source code revisions to use for the ~2'000 contributory consistituent repositories.

While the NEXT branch progresses weekly, MASTER progresses monthly, and may or may not include all inclusions in to NEXT from previous weeks (but typically does).

OTA​

note

Since there is only one piece of OTA infrastructure, this is considered production, but availability of an update in this territory does not mean it is a) published nor b) targeted at production at all.

Production​

The fleet of devices that is in the hands of end-users.

Topics​

In this diagram, topics are used to describe different types of Phorge tickets. Typically, each cycle a number of tickets get resolved, but we haven't the veritical space to accomodate even an example.

1. Production Feedback​

Feedback from end-users based on production releases. This typically progresses through OEM customer care, or Apostrophy support channels, on to the Apostrophy work board in the form of a ticket.

2. Quality Assurance Feedback​

Feedback from Quality Assurance based on weekly development releases. This typically originate from Weekly NEXT Release OTA assignments or publications.

3. Resolution from Development, Topic B​

The development team delivers the resolution to Topic B and submits it to NEXT for inclusion in to the next weekly release.

To illustrate that topics do not necessarily get resolved in the order in which they come in, Topic B gets resolved before Topic A.

4. Request for Enhancement, new Topic C​

Certain topics do not originate with Quality Assurance or production end-users. To avoid introducing Yet Another Column, we're going to pretend Topic C originated with a desire to include the resolution in NEXT, from NEXT.

5. Resolution from Development, Topic A​

Despite Topic B getting resolved first, resolution of Topic A may just still make it in time for inclusion on to the next weekly release.

6. Production Feedback, new Topic D​

Without having upgraded any release, a new topic is raised from among production release end-users.

7. Weekly NEXT Release #1​

If we started on day 1, then the reviews for inclusion in to this week's NEXT release are going to be available for review and discussion are to be available on day 6, and the NEXT release to be built on day 7.

This weekly release, labeled as NEXT, should occur on day 7.

We expect the developer's rolling changelog to be updated with a mention of every change applied.

8. Resolution from Development, Topic D​

To illustrate that development does not just stop because we're in the middle of a weekly release cycle, Development pushes the resolution to Topic D while the weekly release has not yet been provided.

9. OTA Weekly NEXT #1​

As the weekly release has been built, we expect the following to become available:

  • An OTA package from the second-to-last weekly NEXT release (a .U0 if you will)

  • An OTA package from the second-to-last weekly NEXT release critical path OTA testing result (a .U1 if you will)

  • An OTA package for each of the last weekly NEXT release (U0) and its OTA critical path result (U1), from those releases to this release

These OTA packages are not published (yet, see #13), rather assigned to individual devices (through their Controller IDs), that are in the hands of Quality Assurance actors.

10. Critical Path OTA Weekly NEXT #1​

In addition to the OTA update packages from past weekly NEXT releases, another package is made available to validate the resulting weekly release is able to OTA.

11. Topic A Acceptance Feedback​

Quality Assurance actors provide their feedback on Topic A.

12. Topic B Acceptance Feedback​

Quality Assurance actors provide their feedback on Topic B.

13. Publish OTA Weekly NEXT #1​

This is an optional and/or conditional step.

Depending on the judgement of Quality Assurance efforts, the weekly release is either accepted or declined for general availability to all devices running a prior version of NEXT releases.

Clearly, a lack of acceptance should correspond to a number of topics raised that are agreed upon to be blockers for the specific release when compared to its previous release.

14-25. Repeat NEXT Release​

From here we just repeat Weekly NEXT Releases with additional work, feedback and OTA assignments and/or publications.

We should note that the number of iterations is dynamic, such as monthly cycles not aligning with weekly cycles, and other months overlapping with personnel absences reducing the effective number of cycles to perhaps as little as 2.

  • Topics are being worked on.

  • What is proposed to include in to the NEXT release is reviewed and accepted.

  • What is included is built and documented and made available.

26-29. Pre-Prod Weekly NEXT Release​

This weekly NEXT release bears pointing out, because it is special.

We should consider this release the last weekly release aimed at ultimately general acceptance toward a monthly release.

30. Merge NEXT with MASTER​

After a number of weekly cycles, and having available the last weekly NEXT release that is for all intents and purposes the same (source code and build process) as the production release candidate, a push, merge or cherry-pick decision is made from NEXT to MASTER.

This process results in another build, a few OTA packages, and is made available in the form of a download, a composed set of release notes (from the rolling changelog), an SBOM, an Aphyum report, a CTS report and a VTS report.

31. OTA Monthly Production​

At first this monthly production release candidate is assigned to become available to certain devices, which probably are in the possession of Quality Assurance actors, much like would be the case for weekly NEXT releases.

32. Critical Path OTA Monthly Production​

Similarly, select devices get assigned the Critical Path OTA update package, to validate that what we would release to the production end-users can still update itself after the application of the OTA package we are proposing gets published.

33. Publish OTA Monthly Production​

The publication of an OTA package to the end-users' devices.

34. Roll-Out Production OTA​

TBD.

Aphyum Testing​

Aphyum is an Appium-based test suite Apostrophy uses to validate expectations it makes of the platform such as for example;

  • defaults including NFC off, Bluetooth off, Location off, no QR QS tile, etc.

  • our Onboarding (aka. SetupWizard) functioning for a variety of flows,

  • many applications we ship functioning,

  • a large variety of lower-level aspects not specific to Apostrophy or the platform used.

CTS Testing​

TBD

VTS Testing​

TBD

SBOM​

TBD

MobSF Reports​

TBD