Skip to content

Release Process (2.3.z)

Jump to bottom
Hamzah Qudsi edited this page Nov 3, 2022 · 2 revisions

Discussion

The ideal

The platonic ideal of the release process is that the release engineer identifies a commit that they would like to promote to to a release (either RC or GA) and creates and pushes vSEMVER and chart/vSEMVER Git tags on that commit:

version=2.2.0-rc.3
chart_version=7.3.0-rc.3
git tag --annotate --message="Emissary ${version} release" "v${version}"
git tag --annotate --message="Emissary Helm chart ${chart_version} release for Emissary ${version}" "chart/v${chart_version}"
git push origin "v${version}" "chart/v${chart_version}"

The reality

Truthfully, we are quite close to that platonic ideal. However, it is often (always) the case that the release engineer needs to make some commits as part of the release process.

It would be good if this were not true and no edits were necessary, and we actively seek to minimize the edits made. However, unless we recruit a psychic to be an Emissary maintainer, it is unlikely that we will ever be able to remove the step where the release engineer needs to fill in the release date in the release notes.

If no edits were necessary, the release engineer could just tag a commit that is already on master or release/vX.Y; however, because edits are necessary, the release engineer should create a rel/vX.Y.Z branch for the release; that way they can push whatever they need without needing to worry about the branch protection on master and release/v*. The name rel/vX.Y.Z is just a convention; none of the tooling relies on that name. When the release engineer creates this branch, they should also create a PR for it titled "[vX.Y.Z] Release PR".

When pushing to rel/vX.Y.Z, the release engineer should be conscientious that their changes will not undergo code-review before they create the actual release, and will only undergo minimal code-review when the "[vX.Y.Z] Release PR" is merged.

Gotchas / subtleties

Git has two types of tags, "lightweight tags" and "annotated tags". Lightweight tags are just a ref named refs/tags/${tagname} that point directly at "type=commit" object. Annotated tags are also a refs/tags/${tagname} ref, but rather than pointing directly at a commit, the ref points at a special "type=tag" object; that object contains a tagger name/email, a timestamp, a tag message, a commit hash that it points at, and optionally a GPG signature. For Emissary, we prefer to use annotated tags for releases, so that we have a record of who ran the release and when it was performed.

In the shell snippet for the platonic ideal pushes both the Emissary tag and the chart tag in one command: git push origin "v${version}" "chart/v${chart_version}". This is because it is important that both tags exist before CI for either tag starts; that's just the way the CI scripts are written. If you push one tag, wait a few seconds, then push the other: you're going to have a bad time.

You can't pick an arbitrary commit from the Git history; the job-promote-to-passed / build GitHub Action must have successfully run on whichever commit you tag.

Prerequisites

The release engineer must have access to push branches and tags to emissary.git. If anything else is needed, that's a bug.

Checklist

Emissary + chart

RC

  1. Create a rel/vX.Y.Z branch for the release; if you've already created such a branch for a previous RC, then ensure that you're on it.
  2. Look over the release notes (docs/releaseNotes.yml and charts/emissary-ingress/CHANGELOG.md), and edit and commit them as necessary. In particular:
    • Note that both files should contain GA version numbers (no -rc.N suffix).
    • Ensure that releaseNotes.yml reflects the anticipated GA release date.
    • Ensure that the notes reflect the changes you intend to be releasing. You are encouraged to look over the commit log to validate that the release notes are in agreement with the actual changes.
    • Proof-read the entries to make sure that they are written with consistent tone, are ordered in a sane way, et c.
    • If you edit releaseNotes.yml, remember to run make generate-fast (or rm CHANGELOG.md && make $PWD/CHANGELOG.md).
  3. Push that.
  4. (only for the first RC) File a PR for rel/vX.Y.Z. Title it "[vX.Y.Z] Release PR". Feel free to have an empty body. Make sure that you're targeting the appropriate master or release/vX.Y branch.
  5. Wait for the job-promote-to-passed / build GitHub Action to pass on the commit that you pushed. You do not need to wait for any of the tests to pass (if your changes were anything but trivial, then you probably should wait for the tests to pass, but if they're anything but trivial really you should have PRed them separately in order to subject them to code-review, rather than sneaking them in directly on the rel/vX.Y.Z branch).
  6. Create and push Git tags for both Emissary itself and for the chart: 
    version=2.2.0-rc.3
chart_version=7.3.0-rc.3
git tag --annotate --message="Emissary ${version} release candidate" "v${version}"
git tag --annotate --message="Emissary Helm chart ${chart_version} release candidate for Emissary ${version}" "chart/v${chart_version}"
git push origin "v${version}" "chart/v${chart_version}"
  7. That's it! The tags will kick off 2 CI workflows: promote-rc and publish-chart. Both workflows should post in #team-run on datawire.slack.com when they're done, so that folks know that they can begin smoke-testing. Or, if the jobs fail, then they'll post saying that they failed.

GA

Not included in this checklist is smoke-testing, getting the go-ahead from leadership, any coordination with the go-to-market team... this checklist just deals with the mechanics of doing the thing once everyone is ready for it.

  1. On a commit that was previously tagged/promoted to an RC (see above), create and push GA Emissary and chart Git tags: 
    version=2.2.0
chart_version=7.3.0
git tag --annotate --message="Emissary ${version} GA release" "v${version}"
git tag --annotate --message="Emissary Helm chart ${chart_version} GA release for Emissary ${version}" "chart/v${chart_version}"
git push origin "v${version}" "chart/v${chart_version}"
  2. This will kick off 2 CI workflows: promote-ga and publish-chart. Both workflows should post in #team-run on datawire.slack.com when they're done, so that folks know that the release is complete. Or, if the jobs fail, then they'll post saying that they failed.
  3. Because our automation is bad, it's necessary for you to manually run ./releng/release-mirror-images --ga-version=2.2.0. CI should eventually be taught to do this.
  4. You probably have a getambassador.io.git docs PR that you need to go merge.
  5. Merge the "[vX.Y.Z] Release PR".

Chart without Emissary

This doesn't work right now. The machinery won't know which Emissary version the chart should refer to; it would publish a chart containing a dev Emissary version.

Clone this wiki locally