A release is the moment an artifact stops being yours and starts being something other people depend on.
Everything before it can be redone. After it, a version number has been published, somebody has pulled it, and changing what that name means is a breach of an implicit contract. That is why release workflows are worth more rigour than build workflows, even though they are usually less frequently exercised.
The chain
Section titled “The chain”A vertical sequence: a pull request; CI validation; merge to the default branch; a Git tag; a container build; an image digest; an attestation; a registry; and a deployment referencing the digest.
Each arrow is a place identity can be lost. The workflow’s job is to carry it through all of them.
The Git tag is the release decision
Section titled “The Git tag is the release decision”Not the merge. Not a workflow button. A tag.
Why the tag rather than the merge: merging to the default branch means the change is accepted and should be built. Tagging means “this specific state is what we are publishing”. Those are different decisions made by different people at different times, and conflating them means every merge is a release — which forces a team to batch changes in order to control release cadence, producing larger and riskier releases for a reason that has nothing to do with the changes themselves.
Annotated tags, not lightweight. An annotated tag has an author, a date and a message, and it is an object in the repository rather than a bare pointer.
git tag -a v2.4.1 -m "Fix connection pool exhaustion under load"git push origin v2.4.1Tag a commit that is on the default branch. Tagging a branch tip that has not merged produces a release of code that is not in main, which is confusing later and occasionally a genuine incident.
Protect release tags. A tag rule preventing deletion and reassignment turns “we do not move release tags” into something that cannot happen. Without it, a force-pushed tag silently changes what a version means for every consumer, and nothing in their tooling reports it — a rebuild from the moved tag simply produces different content under the same name.
Sign them, if your project’s consumers care about provenance at the source level. Signed commits and tags covers the mechanics.
Build only from tags
Section titled “Build only from tags”The release workflow triggers on a tag and on nothing else.
name: Release
on: push: tags: ['v[0-9]+.[0-9]+.[0-9]+']
permissions: contents: write packages: write id-token: write attestations: write
jobs: release: runs-on: ubuntu-latest steps: - uses: actions/checkout@v7 with: fetch-depth: 0
- name: Verify the tag is on the default branch run: | if ! git merge-base --is-ancestor "$GITHUB_SHA" origin/main; then echo "Tag ${GITHUB_REF_NAME} is not an ancestor of main." >&2 exit 1 fi
- uses: docker/setup-buildx-action@v4
- uses: docker/login-action@v4 with: registry: ghcr.io username: ${{ github.actor }} password: ${{ secrets.GITHUB_TOKEN }}
- uses: docker/metadata-action@v6 id: meta with: images: ghcr.io/${{ github.repository }} tags: | type=semver,pattern=v{{version}} type=semver,pattern={{major}}.{{minor}} type=sha,prefix=sha-,format=long
- uses: docker/build-push-action@v7 id: build with: context: . platforms: linux/amd64,linux/arm64 push: true tags: ${{ steps.meta.outputs.tags }} labels: ${{ steps.meta.outputs.labels }} provenance: true sbom: true
- uses: actions/attest-build-provenance@v4 with: subject-name: ghcr.io/${{ github.repository }} subject-digest: ${{ steps.build.outputs.digest }} push-to-registry: trueThe parts carrying the design:
The tag pattern. Only well-formed version tags trigger a release. A typo in a tag does not publish anything.
fetch-depth: 0. Needed for the ancestry check and for generating notes from history.
The ancestry check. Fails the release if the tagged commit is not reachable from the default branch. Cheap, and it catches a real mistake.
attestations: write. Required for the provenance step, and not a default.
The digest output. steps.build.outputs.digest is what everything downstream should use.
Semantic versioning, applied to images
Section titled “Semantic versioning, applied to images”The version communicates compatibility to whoever consumes the image.
Major — a breaking change for a consumer. A required environment variable added, a configuration format changed, a port changed, a data format that is not backwards compatible.
Minor — new capability, no action required.
Patch — a fix with no interface change.
The category people get wrong is the base image bump. Updating the base is usually a patch — unless it changed the runtime major version, in which case behaviour may differ and it is at least a minor. “We only changed the base image” is not automatically a patch.
The interface of a container image is larger than people assume: environment variables and their defaults, mounted paths, exposed ports, the user and group it runs as, the signals it handles and how it shuts down, the health check it exposes, and its configuration file format. A change to any of them can break a consumer whose code you never see, and none of them appear in the application diff that prompted the release.
Who may release
Section titled “Who may release”A release publishes something under your organisation’s name. The permission to do it deserves more thought than it usually gets.
The tag is the gate. Whoever can push a version tag can release. If tags are unprotected, that is everybody with write access.
Protect the tag pattern. A rule restricting who may create tags matching v* converts the release decision into an explicit permission. This is more effective than controlling the workflow, because the workflow only runs when the tag exists.
Require the underlying commit to have been reviewed. Branch protection on the default branch plus the ancestry check gives you this: a release can only come from a commit that went through review.
Consider requiring a signed tag for anything published externally. It binds the release decision to an identity cryptographically rather than to whoever held a token.
Log it somewhere people see. A release announcement in a channel is not ceremony — it is how somebody notices a release they did not expect, which is the detection mechanism for a compromised token.
The exposure worth understanding: anybody who can push a tag can cause a build with packages: write to run and publish under your namespace. The workflow file itself is the other half of that, which is why .github/workflows/ belongs under CODEOWNERS.
Release notes
Section titled “Release notes”Generated from history, edited by a human.
What to include: what changed from a consumer’s perspective, breaking changes with their migration step, and the image reference.
The image reference belongs in the notes, both forms:
## v2.4.1
**Image**
ghcr.io/example-org/api:v2.4.1 ghcr.io/example-org/api@sha256:0000000000000000000000000000000000000000000000000000000000000000
### Fixed- Connection pool exhaustion under sustained load (#412)
### Changed- Base image updated to node:22.11-slimThe digest in the release notes is what somebody uses to verify what they are running matches what was released, months later, when the tag may have been touched.
Never claim a fix resolves a reported issue unless the commit references it. This is the most common invented claim in generated release notes and it produces support conversations.
Generating the notes
Section titled “Generating the notes”Most of a release note is derivable and the important part is not.
Derivable from history: the commit list, the pull requests merged, the contributors, the file-level scope of the change.
Not derivable: whether a change is breaking for a consumer, what the migration step is, and why the release exists. Those come from the people who made the changes.
The workflow that produces good notes:
Conventional commit prefixes, if your team uses them, let generation group changes by type automatically and infer the version bump. Worth adopting for a repository that releases often; overhead for one that releases twice a year.
A ## Unreleased section in a changelog, maintained as changes merge. Each pull request that changes behaviour adds a line. At release time the section is renamed to the version. This front-loads the work to the moment the author actually knows what they changed, which is the only moment anybody does.
Generation as a draft, always. The pattern from Pillar 6 applies directly: generate the mechanical structure, have a human write the parts that require knowing why, and never publish unedited. A generated note that claims a fix resolves an issue the commit does not reference is the specific failure to watch for.
Draft releases exist for this. Publish as a draft, edit, then release. It costs one click and it is the difference between release notes people read and release notes people learned to ignore.
Write for somebody deciding whether to upgrade. That is the audience. “Refactored the connection handling” tells them nothing; “fixes connection pool exhaustion under sustained load — upgrade if you have seen timeouts under peak traffic” tells them exactly what they need.
Immutability, end to end
Section titled “Immutability, end to end”The property the whole workflow exists to produce.
| Artifact | Immutable? | Enforced by |
|---|---|---|
| Git commit | Yes | Content hashing |
| Git tag | By convention | Tag protection rules |
| Image digest | Yes | Content hashing |
| Image tag | No | Registry policy, where available |
| Attestation | Yes | Signature over the digest |
| Release notes | No | Nothing — edit freely |
The two soft rows are where discipline is required. A Git tag and an image tag are both mutable by default, and both are things consumers depend on. Protection rules on the first and registry immutability on the second, where available, close the gap.
What gets verified before a tag is pushed
Section titled “What gets verified before a tag is pushed”A release workflow that only runs on the tag is running validation too late — the tag has already been pushed and the version number is spent if something fails.
Everything should already have passed on the merge commit. The release build is a re-build of a commit whose tests, scans and reviews are complete. If the release workflow is the first place a scanner runs, you have arranged for failures to happen at the worst moment.
What the release workflow adds beyond the merge build: the multi-platform build, the version tags, the attestations, and publishing. Not new validation.
The checks worth running in the release workflow anyway, because they are release-specific:
The ancestry check shown above.
A version format check — the tag pattern does most of this, and asserting the version matches what is declared in the project’s manifest catches the mismatch where somebody tagged v2.4.1 against a commit whose package.json says 2.4.0.
A “does this version already exist” check. Query the registry before building. A release that overwrites an existing version should fail loudly rather than succeed quietly.
A changelog presence check, if you maintain one. A release with no entry is a release nobody described.
What to do when the release fails halfway. The image may be published; the attestation may not be. Do not re-run into the same version. Delete nothing, publish v2.4.2, and note in its release that v2.4.1 was withdrawn. Version numbers are cheap; ambiguity about what a version means is not.
Promotion after release
Section titled “Promotion after release”A published release is not a deployed one, and the separation is deliberate.
Publishing makes the artifact available.
Promotion moves it toward production through environments, by digest, via pull requests in a deployment repository.
Keeping them separate means a bad release can be published and never promoted, which is a much better outcome than a bad release automatically reaching production because the pipeline was one continuous run.
The digest is what travels. Not the version tag — the version is how a human refers to it, and the digest is what is deployed. This is what makes “we tested exactly this” true.
Pre-releases
Section titled “Pre-releases”For anything consumers install, a pre-release channel is worth having.
git tag -a v2.5.0-rc.1 -m "Release candidate 1 for 2.5.0"Adjust the trigger to match pre-release patterns, and do not apply the rolling 2.5 or latest tags to them — a consumer tracking a minor version should not receive a release candidate.
The value is real for images other teams or external users consume, because it gives them a way to test an upcoming version without it reaching anybody who did not opt in. For an internal service deployed only by you, pre-releases are usually ceremony; the staging environment already serves that purpose and a release candidate adds a version number without adding information.
Versioning several images together
Section titled “Versioning several images together”A repository producing three images has a decision most teams make implicitly.
One version for all of them. A single v2.4.1 tag builds and publishes api, worker and web at the same version. Simple, and it means a consumer knows those three were built together and tested together. The cost is meaningless version bumps — worker goes to 2.4.1 having not changed.
Independent versions per image. Tags become api/v2.4.1 and worker/v1.8.0. Each version means something. The cost is that nothing records which combination was tested together, and for services that must be deployed as a set that is a real loss.
The question that decides it: must these be deployed together? If a version of api only works with a matching worker, one shared version is honest and independent versions are a fiction that will cause an incident. If they are genuinely independent services that happen to share a repository, independent versions are more informative.
The middle path that works for coupled services: shared version, and release notes that say which images actually changed. Consumers get the coupling guarantee, and the notes prevent confusion about why worker has a new version.
Whichever you choose, be consistent, and write it in the README. A repository where some images follow one scheme and some the other is one where nobody can predict what a tag will produce.
Consuming a release
Section titled “Consuming a release”Worth covering from the other side, because a release exists to be consumed.
What a consumer needs from you: a version, a digest, release notes describing what changed, and a stable answer to what the image’s interface is.
What a consumer should do: pin the digest, record the version alongside it in a comment, and upgrade deliberately through a reviewed change. This is the module upgrade pattern applied to images, and it has the same properties.
What breaks consumers most often, in order: a required environment variable added without a default; a change to the user the container runs as, which breaks volume permissions; a changed exposed port; and a base image change that alters the shell or the available utilities their init scripts relied on.
Every item on that list is invisible in the application’s own diff, which is why the release notes are the place they must appear. A consumer cannot read your source to find out that the container now runs as UID 1001.
Verify the release before announcing it. Pull the published image by digest, run it, and check it starts. A release nobody has run is a hypothesis, and the workflow that built it does not necessarily prove the artifact works — a build can succeed and produce an image with a broken entrypoint.
Common mistakes
Section titled “Common mistakes”Releasing on merge rather than on a tag. Every merge becomes a release, so cadence is controlled by batching.
Lightweight tags. No author, no date, no message.
Tagging a commit not on the default branch. A release of code that is not in main.
Re-using a version after a failed release. The same name means two things.
No tag protection. A force-pushed tag silently changes a published version.
Deploying the version tag rather than the digest. Reintroduces mutability at the last step.
No provenance. “It came from our repository” stays an assumption.
Calling a base image bump a patch automatically. Sometimes it changes behaviour.
Publishing release notes claiming fixes the commits do not reference. Support conversations.
No ancestry check. Releases from unmerged branches.
Withdrawing a release
Section titled “Withdrawing a release”Rare, unpleasant, and worth having decided in advance.
Deleting a published version is almost always wrong. Somebody has pulled it. A deployment references its digest. Deleting it means their next node restart fails to pull, and you have converted your problem into their outage.
Publish a fix as a new version instead. v2.4.2 superseding v2.4.1, with release notes on both saying so.
Mark the bad version clearly. Edit its release notes to say it is withdrawn and why, and what to move to. The release notes are mutable for exactly this reason.
Move the rolling tags off it. If 2.4 and latest point at the bad version, re-point them at the good one. This is the legitimate use of mutable tags and the reason they exist.
Delete only when it is actively dangerous — a leaked credential baked into the image, or a version that will cause data loss. Then the calculus changes: the harm from it being pullable exceeds the harm from breaking pulls. Announce it before deleting, not after.
If a credential leaked into an image, deletion is not the fix. The credential has been distributed. Rotate it first, then decide about the image. This is the same ordering as committed Terraform state, for the same reason.
Record what happened. A withdrawn release with no explanation generates the same question repeatedly for years.
The release checklist
Section titled “The release checklist”Condensed, for a team writing its own.
Before tagging
- The commit is merged to the default branch and CI passed on it
- The version does not already exist in the registry
- The changelog has an entry describing consumer-visible changes
- Any breaking change has a documented migration step
At tag time
- Annotated tag, correct semantic version, on a merged commit
- Tag pattern matches what the release workflow triggers on
In the workflow
- Ancestry check passes
- Build succeeds for every supported platform
- Version tags and the full SHA tag are applied
- Provenance and SBOM attestations are produced and pushed
After publishing
- Pull the published image by digest and confirm it starts
- Release notes contain both the tag and the digest
- Rolling tags point where you intend
- Announce it where consumers will see it
Ten minutes, most of it automated, and each item corresponds to a failure this lesson has described. A team that runs it consistently has a release process; one that does most of it most of the time has a habit.
Mental model
Section titled “Mental model”A release is a promise that a name means a specific thing forever. The workflow’s job is to make that promise keepable: an immutable commit, a protected tag, an immutable digest, and a signed statement connecting them.
Everything mutable in that chain is a place the promise can be broken, which is why the protection rules matter more than any individual step.
What you learned
Section titled “What you learned”- The Git tag is the release decision, distinct from the merge
- Use annotated, protected tags, and verify the tagged commit is on the default branch
- Release workflows trigger on version tag patterns only
- A container image’s interface includes environment variables, paths, ports, user and signals
- Put both the tag and the digest in the release notes
- A failed release gets a new version number, never a re-used one
- Publishing and promotion are separate; the digest is what gets promoted
- Provenance attestations turn “it came from our repository” into something checkable
Exercise
Section titled “Exercise”Use a disposable repository under an account you control.
-
Add a release workflow triggering only on
v[0-9]+.[0-9]+.[0-9]+. Push a malformed tag likev2.4. Predict: does it run? -
Push
v0.1.0. Confirm the image is published and note the digest. -
Add the ancestry check. Tag a commit on a branch that is not merged and push it. Predict: does the release fail?
-
Add
provenance: trueand the attestation step. Pushv0.2.0and inspect what attaches to the image. -
Try to delete and re-push
v0.2.0pointing at a different commit. Predict: what happens with and without a tag protection rule? -
Write release notes containing both the tag reference and the digest. Ask somebody to verify a pulled image matches. Predict: which reference do they need?
-
Delete the repository and the packages.
Related lessons
Section titled “Related lessons”The GitOps and infrastructure repository templates are in the Professional Toolkit.