A release is a lot of reading and writing wrapped around one decision.
The reading: what is in this, what changed, what might break, what does an operator need to know. The writing: release notes, upgrade guidance, an announcement, and afterwards an incident timeline if it went badly. The decision: ship or do not.
AI is genuinely strong at the reading and writing. It has no business anywhere near the decision, and this lesson is largely about keeping that line clear while getting everything on the other side of it.
What ships is a human decision
Section titled “What ships is a human decision”Stated first, because everything else depends on it.
The gates that decide a release are deterministic: tests pass, scans are clean, dependencies are not vulnerable, a human approved the change, a human approved the deployment, the deploy window is open, the previous release is healthy.
AI produces the material an approver reads. A summary. A risk list. A note about what an operator should watch. All of it information, delivered to a person who then decides.
The failure to guard against is subtle and common: an approver who reads the generated summary instead of the change. Nothing was configured wrongly. The summary quietly became the gate, and the approval became a formality.
The release cycle
Section titled “The release cycle”A vertical sequence: the commit range is determined; a change summary is generated as advisory material; risk factors are surfaced advisorily; deterministic checks run and block; a human approves the release; release notes are drafted and edited by a person; the release is published; and post-release monitoring uses measured signals.
Three of the eight stages are optional and advisory. Remove all three and the release still happens correctly — which is the test for whether you have layered this properly.
Change summaries for the approver
Section titled “Change summaries for the approver”The highest-value application, because the approver’s alternative is reading 200 commits or reading nothing.
What a good summary contains:
What changed, grouped by area. Not a commit list — a grouping. “Three changes to the billing service, one to auth middleware, eleven dependency updates.”
What an operator needs to know. New environment variables, changed defaults, anything requiring action at deploy time.
What is notable rather than what is large. A one-line change to a rate limit matters more than a 400-line refactor, and a summary that ranks by diff size buries it.
What it should not contain:
A verdict. “This release is safe to deploy” is not a summary.
Claims about testing. “Well tested” written by something that did not run the tests.
Speculation about intent. If a commit’s purpose is unclear, say so rather than constructing one.
The prompt structure that produces this:
Summarise the changes in <range> for somebody deciding whether to approvethis release.
Group by service or area. Within each group, lead with anything thatchanges behaviour an operator or consumer would notice.
Call out separately, if present:- Database migrations- Changes to authentication or authorisation- New or changed environment variables and configuration defaults- Changes to public API contracts- Dependency updates crossing a major version
State facts from the commits and diffs only. Do not assess whether therelease is safe. Do not claim anything about test coverage. If a commit'spurpose is not clear from its message and diff, list it under "unclear"rather than inferring one.The final paragraph does most of the work. Without it, summaries drift toward reassurance, and a reassuring summary is the thing most likely to substitute for reading the change.
Risk surfacing
Section titled “Risk surfacing”Distinct from risk assessment, and the distinction is the whole point.
Surfacing produces a list of factors: this release contains a migration, touches auth, changes a public contract, includes a major dependency bump.
Assessment produces a judgement: this release is low-risk.
The first is useful and largely mechanical — most of those factors can be detected deterministically from paths and diffs, and where they can, they should be. The second is what you must not automate.
Detect deterministically what you can:
| Factor | Deterministic detection |
|---|---|
| Migration present | Path glob on the migrations directory |
| Auth code changed | Path glob |
| Public API changed | Diff against the API surface, or a contract test |
| Major dependency bump | Version comparison in the lockfile diff |
| Large change | Line counts |
| Configuration default changed | Diff on the configuration files |
Use AI for what the globs cannot see: whether a change to a shared utility affects several services, whether two changes in the release interact, whether a rate-limit adjustment has an obvious consumer impact. Reasoning across the diff, presented as observations for a person.
Express policy as rules, not as judgements. “A release containing a migration requires a second approver” is a deterministic rule keyed off a deterministic detection. That is a risk gate, and it is what teams actually want when they reach for an AI risk score.
The workflow, in full
Section titled “The workflow, in full”Putting the advisory pieces into a tag-triggered workflow.
name: Release material
on: push: tags: ['v*']
permissions: contents: write
jobs: prepare: runs-on: ubuntu-latest timeout-minutes: 20 steps: - uses: actions/checkout@v7 with: fetch-depth: 0
- name: Determine the range id: range run: | current="${GITHUB_REF_NAME}" previous="$(git tag --sort=-v:refname | grep -v "^${current}$" | head -n 1)" echo "range=${previous}..${current}" >> "$GITHUB_OUTPUT" echo "Range: ${previous}..${current}"
- name: Deterministic risk factors id: factors run: | range="${{ steps.range.outputs.range }}" files="$(git diff --name-only "$range")" { printf '%s\n' "$files" | grep -q '^db/migrations/' && echo "- Contains a database migration" printf '%s\n' "$files" | grep -q '^src/auth/' && echo "- Touches authentication code" printf '%s\n' "$files" | grep -q '^src/api/' && echo "- Touches the public API surface" printf '%s\n' "$files" | grep -q 'package-lock.json' && echo "- Includes dependency changes" } > /tmp/factors.md || true cat /tmp/factors.md
- name: Change summary run: | git log "${{ steps.range.outputs.range }}" --format='%h %s%n%b' > /tmp/commits.txt git diff --stat "${{ steps.range.outputs.range }}" > /tmp/stat.txt copilot -p "Read /tmp/commits.txt and /tmp/stat.txt. Summarise for somebody deciding whether to approve this release. Group by area. State facts from the commits only. Do not assess safety. Do not claim anything about testing. List unclear commits under 'unclear'." \ --allow-tool='shell(cat)' \ > /tmp/summary.md
- name: Publish a draft release env: GH_TOKEN: ${{ github.token }} run: | cat /tmp/factors.md /tmp/summary.md > /tmp/body.md gh release create "${GITHUB_REF_NAME}" \ --draft \ --title "${GITHUB_REF_NAME}" \ --notes-file /tmp/body.mdThe properties worth noting:
The range is computed deterministically. Tag ordering, not interpretation.
Risk factors come from path globs, run before the model and independent of it. Four grep calls,
correct every time.
The summary is a separate step producing prose, and it reads only files.
--draft. Nothing is public until a person publishes it, which is the single most important line in
the file.
contents: write is the only permission, and it is there for the draft release. No deployment
credentials, no cloud access, nothing that reaches production.
What is deliberately absent: any step that decides. The workflow prepares material and stops. The decision to publish the draft, and the decision to deploy, are made by people looking at what it prepared.
Release notes
Section titled “Release notes”Covered in depth in AI release notes; the release-engineering additions are about where the generation sits.
Generate from the tag range, in a workflow triggered by the tag. The range is deterministic and the input is stable.
Publish as a draft. A release note is a public artefact, and a person should read it before it is public. Draft releases exist for this.
Separate audiences. Consumers want behaviour changes and upgrade steps. Operators want migrations, configuration and rollout notes. One document serving both serves neither well.
Never claim a fix resolves a reported issue unless the commit references it. This is the most common invented claim in generated release notes, and it is the one that causes support conversations.
Link everything. Each entry to its pull request. That makes the note checkable, and checkable is what distinguishes a useful note from a plausible one.
Comparing two releases
Section titled “Comparing two releases”A question that comes up constantly during an incident and is tedious to answer: what is different between what is running and what was running before?
The deterministic part. git diff --stat <deployed>..<previous>, the commit list, the list of
changed services. Fast, exact, and where any answer should start.
Where the reading helps. Turning that into “the deploy at 14:20 changed the retry behaviour in the payment client and added a database index; nothing else touched the payments path” — which is the form somebody debugging actually needs, and which requires reading forty commits to produce.
Ask the right question. “What changed” produces a list. “What changed that could affect the payments path” produces something useful, and the difference is entirely in the question.
The rule during an incident. Use it for orientation, never for conclusion. “These three changes touch the affected area” narrows where to look. “This change caused the incident” is a hypothesis to test, and treating it as an answer sends the whole response down one path early — which is the classic way an incident takes three hours instead of forty minutes.
The rollback decision stays deterministic and human. Error rate, latency, health checks — measured signals and a person deciding. Not an interpretation of a diff, however plausible.
Where this pays off most: a release containing forty commits from six people, at 02:00, with the person who wrote the relevant change asleep. Reading the diff yourself is possible and slow. Having the relevant three changes surfaced in thirty seconds, and then reading those three yourself, is the workflow that actually helps.
Upgrade guides
Section titled “Upgrade guides”The place generation is weakest and the effort is highest, which makes it worth being explicit about.
What it can draft: the mechanical steps. New configuration to set, renamed options, changed defaults, the order of operations for a migration.
What it cannot know: why the change was made, what breaks in an unusual deployment, what the workaround is for consumers who cannot upgrade yet. That knowledge is in the heads of the people who made the change.
The workflow that works: generate the mechanical skeleton from the diff, then have the person who made the breaking change fill in the reasoning and the migration path. The draft removes the blank page, which is the part people avoid, and the expert supplies the part only they have.
The check before publishing: follow the guide yourself, on a clean environment. A generated upgrade guide that has not been executed once is a hypothesis.
Incident timelines
Section titled “Incident timelines”The application most valuable at the moment nobody has capacity for it.
During an incident, do not. Everyone is busy and a generated timeline is a distraction. The exception is a genuinely mechanical query — “which deploy went out at 14:20” — which is a lookup.
Afterwards, generate the skeleton. Commits, deploys, alerts, pull request merges, workflow runs, in one ordered list with timestamps. That is a tedious cross-system reconstruction and it is exactly the kind of thing to automate.
A skeleton is not a timeline. The timeline has causation in it: what somebody believed at 14:30, why they tried the thing that did not work, when the diagnosis changed. None of that is in any system, and it is the part the post-mortem is actually about.
Never generate the analysis. Contributing factors, what would have prevented it, what to change. Those are conclusions the people involved must reach, and a generated version becomes the version that gets discussed — which quietly removes the reflection that was the point.
The honest framing: it fills in the times and the sequence so people can spend their meeting on the reasoning rather than on reconstructing what happened. That is a real contribution and a limited one.
Communicating a release
Section titled “Communicating a release”The outputs that go outside the team, where accuracy matters most and verification is hardest — because the audience cannot check.
The announcement. Shorter than the release notes, aimed at people who will not read them. What is new, what needs action, where the detail is. Generating a draft from the notes works well; the note is already structured, and the announcement is a compression of it.
Customer-facing descriptions need a different voice. Release notes are written for engineers. A customer-facing changelog is written for people who do not know your architecture, and translating one into the other is a task where a draft genuinely saves time — provided somebody who knows the product reads it.
Deprecation notices are the highest-stakes case. A notice announcing a removal date, a replacement and a migration path is something people plan around. Every fact in it must be verified: the date, the replacement’s actual availability, whether the migration path has been tested. A generated deprecation notice with a plausible but wrong replacement causes work for every consumer who acts on it.
Never generate a security advisory. The severity, the affected versions, the exploitability and the mitigation are judgements with consequences, and the audience acts on them immediately. Advisories are written by people who understand the vulnerability. See Pillar 5 on advisories.
The rule for everything external: somebody who could be contradicted by a customer must read it before it goes out. That is a lower bar than it sounds and it catches the entire class of confident invention, because the person reading it knows which claims they would not want to defend.
Pre-release checks a person should still do
Section titled “Pre-release checks a person should still do”Generation makes the reading faster. It does not remove the short list of things somebody looks at directly, and it is worth writing that list down so it does not quietly get delegated to the summary.
The migration, if there is one. Read it. Is it reversible? Does it lock a large table? Does it need to run before or after the deploy? This is the single most common source of a bad release and it is five minutes of reading.
Anything in the auth path. A change to authentication or authorisation gets read by a person regardless of how routine it looks.
Configuration defaults. A changed default applies to every environment that has not overridden it, which is usually the ones nobody is thinking about.
The dependency diff, for anything crossing a major version. The summary can tell you it happened. Only reading the changelog tells you what it means.
Whether the previous release is healthy. Shipping on top of a release that is still settling compounds two problems into one incident with two causes.
Who is available. Not in any diff. A release going out when the person who wrote the risky part is on holiday is a different decision from the same release on a Tuesday morning.
The list is short deliberately. Six items, most of which do not apply to most releases. The point of the generated summary is to tell you which of the six are in play, so that the ten minutes of human attention goes to the right two.
The verification chain, applied to a release
Section titled “The verification chain, applied to a release”The same chain as everywhere in this pillar, at release scale:
Generated material → checked against the diff → checked by the people who made the changes → approved by a human → published
Checked against the diff. Every claim in a summary or note should be traceable to a commit. The linking requirement enforces this by making it visible when something is not.
Checked by the people who made the changes. An author reading the line about their own change catches a mischaracterisation in seconds. Nobody else will.
Approved by a human. The decision, on the deterministic gates plus everything the material told them.
Published. After somebody read it.
Skip any link and the failure is the same: a confident public document nobody verified, and release notes are read by people who cannot check them.
Release notes for an audience that includes agents
Section titled “Release notes for an audience that includes agents”A recent and slightly odd consideration: your release notes are increasingly read by models, because consumers of your library run agents that read changelogs when upgrading.
That does not change what a good release note is. Accurate, specific, linked, and organised by behaviour rather than by implementation. Everything that makes a note useful to a person makes it useful to something reading on their behalf.
Two things it does change:
Structure matters more. A note with consistent headings — Breaking, Features, Fixes, Internal — is parseable. One where the sections vary by release is not, and a consumer’s tooling gets it wrong in ways nobody will report to you.
Machine-readable breaking-change markers earn their place. A BREAKING: prefix, or a structured
field, makes the most consequential category unambiguous. This is the same argument as
AI commit messages, one level up.
What has not changed: the accuracy requirement, which if anything is stricter. A wrong claim in a release note now propagates through automated upgrade attempts rather than being caught by somebody reading sceptically. The verification chain is the answer, and it is the same chain.
What to resist: writing release notes primarily for automated consumption, at the cost of them being readable. The person upgrading at 16:00 on a Friday is still the primary audience, and a note optimised for parsing over comprehension serves them worse for a benefit that is largely hypothetical.
Common mistakes
Section titled “Common mistakes”Gating a release on an AI risk assessment. The most damaging pattern in this lesson.
An approver reading the summary instead of the change. The gate moved without anybody moving it.
Publishing generated release notes unedited. They are a public artefact.
Claiming a fix resolves an issue the commit does not reference. The commonest invented claim.
Generating the incident analysis. It becomes the version that gets discussed.
Detecting risk factors with a model when a path glob would do. Slower, more expensive, less reliable.
An upgrade guide nobody executed. A hypothesis presented as instructions.
One document for consumers and operators. Serves neither.
Summaries that rank by diff size. Buries the one-line change that matters.
Measuring the release process
Section titled “Measuring the release process”If you add generation around releases, it is worth knowing whether it helped, and the tempting metric is the wrong one.
The wrong metric: time to produce release notes. It measures how fast a document appeared. It says nothing about whether the document was accurate or whether anybody read it.
Approver-reported usefulness. Ask the people approving releases whether the summary changed what they looked at. A summary that sends an approver to a specific part of the diff is doing its job; one that replaces the diff is doing the opposite, and the approvers know which is happening.
Corrections at review. How much of the generated release note gets edited before publishing? A note published unedited is either excellent or unread — check a few against the commits to find out which. Consistent edits in the same category tell you what to add to the prompt.
Support questions about the release notes. A note that generates “what does this mean” questions was written for the wrong audience. This is the clearest external feedback available and it arrives without being asked for.
Incidents traced to something in a release that nobody noticed. The failure the summary was supposed to prevent. Rare, and worth reviewing in full when it happens: was the factor in the summary and skipped, absent from the summary, or not detectable from the diff at all? Each has a different fix, and only the second is a prompt problem.
What not to measure: anything that rewards the summary being longer, more confident, or produced faster. Those all move in the wrong direction.
Mental model
Section titled “Mental model”AI reads the release. People decide it.
The reading is genuinely hard and genuinely automatable: 200 commits across four services, condensed into something an approver can hold in their head. That is a real contribution to a task people currently do badly because it is tedious.
The deciding is a judgement about risk, timing, who is on call and what else is in flight. It is made by somebody accountable for the outcome, on deterministic evidence plus whatever the reading gave them.
Keeping those two separate is the whole discipline. Every failure mode in this lesson is an instance of the reading having quietly become the deciding.
What you learned
Section titled “What you learned”- Release gates are deterministic: checks, approvals, windows, health — never an AI assessment
- Change summaries are the highest-value application, and must state facts rather than verdicts
- Risk surfacing produces a list; risk assessment produces a judgement — automate only the first
- Detect risk factors with path globs where possible, and express policy as deterministic rules
- Publish release notes as drafts, link every entry, and never claim an unreferenced fix
- Generate the mechanical skeleton of an upgrade guide; the reasoning comes from the author
- Generate an incident timeline’s sequence, never its analysis
Exercise
Section titled “Exercise”Use a disposable repository with at least two tags. No production credentials.
-
Generate a change summary for the range between two tags, using the prompt structure above.
-
Check every claim in it against the commits. Predict: how many are traceable?
-
Remove the “do not assess whether the release is safe” instruction and regenerate. Compare the tone.
-
Add a deterministic risk-factor detection step — a path glob for migrations — and compare its reliability to asking a model.
-
Generate release notes and publish them as a draft. Edit them as an author would. Note what you changed.
-
Write a policy rule: “any release containing a migration requires a second approver.” Implement it deterministically. Predict: is this what you wanted from a risk score?
-
Delete the repository.