An assessor does not want to know what your settings are. They want to know that a control operated, consistently, over a period, and that you can show it.
That is a different artifact from a configuration export, and producing it is mostly a data problem you can solve in advance rather than a scramble at audit time.
The short answer
Section titled “The short answer”A control is a process with an owner, not a setting. “Changes to production code are reviewed before merge” is a control. A ruleset is one mechanism implementing it.
Evidence has three parts: that the control exists (design), that it applied to everything in scope (coverage), and that it operated over the period (operation).
Configuration exports demonstrate design. They are necessary and they are the part most organisations over-invest in.
Audit and activity data demonstrate operation. This is the part that requires having collected it, and it cannot be produced retroactively.
Scope determination is yours, and it depends on metadata — which repositories are in scope for which control. See custom properties.
The three questions an assessor asks
Section titled “The three questions an assessor asks”“What is the control?” Answered by a written control description: what is required, of what population, enforced how, owned by whom, with what exception process.
“Does it apply to everything it should?” Answered by coverage data: the population in scope, and the demonstration that each member is subject to the control. This is where most evidence is weak, because it requires knowing what is in scope, which requires metadata.
“Did it operate throughout the period?” Answered by activity data: enforcement events, review records, exception approvals, and the record that the control was not disabled at any point. This is where most evidence is missing, because it requires having collected data before anybody asked.
A configuration screenshot answers only the first, and answers it weakly, because it shows the state today rather than throughout the period.
Controls GitHub can support
Section titled “Controls GitHub can support”Common control objectives, and the mechanism and evidence for each. Whether these satisfy any particular framework is your auditor’s determination, not something the mechanism grants.
| Control objective | Mechanism | Evidence of operation |
|---|---|---|
| Changes are reviewed before production | Ruleset requiring pull requests and approvals | Merged pull requests with approvals; bypass records |
| Segregation of duties | Required review from a different person; environment approvals | Approval records showing distinct approver |
| Access is granted through a controlled process | SCIM, teams, grants | Provisioning events; point-in-time access snapshots |
| Access is removed promptly on termination | SCIM deprovisioning | Deprovisioning timing per departure |
| Privileged access is reviewed | Access reviews | Completed reviews with dates and outcomes |
| Changes are attributable | Commit history; audit log | Commit authorship; audit events |
| Production deployments are authorised | Environment required reviewers | Deployment approval records |
| Code is scanned for vulnerabilities | Code scanning, required checks | Scan results over the period; blocked merges |
| Audit records are retained | Audit log streaming | The retained records themselves; stream health |
Note what the evidence column has in common: it is all activity data over a period, and almost none of it is a configuration screenshot.
Design, coverage, operation
Section titled “Design, coverage, operation”Worth expanding, because the three-part structure is what makes an evidence request tractable.
Design is what the control is. A written description, plus the configuration that implements it. Easy to produce and frequently the only thing produced.
Coverage is who it applies to. The in-scope population, and the demonstration that each member is subject to the mechanism. Requires metadata and is where the gaps hide.
Operation is that it worked, throughout. Activity data over the period, plus evidence the control was not disabled. Requires retained data and cannot be produced retroactively.
A control failing on design is rare — organisations generally know what they intend.
Failing on coverage is common and is usually invisible internally. The ruleset is enabled, everybody believes the control is in place, and it targets 80% of the population.
Failing on operation is the most common of all, because it requires having collected data before anybody asked, and organisations do not collect data for questions they have not been asked.
Which produces the priority order for this work: streaming first, metadata second, control descriptions third. The descriptions are quick and can be written any time; the data has to have been accumulating.
Coverage is the hard part
Section titled “Coverage is the hard part”A control applies to a population, and demonstrating coverage means demonstrating that every member of that population was subject to it.
Which requires knowing the population. “All production repositories” is only a population if you can enumerate it, and enumerating it requires the metadata.
The failure that assessors find: a ruleset requiring review, targeted at repositories with a production property, and forty repositories with no property set. Those forty are in scope by any sensible definition and subject to nothing.
Which makes the unset-property report an audit artifact, not just an operational one. A report showing zero unclassified repositories is a coverage demonstration; a non-zero number is a gap you should find before an assessor does.
Report coverage as a number. “The control applies to 412 of 412 repositories in scope” is a strong statement. “The ruleset is enabled” is not, because it says nothing about what it targets.
And demonstrate the scope determination. How you decided which repositories are in scope is part of the evidence, and “we set a property based on this criterion” is a defensible answer where “the platform team knows” is not.
Evidence of operation
Section titled “Evidence of operation”Enforcement events. Blocked pushes, blocked merges, rejected operations. These demonstrate the control actively preventing something, which is the strongest available evidence.
Approval records. Pull request approvals, deployment approvals, exception approvals. These demonstrate the human part of a control operating.
Bypass and exception records. Every bypass invocation, with actor and justification. Do not hide these. An assessor expects exceptions to exist; what they check is whether they were controlled. A control with zero exceptions over a year is more likely to be unenforced than perfect.
Absence of disabling. The audit log shows if a ruleset was deleted or an enforcement changed. Demonstrating that the control was continuously in effect requires showing it was not turned off, and that is an audit log query.
Timing data. Deprovisioning latency, review completion dates, remediation times. Frameworks frequently care about promptness, and promptness is a measurement.
All of which needs the audit data, retained long enough. Which is why streaming is the prerequisite for everything in this article.
Point-in-time snapshots
Section titled “Point-in-time snapshots”The artifact that answers retrospective questions, and the cheapest thing you can build.
A periodic export of state: organisation members, team memberships, repository permissions, outside collaborators, app installations, ruleset configurations, custom property values.
Monthly is usually enough. Quarterly is defensible. Whatever the cadence, it must be consistent — an irregular series is hard to reason about.
Store them immutably. Evidence that could have been edited is weaker evidence.
They answer questions current state cannot: who had access in March, was this ruleset in effect in June, when did this repository become production-tier.
And they are the basis for the coverage demonstration, because the snapshot from the middle of the period shows what was in scope then rather than now.
Automate the export. It is an API job, it takes a day to write, and it converts an audit question from a research project into a file lookup. See API automation.
Change management, specifically
Section titled “Change management, specifically”The control objective GitHub is most often asked about, and the one where the mapping is least straightforward.
What frameworks generally want: changes are authorised, tested, reviewed by somebody other than the author, and traceable to a request.
What GitHub provides: a pull request with required approvals, required status checks, a commit history, and an audit trail of all of it.
The mapping is good but not automatic. Three gaps recur.
Author-approver separation is not guaranteed by default. A ruleset can require an approving review; whether the approver is a different person depends on configuration, and self-approval prevention is a specific setting rather than an inherent property.
Bypass breaks the chain. A merge performed by a bypass actor did not go through the control. That is not a problem — it is an exception — provided it is recorded, justified and reviewed. Undeclared, it is a finding.
Traceability to a request is not something GitHub provides on its own. Linking a change to a ticket is a convention, enforceable through a commit message or branch name rule if the metadata restrictions are available to you, and otherwise a practice.
Automated changes need an answer. Dependency updates merged automatically are changes, and “reviewed by a human” is not true of them. The defensible position is that they are a distinct category with a distinct control — automated testing, scoped to dependency updates only, with the automation itself under change control. State it rather than hoping it is not asked.
And emergency changes need a path. Every organisation has them, every framework expects them, and the control is that they are recorded and reviewed afterwards rather than that they do not happen.
Access management, specifically
Section titled “Access management, specifically”The second objective GitHub is asked about, and the one where coverage is hardest.
What frameworks generally want: access is granted through an approved process, is appropriate to the role, is reviewed periodically, and is removed promptly when no longer needed.
Granting through a process is teams and SCIM: directory group membership drives GitHub access, and the directory has its own approval process. Where access is granted directly in GitHub, that path needs its own approval record.
Appropriateness is the access review, and the evidence is completed reviews with outcomes.
Prompt removal is deprovisioning timing, which is measurable and should be measured. See access reviews and offboarding.
The coverage gaps to declare: outside collaborators, who are outside SAML and SCIM; direct individual grants; machine identities; and administrative roles maintained manually.
Machine identities are the gap most often omitted entirely. An access-management control description that covers only human users, in an estate where automation holds substantial standing access, is incomplete — and it is the gap an experienced assessor asks about.
Point-in-time snapshots are the evidence. Who had what, on which date, throughout the period.
Working with assessors
Section titled “Working with assessors”Write the control descriptions yourself, before being asked. An assessor working from your description evaluates what you meant; one working from a feature list evaluates what they assume.
Be precise about mechanisms. “Enforced by an enterprise ruleset requiring one approving review, targeted at repositories with the production tier property” is specific and checkable.
Declare the gaps. Bypass actors, exempted identities, populations not covered. An assessor finding an undeclared gap treats everything else with more scepticism.
Label preview features. If a control depends on a feature GitHub documents as being in public preview, say so. It does not disqualify the control; concealing it damages your credibility.
Do not claim more than the mechanism does. A ruleset requiring review does not prevent a bypass actor merging without one. Describing it as “all changes are reviewed” is inaccurate and will be found.
Bring the data, not screenshots. A CSV of every merged pull request in the period with its approvers is better evidence than an image of a settings page, and it is easier to produce once the export exists.
Building the evidence pipeline
Section titled “Building the evidence pipeline”The engineering work, which is modest and pays for itself the first time an assessor asks for something.
-
Stream the audit log. Nothing downstream is possible without it, and retention is a rolling window. See audit log streaming.
-
Establish the metadata. Custom properties describing what each repository is, so scope is a query rather than a judgement.
-
Automate the point-in-time snapshot. Members, teams, permissions, collaborators, apps, rulesets, properties. Monthly, immutable, retained.
-
Write the control descriptions. One per objective, four or five sentences, with mechanism, population, owner, exceptions and any preview dependency.
-
Build the coverage report. For each control, the in-scope population and the count subject to the mechanism. This is the artifact assessors most often find missing.
-
Build the operation extracts. For each control, the activity data demonstrating it operated — approvals, blocks, exceptions, timings — over an arbitrary period.
-
Run them monthly, not at audit time. Evidence collected continuously is credible; evidence assembled retrospectively is not, and for the operation part it is frequently impossible.
-
Review the outputs yourself. Find your own gaps before somebody else does.
Steps 1 to 3 are the investment, and they are a few weeks of work. Steps 4 to 6 are then queries over data you already have.
The payoff is disproportionate. An audit request that takes a week of an engineer’s time, three times a year, becomes an afternoon — and the answers are better.
What good evidence looks like
Section titled “What good evidence looks like”Concretely, for one control, to make the abstraction tangible.
The control description. “All changes to repositories with tier production require at least one approving review from a person other than the author before merge to the default branch. Enforced by enterprise ruleset production-baseline. Owned by the platform engineering team. Bypass is granted to the release-automation GitHub App only, for tagged releases. Exceptions are approved by the head of engineering and recorded in the governance log.”
The coverage artifact. A table: 412 repositories with tier production; 412 matched by the ruleset’s targeting; 0 unclassified repositories in the organisations in scope. Produced from the monthly snapshot.
The operation artifact. A table of every merge to a default branch in a production-tier repository during the period, with the approver, the approval timestamp, and a flag for those merged under bypass. Plus the audit log query showing the ruleset was not modified during the period.
The exception artifact. Three bypasses during the period, each with the actor, the date, the release it corresponded to, and the approval.
Four artifacts, all generated. None requires an engineer to assemble anything by hand, and together they answer all three questions completely.
Compare against the common alternative — a screenshot of the ruleset settings page — and the difference in what it demonstrates is the whole of this article.
Common mistakes
Section titled “Common mistakes”Presenting configuration as compliance. The control is the process.
Claiming a framework is satisfied by a setting. It is your auditor’s assessment.
Evidence of design without evidence of operation. The most common gap.
No coverage data. The control may be enabled and apply to a fraction of the population.
Unset properties leaving repositories out of scope. A gap an assessor will find.
Hiding exceptions. Their absence is more suspicious than their presence.
Not labelling preview features. Damages credibility on everything else.
No point-in-time snapshots. Retrospective questions become unanswerable.
Starting evidence collection at audit time. Operation data cannot be produced retroactively.
Over-claiming what a mechanism enforces. It will be checked.
The relationship between engineering and audit
Section titled “The relationship between engineering and audit”A practical note, because this work sits between two functions that frequently misunderstand each other.
Compliance teams ask for screenshots because that is what other systems provide. Offering generated data instead is usually welcomed once explained — it is better evidence and it is less work for them too.
Engineers assume the requirements are arbitrary. Most are not. “Show that changes were reviewed” is a reasonable thing to want, and the friction is usually about the evidence format rather than the requirement.
Translate in both directions. Explain what a ruleset actually enforces, in plain terms, and ask what the control objective actually requires rather than accepting a request for a specific artifact.
Push back on requests for the wrong artifact. If somebody asks for a screenshot of every repository’s settings, the answer is a generated table covering all of them — better evidence, and produced in minutes rather than days.
And engage before the audit. The worst version of this work is discovering, during fieldwork, that a control you believed was in place has a coverage gap. Running your own coverage report a quarter ahead turns that into a task.
Own the control descriptions jointly. Written by engineering, reviewed by compliance, agreed by both. A description written by one side alone is either technically inaccurate or does not answer the objective.
What to do when evidence does not exist
Section titled “What to do when evidence does not exist”Sometimes the honest answer is that you cannot demonstrate a control operated, and how you handle that matters.
Say so. An undeclared gap discovered by an assessor is worse than a declared one, by a large margin.
Explain why. “Git event data is retained for seven days by default and we began streaming in March, so we can demonstrate this from March onward” is a clear, credible statement.
State the remediation and the date. A gap with a plan is a finding with a resolution; a gap without one is a finding that recurs.
Start collecting immediately. The evidence for the next period is created by what you do today, and every week of delay is a week that will be missing.
Do not reconstruct. Assembling evidence retrospectively from partial data, presented as though it were collected contemporaneously, is a serious problem that goes well beyond the original gap.
And use it. A documented evidence gap is frequently the most effective argument for the streaming, snapshot and metadata work described in this cluster — more effective than any technical case, because it comes with an external deadline.
Mental model
Section titled “Mental model”Evidence answers three questions: what is the control, did it cover everything in scope, and did it operate throughout the period. Configuration answers the first. Metadata answers the second. Retained audit data answers the third — and only if you were collecting it before anybody asked.
What you learned
Section titled “What you learned”- No GitHub configuration produces compliance with any framework by itself; the control is your process and the assessment is your auditor’s
- Evidence has three parts: design, coverage and operation
- Configuration exports demonstrate design only, and only as of today
- Coverage requires knowing the in-scope population, which requires metadata
- Unset custom properties are a coverage gap that leaves repositories subject to no targeted control
- Operation is demonstrated by enforcement events, approvals, exception records and the absence of disabling
- Point-in-time snapshots answer retrospective questions that current state cannot
- Exceptions should be declared, not hidden; their total absence is itself suspicious
- Preview-feature dependencies must be labelled in any control description
Exercise
Section titled “Exercise”Use an enterprise you administer.
-
Pick one control your organisation claims — for example, that changes are reviewed before merge. Write the control description in four sentences.
-
Define the population it applies to. Predict: can you enumerate it exactly?
-
Count repositories in that population with no relevant custom property set. That is your coverage gap.
-
Produce evidence of operation for one month: merged pull requests with approvals, and any bypasses.
-
Query the audit log to show the ruleset was not modified or disabled during that month.
-
List every bypass actor and exempted identity for that control. Predict: would you have declared all of them?
-
Determine whether any part of the control depends on a preview feature.
-
Write the point-in-time snapshot you would export monthly, and estimate the work to automate it.