Copilot Autofix reads a code scanning alert, the surrounding code, and the query’s description of the vulnerability class, and proposes a code change that resolves the alert.
It is genuinely useful, and this page maintains a specific scepticism about it — not because AI assistance is suspect in general, but because a security fix has a property most code changes do not: it can appear to work while leaving the vulnerability in place.
The short answer
Section titled “The short answer”What it does. Suggests a code change resolving a code scanning alert, presented as a proposed diff you accept, edit or reject.
Coverage. A subset of the queries in the default and security-extended CodeQL suites, for C#,
C/C++, Go, Java/Kotlin, Swift, JavaScript/TypeScript, Python, Ruby and Rust. Not every CodeQL alert
has a suggestion.
Availability. Allowed by default and enabled for every repository using CodeQL, and it can be disabled at enterprise, organisation or repository level. No GitHub Copilot subscription is required.
What GitHub does before showing you one. Runs code scanning and the repository’s unit tests against the resulting code. Suggestions that fail that internal testing are not displayed.
What it is not. A remediation service. It is a first draft with an unusually good starting point.
What GitHub already checks
Section titled “What GitHub already checks”Worth knowing precisely, because it is better than most people assume and it is not what people assume.
Before a suggestion is displayed, GitHub runs code scanning and the repository’s unit tests against the modified code, to verify that the change resolves the alert without introducing new problems or syntax errors. Suggestions failing that internal testing are not shown.
So a suggestion you see has already cleared two bars: the alert closes, and your existing tests still pass.
That is a meaningful filter, and it is precisely bounded. It establishes that the change satisfies the analysis and does not break what you already test. It establishes nothing about whether the change is correct, whether it preserves behaviour your tests do not cover, or whether it fixes the underlying problem rather than the pattern the query matched.
The documented failure modes
Section titled “The documented failure modes”GitHub’s own documentation is unusually direct about this, and the list is worth reproducing because it is more specific than a general caution.
Autofix may:
- Suggest fixes that fail to remediate the underlying vulnerability
- Introduce new vulnerabilities
- Produce syntax errors
- Produce location errors — changes in the wrong place
- Produce semantic errors — changes that compile and mean the wrong thing
And it is non-deterministic: the same alert, the same code, and the same request can produce different suggestions.
Read the first two again. The two failure modes that matter most for a security fix are both on the vendor’s own list, and neither is caught by “the alert closed and the tests passed” — which is exactly what the internal validation checks.
The review workflow
Section titled “The review workflow”A vertical sequence: security alert, AI suggested fix, engineer review, tests, security validation, merge. Every stage is required.
The shape to avoid is the two-stage version — alert, AI, trust — which is what happens when the suggestion is accepted from the alert view and merged on a green build.
Reviewing a suggestion
Section titled “Reviewing a suggestion”-
Understand the vulnerability first. Read the alert’s data flow before reading the suggestion. Reviewing a fix without understanding the flaw is reviewing whether the code looks reasonable, which is a different question.
-
Read the change as code. Would you approve this diff in an ordinary review, from a colleague, with no security framing? If not, the security framing does not improve it.
-
Ask whether it fixes the flaw or breaks the pattern. The critical question. Does the change sanitise, validate or restructure the dangerous path — or does it move code around such that the analysis loses the trail?
-
Check behaviour. Does the change alter what the code does for legitimate input? An over-aggressive validation that rejects valid data is an outage rather than a security fix.
-
Look for the same pattern elsewhere. A fix at one call site leaves the other four. Autofix addresses the alert; the alert is one instance.
-
Write a test. For the vulnerability, not for the code. A test asserting that the malicious input is rejected is the only artefact that keeps this fixed through future refactors.
Steps 3 and 6 are the ones that distinguish a reviewed fix from an accepted suggestion, and both take minutes.
What “the alert closed” actually proves
Section titled “What “the alert closed” actually proves”It is worth being concrete about the logical gap, because it is the whole argument of this page.
CodeQL’s security queries model a vulnerability as a path: a source of untrusted data, a sink where that data has consequences, and the absence of a recognised sanitiser between them. The alert exists when the query can trace such a path.
A change makes the alert disappear if it does any of the following:
- Introduces a sanitiser the query recognises. A genuine fix.
- Removes the path. Also genuine — the dangerous call no longer receives that data.
- Introduces something the query treats as a sanitiser that does not sanitise. A validation function that checks the wrong property, an escaping call for the wrong context.
- Makes the flow untraceable. Passing the value through a dynamic dispatch, a serialisation round-trip, or a data structure the analysis does not model through.
Outcomes 1 and 2 fix the vulnerability. Outcomes 3 and 4 close the alert and leave it, and 3 and 4 pass GitHub’s internal validation exactly as well as 1 and 2 do — the alert closes, the tests pass, the suggestion is displayed.
Outcome 3 is the more likely of the two in practice. Escaping is context-sensitive: HTML escaping a value that ends up in a JavaScript string literal, or in an SQL identifier, is a real fix for the wrong context, and it looks entirely correct in review unless the reviewer is thinking about where the value lands.
This is why step 3 of the review is the whole review. Everything else — does it compile, do tests pass, is it readable — has already been checked by somebody or something.
Where it works well, and less well
Section titled “Where it works well, and less well”Well: mechanical, well-understood vulnerability classes with a standard fix. Parameterising a SQL query. Escaping output. Adding a path check before a file operation. Replacing a weak hash. These have a canonical shape, the model has seen thousands of examples, and the fix is local.
Less well: anything requiring architectural judgement. An authorisation flaw where the fix depends on your permission model. A vulnerability whose correct resolution is to restructure how data flows rather than to sanitise at the sink. Cases where the right answer is “this feature should not exist”.
The distinguishing question is whether the fix is local and canonical. Where a competent engineer would apply a known pattern without much thought, Autofix does that well and quickly. Where a competent engineer would need to understand the system’s intent, it is a suggestion with no more authority than any other.
The threat model
Section titled “The threat model”Unusually for this pillar, the threat here is partly a property of the tool rather than only of the code.
Threat. A vulnerability remains in production because a fix that appeared to resolve it did not — either because it addressed the analysis rather than the flaw, or because it introduced a different one.
Attack surface. Every accepted suggestion, weighted by how security-sensitive the surrounding code is and by how thoroughly it was reviewed. The surface grows with adoption, which is worth saying plainly: a tool that makes fixes cheap increases the number of security-relevant changes flowing through review.
Impact. The original vulnerability, still present, now with an alert closed and a commit in the history that says it was fixed. That combination is worse than the untouched vulnerability, because the second look never happens.
Control. Human review that asks specifically whether the flaw is gone rather than whether the alert is; a test asserting the vulnerable behaviour is rejected; and a search for the same pattern elsewhere.
Verification. Attempt the exploit against the fixed code. For a mechanical class this is a unit test with malicious input; for anything more involved it is the same work you would do for any security fix. If the fix is not worth verifying, the alert was not worth fixing.
That last sentence is a useful test in both directions. It sorts genuine findings from noise, and it sets the standard for what “fixed” means.
Using it at scale
Section titled “Using it at scale”The high-value application is a backlog. A repository with two hundred alerts is not going to be triaged alert by alert, and Autofix genuinely changes the economics.
Security campaigns build on exactly this: alerts in a code campaign are automatically submitted to Autofix, so the developer who picks one up starts from a suggestion rather than from a blank editor.
The discipline that makes this safe at volume is the same discipline as one alert, held under more pressure:
Review each suggestion individually. Batch-accepting fifty AI-generated security fixes is fifty unreviewed changes to security-sensitive code, arriving together, which is a worse outcome than the backlog.
Group by vulnerability class, not by count. Twenty fixes for the same class are reviewable together because the reviewer builds up context and the same question applies to each. Twenty fixes across nine classes are twenty separate reviews wearing a single pull request, and they will be approved as one.
Keep a rejection rate. If nobody is ever rejecting a suggestion, nobody is evaluating them. A healthy process rejects some, and knowing roughly what proportion tells you whether review is happening. This is not a metric to optimise — it is a smoke test on the process, and a rate of zero is the only value that reliably means something.
Policy: where to allow it
Section titled “Policy: where to allow it”Autofix is on by default, and there are situations where a deliberate decision is worth making.
Leave it on everywhere by default. The suggestions are a starting point, they cost nothing when ignored, and for the mechanical cases they save real time. Disabling it organisation-wide is an over-correction that removes value without removing the risk — which comes from accepting suggestions without review, not from suggestions existing.
Consider requiring stronger review on accepted suggestions. A CODEOWNERS entry routing changes
in security-sensitive paths to a named reviewer applies whether the change came from a person or a
model, and it is the right instrument here.
Do not auto-merge them. There is no configuration that makes an AI-generated security fix safe to merge unreviewed. If you have auto-merge for dependency updates, be explicit that Autofix pull requests are not in that category.
Be careful in code with unusual security properties. Cryptographic code, authentication and authorisation logic, and anything implementing a protocol are places where a plausible-looking change can be subtly wrong in ways neither tests nor static analysis detect. Treat suggestions there as prompts for a human to think about, not as drafts to edit.
The framing that keeps this proportionate: Autofix changes who writes the first draft. It does not change who is responsible for the code. Everything about your review process that was appropriate before remains appropriate, and the only thing that has changed is the speed at which changes arrive for review.
Common mistakes
Section titled “Common mistakes”Accepting because the alert closed. The alert closing is the weakest possible evidence, and it is the evidence most prominently displayed.
Trusting the internal validation to mean more than it does. It confirms the alert closes and existing tests pass. Those were never the hard questions.
Not writing a test. Without one, the next refactor can reintroduce the flaw with nothing to notice.
Fixing the instance rather than the pattern. Autofix addresses one alert. The same mistake usually exists elsewhere.
Batch-accepting during a backlog push. Volume pressure applied to security-sensitive code review.
Assuming coverage. Not every alert has a suggestion, and the ones without are frequently the complex ones that most needed help.
Treating non-determinism as a bug. Two different suggestions for the same alert are both drafts. Neither is authoritative.
What this page is not saying
Section titled “What this page is not saying”Scepticism about a specific claim is easy to read as opposition to the tool, so it is worth separating the two.
This is not an argument against using Autofix. For the mechanical vulnerability classes it covers, it produces good drafts quickly, and starting from a draft is materially better than starting from an alert and an unfamiliar codebase. Refusing it on principle costs real time and prevents real fixes.
It is not an argument that AI-written code is worse. The relevant property is not authorship. It is that a security fix has an unusual failure mode — appearing correct while leaving the flaw — and that the validation available happens to be blind to exactly that mode. A human fix has the same failure mode and gets caught by review; the concern is that a pre-validated suggestion invites the review to be shorter.
It is not a claim that GitHub’s validation is inadequate. Running code scanning and the test suite before displaying a suggestion is more than most tools do. The point is only that those two checks answer questions that were never the hard ones.
The narrow claim is this: the confidence signal attached to an Autofix suggestion is stronger than the evidence behind it. Adjust for that in review, and the tool is straightforwardly good.
Mental model
Section titled “Mental model”Autofix is a knowledgeable colleague who has read the alert and written a first draft. The draft is often good and has been checked against the analysis and your tests. It carries exactly the authority of a draft — which is to say, none until somebody who understands the system has read it.
What you learned
Section titled “What you learned”- Autofix covers a subset of
defaultandsecurity-extendedCodeQL queries across nine languages - It is enabled by default for repositories using CodeQL and needs no Copilot subscription
- GitHub runs code scanning and your unit tests before showing a suggestion, and hides failures
- That validation confirms the alert closes and tests pass — neither of which is “the vulnerability is fixed”
- GitHub documents that suggestions may fail to remediate, or introduce new vulnerabilities
- Suggestions are non-deterministic; the same alert can produce different drafts
- Closing an alert by obscuring the data flow is a real failure mode and looks identical to a fix
- It works best on local, canonical fixes and worst on anything needing architectural judgement
- Write a test for the vulnerability, because it is the only artefact that survives a refactor
- A process that never rejects a suggestion is a process where nobody is reviewing
Exercise
Section titled “Exercise”Use a disposable public repository with CodeQL enabled.
-
Introduce a clear vulnerability in a supported language — a request parameter concatenated into a database query.
-
Wait for the alert, then request a fix. Predict: is a suggestion offered for this alert class?
-
Before reading the suggestion, write down how you would fix it yourself.
-
Read the suggestion and compare. Predict: does it match your approach, and if not, is the difference an improvement or a shortcut?
-
Ask specifically: does this sanitise the dangerous path, or does it restructure the code so the analysis no longer traces it?
-
Accept it and confirm the alert closes. Then write a test asserting the malicious input is rejected. Predict: does the test pass?
-
Add the same vulnerable pattern in a second location. Predict: did the first fix address it?
-
Delete the repository.