Requesting a review each time works, and nobody does it consistently. Automatic review is how the layer actually gets applied.
The decision is not whether to turn it on. It is what it reviews, and what happens to the comments — because an automatic review that produces findings nobody reads is worse than no review at all. It trains people to scroll past a category of comment, and that habit does not distinguish between the finding that did not matter and the one that did.
The short answer
Section titled “The short answer”Enable automatic review through a coding guidelines / code review setting, at repository or organisation level, depending on your plan and administrative access. Once enabled, Copilot is added as a reviewer when a pull request opens.
Three things to decide before you turn it on:
- Scope — every repository, or the ones where you have review instructions worth applying.
- Draft pull requests — whether work-in-progress gets reviewed.
- Each push, or once — whether new commits trigger a fresh review.
Everything else in this lesson is about the consequences of those three.
Where the setting lives
Section titled “Where the setting lives”Repository level. A repository setting enabling automatic review for pull requests in that repository. The right place to start: one repository, instructions you have tuned, a team that has agreed to try it.
Organisation level. The same behaviour applied across repositories. Efficient, and the mechanism by which a review layer arrives in repositories whose owners did not ask for it — which is the source of most complaints about noise.
Rulesets. On plans that support it, automatic review can be expressed as a rule targeting specific
branches, which is a considerably better fit than a blanket repository toggle. Reviewing pull requests
targeting main and release/* while leaving experimental branches alone is a coherent policy;
“everything in this repository” usually is not.
Draft pull requests
Section titled “Draft pull requests”The most consequential of the three decisions, and the one most teams get backwards on the first attempt.
The case for reviewing drafts: feedback arrives while the change is still cheap to alter. A defect found in draft costs a commit; the same defect found at review costs a round trip and somebody else’s attention.
The case against: a draft is unfinished by definition. Reviewing it produces findings about code the author was already going to change, and those findings are noise in the most literal sense — comments about a state that no longer exists by the time anyone reads them.
What usually works: review on ready-for-review, not on draft open. Authors who want early feedback can request a review manually, which is a small enough action that the people who want it will do it.
The exception is a team whose drafts are genuinely near-complete — where “draft” means “I want a look before I ask for approval” rather than “I am halfway through”. For them, draft review is early feedback rather than noise. You will know which kind of team you have by looking at how much your drafts change between opening and ready-for-review.
Reviewing each push
Section titled “Reviewing each push”Copilot reviews a pull request once unless configured otherwise. Enabling review on each push changes that.
What you gain: the review reflects the current state of the branch. Without it, a pull request that changed substantially after its first review has been reviewed only in its earlier form — which is a real gap, and one people forget about precisely because the first review’s comments are still sitting there looking current.
What you pay: a review per push. On a branch where the author pushes eleven times while iterating, that is eleven reviews, most of them on intermediate states nobody cares about. The comment volume climbs, the signal per comment falls, and people start collapsing the review section by reflex.
The middle path most teams land on: review once automatically, then re-request manually after substantive changes. It puts a human decision in the loop about when a re-review is worth having, and that decision is usually correct because the person making it knows what changed.
The push-review setting earns its cost on long-lived pull requests — a large migration, a change that sits open for a week while decisions get made. If most of your pull requests merge within a day, once is enough.
The noise problem
Section titled “The noise problem”The failure mode is not wrong findings. It is right findings nobody reads.
How it develops. Automatic review is enabled everywhere. Instructions are generic, because they were never written for most of these repositories. Reviews produce a stream of correct-but-unremarkable comments — style preferences, defensive suggestions, observations about code that is fine. People learn that Copilot comments are usually skippable. They start skipping them. Six weeks later a real finding arrives and gets the same treatment.
Why it is hard to detect. Every metric looks healthy. Reviews are happening. Comments are being produced. The thing that has degraded — whether anybody reads them — is not on any dashboard.
The fixes, in the order they help:
Instructions before scope. A repository with tuned review instructions produces findings that fit the codebase. A repository without them produces generic ones. Turn automatic review on where the instructions exist, and treat writing instructions as the precondition rather than a follow-up. See custom instructions for AI code review.
Say what not to comment on. Instructions can exclude as well as include. “Do not comment on formatting; it is handled by the formatter” removes an entire category of finding that your toolchain already covers, and every category you remove raises the average value of what remains.
Path-specific scope. Review criteria that fit migrations do not fit React components. Path-specific instruction files are the mechanism, and they are underused relative to how much they help.
Watch the acted-on proportion, not the volume. Of the last fifty findings, how many led to a change or a considered dismissal? A low proportion means the instructions are wrong, not that the feature is.
Cost and quota
Section titled “Cost and quota”Worth knowing before you enable it across an organisation.
Copilot code review consumes from your plan’s allowance, and automatic review multiplies the number of reviews by the number of pull requests rather than by the number of times somebody chose to ask. On a busy organisation that is a substantial difference — and the reviews it adds are disproportionately the low-value ones, because the pull requests where somebody would have requested a review manually were already getting reviewed.
Check your plan’s current terms rather than assuming; allowances and what counts against them change, and this lesson deliberately does not quote numbers that will be wrong by the time you read it. The structural point survives the details: scoping automatic review to branches and paths where it earns its cost is both a noise decision and a budget decision, and they point in the same direction.
Automatic is not required
Section titled “Automatic is not required”Two independent settings, and the distinction matters more than any other configuration detail in this lesson.
Automatic review means Copilot is added as a reviewer without anybody asking.
A required review means the merge is blocked until an approval exists.
Copilot’s review does not satisfy a branch protection rule requiring approvals. It comments; it does not approve in the sense that unblocks a merge. Nor should it — a merge gate satisfied by a non-human participant is not a review requirement, it is a delay.
The consequence worth being explicit about with your team: a pull request with a clean Copilot review and no human approval is not reviewed. It has passed one layer. Where merges have historically been loose, the arrival of an automatic reviewer can create a feeling of coverage that the configuration does not support, and that feeling is more dangerous than the original looseness because it is invisible.
If you want review to be enforced, enforce it in branch protection or a ruleset. Instructions shape; rulesets enforce. That separation runs through this entire pillar and it applies exactly here.
Interaction with CODEOWNERS
Section titled “Interaction with CODEOWNERS”Automatic review sits alongside CODEOWNERS, not inside it.
Copilot’s review does not consume, satisfy or substitute for a code owner’s review. A pull request touching an owned path still needs the owner. What automatic review changes is what the owner is looking at when they arrive: a diff that has already had a mechanical pass, with the obvious things either fixed or explicitly dismissed in a thread.
That is a genuine improvement to the owner’s experience, and it is the strongest argument for automatic
review in repositories with heavy CODEOWNERS coverage — the scarce resource is the owner’s attention,
and this spends less of it on things that did not need an expert.
What to review, and what not to
Section titled “What to review, and what not to”Some pull requests are worth an automatic review and some are pure cost.
Worth it: application code, infrastructure definitions, CI workflow changes, anything with security-relevant surface, anything where a defect is expensive to find later.
Usually not: dependency bump pull requests from an automated tool, generated files, translation updates, changes that are entirely additions of test fixtures. Copilot will review these, produce findings, and the findings will be about generated code nobody wrote.
The mechanism: path-specific instructions can tell the review what to skip, and repositories that are entirely generated content are better off not enabling automatic review at all than enabling it and teaching people to ignore the results.
Dependency pull requests deserve a specific note. They are numerous, they are automated, and the review question about them is “is this version safe and compatible”, which is a Dependabot and supply chain question rather than a code review question. An AI review of a lockfile diff is almost pure noise.
Who owns the configuration
Section titled “Who owns the configuration”An organisational question that decides whether the rollout holds.
The toggle usually sits with administrators. Repository admins, or organisation owners for the broader settings. That is appropriate — it is a policy control.
The instructions belong to the team. They encode conventions that only the people writing the code know. An administrator writing review instructions for forty teams produces guidance that is either generic or wrong.
The split that fails is administrators enabling review broadly and expecting teams to notice they should tune it. Teams do not notice. They experience a reviewer that appeared without warning, producing comments about a codebase it does not understand, and their rational response is to ignore it.
The split that works: administrators own whether the feature is available and which branches it targets; teams own the instruction files in their own repositories; somebody checks periodically which repositories have instructions at all. That last check is the one nobody schedules and the one that determines whether the layer is doing anything.
If you administer this for an organisation, the useful question to ask before expanding scope is not “are reviews happening” but “in how many of these repositories would a review find something the team would care about”. Where the answer is no, the honest move is to leave it off until instructions exist.
Rolling it out
Section titled “Rolling it out”-
Pick one repository where somebody cares about review quality and is willing to tune.
-
Write review instructions first. Repository-wide for the standards that always apply, path-specific where criteria genuinely differ. Without this step the rest is generic.
-
Enable automatic review on ready-for-review, once per pull request rather than per push.
-
Run it for two weeks. Read every finding. Note which ones were acted on, which were dismissed with a reason, and which were ignored without comment — the third category is your noise.
-
Tune the instructions against that record. Exclude what the toolchain already covers. Add specifics for the findings you wished it had made.
-
Expand to the repositories where the same instructions apply. Not to all repositories — to the ones the instructions actually fit.
-
Re-check the acted-on proportion after expansion. A drop means the instructions did not travel as well as you thought.
The reason this sequence works is that it treats the instructions as the product and the toggle as the delivery mechanism. Teams that reverse the order — enable everywhere, tune later — usually never reach the tuning step, because by then the reviews are already background noise and nobody is reading them closely enough to know what to fix.
Common mistakes
Section titled “Common mistakes”Enabling it organisation-wide on day one. The instructions do not exist yet, so the reviews are generic, so people learn to ignore them. Recovering from that is harder than the original rollout.
Assuming it satisfies required reviews. It does not. A pull request with a Copilot review and no human approval is not reviewed.
Reviewing every push during active development. Eleven reviews of eleven intermediate states, and comment fatigue by the fourth.
Reviewing drafts on a team whose drafts are genuinely rough. Findings about code that was already going to change.
Leaving it on for dependency bumps and generated files. High volume, near-zero value, and it is the category that most efficiently trains people to skip Copilot comments entirely.
Measuring comment volume. It measures activity, not usefulness, and optimising it makes the layer worse. The acted-on proportion is the number that means something.
Never revisiting the instructions. Review criteria drift out of date exactly like documentation does, and stale review guidance produces findings about conventions the team abandoned a year ago.
Mental model
Section titled “Mental model”Automatic review is a default, not an obligation. It decides that this class of pull request gets a mechanical pass without anybody remembering to ask.
The value of the default depends entirely on whether the pass is tuned. A tuned review applied automatically compounds — every pull request gets criteria the team agreed on, consistently, including on Friday evenings. An untuned review applied automatically also compounds, in the other direction, by teaching people that this section of the pull request is safe to skip.
The toggle is trivial. The instructions are the work.
What you learned
Section titled “What you learned”- Automatic review is enabled at repository, organisation or ruleset level, and rulesets give the best scope control
- Three decisions matter: scope, draft handling, and once-versus-each-push
- Review on ready-for-review and once per pull request is the default that survives contact with a real team
- Automatic review does not satisfy required approvals and does not substitute for
CODEOWNERS - The failure mode is right findings nobody reads, and it is invisible on every metric except the acted-on proportion
- Instructions are the product; the toggle is the delivery mechanism
Exercise
Section titled “Exercise”Use a disposable repository with Copilot code review available.
-
Enable automatic review at repository level. Open a pull request. Predict: is Copilot added as a reviewer without you asking?
-
Open a draft pull request. Predict: does it review before you mark it ready?
-
Push a second commit to a reviewed pull request. Predict: does a second review appear?
-
Add a branch protection rule requiring one approval. With a clean Copilot review and no human approval, predict: does the merge button unblock?
-
Add
.github/copilot-instructions.mdcontaining an explicit exclusion — “do not comment on formatting”. Open a pull request with formatting inconsistencies. Compare the review to one from before the exclusion. -
Open a pull request that only changes a lockfile. Read the review. Predict: how much of it is useful?
-
Delete the repository.