◆Lesson 4 of 8Intermediate14 min readGitHub Copilot & AI Engineering · GitHub CopilotVerified: GitHub Copilot Chat documentation and IDE integrations, September 2026
Most writing about chat is about prompting. Almost none of the difference between a useful answer and a
useless one is prompting.
It is context — what the model can see when it answers — and verification — what you do with
what comes back. This lesson is about those two, applied to the things engineers actually use chat for.
What the model sees, in descending order of reliability:
What you explicitly attach. Files, selections, symbols. Deterministic. This is the lever.
What the editor contributes. The active file, your selection. Usually present, occasionally not
what you assumed.
What a search retrieved. Where the surface can search the workspace. Powerful and
non-deterministic — it may not find the file you had in mind.
Repository instructions. Always present where supported, and the reason your conventions apply
without restating them.
Conversation history. Everything earlier in this chat, until the context window fills.
Nothing else. Not the rest of your repository, not your issue tracker, not what you told it
yesterday in a different conversation.
That last line explains most disappointing answers. A model asked about a function it cannot see will
produce an answer about a function like it — fluently, and without indicating that it is guessing.
Explaining unfamiliar code. The highest-value use, and the safest — the output is an explanation you
can check against the code in front of you. Particularly good for dense code: a regular expression, a
complex conditional, an unfamiliar language idiom.
Generating tests. Given a function and your existing test file for conventions, it produces
reasonable cases quickly. It is good at the mechanical coverage — happy path, empty input, boundary —
and it will not think of the case that actually breaks your system, because that requires knowing your
system.
Debugging with real output. Given an error, a stack trace and the relevant code, chat is a good
hypothesis generator. The requirement is the real output rather than a description of it.
Refactoring proposals. “What are three ways to restructure this?” is a better question than “refactor
this”, because it returns options you choose between rather than a change you have to evaluate.
Translating between idioms. Explaining a pattern from another language in terms of yours, or
adapting an approach to your framework’s conventions.
Anything requiring your system’s runtime state. What is actually in the database, what the load
looks like, which feature flag is on.
Judging significance. Whether this bug matters, whether this refactor is worth doing.
Knowing what changed recently unless you tell it. It has no view of your deploy history.
Being reliably right about specific APIs. A method signature that is plausible and wrong is the
characteristic failure, and it compiles in your head before it fails on the machine.
My tests are failing with some kind of import error.
This:
Running pytest tests/test_orders.py gives:
ImportError: cannot import name 'OrderStatus' from partially initialized module
'app.models' (most likely due to a circular import)
Here is app/models.py and app/services/orders.py. What is the cycle and what are my options for
breaking it?
The second contains: the command, the exact error, the relevant files, and a specific question. Every
element narrows the answer, and the error text alone frequently determines the diagnosis.
This is the same discipline as AI Git troubleshooting, and it applies
to every kind of failure: paste the output, not a summary of the output.
The second set has answers you can verify by looking. That is not a small difference: it converts chat
from something you trust into something you use.
The same applies to design questions, with a rephrasing. Instead of “how should I structure this”,
ask “what are three ways to structure this, and what does each make harder?” Options with
trade-offs are evaluable; a recommendation is not.
Worth its own treatment because it is the most common generation task and the one with the most
misleading success signal.
Give it the conventions. Attach an existing test file. Your framework, your fixtures, your naming,
your assertion style — all of it is inferable from one example and none of it from the function alone.
Ask for the cases, then the code. “What cases should this function be tested for?” produces a list
you can add to from your knowledge of the system. Generating the code for an agreed list beats
generating both at once.
Expect the mechanical cases. Happy path, empty input, boundary values, type errors. It is good at
these and they are worth having.
Supply the interesting cases yourself. The concurrent modification, the partial failure, the input
that broke production in March. These come from knowing the system.
Verify by breaking. The non-negotiable step:
Run the generated test. It passes.
Break the function deliberately — invert a condition, remove a guard.
Run again. The test must fail.
Restore the function. Confirm it passes.
A test that passes in both states asserts nothing. This happens more often than people expect,
particularly with mocked dependencies where a generated test can end up asserting that a mock was
called rather than that the behaviour is right.
Chat degrades over a long session, and the failure is gradual rather than announced.
What happens. The context window fills. Earlier messages are dropped or summarised. Constraints you
established twenty exchanges ago stop being applied, and nothing tells you.
The symptoms. It starts re-asking things you answered. It contradicts an earlier decision. Answers
get vaguer. It re-reads a file it already read.
What to do:
Start a new conversation for a new problem. The most effective habit. Conversations are cheap; a
polluted context is not.
Re-state constraints that matter. If a decision from earlier is load-bearing, say it again rather
than assuming it survived.
Use compaction where available. Some surfaces can summarise the conversation to reclaim space —
/compact in the CLI, equivalents elsewhere.
Prefer several focused conversations to one long one. A chat covering three problems is worse at
all three than three chats would have been.
The output is a claim. The verification is cheap and specific to the kind of claim.
Output
Verify by
An explanation of your code
Reading the code with the explanation in mind
A code snippet
Compiling it
An API usage
Checking the actual documentation
A test
Running it, and confirming it fails when it should
A debugging hypothesis
Testing it
A claim about your repository
Looking
The one worth emphasising is tests. A generated test that passes tells you nothing until you have
confirmed it fails against broken code. A test asserting something trivially true is worse than no
test, because it appears in your coverage and reassures somebody.
The check takes seconds: break the thing deliberately, confirm the test fails, fix it, confirm it
passes.
The lowest-risk and highest-return use, worth a method rather than an ad-hoc question.
Start broad, then narrow. “What is this module responsible for?” before “what does this function
do?” before “why does this line check for null?”. Each answer gives you the vocabulary for the next
question.
Ask about the boundaries. “What calls into this, and what does it call out to?” locates the code in
the system, which is usually what you actually need.
Ask what would break. “If I changed this function’s return type, what would I need to update?” is a
question about coupling that a model reading the file can partly answer and that tells you what to
look at.
Cross-check against the IDE. Where your editor can show usages and definitions, use them.
The model’s answer about how many callers there are is an inference; the IDE’s is a fact.
Ask about the history when the code is surprising. Chat can explain what code does; it cannot
explain why it is like that unless the reason is written down. That question belongs to
history analysis, and switching to it is often the right move when an
explanation ends in “this appears to be a workaround for something”.
The failure to watch for: an explanation that describes what the code should do given its names. A
function called validateEmail that does not validate anything will be described as validating email
addresses, because the name is the strongest signal available. Read the explanation against the code,
not instead of it.
Distinct from the IDE surface and worth knowing about.
What it is good for: questions about a repository you do not have open — during review, when
triaging an issue, when investigating something on a phone. It has repository context without you
cloning anything.
What it does not have: your working tree, your uncommitted changes, your local environment. It
answers about the repository as committed.
Customisation differs. Repository-wide and organisation instructions apply; prompt files do not
work here at all.
The natural division: github.com chat for questions about the repository as it exists; IDE chat for
questions about the code you are currently changing.
Chat makes it easy to paste things, which makes it easy to paste the wrong thing.
Before pasting, look at what you are pasting. A config file with a real credential. A stack trace
containing a customer identifier. A database row from production. Once sent, it is sent.
Know your organisation’s position. Content exclusion, approved tooling, repositories with
restrictions. Worth establishing before it matters.
Prefer minimal reproduction. A trimmed example is usually a better prompt and discloses less —
the two goals align, which is unusual and worth taking advantage of.
Most surfaces now offer several modes, and using chat for something that wants an agent — or the
reverse — is a common inefficiency.
You want
Mode
An explanation, no changes
Chat / ask
A change to one region you have selected
Inline chat / edit
Changes across several files
Edit or agent
Something that needs to run and check
Agent
A plan before any changes
Plan mode, where available
The distinction that matters: chat produces text you apply; an agent applies it and observes the
result. For anything where the answer depends on what happens when you run it — a failing test, a
build error, an integration — the agent’s observe step is the whole value, and chat will produce a
confident guess instead.
Conversely, when you want to understand rather than change, chat’s inability to edit is a feature.
JetBrains’ ask mode exists for exactly this, and agent mode’s default-to-action
is unhelpful when you were still thinking.
The escalation habit from the VS Code lesson applies: start with the least
autonomous mode that can do the job, and move up when it cannot.
Chat is fast enough that it can substitute for understanding, which is a bad trade on anything you will
maintain.
Signals that you should be reading the code instead:
You have asked three follow-ups and are more confused. The explanations are not converging, which
usually means the model lacks context you have not identified.
You cannot evaluate the answer. If you could not tell a correct explanation from a plausible one,
you are accepting it on faith — which is fine for a library you will never touch and not fine for the
module you are about to change.
The answer contradicts what you observe. The code is authoritative. An explanation that does not
match the behaviour you are seeing is wrong, however confident.
You are about to make a decision that is hard to reverse. Architecture, a data migration, anything
touching money or auth. Understanding is the requirement, and chat is an aid to it rather than a
substitute.
The productive relationship is that chat accelerates reading — it tells you where to look and what a
dense fragment means — rather than replacing it. A ten-minute conversation that leaves you able to
explain the code to somebody else is a good use. One that leaves you able to quote an explanation you
could not defend is not.
Most chat surfaces expose shortcuts for common operations — variations on explain, fix, generate tests,
and generate documentation, plus ways to reference files and symbols directly in a prompt.
Two things worth knowing about them rather than a list that will age.
They are prompts with context pre-attached. An “explain” action is a prompt plus the current
selection. Knowing that demystifies them: where the shortcut does not exist, typing the equivalent with
the file attached gets the same result.
File and symbol references are the useful ones. Where a surface lets you reference a file or symbol
inline, that is explicit attachment with less friction — the deterministic context from the hierarchy
above, obtained without leaving the prompt.
The specific commands differ by surface and change between versions. The concept — pre-built prompts
and inline references — is what transfers.
Easy to think of instructions as an agent concern. They apply to
chat, and they are what stops you restating the same context in every conversation.
The things worth putting there specifically for chat’s benefit:
Your stack and versions. “Python 3.12, FastAPI, SQLAlchemy 2.x” prevents answers written for the
1.x API — a common and subtle source of plausible-but-wrong snippets.
Your test conventions. Framework, fixture style, where tests live. Saves attaching an example test
file every time.
Things that are counterintuitive. “All outbound HTTP must go through clients/base.py, which adds
tracing headers.” A model has no way to know this and will write requests.get forever without it.
What not to suggest. “Do not suggest adding new dependencies without noting it explicitly.” Useful,
because dependency suggestions are otherwise offered freely.
The effect is cumulative rather than dramatic: each conversation starts from a better baseline, and the
corrections you were making repeatedly stop being necessary. That is also the test for whether an
instruction is worth its context cost — are you currently correcting this more than once a week?
Chat is a well-read colleague who can see exactly what you show them and nothing else, who never says
“I don’t know”, and who has not run your code. Show them more, ask narrower questions, and check the
answer.
Ask what a specific function does, without attaching the file. Then attach it and ask again.
Predict: how much of the first answer was inferred from the name?
Ask for a test for that function. Run it. Predict: does it pass? Now break the function
deliberately. Does the test fail?
Introduce a real error, run the command, and paste the full output plus the file. Compare with
describing the error in your own words.
Ask for a snippet using a library you use. Predict: does every method it calls exist? Check the
documentation.
Have a long conversation about one problem, then switch to an unrelated one in the same thread.
Predict: does the earlier context help or interfere?
Ask “is this code good?” and then “does this function handle an empty input?” Compare how actionable
each answer is.