Git was designed for the Linux kernel, which is a large repository by any ordinary standard and small compared with what several organisations now operate.
At a certain point the defaults stop working. A clone that takes eleven minutes is a clone developers avoid, and a CI system paying that cost on every run is paying it thousands of times a day. This cluster is about the engineering that makes such repositories habitable — and about knowing which technique addresses which problem, because they are frequently confused.
Start with Git MonoreposWhat this cluster answers
Section titled “What this cluster answers”Pillar 2 explains the features; this cluster deploys them
Section titled “Pillar 2 explains the features; this cluster deploys them”The distinction matters, because three of these topics already have articles on this site.
Pillar 2’s sparse checkout article explains cone mode, the commands and the behaviour. Sparse checkout at scale covers rolling it out to four hundred engineers: which persona gets which cone, how the configuration is generated, what breaks in their IDE, and how you measure whether it helped.
Pillar 2’s partial clone article explains object filters. Partial clone at scale covers promisor remotes, the failure mode when the remote is unreachable mid-operation, and why the interaction with CI caching is the part that bites.
Pillar 2’s maintenance article explains the tasks. Maintenance at scale covers deploying it across a workstation fleet and why gc is disabled in the default strategy.
Read the Pillar 2 article first if the feature is new to you. These pages assume you know what the command does and are asking how to operate it.
Diagnose before you prescribe
Section titled “Diagnose before you prescribe”The most valuable habit in this cluster, and the one most often skipped.
“The repository is too big” is not a diagnosis. Repository size is at least seven independent dimensions, and each has a different remedy:
| Symptom | Likely dimension | Relevant technique |
|---|---|---|
| Clone is slow, checkout is fast | Object count, history depth | Partial clone |
Checkout and status are slow | Working-tree size, file count | Sparse checkout |
log and blame are slow | History depth | Commit-graph |
| Repository grows without new code | Binary content | LFS or artifact storage |
| Fetch is slow, everything else fine | Reference count | pack-refs, ref pruning |
| Gradual slowdown over months | Packfile count, loose objects | Maintenance |
| CI is expensive, laptops are fine | Clone frequency | CI clone optimization |
Measure first. git count-objects -v gives object and pack counts in a second. git-sizer reports every dimension in the table at once and is the right first move on any repository somebody has described as slow.
The three techniques people confuse
Section titled “The three techniques people confuse”Stated once, clearly, because the confusion recurs throughout the cluster.
Partial clone changes what is downloaded. --filter=blob:none clones commits and trees but no file contents; blobs arrive on demand from a promisor remote. History is complete.
Sparse checkout changes what is written to the working tree. The repository still contains everything; your directory contains a subset.
Shallow clone changes how much history exists at all. --depth=1 gives you one commit. It is lossy: merge-base calculations, git describe, blame and anything needing ancestry either fail or lie.
They compose. A large-monorepo developer setup is frequently partial clone plus sparse checkout and no shallow clone at all — full history, few objects downloaded, few files on disk.
Shallow is the CI tool, and even there not always the right one, because a workflow computing a diff against a base branch needs a merge base that a depth-1 clone does not have.
Monorepo is an architecture, not a size
Section titled “Monorepo is an architecture, not a size”Two articles open this cluster and they answer different questions.
Git monorepos is the architectural guide: what a monorepo is, what apparatus it requires, and why the Git repository alone does not make one work.
Monorepo vs polyrepo is the decision guide, for software engineering organisations generally. It deliberately does not repeat Pillar 7’s version, which is specifically about infrastructure repository boundaries.
The definition to carry through both: a monorepo is one repository containing multiple logical projects an organisation chooses to version together. A repository can be enormous and contain one project — that is a large repository with Git-performance problems, and none of the monorepo machinery applies to it.
Binaries deserve their own decision
Section titled “Binaries deserve their own decision”Two articles, because the mechanism and the strategy are separable.
Git LFS is the mechanism: pointer files, external object storage, .gitattributes, migration and the operational costs.
Binary file strategy is the decision: for a given kind of large file, should it be in Git at all, in LFS, in a package registry, in release assets, in object storage, or in a container registry?
The answer for most generated binaries is none of the first two. A compiled artifact that a build produces from source does not belong in version control in any form, and LFS is frequently adopted as a way of avoiding that conversation.
Learning objectives
Section titled “Learning objectives”By the end of this cluster you should be able to:
Diagnose which dimension of a repository is actually large, using object counts and blob analysis rather than the reported clone time.
Decide whether a monorepo is the right architecture for a given organisation, and name the apparatus it would require.
Choose between Git, LFS, a package registry, release assets and object storage for a specific class of large file, and justify the choice.
Deploy sparse checkout to a large engineering organisation as a developer-experience programme rather than a command people are told to run.
Explain the difference between partial clone, sparse checkout and shallow clone without hesitating, including which one breaks merge-base calculations.
Design a CI checkout strategy per workflow type, knowing which workflows genuinely need full history.
Roll out git maintenance across a workstation fleet, and say why the default strategy disables gc.
Explain what the commit-graph and multi-pack-index do and why they exist.
State GitHub’s repository limits from documentation rather than memory, and recognise the symptoms of approaching them.
Enterprise use cases
Section titled “Enterprise use cases”Where this cluster’s material actually gets used.
A monorepo migration. An organisation consolidating dozens of repositories into one, needing sparse checkout, path-aware CI and an ownership model before the merge rather than after.
An inherited large repository. A ten-year-old codebase where clones now take twenty minutes and nobody knows why. This is the diagnosis-first case, and the answer is frequently binaries nobody noticed accumulating.
A game or media studio. Enormous binary assets, LFS in production, file locking, and a repository whose size is dominated by content rather than code.
A CI cost investigation. A platform team finding that checkout is a material fraction of the compute bill, which turns out to be the cheapest problem in this cluster to fix.
A repository approaching a GitHub limit. A push rejected at 2 GB, or a repository nearing the 10 GB recommendation, needing a remediation plan rather than a support ticket.
An acquisition. Repositories arriving from another platform with unknown shape, needing assessment before migration — which is where this cluster meets the fleet cluster.
What this cluster is not
Section titled “What this cluster is not”Not a Git tutorial. The commands are covered in Pillars 1 and 2.
Not a build-system guide. Monorepos need a build graph, and choosing one is out of scope.
Not infrastructure repository architecture. Pillar 7 covers that.
Not a claim that big repositories are a mistake. Several of the largest and most productive engineering organisations run one repository deliberately. The cost is real and payable.
The order these techniques should be adopted
Section titled “The order these techniques should be adopted”Teams frequently adopt the most-discussed technique rather than the most valuable one. A rough sequence that reflects cost against benefit:
1. Measure. git-sizer, object counts, largest blobs. An afternoon, and it changes what you do next.
2. Stop the bleeding. If binaries are accumulating, fix that first — every day of delay adds permanent history. Binary file strategy before anything else.
3. Fix CI checkout. The highest-value change for most organisations, because the cost is paid thousands of times a day rather than once per developer per week. Frequently a one-line change to a workflow.
4. Enable maintenance. Cheap, invisible, and it stops the gradual degradation that makes a repository feel worse every quarter.
5. Partial clone for developers. Reduces onboarding clone time substantially with no workflow change.
6. Sparse checkout, if the working tree is the problem. Higher cost — it needs a persona model, generated configuration and support — and it is the technique that most needs the diagnosis in step 1 to justify it.
7. History rewriting, last and reluctantly. Removing large objects from history is the only way to shrink an existing repository, and it invalidates every clone. Repository health covers when it is warranted and how to coordinate it.
Steps 3 and 4 are the ones with the best return and the least attention. Sparse checkout gets the conference talks; a corrected fetch-depth gets the actual savings.