Git at Scale & Enterprise Engineering

13 min read

Everything in the previous seven pillars assumed a repository you can hold in your head and a team you can name.

Scale removes both assumptions, and it does not remove them gradually. A repository that clones in four seconds at 200 MB takes eleven minutes at 40 GB. A policy you can apply by hand to twelve repositories cannot be applied by hand to two thousand. An access list you can review over coffee becomes an access review programme. The techniques do not merely get harder — several of them stop working and have to be replaced.

Start with Large Repositories & Monorepos

Six kinds of scale

“Enterprise Git” is not one problem. It is six, and they arrive independently — an organisation can have severe repository-scale problems and trivial identity problems, or the reverse.

Repository scale. Millions of objects, deep history, large binaries, wide working trees, thousands of refs. This is a Git engineering problem, and the solutions are partial clone, sparse checkout, LFS and maintenance.

Team scale. Hundreds of engineers in one codebase. A code review process that worked for eight people does not survive eight hundred, and neither does an ownership model that lives in somebody’s memory.

Repository-fleet scale. Two thousand repositories that all need a CODEOWNERS file, a security policy and a current CI workflow. Nothing about any individual repository is hard; the aggregate is a different discipline.

Identity scale. Employees, contractors, acquisitions, machine identities. Joiners and leavers every week. Manual access management stops being merely tedious and becomes a control failure.

Automation scale. Thousands of workflow runs an hour, runner fleets, shared workflows consumed by hundreds of repositories, and the question of who is allowed to run what.

Governance scale. A security standard is easy to write and hard to apply consistently to repositories you have never seen, owned by teams you have not met.

The enterprise control path

Every action a developer takes at enterprise scale passes through a chain, and each link is somebody’s responsibility.

From identity to audit

Read it twice. Downward, it is an authorisation chain — each step narrows what is possible. Upward, it is an evidence chain — each step records what occurred. Enterprise engineering is mostly the work of making both directions reliable.

Repository scale is multidimensional

The most common mistake in this domain is treating “large repository” as one thing with one fix.

What actually makes a repository slow

Git Performance │ ├── Number of objects → clone and fetch cost ├── History depth → log, blame, bisect cost ├── Working-tree size → checkout and status cost ├── Binary content → storage, delta failure ├── Ref count → fetch negotiation, pack-refs ├── Packfiles → lookup cost, MIDX territory └── CI clone frequency → aggregate network and time

These produce different symptoms and take different remedies:

A small working tree with enormous history — a decade of commits on a modest codebase. git log and git blame are slow; checkout is fine. Commit-graph helps; sparse checkout does not.

An enormous working tree with shallow history — a recently imported monolith. Checkout and status are slow; history operations are fine. Sparse checkout helps; shallow clone does not.

Millions of tiny files. Index operations dominate. The sparse index is the relevant feature.

A few enormous binaries. Delta compression fails, every clone carries every version forever. This is Git LFS or, more often, an artifact store.

Thousands of refs. Fetch negotiation and ref iteration dominate. pack-refs and ref pruning are the answer.

Diagnose before prescribing. The single most useful habit in this cluster is measuring which dimension is actually large before adopting a technique that addresses a different one. git count-objects -v and a survey of the largest blobs answer more in two minutes than a week of speculation, and git-sizer reports every dimension above at once.

The large-repository toolkit

Techniques for large repositories

Large Repository │ ├── Partial Clone → fetch fewer objects ├── Sparse Checkout → populate fewer paths ├── Git LFS → move large blobs out of history ├── Maintenance → keep the local repo fast ├── Commit Graph → make history queries fast └── CI Clone Optimization → stop paying the cost per run

They compose, and they solve different problems. Partial clone reduces what is downloaded. Sparse checkout reduces what is written to disk. Confusing the two is the most common error in this area, and it produces engineers who enable sparse checkout and wonder why the clone is still slow.

The monorepo, precisely

Worth defining on the hub because the word is used loosely enough to cause real confusion.

A monorepo is one Git repository containing multiple logical projects or components that an organisation chooses to version together.

A large repository is not automatically a monorepo. A single application with a decade of history and a hundred thousand commits is enormous and contains one project. Its problems are Git-performance problems and none of the monorepo apparatus applies.

A monorepo is not automatically large. Five small services in one repository is a monorepo and clones instantly.

What a monorepo actually contains

One Repository │ ├── Service A ├── Service B ├── Library C ├── Frontend ├── Infrastructure └── Tooling Supported by: CODEOWNERS → who reviews which paths Sparse Checkout → what a developer materialises Partial Clone → what gets downloaded Path-Aware CI → what runs for a given change Build Graph → what actually needs rebuilding Repository Policy → what is permitted where

The Git repository alone does not make a monorepo work. The apparatus in the second group is what makes it habitable, and organisations that adopt a monorepo without building it experience the costs and none of the benefits. Git monorepos covers the whole system.

The governance layering model

Policy at enterprise scale is applied in layers, and the layering has a specific rule.

How policy narrows as it descends

Rules aggregate; they do not override. GitHub’s own documentation puts it plainly: creating a new ruleset can make the rules targeting a branch or tag more restrictive, but never less restrictive. A repository owner cannot relax an organisation rule by adding a repository rule — the union of all applicable rules applies, and the most restrictive requirement wins.

That single property is what makes enterprise governance tractable. It means a policy applied at the top cannot be undone below it, so you can reason about the floor without auditing every repository. Enterprise and organization rulesets works through the mechanics, including targeting and bypass.

From naming repositories to describing them

The shift that makes fleet-scale governance possible.

A policy that says “apply this to payments-api, payments-worker and payments-ledger” requires somebody to maintain a list. It is wrong the day a fourth service is created, and nobody notices.

A policy that says “apply this to every repository where data_classification is restricted” is correct for repositories that do not exist yet.

That is what custom properties are for — structured metadata attached to repositories, which rulesets and policies can target:

{
  "repository": "payments-api",
  "owner": "payments-platform",
  "service_tier": "critical",
  "production": true,
  "data_classification": "restricted",
  "lifecycle": "active"
}

That schema is an illustration, not a GitHub standard. The property names are yours to choose, and choosing them well is the actual work — a property nobody maintains is worse than no property, because policies target it and quietly miss.

The repository fleet

The mental shift that makes the last cluster make sense.

A repository fleet as an operational system

1,000 Repositories │ ├── Metadata → what is this, who owns it, how critical ├── Owners → a person or team accountable ├── Rulesets → the controls that apply ├── CI Standards → which shared workflow, at which version ├── Security → scanning, secrets, dependencies ├── Runtime Versions → what language, how old ├── Dependencies → what it consumes, what is vulnerable └── Lifecycle → active, maintenance, archived, orphaned

Each column is answerable for one repository by looking. None is answerable for a thousand without automation, and the gap between those two facts is the entire discipline.

Configuration drift is the fleet’s characteristic failure. No individual repository is wrong; the set has diverged. A workflow pinned to a version withdrawn a year ago. Twelve repositories with no CODEOWNERS. Forty on a runtime that stopped receiving security updates. Nothing failed, nothing alerted, and the aggregate is a risk nobody can see from any single repository.

Detecting and remediating that is repository fleet management, and it starts — as everything in this pillar does — with an inventory.

The distinction this pillar insists on, because conflating these terms produces real security gaps.

Concern	Mechanism	Answers
Authentication	SAML or OIDC SSO	Who are you?
Provisioning	SCIM	Does an account exist, and should it?
Membership	IdP groups and team mapping	Which teams are you in?
Authorization	Repository roles	What may you do here?

SSO is not deprovisioning. An organisation with SAML SSO and no SCIM has centralised the login and not the lifecycle: when somebody leaves, their identity provider account is disabled, and their GitHub account, their personal access tokens and their SSH keys keep working until a human remembers to remove them.

That gap is the single most common enterprise identity finding, and SCIM provisioning exists to close it. Enterprise IAM covers the whole architecture.

Where the boundary between Git and GitHub sits

Worth establishing early, because enterprise problems get misattributed constantly.

Git’s problems are about objects. Clone time, checkout time, history traversal, index size, packfile count. These are properties of the repository’s shape and they exist identically whether the remote is GitHub, GitLab or a bare repository on a server. Changing platform does not fix them.

GitHub’s problems are about governance and identity. Who can do what, across how many repositories, provisioned by whom, recorded where. These have nothing to do with Git and would exist if every repository were empty.

They meet in two places. Repository limits, where GitHub imposes ceilings on Git-shaped things — a 100 MB file rejection is GitHub’s decision, not Git’s. And CI, where the aggregate cost of Git operations becomes a platform-scale bill.

Why this matters practically: a team whose clones are slow does not need GitHub Enterprise. A team that cannot deprovision leavers does not need sparse checkout. Both are common, and both are addressed by the wrong half of this pillar surprisingly often — usually because whoever escalated the problem described it as “GitHub is slow” or “we need better GitHub security”, and nobody asked which layer was actually failing.

The first two clusters are deliberately ordered to reflect this: Git-shaped problems first, platform-shaped problems after.

What this pillar covers

Large Repositories
Monorepos, Git LFS, and the Git features that make enormous repositories workable.
10 lessons150+ min
Enterprise Governance
Enterprise architecture, rulesets, repository policies and metadata-driven governance.
10 lessons145+ min
Enterprise Identity
SSO, provisioning, managed users, machine identities and the access lifecycle.
8 lessons120+ min
Enterprise Operations
Audit logs, streaming, Actions governance, runner fleets and compliance evidence.
9 lessons135+ min
Repository Fleet
Operating hundreds of repositories as a system, and migrating them between platforms.
8 lessons120+ min

Large Repositories & Monorepos — engineering Git for repositories that exceed desktop-scale assumptions. Monorepo architecture, the monorepo-versus-polyrepo decision, LFS and binary strategy, sparse checkout and partial clone deployed as a developer-experience programme, CI clone economics, maintenance across a workstation fleet, the data structures that make large history fast, and GitHub’s actual repository limits.

Enterprise GitHub Governance — translating engineering standards into policy that applies to repositories you have never seen. Enterprise account architecture, internal repositories and innersource, rulesets and their layering, repository lifecycle policies, metadata-driven targeting, and custom roles for delegated administration.

Enterprise Identity & Access — connecting GitHub access to a central identity lifecycle. SAML and OIDC, SCIM provisioning, Enterprise Managed Users and what they genuinely cost you, team synchronisation, machine identities, and access reviews that find what automation missed.

Enterprise Operations & Compliance — running the platform. Audit logs and their retention, streaming to a SIEM, Actions governance and runner fleets, safely rolling out policy to thousands of repositories, API automation at scale, and what GitHub data can and cannot evidence for a compliance programme.

Repository Fleet Engineering & Migration — operating repositories as a managed system. Inventory, ownership, templates, standardised CI/CD, lifecycle, the Cloud-versus-Server decision, and migrations from planning through automation to validation.

Standardisation without rigidity

The tension that decides whether enterprise governance is experienced as support or obstruction, and it resolves into three categories that should never be confused.

Mandatory policy. A security or compliance control that must hold everywhere, enforced so it cannot be bypassed without a recorded decision. Signed commits on production repositories. No secrets in code. Required review on default branches. These are few, and being few is what makes them credible.

Golden defaults. A strong recommended starting point that a team can deviate from with a reason. A standard CI workflow, a repository template, a preferred branching model. Deviation is allowed and costs the team support rather than an argument.

Team customisation. Genuinely local decisions. Directory layout, test framework, review conventions, how many approvals a team wants beyond the mandatory floor.

The failure is promoting category three into category one. A platform team that centrally controls every repository setting has built a bottleneck and will be routed around. The organisations that govern well at scale are strict about a short list and deliberately permissive about everything else.

The reverse failure is real too. An organisation with no mandatory floor has a security posture equal to its least careful team, and no amount of documentation fixes that — which is why policy rollout treats moving something into the mandatory category as a change-management exercise rather than a settings change.

What this pillar will not tell you

Being explicit, because enterprise content attracts overclaiming.

It will not price anything. Plans, seats and quotas change and vary by agreement. Where a limit matters, verify it against your own account.

It will not claim a configuration makes you compliant. GitHub settings can produce evidence and enforce controls. SOC 2, ISO 27001, PCI DSS and the rest are programmes involving people, process and scope that no repository setting addresses. Compliance evidence is precise about the difference.

It will not present preview features as permanent. Several capabilities central to modern GitHub governance — repository policies, Actions policies, API request events, multiple audit streams — are in public preview at the time of writing. Each page says so where it matters, because building an architecture whose only control is a preview feature is a decision you should make knowingly.

It will not tell you that Enterprise Managed Users is the secure option. It is a different collaboration model with real costs, and the article covers both sides.

What changes when you cross the threshold

A rough guide to when each problem appears, because adopting solutions early is its own cost.

Under ~50 repositories and ~50 engineers. Most of this pillar is premature. Repository settings by hand are fine, access review is a conversation, and a monorepo decision can wait. The one thing worth doing early is an inventory habit, because it is much cheaper to start than to retrofit.

~50 to ~200 repositories. Fleet inconsistency becomes visible. Somebody notices that a third of repositories have no owner. Templates and organisation-level rulesets start earning their cost, and CODEOWNERS stops being optional.

~200 to ~1,000 repositories. Manual anything stops working. Metadata-driven targeting becomes necessary because enumerating repositories is no longer feasible. Identity automation moves from nice to load-bearing, because the joiner-leaver rate exceeds what a human tracks reliably.

Above ~1,000. The fleet is a system with its own operational discipline, and somebody’s job is running it. Migrations become programmes. Policy rollout requires evaluation before enforcement because the blast radius is the organisation.

Repository scale is independent of all of this. A ten-person team can have a 40 GB repository, and a thousand-person organisation can have nothing larger than 500 MB. The first cluster applies whenever the repository is large, not whenever the company is.

The number that actually predicts pain is neither repositories nor engineers — it is how many people must coordinate to make a routine change. When that number exceeds about three, something structural is wrong, and most of this pillar is about finding which layer it is in.

Reading order

The clusters build on each other and are usable independently if you have a specific problem.

Start at Large Repositories if clone times, checkout times or repository size are the pain. This cluster is Git engineering and applies regardless of platform.

Start at Governance if the problem is inconsistency — repositories that should have the same controls and do not.

Start at Identity if the problem is access: who has it, who should, and what happens when somebody leaves.

Start at Operations if the problem is visibility — you cannot answer what happened, or you cannot roll out a change safely.

Start at Fleet if you are managing hundreds of repositories as a set, or planning a migration.

A caution about sequencing at work. The instinct on arriving at a scaled organisation is to start with governance, because policy feels like the lever. In practice the ordering that succeeds is: inventory first (you cannot govern what you cannot enumerate), identity second (because a policy applied to the wrong people achieves nothing), then policy, then enforcement. Several articles here return to that ordering, because reversing it is the most common way an enterprise programme stalls in its first quarter.

The principle underneath

Git at enterprise scale is not simply Git with more users. Scale changes repository architecture, identity, governance, automation, observability, and operational responsibilities.

Every cluster is an application of that sentence. Repository techniques change because the object count changed. Governance becomes metadata-driven because you cannot enumerate the targets. Identity becomes a provisioning pipeline because the joiner-leaver rate exceeded manual capacity. Audit becomes streaming because retention windows are shorter than investigation timelines.

A team that has read this pillar should be able to look at a scaling problem and say which of the six kinds of scale it is, which layer owns the fix, and what the fix will cost.

Begin: Large Repositories & Monorepos