Kubernetes is already a reconciliation engine. You declare that you want three replicas; a controller notices there are two and creates one.
GitOps extends that loop outward by one step. Instead of the desired state arriving through kubectl from whoever ran it last, an agent inside the cluster reads it from a versioned source and continuously converges toward it. The cluster’s own reconciliation keeps running; what changes is where the declaration comes from and what happens when reality diverges from it.
What this cluster answers
Section titled “What this cluster answers”Eleven lessons — the largest cluster in the pillar, because this is where the concepts are load-bearing.
The loop
Section titled “The loop”A vertical chain: a developer commits; the Git repository holds desired state; a GitOps controller pulls it; the controller applies through the Kubernetes API; producing actual state; and the controller continuously re-compares actual against desired.
The arrow that matters is the last one, and it points backwards. Without continuous reconciliation this is a deployment pipeline drawn vertically. With it, the system has a defined behaviour when somebody changes the cluster by hand — which is the property teams actually adopt GitOps to get.
GitOps, defined properly
Section titled “GitOps, defined properly”The OpenGitOps project publishes four principles, and this cluster follows them rather than the marketing usage. Version 1.0.0, verbatim:
- Declarative — “A system managed by GitOps must have its desired state expressed declaratively.”
- Versioned and Immutable — “Desired state is stored in a way that enforces immutability, versioning and retains a complete version history.”
- Pulled Automatically — “Software agents automatically pull the desired state declarations from the source.”
- Continuously Reconciled — “Software agents continuously observe actual system state and attempt to apply the desired state.”
Two things are worth noticing immediately.
The principles say “the source”, not “Git”. An OCI registry holding versioned, immutable artifacts satisfies principle 2 perfectly well, and Flux’s OCIRepository is built on exactly that. GitOps is not defined by the storage being Git.
Principles 3 and 4 are what most “GitOps” pipelines are missing. A workflow that runs kubectl apply on merge satisfies 1 and 2. It does not pull, and it does not continuously reconcile — it acts once and stops. That is a perfectly good deployment pipeline, and calling it GitOps costs you the vocabulary to describe what you would gain by adding a controller.
GitOps explained works through all four properly.
Push and pull
Section titled “Push and pull”| Push (pipeline applies) | Pull (controller reconciles) | |
|---|---|---|
| Who holds cluster credentials | The CI system, externally | Nothing external — the agent is inside |
| Network direction | Inward, to the cluster API | Outward, from the cluster |
| When state changes | On pipeline run | Continuously |
Manual kubectl change | Persists silently | Detected; optionally reverted |
| Ordered, imperative steps | Natural | Awkward |
| Multi-cluster | Credentials per cluster | An agent per cluster |
| Failure visibility | The pipeline run | Controller status, continuously |
Neither wins. Push handles ordered operations — a database migration that must complete before the new version starts — which reconciliation models express badly. Pull handles convergence and self-healing that push models cannot express at all.
Mature platforms run both and are explicit about which changes take which path. The mistake is not choosing one; it is having both and no rule about which is authoritative for a given resource, so they fight.
Manifests, Kustomize or Helm
Section titled “Manifests, Kustomize or Helm”Three ways to get YAML into a repository, and the choice affects diff quality more than anything else.
Plain manifests are the most reviewable and the least reusable. A pull request diff is exactly what will be applied. For a small number of environments this is genuinely fine.
Kustomize overlays patches onto a base. No templating language, output is still YAML, and kustomize build shows exactly what a change produces. The diff review story is good.
Helm templates charts with values. Far better for packaging and distributing something other people install; harder to review, because a values change and the manifests it produces are separated by a template. helm template in CI closes most of that gap.
Most real repositories use more than one — Helm for third-party components, Kustomize or plain manifests for their own workloads.
Argo CD and Flux
Section titled “Argo CD and Flux”Two mature controllers, both CNCF graduated, both correct implementations of the principles above.
Argo CD is application-centric with a strong web UI. An Application resource points at a source and a destination. Teams that want visible sync status and a console gravitate here.
Flux is a set of composable controllers with no bundled UI. Sources, Kustomizations and HelmReleases are separate resources you assemble. Teams that want everything expressible as Kubernetes objects gravitate here.
The comparison is deliberately not a verdict. They differ in operating model far more than in capability, and the right answer depends on who operates the platform.
The hard parts
Section titled “The hard parts”Secrets — desired state is in a repository and secrets cannot be. This is the genuinely difficult problem in GitOps, and base64 is not a solution to it.
Promotion — moving a tested artifact toward production without rebuilding it, and what “rollback” means when the change was a schema migration.
Drift — what happens when somebody fixes production by hand at 3am, and how to make the incident recoverable rather than silently reverted.