Helm + Git: Charts, Values and Versioned Releases

Helm’s strength is packaging: a versioned, distributable unit that somebody else can install without reading your manifests.

Its weakness in a Git workflow is the same property. A values file and the resources it produces are separated by a templating language, so a pull request diff shows the input rather than the effect. Closing that gap is most of what this lesson is about.

Before you start

• Kubernetes + Git — what belongs in the repository
• Kustomize + Git — the non-templating alternative

Availability

Two versions, two meanings

Chart.yaml carries both and conflating them causes real confusion.

apiVersion: v2
name: api
description: The API service
type: application
version: 2.4.1        # The chart's version
appVersion: "1.18.0"  # The application's version

version is the chart’s. Bump it when the templates, defaults or structure change — a new value, a changed resource, a fixed template bug.

appVersion is the software the chart deploys. Bump it when the application changes.

They move independently. A chart fix with no application change bumps version only. An application release with no template change bumps appVersion only.

version follows semantic versioning for the chart’s interface, which is its values. Removing a value or changing its meaning is a major bump; adding an optional one is minor.

appVersion is a string and does not have to be semver, though it usually is.

The mistake to avoid: keeping them in lockstep. Bumping the chart version because the application released produces a chart history where every version looks like a change to the chart, and nobody can tell which releases actually changed the templates.

Where charts live

In the application’s repository, under charts/, when the chart exists only to deploy that application. Chart and code change together, which is usually what you want.

In a chart repository, when several applications share it or when it is published for others. Then it is a versioned artifact with the same module-style discipline: tagged releases, semantic versioning, release notes.

In an OCI registry, which is now the standard distribution mechanism. Helm’s OCI support is generally available and a chart is pushed and pulled like any other OCI artifact.

helm package charts/api                          # produces api-2.4.1.tgz
helm push api-2.4.1.tgz oci://ghcr.io/example-org/charts
helm pull oci://ghcr.io/example-org/charts/api --version 2.4.1

Note the asymmetry: helm push takes the repository path without the chart name or version — both are inferred from the packaged file. helm pull and helm install include the chart name.

Digest references work, which is the property this pillar cares about:

helm install api oci://ghcr.io/example-org/charts/api@sha256:0000000000000000000000000000000000000000000000000000000000000000

A chart version tag is mutable the way an image tag is. A digest is not. For anything deployed to an environment that matters, the digest is the honest reference.

Values, per environment

The base chart holds defaults. Environment values override them.

charts/api/
├── Chart.yaml
├── values.yaml          # Defaults — safe, minimal
└── templates/

environments/
├── dev/values.yaml
├── staging/values.yaml
└── production/values.yaml

environments/production/values.yaml

replicaCount: 6

image:
  repository: ghcr.io/example-org/api
  digest: sha256:0000000000000000000000000000000000000000000000000000000000000000

resources:
  requests:
    cpu: 500m
    memory: 512Mi
  limits:
    memory: 1Gi

autoscaling:
  enabled: true
  minReplicas: 6
  maxReplicas: 20

Defaults should be safe rather than production-shaped. A chart whose defaults deploy a single replica with modest resources fails safe if somebody installs it without values. Defaults sized for production mean an accidental install in the wrong place is expensive.

Never put secrets in a values file. They are committed, they are plaintext, and a values file is exactly where people put a database password. GitOps secrets covers the alternatives.

Prefer a digest to a tag in the image values, for the same reason as everywhere else in this pillar.

Making values reviewable

The core problem, and it has a mechanical solution.

helm template api charts/api -f environments/production/values.yaml > /tmp/new.yaml
git stash
helm template api charts/api -f environments/production/values.yaml > /tmp/old.yaml
git stash pop
diff -u /tmp/old.yaml /tmp/new.yaml

Run this in CI for every environment a change touches and post the difference as a pull request comment.

Why it matters more than for Kustomize. A Kustomize patch looks like the resource it patches; a reviewer can usually predict the effect. A Helm values change goes through a template, and predicting the effect requires reading the template. Rendering removes the guesswork.

helm template does not contact the cluster and needs no credentials, which makes it safe to run on pull requests from anywhere.

helm diff as a plugin compares against the release currently installed, which additionally catches drift. It needs cluster access, which is a real trade-off.

Caution

The specific failure this prevents: a one-line values change enabling a feature flag that a template uses in six places, one of which alters a Deployment’s selector labels. The values diff is one line. The rendered diff shows an immutable field change, which will fail to apply and require deleting the Deployment. Only one of those is visible without rendering.

Dependencies

Chart.yaml

dependencies:
  - name: postgresql
    version: "<pinned>"
    repository: oci://registry-1.docker.io/bitnamicharts
    condition: postgresql.enabled

Pin exact versions. A range means helm dependency update can resolve differently on different days, which reintroduces the mutability this pillar keeps arguing against.

Commit Chart.lock. It records what a range actually resolved to. Committing it is the same decision as committing any lockfile, and for the same reason.

Decide about charts/*.tgz. Vendoring the dependency archives makes builds offline and deterministic; not vendoring keeps the repository small and requires a fetch. Both are defensible; vendoring is usually right for a repository a GitOps controller reconciles, because a network failure during a fetch becomes a failed reconciliation.

condition: lets a dependency be disabled. Useful for a chart that bundles a database for development and uses a managed one in production — but be careful that the two paths are genuinely tested, because a dependency that is only enabled in dev is a dependency whose production path nobody exercises.

Be sceptical of large dependency trees. A chart that pulls in four subcharts deploys forty resources you did not write. Read what it creates before adopting it, not after the first bill.

Helm 4

The current major version, with a handful of changes that affect scripts and pipelines.

Renamed flags. --atomic became --rollback-on-failure; --force became --force-replace. The old names still work and emit deprecation warnings — which means a pipeline using them works today and will need updating, and the warning is easy to miss in CI output.

--post-renderer takes a plugin name, not an executable path. Any workflow using a post-renderer script needs migrating to the new plugin system.

helm registry login accepts a domain name only — no scheme, no path. A login command carrying https:// fails.

Charts remain compatible. A chart deployable with Helm 3 is generally deployable with Helm 4, and Helm 4 manages existing Helm 3 releases without a migration step. That is the important compatibility guarantee and it makes the upgrade low-risk for the charts themselves.

The risk is in the tooling around it. Scripts using renamed flags, post-renderer workflows, and registry login commands. Grep your workflows for those three before upgrading — the flag renames in particular fail silently for now, which means a pipeline can be running on deprecated flags for months before anybody notices the warning.

Check what your GitOps controller uses. A controller embeds its own Helm implementation, which may be on a different major version from your CLI. A chart that renders correctly locally and differently under the controller is usually this.

With Argo CD and Flux

Neither installs charts by running helm install; both render and apply.

Argo CD renders the chart with helm template and applies the output. Consequences: no Helm release object exists in the cluster, helm list shows nothing, and Helm hooks are translated to Argo CD’s own hook mechanism rather than run by Helm.

Flux uses a HelmRelease resource (helm.toolkit.fluxcd.io/v2) and does maintain Helm releases. The current recommendation is chartRef pointing at an OCIRepository, rather than the older chart.spec.sourceRef pattern — the latter still works and appears in most existing examples.

Values come from the repository in both cases: inline in the resource, or referenced from a ConfigMap or Secret.

The practical difference: Flux gives you Helm’s release lifecycle, including rollback and drift detection against the release. Argo CD gives you a rendered-manifest model where Helm is a templating step. Neither is wrong; they lead to different debugging experiences, and it is worth knowing which you have when something behaves unexpectedly.

Chart testing

A chart is consumed by people who cannot easily verify it, which raises the bar.

helm lint catches structural problems. Cheap, necessary, insufficient.

helm template with every values file you support. This catches the most common real defect: a template that breaks when an optional value is absent.

Schema-validate the rendered output against the Kubernetes version you target.

values.schema.json validates the values themselves, so a typo in a values file fails with a clear message rather than rendering something unexpected. Underused and worth the effort for any chart others consume.

helm test runs test hooks against an installed release. The only layer that exercises the chart in a real cluster.

Test the defaults. helm template charts/api with no values file at all should produce something valid. A chart that only works with a values file has undocumented required values.

Releasing a chart

A chart consumed by anybody other than its author is a published artifact, and the release discipline applies.

1. Change the chart on a branch. Templates, defaults, schema.

2. Decide the version. Removing a value or changing its meaning is major; adding an optional one is minor; a template fix is a patch. Apply this to the values interface, not to how much the templates changed internally.

3. Update the chart’s README. For a chart others consume, the values table is the interface documentation, and generating it from the schema keeps it accurate.

4. Render every supported values combination in CI.

5. Merge, then tag. The Git tag is the release decision, as it is for images.

6. Package and push to the registry from CI, on the tag.

7. Write release notes describing what changed for a consumer — which values moved, what a consumer must do.

Breaking changes need a migration note. A renamed value silently reverts to its default when a consumer upgrades, because Helm does not know the old name was meaningful. That is worse than an error: the deployment succeeds with different behaviour. Where possible, keep the old value working for one major version and warn in a template comment or a NOTES.txt.

Do not move a published chart version. Same rule as image tags: a consumer who pulled 2.4.1 yesterday and pulls it again today must get the same chart. If a release is wrong, publish 2.4.2.

Test upgrades, not just installs. A chart that installs cleanly can fail on upgrade — an immutable field changed, a hook that assumes a fresh install, a removed resource that leaves an orphan. Install the previous version, then upgrade to the new one, in CI.

Debugging a chart

The failures are distinctive and the diagnosis is usually fast if you know where to look.

helm template --debug renders and shows what it produced even when the output is invalid. This is the first thing to reach for.

A rendered manifest that is structurally wrong is almost always indentation in the template. Helm templates produce text, not YAML — a misplaced nindent produces valid text and invalid YAML, and the error is reported at a line number in the rendered output rather than in the template.

helm get manifest RELEASE shows what a release actually applied, which is the ground truth when the cluster disagrees with what you expected.

helm get values RELEASE shows what values a release was installed with, including ones from files nobody can find any more. Under Argo CD there is no release, so this does not apply — another reason to know which model you are on.

“cannot patch: field is immutable” is a selector or another immutable field. Rendering the diff before applying would have caught it.

Values that appear ignored are usually a nesting mismatch — a value set at the top level that the template reads from under a subchart’s key, or vice versa. helm template --debug with --set on the command line confirms which path the template actually reads.

Subchart values need the subchart’s name as a prefix. postgresql.auth.password, not auth.password. This is the single most common values confusion and it produces silence rather than an error.

Writing templates people can review

Chart authorship choices that make the rendered diff easier to reason about.

Keep conditionals shallow. A template with four nested if blocks produces output nobody can predict from the values. Two levels is a reasonable ceiling.

One value should change one thing. A flag that alters six unrelated resources is a flag whose effect nobody remembers. Where a group of changes genuinely belong together, name the value after the outcome rather than the mechanism.

Avoid computing names. A resource whose name is assembled from three values is a resource whose name changes unexpectedly — and a changed name means delete-and-create rather than update.

Do not put logic in values. A value containing a template string is evaluated in a way that is hard to follow and harder to review.

Use values.schema.json. It turns a typo from silent-default into a clear error, and it documents the interface in a machine-readable way.

Name things predictably. Helm’s convention of {{ include "chart.fullname" . }} exists so resource names follow from the release name. Deviating means somebody cannot find the resources a release created.

Keep NOTES.txt short and accurate. It is the first thing a consumer sees after installing, and a stale one that references removed values is worse than none.

The test for a reviewable chart: can somebody change one value, read the rendered diff, and understand why every changed line changed? If the answer needs the template open alongside, the templates are doing too much.

Common mistakes

Bumping version and appVersion together always. Nobody can tell which releases changed the chart.

Secrets in a values file. Committed plaintext.

Reviewing the values diff rather than the rendered diff. One line in, forty resources out.

Version ranges in dependencies without a committed lock. Different resolutions on different days.

Production-shaped defaults. An accidental install is expensive.

A tag rather than a digest for the image. Mutability in the deployment path.

Assuming your controller uses your CLI’s Helm version. They frequently differ.

Adopting a chart without reading what it creates. Forty resources you did not write.

Only testing with the production values file. Optional-value paths break silently.

Third-party charts

Most Helm usage is consuming somebody else’s chart, and that is a supply-chain decision.

A chart runs templates you did not write and creates resources you did not specify. A widely used chart can produce sixty resources including RBAC, CRDs and webhooks from a fifteen-line values file. Read what it creates before the first install — helm template against your values, into a file, and actually look at it.

Pin the version, always. A floating dependency means an upgrade happens when the maintainer publishes rather than when you decide.

Prefer charts from the project that maintains the software. A community chart for a project that also publishes one is a fork with a different lifecycle.

Check what RBAC it requests. A chart creating a ClusterRole with broad verbs is granting itself that access. Sometimes necessary; always worth reading.

Watch for CRDs. Helm’s CRD handling is deliberately conservative — CRDs installed by a chart are typically not upgraded or deleted by Helm, which means chart upgrades can leave CRD schemas stale. Managing CRDs separately from the chart that uses them is more work and avoids a category of confusing failure.

Consider vendoring for anything critical. Copying a chart into your own repository costs you upstream fixes and buys you control over when anything changes. For a chart deploying something central, that trade is often correct — and it also means you can read the templates in your own review process.

Override, do not fork, where you can. Kustomize post-rendering handles the case where a chart does not expose the value you need, without taking on maintenance of the whole chart.

Mental model

A chart is a package with a versioned interface: its values. A values file is a configuration of that interface. The rendered output is the only thing that reaches the cluster — and the only thing worth reviewing.

That framing settles most questions. Version the chart when the interface changes. Keep defaults safe because somebody will install without values. Render in CI because the input is not the effect.

What you learned

• version is the chart’s, appVersion is the application’s; they move independently
• Helm’s OCI registry support is generally available, and charts can be referenced by digest
• helm push omits the chart name and version; helm pull and install include them
• Chart defaults should be safe, not production-shaped
• Render with helm template in CI and diff — the values diff does not show the effect
• Pin dependency versions, commit Chart.lock, and consider vendoring for GitOps repositories
• Helm 4 renamed --atomic to --rollback-on-failure and --force to --force-replace; charts stay compatible
• Argo CD renders charts and applies output; Flux maintains actual Helm releases

Exercise

Use a disposable local cluster and repository.

1. Create a chart with a Deployment whose replica count comes from a value. Render it with helm template and no values file. Predict: does it produce something valid?

2. Add two environment values files differing in replica count. Render both and diff.

3. Add a template conditional that changes a label when a feature flag value is true. Change only the flag in a values file. Diff the values, then diff the rendered output. Compare how much each tells you.

4. Add values.schema.json requiring image.repository. Render with it missing. Predict: is the error clearer?

5. Add a dependency with a version range. Run helm dependency update and inspect Chart.lock.

6. Bump appVersion without touching version. Predict: does anything about the packaged chart change?

7. Run helm lint and helm template in a CI step for every values file.

8. Delete the cluster.

Related lessons

Kustomize + GitPatching instead of templating.

FluxHelmRelease and chartRef.

Argo CDRendering charts as manifests.

GitHub Container RegistryPublishing charts as OCI artifacts.