Docker Compose with Git: Environments, Overrides and Secrets

Compose is the file that gets a new engineer running in ten minutes, and the file that most often carries a real credential into a repository.

Both facts come from the same property: it describes a complete working environment, and complete working environments need configuration. The discipline is keeping the description in the repository and the values out of it.

Before you start

• Docker + Git Workflow — how images get built
• Preventing .env file commits — the specific failure this lesson guards against

What Compose is for

Worth stating, because scope creep here is the source of most Compose problems.

Good uses: a local development environment with the services an application depends on; integration tests in CI that need a real database rather than a mock; a demonstration or review environment; a small single-host deployment where an orchestrator would be disproportionate.

Not an orchestrator. No scheduling across hosts, no rolling updates with health-gated rollback, no reconciliation loop, no autoscaling, no self-healing after a host failure. It starts containers on one machine and keeps them running according to a restart policy. That is a genuinely useful thing to do and it is a smaller thing than an orchestrator does.

Not a deployment tool for anything with availability requirements. Teams routinely outgrow it without noticing, and the symptom is recognisable: a Compose file with three environment-specific overrides, a shell script wrapping the invocation, and a runbook describing the manual steps around it. Each of those was individually reasonable; collectively they are an orchestrator somebody wrote by accident.

The files

The current specification uses compose.yaml as the canonical name; docker-compose.yml still works and is what you will see in older repositories.

compose.yaml               # Base definition — committed
compose.override.yaml      # Local development — committed
compose.ci.yaml            # CI adjustments — committed
.env.example               # Documented variables, placeholder values — committed
.env                       # Real local values — NEVER committed

compose.override.yaml is loaded automatically alongside compose.yaml. Others are explicit:

docker compose up                                        # base + override
docker compose -f compose.yaml -f compose.ci.yaml up     # base + CI

Everything except .env is committed. The override file is committed deliberately: it is the shared development configuration, not a per-developer one. That distinction is worth defending, because the natural drift is for the override file to accumulate one person’s preferences — a port they like, a mount for a directory only they have — and then to stop working for everybody else.

Personal deviations belong in .env, or in a file listed in .gitignore and passed explicitly with -f. If two developers genuinely need different Compose configuration for the same task, that is usually a signal that something environment-specific has leaked into the base file.

Base and override

The base file describes structure. Overrides adjust it.

compose.yaml

services:
  api:
    build:
      context: .
      target: runtime
    environment:
      NODE_ENV: production
      DATABASE_URL: ${DATABASE_URL:?DATABASE_URL is required}
    depends_on:
      db:
        condition: service_healthy
    ports:
      - "3000:3000"

  db:
    image: postgres:17-alpine
    environment:
      POSTGRES_USER: ${POSTGRES_USER:-app}
      POSTGRES_PASSWORD: ${POSTGRES_PASSWORD:?POSTGRES_PASSWORD is required}
      POSTGRES_DB: ${POSTGRES_DB:-app}
    volumes:
      - pgdata:/var/lib/postgresql/data
    healthcheck:
      test: ["CMD-SHELL", "pg_isready -U ${POSTGRES_USER:-app}"]
      interval: 5s
      timeout: 3s
      retries: 10

volumes:
  pgdata:

# compose.override.yaml — development
services:
  api:
    build:
      target: dev
    environment:
      NODE_ENV: development
    volumes:
      - ./src:/app/src
    command: npm run dev

Three things worth copying.

${VAR:?message} fails fast when a required variable is missing, with your message rather than an obscure downstream error. ${VAR:-default} supplies a default for optional ones. Using both makes the file self-documenting about which variables are mandatory.

No default for POSTGRES_PASSWORD. A default password in a committed file is a password somebody will carry into an environment that matters.

depends_on with condition: service_healthy waits for the database to be ready rather than merely started. Without the health check, depends_on only waits for the container to exist, and the application starts against a database that is not yet accepting connections.

.env is the dangerous file

The single most common secret-commit vector in container repositories.

Commit .env.example with every variable documented and obviously fake values:

# Copy to .env and fill in. Never commit .env.
POSTGRES_USER=app
POSTGRES_PASSWORD=change-me-locally
POSTGRES_DB=app
DATABASE_URL=postgres://app:change-me-locally@db:5432/app

# Obtain from the team password manager — do not put a real value here.
STRIPE_API_KEY=

Ignore .env with a default-deny pattern:

.env
.env.*
!.env.example

Deny broadly, allow the example back. A new .env.staging is ignored automatically rather than depending on somebody remembering.

Enable push protection. It blocks recognised credential patterns at push time, which catches the case where somebody used git add -f.

Danger

An .env file committed with a real credential is a credential exposed in every clone and every fork. The response is the rotation-first sequence: rotate the credential, then add the ignore rule, then consider history. History rewriting does not reach existing clones, and the credential works the whole time you are doing it.

Image references

Compose is a deployment description, so the tag versus digest argument applies.

services:
  db:
    image: postgres:17-alpine        # Development: readable, floating

services:
  api:
    image: ghcr.io/example-org/api@sha256:0000000000000000000000000000000000000000000000000000000000000000

For development, a readable tag is fine. Reproducibility matters less than a file people can read, and a floating postgres:17-alpine picking up patches locally is desirable.

For anything deployed, use a digest. What was tested and what runs should be the same content.

Do not use latest anywhere. It is a tag like any other with no semantics beyond convention, and it changes without notice.

Compose in CI

Genuinely useful for integration tests, with adjustments.

compose.ci.yaml

services:
  api:
    build:
      target: runtime
    environment:
      NODE_ENV: test
    ports: !reset []

  db:
    tmpfs:
      - /var/lib/postgresql/data
    volumes: !reset []

Test the runtime target, not the dev one. Otherwise CI tests an image you do not ship.

!reset [] removes inherited values rather than adding to them. Compose’s default merge behaviour appends to lists, so without it a CI override that omits ports still inherits the base file’s. Published ports on a runner are unnecessary and can collide with other jobs.

The companion tag is !override, which replaces an attribute wholesale rather than merging into it:

services:
  api:
    command: !override ["npm", "run", "test:integration"]

Both exist because the default merge is additive, and additive merging is right for the common case and wrong when you specifically want less than the base file has. Knowing they exist saves the workaround people otherwise reach for: duplicating the whole service definition in the override file, which then drifts.

tmpfs for the database makes tests faster and guarantees a clean state per run.

Secrets come from CI secrets, injected as environment variables. The .env file does not exist on a runner and should not be created there — a workflow step that writes one is a workflow step that has put a credential on disk where a later step, or a debug log, can pick it up.

Test credentials are still credentials. A password for a throwaway Postgres container that lives for ninety seconds does not need to be a real one, and it does not need to be in the repository either. Generate it in the workflow, or use a value that is obviously a placeholder and could not work anywhere else.

      - name: Integration tests
        env:
          POSTGRES_PASSWORD: ${{ secrets.TEST_DB_PASSWORD }}
        run: |
          docker compose -f compose.yaml -f compose.ci.yaml up -d --wait
          docker compose -f compose.yaml -f compose.ci.yaml exec -T api npm test
          docker compose -f compose.yaml -f compose.ci.yaml down -v

--wait blocks until health checks pass, which removes the sleep-and-hope pattern. down -v removes volumes so a failed run does not affect the next.

Reviewing a Compose change

Compose files change rarely and matter disproportionately when they do, which is exactly the profile of a file that gets skimmed.

A new service. What is it, where does its image come from, and does it need to be exposed?

A new published port. ports: binds to the host. On a shared machine or a CI runner that is a wider exposure than expose:, which only opens the port to other services on the Compose network. The distinction is worth knowing: most services need expose, not ports.

A new volume mount, especially a bind mount. - /:/host in a development file is somebody debugging; - /var/run/docker.sock:/var/run/docker.sock gives the container control of the host’s Docker daemon, which is effectively root on the host.

privileged: true or added capabilities. Almost never necessary and always worth an explanation.

A default value appearing for a credential. The failure mode this lesson opens with.

An image reference changing from a digest to a tag. A quiet loss of the property that made deployments predictable.

Environment variables added inline rather than through ${...}. A value written directly in the committed file is a value in the repository, and the reason it was convenient is usually that it was a real one.

A network set to host mode. Removes network isolation between the container and the host.

That list takes thirty seconds to check and covers most of what goes wrong in a Compose file.

Multiple projects on one machine

A practical problem that produces confusing failures.

Compose derives a project name from the directory, and that name namespaces containers, networks and volumes. Two repositories both containing a db service coexist fine — until one of them is checked out twice, or two directories share a name.

Set the project name explicitly when it matters:

name: example-api

Or per invocation with -p. Either way, an explicit name means docker compose down -v in one project cannot remove another’s volumes, which is a mistake with real consequences when the volume held a database somebody was using.

Volumes are the part that surprises people. docker compose down leaves named volumes in place; down -v removes them. In development that is usually what you want when resetting state, and it is worth knowing that the data is gone rather than stopped.

Port collisions between projects are the other common symptom. Two projects both publishing 5432:5432 cannot run simultaneously. Either vary the host port per project or, better, stop publishing database ports at all and reach them from other containers by service name.

Production Compose

Sometimes correct — a single host, an internal tool, a small deployment where an orchestrator is disproportionate. If you do:

Digests, not tags.

Secrets from a secret manager, injected at runtime. Compose supports a secrets mechanism backed by files, and those files come from somewhere outside the repository.

Restart policies and health checks on everything.

Resource limits, or one runaway container takes the host.

A deliberate update procedure. docker compose pull && docker compose up -d recreates changed containers, which means a brief interruption per service. There is no rolling update, no health-gated rollout and no automatic rollback: if the new container fails to start, you have a stopped service and a manual recovery. Write down what that recovery is before you need it, because the answer — re-point at the previous digest and bring it back up — is obvious in calm conditions and less so at 3am.

Know what you do not have: no rescheduling if the host dies, no rollback beyond re-pointing at the previous digest, no horizontal scaling.

The signals you have outgrown it: more than one host, a need for zero-downtime deploys, autoscaling, or a Compose file with three environment-specific override files and a shell script wrapping it. At that point the answer is Kubernetes and the GitOps model rather than more Compose.

Keeping it working for new joiners

The main justification for a Compose file is that somebody can clone the repository and be running in ten minutes. That property decays unless somebody checks.

The decay is invisible to the team. Everybody already has a working .env from six months ago, containing three variables that are no longer in .env.example and missing two that were added since. Nobody notices, because nobody starts from scratch.

The check is cheap. Once a quarter, in a clean clone, with no .env: copy the example, follow the README, and see how far you get. Every step that fails is a step a new joiner will hit.

Keep .env.example synchronised. The most common failure is a variable added to compose.yaml with ${VAR:?...} and never added to the example. The person who added it had it locally; nobody else does. A CI check comparing the variables referenced in Compose files against those documented in .env.example catches this mechanically:

grep -ohE '\$\{[A-Z_][A-Z0-9_]*' compose*.yaml   | sed 's/\${//' | sort -u > /tmp/used.txt
grep -ohE '^[A-Z_][A-Z0-9_]*' .env.example | sort -u > /tmp/documented.txt
comm -23 /tmp/used.txt /tmp/documented.txt

Anything that prints is a variable the Compose files need and the example does not mention.

Document the prerequisites. Docker version, available memory, whether anything must be running first. “It does not work” from a new joiner is usually one of these.

One command should do it. docker compose up --wait and nothing else. Every additional manual step is a step somebody will get wrong, and a Makefile target wrapping the sequence is worth the four lines.

Common mistakes

Committing .env. The commonest secret exposure in container repositories.

Default passwords in the committed file. They reach environments that matter.

latest in a Compose file. No semantics, changes silently.

Tags rather than digests for deployed services. Tested and running can differ.

depends_on without a health check condition. Waits for the container, not for readiness.

Testing the dev target in CI. Tests an image you do not ship.

Not cleaning volumes between CI runs. State leaks between tests.

A separate Dockerfile.dev. Diverges from production; use a build target.

Growing Compose into an orchestrator. Override files and wrapper scripts are the symptom.

Compose and the rest of the pillar

Where it sits relative to everything else here, because the boundaries are not obvious.

It consumes what the container workflow produces. The image: field references an artifact CI built and pushed. Compose does not replace that pipeline; it points at its output.

It can also build, via build:, which is right for development and wrong for anything deployed. A production Compose file that builds on the host has no reviewable provenance and no attestation — the same objection as pushing images from a laptop.

It overlaps with Kubernetes conceptually and not operationally. Both describe a set of services declaratively. Only one reconciles continuously, reschedules on failure, or does anything when the host dies. A Compose file is a description that something runs once; a Kubernetes manifest is a description something converges toward forever.

It is not GitOps, however it is stored. Running docker compose up from a pipeline after a merge is a push deployment. There is no agent pulling desired state and no continuous reconciliation, which are two of the four principles. This matters because the term gets applied loosely, and losing the distinction makes it harder to reason about what a reconciler would add.

The migration path, when it comes, is not mechanical. Tools that convert Compose files to Kubernetes manifests exist and produce a starting point rather than a result. The concepts that have no Compose equivalent — probes, resource requests, rollout strategy, ingress — are exactly the ones you adopted Kubernetes for, and they need deciding rather than converting.

Mental model

compose.yaml describes an environment. .env fills it in. The description is committed; the values are not.

Every rule follows: the example file documents the shape, the ignore rule is default-deny, required variables fail fast with a message, and defaults exist only for values that are safe to publish.

What you learned

• compose.yaml is the current canonical name; compose.override.yaml loads automatically
• Commit the base, the override and .env.example; never commit .env
• ${VAR:?message} fails fast on required values; ${VAR:-default} supplies safe defaults
• Never give a password a default in a committed file
• depends_on needs condition: service_healthy to wait for readiness
• Use !reset [] to clear inherited lists in a CI override
• Development can use readable tags; anything deployed uses digests
• Compose is not an orchestrator, and override sprawl is the signal you have outgrown it

Exercise

Use a disposable directory with Docker and Compose. No real credentials — placeholders only.

1. Write compose.yaml with an application service and Postgres, using ${POSTGRES_PASSWORD:?...} with no default.

2. Run docker compose config without a .env. Predict: does it fail, and is the message useful?

3. Create .env with a placeholder password. Run docker compose config again.

4. Add the .gitignore block. Run git status. Confirm .env is ignored and .env.example is not.

5. Add a health check to the database and condition: service_healthy to depends_on. Run docker compose up --wait. Predict: does it wait?

6. Remove the health check condition and start again with the application logging its first database query. Predict: does it connect on the first attempt?

7. Add a compose.ci.yaml with ports: !reset []. Run docker compose -f compose.yaml -f compose.ci.yaml config and confirm the ports are gone.

8. Delete the directory and run docker compose down -v.

Related lessons

Version Controlling DockerfilesThe images Compose runs.

Container Image Tagging with GitWhat to put in the image field.

Preventing .env File CommitsThe specific failure this lesson guards against.

Kubernetes + GitWhere to go when Compose is not enough.