Python CI with GitHub Actions: Complete Pipeline

Python CI is a good first pipeline: the toolchain is simple, the caching is handled by the setup action, and version matrices matter because Python’s release cadence means real projects support several at once.

This lesson builds a complete workflow and explains each decision. The finished file is in the repository at examples/github-actions/python-ci/ci.yml, and it is parsed and version-checked by npm run check:workflows — so it is a file that must remain valid, not a snippet.

Before you start

• GitHub Actions Fundamentals — all eleven lessons
• A Python project with a dependency file and a test suite

The project this assumes

my-project/
├── src/
│   └── myapp/
│       └── __init__.py
├── tests/
│   └── test_myapp.py
├── requirements.txt
├── requirements-dev.txt
└── pyproject.toml

Split dependency files matter for CI. requirements.txt holds what the application needs; requirements-dev.txt holds pytest, ruff, mypy and anything else only CI and developers use. Installing both in CI is correct; shipping both is not.

Starting simple

The minimum useful pipeline:

name: Python CI

on:
  pull_request:
    branches: [main]

permissions:
  contents: read

jobs:
  test:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v7

      - uses: actions/setup-python@v7
        with:
          python-version: "3.13"

      - run: pip install -r requirements.txt -r requirements-dev.txt
      - run: pytest

That works and it is slow, because every run downloads every dependency from scratch.

Caching dependencies

actions/setup-python caches pip’s download cache natively:

- uses: actions/setup-python@v7
  with:
    python-version: "3.13"
    cache: pip
    cache-dependency-path: |
      requirements.txt
      requirements-dev.txt

What it doesInstalls Python and restores pip's cache, keyed on the contents of the named dependency files.

Why we run itWithout caching, every run re-downloads every wheel. The cache key derives from the dependency files, so it invalidates exactly when the dependencies change and not otherwise.

Expected resultA 'Cache restored' line on runs after the first, and a noticeably shorter install step.

cache-dependency-path is worth setting explicitly. The default looks for a single requirements.txt, so a project with split files gets a key that ignores the development dependencies — and the cache then does not invalidate when those change.

This caches downloads, not the environment

cache: pip caches pip’s HTTP cache — the wheels it downloaded. pip install still runs and still installs them, which takes a few seconds.

It does not cache the installed site-packages. Caching an installed virtual environment is possible with actions/cache and is more fragile: the cache must key on the Python version and platform as well as the dependencies, and a stale one produces failures that look like dependency bugs. The download cache gives most of the benefit with none of that risk.

Testing several Python versions

Most libraries support several versions, and a matrix tests them all from one definition:

jobs:
  test:
    name: Test (Python ${{ matrix.python-version }})
    runs-on: ubuntu-latest
    strategy:
      fail-fast: false
      matrix:
        python-version: ["3.11", "3.12", "3.13"]
    steps:
      - uses: actions/checkout@v7
      - uses: actions/setup-python@v7
        with:
          python-version: ${{ matrix.python-version }}
          cache: pip

Two details that matter more than they look.

The versions are quoted. 3.10 unquoted is the number 3.1, and the setup action will fail to find it — or worse, silently install something else. This is the single most common Python CI bug and the YAML lesson covers why.

fail-fast: false stops one failing version cancelling the others. The default cancels everything, which is efficient and hides whether a failure is universal or specific to one version — usually the first thing you want to know.

Lint, format and type check

Three checks that catch different things, run before tests because they are fast:

- name: Lint
  run: ruff check --output-format=github .

- name: Check formatting
  run: ruff format --check .

- name: Type check
  run: mypy src/

--output-format=github is the important flag. It makes ruff emit GitHub’s annotation format, so findings appear on the pull request’s Files changed tab next to the offending line rather than in a log someone has to open. Many Python tools have an equivalent, and using it is far better than parsing output yourself.

These belong in the same job as the tests rather than a separate one: they share the same expensive dependency install, and splitting them would pay that cost twice for a check that takes two seconds.

Tests, coverage and artifacts

- name: Test
  run: |
    pytest \
      --junitxml=results-${{ matrix.python-version }}.xml \
      --cov=src \
      --cov-report=xml \
      --cov-report=term-missing

- name: Upload test results
  if: always()
  uses: actions/upload-artifact@v7
  with:
    name: results-python-${{ matrix.python-version }}
    path: |
      results-${{ matrix.python-version }}.xml
      coverage.xml
    retention-days: 14

if: always() on the upload is the line that matters. Test results are most valuable when tests fail, and without it the upload is skipped precisely then — leaving a red run and no report.

The artifact name includes the matrix value, because names must be unique within a run and every variant uploads.

--cov-report=term-missing prints uncovered lines in the log, which is often enough that nobody needs to download the XML at all.

A stable required check

A matrix job reports one status check per variant — Test (Python 3.11), Test (Python 3.12) and so on. Requiring those individually means updating branch protection every time the matrix changes.

The fix is a gate job:

  ci:
    name: Python CI
    needs: test
    if: always()
    runs-on: ubuntu-latest
    steps:
      - name: Verify the matrix succeeded
        run: |
          if [ "${{ needs.test.result }}" != "success" ]; then
            echo "::error::one or more Python versions failed"
            exit 1
          fi
          echo "all Python versions passed"

needs.test.result for a matrix dependency is the aggregate — success only when every variant succeeded. Requiring the single check Python CI is then stable across matrix changes, which is what makes it maintainable. See Required Reviews for how a check becomes binding.

The complete workflow

name: Python CI

on:
  pull_request:
    branches: [main]
    paths-ignore: ['**.md', 'docs/**']
  push:
    branches: [main]
  merge_group:
  workflow_dispatch:

permissions:
  contents: read

concurrency:
  group: python-ci-${{ github.ref }}
  cancel-in-progress: true

defaults:
  run:
    shell: bash

jobs:
  test:
    name: Test (Python ${{ matrix.python-version }})
    runs-on: ubuntu-latest
    timeout-minutes: 15
    strategy:
      fail-fast: false
      matrix:
        python-version: ["3.11", "3.12", "3.13"]

    steps:
      - uses: actions/checkout@v7

      - uses: actions/setup-python@v7
        with:
          python-version: ${{ matrix.python-version }}
          cache: pip
          cache-dependency-path: |
            requirements.txt
            requirements-dev.txt

      - name: Install dependencies
        run: |
          python -m pip install --upgrade pip
          pip install -r requirements.txt -r requirements-dev.txt

      - name: Lint
        run: ruff check --output-format=github .

      - name: Check formatting
        run: ruff format --check .

      - name: Type check
        run: mypy src/

      - name: Test
        run: |
          pytest \
            --junitxml=results-${{ matrix.python-version }}.xml \
            --cov=src --cov-report=xml --cov-report=term-missing

      - name: Upload test results
        if: always()
        uses: actions/upload-artifact@v7
        with:
          name: results-python-${{ matrix.python-version }}
          path: |
            results-${{ matrix.python-version }}.xml
            coverage.xml
          retention-days: 14

  ci:
    name: Python CI
    needs: test
    if: always()
    runs-on: ubuntu-latest
    steps:
      - run: |
          if [ "${{ needs.test.result }}" != "success" ]; then
            echo "::error::one or more Python versions failed"
            exit 1
          fi

Six workflow-level decisions worth noting: paths-ignore so documentation changes do not run the suite, merge_group so it works with a merge queue, concurrency with cancellation because a superseded CI result is worthless, defaults.run.shell for consistency, timeout-minutes so a hang fails fast, and permissions: contents: read because this pipeline needs nothing more.

Installing dependencies the way your project declares them

The install command has to match how the project actually declares its dependencies, and Python has more than one convention in active use.

Project declares	Install in CI	Notes
requirements.txt	pip install -r requirements.txt	Pin exact versions, or CI drifts from production
pyproject.toml (PEP 621)	pip install -e ".[dev]"	The extras group carries the test tooling
Poetry	poetry install --sync	--sync removes packages no longer in the lock file
PDM	pdm install --check	--check fails if the lock file is stale
uv	uv sync --frozen	--frozen refuses to update the lock file

The --sync, --check and --frozen flags are the equivalents of npm ci: they make CI fail when the lock file and the manifest disagree, rather than quietly resolving something new. A pipeline without that check can pass on a dependency set that exists on no developer’s machine and in no deployment.

Plain pip has no lock file of its own, so the equivalent guarantee comes from hashes:

- run: pip install --require-hashes -r requirements.lock

--require-hashes refuses to install anything whose hash is not pinned in the file — including transitive dependencies, which is the point. Generate the file with pip-compile --generate-hashes or uv pip compile --generate-hashes. It is the strongest form of “install exactly this” available without switching tooling, and it turns a compromised package release into a failed install rather than a silent substitution.

uv is worth benchmarking on your own project

uv resolves and installs substantially faster than pip on most projects, and actions/setup-python is not required to use it. If dependency installation dominates your CI time and caching has not fixed it, an install-tooling change is often a larger win than anything else on this page. Measure rather than assume — a project whose slowness is actually the test suite will see no benefit.

Integration tests with service containers

Unit tests that mock the database prove the code is internally consistent. They do not prove a query is valid SQL. Service containers give the job a real database with no external infrastructure:

jobs:
  integration:
    runs-on: ubuntu-latest
    services:
      postgres:
        image: postgres:18
        env:
          POSTGRES_PASSWORD: ci-only-not-a-real-secret
          POSTGRES_DB: app_test
        ports:
          - 5432:5432
        options: >-
          --health-cmd "pg_isready -U postgres"
          --health-interval 10s
          --health-timeout 5s
          --health-retries 5

      redis:
        image: redis:8
        ports:
          - 6379:6379
        options: >-
          --health-cmd "redis-cli ping"
          --health-interval 10s
          --health-timeout 5s
          --health-retries 5

    steps:
      - uses: actions/checkout@v7
      - uses: actions/setup-python@v7
        with:
          python-version: "3.13"
          cache: pip
      - run: pip install -e ".[dev]"
      - name: Run integration tests
        run: pytest tests/integration
        env:
          DATABASE_URL: postgresql://postgres:ci-only-not-a-real-secret@localhost:5432/app_test
          REDIS_URL: redis://localhost:6379/0

The health check options are the part that matters. Without them the container is started and the steps begin immediately, so the first test connects to a Postgres that has not finished initialising and fails with a connection error that looks like a configuration problem. With them, the job waits until pg_isready succeeds.

Two details that trip people up:

The hostname is localhost when the job runs directly on the runner, because ports: publishes the container port onto the runner’s network. If the job itself runs in a container, the services are reachable by their label instead — postgres:5432 — and ports: is unnecessary.

The password is not a secret. It exists for the lifetime of one job, on a database reachable only from that runner, containing generated test data. Putting it in a repository secret adds masking that makes the connection string unreadable in logs, and protects nothing. Write it inline and make it obviously disposable, as above.

Making the test suite finish

Two mechanisms, and they solve different problems.

Parallelism within a job uses the runner’s cores:

- run: pytest -n auto --dist loadgroup

pytest-xdist with -n auto spawns one worker per available core. --dist loadgroup keeps tests marked as belonging to the same group on the same worker, which matters when tests share a fixture that is expensive or not safe to run concurrently.

The constraint is that parallel tests must be independent. A suite that shares a database, writes to fixed file paths, or depends on execution order will fail in ways that look like flakiness. If enabling -n auto produces new failures, those are real bugs in test isolation rather than an argument against parallelism.

Sharding across jobs uses more machines:

strategy:
  fail-fast: false
  matrix:
    shard: [1, 2, 3, 4]

steps:
  - run: pytest --splits 4 --group ${{ matrix.shard }}

Four runners, each running a quarter of the suite. Wall-clock time drops toward a quarter; total runner minutes go up slightly because each shard repeats the setup. Worth it when the suite takes longer than the feedback loop can absorb.

Sharding interacts with coverage: each shard produces a partial report, and you have to combine them before applying a threshold. coverage combine over downloaded artifacts is the usual approach.

Sharding changes your required check names

Each shard is a separate job with the matrix value in its name. If a required status check was test, it is now test (1) through test (4) — and the old name never reports again, which blocks every pull request. Require an aggregating job instead, as described in matrix builds.

Coverage thresholds that people accept

A global coverage floor is easy to configure and mostly generates resentment: it fails pull requests for reasons unrelated to their content, and the usual response is to lower the number until it stops complaining.

- run: pytest --cov=src --cov-report=xml --cov-fail-under=80

The more useful measure is coverage of the lines this pull request changed. It asks the contributor to test what they wrote, which is a request they can act on, and it does not hold them responsible for a module written three years ago.

- name: Check coverage of changed lines
  run: |
    git fetch origin "${GITHUB_BASE_REF}" --depth=1
    diff-cover coverage.xml --compare-branch "origin/${GITHUB_BASE_REF}" --fail-under=90

GITHUB_BASE_REF is set on pull request events and is empty on push, so guard the step with if: github.event_name == 'pull_request'. The shallow fetch is required because the default checkout does not include the base branch.

Determinism

Tests that pass locally and fail in CI are usually depending on something about the machine. The recurring causes, in rough order of frequency:

• Time zone. The runner is UTC; developer machines are not. A test asserting a formatted local date passes in one and fails in the other. Set TZ explicitly in the job rather than fixing the test twice a year.
• Locale. String sorting, number formatting and case rules differ. Set LC_ALL if any test depends on them.
• Dictionary and set ordering. Python dictionaries preserve insertion order; sets do not have a defined order, and PYTHONHASHSEED varies per process. A test that iterates a set and asserts on order will fail eventually.
• Filesystem case sensitivity. macOS is case-insensitive by default and Linux is not, so an import with the wrong case works locally and fails on the runner.
• Available cores. Code that sizes a thread pool from os.cpu_count() behaves differently on a runner than on a workstation.

env:
  TZ: UTC
  PYTHONHASHSEED: "0"
  PYTHONDONTWRITEBYTECODE: "1"

Setting PYTHONHASHSEED makes hash-order-dependent tests deterministic, which is useful for reproducing a failure — but a test that needs it is asserting on something it should not, and the fix is the test rather than the environment variable.

Publishing to PyPI without an API token

If the pipeline releases a package, the last long-lived credential in it is usually the PyPI token. PyPI supports trusted publishing, which is OIDC applied to a package index:

jobs:
  publish:
    needs: test
    runs-on: ubuntu-latest
    environment: pypi
    permissions:
      id-token: write
    steps:
      - uses: actions/download-artifact@v8
        with:
          name: dist
          path: dist/
      - uses: pypa/gh-action-pypi-publish@release/v1

No token is stored. PyPI is configured with the repository, workflow filename and environment it will accept a publish from, and it validates the workflow’s OIDC token against that configuration — the same trust architecture as the cloud providers, with PyPI as the relying party.

That `@release/v1` is a branch, and it contradicts the pinning rule

Pinning actions says never reference an action by branch, and release/v1 is a branch. It is what the PyPA project documents, and it is what most published examples use — which does not make it safe, it makes it widely copied.

For a workflow that publishes packages under your organisation’s name, pin the commit SHA and let Dependabot propose the bumps. This is exactly the class of workflow where change control matters most: the action runs at the moment your artifact is signed and published.

Pair it with an environment: that has required reviewers, so a release is a decision rather than a consequence of a tag being pushed. Note that the environment name is part of what PyPI matches on, so it has to agree with the configuration on both sides.

Running the checks developers already run

Most Python projects have a pre-commit configuration. Reimplementing those same checks as separate CI steps means two definitions that drift, and a contributor who ran the hooks locally can still fail CI on formatting.

- name: Run pre-commit
  run: |
    pip install pre-commit
    pre-commit run --all-files --show-diff-on-failure

--all-files checks the whole tree rather than only what changed, which is what you want in CI — a hook added last month should apply to files nobody has touched since. --show-diff-on-failure prints the change the hook would have made, so a formatting failure comes with the fix attached instead of just a file name.

Cache the hook environments, which are otherwise rebuilt from scratch on every run:

- uses: actions/cache@v6
  with:
    path: ~/.cache/pre-commit
    key: pre-commit-${{ runner.os }}-${{ hashFiles('.pre-commit-config.yaml') }}

The key hashes the config, so adding a hook invalidates the cache and everything else reuses it — the key design rule from caching.

One caution: pre-commit hooks pin their own tool versions in .pre-commit-config.yaml, and those pins are independent of anything in requirements.txt. A project can end up running one ruff version in the hook and a different one in a direct CI step, which produces the confusing situation where the lint step and the pre-commit step disagree. Pick one as the source of truth.

Scanning dependencies

Two different questions, and they need different tools:

Do our dependencies have known vulnerabilities?

- name: Audit dependencies
  run: |
    pip install pip-audit
    pip-audit --strict --requirement requirements.lock

pip-audit checks the installed set against advisory databases. --strict makes it fail on any finding rather than reporting and exiting zero.

Does our own code contain risky patterns? That is a static analysis question, and ruff already covers a useful portion of it through its security rules — so a project already running ruff can enable them rather than adding another tool.

For either, decide the failure policy before turning it on. Failing on every advisory regardless of whether a fix exists produces a pipeline that is red for reasons nobody can act on, and a red pipeline that is normal is not a signal. Failing only on findings with an available fixed version is usually the sustainable rule, with a separate scheduled job that reports everything.

Note the interaction with artifacts: audit reports are useful to keep, and they are also a list of your unpatched vulnerabilities. On a public repository they are downloadable by anyone. Keep the detail in the job log and the summary rather than in a published artifact.

Common mistakes

Unquoted 3.10. Becomes 3.1. Quote every version.

fail-fast: true by default. Hides whether a failure is version-specific.

Uploading results without if: always(). Skipped exactly when needed.

Caching the virtual environment. Fragile; cache pip’s downloads instead.

Requiring individual matrix checks. Breaks whenever the matrix changes; use a gate job.

Not using --output-format=github. Findings end up in a log rather than on the diff.

Installing production and development dependencies with one file. Ships test tooling.

Exercise

1. Add the minimal four-step workflow to a Python project and confirm it runs.
2. Add cache: pip and compare install times between the first and second run.
3. Add a matrix of three versions with fail-fast: false.
4. Deliberately unquote one version as 3.10 and read which Python the log reports.
5. Add coverage and the if: always() artifact upload, then make a test fail and confirm the report still uploads.
6. Add the gate job and require it as a status check.

What you learned

• actions/setup-python caches pip natively; cache-dependency-path matters with split requirements.
• Quote every version — 3.10 unquoted is 3.1.
• fail-fast: false keeps the other matrix variants running.
• Lint and type checks belong in the same job as tests, because they share the install.
• --output-format=github puts findings on the diff instead of in a log.
• if: always() is what makes test reports survive a failure.
• A gate job gives one stable check name across matrix changes.

Related lessons

Node.js CINext: lockfiles, npm ci, and the JavaScript ecosystem's differences.

Matrix BuildsInclude, exclude, dynamic matrices and cost.

CachingKeys, restore keys and invalidation when a setup action is not enough.

.NET CI with GitHub Actions: Complete PipelineBuild a .NET CI pipeline with GitHub Actions