Lab: Build a Python CI Pipeline

Time25 minutes

LevelIntermediate

You needPython 3.11+ locally (or Docker), and a GitHub repository to push to

Scenario

A small Python library has tests that developers run “usually”. A regression shipped last week because one person’s machine had a different pytest version. The team wants CI, and they want it to be the same thing every developer runs — not a second, slightly different process.

Objective

Build a pipeline in three stages — lint, test, artifact — where every command in the workflow is one you have already run locally and seen pass and fail. Then add a version matrix and dependency caching without making the workflow harder to read.

Prerequisites

• What is GitHub Actions
• Python CI with GitHub Actions — the reference this lab builds toward, one stage at a time
• Python 3.11+ with pip, or Docker (the commands below show the Docker form)

Starting state

mkdir -p /tmp/lab-pyci/src/calc /tmp/lab-pyci/tests && cd /tmp/lab-pyci
git init -q -b main .
git config user.email "lab@example.com"
git config user.name "Lab User"

cat > src/calc/__init__.py <<'EOF'
"""A deliberately small module: enough surface for lint, types and tests."""


def add(a: int, b: int) -> int:
    return a + b


def divide(a: float, b: float) -> float:
    if b == 0:
        raise ZeroDivisionError("cannot divide by zero")
    return a / b
EOF

cat > tests/test_calc.py <<'EOF'
import pytest

from calc import add, divide


def test_add() -> None:
    assert add(2, 3) == 5


def test_divide() -> None:
    assert divide(6, 3) == 2


def test_divide_by_zero() -> None:
    with pytest.raises(ZeroDivisionError):
        divide(1, 0)
EOF

cat > pyproject.toml <<'EOF'
[project]
name = "calc"
version = "0.1.0"
requires-python = ">=3.11"

[build-system]
requires = ["setuptools>=68"]
build-backend = "setuptools.build_meta"

[tool.setuptools.packages.find]
where = ["src"]

[tool.pytest.ini_options]
testpaths = ["tests"]

[tool.ruff]
line-length = 100
EOF

cat > requirements-dev.txt <<'EOF'
pytest==8.3.4
ruff==0.8.4
EOF

git add . && git commit -q -m "Python package with tests, no CI yet"

Tasks

1. Run the pipeline locally first. Install, lint, test — the exact commands the workflow will run:

   pip install -r requirements-dev.txt -e .
   ruff check .
   pytest -q

   (Without a local Python: docker run --rm -v "$PWD":/app -w /app python:3.12-slim sh -c 'pip install -q -r requirements-dev.txt -e . && ruff check . && pytest -q'.)

   Both should pass. Note the output — you will compare it against CI.

2. Make lint fail, and see what that looks like. Append a stray import and re-run ruff:

   printf '\n\nimport os\n' >> src/calc/__init__.py
   ruff check .

   Read the two findings, then remove the line.

3. Write the minimal workflow — one job, least privilege, the three commands from step 1:

   mkdir -p .github/workflows
   cat > .github/workflows/ci.yml <<'EOF'
   name: Python CI
   
   on:
     pull_request:
     push:
       branches: [main]
   
   permissions:
     contents: read
   
   jobs:
     test:
       runs-on: ubuntu-latest
       steps:
         - uses: actions/checkout@v7
         - uses: actions/setup-python@v7
           with:
             python-version: "3.12"
         - run: pip install -r requirements-dev.txt -e .
         - run: ruff check .
         - run: pytest -q
   EOF
   git add . && git commit -q -m "Add minimal CI"

4. Add caching and a matrix. Dependencies are reinstalled on every run; and the package claims to support 3.11+, which nothing verifies. Change the setup-python step and the job header:

   jobs:
     test:
       name: Test (Python ${{ matrix.python-version }})
       runs-on: ubuntu-latest
       timeout-minutes: 10
       strategy:
         fail-fast: false
         matrix:
           python-version: ["3.11", "3.12", "3.13"]
       steps:
         - uses: actions/checkout@v7
         - uses: actions/setup-python@v7
           with:
             python-version: ${{ matrix.python-version }}
             cache: pip
             cache-dependency-path: requirements-dev.txt
         - run: pip install -r requirements-dev.txt -e .
         - run: ruff check .
         - run: pytest -q

   Why fail-fast: false?

5. Keep evidence of failures. Add a JUnit report and upload it even when tests fail:

         - run: pytest -q --junitxml=test-results.xml
         - uses: actions/upload-artifact@v7
           if: always()
           with:
             name: test-results-${{ matrix.python-version }}
             path: test-results.xml

6. Push and open a pull request that breaks a test, then one that fixes it. Confirm the matrix runs three jobs, the failure is reported per version, and the artifact is present on the failed run.

Validation

Locally: ruff check . prints All checks passed! and pytest -q prints 3 passed. On GitHub: three matrix jobs, all green on the fixed branch; on the broken branch, the failing job shows the same assertion you saw locally, and its artifact downloads.

Hints

Step 2. Ruff reports the line, the rule code and a caret under the problem. E402 is “import not at top”, F401 is “imported but unused”. CI will show exactly this output.

Step 4. With fail-fast: true (the default), the first failing version cancels the others — so you learn “3.11 failed” but not whether 3.13 also does. false costs a few minutes and gives the full picture.

Step 5. Without if: always(), the upload step is skipped when pytest fails — which is precisely when you want the report.

Solution

Local run, step 1:

All checks passed!
...                                                                      [100%]
3 passed in 0.03s

Step 2, lint failure:

src/calc/__init__.py:14:1: E402 Module level import not at top of file
   |
14 | import os
   | ^^^^^^^^^ E402
   |

src/calc/__init__.py:14:8: F401 `os` imported but unused

The complete workflow after steps 3–5:

name: Python CI

on:
  pull_request:
  push:
    branches: [main]

permissions:
  contents: read

concurrency:
  group: ci-${{ github.ref }}
  cancel-in-progress: true

jobs:
  test:
    name: Test (Python ${{ matrix.python-version }})
    runs-on: ubuntu-latest
    timeout-minutes: 10
    strategy:
      fail-fast: false
      matrix:
        python-version: ["3.11", "3.12", "3.13"]
    steps:
      - uses: actions/checkout@v7

      - uses: actions/setup-python@v7
        with:
          python-version: ${{ matrix.python-version }}
          cache: pip
          cache-dependency-path: requirements-dev.txt

      - run: pip install -r requirements-dev.txt -e .
      - run: ruff check .
      - run: pytest -q --junitxml=test-results.xml

      - uses: actions/upload-artifact@v7
        if: always()
        with:
          name: test-results-${{ matrix.python-version }}
          path: test-results.xml

Explanation

CI runs what you run. The three commands in the workflow are the three you ran in step 1. When CI is red and local is green, the difference is the environment, and pinned tool versions in requirements-dev.txt are how you remove that difference.

permissions: contents: read is the whole permissions story for a test job. Declaring it sets every other scope to none. Nothing here writes to the repository, so nothing needs more.

Caching keys off the dependency file. cache-dependency-path tells setup-python to hash requirements-dev.txt; the cache is reused until that file changes. A cache keyed on something that never changes would serve stale packages forever — the failure mode in lab 04.

A matrix tests a claim. requires-python = ">=3.11" is a promise. The matrix is what makes it true.

Tip

concurrency with cancel-in-progress means pushing a fix cancels the run for the commit you just replaced. On a busy branch this is the single biggest saving in CI minutes.

Troubleshooting

ModuleNotFoundError: calc in CI but not locally. The editable install (-e .) was skipped. It is what puts src/calc on the path.

Cache never hits. cache-dependency-path must name a file that exists at that path in the checkout; a typo silently disables caching.

ruff passes locally, fails in CI. Different ruff version. Pin it in requirements-dev.txt (this lab does) and check pip list in both places.

Clean up

cd /tmp && rm -rf lab-pyci

Related lessons

Python CI with GitHub ActionsThe full reference pipeline, with coverage and type checking.

Least-Privilege PermissionsWhy one line of permissions matters.

CachingKeys, restore-keys and when caching hurts.

Lab: Debug a failing workflowFour failures with misleading symptoms.

Next lab

Build a Node.js CI pipeline — the same discipline with a lockfile, npm ci, and the built-in test runner.