Ansible CI: Validation That Never Touches Production

Ansible CI has one rule that shapes everything else: it validates, and it does not execute against anything real.

The temptation to cross that line is genuine — “the tests should be realistic” is a reasonable instinct — and crossing it means arbitrary pull request code runs against your infrastructure with your credentials.

Before you start

• Version Controlling Ansible — the validation/execution boundary
• Ansible CI with GitHub Actions — the workflow mechanics

The ladder

Cheapest first, so fast failures fail fast.

Check	Catches	Needs a target?	Cost
YAML parse	Indentation, structure	No	Seconds
--syntax-check	Playbook structure, missing files	No	Seconds
ansible-lint	Deprecations, bad practice, idempotency risks	No	Seconds
Secret scan	Committed credentials	No	Seconds
Inventory validation	Missing groups, undefined variables	No	Seconds
Molecule converge	Does the role work?	Disposable container	Minutes
Molecule idempotence	Does it work twice?	Disposable container	Minutes
Molecule verify	Did it produce the right state?	Disposable container	Minutes

Everything above Molecule needs no target at all. That is most of the value, it runs in under a minute, and it needs no credentials of any kind — which is what makes it safe to run on a pull request from anybody.

Molecule needs a target and creates its own — a container it destroys afterwards. Never a host from your inventory.

The boundary

Danger

CI must never connect to a host in a real inventory. A workflow with an SSH key and a production inventory is a deployment system triggered by whatever anybody pushed — including a pull request from somebody who added a task that runs a command on every host. The check that this has not happened: does any CI workflow have an SSH private key, a vault password, or a reference to a production inventory? If yes, that is the finding.

What CI may have: the repository, a container runtime, and public package access.

What CI must not have: SSH keys to managed hosts, vault passwords for production, cloud credentials that reach infrastructure, or any inventory naming real machines.

The exception, handled deliberately: a vault password for test data, so a Molecule scenario can use encrypted fixtures. Even that is largely avoidable — ansible-lint handles encrypted files without decrypting them, and a test vault containing obvious placeholders, encrypted with a password used nowhere else, is a much smaller exposure than the production one. If a CI secret is genuinely needed, it should be that, and it should be scoped to the one job that needs it.

YAML and syntax

yamllint .

Catches indentation and structural errors, which in YAML are the ones that produce baffling failures elsewhere. Configure it in a committed .yamllint so the rules are reviewable.

ansible-playbook --syntax-check playbooks/site.yml

Parses the playbook, resolves imports and includes, and confirms referenced files exist. It does not connect to anything.

It needs an inventory to parse, which is a trap: pointing it at a production inventory means CI has a file listing production hosts. Point it at a test inventory containing localhost or documentation-range addresses.

Run it for every playbook, not just site.yml. A playbook that nothing imports is a playbook nothing validates, and those are frequently the ad-hoc ones somebody runs during an incident — precisely when a syntax error is least welcome.

Linting

The highest value per second in the whole pipeline.

ansible-lint catches deprecated modules, removed arguments, missing task names, shell where a module exists, missing changed_when on commands, unsafe permissions, and a large set of idempotency risks.

The official GitHub Action:

      - name: Run ansible-lint
        uses: ansible/ansible-lint@main
        with:
          args: ""
          setup_python: "true"
          working_directory: ""
          requirements_file: requirements.yml

Pin it to a commit SHA rather than @main for the supply-chain reasons in Pillar 5. The documentation’s example uses @main; a pinned SHA is the better practice and both work.

requirements_file matters: without the collections installed, lint reports unknown modules for anything from a collection.

Configure in .ansible-lint, committed:

profile: production

exclude_paths:
  - .github/
  - molecule/
  - tests/fixtures/

skip_list:
  - yaml[line-length]

warn_list:
  - experimental

Profiles are the useful feature. min, basic, moderate, safety, shared and production are progressively stricter. Start at a level your repository passes, and move up deliberately.

Baseline rather than fixing everything at once. A mature repository at production profile reports hundreds of findings. Start lower, block new findings, and raise the profile as a separate piece of work.

Skip with a reason. A skip_list entry with a comment explaining why is reviewable; a long unexplained list is a lint configuration nobody trusts.

Validating inventories

Underused, and it catches a class of error that otherwise appears at run time.

ansible-inventory -i inventories/test/hosts.yml --list > /dev/null

Parses the inventory and fails on structural errors.

Check that group_vars files correspond to real groups. A group_vars/webserver.yml when the group is webservers is silently ignored, and the variables it contains simply never apply. This is a common and confusing failure:

for f in inventories/*/group_vars/*.yml; do
  group="$(basename "$f" .yml)"
  [ "$group" = "all" ] && continue
  env_dir="$(dirname "$(dirname "$f")")"
  if ! ansible-inventory -i "$env_dir/hosts.yml" --graph 2>/dev/null | grep -q "@$group:"; then
    echo "group_vars file with no matching group: $f" >&2
  fi
done

Check for undefined variables that playbooks reference and no inventory defines. Harder to automate completely, and a --syntax-check with a strict undefined behaviour catches some.

Molecule in CI

Where roles are actually tested, against a disposable target.

Molecule’s default driver is delegated, and Docker and Podman create/destroy playbooks are bundled with Molecule itself. Other drivers install separately, mostly via the molecule-plugins package.

python -m pip install "molecule" "molecule-plugins[docker]" "ansible-lint"
molecule test --scenario-name default

If you are upgrading from older Molecule, remove separately installed molecule-docker or molecule-podman packages — those were superseded by molecule-plugins, and having both installed produces confusing driver errors.

molecule test runs the full sequence: create, converge, idempotence, verify, destroy. In CI that is what you want; locally, molecule converge and molecule verify iterate faster.

Test in a matrix across platforms you claim to support. A role declaring three distributions and testing one supports one.

Give it enough time. Molecule scenarios are minutes, not seconds, and a job timeout tuned for lint will kill them.

Molecule testing covers scenarios and drivers properly.

Scoping to what changed

At scale, running everything on every pull request is slow enough that people work around it.

Detect changed roles:

git diff --name-only "origin/${BASE_REF}...HEAD" \
  | grep '^roles/local/' \
  | cut -d/ -f1-3 \
  | sort -u

Run Molecule only for those, as a matrix.

Handle shared changes. A change to requirements.yml, ansible.cfg or a widely used template affects everything. Either maintain a dependency map or run everything when those paths change — the second is cruder and much harder to get wrong.

Always run lint on everything. It is seconds, and scoping it saves nothing while risking a missed finding in a file the change did not touch but affected — a role whose variable a changed group_vars file now sets differently, for instance.

Secret scanning

Ansible repositories attract credentials — inventories, group_vars, vault files.

Push protection blocks recognised patterns at push time.

A CI check for unencrypted vault-shaped files:

found=0
for f in $(git ls-files '*vault*.yml' 'group_vars/*/vault*.yml'); do
  if ! head -1 "$f" | grep -q '^\$ANSIBLE_VAULT'; then
    echo "Unencrypted vault file: $f" >&2
    found=1
  fi
done
exit "$found"

Any file whose name says vault and whose first line is not the Ansible Vault header is a plaintext secret with a misleading name.

A check for committed vault passwords:

if git ls-files | grep -qE '(\.vault_pass|vault-password|\.vaultpass)'; then
  echo "Vault password file is tracked." >&2
  exit 1
fi

Both take a second and catch the specific mistakes this cluster warns about.

Making failures actionable

A red check that does not say what to do produces a message in a chat channel rather than a fix.

Lint output is already good — it names the rule, the file, the line and usually the fix. Do not swallow it into a summary; print it.

Name the role or playbook in every failure. In a repository with forty roles, “molecule failed” is not a diagnosis.

Distinguish a real failure from an infrastructure one. A container image that failed to pull and a role that genuinely broke should not look the same. Where a step can fail for environmental reasons, say so in the message.

Surface the first error. Ansible’s output is verbose and the useful part is the first failed task, not the recap at the bottom. A log tail frequently shows only the summary.

Run the cheap checks first and let them fail fast. A developer waiting six minutes for Molecule to discover a YAML indentation error learns to distrust the pipeline.

Keep the whole pull request pipeline under a few minutes for the non-Molecule parts. Above that, people push and go do something else, which removes the feedback loop that makes CI useful.

The standard to aim for: a developer can fix a failed check from the pull request page without opening a log. Most of the work to get there is in the messages rather than the checks.

What CI cannot tell you

Being honest about the limits, because a green pipeline is not proof.

It does not know your hosts. A role passing against a clean container can fail against a host with existing configuration, a different filesystem layout, or a service already running.

It does not test at scale. A playbook working against one container may behave differently against two hundred hosts with serial and handlers.

It does not test your inventory’s correctness. That the syntax parses is not that the groups are right, that the hosts exist, or that a host is in the group somebody intended. Those are discovered when a playbook runs.

Check mode is not a plan. Modules implement --check individually and dependent tasks report inaccurately.

The residual risk is managed by how you run, not by CI: development first, --limit to one host, --check and --diff, and a person watching. Ansible with GitHub Actions covers making execution deliberate.

Turning checks into gates

Validation that does not block is advice.

Required status checks in a ruleset make the checks unavoidable. Without this, everything above is optional and a merge can proceed with a red pipeline.

Distinguish must-pass from advisory. Syntax, lint at your adopted profile, and the vault checks must pass. A newer lint profile you are working toward can be advisory. Making everything blocking produces bypass habits; making nothing blocking produces a pipeline nobody watches.

Required review from a code owner for inventories and vault files, which is where the consequential changes are.

Watch the bypass list. A ruleset with a long bypass list describes an intention.

Check what happens when a job is skipped. Path filters can cause a required check to report as skipped, and whether that leaves the merge blocked depends on your ruleset configuration. Verify the behaviour on your own repository rather than assuming — the safe pattern is running the workflow always and exiting early inside the job.

A complete pipeline

Assembled, and it is shorter than people expect.

1. Checkout, with no credentials beyond the repository.

2. yamllint across the repository.

3. Install collections from requirements.yml into a clean directory.

4. ansible-lint with the requirements file, at your adopted profile.

5. --syntax-check for every playbook, against a test inventory.

6. Inventory validation — parse, and check group_vars correspond to groups.

7. Vault checks — encrypted files are encrypted, no password file tracked.

8. Detect changed roles and build a matrix.

9. Molecule for each changed role, across the platforms it claims to support.

10. Report — every failure naming the file and the fix.

Steps 1 to 7 need no target, no credentials and under a minute. That is the bulk of the value, and a repository running only those is in far better shape than one running nothing while planning to add Molecule someday.

Common mistakes

SSH keys or production inventories in CI. The line this lesson exists to draw.

Running a real playbook as validation. Arbitrary code reaching real hosts.

--syntax-check against a production inventory. CI now holds a list of production hosts.

No requirements_file for lint. Unknown-module errors for everything in a collection.

Adopting the production lint profile immediately. Hundreds of findings, then the check is disabled.

Committing an unencrypted file named vault.yml. The name implies protection that is not there.

Testing one platform while claiming three. Consumers find out at run time.

Not removing superseded Molecule driver packages. Confusing driver errors after an upgrade.

A job timeout tuned for lint. Molecule scenarios killed midway.

Treating green CI as proof it will work. It proves the role works on a clean container.

Testing collections

A collection repository has a different test story from a playbook repository, and Ansible ships tooling for it.

ansible-test sanity runs a set of checks across the collection — import validation, documentation completeness, Python compatibility, YAML correctness. For a collection containing custom modules this catches a great deal, and it is the check whose absence is most visible when somebody consumes your collection.

ansible-test units runs unit tests for modules and plugins. Applicable where you ship Python code; irrelevant to a collection of roles.

ansible-test integration runs integration targets against a container or a remote host. Powerful, and the same rule applies: a container, never anything from a real inventory.

Molecule still covers the roles inside a collection, with a scenario per role.

Test against the Ansible versions you claim to support. meta/runtime.yml declares a minimum requires_ansible, and a matrix across that range is what makes the claim true rather than aspirational.

The documentation checks matter more than they look. A collection whose module documentation does not match its arguments is one where consumers cannot use the built-in help, and sanity tests catch exactly that.

Ansible version compatibility

Worth checking in CI, because it degrades quietly.

Modules are removed on a schedule. Ansible announces deprecations several releases ahead and then removes them. A playbook using a deprecated module works until the control node is upgraded and then fails entirely.

ansible-lint reports deprecations, which is the cheapest early warning and the main reason to run it at a reasonably strict profile.

Test against the version you run and the next one. A matrix with the current control node version and the next release turns an upgrade surprise into a pull request failure months earlier.

Pin the control node version in CI. Otherwise a new Ansible release changes your validation results with no commit in your repository, and the failure arrives on a Monday morning attached to an unrelated change.

Record which version you run in the repository README. “Which Ansible are we on” is a question that comes up during every upgrade discussion, and it should not require asking somebody.

Adopting CI on an existing repository

A repository with no validation, and a team that wants some. The order that avoids the pipeline being disabled in week two.

1. Start with yamllint and --syntax-check. Both are fast, both catch real errors, and both usually pass on an existing repository — so the pipeline starts green and stays credible.

2. Add ansible-lint at a low profile. basic or moderate. Find the level your repository passes today and start there.

3. Make those required. A check that can be ignored trains people to ignore it.

4. Add the vault checks. Seconds, and they catch the mistakes with the worst consequences.

5. Raise the lint profile one step, as a separate piece of work with its own pull request fixing the findings. Not as a change that breaks everybody’s builds.

6. Add Molecule to one role, the most-used one. Learn the timing and the flakiness before committing to it everywhere.

7. Extend Molecule to the roles that matter, not to all of them. A role that creates one file does not need a container test.

8. Add change detection once the runtime becomes a complaint.

The failure to avoid is turning everything on at the strictest setting in one pull request. It produces hundreds of findings, a red pipeline for a week, and a team that adds a bypass. Every step above is individually green before the next one starts.

The first three steps take an afternoon and deliver most of the value. Teams that treat CI adoption as a large project frequently never start; teams that ship a yamllint job on Tuesday have something.

Mental model

CI proves the automation is well-formed, current, idempotent and does what its tests say on a clean target. It cannot prove it is safe on your hosts, and it must never be given the access that would let it try.

That boundary is the whole design. Everything CI does needs no credentials beyond the repository, which is what makes it safe to run on any pull request from anybody.

What you learned

• CI validates; it never connects to a host in a real inventory
• Everything up to Molecule needs no target and runs in under a minute
• The official ansible/ansible-lint action takes args, setup_python, python_version, working_directory and requirements_file
• Lint profiles run from min to production — adopt progressively rather than all at once
• A group_vars file whose name does not match a group is silently ignored
• Molecule’s default driver is delegated; Docker and Podman playbooks are bundled, other drivers come from molecule-plugins
• A file named vault.yml without the $ANSIBLE_VAULT header is a plaintext secret
• Green CI proves the role works on a clean container, not that it is safe on your hosts

Exercise

Use a disposable repository with containers available. No real hosts, no real credentials.

1. Add a workflow running yamllint, --syntax-check and ansible-lint against a test inventory of documentation-range addresses.

2. Introduce a task using shell with no changed_when. Predict: does lint catch it?

3. Set the lint profile to production. Predict: how many new findings appear?

4. Add a group_vars/webserver.yml when the group is webservers. Run a playbook that uses a variable from it. Predict: does anything fail, or does the variable silently not apply?

5. Add the group_vars check above and confirm it catches the mismatch.

6. Create group_vars/production/vault.yml in plaintext. Run the vault-header check. Predict: does it fail?

7. Add a Molecule scenario and run molecule test. Note the sequence of steps and the total time.

8. Try adding a step that runs a playbook against a production inventory. Before running it, write down what a malicious pull request could do. Then delete it.

9. Delete the repository and the containers.

Related lessons

Ansible Lint with GitHub ActionsThe lint workflow in detail.

Molecule Testing with GitScenarios, drivers and disposable targets.

Ansible + GitHub ActionsExecution, kept separate from validation.

Ansible Vault and GitThe encrypted files CI must not decrypt.