GitHub CLI Scripting: Production-Quality Automation

The previous lesson covered writing scripts that work. This one covers writing scripts you are willing to leave running.

The difference is mostly about failure: what the script does when GitHub is slow, when a repository has been renamed, when a token expires mid-run, and when someone runs it against the wrong organisation.

Before you start

• Automating GitHub with Bash
• gh api

Output formatting, properly

Three mechanisms, and they suit different jobs.

--json selects fields. It is the foundation and it is a contract — field names are stable in a way that column layout is not.

--jq filters and reshapes, using built-in jq.

--template renders Go templates, which suit fixed-format human-readable output.

# Data for another program
gh pr list --json number,title,author --jq '.[] | {number, title, login: .author.login}'

# Tabular for a human
gh pr list --json number,title,author \
  --template '{{range .}}{{printf "#%v" .number | autocolor "green"}}  {{.title}} ({{.author.login}}){{"\n"}}{{end}}'

# Machine-readable records
gh pr list --json number,title --jq '.[] | [.number, .title] | @tsv'

Discover available fields for any command by passing --json with no value — it lists them for your installed version, which is more reliable than documentation.

Configuration through the environment

gh reads several environment variables, and using them makes scripts portable and explicit.

Variable	Purpose
GH_TOKEN	Authentication token, overriding stored credentials
GH_REPO	Default repository, so --repo can be omitted
GH_HOST	Default host, for GitHub Enterprise Server
GH_PAGER	Pager, or empty to disable
NO_COLOR	Disable colour output
GH_FORCE_TTY	Force terminal formatting

A script that sets these at the top is self-documenting and behaves identically regardless of where it runs:

#!/usr/bin/env bash
set -euo pipefail

export GH_REPO="${GH_REPO:-acme/platform}"
export GH_PAGER=""
export NO_COLOR=1

GH_PAGER="" matters more than it looks: without it, a command producing long output can invoke a pager and hang forever in a non-interactive context, which manifests as a CI job that times out with no error.

Never rely on repository inference in a script

gh infers the repository from the current directory’s Git remotes. That is convenient interactively and unpredictable in automation — the same script does different things depending on where it is invoked. Set GH_REPO, or pass --repo explicitly, every time.

Exit codes

gh uses meaningful exit codes, documented under gh help exit-codes. The values worth handling:

Code	Meaning
0	Success
1	Error
2	Cancelled, or invalid usage
4	Authentication required

Distinguishing them lets a script react rather than simply die:

if ! output=$(gh api "repos/$OWNER/$REPO" 2>&1); then
  case $? in
    4) echo "not authenticated — set GH_TOKEN" >&2; exit 78 ;;
    *) echo "request failed: $output" >&2; exit 1 ;;
  esac
fi

Your own script should also exit meaningfully. A script that always exits 0 cannot be used in a pipeline, and one that exits 1 for everything gives a caller nothing to act on.

Retrying transient failures

Network calls fail. A script that gives up on the first timeout will fail regularly for reasons that have nothing to do with your logic.

retry() {
  local max="${RETRY_MAX:-4}" delay=1 attempt=1
  until "$@"; do
    if [ "$attempt" -ge "$max" ]; then
      printf 'failed after %d attempts: %s\n' "$max" "$*" >&2
      return 1
    fi
    printf 'attempt %d failed, retrying in %ds\n' "$attempt" "$delay" >&2
    sleep "$delay"
    delay=$((delay * 2))
    attempt=$((attempt + 1))
  done
}

retry gh api "repos/$OWNER/$REPO" --silent

What it doesRetries a command with exponential backoff, up to a maximum number of attempts.

Why we run itTransient failures — timeouts, 502s, brief rate-limit pauses — succeed on retry. Permanent failures do not, so the attempt cap prevents an infinite loop on a genuine error.

Expected resultSilent success on the first attempt; a message per retry otherwise.

Retry only what is safe to repeat. Retrying a GET is free. Retrying a POST that creates an Issue may create two — which is why the idempotency patterns from the previous lesson matter before retries are added.

Working across many repositories

The common shape of organisation-wide automation:

#!/usr/bin/env bash
set -euo pipefail

ORG="${1:?usage: sweep.sh ORG}"
DRY_RUN="${DRY_RUN:-true}"

mapfile -t repos < <(
  gh repo list "$ORG" --limit 500 --no-archived --source \
    --json nameWithOwner --jq '.[].nameWithOwner'
)

printf 'found %d repositories\n' "${#repos[@]}"

for repo in "${repos[@]}"; do
  if ! settings=$(gh repo view "$repo" --json deleteBranchOnMerge 2>/dev/null); then
    printf 'skipping %s (not accessible)\n' "$repo" >&2
    continue
  fi

  if [ "$(jq -r '.deleteBranchOnMerge' <<<"$settings")" = "true" ]; then
    continue
  fi

  if [ "$DRY_RUN" = "true" ]; then
    printf 'would enable delete-branch-on-merge: %s\n' "$repo"
  else
    gh repo edit "$repo" --delete-branch-on-merge
    printf 'enabled: %s\n' "$repo"
  fi
done

Four properties make this safe to run against a real organisation:

• --source excludes forks, which usually should not be modified.
• A per-repository failure is skipped, not fatal — one inaccessible repository does not abandon the sweep.
• Already-correct repositories are skipped, which keeps it idempotent and reduces API calls.
• Dry-run by default, so the dangerous mode is explicit.

mapfile reads the list into an array before the loop, so the loop body runs in the main shell rather than a subshell — which means counters and accumulated results survive.

Reusable functions

Once you have written three of these, the shared parts are worth extracting.

# gh-lib.sh — source this from scripts
set -euo pipefail

log()  { printf '[%s] %s\n' "$(date -u +%H:%M:%S)" "$*" >&2; }
die()  { log "ERROR: $*"; exit 1; }

require_auth() {
  gh auth status >/dev/null 2>&1 || die "not authenticated; set GH_TOKEN or run gh auth login"
}

require_budget() {
  local need="${1:-500}"
  local remaining
  remaining=$(gh api rate_limit --jq '.resources.core.remaining')
  [ "$remaining" -ge "$need" ] || die "only $remaining API requests remaining, need $need"
}

repo_exists() {
  gh api "repos/$1" --silent >/dev/null 2>&1
}

Then a script becomes mostly its own logic:

#!/usr/bin/env bash
source "$(dirname "$0")/gh-lib.sh"

require_auth
require_budget 1000

repo_exists "$GH_REPO" || die "no such repository: $GH_REPO"
log "starting sweep of $GH_REPO"

Logging to stderr rather than stdout is deliberate: it keeps stdout clean for data, so the script can be piped into another one.

Authentication in CI

- name: Report failing checks
  env:
    GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
    GH_REPO: ${{ github.repository }}
  run: ./scripts/report-failing-checks.sh

Three points. The workflow token is provided automatically and is scoped to the repository. It is short-lived, which is a security property worth preferring over a personal token. And for anything crossing repository boundaries, the workflow token is insufficient — that needs a GitHub App rather than someone’s personal access token.

When to stop using Bash

Bash is excellent up to a point, and past it the signals are clear:

• You are building nested JSON in shell strings
• You need real error types rather than exit codes
• The script has grown past a few hundred lines
• You want tests
• You need concurrency beyond backgrounding a few jobs
• Someone other than you has to maintain it

At that point, use a real language with an HTTP client. That is the GitHub API cluster, and Python + GitHub API builds exactly the small client this leads to.

The boundary is roughly: shell for orchestration, a language for logic. Calling gh a few times and reacting is shell work. Modelling state, retrying selectively and reporting structurally is not.

Structuring a larger script

Past a hundred lines or so, a flat script becomes hard to follow. Functions plus a main at the bottom keeps it navigable:

#!/usr/bin/env bash
set -euo pipefail

readonly SCRIPT_NAME="${0##*/}"
readonly DEFAULT_LIMIT=100

log()  { printf '[%s] %s\n' "$(date -u +%H:%M:%S)" "$*" >&2; }
die()  { log "ERROR: $*"; exit 1; }
warn() { log "WARN: $*"; }

fetch_repos() {
  local org="$1" limit="${2:-$DEFAULT_LIMIT}"
  gh repo list "$org" --limit "$limit" --no-archived --source \
    --json nameWithOwner --jq '.[].nameWithOwner'
}

audit_repo() {
  local repo="$1" settings
  if ! settings=$(gh repo view "$repo" --json deleteBranchOnMerge,hasWikiEnabled 2>/dev/null); then
    warn "cannot read $repo"
    return 0                       # skip, do not abort the sweep
  fi
  jq -r --arg r "$repo" \
    'select(.deleteBranchOnMerge | not) | $r' <<<"$settings"
}

main() {
  local org="${1:?usage: $SCRIPT_NAME ORG}"
  log "auditing $org"

  local -a needs_fix=()
  while read -r repo; do
    local result
    result=$(audit_repo "$repo")
    [ -n "$result" ] && needs_fix+=("$result")
  done < <(fetch_repos "$org")

  log "${#needs_fix[@]} repositories need attention"
  printf '%s\n' "${needs_fix[@]}"
}

main "$@"

Three things make this maintainable. Functions are small and testable in isolation. main "$@" at the bottom means the whole file parses before anything executes — so a syntax error near the end does not happen halfway through a sweep. And a per-repository failure returns rather than aborting, so one inaccessible repository does not lose the run.

Testing shell

Shell is testable, and a sweep that touches production is worth testing.

shellcheck catches a large class of bugs statically — unquoted expansions, unreachable code, misuse of [ versus [[:

shellcheck -x scripts/*.sh

Run it in CI. It is the single highest-value addition to any shell codebase.

bats runs unit tests, with gh stubbed:

#!/usr/bin/env bats

setup() {
  export PATH="$BATS_TEST_DIRNAME/stubs:$PATH"
}

@test "skips repositories that already have the setting" {
  run ./scripts/sweep.sh acme
  [ "$status" -eq 0 ]
  [[ "$output" != *"already-correct-repo"* ]]
}

The stub is a script named gh earlier on PATH that echoes canned JSON. That is enough to test your filtering and control flow without touching the network — and the filtering is where the bugs are.

Structured logging

For anything scheduled, output that a machine can read is worth the small extra effort:

log_json() {
  local level="$1"; shift
  printf '{"ts":"%s","level":"%s","msg":"%s"}\n' \
    "$(date -u +%Y-%m-%dT%H:%M:%SZ)" "$level" "$*" >&2
}

log_json info "starting sweep of $ORG"
log_json warn "cannot read $repo"

A log aggregator can filter and alert on that; it cannot reliably parse prose. Keep it on stderr so stdout stays clean for data.

Handling partial failure

A sweep over two hundred repositories will have some failures. The question is what the script reports afterwards.

declare -i ok=0 failed=0
declare -a failures=()

for repo in "${repos[@]}"; do
  if gh repo edit "$repo" --delete-branch-on-merge 2>/dev/null; then
    ok=$((ok + 1))
  else
    failed=$((failed + 1))
    failures+=("$repo")
  fi
done

log "succeeded: $ok, failed: $failed"
if [ "$failed" -gt 0 ]; then
  printf 'failed: %s\n' "${failures[@]}" >&2
  exit 1
fi

Exiting non-zero when anything failed matters — a scheduled job that always exits 0 will never alert, and a sweep that silently skipped forty repositories looks identical to one that worked.

Listing which failed is what makes the run actionable. “38 of 200 failed” is a mystery; a list is a task.

Configuration files

Once a script has more than a few settings, move them out of the code:

# sweep.conf
ORG=acme
LIMIT=500
SKIP_REPOS="acme/legacy-thing acme/archived-tool"

CONFIG="${CONFIG:-./sweep.conf}"
# shellcheck source=/dev/null
[ -f "$CONFIG" ] && . "$CONFIG"

should_skip() {
  local repo="$1"
  for skip in $SKIP_REPOS; do
    [ "$repo" = "$skip" ] && return 0
  done
  return 1
}

Sourcing a config file executes it, so treat it as code: never source one from an untrusted location, and never accept a path to it from an unvalidated argument. The convenience is real and so is the risk.

Common mistakes

Relying on repository inference. Set GH_REPO.

Forgetting GH_PAGER="". Hangs in CI with no error message.

Ignoring exit codes. Authentication and usage errors look identical to real failures.

Retrying non-idempotent operations. Creates duplicates.

One failure aborting a sweep. Skip and continue.

Logging to stdout. Corrupts data for anything downstream.

Using a personal token in CI. Use the workflow token, or an App.

Staying in Bash too long. JSON construction in shell strings is the clearest signal.

Caching during development

Iterating on a filter re-requests every time, which is slow and spends rate limit on data you already have:

gh api "repos/OWNER/REPO/issues?per_page=100" --cache 10m --jq '.[].title'
gh api "repos/OWNER/REPO/issues?per_page=100" --cache 10m --jq '.[] | select(.comments > 5) | .number'

The second reads from the local cache. --cache accepts 30s, 10m, 1h and applies only to GET requests — caching a write would be actively dangerous, so it does not.

Remove it before the script goes anywhere real. A cached response in production means acting confidently on ten-minute-old state, which is a difficult bug to see because everything appears to work.

Concurrency without corruption

Parallelism helps when a sweep is dominated by request latency. Two things must be handled.

gh repo list ORG --limit 200 --source --json nameWithOwner --jq '.[].nameWithOwner' \
| xargs -P 4 -I {} bash -c '
    repo="$1"
    out=$(mktemp)
    if gh repo view "$repo" --json nameWithOwner,deleteBranchOnMerge > "$out" 2>/dev/null; then
      jq -r "select(.deleteBranchOnMerge | not) | .nameWithOwner" < "$out"
    else
      printf "SKIP %s\n" "$repo" >&2
    fi
    rm -f "$out"
  ' _ {}

Rate limits are shared. Four workers do not get four budgets; they spend one budget four times as fast. If a sequential run uses 80% of your limit, a parallel one exhausts it.

Output must be whole lines. Two workers writing simultaneously can interleave partial lines. A single printf of a complete line is effectively atomic for short output; anything longer should go to per-worker files and be concatenated afterwards.

Four workers is a reasonable default. Beyond about eight the returns diminish and the risk of secondary rate limiting rises sharply.

Making scripts resumable

Any sweep over hundreds of items will eventually be interrupted — a rate limit, a network failure, a laptop closing. Resumability turns that from a restart into a continuation.

STATE="${STATE:-.sweep-state}"
touch "$STATE"

gh repo list "$ORG" --limit 500 --source --json nameWithOwner --jq '.[].nameWithOwner' \
| while read -r repo; do
    if grep -qxF "$repo" "$STATE"; then
      continue                              # already done
    fi

    process_repo "$repo"
    printf '%s\n' "$repo" >> "$STATE"      # record only after success
  done

Recording after the work means an interruption mid-item leaves it unrecorded and it is retried, which is correct provided process_repo is idempotent. Recording before would skip an item that never completed.

Deleting the state file starts over. Keeping it means a re-run costs nothing for work already done — which also makes the script safe to schedule, since a duplicate run does almost nothing.

Where the boundary really is

The signals that a script has outgrown shell, restated as specifics rather than a feeling:

You are building JSON with string concatenation. jq -n --arg postpones this; nested structures do not.

You need to retry selectively. Distinguishing retryable from permanent failures across several call sites is where shell error handling becomes unmanageable.

You want tests. bats works and is limited. If the logic deserves a test suite, it deserves a language with one.

You need concurrency with shared state. Counters, aggregation and coordinated work are painful in shell and ordinary elsewhere.

Someone else maintains it. Shell is write-once for many teams. A hundred lines of Python is more likely to survive contact with a colleague than a hundred lines of jq filters.

It runs unattended and matters. The threshold for “correct under all conditions” is higher than shell comfortably reaches.

None of these mean shell was the wrong choice initially. A script that grew past its shape is a sign it was useful, and rewriting a working two-hundred-line script into Python is usually an afternoon.

Exercise

1. Write gh-lib.sh with log, die, require_auth and require_budget.
2. Write a script sourcing it that reports, for a repository, how many open pull requests are awaiting review.
3. Add the retry wrapper and confirm it recovers from a deliberately wrong hostname on the first attempt.
4. Add DRY_RUN and a run wrapper for any mutating call.
5. Run it with GH_TOKEN unset and confirm it fails with a clear message and a non-zero exit.
6. Run it in a GitHub Actions workflow using the workflow token.

Documenting a script

A script that runs unattended will eventually be read by someone who did not write it, usually while it is broken.

#!/usr/bin/env bash
#
# sweep-settings.sh — apply repository hygiene settings across an organisation.
#
# Usage:
#   DRY_RUN=false ./sweep-settings.sh acme
#
# Environment:
#   DRY_RUN      "true" (default) prints what would change, "false" applies it
#   GH_TOKEN     authentication; falls back to `gh auth` if unset
#   LIMIT        maximum repositories to consider (default 500)
#
# Exit codes:
#   0  success
#   1  one or more repositories failed
#   2  usage error
#   75 insufficient rate-limit budget (retry later)
#
# Requires: gh >= 2.60, jq >= 1.6

Ten lines that answer every question someone will have at three in the morning: how to run it, what controls it, what the exit code means, and what it needs installed.

The exit-code table is the part most often missing and most useful, because it tells whoever is looking at a failed scheduled job whether to retry, fix a credential, or read the log.

Version pinning

Scripts depending on a recent gh feature should say so rather than failing obscurely:

require_gh() {
  local want="$1" have
  have=$(gh --version | head -1 | awk '{print $3}')
  if [ "$(printf '%s\n%s\n' "$want" "$have" | sort -V | head -1)" != "$want" ]; then
    die "gh $want or newer required; found $have"
  fi
}

require_gh 2.60.0

sort -V compares version strings correctly where string comparison does not — 2.9.0 sorts after 2.10.0 alphabetically and before it numerically, which is exactly the case that bites.

The same applies to jq, whose behaviour changed between 1.6 and 1.7 in ways that affect some filters.

What you learned

• --json, --jq and --template cover data, filtering and human output respectively.
• GH_REPO, GH_TOKEN and GH_PAGER="" make scripts portable and non-hanging.
• gh exit codes distinguish authentication failure from other errors; handle them separately.
• Retry with backoff, but only operations that are safe to repeat.
• Sweeps should skip inaccessible repositories, skip already-correct ones, and default to dry-run.
• Log to stderr so stdout stays usable as data.
• Building JSON in shell strings is the signal to move to a real language.

The short version

A script you are willing to leave running differs from one that merely works in five specific ways: it sets GH_REPO rather than inferring the repository, it sets GH_PAGER="" so it cannot hang at a pager, it distinguishes gh’s exit codes, it retries only what is safe to repeat, and it defaults to dry-run.

Beyond that, the signal that shell has been outgrown is consistent — building JSON with string concatenation. At that point a hundred lines of Python is simpler than the shell replacing it, not more complex.

Distributing scripts as an extension

Once a team has several gh scripts, the distribution problem appears: they live in someone’s ~/bin, get copied around, and drift. Packaging them as a gh extension solves it with the tooling already present on everyone’s machine.

gh extension create acme-tools

That scaffolds a repository containing an executable named gh-acme-tools. Anything in it becomes a subcommand of gh:

#!/usr/bin/env bash
set -euo pipefail

case "${1:-help}" in
  stale)   shift; exec "$(dirname "$0")/lib/stale.sh" "$@" ;;
  audit)   shift; exec "$(dirname "$0")/lib/audit.sh" "$@" ;;
  help|*)  cat <<'USAGE'
gh acme-tools <command>

  stale   report stale pull requests
  audit   audit repository settings across the org
USAGE
           ;;
esac

Installed and updated like anything else:

gh extension install acme/gh-acme-tools
gh acme-tools audit
gh extension upgrade acme-tools

Three things this gets you over a shared directory of scripts: versioning, because the extension is a repository with tags; discovery, because gh extension list shows what someone has; and updates, because gh extension upgrade --all is one command rather than a request to re-copy files.

The credential caution from the install lesson applies to your own extensions too — an extension runs with the user’s authenticated session, so an internal one should go through the same review as anything else that acts with people’s access.

Related lessons

GitHub APINext cluster: REST, GraphQL, authentication and Apps.

Python + GitHub APIWhere these patterns go when Bash runs out.

Git AliasesThe same shortening instinct, applied to Git.