Skip to content

Releasing agent-audit-kit

This file documents the manual + automated steps for cutting a release. The atomic claims-vs-code chain is enforced by scripts/sync_*.py (run by .github/workflows/sync-rule-count.yml and the local pre-commit hook). Most surfaces are anchor-pinned. One surface is not — see §3.

1. Pre-tag

  • Bump pyproject.toml version.
  • Bump agent_audit_kit/__init__.py __version__ (and re-Read after the bump per the v0.3.11 lesson — the on-disk grep is the authoritative pin).
  • ~~Bump tests/test_phase5.py version assertion.~~ No longer needed (v0.3.88): it derives from pyproject.toml. It used to be a hardcoded literal, so every release hand-edited a test file — and the assertion could only disagree with a bump that had already happened, never catch a missed one.
  • Run python3 scripts/sync_rule_count.py --regenerate.
  • Run python3 scripts/sync_scanner_count.py.
  • Run python3 scripts/sync_repo_metadata.py --write.
  • Re-grep pyproject.toml ^version and __init__.py __version__. Both must read the new version.
  • pytest -q && ruff check . && mypy agent_audit_kit/ && agent-audit-kit rule lint.
  • Update CHANGELOG.md and CHANGELOG.cves.md.

2. Tag

  • git tag vX.Y.Z <merge-commit> after the PR is squash-merged.
  • git push origin vX.Y.Z triggers release.yml.
  • Watch the 5 release jobs: CVE-response gate, Publish to PyPI, Push Docker image to GHCR, Rule bundle + SBOM (Sigstore), Create GitHub Release.

3. Post-tag — re-PATCH the GitHub repo description

The GitHub repo description field is the highest-leverage marketing surface (procurement reviewers read it first) and is not wired into the sync scripts. It is a one-time-set field unless we re-set it on every release.

After every tag push, re-PATCH the description so it matches the live RULE_COUNT and the canonical framework count — both read from code, never laundered from the README's own claim:

# Use the renderer. Do NOT retype the sentence: `scripts/render_repo_metadata.py`
# is the single source the `description-liveness` job compares against, and the
# hand-written command that used to live here drifted from it by one character
# (an em-dash where the template has a full stop). Following the documented step
# then produced a description that failed the guard -- observed at v0.4.0.
gh repo edit sattyamjjain/agent-audit-kit \
  --description "$(PYTHONPATH=. python scripts/render_repo_metadata.py)"

# Verify with the same comparison the job runs, rather than by eye:
PYTHONPATH=. python scripts/render_repo_metadata.py --check-live sattyamjjain/agent-audit-kit

This drift was observed at v0.3.15 ship time: the description still read "77 rules, 13 scanners" when the live RULE_COUNT was 193.

Wired at v0.6.1. The commands above are now the fallback, not the procedure. release.yml's final job renders the line and, when a REPO_ADMIN_TOKEN secret is present, sets it with gh repo edit and verifies the result with the same --check-live comparison the liveness job runs. With no such secret the job prints the line to the step summary and the release still succeeds, so a missing secret degrades to the old behaviour rather than breaking a tag.

REPO_ADMIN_TOKEN is a fine-grained PAT with Administration: write on this repository. The default GITHUB_TOKEN cannot set a repo description, which is the whole reason this was manual.

This paragraph previously read "Manual is acceptable until v0.4.0; wire it then." It stayed manual through v0.4.0, v0.5.0 and v0.5.1, and on 2026-09-12 the description was set by hand three times in one day. A deadline a file sets for itself and then passes without comment is the same class of defect as the stale counts the sync scripts exist to prevent, so it is recorded here rather than quietly deleted.

v0.3.16 self-bug: the original len(FRAMEWORKS) form shipped here on 2026-05-09 returned 6 (dict size), not 12 (README claim). Fixed inline above; gh repo edit re-PATCH was redone post-tag with the correct number. Future releases use the README-grep form.

4. Verify

  • PyPI index shows new version as latest. Index can lag the workflow's Publish to PyPI: success by 1–2 minutes; poll until propagated.
  • GitHub Release published, non-draft, non-prerelease.
  • GHCR Docker image pushed.
  • Sigstore SBOM + bundle uploaded.
  • README badges (rule count, framework count, version pin) all atomic.
  • GitHub repo description re-PATCHED per §3.

5. CVE-gate hygiene

Before tagging, disposition every open cve-response issue (the sla-48h label was retired with the 48h SLA in PR #432). The release workflow's CVE-response gate blocks the tag-push pipeline on anything untriaged. Each issue ends in exactly one of:

Disposition Action
rule shipped close (completed), citing the rule and the CHANGELOG.cves.md row
out of scope / unreachable close (not planned) with the one-line reason, label wontfix-static
in scope, rule queued stays open, label cve-deferred, dated disposition comment

A disposition comment carries three fields and nothing else, so the queue can be read by a person and by a script:

disposition:  NEW-RULE | DEFERRED | OUT-OF-SCOPE
target date:  YYYY-MM-DD    (required for DEFERRED, omitted otherwise)
reason:       one sentence

cve-deferred does not block the gate. That exemption was added on 2026-09-01 and it is not a loophole — it is the difference between "has this disclosure been looked at?" and "is the queue empty?". Those were the same question while every triage ended in a close, and they came apart when the watcher's 6-hour cron outran the triage rate: 27 issues open, so count == 0 was a state the repo could not reach on purpose, and v0.3.91 sat declared-but-unpublished for a day. Untriaged issues still block exactly as before.

The rule for using it honestly: label cve-deferred only when the issue has a disposition comment naming what is queued and why. A label without that comment turns the gate off rather than satisfying it.

And it has to say when. That obligation used to be prose, checked by nobody. The 2026-08-31 wave honoured it — all ten of those deferrals carry a **Target: YYYY-MM-DD.** line — which is precisely why it took until 2026-09-04 to notice that nothing enforced it. A deferral with no date is not a deferral; it is a silent drop wearing the one label that switches the gate off. scripts/check_cve_deferrals.py now runs inside the CVE-response gate and refuses the tag when a cve-deferred issue names no target date:. It accepts the older **Target: …** spelling too, because the obligation is say when, not say when in the approved punctuation.

A target date in the past is listed on every run and fails nothing. Making it fatal was the obvious next step and is a trap: it would turn every scheduling note in the tree into a time bomb that detonates on an unrelated release, some morning nobody chose. Visibility at the moment somebody is already looking at the queue is the useful half; holding a tag hostage to a date typed a month ago is not.

The cve-watcher dedup bug (issue #163) re-fired closed CVE IDs across daily cycles. Fixed in v0.3.20 — scripts/cve_watcher.py now queries state=all so a closed issue still suppresses a re-file. This line read "Fix queued for v0.3.17" until 2026-09-12, three patch releases after the fix had actually shipped.