airlock kill-switch — HMAC-signed freeze with quorum reset¶
Landed in v0.5.9. Implementation src/agent_airlock/kill_switch/; CLI
src/agent_airlock/cli/kill_switch.py.
Status: operational as of v0.8.86¶
Through v0.8.85 this froze nothing. The HMAC signing, the envelope, the listener state
machine and the M-of-N quorum were all correct and all inert, because
KillSwitchListener was never constructed inside the library — the only references
outside kill_switch/ were the CLI and the CLI dispatcher table. Triggering the switch
halted no tool call anywhere.
Three things changed:
| before v0.8.85 | now | |
|---|---|---|
@Airlock consults the switch |
never | yes, ahead of every other gate |
| Cross-process transport | NATS/Redis/S3 were stubs delegating to an in-process queue | RedisStreamTransport under the [redis] extra |
CLI trigger / reset |
published into a transport discarded at exit, printed OK |
require a real transport, or fail |
It is opt-in and it is two steps. A process must both construct a listener and register it; constructing one alone still changes nothing:
from agent_airlock.kill_switch import HMACBroadcastSigner, KillSwitchListener, registry
from agent_airlock.kill_switch.transports import RedisStreamTransport
registry.install(KillSwitchListener(
signers=(HMACBroadcastSigner(keyid="ops-a", key=KEY),),
transport=RedisStreamTransport.from_url("redis://localhost:6379/0"),
))
After install, a blocked call returns block_reason: "kill_switch" — a distinct reason
from policy_violation, because "an operator halted the fleet" and "this call is
forbidden" need different responses.
What it does¶
An operator signs a trigger broadcast with a shared key. Every process running a
KillSwitchListener that holds a matching key sees state == TRIGGERED on its next
poll() and reports is_frozen(). Returning to service needs a quorum of distinct
keyids — 2-of-3 by default — so one compromised key cannot unilaterally re-enable agents.
The wire envelope is compact JSON with sorted keys, signed over its canonical form:
{"action":"trigger","keyid":"ops-a","reason":"prod incident 412",
"signature":"<hex hmac-sha256>","ts_epoch":1757000000.0,"version":1}
version is BROADCAST_VERSION, bumped only on incompatible payload changes. Signing keys
must be at least 32 bytes; shorter keys raise InvalidBroadcastSignature at construction.
Runnable example¶
The whole feature in one script. Output is from an actual run:
from agent_airlock import Airlock
from agent_airlock.kill_switch import (
HMACBroadcastSigner, InMemoryTransport, KillSwitchBroadcast,
KillSwitchListener, registry,
)
@Airlock()
def deploy(target: str) -> str:
return f"deployed to {target}"
bus = InMemoryTransport() # swap for RedisStreamTransport to cross processes
ops_a = HMACBroadcastSigner(keyid="ops-a", key=b"a" * 32)
ops_b = HMACBroadcastSigner(keyid="ops-b", key=b"b" * 32)
listener = KillSwitchListener(signers=(ops_a, ops_b), transport=bus, poll_interval_seconds=0)
registry.install(listener) # <- without this, @Airlock never asks
print(listener.state.value, "|", deploy(target="prod"))
# armed | deployed to prod
KillSwitchBroadcast(signer=ops_a, transport=bus).trigger(reason="incident 412")
print(deploy(target="prod")["block_reason"])
# kill_switch -- the decorated function is now refused
# One key is not enough to come back: the default quorum is 2-of-3.
KillSwitchBroadcast(signer=ops_a, transport=bus).reset(reason="all clear")
print(deploy(target="prod")["block_reason"], listener.quorum_progress())
# kill_switch (1, 2)
KillSwitchBroadcast(signer=ops_b, transport=bus).reset(reason="all clear")
print(deploy(target="prod"))
# deployed to prod
# A broadcast signed by a key the listener does not hold is dropped, not applied.
intruder = HMACBroadcastSigner(keyid="ops-a", key=b"z" * 32) # right keyid, wrong key
KillSwitchBroadcast(signer=intruder, transport=bus).trigger(reason="forged")
print(deploy(target="prod"))
# deployed to prod
Two things worth reading twice. The forged message claims a keyid the listener trusts and is still rejected, because acceptance is decided by the MAC and not by the claimed identity. And the same key voting to reset five times is still one vote — the quorum counts distinct keyids, so a single compromised key cannot re-enable a fleet.
Commands¶
export AIRLOCK_KILL_SWITCH_KEY="<>= 32 bytes, hex or raw>"
export AIRLOCK_KILL_SWITCH_REDIS_URL="redis://localhost:6379/0" # or --redis-url
airlock kill-switch status --keyid ops-a
airlock kill-switch trigger "prod incident 412" --keyid ops-a
airlock kill-switch reset "all clear" --keyid ops-a --quorum 2-of-3
airlock kill-switch arm ops-a # validates key material, reads fleet state
--key-env and --stream-key override the key variable and the Redis stream on any
subcommand. status exits 1 when the fleet is frozen, so it composes in a shell check.
Without a transport, the publishing commands fail rather than reporting success:
no broadcast transport configured — this command would reach no process.
Pass --redis-url redis://host:6379/0, or set AIRLOCK_KILL_SWITCH_REDIS_URL.
Requires the redis extra: pip install "agent-airlock[redis]"
That refusal is the point. Until v0.8.86 these commands printed
OK: trigger queued (transport=InMemoryTransport) and reached nothing.
What this does not cover¶
- A freeze takes effect within one poll interval, not instantly. The listener polls at
most once per
poll_interval_seconds(5 s by default) so a transport read does not land on every tool call. Set it to0to poll on every call. A process that makes no tool calls never polls, and is therefore never frozen — the switch stops calls, not processes. - Transport failures fail open by default. If a poll raises, the previous state is kept
and the error is logged, because a kill switch that halts every agent when its own
transport hiccups is an outage generator. This is the opposite of the library's usual
deny-by-default posture, so it is a deliberate exception: pass
strict=Truetoregistry.installto freeze on transport errors instead. --quorumdoes not travel in the broadcast. The threshold is listener configuration (reset_quorum_threshold/reset_quorum_total). The CLI now parses and validates the flag —--quorum 9-of-3is rejected rather than echoed — but uses it only to tell you how many more votes are needed.- NATS and S3 transports are still stubs that delegate to an in-process queue, as is
the deprecated
RedisTransportStub. OnlyRedisStreamTransportcrosses a process boundary. Anything else needs the two-methodBroadcastTransportprotocol implemented against your own bus. - There is no replay protection.
ts_epochis signed but never checked for freshness, so a capturedresetenvelope stays valid indefinitely. Two captured votes from two distinct keyids would disarm a fleet. docs/kill-switch.md, cited as the feature spec by the package docstring, does not exist. This page documents the code as it stands.