Skip to content

AAK-HEALTHCARE-AI-005: Crisis keywords handled without escalation path

Severity
HIGH
Category
Legal Compliance (legal-compliance)
Scanner
agent_audit_kit/scanners/healthcare_ai.py
Auto-fixable
No.
SARIF name
HealthcareAiNoCrisisEscalation

What it detects

A healthcare AI surface mentions suicide / self-harm / crisis but never references 988 / 911 / 112 / 999 / a crisis line. Tennessee HB 1951 (2026) creates criminal liability for encouraging suicide; lacking an escalation path materially worsens the exposure.

How to fix

Add explicit crisis-line escalation instructions in the prompt / system message; test for the most common suicide / self-harm phrases and escalate before generating any other reply.

To confirm the finding is gone, report only this rule:

agent-audit-kit scan . --rules AAK-HEALTHCARE-AI-005

Limitations

Reads .md, .json, .yaml, .yml and .txt files and matches suicid*, self-harm, overdose, harm myself, kill yourself, end my life and similar; want to die, self-injury and hurting themselves are not matched. The escalation check passes on any 988, 911, crisis line, emergency services or call 112 anywhere in the file, an unrelated number such as ORD-911 included.

References

No external references are recorded for this rule.


All rules. This page is generated from the rule registry when the documentation is built; to change it, change the rule in agent_audit_kit/rules/builtin.py. To give the rule a hand-written write-up, add docs/rules/AAK-HEALTHCARE-AI-005.md: the page then shows that text, with the registry reference appended under it.