AAK-METIS-SCORING-SINK-001: Scoring / judge value flows into prompt-sink call (Metis, research-grade)
- Severity
- MEDIUM
- Category
- Tool Poisoning (
tool-poisoning) - Scanner
agent_audit_kit/scanners/metis_pomdp.py- Auto-fixable
- No.
- SARIF name
MetisScoringSink
What it detects
A function consumes a scoring / judge signal (parameter or local variable matches score/scoring/judge/reward/rating/critique) and passes it into a prompt-sink call (format / append / add_message / build_prompt / etc.) without discretizing the signal first. Per Metis (arXiv:2605.10067), numeric or verbose scoring strings are exactly the semantic gradient an adversary uses to refine its policy across closed-loop reasoning iterations. Research-grade — MEDIUM severity reflects non-trivial false-positive risk.
How to fix
Discretize scoring signals into an opaque bucket (e.g., PASS / FAIL / PARTIAL) before re-injecting. If a numeric score must be exposed, cap it (e.g., 0-3 ordinal) and avoid free-text critiques. Per Metis: any verbose feedback string is fuel for the metacognitive policy refinement the paper demonstrates.
To confirm the finding is gone, report only this rule:
agent-audit-kit scan . --rules AAK-METIS-SCORING-SINK-001
Limitations
Follows a score only through a parameter or variable whose name contains score, judge, reward, rating or critique (and plain copies of it), and reports it only when it is passed as a bare argument to a call named format, append, extend, add_message, build_prompt and the like. Putting it into an f-string (f"Your last draft scored {score}"), or into a dict passed to append, is not reported. A Python file over 512 KB is not read.
References
- OWASP Agentic Top 10: ASI01 (Agent Goal Hijack)
- Incidents and advisories: ARXIV-2605.10067
- CSA AI Controls Matrix (AICM): AIS-07, AIS-12
All rules. This page is generated from the rule registry when the documentation is built; to change it, change the rule in agent_audit_kit/rules/builtin.py. To give the rule a hand-written write-up, add docs/rules/AAK-METIS-SCORING-SINK-001.md: the page then shows that text, with the registry reference appended under it.