Skip to content

AAK-METIS-SCORING-SINK-001: Scoring / judge value flows into prompt-sink call (Metis, research-grade)

Severity
MEDIUM
Category
Tool Poisoning (tool-poisoning)
Scanner
agent_audit_kit/scanners/metis_pomdp.py
Auto-fixable
No.
SARIF name
MetisScoringSink

What it detects

A function consumes a scoring / judge signal (parameter or local variable matches score/scoring/judge/reward/rating/critique) and passes it into a prompt-sink call (format / append / add_message / build_prompt / etc.) without discretizing the signal first. Per Metis (arXiv:2605.10067), numeric or verbose scoring strings are exactly the semantic gradient an adversary uses to refine its policy across closed-loop reasoning iterations. Research-grade — MEDIUM severity reflects non-trivial false-positive risk.

How to fix

Discretize scoring signals into an opaque bucket (e.g., PASS / FAIL / PARTIAL) before re-injecting. If a numeric score must be exposed, cap it (e.g., 0-3 ordinal) and avoid free-text critiques. Per Metis: any verbose feedback string is fuel for the metacognitive policy refinement the paper demonstrates.

To confirm the finding is gone, report only this rule:

agent-audit-kit scan . --rules AAK-METIS-SCORING-SINK-001

Limitations

Follows a score only through a parameter or variable whose name contains score, judge, reward, rating or critique (and plain copies of it), and reports it only when it is passed as a bare argument to a call named format, append, extend, add_message, build_prompt and the like. Putting it into an f-string (f"Your last draft scored {score}"), or into a dict passed to append, is not reported. A Python file over 512 KB is not read.

References


All rules. This page is generated from the rule registry when the documentation is built; to change it, change the rule in agent_audit_kit/rules/builtin.py. To give the rule a hand-written write-up, add docs/rules/AAK-METIS-SCORING-SINK-001.md: the page then shows that text, with the registry reference appended under it.