ai-agent-methodology

Installation
SKILL.md

AI Agent Red-Team Methodology

You attack an AI agent through its tools, not through URLs and forms. The LLM is middleware; the tools are the sink; the real prize is what those tools can reach — cloud IAM, customer data, secrets managers, internal services, actions taken on a user's behalf. This skill routes you to the right hunt-agent-* skill AND enforces the verification discipline that separates an impactful, genuine finding from a confident-wrong one. In real engagements a false "the copilot leaked customer PII" is worse than useless — it burns client trust and gets the whole report dismissed. The discipline is not optional.


PART 0 — THE VERIFICATION LAW (applies to every skill, every finding)

Agents generate text; they don't retrieve state. That is why a verification step is mandatory — a fluent answer can be a hallucination that looks identical to a real leak. Before you claim ANYTHING:

  1. Self-description is untrusted. The agent's account of its own tools/limits is generated and frequently false ("I pass your input verbatim," "I don't have access to that"). Verify by effect.
  2. Verify the channel discriminates. Prove known-true vs known-false give different results before trusting an oracle; confirm your OOB listener receives a literal marker before trusting exfil.
  3. Marker-fired ≠ data-fired. A control landing is not your data leaving. Unique, non-guessable marker per hypothesis.
  4. Reproduce ≥3×. Non-determinism means one hit can be luck. Report hit-rate for probabilistic bugs.
  5. OOB-or-it-didn't-happen. Exfil confirmed only when the listener receives your marker + real data.
  6. Refusal ≠ secure; compliance ≠ vuln; enumeration ≠ impact; "it parsed" ≠ RCE. The finding is the state/data/action obtained, not what the model said.
  7. Actioned secret. A leaked credential/secret/record is confirmed only when it authenticates, the record verifiably belongs to another tenant/user, or the OOB callback carries it.
  8. Minimum-necessary extraction. Prove the missing control with 2–3 records; volume is client liability, not stronger evidence. Synthetic markers; don't persist customer data.
Installs
1
GitHub Stars
21
First Seen
3 days ago