human-in-the-loop-training
Pass
Audited by Gen Agent Trust Hub on Sep 14, 2026
Risk Level: SAFEINDIRECT_PROMPT_INJECTION
Full Analysis
- [INDIRECT_PROMPT_INJECTION]: The skill describes a system architecture designed to ingest untrusted human feedback and process it through LLMs to generate new rules or training data.
- Ingestion points: The
FeedbackCollector,HumanReviewQueue, andPatternLearnerclasses inSKILL.mdare designed to handle external human input. - Boundary markers: The provided code snippets lack explicit boundary markers or instructions to the LLM to ignore embedded commands within the human corrections.
- Capability inventory: The design suggests capabilities for triggering model training jobs (
submit_training_job) and generating text based on corrections (llm.generate). - Sanitization: There is no evidence of sanitization or filtering for the human-provided feedback before it is interpolated into prompts in the
derive_ruleandgenerate_with_correctionsmethods. This represents a vulnerability surface where a malicious reviewer could inject instructions that influence model behavior or rule generation.
Audit Metadata