check-training-data

Pass

Audited by Gen Agent Trust Hub on Nov 16, 2026

Risk Level: SAFEINDIRECT_PROMPT_INJECTIONDYNAMIC_EXECUTIONEXTERNAL_DOWNLOADS
Full Analysis
  • [INDIRECT_PROMPT_INJECTION]: The skill is designed to ingest and audit untrusted datasets (e.g., web text, chat transcripts) and utilizes LLMs to judge or score this data.
  • Ingestion points: Untrusted data enters the context via pb.Validate(data=...) in SKILL.md.
  • Boundary markers: No explicit delimiters or instructions to ignore embedded commands are mentioned in the prompt() or calibrate_prompt() methods.
  • Capability inventory: The skill can perform network operations (API calls to Anthropic/Cohere), write files (dataset cards, reports), and execute local code (ablation evaluation).
  • Sanitization: There is no evidence of input sanitization or escaping of the processed text before it is sent to the LLM judge.
  • [DYNAMIC_EXECUTION]: The skill allows the definition and execution of arbitrary code for scorers and evaluation functions.
  • Evidence: pb.Ablation and pb.Validate take user-provided Python functions (evaluate, scorer) which are executed during the interrogation and ablation process.
  • Evidence: pb.yaml_ablation() supports an allow_python=True flag that imports and executes code defined in a YAML file's evaluate: block.
  • [EXTERNAL_DOWNLOADS]: The skill relies on and interacts with well-known external services to download models and perform remote inference.
  • Evidence: Utilizes the transformers and sentence-transformers libraries to download and cache models from Hugging Face.
  • Evidence: Loads specific third-party models for language identification (e.g., lid.176.bin for fastText) and perplexity (kenlm).
  • Evidence: Performs remote API calls to services like Cohere (via pb.scorers.reranker) and Anthropic (via .prompt()).
Audit Metadata
Risk Level
SAFE
Analyzed
Nov 16, 2026, 08:14 PM