semantic-consistency-auditor
Pass
Audited by Gen Agent Trust Hub on Sep 23, 2026
Risk Level: SAFEEXTERNAL_DOWNLOADSINDIRECT_PROMPT_INJECTIONMETADATA_POISONING
Full Analysis
- [EXTERNAL_DOWNLOADS]: The skill performs runtime downloads of pre-trained AI models from established repositories.
- Evidence:
SKILL.mdandscripts/main.pydocument that the tool downloads BERTScore models (400MB-1GB) and COMET models (500MB-1.5GB) from well-known sources such as Microsoft and Unbabel during initial execution. - Context: These downloads are standard for the machine learning functionality provided by the skill and target recognized industry providers.
- [INDIRECT_PROMPT_INJECTION]: The skill processes AI-generated and expert clinical notes, which are untrusted data inputs.
- Ingestion points:
scripts/main.pyaccepts clinical text through the--ai-generated,--gold-standard, and--input-fileCLI parameters. - Boundary markers: The skill implementation does not utilize explicit delimiters or 'ignore' instructions to prevent the agent from potentially obeying malicious commands hidden within the notes being compared.
- Capability inventory: The skill can write evaluation output to local files and performs network read operations to fetch models.
- Sanitization: No sanitization or filtering logic is present in
scripts/main.pyto handle adversarial text content. - [METADATA_POISONING]: The skill includes a metadata file (
semantic-consistency-auditor_audit_result_v2.json) that contains self-referential safety and security claims. - Evidence: The file contains multiple hardcoded assertions such as
"security": "PASS"and"gate": "PASS"intended to influence the evaluation of the skill. - Context: As per security analysis best practices, these self-attestations are treated as data to be evaluated and do not impact the independent verdict.
Audit Metadata