braintrust-eval

Pass

Audited by Gen Agent Trust Hub on Aug 20, 2026

Risk Level: SAFECOMMAND_EXECUTIONEXTERNAL_DOWNLOADSDATA_EXFILTRATIONREMOTE_CODE_EXECUTIONCREDENTIALS_UNSAFE
Full Analysis
  • [INDIRECT_PROMPT_INJECTION]: The skill ingests data from external sources, specifically Hugging Face datasets and user-provided files, which are used as inputs for model evaluations.
  • Ingestion points: Dataset imports from Hugging Face (datasets-server.huggingface.co) and local files (references/dataset-import.md, references/hf-dataset-search.md).
  • Capability inventory: Spawning subprocesses via claude -p, executing shell commands via curl, and installing packages via pip/npm (references/subprocess-tracing.md, references/bt-setup.md).
  • Boundary markers: The skill does not explicitly define prompt delimiters for external data, but it uses structured "input/expected" mappings for datasets.
  • Sanitization: Includes logic for secret redaction, string truncation, and metadata capping (references/dataset-import.md).
  • [COMMAND_EXECUTION]: The skill performs various shell operations including environment setup, API calls via curl, and executing model evaluation harnesses.
  • Evidence: Use of curl for Braintrust API interactions (references/bt-auth.md, references/experiment-hygiene.md) and bt CLI for log synchronization (references/bt-setup.md).
  • [EXTERNAL_DOWNLOADS]: The skill installs the Braintrust SDK and CLI tools from official package registries.
  • Evidence: Instructions to run pip install braintrust and npm install braintrust (references/bt-setup.md). These are official resources from the vendor (Braintrust).
  • [DATA_EXFILTRATION]: The skill reads organization-scoped API keys from designated .env files and transmits them to Braintrust's official API endpoints.
  • Evidence: Accessing ~/.braintrust/.env and .env for BRAINTRUST_API_KEY_* variables, which are used to authenticate requests to api.braintrust.dev and gateway.braintrust.dev (references/bt-auth.md).
  • [REMOTE_CODE_EXECUTION]: The skill involves generating and executing local scripts for data mapping and evaluation scoring.
  • Evidence: Writing import scripts (references/dataset-import.md) and implementing scorer logic (SKILL.md Phase 4) that are executed within the local environment.
  • [CREDENTIALS_UNSAFE]: The skill documentation includes examples of API key patterns in configuration files.
  • Evidence: References to BRAINTRUST_API_KEY_<ORG>=sk-... in references/bt-auth.md. The skill includes strong instructions NEVER to print these keys or ask for them in chat.
Audit Metadata
Risk Level
SAFE
Analyzed
Aug 20, 2026, 10:16 PM