braintrust-eval
Pass
Audited by Gen Agent Trust Hub on Aug 20, 2026
Risk Level: SAFECOMMAND_EXECUTIONEXTERNAL_DOWNLOADSDATA_EXFILTRATIONREMOTE_CODE_EXECUTIONCREDENTIALS_UNSAFE
Full Analysis
- [INDIRECT_PROMPT_INJECTION]: The skill ingests data from external sources, specifically Hugging Face datasets and user-provided files, which are used as inputs for model evaluations.
- Ingestion points: Dataset imports from Hugging Face (
datasets-server.huggingface.co) and local files (references/dataset-import.md,references/hf-dataset-search.md). - Capability inventory: Spawning subprocesses via
claude -p, executing shell commands viacurl, and installing packages viapip/npm(references/subprocess-tracing.md,references/bt-setup.md). - Boundary markers: The skill does not explicitly define prompt delimiters for external data, but it uses structured "input/expected" mappings for datasets.
- Sanitization: Includes logic for secret redaction, string truncation, and metadata capping (
references/dataset-import.md). - [COMMAND_EXECUTION]: The skill performs various shell operations including environment setup, API calls via
curl, and executing model evaluation harnesses. - Evidence: Use of
curlfor Braintrust API interactions (references/bt-auth.md,references/experiment-hygiene.md) andbtCLI for log synchronization (references/bt-setup.md). - [EXTERNAL_DOWNLOADS]: The skill installs the Braintrust SDK and CLI tools from official package registries.
- Evidence: Instructions to run
pip install braintrustandnpm install braintrust(references/bt-setup.md). These are official resources from the vendor (Braintrust). - [DATA_EXFILTRATION]: The skill reads organization-scoped API keys from designated
.envfiles and transmits them to Braintrust's official API endpoints. - Evidence: Accessing
~/.braintrust/.envand.envforBRAINTRUST_API_KEY_*variables, which are used to authenticate requests toapi.braintrust.devandgateway.braintrust.dev(references/bt-auth.md). - [REMOTE_CODE_EXECUTION]: The skill involves generating and executing local scripts for data mapping and evaluation scoring.
- Evidence: Writing import scripts (
references/dataset-import.md) and implementing scorer logic (SKILL.mdPhase 4) that are executed within the local environment. - [CREDENTIALS_UNSAFE]: The skill documentation includes examples of API key patterns in configuration files.
- Evidence: References to
BRAINTRUST_API_KEY_<ORG>=sk-...inreferences/bt-auth.md. The skill includes strong instructions NEVER to print these keys or ask for them in chat.
Audit Metadata