The Agent Skills Directory

[PROMPT_INJECTION]: The skill contains strings such as "Ignore all previous instructions" and "SYSTEM: You are now DAN". These are identified as adversarial test cases located within safety evaluation scripts (evals/test_safety.py), JSON test fixtures (evals/fixtures/unit_cases.json), and configuration files (promptfooconfig.yaml). These strings are intended for testing the agent's refusal capabilities and do not represent instructions to override the agent's behavior during normal operation.
[INDIRECT_PROMPT_INJECTION]: The skill defines a surface for processing external data by ingesting JSON fixtures and interpolating user queries, agent responses, and reference answers into a judge prompt (JUDGE_PROMPT) used by an LLM to grade outputs.
Ingestion points: Fixture files located in evals/fixtures/*.json are loaded into memory and processed by the test runner.
Boundary markers: The JUDGE_PROMPT template uses basic string formatting without explicit boundary markers or delimiters for the interpolated content.
Capability inventory: The evaluation scripts utilize the Anthropic SDK to invoke LLMs and uses pytest for execution control.
Sanitization: No explicit sanitization or escaping of the interpolated test data is present in the judge template, as the skill is designed to handle adversarial content for testing purposes.
[EXTERNAL_DOWNLOADS]: The skill refers to standard, well-known libraries and tools including the anthropic SDK, pytest, and promptfoo. The included GitHub Actions workflow utilizes official, trusted actions from the actions/ organization for environment setup and artifact management.
[COMMAND_EXECUTION]: The documentation provides examples of using pytest, npx promptfoo, and make to execute the evaluation suites. These are standard development workflows and do not involve the execution of unauthorized or hidden commands.

agent-evals