The Agent Skills Directory

[Dynamic Code Execution]: The script scripts/calibrate_grader.py utilizes the exec() and compile() functions to load and execute user-provided Python grader files at runtime. This pattern is a security consideration because it allows for the execution of arbitrary Python code from local files. The script includes an explicit developer warning advising users to only load grader files they have reviewed or written themselves.
[Command Execution]: The script scripts/deploy_model.py uses subprocess.run() to invoke the Azure CLI (az) to retrieve authentication tokens for resource deployment. While this is a standard approach for integrating with Azure services, the invocation of external shell commands is a pattern that should be noted for security review.
[Indirect Prompt Injection Surface]: Several scripts, such as scripts/evaluate_model.py and scripts/score_dataset.py, ingest content from user-provided JSONL training or test files and interpolate that content directly into prompts sent to judge models. Without rigorous sanitization or explicit boundary markers, malicious instructions embedded within these data files could potentially influence the behavior of the evaluation or quality scoring models.
Ingestion points: Training and test data is loaded from JSONL files in scripts/evaluate_model.py, scripts/score_dataset.py, and scripts/calibrate_grader.py.
Boundary markers: The prompts in scripts/evaluate_model.py and scripts/score_dataset.py use structural markers like ## Task prompt but do not include explicit instructions to the judge model to ignore instructions embedded within the data.
Capability inventory: The skill possesses capabilities for network API calls to OpenAI/Azure endpoints and local code execution via the exec() function in the grader calibration script.
Sanitization: The scripts do not appear to perform specific filtering or escaping of the user-provided text content before it is interpolated into judge prompts.

finetuning