databricks-ml-training

Pass

Audited by Gen Agent Trust Hub on Jun 26, 2026

Risk Level: SAFEREMOTE_CODE_EXECUTIONPROMPT_INJECTION
Full Analysis
  • [REMOTE_CODE_EXECUTION]: The reference implementation for custom MLflow models in references/custom-pyfunc.md utilizes pickle.load() to deserialize preprocessor artifacts. While this is a standard practice in many machine learning environments, pickle is an inherently unsafe serialization format that can lead to arbitrary code execution if an artifact file is replaced with a malicious version.
  • [PROMPT_INJECTION]: The GenAI agent template in references/genai-agents.md defines a ResponsesAgent that processes user-supplied chat messages. This architecture presents a surface for indirect prompt injection as the provided implementation lacks robust boundary delimiters or sanitization for external input.
  • Ingestion points: The predict and predict_stream methods in agent.py ingest ResponsesAgentRequest objects containing untrusted user content.
  • Boundary markers: The agent uses a SYSTEM_PROMPT but does not wrap user-provided messages in specific isolation markers (e.g., XML tags or distinct delimiters) to prevent them from influencing agent instructions.
  • Capability inventory: The agent has the capability to query Unity Catalog functions and Vector Search indexes, which could be misused if the agent is successfully injected.
  • Sanitization: The reference code does not include logic for filtering, escaping, or validating the input data before it is passed to the LangGraph orchestration.
Audit Metadata
Risk Level
SAFE
Analyzed
Jun 26, 2026, 01:25 PM
Security Audit — agent-trust-hub — databricks-ml-training