nemo-evaluator-plugin
Audited by Socket on Jul 22, 2026
3 alerts found:
AnomalySecurityx2No direct indicators of classic malware (e.g., reverse shells, file destruction, or exfiltration to unknown domains) are visible in this configuration fragment. The dominant supply-chain/security concern is the embedding of metric logic as cloudpickle payloads: if the runtime deserializes these blobs, arbitrary code execution is possible during unpickling. Treat the cloudpickle blobs and the consumer runtime as a trusted boundary—verify provenance/integrity and ensure deserialization is safe/locked down. Otherwise, this artifact functions as an LLM evaluation harness that sends prompts to a specified external API using a secret reference.
No explicit malicious network targets beyond the legitimate NVIDIA chat-completions endpoint are visible in the JSON, and there is no direct evidence of file/system modification. However, the artifact embeds an opaque cloudpickle payload intended for runtime deserialization, which is a well-known arbitrary code execution vector if trust boundaries or integrity verification are missing. Combined with authenticated outbound API usage, this creates a high supply-chain/runtime execution risk that should be treated as untrusted until the consumer’s deserialization path and provenance/integrity protections are confirmed.
No explicit malicious behavior is visible in the snippet itself, but the artifact contains an embedded opaque cloudpickle payload that is intended for deserialization/execution by the consumer. If payload.blob is not cryptographically verified and the consumer unpickles it with standard cloudpickle/pickle loaders, it creates a significant supply-chain risk of arbitrary code execution. Treat this package as potentially dangerous unless the deserialized content is strictly trusted and loaded using hardened controls (signature/allowlisting and sandboxing).