llm-serving-auto-benchmark

Warn

Audited by Socket on Aug 28, 2026

8 alerts found:

Anomalyx8
AnomalyLOW
configs/cookbook-llm/nemotron-3-nano-30b-a3b-bf16.yaml

This YAML is not itself malicious code; it is a model-serving/benchmark configuration. The dominant supply-chain security risk is the repeated trust_remote_code: true across multiple serving backends combined with loading a remote model/tokenizer from an external identifier. If model artifacts are not pinned and integrity-verified, this configuration can enable arbitrary code execution during model initialization. No direct exfiltration or credential-stealing behavior is evident in the YAML alone.

Confidence: 66%Severity: 58%
AnomalyLOW
configs/cookbook-llm/llada2-1-mini.yaml

This YAML fragment does not show overt malicious logic (no embedded payloads, secrets, or suspicious network destinations). However, it enables trust_remote_code: true for multiple serving frameworks while loading a remote model repository, which can allow execution of untrusted code during model initialization—a significant supply-chain risk. Treat the model source as untrusted unless integrity/provenance is verified (e.g., pinned revisions, allowlisting, and isolation/sandboxing).

Confidence: 66%Severity: 62%
AnomalyLOW
configs/cookbook-llm/intern-s1.yaml

No direct malware is evident in this YAML configuration fragment (it is declarative and contains no embedded malicious logic). The main supply-chain security risk is the repeated use of trust_remote_code: true across multiple inference frameworks when loading the external internlm/Intern-S1 model/tokenizer, which can permit remote/custom code execution during server initialization depending on framework behavior. Review and pin model/tokenizer revisions and restrict remote code execution in your deployment/orchestration pipeline.

Confidence: 64%Severity: 52%
AnomalyLOW
configs/cookbook-llm/glm-4.6.yaml

No explicit malware or data-theft logic is visible in this YAML fragment (it is configuration-only). The primary security concern is elevated supply-chain execution risk: trust_remote_code: true is enabled for multiple inference frameworks while using a remote model/tokenizer reference that is not pinned to an immutable artifact/version in this snippet. If that upstream source is compromised, arbitrary code could execute during model/tokenizer loading. Otherwise, the remainder of the configuration appears consistent with benchmarking/serving and local result output.

Confidence: 60%Severity: 60%
AnomalyLOW
configs/cookbook-llm/kimi-k2.6.yaml

No explicit malicious payload is present in this configuration snippet. However, it materially increases supply-chain execution risk by enabling trust_remote_code: true while loading a remote model/tokenizer (moonshotai/Kimi-K2.6) across multiple inference backends; if the referenced artifacts are compromised, arbitrary code could execute during model/tokenizer loading. Other aspects (benchmark endpoint, local output_dir) look like standard benchmarking operations with no obvious exfiltration or credential handling in this file.

Confidence: 70%Severity: 60%
AnomalyLOW
configs/cookbook-llm/glm-4.7-flash.yaml

This configuration is not overtly malicious in the YAML itself, but it carries a significant supply-chain security risk because it enables trust_remote_code: true while loading a remote model/tokenizer (zai-org/GLM-4.7-Flash) across multiple inference backends. If the upstream model repository (or dependencies it references) is compromised, attacker-controlled code could execute during server/model initialization. Apart from this remote-code trust setting, other elements (benchmark endpoint, randomized dataset parameters, and output_dir) are consistent with standard load-testing/benchmarking and do not, by themselves, indicate malware.

Confidence: 62%Severity: 62%
AnomalyLOW
configs/cookbook-llm/kimi-linear-48b-a3b-instruct.yaml

This YAML fragment is not malicious by itself (it is configuration only), but it significantly increases supply-chain execution risk by enabling trust_remote_code: true across multiple inference frameworks. If the referenced moonshotai model repository/tokenizer artifacts are compromised or contain unexpected code, that code may be executed during model loading. No direct indicators of data theft, credential exfiltration, or hidden backdoors are present in this snippet.

Confidence: 68%Severity: 60%
AnomalyLOW
configs/cookbook-llm/glm-4.5.yaml

No explicit malicious payloads, credential theft, or exfiltration logic are present in this YAML fragment; it is primarily a declarative LLM serving/benchmark recipe. The dominant security concern is supply-chain risk: trust_remote_code: true is enabled across multiple inference frameworks while referencing an external model/tokenizer repository, which can allow repository-supplied code to execute during model initialization. Additionally, benchmark results are written to a relative output_dir and a network-facing /v1/completions endpoint is configured, increasing operational exposure, but no direct abuse mechanisms are evident in the provided configuration.

Confidence: 62%Severity: 58%
Audit Metadata
Analyzed At
Aug 28, 2026, 03:02 AM
Package URL
pkg:socket/skills-sh/bbuf%2Fai-infra-auto-driven-skills%2Fllm-serving-auto-benchmark%2F@4d4b8c499a5ec8f618b7fc3093f443f46dab5c504ec423df0041fab62e22d4fe
Security Audit — socket — llm-serving-auto-benchmark