The Agent Skills Directory

[EXTERNAL_DOWNLOADS]: The skill recommends installing well-known Python packages from PyPI (vllm, torch, transformers, locust, autoawq, auto-gptq, flash-attn) and downloading models from the Hugging Face Hub, which are established and trusted sources in the machine learning ecosystem.\n- [REMOTE_CODE_EXECUTION]: Instructions include the use of the --trust-remote-code flag when serving models with vLLM. While this flag allows the execution of code defined within a model repository, its inclusion is documented as a necessary step for loading models with custom architectures and is a standard feature of the underlying software.\n- [COMMAND_EXECUTION]: The skill provides various shell commands for setup and deployment, including administrative tasks such as configuring firewall rules (sudo ufw). These commands are contextually appropriate for the stated goal of deploying and troubleshooting a production-grade inference server.

serving-llms-vllm