llm-inference
LLM Inference
High-performance inference engines for serving large language models.
Engine Comparison
| Engine | Best For | Hardware | Throughput | Setup |
|---|---|---|---|---|
| vLLM | Production serving | GPU | Highest | Medium |
| llama.cpp | Local/edge, CPU | CPU/GPU | Good | Easy |
| TGI | HuggingFace models | GPU | High | Easy |
| Ollama | Local desktop | CPU/GPU | Good | Easiest |
| TensorRT-LLM | NVIDIA production | NVIDIA GPU | Highest | Complex |
Decision Guide
More from eyadsibai/ltk
document-processing
Use when working with "PDF", "Excel", "Word", "PowerPoint", "XLSX", "DOCX", "PPTX", "spreadsheets", "presentations", "extract text", "merge documents", "convert documents", or asking about "office document manipulation
892file-organization
Use when "organizing files", "cleaning up folders", "finding duplicates", "structuring directories", or asking about "Downloads cleanup", "folder structure", "file management
336literature-review
Use when "literature review", "research synthesis", "systematic review", "academic search", or asking about "find papers", "cite sources", "research gaps", "meta-analysis", "bibliography
226resume-generator
Use when "tailoring resume", "job application", "CV customization", "ATS optimization", or asking about "resume writing", "career transition", "job description matching
138content-writing
Use when "writing articles", "blog posts", "content creation", "research writing", "technical writing", or asking about "outlining", "citations", "improving hooks", "writing feedback
120agent-browser
Use when automating browser interactions via CLI, filling forms, taking screenshots, scraping pages, or asking about "agent-browser", "browser automation", "headless browser", "web scraping", "form filling", "Vercel browser
103