model-serving

SKILL.md

Deploy LLM and ML models for production inference with optimized serving engines, streaming response patterns, and orchestration frameworks. Focuses on self-hosted model serving, GPU optimization, and integration with frontend applications.

This skill provides the backend serving layer for the ai-chat skill.

See references/langchain-orchestration.md and examples/langchain-rag-qdrant/ for complete patterns.

LLM and ML model deployment for inference. Use when serving models in production, building AI APIs, or optimizing inference. Covers vLLM (LLM serving), TensorRT-LLM (GPU optimization), Ollama (local), BentoML (ML deployment), Triton (multi-model), LangChain (orchestration), LlamaIndex (RAG), and streaming patterns. Source: ancoleman/ai-design-components.

View raw