high-performance-inference

SKILL.md

Optimize LLM inference for production with vLLM 0.14.x, quantization, and speculative decoding.

vLLM 0.14.0 (Jan 2026): PyTorch 2.9.0, CUDA 12.9, AttentionConfig API, Python 3.12+ recommended.

| PagedAttention | Up to 24x throughput via efficient KV cache | | Continuous Batching | Dynamic request batching for max utilization | | CUDA Graphs | Fast model execution with graph capture | | Tensor Parallelism | Scale across multiple GPUs | | Prefix Caching | Reuse KV cache for shared prefixes |

Hochleistungs-LLM-Inferenz mit vLLM, Quantisierung (AWQ, GPTQ, FP8), spekulativer Dekodierung und Edge-Bereitstellung. Zur Optimierung der Inferenzlatenz, des Durchsatzes oder des Speichers. Quelle: yonatangross/orchestkit.

Original anzeigen