< cd ../gallery
[AI & ML]
Efficient-LLM-Inference-Serving-Systems
Why is LLM inference slow — and how do you make it fast? A hands-on, first-principles course: roofline → KV cache → quantization → parallelism → vLLM/SGLang, with GPU labs on open models.
@jiahongsigma
maintainer
★ 19 stars
# README
Why is LLM inference slow — and how do you make it fast? A hands-on, first-principles course: roofline → KV cache → quantization → parallelism → vLLM/SGLang, with GPU labs on open models. Maintained by jiahongsigma on GitHub, where it has earned 19 stars from the community.
It's actively developed around cuda, deep-learning, flash-attention, and is a solid reference for anyone building with these tools.
# tags
# install
npm install Efficient-LLM-Inference-Serving-Systemsmore vLLM repos
