piqc by paralleliq

Why is LLM inference slow — and how do you make it fast? A hands-on, first-principles course: roofline → KV cache → quantization → parallelism → vLLM/SGLang, with GPU labs on open models. Maintained by jiahongsigma on GitHub, where it has earned 20 stars from the community.

TokenPowerBench: Benchmarking the power consumption of LLM inference across engines and deployment configurations. Maintained by chenxuniu on GitHub, where it has earned 15 stars from the community.

A hybrid router that uses Spot GPU instances to reduce costs and Serverless GPUs for making Cold Starts faster. Maintained by Tandemn-Labs on GitHub, where it has earned 27 stars from the community.