< cd ../gallery
[AI & ML]
vllm-windows-build by aivrar
Native Windows vLLM 0.26.0: CPython 3.13, CUDA 12.8, SM 7.5/8.6/8.9/12.0 for RTX 20/30/40/50, OpenAI-compatible serving, Triton/FlashAttention, 10 KV-cache formats, Multi-TurboQuant, and experimental CPU/RAM/NVMe prompt-KV offload. No WSL or Docker. Maintained by aivrar on GitHub, where it has earned 36 stars from the community.
more vLLM repos
AI & ML
Unified KV-cache compression for LLM inference: 12 Python-native methods, Debian-tested isolated add-ons, Godzilla KVarN/TriAttention, exact Godzilla/Gigatoken profiles, CUDA weight sharing, and multi-GPU planning. Maintained by aivrar on GitHub, where it has earned 25 stars from the community.
2mo ago
AI & ML
TurboQuant KV cache compression plugin for vLLM — asymmetric K/V, 8 models validated, consumer GPUs. Maintained by Alberto-Codes on GitHub, where it has earned 48 stars from the community.
6mo ago
AI & ML
Why is LLM inference slow — and how do you make it fast? A hands-on, first-principles course: roofline → KV cache → quantization → parallelism → vLLM/SGLang, with GPU labs on open models. Maintained by jiahongsigma on GitHub, where it has earned 20 stars from the community.
3mo ago
