vllm-chatterbox-stream by wuxuedaifu

OpenAI-compatible, vLLM-served OCR API for the Surya-OCR-2 model — multilingual document OCR (layout + text recognition) with request batching, a local CLI, and Docker packaging. Maintained by wuxuedaifu on GitHub, where it has earned 21 stars from the community.

Dual-engine (llama.cpp + vLLM) LLM benchmarking pipeline for GGUF & safetensors on NVIDIA GPUs — speed, quality, live dashboard, publishable cards. Maintained by notwitcheer on GitHub, where it has earned 26 stars from the community.

Why is LLM inference slow — and how do you make it fast? A hands-on, first-principles course: roofline → KV cache → quantization → parallelism → vLLM/SGLang, with GPU labs on open models. Maintained by jiahongsigma on GitHub, where it has earned 20 stars from the community.