vllm-doctor by vllm-doctor

A Scheduler for Batched LLM Inference. Maintained by iPieter on GitHub, where it has earned 19 stars from the community.

An endpoint server for efficiently serving quantized open-source LLMs for code. Maintained by wangcx18 on GitHub, where it has earned 59 stars from the community.

High-performance KV cache storage for LLM inference — GPU offloading, SSD caching, and cross-node sharing via RDMA. Works with vLLM and SGLang. Maintained by novitalabs on GitHub, where it has earned 184 stars from the community.