WhiteLotus by Anirudh171202

a community oriented 1:1, vLLM-alike (Continuous batching, paged KV) engine in C++ with additional features. Maintained by mudler on GitHub, where it has earned 44 stars from the community.

High-performance KV cache storage for LLM inference — GPU offloading, SSD caching, and cross-node sharing via RDMA. Works with vLLM and SGLang. Maintained by novitalabs on GitHub, where it has earned 184 stars from the community.

Layered prefill changes the scheduling axis from tokens to layers and removes redundant MoE weight reloads while keeping decode stall free. The result is lower TTFT, lower end-to-end latency, and lower energy per token without hurting TBT stability. Maintained by scale-snu on GitHub, where it has earned 20 stars from the community.