nano-vLLM-MS2 by banfeb

Benchmark OpenAI-compatible AI endpoints and AI Accelerators in a reproducible structured way. Maintained by flexaihq on GitHub, where it has earned 12 stars from the community.

Layered prefill changes the scheduling axis from tokens to layers and removes redundant MoE weight reloads while keeping decode stall free. The result is lower TTFT, lower end-to-end latency, and lower energy per token without hurting TBT stability. Maintained by scale-snu on GitHub, where it has earned 20 stars from the community.

Official implementation for Text Generation Beyond Discrete Token Sampling. Maintained by EvanZhuang on GitHub, where it has earned 26 stars from the community.