Mooncake by kvcache-ai

Community maintained hardware plugin for vLLM on Ascend. Maintained by vllm-project on GitHub, where it has earned 2,560 stars from the community.

Build your own high performance LLM inference engine in C++ and CUDA - a smaller version of vLLM. Maintained by jmaczan on GitHub, where it has earned 994 stars from the community.

LLM inference engine built from scratch in C++. No PyTorch, no frameworks. Maintained by Anirudh171202 on GitHub, where it has earned 12 stars from the community.