< cd ../gallery

[AI & ML]

tiny-vllm by jmaczan

Build your own high performance LLM inference engine in C++ and CUDA - a smaller version of vLLM. Maintained by jmaczan on GitHub, where it has earned 994 stars from the community.

maintainer

★ 994 stars

61 forks

3 watchers

# install

# tags

C++Ai Hpc Llm Llm Inference

languages

C++98%

C0%

last commit2mo ago

licenseApache-2.0

more vLLM repos

A High-Performance LLM Inference Engine with vLLM-Style Continuous Batching. Maintained by lumia431 on GitHub, where it has earned 120 stars from the community.

@lumia4319mo ago

Mooncake is the serving platform for Kimi, a leading LLM service provided by Moonshot AI. Maintained by kvcache-ai on GitHub, where it has earned 6,157 stars from the community.

@kvcache-ai2mo ago

a community oriented 1:1, vLLM-alike (Continuous batching, paged KV) engine in C++ with additional features. Maintained by mudler on GitHub, where it has earned 44 stars from the community.