< cd ../gallery
[AI & ML]
tiny-vllm
Build your own high performance LLM inference engine in C++ and CUDA - a smaller version of vLLM
@jmaczan
maintainer
★ 919 stars
# README
Build your own high performance LLM inference engine in C++ and CUDA - a smaller version of vLLM. Maintained by jmaczan on GitHub, where it has earned 919 stars from the community.
It's actively developed around ai, attention, batching, and is a solid reference for anyone building with these tools.
# tags
# install
npm install tiny-vllmmore vLLM repos
