< cd ../gallery
[DevTools]
sardeenz
Sardeenz is a proof-of-concept application that allows you to load more than one model on a given GPU. It allows you to add more and more models onto a GPU, until it is fully utilized.
@rh-aiservices-bu
maintainer
★ 60 stars
# README
Sardeenz is a proof-of-concept application that allows you to load more than one model on a given GPU. It allows you to add more and more models onto a GPU, until it is fully utilized. Maintained by rh-aiservices-bu on GitHub, where it has earned 60 stars from the community.
It's actively developed around kvcache, kvcached, vllm, and is a solid reference for anyone building with these tools.
# tags
# install
npm install sardeenzmore vLLM repos
