< cd ../gallery
[AI & ML]
ray_vllm_inference
A simple service that integrates vLLM with Ray Serve for fast and scalable LLM serving.
@asprenger
maintainer
★ 79 stars
# README
A simple service that integrates vLLM with Ray Serve for fast and scalable LLM serving. Maintained by asprenger on GitHub, where it has earned 79 stars from the community.
It's actively developed around inference, llm, llm-serving, and is a solid reference for anyone building with these tools.
# tags
# install
npm install ray_vllm_inferencemore vLLM repos
