< cd ../gallery

[AI & ML]

mvllm by xerrors

Intelligent load balancer for distributed vLLM server clusters vLLM. Maintained by xerrors on GitHub, where it has earned 19 stars from the community.

maintainer

★ 19 stars

3 forks

# install

pip install mvllm

# tags

Python Balancer Inference Llms Vllm

languages

last commit11mo ago

licenseMIT

more vLLM repos

Benchmark OpenAI-compatible AI endpoints and AI Accelerators in a reproducible structured way. Maintained by flexaihq on GitHub, where it has earned 12 stars from the community.

@flexaihq7mo ago

nano-vllmmoeSpeculative Decoding. Maintained by banfeb on GitHub, where it has earned 11 stars from the community.

Official implementation for Text Generation Beyond Discrete Token Sampling. Maintained by EvanZhuang on GitHub, where it has earned 26 stars from the community.

@EvanZhuang1y ago