< cd ../gallery
Accurate, large-scale, and extensible simulator for LLM inference Systems. Maintained by microsoft on GitHub, where it has earned 654 stars from the community.
# install
pip install vidurmore vLLM repos
AI & ML
Nano vLLM with vLLM v1's request scheduling strategy and chunked prefill. Maintained by slwang-ustc on GitHub, where it has earned 95 stars from the community.
8mo ago
AI & ML
A simple service that integrates vLLM with Ray Serve for fast and scalable LLM serving. Maintained by asprenger on GitHub, where it has earned 79 stars from the community.
2y ago
AI & ML
Benchmark OpenAI-compatible AI endpoints and AI Accelerators in a reproducible structured way. Maintained by flexaihq on GitHub, where it has earned 12 stars from the community.
7mo ago
