< cd ../gallery
[AI & ML]
nano-vllm-v1 by slwang-ustc
Nano vLLM with vLLM v1's request scheduling strategy and chunked prefill. Maintained by slwang-ustc on GitHub, where it has earned 95 stars from the community.
# install
pip install nano-vllm-v1more vLLM repos
AI & ML
A simple service that integrates vLLM with Ray Serve for fast and scalable LLM serving. Maintained by asprenger on GitHub, where it has earned 79 stars from the community.
2y ago
AI & ML
Accurate, large-scale, and extensible simulator for LLM inference Systems. Maintained by microsoft on GitHub, where it has earned 654 stars from the community.
1y ago
AI & ML
The htop for LLM inference see exactly where every GB of VRAM goes and get measured quantization savings. Maintained by helasaoudi on GitHub, where it has earned 68 stars from the community.
8w ago
