< cd ../gallery
[AI & ML]
llm-inspector by helasaoudi
The htop for LLM inference see exactly where every GB of VRAM goes and get measured quantization savings. Maintained by helasaoudi on GitHub, where it has earned 68 stars from the community.
@helasaoudi
maintainer
★ 68 stars
# install
pip install llm-inspectormore vLLM repos
AI & ML
Nano vLLM with vLLM v1's request scheduling strategy and chunked prefill. Maintained by slwang-ustc on GitHub, where it has earned 95 stars from the community.
8mo ago
AI & ML
A simple service that integrates vLLM with Ray Serve for fast and scalable LLM serving. Maintained by asprenger on GitHub, where it has earned 79 stars from the community.
2y ago
AI & ML
ElasticMM: Elastic and Efficient MLLM Serving System. Maintained by hpdps-group on GitHub, where it has earned 44 stars from the community.
5mo ago
