< cd ../gallery
[AI & ML]
dynamic-batching by KevinLee1110
The official repo for the paper "Optimizing LLM Inference Throughput via Memory-aware and SLA-constrained Dynamic Batching". Maintained by KevinLee1110 on GitHub, where it has earned 18 stars from the community.
# install
npm install dynamic-batching# tags
more vLLM repos
AI & ML
Official implementation for Text Generation Beyond Discrete Token Sampling. Maintained by EvanZhuang on GitHub, where it has earned 26 stars from the community.
1y ago
AI & ML
A Scheduler for Batched LLM Inference. Maintained by iPieter on GitHub, where it has earned 19 stars from the community.
1y ago
AI & ML
Diagnose vLLM inference servers. Maintained by vllm-doctor on GitHub, where it has earned 16 stars from the community.
2mo ago
