< cd ../gallery
[AI & ML]
fastassert by phospho-app
Dockerized LLM inference server with constrained output (JSON mode), built on top of vLLM and outlines. Faster, cheaper and without rate limits. Compare the quality and latency to your current LLM API provider. Maintained by phospho-app on GitHub, where it has earned 27 stars from the community.
# install
pip install fastassertmore vLLM repos
AI & ML
A Scheduler for Batched LLM Inference. Maintained by iPieter on GitHub, where it has earned 19 stars from the community.
1y ago
AI & ML
An endpoint server for efficiently serving quantized open-source LLMs for code. Maintained by wangcx18 on GitHub, where it has earned 59 stars from the community.
2y ago
AI & ML
Batch Deployment for Document Parsing with AWS Batch & Qwen-2.5-VL. Maintained by jeremyarancio on GitHub, where it has earned 48 stars from the community.
1y ago
