VLM-Batch-Deployment by jeremyarancio

Low latency JSON generation using LLMs. Maintained by varunshenoy on GitHub, where it has earned 396 stars from the community.

Official implementation for Text Generation Beyond Discrete Token Sampling. Maintained by EvanZhuang on GitHub, where it has earned 26 stars from the community.

Dockerized LLM inference server with constrained output (JSON mode), built on top of vLLM and outlines. Faster, cheaper and without rate limits. Compare the quality and latency to your current LLM API provider. Maintained by phospho-app on GitHub, where it has earned 27 stars from the community.