semantic-router by vllm-project

Reliable model swapping for any local OpenAI/Anthropic compatible server - llama.cpp, vllm, etc. Maintained by mostlygeek on GitHub, where it has earned 5,268 stars from the community.

Standardized Distributed Generative and Predictive AI Inference Platform for Scalable, Multi-Framework Deployment on Kubernetes. Maintained by kserve on GitHub, where it has earned 5,773 stars from the community.

Enterprise-grade API gateway that helps you monitor and impose cost or rate limits per API key. Get fine-grained access control and monitoring per user, application, or environment. Supports OpenAI, Azure OpenAI, Anthropic, vLLM, and open-source LLMs. Maintained by bricks-cloud on GitHub, where it has earned 1,221 stars from the community.