vllm-router

vLLM Router. Maintained by llm-semantic-router on GitHub, where it has earned 55 stars from the community.

It's actively developed around huggingface, kubernetes, llama2, and is a solid reference for anyone building with these tools.