julius by praetorian-inc

Zero trust LLM gateway. OpenAI-compatible proxy with semantic routing and load balancing across OpenAI, Anthropic, Ollama, vLLM, and any compatible backend. Identity-based access, virtual API keys, and end-to-end encryption via OpenZiti. Maintained by openziti on GitHub, where it has earned 86 stars from the community.

High-performance lightweight proxy and load balancer for LLM infrastructure. Intelligent routing, automatic failover and unified model discovery across local and remote inference backends. Maintained by thushan on GitHub, where it has earned 273 stars from the community.

Easy, advanced inference platform for large language models on Kubernetes. Star to support our work!. Maintained by InftyAI on GitHub, where it has earned 308 stars from the community.