< cd ../gallery
[AI & ML]
olla by thushan
High-performance lightweight proxy and load balancer for LLM infrastructure. Intelligent routing, automatic failover and unified model discovery across local and remote inference backends. Maintained by thushan on GitHub, where it has earned 273 stars from the community.
# install
pip install ollamore vLLM repos
AI & ML
Privacy-first LLM proxy and AI gateway - load balancing, multi-provider routing, API key management, usage tracking, rate limiting. Self-hosted. Zero knowledge of your prompts. Maintained by voidmind-io on GitHub, where it has earned 125 stars from the community.
2mo ago
AI & ML
Easy, advanced inference platform for large language models on Kubernetes. Star to support our work!. Maintained by InftyAI on GitHub, where it has earned 308 stars from the community.
11mo ago
AI & ML
Open Model Engine (OME) — Kubernetes operator for LLM serving, GPU scheduling, and model lifecycle management. Works with SGLang, vLLM, TensorRT-LLM, and Triton. Maintained by ome-projects on GitHub, where it has earned 485 stars from the community.
2mo ago
