< cd ../gallery
[AI & ML]
smg by smg-project
Engine-agnostic LLM gateway in Rust. Full OpenAI & Anthropic API compatibility across vLLM, TRT-LLM, TokenSpeed, SGLang, OpenAI, Gemini & more. Industry-first gRPC pipeline, KV cache-aware routing, chat history, tokenization caching, Responses API, embeddings, WASM plugins, MCP, and multi-tenant auth. Maintained by smg-project on GitHub, where it has earned 442 stars from the community.
# install
pip install smgmore vLLM repos
AI & ML
Blazing-fast LLM inference in pure Rust. No PyTorch and Python runtime. Maintained by guoqingbao on GitHub, where it has earned 298 stars from the community.
2mo ago
AI & ML
Open source Enterprise AI Gateway for API, MCP and agent, and AI model traffic. One Apache-2.0 binary: 72 native providers behind an OpenAI-compatible API, or serve vLLM and llama.cpp on your own GPUs. Keys, budgets, guardrails, semantic cache, WAF. Maintained by soapbucket on GitHub, where it has earned 49 stars from the community.
2mo ago
AI & ML
High-performance KV cache storage for LLM inference — GPU offloading, SSD caching, and cross-node sharing via RDMA. Works with vLLM and SGLang. Maintained by novitalabs on GitHub, where it has earned 184 stars from the community.
2mo ago
