xinfer by guoqingbao

High-performance KV cache storage for LLM inference — GPU offloading, SSD caching, and cross-node sharing via RDMA. Works with vLLM and SGLang. Maintained by novitalabs on GitHub, where it has earned 184 stars from the community.

Diagnose vLLM inference servers. Maintained by vllm-doctor on GitHub, where it has earned 16 stars from the community.

Rust SDK for building AI agents with local OpenAI-compatible servers (LMStudio, Ollama, llama.cpp, vLLM). Features streaming, tools, hooks, retry logic, and comprehensive examples. Maintained by slb350 on GitHub, where it has earned 24 stars from the community.