pegainfer

Pure Rust + CUDA LLM inference engine — no PyTorch, OpenAI-compatible, serves Qwen3 to Kimi-K2. Maintained by openinfer-project on GitHub, where it has earned 626 stars from the community.

It's actively developed around inference, kimi, llm-serving, and is a solid reference for anyone building with these tools.