xinfer

Blazing-fast LLM inference in pure Rust. No PyTorch and Python runtime. Maintained by guoqingbao on GitHub, where it has earned 286 stars from the community.

It's actively developed around agent, llm, qwen, and is a solid reference for anyone building with these tools.