LLM-Inference-Bench

LLM-Inference-Bench. Maintained by argonne-lcf on GitHub, where it has earned 63 stars from the community.

It's actively developed around benchmark, deepspeed, inference, and is a solid reference for anyone building with these tools.