deepmark

Deepmark AI enables a unique testing environment for language models (LLM) assessment on task-specific metrics and on your own data so your GenAI-powered solution has predictable and reliable performance. Maintained by IngestAI on GitHub, where it has earned 104 stars from the community.

It's actively developed around benchmark, benchmarking, benchmarks, and is a solid reference for anyone building with these tools.