opik

Debug, evaluate, and monitor your LLM applications, RAG systems, and agentic workflows with comprehensive tracing, automated evaluations, and production-ready dashboards. Maintained by comet-ml on GitHub, where it has earned 21,117 stars from the community.

It's actively developed around open-source, langchain, openai, and is a solid reference for anyone building with these tools.