OpenLLM-Monitor

OpenLLM Monitor is a plug-and-play, real-time observability dashboard for monitoring and debugging LLM API calls across OpenAI, Ollama, OpenRouter, and more. Tracks tokens, latency, cost, retries, and lets you replay prompts — fully open-source and self-hostable. Maintained by prajeesh-chavan on GitHub, where it has earned 42 stars from the community.

It's actively developed around ai, llm, machine-learning, and is a solid reference for anyone building with these tools.