This is an opensource project allowing you to compare two LLM's head to head with a given prompt, it has a wide range of supported models, from opensource ollama ones to the likes of openai and claude. Maintained by Supahands on GitHub, where it has earned 25 stars from the community.
