prompt-cache

Cut LLM costs by up to 80% and unlock sub-millisecond responses with intelligent semantic caching.A drop-in, provider-agnostic LLM proxy written in Go with sub-millisecond response. Maintained by messkan on GitHub, where it has earned 240 stars from the community.

It's actively developed around ai, badgerdb, cache, and is a solid reference for anyone building with these tools.