LMCache

LMCache: Supercharge Your LLM with the Fastest KV Cache Layer. Maintained by LMCache on GitHub, where it has earned 11,024 stars from the community.

It's actively developed around amd, cuda, inference, and is a solid reference for anyone building with these tools.