llmaz

Easy, advanced inference platform for large language models on Kubernetes. Star to support our work!. Maintained by InftyAI on GitHub, where it has earned 308 stars from the community.

It's actively developed around kubernetes, llm, llamacpp, and is a solid reference for anyone building with these tools.