ramalama

RamaLama is an open-source developer tool that simplifies the local serving of AI models from any source and facilitates their use for inference in production, all through the familiar language of containers. Maintained by containers on GitHub, where it has earned 2,956 stars from the community.

It's actively developed around ai, containers, cuda, and is a solid reference for anyone building with these tools.