dynamo

A Datacenter Scale Distributed Inference Serving Framework. Maintained by ai-dynamo on GitHub, where it has earned 7,479 stars from the community.

It's actively developed around diffusion, disaggregated-serving, kubernetes, and is a solid reference for anyone building with these tools.