tightwad

Mixed-vendor GPU inference cluster manager with speculative decoding. Maintained by youngharold on GitHub, where it has earned 31 stars from the community.

It's actively developed around cuda, deep-learning, distributed-inference, and is a solid reference for anyone building with these tools.