inference

Swap GPT for any LLM by changing a single line of code. Xinference lets you run open-source, speech, and multimodal models on cloud, on-prem, or your laptop — all through one unified, production-ready inference API. Maintained by xorbitsai on GitHub, where it has earned 9,429 stars from the community.

It's actively developed around artificial-intelligence, chatglm, deployment, and is a solid reference for anyone building with these tools.