FastDeploy

High-performance Inference and Deployment Toolkit for LLMs and VLMs based on PaddlePaddle. Maintained by PaddlePaddle on GitHub, where it has earned 3,702 stars from the community.

It's actively developed around ernie, ernie-45, ernie-45-vl, and is a solid reference for anyone building with these tools.