vllm-ascend by vllm-project

A simple service that integrates vLLM with Ray Serve for fast and scalable LLM serving. Maintained by asprenger on GitHub, where it has earned 79 stars from the community.

High-performance Inference and Deployment Toolkit for LLMs and VLMs based on PaddlePaddle. Maintained by PaddlePaddle on GitHub, where it has earned 3,703 stars from the community.

Accurate, large-scale, and extensible simulator for LLM inference Systems. Maintained by microsoft on GitHub, where it has earned 654 stars from the community.