LazyLLM by LazyAGI

A GPU cluster manager for high-performance AI model serving (vLLM, SGLang) and on-demand SSH-accessible GPU instances. Maintained by gpustack on GitHub, where it has earned 5,436 stars from the community.

a simple lightweight large language model pipeline framework. Maintained by sherlockchou86 on GitHub, where it has earned 27 stars from the community.

Nano vLLM with vLLM v1's request scheduling strategy and chunked prefill. Maintained by slwang-ustc on GitHub, where it has earned 95 stars from the community.