< cd ../gallery
[AI & ML]
Lvllm
LvLLM is a special NUMA extension of vllm that makes full use of CPU and memory resources, reduces GPU memory requirements, and features an efficient GPU parallel and NUMA parallel architecture, supporting hybrid inference for MOE large models.
@guqiong96
maintainer
★ 401 stars
# README
LvLLM is a special NUMA extension of vllm that makes full use of CPU and memory resources, reduces GPU memory requirements, and features an efficient GPU parallel and NUMA parallel architecture, supporting hybrid inference for MOE large models. Maintained by guqiong96 on GitHub, where it has earned 401 stars from the community.
It's actively developed around hybrid, inference, model, and is a solid reference for anyone building with these tools.
# tags
# install
npm install Lvllmmore vLLM repos
