< cd ../gallery

[AI & ML]

grps by NetEase-Media

go to website ↗github

Deep Learning Deployment Framework: Supports tf/torch/trt/trtllm/vllm and other NN frameworks. Support dynamic batching, and streaming modes. It is dual-language compatible with Python and C++, offering scalability, extensibility, and high performance. It helps users quickly deploy models and provide services through HTTP/RPC interfaces. Maintained by NetEase-Media on GitHub, where it has earned 169 stars from the community.

maintainer

★ 169 stars

11 forks

9 watchers

# install

pip install grps

# tags

C++Tensorflow Tensorrt Torch Vllm

languages

C++55%

last commit1y ago

licenseApache-2.0

more vLLM repos

a community oriented 1:1, vLLM-alike (Continuous batching, paged KV) engine in C++ with additional features. Maintained by mudler on GitHub, where it has earned 44 stars from the community.

LLM inference engine built from scratch in C++. No PyTorch, no frameworks. Maintained by Anirudh171202 on GitHub, where it has earned 12 stars from the community.

@Anirudh1712026mo ago

Community maintained hardware plugin for vLLM on Ascend. Maintained by vllm-project on GitHub, where it has earned 2,560 stars from the community.

@vllm-project2mo ago