< cd ../gallery
[AI & ML]
GPTQModel
LLM model quantization (compression) toolkit with HW acceleration support for Nvidia, AMD, Intel GPU and Intel/AMD/Apple CPU via HF, vLLM, and SGLang.
@ModelCloud
maintainer
★ 1.2k stars
# README
LLM model quantization (compression) toolkit with HW acceleration support for Nvidia, AMD, Intel GPU and Intel/AMD/Apple CPU via HF, vLLM, and SGLang. Maintained by ModelCloud on GitHub, where it has earned 1,205 stars from the community.
It's actively developed around gptq, optimum, peft, and is a solid reference for anyone building with these tools.
# tags
# install
npm install GPTQModelmore FastAPI repos
