< cd ../gallery
A SOTA quantization algorithm for high-accuracy low-bit LLM inference, seamlessly optimized for CPU/XPU/CUDA, with multi-datatype support and full compatibility with vLLM, SGLang, and Transformers. Maintained by intel on GitHub, where it has earned 1,552 stars from the community.
# install
pip install auto-roundmore vLLM repos
AI & ML
LLM model quantization (compression) toolkit with HW acceleration support for Nvidia, AMD, Intel GPU and Intel/AMD/Apple CPU via HF, vLLM, and SGLang. Maintained by ModelCloud on GitHub, where it has earned 1,222 stars from the community.
2mo ago
AI & ML
Python package for LLM compression. Maintained by FujitsuResearch on GitHub, where it has earned 398 stars from the community.
2mo ago
AI & ML
KVarN is a native vLLM KV-cache quantization backend for your agents: 3-5x more context, throughput above FP16, and FP16-level accuracy. Calibration-free, one flag. Maintained by huawei-csl on GitHub, where it has earned 452 stars from the community.
3mo ago
