< cd ../gallery
[AI & ML]
KVarN by huawei-csl
KVarN is a native vLLM KV-cache quantization backend for your agents: 3-5x more context, throughput above FP16, and FP16-level accuracy. Calibration-free, one flag. Maintained by huawei-csl on GitHub, where it has earned 452 stars from the community.
# install
pip install kvarnmore vLLM repos
AI & ML
This project aims to provide a high effective KV cache manage framework for llm inference and improve memory utilization and inference speed. Maintained by TheToughCrane on GitHub, where it has earned 71 stars from the community.
5mo ago
AI & ML
Python package for LLM compression. Maintained by FujitsuResearch on GitHub, where it has earned 398 stars from the community.
2mo ago
AI & ML
LLM notes, including model inference, transformer model structure, and llm framework code analysis notes. Maintained by harleyszhang on GitHub, where it has earned 887 stars from the community.
2mo ago
