[AI & ML]

KVarN

KVarN is a native vLLM KV-cache quantization backend for your agents: 3-5x more context, throughput above FP16, and FP16-level accuracy. Calibration-free, one flag.

@huawei-csl

maintainer

★ 437 stars

go to website ↗github

kvarn — preview

# README

KVarN is a native vLLM KV-cache quantization backend for your agents: 3-5x more context, throughput above FP16, and FP16-level accuracy. Calibration-free, one flag. Maintained by huawei-csl on GitHub, where it has earned 437 stars from the community.

It's actively developed around agentic-ai, kv-cache, llm, and is a solid reference for anyone building with these tools.

# tags

Python Agentic Ai Kv Cache Llm Llm Inference

# install

npm install KVarN

languages

Python83.989%

Cuda5.776%

Rust4.799%

C++3.758%

Shell1.011%

CMake0.276%