< cd ../gallery

[AI & ML]

tangram by aiha-lab

A high-throughput LLM serving engine with non-uniform KV cache compression, built on vLLM. Maintained by aiha-lab on GitHub, where it has earned 13 stars from the community.

maintainer

★ 13 stars

# install

pip install tangram

# tags

Python Inference Optimization Kv Cache Compression Llm Inference Llm Serving

languages

last commit8w ago

licenseApache-2.0

more vLLM repos

A Scheduler for Batched LLM Inference. Maintained by iPieter on GitHub, where it has earned 19 stars from the community.

Source code for the paper: Hear Both Sides: Efficient Multi-Agent Debate via Diversity-Aware Message Retention. Maintained by DA2I2-SLM on GitHub, where it has earned 18 stars from the community.

@DA2I2-SLM6mo ago

Interested in running a conspiracy of agents (technical term) on your Local AI Infra? Who isn't!. Maintained by digitalspaceport on GitHub, where it has earned 16 stars from the community.

@digitalspaceport6mo ago