< cd ../gallery
[AI & ML]
CacheRoute by AstraNetLab
CacheRoute is an innovative LLM scheduling scheme dedicated to enabling flexible KV cache reuse across LLM systems, improving task performance and system efficiency. Maintained by AstraNetLab on GitHub, where it has earned 293 stars from the community.
# install
pip install cacheroutemore vLLM repos
AI & ML
An endpoint server for efficiently serving quantized open-source LLMs for code. Maintained by wangcx18 on GitHub, where it has earned 59 stars from the community.
2y ago
AI & ML
This project aims to provide a high effective KV cache manage framework for llm inference and improve memory utilization and inference speed. Maintained by TheToughCrane on GitHub, where it has earned 71 stars from the community.
5mo ago
AI & ML
LLM notes, including model inference, transformer model structure, and llm framework code analysis notes. Maintained by harleyszhang on GitHub, where it has earned 887 stars from the community.
2mo ago
