< cd ../gallery
[AI & ML]
nano-kvllm by TheToughCrane
This project aims to provide a high effective KV cache manage framework for llm inference and improve memory utilization and inference speed. Maintained by TheToughCrane on GitHub, where it has earned 71 stars from the community.
# install
pip install nano-kvllmmore vLLM repos
AI & ML
An endpoint server for efficiently serving quantized open-source LLMs for code. Maintained by wangcx18 on GitHub, where it has earned 59 stars from the community.
2y ago
AI & ML
A Scheduler for Batched LLM Inference. Maintained by iPieter on GitHub, where it has earned 19 stars from the community.
1y ago
AI & ML
LLM notes, including model inference, transformer model structure, and llm framework code analysis notes. Maintained by harleyszhang on GitHub, where it has earned 887 stars from the community.
2mo ago
