kvcached by ovg-project

LLM notes, including model inference, transformer model structure, and llm framework code analysis notes. Maintained by harleyszhang on GitHub, where it has earned 887 stars from the community.

Evaluate your LLM's response with Prometheus and GPT4. Maintained by prometheus-eval on GitHub, where it has earned 1,103 stars from the community.

Accurate, large-scale, and extensible simulator for LLM inference Systems. Maintained by microsoft on GitHub, where it has earned 654 stars from the community.