inference-cookbook by SuperMarioYL

A 10-week, 30-minutes-a-day roadmap for LLM inference serving and optimization. vLLM, SGLang, quantization, speculative decoding, benchmarking. Maintained by patchy631 on GitHub, where it has earned 296 stars from the community.

Source-code level analysis of LLM RL training infra: async RL, weight sync, FP8, MoE routing | LLM RL. Maintained by zpqiu on GitHub, where it has earned 14 stars from the community.

A tool for benchmarking LLMs on Modal. Maintained by modal-labs on GitHub, where it has earned 62 stars from the community.