coloma by colomalabs

A Scheduler for Batched LLM Inference. Maintained by iPieter on GitHub, where it has earned 19 stars from the community.

[ICPP'25] TD-Pipe: Temporally-Disaggregated Pipeline Parallelism Architecture for High-Throughput LLM Inference. Maintained by MLSysU on GitHub, where it has earned 52 stars from the community.

An endpoint server for efficiently serving quantized open-source LLMs for code. Maintained by wangcx18 on GitHub, where it has earned 59 stars from the community.