distributed-inference-vllm by KempnerInstitute

Intelligent load balancer for distributed vLLM server clusters vLLM. Maintained by xerrors on GitHub, where it has earned 19 stars from the community.

Tencent Hy3 295B MoE (NVFP4) on 2x NVIDIA DGX Spark — TP2 over 200GbE, 256K context, 26 tok/s end-to-end. Tuned, benchmarked, agent-ready. Maintained by joeynyc on GitHub, where it has earned 11 stars from the community.

Serve Poolside Laguna S 2.1 (NVFP4) on the NVIDIA DGX Spark (GB10) without hanging your box. Working stack, crash-safe configs, benchmarks, and the exact gotchas. Maintained by sudoingX on GitHub, where it has earned 18 stars from the community.