vector-inference

Efficient LLM inference on Slurm clusters. Maintained by VectorInstitute on GitHub, where it has earned 106 stars from the community.

It's actively developed around inference, text-embedding, vllm, and is a solid reference for anyone building with these tools.