vector-inference

Efficient LLM inference on Slurm clusters. Maintained by VectorInstitute on GitHub, where it has earned 106 stars from the community.

It's actively developed around audio-transcription, inference, llm, and is a solid reference for anyone building with these tools.