$ Discover the best projects built with vLLM

Tencent Hy3 295B MoE (NVFP4) on 2x NVIDIA DGX Spark — TP2 over 200GbE, 256K context, 26 tok/s end-to-end. Tuned, benchmarked, agent-ready. Maintained by joeynyc on GitHub, where it has earned 11 stars from the community.

Professional CLI toolkit for Modal GPU workflows, Docker Compose deployments, Python runtimes, model serving, and account/billing management. Maintained by PuxHocDL on GitHub, where it has earned 11 stars from the community.

Spark Pulse is a web control plane for spark-vllm-docker. Maintained by kharkevich-engineering-lab on GitHub, where it has earned 11 stars from the community.

Qwen3.6 vLLM Toolkit — Launcher + Templates, Optimized for 2×24 GB. vLLM launcher with an optimized hybrid chat template drawing from the best community fixes for Qwen 3.6 27B. Maintained by pajitosingh on GitHub, where it has earned 11 stars from the community.

An out-of-tree vLLM plugin for Mobilint NPU runtime integration. Maintained by mobilint on GitHub, where it has earned 11 stars from the community.

Kubernetes-native control plane for scale-to-zero serving of long-tail LLMs. Maintained by noctaya on GitHub, where it has earned 11 stars from the community.

GPU orchestrator with vLLM sleep mode. Maintained by arkorlab on GitHub, where it has earned 11 stars from the community.

Ask Poddy: Run Open Source LLMs and Embeddings as OpenAI-Compatible Serverless Endpoints (Tutorial). Maintained by blib-la on GitHub, where it has earned 11 stars from the community.

Docker Compose setup that integrates vLLM, Open WebUI and NGINX (for SSL termination). Maintained by marib00 on GitHub, where it has earned 11 stars from the community.

Repository for Multililngual Generation, RAG evaluations, and surrogate judge training for Arena RAG leaderboard (NAACL'25). Maintained by vectara on GitHub, where it has earned 11 stars from the community.

Multi-model LLM serving for NVIDIA DGX Spark with vLLM, web UI, and tool calling. Maintained by dataforgex on GitHub, where it has earned 11 stars from the community.

The official implementation of CVPR Workshop 2025 paper: Window Token Concatenation for Efficient Visual Large Language Models. Maintained by JackYFL on GitHub, where it has earned 11 stars from the community.

agentsculptor is an experimental AI-powered development agent designed to analyze, refactor, and extend Python projects automatically. It uses an OpenAI-like planner–executor loop on top of a vLLM backend, combining project context analysis, structured tool calls, and iterative refinement. It has only been tested with gpt-oss-120b via vLLM. Maintained by Perpetue237 on GitHub, where it has earned 11 stars from the community.

A slim, local-LLM-first AI coding assistant. TUI, Web UI, and desktop app. Runs air-gapped with Ollama, vLLM, or any OpenAI-compatible endpoint. Maintained by bobbyjohnstx on GitHub, where it has earned 11 stars from the community.

Tiny, stateless Go router that dispatches OpenAI-compatible requests to single-model vLLM and sglang backends with zero external dependencies. Maintained by mirkolenz on GitHub, where it has earned 11 stars from the community.

Terraform setup for deploying a private coding LLM on Vast.ai with vLLM, Qwen3 Coder, and OpenCode. Maintained by anqorithm on GitHub, where it has earned 11 stars from the community.

Extend LLM context windows beyond GPU memory limits with disk-backed KV cache. Maintained by tirdyhouse on GitHub, where it has earned 11 stars from the community.

npm pacakge to Craft files into Markdown with ease. Maintained by jparkerweb on GitHub, where it has earned 11 stars from the community.

General Information, model certifications, and benchmarks for nm-vllm enterprise distributions. Maintained by neuralmagic on GitHub, where it has earned 11 stars from the community.

Automated comic cataloging tool that identifies issues directly from cover images using a vision-language model, then cross-references results with the Grand Comics Database and the ComicVine API to generate structured, high-confidence collection data with minimal manual entry. Maintained by boyobob on GitHub, where it has earned 10 stars from the community.

A curated collection of NLP and LLM resources. Covers essential papers and blogs on Transformers, Reinforcement Learning (RLHF, DPO, GRPO), Mechanistic Interpretability, Scaling Laws, and MLSys. Maintained by rraghavkaushik on GitHub, where it has earned 10 stars from the community.

Open-source, self-hosted LLM chat application, featuring local-first data storage and real-time streaming responses. Maintained by GHuyHuynh on GitHub, where it has earned 10 stars from the community.

A voice assistant with local LLM as a backend. Maintained by Saga9103 on GitHub, where it has earned 10 stars from the community.

Perl Framework for AI - Langertha - the viking of AI. Maintained by Getty on GitHub, where it has earned 10 stars from the community.

nvtop for vLLM — an interactive terminal dashboard for vLLM serving performance (concurrency, throughput, cache & KV memory, latency, spec-decode, GPU). Maintained by bryanvine on GitHub, where it has earned 10 stars from the community.

inference cookbook / inference. Maintained by SuperMarioYL on GitHub, where it has earned 10 stars from the community.

Benchmarks and notes for running modern LLMs with vLLM on 8x Tesla V100-32GB in 2026. Maintained by KumphanartDansiri on GitHub, where it has earned 10 stars from the community.

Lna-Lab production pipeline: GGUF -> modelopt-format NVFP4 + working MTP head for vLLM on RTX PRO 6000 Blackwell (SM120). Stages 2 (NVFP4) and 3 (MTP graft) are Lna-Lab originals; stage 1 (GGUF->bf16) reuses li-yifei/gguf-to-nvfp4. Maintained by lna-lab on GitHub, where it has earned 10 stars from the community.

Read PGS (Bluray) and VobSub (DVD) image subtitles and extract their text using external Vision Language Models. Maintained by hekmon on GitHub, where it has earned 10 stars from the community.

VS Code extension to connect to local LLM inference servers (Ollama, llama.cpp, LocalAI) via OpenAI API compatible endpoints. Maintained by krevas on GitHub, where it has earned 10 stars from the community.

AI-,. XTTS v2, real-time (Vosk/Whisper) offline LLM. - (Vue 3), Telegram-,, fine-tuning pipeline. Self-hosted,,. Maintained by ShaerWare on GitHub, where it has earned 10 stars from the community.

GLM-5.2 QuantTrio (Unpruned ) on 4xDGX Spark (GB10): 327K-655K context or up to 5 concurrent agents, one receipe. TP4 + DCP + MTP. Maintained by XanuNetworks on GitHub, where it has earned 10 stars from the community.