// 332 open-source projects indexed

$ Discover the best projects built with vLLM

A hand-curated, daily-updated gallery of open-source projects. Browse the ecosystem, ranked by GitHub stars.

>

sort --stars ▾

#all#aiml278#devtools44#docs4#dashboards3#blogs1

// 332 results

Optimized vLLM setup for Qwen3.6-27B-FP8 on dual RTX PRO 6000 Blackwell (192 GB GDDR7, no NVLink); config, benchmark sweep results, and custom chat template with thinking mode off by default. Maintained by theogravity on GitHub, where it has earned 18 stars from the community.

@theogravity5mo ago

Serve Poolside Laguna S 2.1 (NVFP4) on the NVIDIA DGX Spark (GB10) without hanging your box. Working stack, crash-safe configs, benchmarks, and the exact gotchas. Maintained by sudoingX on GitHub, where it has earned 18 stars from the community.

@sudoingX3mo ago

Collection of recent advanced RAG techniques. Maintained by hienhayho on GitHub, where it has earned 18 stars from the community.

@hienhayho11mo ago

Source code for the paper: Hear Both Sides: Efficient Multi-Agent Debate via Diversity-Aware Message Retention. Maintained by DA2I2-SLM on GitHub, where it has earned 18 stars from the community.

@DA2I2-SLM6mo ago

An OpenAI Compatible API which integrates LLM, Embedding and Reranker. LLM、Embedding Reranker OpenAI API. Maintained by hcd233 on GitHub, where it has earned 18 stars from the community.

The official repo for the paper "Optimizing LLM Inference Throughput via Memory-aware and SLA-constrained Dynamic Batching". Maintained by KevinLee1110 on GitHub, where it has earned 18 stars from the community.

@KevinLee11101y ago

DGX Spark research and tests - containers, benchmarks, and investigation notes for running models on GB10 (SM 12.1). Maintained by Sggin1 on GitHub, where it has earned 18 stars from the community.

Object-storage-native KV cache for LLM inference & RL. Cross-restart, cross-conversation, cross-engine via shared S3 bucket. Maintained by Venkat2811 on GitHub, where it has earned 18 stars from the community.

@Venkat28112mo ago

htop for your LLM inference cluster. Maintained by InfraWhisperer on GitHub, where it has earned 17 stars from the community.

@InfraWhisperer5mo ago

KONASH: Train knowledge agents that search, retrieve, and reason. Based on KARL (Databricks, 2026). Maintained by konaequity on GitHub, where it has earned 17 stars from the community.

@konaequity6mo ago

An open-source, model-agnostic agent harness for local LLMs. Define agents in YAML (tools, memory, deny-first permissions) and run them against any OpenAI-compatible endpoint: vLLM, Ollama, LM Studio, or llama.cpp. Maintained by ahwurm on GitHub, where it has earned 17 stars from the community.

Agentic SDLC with local LLM's. Maintained by xencon on GitHub, where it has earned 17 stars from the community.

Convert pdf and image files into markdown. Maintained by arc53 on GitHub, where it has earned 17 stars from the community.

Dual-mode cloud burst LLM router. Edge-first or cloud-first inference with data sovereignty, cost controls, and automatic failover. https://aiburstcloud.com. Maintained by aiburstcloud on GitHub, where it has earned 17 stars from the community.

@aiburstcloud2mo ago

A project worth exploring. Maintained by ampir-nn on GitHub, where it has earned 16 stars from the community.

@ampir-nn9mo ago

Diagnose vLLM inference servers. Maintained by vllm-doctor on GitHub, where it has earned 16 stars from the community.

@vllm-doctor2mo ago

vLLM + Qwen3.6-27B (BF16) OpenAI-compatible inference server on AMD Strix Halo (Ryzen AI Max+ 395, gfx1151). Vision input, 256K context, /v1/responses with separated reasoning, via TheRock ROCm. Maintained by hec-ovi on GitHub, where it has earned 16 stars from the community.

@hec-ovi5mo ago

Open recipes, engine patches, and benchmark harnesses for LLM inference on Intel Arc Pro B60/B70 (Battlemage, Xe2). MoE 35B at 126 t/s decode / 7.5K t/s prefill single-stream — vLLM XPU MTP unlocked. Maintained by SergiioB on GitHub, where it has earned 16 stars from the community.

@SergiioB8w ago

Interested in running a conspiracy of agents (technical term) on your Local AI Infra? Who isn't!. Maintained by digitalspaceport on GitHub, where it has earned 16 stars from the community.

@digitalspaceport6mo ago

The official implementation of NeurlPS 2025 D&B paper: IndustryEQA: Pushing the frontiers of Embodied Question Answering in Industrial Scenarios. Maintained by JackYFL on GitHub, where it has earned 16 stars from the community.

Thinking-native compiler for LEET Language Understanding item sets — killer-item design grammar, verification separation, deterministic gate chain, 3-model benchmark. Maintained by Glockevonpavlov on GitHub, where it has earned 15 stars from the community.

@Glockevonpavlov2mo ago

A curated collection of Claude Code agent skills that accelerate the entire vLLM development lifecycle. Maintained by shen-shanshan on GitHub, where it has earned 15 stars from the community.

@shen-shanshan8w ago

Easily boost the speed of pulling your models and datasets from various of inference runtimes. (e.g. HuggingFace, Ollama, vLLM, and more!). Maintained by moeru-ai on GitHub, where it has earned 15 stars from the community.

@moeru-ai1y ago

AI-Inference-Managed-by-AI: Go binary for managing AI inference on edge devices. Maintained by Approaching-AI on GitHub, where it has earned 15 stars from the community.

@Approaching-AI8w ago

Hands-on GPU/HPC infrastructure operations: K8s GPU scheduling, HAMi sharing, Slurm, observability & vLLM inference. Learn it free on a laptop; validate on one cheap GPU. Maintained by ld-singh on GitHub, where it has earned 15 stars from the community.

@ld-singh2mo ago

Hikari's curated English measured notes on local LLM systems (serving, specdec, training, steering). Maintained by hikarioyama on GitHub, where it has earned 15 stars from the community.

@hikarioyama2mo ago

TokenPowerBench: Benchmarking the power consumption of LLM inference across engines and deployment configurations. Maintained by chenxuniu on GitHub, where it has earned 15 stars from the community.

@chenxuniu6mo ago

Exocomp Agentic Environment for Go. Maintained by cookiengineer on GitHub, where it has earned 15 stars from the community.

@cookiengineer4mo ago

Reproducible recipe: serve abliterated Gemma-4-12B (gemma4_unified) at 50-118 tok/s on no-NVLink Blackwell (SM120) via vLLM nightly + ModelOpt FP8/NVFP4 + MTP spec-decode. Maintained by lna-lab on GitHub, where it has earned 15 stars from the community.

@lna-lab4mo ago

Turn any NVIDIA GPU into a local AI platform. Inference + fine-tuning in your browser. One command to start, automatic clustering. Maintained by getainode on GitHub, where it has earned 14 stars from the community.

@getainode3mo ago

Source-code level analysis of LLM RL training infra: async RL, weight sync, FP8, MoE routing | LLM RL. Maintained by zpqiu on GitHub, where it has earned 14 stars from the community.

Running Large Language Model easily. Maintained by janelu9 on GitHub, where it has earned 14 stars from the community.

@janelu92mo ago

FlashAttention-style custom attention backend for vLLM on AMD MI50/MI60/Radeon VII (gfx906). Downstream fork of mixa3607/ML-gfx906 with replacement HIP kernels and a vllm.general_plugins entry point. Maintained by nick413-bit on GitHub, where it has earned 14 stars from the community.

@nick413-bit6mo ago

AI hardware fit calculator for LLM, embedding, reranker, OCR and VLM workloads — VRAM, throughput, licensing and multi-GPU planning. Maintained by jaeseok614 on GitHub, where it has earned 13 stars from the community.

@jaeseok6148w ago

A high-throughput LLM serving engine with non-uniform KV cache compression, built on vLLM. Maintained by aiha-lab on GitHub, where it has earned 13 stars from the community.

@aiha-lab8w ago

GPU Service Manager for LLM workloads on Linux/NVIDIA systems. Maintained by jaigouk on GitHub, where it has earned 13 stars from the community.

@jaigouk3mo ago

Honeycomb Lab — hex map + OpenAI gateway control plane for a home AI fleet. Maintained by joeynyc on GitHub, where it has earned 13 stars from the community.

@joeynyc2mo ago

Multi-tenant streaming ASR server for Qwen3-ASR (vLLM AsyncLLMEngine). Drop-in replacement for qwen-asr-demo-streaming. Maintained by jayter-official on GitHub, where it has earned 13 stars from the community.

@jayter-official4mo ago

Kubernetes scanner that discovers LLMs running on vLLM and extracts their deployment and runtime facts. Maintained by paralleliq on GitHub, where it has earned 13 stars from the community.

@paralleliq3mo ago

Local LLM speed measurements from a 4x RTX 3090 rig, with exact launch commands and per-run provenance. Maintained by alesha-pro on GitHub, where it has earned 13 stars from the community.

@alesha-pro2mo ago

LLM, Fine Tuning, Llama 2, Gemma, Mixtral, vLLM, LangChain, RAG, ChromaDB, FAISS. Maintained by joydeb28 on GitHub, where it has earned 13 stars from the community.

@joydeb282y ago

Automated Triton w8a8 block FP8 kernel tuning tool for vLLM. Auto-detects model architecture, supports Qwen3-Coder-30B-A3B-Instruct-FP8/DeepSeek-V3/custom models, multi-GPU parallel tuning, and generates optimized kernel configs for quantization. Maintained by massif-01 on GitHub, where it has earned 13 stars from the community.

@massif-015mo ago

SAM — Smart Agentic Model: CLI coding agent for open-source LLMs. pip install sam-agent. Maintained by SecFathy on GitHub, where it has earned 13 stars from the community.

@SecFathy7mo ago

FlashHead: Efficient Drop-In Replacement for the Classification Head in Language Model Inference. Maintained by embedl on GitHub, where it has earned 13 stars from the community.

A lean, local AI research agent framework that runs on your machine. Maintained by KabakaWilliam on GitHub, where it has earned 13 stars from the community.

@KabakaWilliam6mo ago

LLM Performance Testing | K6 + Grafana + InfluxDB | A tiny toolkit for load testing and benchmarking OpenAI-like inference endpoints using K6 + Grafana + InfluxDB. Maintained by 0xnyn on GitHub, where it has earned 13 stars from the community.

Batch LLM Inference with Ray Data LLM: From Simple to Advanced. Maintained by 0-mostafa-rezaee-0 on GitHub, where it has earned 12 stars from the community.

@0-mostafa-rezaee-08mo ago

LLM inference engine built from scratch in C++. No PyTorch, no frameworks. Maintained by Anirudh171202 on GitHub, where it has earned 12 stars from the community.

@Anirudh1712026mo ago

Benchmark OpenAI-compatible AI endpoints and AI Accelerators in a reproducible structured way. Maintained by flexaihq on GitHub, where it has earned 12 stars from the community.

@flexaihq7mo ago

GLM-5.2 744B with a true 4-bit NVFP4 KV cache on 4x DGX Spark: 42 tok/s peak, 317K-token KV pool (+58.6% vs fp8), needle-verified at 250K depth, serving 316K context. Maintained by tonyd2wild on GitHub, where it has earned 12 stars from the community.

@tonyd2wild3mo ago

An autonomous agentic pipeline that finds, proves, and patches real C memory-safety vulnerabilities end-to-end using a single 7B open-source LLM (Qwen2.5-Coder) on vLLM, with AddressSanitizer as a ground-truth oracle. Maintained by FareedKhan-dev on GitHub, where it has earned 12 stars from the community.

@FareedKhan-dev4mo ago

The Continuous Verification and Optimization layer for self-hosted LLMs. Maintained by colomalabs on GitHub, where it has earned 12 stars from the community.

@colomalabs3mo ago

Open-source local AI server configs, GFX906 runtime maintenance, reproducible benchmarks, and QC methods for affordable AI research infrastructure. Maintained by joe2gaan on GitHub, where it has earned 12 stars from the community.

@joe2gaan3mo ago

Local-first AI coding agent runtime — your models, fail-closed safety, evidence-gated self-improvement. Maintained by cdnwetzel on GitHub, where it has earned 12 stars from the community.

@cdnwetzel8w ago

vLLM tool parser for Qwen2.5-Coder models using <tools> tag format. Maintained by hanXen on GitHub, where it has earned 12 stars from the community.

Distributed Inference with vLLM. Maintained by KempnerInstitute on GitHub, where it has earned 12 stars from the community.

@KempnerInstitute3mo ago

Run DeepSeek-V4-Flash 304B on a single NVIDIA GB10 / DGX Spark - 2-bit MoE planes, FP4 quality recovery, speculative decoding. Maintained by lrozewicz on GitHub, where it has earned 11 stars from the community.

@lrozewicz8w ago

nano-vllmmoeSpeculative Decoding. Maintained by banfeb on GitHub, where it has earned 11 stars from the community.

MinerU AMD vLLM + hybrid-auto-engine ！ AMD/ROCm 。 MinerU 3.x ROCm 7.x + PyTorch 2.11.0 + vLLM ， NVIDIA 。. Maintained by buptanswer on GitHub, where it has earned 11 stars from the community.

@buptanswer4mo ago

GLM-5.2 on 4x DGX Spark with adaptive MTP K2/K4/K5, FULL CUDA graphs, DCP2, 520K context, and a downloadable ARM64 runtime image. Maintained by 0xdfi on GitHub, where it has earned 11 stars from the community.

From the blog

Latest vLLM guides & deep-dives

All articles →

HaystackGuideMigrating Haystack to vLLM: decision & execution guide

Step-by-step migration guidance to adopt vLLM as the inference/serving backend for Haystack orchestration. Includes suitability, inventory, phased plan, tests, data/API considerations, and rollback gates.

AAndrea Lunelio · Jul 15, 2026 · 15 min

vLLMGuidevLLM ecosystem guide: architecture, integrations, evaluation

A practical, evidence-grounded ecosystem map for vLLM covering the core repository, integration patterns, evaluation criteria, maintenance signals, licensing, and a discovery workflow.

MMadeWithWhat Editorial Team · Jul 9, 2026 · 15 min

Video archive

vLLM talks, tutorials & deep-dives

DevOps + LLM +AI Project w/ Docker, Kubernetes, vLLM | Resume Project for Beginners

DevOps + LLM +AI Project w/ Docker, Kubernetes, vLLM | Resume Project for Beginners

VVishakha Sadhwani · 18K views · Jun 22, 2026

Understanding vLLM with a Hands On Demo

Understanding vLLM with a Hands On Demo

KKodeKloud · 84K views · Mar 31, 2026

NVIDIA NemoClaw + OpenShell: OpenClaw Agent in a Secure Sandbox - Local vLLM Setup

NVIDIA NemoClaw + OpenShell: OpenClaw Agent in a Secure Sandbox - Local vLLM Setup

FFahd Mirza · 15K views · Mar 17, 2026