https://madewithwhat.net/vllm/ A curated, daily-updated gallery of the best open-source projects built with vLLM, ranked by GitHub stars. Discover dashboards, UI kits, e-commerce, blogs and dev tools. en Wed, 22 Jul 2026 09:29:08 GMT MadeWithWhat catalog https://madewithwhat.net/vllm/project/rag-lcc/ https://madewithwhat.net/vllm/project/rag-lcc/ Tue, 21 Jul 2026 11:09:09 GMT A hands‑on RAG experimentation lab. Largely configurable with debug insights. Classification‑driven corpus construction, filter chains, document loading, chat interaction, Open WebUI integration. Experimental by design and not production‑ready. · 15 GitHub stars · by HarinezumIgel AI & ML https://madewithwhat.net/vllm/project/tinycode/ https://madewithwhat.net/vllm/project/tinycode/ Tue, 21 Jul 2026 11:09:09 GMT A slim, local-LLM-first AI coding assistant. TUI, Web UI, and desktop app. Runs air-gapped with Ollama, vLLM, or any OpenAI-compatible endpoint. · 10 GitHub stars · by bobbyjohnstx AI & ML https://madewithwhat.net/vllm/project/dspark-vllm-gx10/ https://madewithwhat.net/vllm/project/dspark-vllm-gx10/ Tue, 21 Jul 2026 11:09:09 GMT Two-node DGX Spark/ASUS GX10 DeepSeek V4 Flash DSpark NVFP4 port for vLLM 0.25, with live dashboard and reproducible deployment. · 10 GitHub stars · by Anemll Dashboards https://madewithwhat.net/vllm/project/funasr/ https://madewithwhat.net/vllm/project/funasr/ Wed, 15 Jul 2026 22:28:33 GMT Industrial-grade speech recognition toolkit: 170x realtime, 50+ languages, speaker diarization, emotion detection, streaming, and OpenAI-compatible API. · 19,219 GitHub stars · by modelscope AI & ML https://madewithwhat.net/vllm/project/llama-cookbook/ https://madewithwhat.net/vllm/project/llama-cookbook/ Wed, 15 Jul 2026 22:28:33 GMT Welcome to the Llama Cookbook! This is your go to guide for Building with Llama: Getting started with Inference, Fine-Tuning, RAG. We also show you how to solve end to end problems using Llama model family and using them on various provider services · 18,401 GitHub stars · by meta-llama AI & ML https://madewithwhat.net/vllm/project/halfrost-field/ https://madewithwhat.net/vllm/project/halfrost-field/ Wed, 15 Jul 2026 22:28:33 GMT Source Code Deep Dives, System Design & Engineering Blogs | Halfrost-Field ：、 · 13,206 GitHub stars · by halfrost Blogs https://madewithwhat.net/vllm/project/ai-research-skills/ https://madewithwhat.net/vllm/project/ai-research-skills/ Wed, 15 Jul 2026 22:28:33 GMT Comprehensive open-source library of AI research and engineering skills for any AI model. Package the skills and your claude code/codex/gemini agent will be an AI research agent with full horsepower. Maintained by Orchestra Research. · 10,689 GitHub stars · by Orchestra-Research AI & ML https://madewithwhat.net/vllm/project/lmcache/ https://madewithwhat.net/vllm/project/lmcache/ Wed, 15 Jul 2026 22:28:33 GMT LMCache: Supercharge Your LLM with the Fastest KV Cache Layer · 10,541 GitHub stars · by LMCache AI & ML https://madewithwhat.net/vllm/project/openrlhf/ https://madewithwhat.net/vllm/project/openrlhf/ Wed, 15 Jul 2026 22:28:33 GMT An Easy-to-use, Scalable and High-performance Agentic RL Framework based on Ray (PPO & DAPO & REINFORCE++ & VLM & TIS & vLLM & Ray & Async RL) · 9,785 GitHub stars · by OpenRLHF AI & ML https://madewithwhat.net/vllm/project/inference/ https://madewithwhat.net/vllm/project/inference/ Wed, 15 Jul 2026 22:28:33 GMT Swap GPT for any LLM by changing a single line of code. Xinference lets you run open-source, speech, and multimodal models on cloud, on-prem, or your laptop — all through one unified, production-ready inference API. · 9,429 GitHub stars · by xorbitsai AI & ML https://madewithwhat.net/vllm/project/dynamo/ https://madewithwhat.net/vllm/project/dynamo/ Wed, 15 Jul 2026 22:28:33 GMT A Datacenter Scale Distributed Inference Serving Framework · 7,479 GitHub stars · by ai-dynamo AI & ML https://madewithwhat.net/vllm/project/mooncake/ https://madewithwhat.net/vllm/project/mooncake/ Wed, 15 Jul 2026 22:28:33 GMT Mooncake is the serving platform for Kimi, a leading LLM service provided by Moonshot AI. · 5,816 GitHub stars · by kvcache-ai AI & ML https://madewithwhat.net/vllm/project/kserve/ https://madewithwhat.net/vllm/project/kserve/ Wed, 15 Jul 2026 22:28:33 GMT Standardized Distributed Generative and Predictive AI Inference Platform for Scalable, Multi-Framework Deployment on Kubernetes · 5,681 GitHub stars · by kserve AI & ML https://madewithwhat.net/vllm/project/ultrarag/ https://madewithwhat.net/vllm/project/ultrarag/ Wed, 15 Jul 2026 22:28:33 GMT A Low-Code MCP Framework for Building Complex and Innovative RAG Pipelines · 5,643 GitHub stars · by OpenBMB AI & ML https://madewithwhat.net/vllm/project/gpustack/ https://madewithwhat.net/vllm/project/gpustack/ Wed, 15 Jul 2026 22:28:33 GMT A GPU cluster manager for high-performance AI model serving (vLLM, SGLang) and on-demand SSH-accessible GPU instances. · 5,317 GitHub stars · by gpustack AI & ML https://madewithwhat.net/vllm/project/sparrow/ https://madewithwhat.net/vllm/project/sparrow/ Wed, 15 Jul 2026 22:28:33 GMT Structured data extraction, instruction calling and agentic workflows with ML, LLM and Vision LLM · 5,181 GitHub stars · by katanaml AI & ML https://madewithwhat.net/vllm/project/llama-swap/ https://madewithwhat.net/vllm/project/llama-swap/ Wed, 15 Jul 2026 22:28:33 GMT Reliable model swapping for any local OpenAI/Anthropic compatible server - llama.cpp, vllm, etc · 4,992 GitHub stars · by mostlygeek AI & ML https://madewithwhat.net/vllm/project/semantic-router/ https://madewithwhat.net/vllm/project/semantic-router/ Wed, 15 Jul 2026 22:28:33 GMT Intelligent Mixture-of-Models Router for Efficient Heterogeneous LLMs Inference · 4,962 GitHub stars · by vllm-project AI & ML https://madewithwhat.net/vllm/project/tiny-llm/ https://madewithwhat.net/vllm/project/tiny-llm/ Wed, 15 Jul 2026 22:28:33 GMT A course of learning LLM inference serving on Apple Silicon for systems engineers: build a tiny vLLM + Qwen. · 4,361 GitHub stars · by skyzh AI & ML https://madewithwhat.net/vllm/project/lazyllm/ https://madewithwhat.net/vllm/project/lazyllm/ Wed, 15 Jul 2026 22:28:33 GMT Easiest and laziest way for building multi-agent LLMs applications. · 3,853 GitHub stars · by LazyAGI AI & ML https://madewithwhat.net/vllm/project/fastdeploy/ https://madewithwhat.net/vllm/project/fastdeploy/ Wed, 15 Jul 2026 22:28:33 GMT High-performance Inference and Deployment Toolkit for LLMs and VLMs based on PaddlePaddle · 3,702 GitHub stars · by PaddlePaddle AI & ML https://madewithwhat.net/vllm/project/cascadeflow/ https://madewithwhat.net/vllm/project/cascadeflow/ Wed, 15 Jul 2026 22:28:33 GMT Cascading runtime for AI agents. Optimize cost, latency, quality, and policy decisions inside the agent loop. · 3,295 GitHub stars · by lemony-ai AI & ML https://madewithwhat.net/vllm/project/rapid-mlx/ https://madewithwhat.net/vllm/project/rapid-mlx/ Wed, 15 Jul 2026 22:28:33 GMT The fastest local AI engine for Apple Silicon. 4.2x faster than Ollama, 0.08s cached TTFT, 100% tool calling. 17 tool parsers, prompt cache, reasoning separation, cloud routing. Drop-in OpenAI replacement. Works with Claude Code, Cursor, Aider. · 3,262 GitHub stars · by raullenchai AI & ML https://madewithwhat.net/vllm/project/ramalama/ https://madewithwhat.net/vllm/project/ramalama/ Wed, 15 Jul 2026 22:28:33 GMT RamaLama is an open-source developer tool that simplifies the local serving of AI models from any source and facilitates their use for inference in production, all through the familiar language of containers. · 2,956 GitHub stars · by containers AI & ML https://madewithwhat.net/vllm/project/vllm-ascend/ https://madewithwhat.net/vllm/project/vllm-ascend/ Wed, 15 Jul 2026 22:28:33 GMT Community maintained hardware plugin for vLLM on Ascend · 2,496 GitHub stars · by vllm-project AI & ML https://madewithwhat.net/vllm/project/auto-round/ https://madewithwhat.net/vllm/project/auto-round/ Wed, 15 Jul 2026 22:28:33 GMT A SOTA quantization algorithm for high-accuracy low-bit LLM inference, seamlessly optimized for CPU/XPU/CUDA, with multi-datatype support and full compatibility with vLLM, SGLang, and Transformers. · 1,519 GitHub stars · by intel AI & ML https://madewithwhat.net/vllm/project/vllm-mlx/ https://madewithwhat.net/vllm/project/vllm-mlx/ Wed, 15 Jul 2026 22:28:33 GMT OpenAI and Anthropic compatible server for Apple Silicon. Run LLMs and vision-language models (Llama, Qwen-VL, LLaVA) with continuous batching, MCP tool calling, and multimodal support. Native MLX backend, 400+ tok/s. Works with Claude Code. · 1,428 GitHub stars · by waybarrios AI & ML https://madewithwhat.net/vllm/project/local-studio/ https://madewithwhat.net/vllm/project/local-studio/ Wed, 15 Jul 2026 22:28:33 GMT Control panel for VLLM, Sglang, llama.cpp, exllamav3 · 1,359 GitHub stars · by sybil-solutions AI & ML https://madewithwhat.net/vllm/project/inferencex/ https://madewithwhat.net/vllm/project/inferencex/ Wed, 15 Jul 2026 22:28:33 GMT Open Source Continuous Inference Benchmark Research Platform — Kimi K2.7-Code, MiniMax M3, DeepSeekv4, GLM5 - GB200 NVL72 vs MI355X vs B200 vs GB300 NVL72 & soon™ TPUv6e/v7/Trainium2/3 | — Kimi K2.7-Code、MiniMax M3、DeepSeekv4、GLM5 - GB200 NVL72 vs MI355X vs B200 vs GB300 NVL72，™ TPUv6e/v7/Trainium2/3 · 1,244 GitHub stars · by SemiAnalysisAI AI & ML https://madewithwhat.net/vllm/project/kubeai/ https://madewithwhat.net/vllm/project/kubeai/ Wed, 15 Jul 2026 22:28:33 GMT AI Inference Operator for Kubernetes. The easiest way to serve ML models in production. Supports VLMs, LLMs, embeddings, and speech-to-text. · 1,222 GitHub stars · by kubeai-project AI & ML https://madewithwhat.net/vllm/project/bricksllm/ https://madewithwhat.net/vllm/project/bricksllm/ Wed, 15 Jul 2026 22:28:33 GMT Enterprise-grade API gateway that helps you monitor and impose cost or rate limits per API key. Get fine-grained access control and monitoring per user, application, or environment. Supports OpenAI, Azure OpenAI, Anthropic, vLLM, and open-source LLMs. · 1,217 GitHub stars · by bricks-cloud AI & ML https://madewithwhat.net/vllm/project/gptqmodel/ https://madewithwhat.net/vllm/project/gptqmodel/ Wed, 15 Jul 2026 22:28:33 GMT LLM model quantization (compression) toolkit with HW acceleration support for Nvidia, AMD, Intel GPU and Intel/AMD/Apple CPU via HF, vLLM, and SGLang. · 1,205 GitHub stars · by ModelCloud AI & ML https://madewithwhat.net/vllm/project/prometheus-eval/ https://madewithwhat.net/vllm/project/prometheus-eval/ Wed, 15 Jul 2026 22:28:33 GMT Evaluate your LLM's response with Prometheus and GPT4 · 1,102 GitHub stars · by prometheus-eval AI & ML https://madewithwhat.net/vllm/project/kvcached/ https://madewithwhat.net/vllm/project/kvcached/ Wed, 15 Jul 2026 22:28:33 GMT Virtualized Elastic KV Cache for Dynamic GPU Sharing and Beyond · 1,097 GitHub stars · by ovg-project AI & ML https://madewithwhat.net/vllm/project/tiny-vllm/ https://madewithwhat.net/vllm/project/tiny-vllm/ Wed, 15 Jul 2026 22:28:33 GMT Build your own high performance LLM inference engine in C++ and CUDA - a smaller version of vLLM · 919 GitHub stars · by jmaczan AI & ML https://madewithwhat.net/vllm/project/llm-note/ https://madewithwhat.net/vllm/project/llm-note/ Wed, 15 Jul 2026 22:28:33 GMT LLM notes, including model inference, transformer model structure, and llm framework code analysis notes. · 883 GitHub stars · by harleyszhang AI & ML https://madewithwhat.net/vllm/project/llmcord/ https://madewithwhat.net/vllm/project/llmcord/ Wed, 15 Jul 2026 22:28:33 GMT Make Discord your LLM frontend - Supports any OpenAI compatible API (OpenRouter, Ollama and more) · 814 GitHub stars · by jakobdylanc AI & ML https://madewithwhat.net/vllm/project/unirl/ https://madewithwhat.net/vllm/project/unirl/ Wed, 15 Jul 2026 22:28:33 GMT UniRL is a Framework for Unified Multimodal Model Reinforcement Learning · 810 GitHub stars · by Tencent-Hunyuan DevTools https://madewithwhat.net/vllm/project/bambooai/ https://madewithwhat.net/vllm/project/bambooai/ Wed, 15 Jul 2026 22:28:33 GMT A Python library powered by Language Models (LLMs) for conversational data discovery and analysis. · 782 GitHub stars · by pgalko AI & ML https://madewithwhat.net/vllm/project/local-ai-ocr/ https://madewithwhat.net/vllm/project/local-ai-ocr/ Wed, 15 Jul 2026 22:28:33 GMT An local, offline (after initial setup), portable OCR software that can process images and PDF files, using DeepSeek-OCR-2 AI (running directly on your machine). · 773 GitHub stars · by th1nhhdk AI & ML https://madewithwhat.net/vllm/project/lightcompress/ https://madewithwhat.net/vllm/project/lightcompress/ Wed, 15 Jul 2026 22:28:33 GMT [EMNLP 2024 & AAAI 2026] A powerful toolkit for compressing large models including LLMs, VLMs, and video generative models. · 733 GitHub stars · by ModelTC DevTools https://madewithwhat.net/vllm/project/vidur/ https://madewithwhat.net/vllm/project/vidur/ Wed, 15 Jul 2026 22:28:33 GMT Accurate, large-scale, and extensible simulator for LLM inference Systems · 641 GitHub stars · by microsoft AI & ML https://madewithwhat.net/vllm/project/efficientsam3/ https://madewithwhat.net/vllm/project/efficientsam3/ Wed, 15 Jul 2026 22:28:33 GMT EfficientSAM3 compresses SAM3 into lightweight, edge-friendly models via progressive knowledge distillation for fast promptable concept segmentation and tracking. · 628 GitHub stars · by SimonZeng7108 AI & ML https://madewithwhat.net/vllm/project/flashtts/ https://madewithwhat.net/vllm/project/flashtts/ Wed, 15 Jul 2026 22:28:33 GMT SparkTTS、OrpheusTTS，。 · 612 GitHub stars · by HuiResearch DevTools https://madewithwhat.net/vllm/project/ai-bank-statement-document-automation-by-llm-and-personal-finanical-analysis-prediction/ https://madewithwhat.net/vllm/project/ai-bank-statement-document-automation-by-llm-and-personal-finanical-analysis-prediction/ Wed, 15 Jul 2026 22:28:33 GMT AI Bank Statement Document Automation By LLM model and Personal Finanical Analysis · 597 GitHub stars · by johnsonhk88 AI & ML https://madewithwhat.net/vllm/project/robobrain/ https://madewithwhat.net/vllm/project/robobrain/ Wed, 15 Jul 2026 22:28:33 GMT [CVPR 2025] RoboBrain: A Unified Brain Model for Robotic Manipulation from Abstract to Concrete. Official Repository. · 558 GitHub stars · by FlagOpen AI & ML https://madewithwhat.net/vllm/project/verl-omni/ https://madewithwhat.net/vllm/project/verl-omni/ Wed, 15 Jul 2026 22:28:33 GMT Multimodal RL training framework for diffusion & omni models · 548 GitHub stars · by verl-project Docs https://madewithwhat.net/vllm/project/crater/ https://madewithwhat.net/vllm/project/crater/ Wed, 15 Jul 2026 22:28:33 GMT Crater is a cloud-native AI training & inference platform. · 542 GitHub stars · by raids-lab AI & ML https://madewithwhat.net/vllm/project/openinfer/ https://madewithwhat.net/vllm/project/openinfer/ Wed, 15 Jul 2026 22:28:33 GMT Pure Rust + CUDA LLM inference engine — no PyTorch, OpenAI-compatible, serves Qwen3 to Kimi-K2 · 535 GitHub stars · by openinfer-project AI & ML https://madewithwhat.net/vllm/project/restai/ https://madewithwhat.net/vllm/project/restai/ Wed, 15 Jul 2026 22:28:33 GMT RESTai is an AIaaS (AI as a Service) open-source platform. Supports many public and local LLM suported by Ollama/vLLM/etc. Precise embeddings usage, tuning, analytics etc. Built-in image/audio generation with dynamic loading generators. Live chat deployment. Built-in block based graphical language. Prompt versioning and much more... · 512 GitHub stars · by apocas AI & ML