// 332 open-source projects indexed

$ Discover the best projects built with vLLM

A hand-curated, daily-updated gallery of open-source projects. Browse the ecosystem, ranked by GitHub stars.
>
sort --stars ▾
#all#aiml278#devtools44#docs4#dashboards3#blogs1
// 332 results
AI & ML
Open-source speech recognition toolkit for training, inference, streaming ASR, VAD, punctuation, speaker diarization pipelines, and OpenAI-compatible/MCP serving. Maintained by modelscope on GitHub, where it has earned 19,644 stars from the community.
@modelscope2mo ago
AI & ML
Welcome to the Llama Cookbook! This is your go to guide for Building with Llama: Getting started with Inference, Fine-Tuning, RAG. We also show you how to solve end to end problems using Llama model family and using them on various provider services. Maintained by meta-llama on GitHub, where it has earned 18,552 stars from the community.
@meta-llama5mo ago
Blogs
Source Code Deep Dives, System Design & Engineering Blogs | Halfrost-Field :、. Maintained by halfrost on GitHub, where it has earned 13,214 stars from the community.
@halfrost2mo ago
AI & ML
Comprehensive open-source library of AI research and engineering skills for any AI model. Package the skills and your claude code/codex/gemini agent will be an AI research agent with full horsepower. Maintained by Orchestra Research. Maintained by Orchestra-Research on GitHub, where it has earned 11,405 stars from the community.
@Orchestra-Research4mo ago
AI & ML
LMCache: Supercharge Your LLM with the Fastest KV Cache Layer. Maintained by LMCache on GitHub, where it has earned 11,024 stars from the community.
@LMCache2mo ago
AI & ML
An Easy-to-use, Scalable and High-performance Agentic RL Framework based on Ray (PPO & DAPO & REINFORCE++ & VLM & TIS & vLLM & Ray & Async RL). Maintained by OpenRLHF on GitHub, where it has earned 9,882 stars from the community.
@OpenRLHF3mo ago
AI & ML
Swap GPT for any LLM by changing a single line of code. Xinference lets you run open-source, speech, and multimodal models on cloud, on-prem, or your laptop — all through one unified, production-ready inference API. Maintained by xorbitsai on GitHub, where it has earned 9,479 stars from the community.
@xorbitsai2mo ago
AI & ML
A Datacenter Scale Distributed Inference Serving Framework. Maintained by ai-dynamo on GitHub, where it has earned 7,683 stars from the community.
@ai-dynamo2mo ago
AI & ML
Mooncake is the serving platform for Kimi, a leading LLM service provided by Moonshot AI. Maintained by kvcache-ai on GitHub, where it has earned 6,157 stars from the community.
@kvcache-ai2mo ago
AI & ML
Standardized Distributed Generative and Predictive AI Inference Platform for Scalable, Multi-Framework Deployment on Kubernetes. Maintained by kserve on GitHub, where it has earned 5,773 stars from the community.
@kserve2mo ago
AI & ML
A Low-Code MCP Framework for Building Complex and Innovative RAG Pipelines. Maintained by OpenBMB on GitHub, where it has earned 5,713 stars from the community.
@OpenBMB1d ago
AI & ML
A GPU cluster manager for high-performance AI model serving (vLLM, SGLang) and on-demand SSH-accessible GPU instances. Maintained by gpustack on GitHub, where it has earned 5,436 stars from the community.
@gpustack2mo ago
AI & ML
Reliable model swapping for any local OpenAI/Anthropic compatible server - llama.cpp, vllm, etc. Maintained by mostlygeek on GitHub, where it has earned 5,268 stars from the community.
@mostlygeek2mo ago
AI & ML
Structured data extraction, instruction calling and agentic workflows with ML, LLM and Vision LLM. Maintained by katanaml on GitHub, where it has earned 5,188 stars from the community.
@katanaml2mo ago
AI & ML
A programmable Mixture-of-Models router for heterogeneous LLM inference. Maintained by vllm-project on GitHub, where it has earned 5,116 stars from the community.
@vllm-project2mo ago
AI & ML
A course of learning LLM inference serving on Apple Silicon for systems engineers: build a tiny vLLM + Qwen. Maintained by skyzh on GitHub, where it has earned 4,441 stars from the community.
@skyzh2mo ago
AI & ML
Easiest and laziest way for building multi-agent LLMs applications. Maintained by LazyAGI on GitHub, where it has earned 3,860 stars from the community.
@LazyAGI2mo ago
AI & ML
Cascading runtime for AI agents. Optimize cost, latency, quality, and policy decisions inside the agent loop. Maintained by lemony-ai on GitHub, where it has earned 3,838 stars from the community.
@lemony-ai3mo ago
AI & ML
High-performance Inference and Deployment Toolkit for LLMs and VLMs based on PaddlePaddle. Maintained by PaddlePaddle on GitHub, where it has earned 3,703 stars from the community.
@PaddlePaddle2mo ago
AI & ML
The fastest local AI engine for Apple Silicon. 4.2x faster than Ollama, 0.08s cached TTFT, 100% tool calling. 17 tool parsers, prompt cache, reasoning separation, cloud routing. Drop-in OpenAI replacement. Works with Claude Code, Cursor, Aider. Maintained by raullenchai on GitHub, where it has earned 3,400 stars from the community.
@raullenchai2mo ago
AI & ML
RamaLama is an open-source developer tool that simplifies the local serving of AI models from any source and facilitates their use for inference in production, all through the familiar language of containers. Maintained by containers on GitHub, where it has earned 2,987 stars from the community.
@containers2mo ago
AI & ML
Community maintained hardware plugin for vLLM on Ascend. Maintained by vllm-project on GitHub, where it has earned 2,560 stars from the community.
@vllm-project2mo ago
AI & ML
Control panel for VLLM, Sglang, llama.cpp, exllamav3. Maintained by sybil-solutions on GitHub, where it has earned 1,557 stars from the community.
@sybil-solutions2mo ago
AI & ML
A SOTA quantization algorithm for high-accuracy low-bit LLM inference, seamlessly optimized for CPU/XPU/CUDA, with multi-datatype support and full compatibility with vLLM, SGLang, and Transformers. Maintained by intel on GitHub, where it has earned 1,552 stars from the community.
@intel2mo ago
AI & ML
OpenAI and Anthropic compatible server for Apple Silicon. Run LLMs and vision-language models (Llama, Qwen-VL, LLaVA) with continuous batching, MCP tool calling, and multimodal support. Native MLX backend, 400+ tok/s. Works with Claude Code. Maintained by waybarrios on GitHub, where it has earned 1,488 stars from the community.
@waybarrios2mo ago
AI & ML
Open Source Continuous Inference Benchmark Research Platform — Kimi K3 2.8T, MiniMax M3, DeepSeekv4, GLM5 - GB200 NVL72 vs MI355X vs B200 vs GB300 NVL72 & soon™ TPUv6e/v7/Trainium2/3 | — Kimi K2.7-Code、MiniMax M3、DeepSeekv4、GLM5 - GB200 NVL72 vs MI355X vs B200 vs GB300 NVL72,™ TPUv6e/v7/Trainium2/3. Maintained by SemiAnalysisAI on GitHub, where it has earned 1,318 stars from the community.
@SemiAnalysisAI2mo ago
AI & ML
AI Inference Operator for Kubernetes. The easiest way to serve ML models in production. Supports VLMs, LLMs, embeddings, and speech-to-text. Maintained by kubeai-project on GitHub, where it has earned 1,238 stars from the community.
@kubeai-project2mo ago
AI & ML
LLM model quantization (compression) toolkit with HW acceleration support for Nvidia, AMD, Intel GPU and Intel/AMD/Apple CPU via HF, vLLM, and SGLang. Maintained by ModelCloud on GitHub, where it has earned 1,222 stars from the community.
@ModelCloud2mo ago
AI & ML
Enterprise-grade API gateway that helps you monitor and impose cost or rate limits per API key. Get fine-grained access control and monitoring per user, application, or environment. Supports OpenAI, Azure OpenAI, Anthropic, vLLM, and open-source LLMs. Maintained by bricks-cloud on GitHub, where it has earned 1,221 stars from the community.
@bricks-cloud1y ago
AI & ML
Virtualized Elastic KV Cache for Dynamic GPU Sharing and Beyond. Maintained by ovg-project on GitHub, where it has earned 1,128 stars from the community.
@ovg-project2mo ago
AI & ML
Evaluate your LLM's response with Prometheus and GPT4. Maintained by prometheus-eval on GitHub, where it has earned 1,103 stars from the community.
@prometheus-eval1y ago
AI & ML
Build your own high performance LLM inference engine in C++ and CUDA - a smaller version of vLLM. Maintained by jmaczan on GitHub, where it has earned 994 stars from the community.
@jmaczan2mo ago
AI & ML
LLM notes, including model inference, transformer model structure, and llm framework code analysis notes. Maintained by harleyszhang on GitHub, where it has earned 887 stars from the community.
@harleyszhang2mo ago
DevTools
UniRL is a Framework for Unified Multimodal Model Reinforcement Learning. Maintained by Tencent-Hunyuan on GitHub, where it has earned 881 stars from the community.
@Tencent-Hunyuan2mo ago
AI & ML
Make Discord your LLM frontend - Supports any OpenAI compatible API (OpenRouter, Ollama and more). Maintained by jakobdylanc on GitHub, where it has earned 826 stars from the community.
@jakobdylanc3mo ago
AI & ML
A Python library powered by Language Models (LLMs) for conversational data discovery and analysis. Maintained by pgalko on GitHub, where it has earned 784 stars from the community.
@pgalko4mo ago
AI & ML
An local, offline (after initial setup), portable OCR software that can process images and PDF files, using DeepSeek-OCR-2 AI (running directly on your machine). Maintained by th1nhhdk on GitHub, where it has earned 778 stars from the community.
@th1nhhdk3mo ago
DevTools
[EMNLP 2024 & AAAI 2026] A powerful toolkit for compressing large models including LLMs, VLMs, and video generative models. Maintained by ModelTC on GitHub, where it has earned 738 stars from the community.
@ModelTC5mo ago
Docs
Multimodal RL training framework for diffusion & omni models. Maintained by verl-project on GitHub, where it has earned 728 stars from the community.
@verl-project2mo ago
AI & ML
Accurate, large-scale, and extensible simulator for LLM inference Systems. Maintained by microsoft on GitHub, where it has earned 654 stars from the community.
@microsoft1y ago
AI & ML
EfficientSAM3 compresses SAM3 into lightweight, edge-friendly models via progressive knowledge distillation for fast promptable concept segmentation and tracking. Maintained by SimonZeng7108 on GitHub, where it has earned 647 stars from the community.
@SimonZeng71083mo ago
AI & ML
Pure Rust + CUDA LLM inference engine — no PyTorch, OpenAI-compatible, serves Qwen3 to Kimi-K2. Maintained by openinfer-project on GitHub, where it has earned 626 stars from the community.
@openinfer-project2mo ago
AI & ML
AI Bank Statement Document Automation By LLM model and Personal Finanical Analysis. Maintained by johnsonhk88 on GitHub, where it has earned 616 stars from the community.
@johnsonhk882mo ago
DevTools
SparkTTS、OrpheusTTS,。. Maintained by HuiResearch on GitHub, where it has earned 609 stars from the community.
@HuiResearch1y ago
AI & ML
Fine-tune LLMs from one YAML. Layer streaming trains an 8B model on a 4 GB laptop GPU. Maintained by MakazhanAlpamys on GitHub, where it has earned 586 stars from the community.
@MakazhanAlpamys8w ago
AI & ML
[CVPR 2025] RoboBrain: A Unified Brain Model for Robotic Manipulation from Abstract to Concrete. Official Repository. Maintained by FlagOpen on GitHub, where it has earned 560 stars from the community.
@FlagOpen12mo ago
AI & ML
Crater is a cloud-native AI training & inference platform. Maintained by raids-lab on GitHub, where it has earned 547 stars from the community.
@raids-lab2mo ago
AI & ML
RESTai is an AIaaS (AI as a Service) open-source platform. Supports many public and local LLM suported by Ollama/vLLM/etc. Precise embeddings usage, tuning, analytics etc. Built-in image/audio generation with dynamic loading generators. Live chat deployment. Built-in block based graphical language. Prompt versioning and much more. Maintained by apocas on GitHub, where it has earned 512 stars from the community.
@apocas8w ago
DevTools
A command-line interface tool for serving LLM using vLLM. Maintained by Chen-zexi on GitHub, where it has earned 507 stars from the community.
@Chen-zexi8mo ago
AI & ML
A modern web interface for managing and interacting with vLLM servers (www.github.com/vllm-project/vllm). Supports both GPU and CPU modes, with special optimizations for macOS Apple Silicon and enterprise deployment on OpenShift/Kubernetes. Maintained by micytao on GitHub, where it has earned 506 stars from the community.
@micytao6mo ago
AI & ML
Open Model Engine (OME) — Kubernetes operator for LLM serving, GPU scheduling, and model lifecycle management. Works with SGLang, vLLM, TensorRT-LLM, and Triton. Maintained by ome-projects on GitHub, where it has earned 485 stars from the community.
@ome-projects2mo ago
AI & ML
Self-hosted AI agent OS. Your memory, chat, agents, and files stay on hardware you own, offline by default, cloud by choice. Offline AI memory (taOSmd), self-hosted multi-framework group chat, a full web desktop + app store, and auto-clustering across the consumer hardware you already have (Orange/Raspberry Pi, Mac mini, gaming PC). Maintained by jaylfc on GitHub, where it has earned 473 stars from the community.
@jaylfc2mo ago
AI & ML
The open-source, agent-first creative workspace. Maintained by nodetool-ai on GitHub, where it has earned 463 stars from the community.
@nodetool-ai8w ago
AI & ML
The Runpod worker template for serving our large language model endpoints. Powered by vLLM. Maintained by runpod-workers on GitHub, where it has earned 461 stars from the community.
@runpod-workers2mo ago
AI & ML
KVarN is a native vLLM KV-cache quantization backend for your agents: 3-5x more context, throughput above FP16, and FP16-level accuracy. Calibration-free, one flag. Maintained by huawei-csl on GitHub, where it has earned 452 stars from the community.
@huawei-csl3mo ago
AI & ML
Engine-agnostic LLM gateway in Rust. Full OpenAI & Anthropic API compatibility across vLLM, TRT-LLM, TokenSpeed, SGLang, OpenAI, Gemini & more. Industry-first gRPC pipeline, KV cache-aware routing, chat history, tokenization caching, Responses API, embeddings, WASM plugins, MCP, and multi-tenant auth. Maintained by smg-project on GitHub, where it has earned 442 stars from the community.
@smg-project2mo ago
AI & ML
Ready-to-use DeepSeek-OCR Web UI | Modern Interface | 7 Recognition Modes | Batch Processing | Real-time Logging | Fully Responsive. Maintained by neosun100 on GitHub, where it has earned 441 stars from the community.
@neosun1007mo ago
AI & ML
Fully uncensored, capability-enhanced abliteration of Qwen3.6-27B. NVFP4 + z-lab DFlash speculative decoding (n=12) on the unified ghcr.io/aeon-7/aeon-vllm-ultimate:latest container, tuned for long-context draft acceptance on DGX Spark. 6 HF variants (BF16/NVFP4/MTP/MTP-XS), docker-compose, and QuickStart. Maintained by AEON-7 on GitHub, where it has earned 434 stars from the community.
@AEON-73mo ago
AI & ML
sparkrun - launch, manage, and stop LLM inference workloads on NVIDIA DGX Spark systems. Maintained by spark-arena on GitHub, where it has earned 422 stars from the community.
@spark-arena2mo ago
AI & ML
TopicGPT: A Prompt-Based Framework for Topic Modeling [NAACL'24]. Maintained by chtmp223 on GitHub, where it has earned 413 stars from the community.
@chtmp2234mo ago

Ask MadeWithWhat

AI answers may contain mistakes — please double-check important details.