restai

RESTai is an AIaaS (AI as a Service) open-source platform. Supports many public and local LLM suported by Ollama/vLLM/etc. Precise embeddings usage, tuning, analytics etc. Built-in image/audio generation with dynamic loading generators. Live chat deployment. Built-in block based graphical language. Prompt versioning and much more. Maintained by apocas on GitHub, where it has earned 512 stars from the community.

It's actively developed around blocky, embeddings, fastapi, and is a solid reference for anyone building with these tools.