Batch_LLM_Inference_with_Ray_Data_LLM by 0-mostafa-rezaee-0

Dockerized LLM inference server with constrained output (JSON mode), built on top of vLLM and outlines. Faster, cheaper and without rate limits. Compare the quality and latency to your current LLM API provider. Maintained by phospho-app on GitHub, where it has earned 27 stars from the community.

Batch Deployment for Document Parsing with AWS Batch & Qwen-2.5-VL. Maintained by jeremyarancio on GitHub, where it has earned 48 stars from the community.

LLM, Fine Tuning, Llama 2, Gemma, Mixtral, vLLM, LangChain, RAG, ChromaDB, FAISS. Maintained by joydeb28 on GitHub, where it has earned 13 stars from the community.