OpenRLHF

An Easy-to-use, Scalable and High-performance Agentic RL Framework based on Ray (PPO & DAPO & REINFORCE++ & VLM & TIS & vLLM & Ray & Async RL). Maintained by OpenRLHF on GitHub, where it has earned 9,882 stars from the community.

It's actively developed around transformers, vllm, large-language-models, and is a solid reference for anyone building with these tools.