Recent advancements in reinforcement learning from human feedback (RLHF) have introduced a new paradigm known as Asynchronous RLHF, which promises faster and more efficient off-policy reinforcement learning for language models. Researchers, led by mnoukhov, have demonstrated that this approach not only accelerates the training process but also maintains performance levels comparable to state-of-the-art (SOTA) methods, with improvements seen as the model scales. The release of code accompanying this research aims to facilitate broader adoption within the community, particularly for applications like decision process optimization (DPO) in language models. The initiative reflects a growing trend in the AI community to adapt traditional reinforcement learning techniques for language model training, enhancing both efficiency and effectiveness.