DeepMind has introduced a new approach to language model training using Scalable Inverse Reinforcement Learning (IRL). This method presents an effective alternative to traditional supervised Maximum Likelihood Estimation (MLE) in the fine-tuning pipeline, resulting in more robust reward functions and increased performance and diversity of model generations. The foundation of this approach lies in imitation learning, which is considered a reinforcement learning problem. Compared to supervised learning, IRL better exploits sequential structure, online data, and further extracts rewards. The insights were shared in a recent paper by GoogleDeepMind.