New Methodology in Deep RL Shows Promising Scalability and Performance Improvements

Recent discussions among researchers highlight a significant shift in deep reinforcement learning (RL) methodologies. By framing regression tasks as classification problems, a method involving training value functions with categorical cross-entropy, improvements in performance and scalability of deep RL have been reported. This approach, argued to enhance the scalability of deep RL with little to no additional cost, has shown promise across various domains, including offline and online settings, and even in complex applications like robotics. The potential of this method to work robustly across models and tasks has been recognized as a potentially groundbreaking development in the field. However, the implications of this shift on the dynamic programming aspect of RL training and the knowledge within pre-trained models are still under exploration.