DeepSeek has announced a new framework for enhancing large language model (LLM) reward models, focusing on inference-time scaling. The framework, named GRM-27B, reportedly outperforms the existing GPT-4o in reward modeling. This advancement is based on a novel approach called Self-Principled Critique Tuning (SPCT), which aims to improve the quality and scalability of generalist reward models (GRMs) without introducing significant biases. The new model is set to be open-sourced soon, and it has gained traction on alphaXiv, a platform for research dissemination. The research highlights the importance of inference compute in enhancing reward generation behaviors in LLMs, suggesting that improvements in inference-time scaling could lead to better overall performance in AI systems.