Researchers from the Allen Institute for AI and the University of Washington have introduced a new benchmark called SUPER. This benchmark is designed to evaluate the capabilities of large language models (LLMs) in autonomously setting up and executing research experiments from research repositories. The initiative emphasizes the importance of reproducibility in scientific research. The SUPER benchmark assesses the performance of various LLMs, including the SWE-agent, which has shown impressive results in these tasks.