Eric Jang discusses the challenges of evaluating humanoid AI before home deployment. Unlike self-driving models that can be tested on public roads with QA drivers or user trips, robots in private settings require a different approach. A 'World model' is needed to simulate robot actions. 1X's World Model acts as a virtual simulator, predicting how robots will interact with their environment, simulating a range of tasks and complex object interactions without manual setup. This approach involves learning a simulator directly from raw sensor data to evaluate policies across millions of scenarios. The generative interactive simulation by 1X addresses the issue of policy network degradation over time due to subtle environmental changes, providing a scalable solution for reproducible evaluations in multi-agent settings. Multi-day evaluations in customer homes are impractical to confirm model reliability and safety for hundreds of tasks.
Robot policy networks may experience degradation over time due to subtle environmental changes You can't do multi-day evaluations in customer homes to confirm model reliability and safety for hundreds of tasks 1X's generative interactive simulation (world model) addresses this https://t.co/gVdUzwmZDT https://t.co/3Infrg5hMB
Promising progress from 1X on learned world models which improve with more experience and physical interaction data. What I'm excited about: - World models are likely the only path forward for reproducible and scalable evaluations in *multi-agent settings*; see success of world… https://t.co/Do4snLjlju
A world model for robotic. Sounds very promising: „We’re taking a radically new approach to evaluation of general-purpose robots: learning a simulator directly from raw sensor data and using it to evaluate our policies across millions of scenarios. By learning a simulator… https://t.co/pnTfGmDAuB