New Robotics Models Enhance Control, Teleoperation, and Learning Capabilities with 12 Cans Nonstop

OpenVLA, a vision-language-action model for robotics, allows developers to control robots using natural language and images, enhancing customization in multi-task environments with multiple objects affordably. Open-TeleVision, a real-time teleoperation system, streams stereo vision and allows users to control robots with a VR headset across the United States, offering highly precise and smooth bimanual manipulation and active egocentric vision, demonstrated by inserting 12 cans nonstop. RoboPack, a framework integrating tactile-informed state estimation, dynamics prediction, and planning, improves robots' understanding of world dynamics through visual and tactile sensing for complex tasks like packing. EquiBot is a generalizable and data-efficient method for visuomotor policy learning, enabling robots to learn household tasks by watching human videos for just five minutes, robust to changes in object shapes, lighting, and scene makeup.