Haozhi Qi (UC Berkeley)- Multisensory Dexterity for Robotics

Abstract: Human hands are essential for sensing and interacting with the physical world, allowing us to grasp and manipulate objects with ease. Replicating this dexterity in robots is the key to unlocking general-purpose robotics in unstructured environments. While modern AI has achieved breakthroughs in many domains, robot dexterity remains an unsolved challenge due to the complexity of high-dimensional control, limited real-world data, and the need for rich multisensory feedback. In this talk, I will present my work on multisensory dexterity for robotics and demonstrate how robots can achieve a broad range of dexterous manipulation capabilities. First, I will introduce how robots develop dexterous manipulation using simple sensory inputs and identify the key ingredients that enable generalizable manipulation across diverse objects, with examples in in-hand and bimanual manipulation. Building on these ingredients, I will then show how integrating rich multisensory feedback—including proprioception, vision, and tactile sensing—improves both perception and control, allowing robots to perform tasks that would be impossible with simple sensors. Finally, I will conclude with future opportunities and open challenges in scaling robotic dexterity and developing robots capable of general-purpose physical interaction.