Foundation models have transformed perception and language, but true intelligence requires embodiment. Agents must not only interpret the world, but also predict dynamics, plan actions, and explore autonomously.
In this talk, I present a new paradigm for embodied foundation models built on three pillars: reasoning through simulation, learning through abstraction, and exploration through imagination. I will show how (1) physics simulators can be integrated into VLM reasoning, (2) transferable world models can be learned from 3D motion traces distilled from video, and (3) curiosity-driven exploration can be achieved via imagination and verification.
Together, these approaches enable agents to acquire physical intuition, adapt across embodiments, and learn from interaction—bringing us closer to general-purpose embodied intelligence.
Jia-Bin Huang is the Capital One Endowed Associate Professor in the Department of Computer Science at the University of Maryland, College Park (UMD). He received his B.S. degree from National Chiao-Tung University, and his M.S. and Ph.D. degrees in Electrical and Computer Engineering from the University of Illinois at Urbana-Champaign (UIUC). His research explores the intersection of computer vision, computer graphics, and machine learning, focusing on areas such as 3D reconstruction, generative modeling, world models, and robotics perception. He actively mentors graduate and undergraduate researchers, and his work is regularly published at top-tier venues including CVPR, NeurIPS, ICCV, and SIGGRAPH.