log in  |  register  |  feedback?  |  help  |  web accessibility
PhD Proposal: Embodied Action Understanding
Eadom Dessalene
IRB-4107 https://umd.zoom.us/j/8756553504?pwd=Q01uai83UkhnVVBOQXhXSVRwNERHdz09
Tuesday, November 4, 2025, 10:00-11:30 am
  • You are subscribed to this talk through .
  • You are watching this talk through .
  • You are subscribed to this talk. (unsubscribe, watch)
  • You are watching this talk. (unwatch, subscribe)
  • You are not subscribed to this talk. (watch, subscribe)
Abstract

There is consensus in neuroscience that object understanding arises from hierarchical visual processes, inspiring most of computer vision’s advances in image perception. However, action understanding - how humans perceive and predict the behavior of others - is an embodied process. Mounting evidence from cognitive science shows that observing others’ actions also engages our motor systems, suggesting that understanding action involves both seeing and simulating movement. We aim to create a framework inspired by the rich interplay between perceptual processes and motoric processes in the brain, based on two prior works of ours - LEAP and EmbodiSwap. LEAP uses LLMs to translate egocentric human videos into structured, symbolic action programs. EmbodiSwap composites robot embodiments into human videos for zero-shot robot imitation learning. Combining these works, we propose a framework in which visual processes that extract programs of action from human video benefit from embodied interaction enabled through the deployment of visuomotor policies. Through this framework, we aim to demonstrate that a real-world robot can actively refine its visual representations through action.

 

Bio

Eadom Dessalene is a PhD student in Computer Science at the University of Maryland, College Park, advised by Prof. Yiannis Aloimonos. His research focuses on the interplay between the understanding of action from egocentric videos, and robotics. He has published at venues such as PAMI, CVPR, ICLR, WACV and ICRA.

This talk is organized by Migo Gui