log in  |  register  |  feedback?  |  help  |  web accessibility
On Building General-Purpose Home Robots
Lerrel Pinto - Assistant Professor of Computer Science at NYU
IRB 4105 or https://umd.zoom.us/j/95527564297?pwd=VKvEJkTzBK7TvzXkRsDz1dmbrl7GvB.1
Monday, September 16, 2024, 1:30-2:30 pm
  • You are subscribed to this talk through .
  • You are watching this talk through .
  • You are subscribed to this talk. (unsubscribe, watch)
  • You are watching this talk. (unwatch, subscribe)
  • You are not subscribed to this talk. (watch, subscribe)
Abstract

The concept of a "generalist machine" in homes — a domestic assistant that can adapt and learn from our needs, all while remaining cost-effective — has long been a goal in robotics that has been steadily pursued for decades. In this talk, I will present our recent efforts towards building such capable home robots. First, I will discuss how large, pretrained vision-language models can induce strong priors for mobile manipulation tasks like pick-and-drop. But pretrained models can only take us so far. To scale beyond basic picking, we will need systems and algorithms to rapidly learn new skills. This requires creating new tools to collect data, improving representations of the visual world, and enabling trial-and-error learning during deployment. While much of the work presented focuses on two-fingered hands, I will briefly introduce learning approaches for multi-fingered hands which support more dexterous behaviors and rich touch sensing combined with vision. Finally, I will outline unsolved problems that were not obvious initially, which, when solved, will bring us closer to general-purpose home robots.

Bio
Lerrel Pinto is an Assistant Professor of Computer Science at NYU. His research focuses on machine learning for robots. He received a Ph.D. degree from CMU after which he did a Postdoc at UC Berkeley. His research on robot learning has received best paper awards at ICRA 2016, RSS 2023, ICRA 2024 and finalist at IROS 2019, and CoRL 2022. Lerrel has received the Packard Fellowship and was named a TR35 innovator under 35 in 2023. Several of his works have been featured in popular media such as The Wall Street Journal, TechCrunch, MIT Tech Review, Wired, and BuzzFeed among others. His recent work can be found on www.lerrelpinto.com.
 
Meetings: Please contact Abhinav Shrivastava (abhinav@cs.umd.edu) if you'd like to meet Lerrel on Monday. 
This talk is organized by Samuel Malede Zewdu