Robots acting in human-scale environments must plan under uncertainty in large state–action spaces and face constantly changing reward functions as requirements and goals change. We introduce a new hierarchical planning framework called Abstract Markov Decision Processes (AMDPs) that can plan in a fraction of the time needed for complex decision making in ordinary MDPs. AMDPs provide abstract states, actions, and transition dynamics in multiple layers above a base-level “flat” MDP. AMDPs decompose problems into a series of subtasks with both local reward and local transition functions used to create policies for subtasks. The resulting hierarchical planning method is independently optimal at each level of abstraction, and is recursively optimal when the local reward and transition functions are correct.
I will present empirical results in several domains showing significantly improved planning speed, while maintaining solution quality. I will also discuss related work within the same project on automated option discovery, abstraction construction, language learning, and initial steps towards automated methods for learning AMDPs from base MDPs, from teacher demonstrations, and from direct observations in the domain.
This work is collaborative research with Dr. Michael Littman and Dr. Stefanie Tellex of Brown University, and Dr. Smaranda Muresan of Columbia University. Dr. James MacGlashan of SIFT and Dr. Smaranda Muresan of Columbia University collaborated on earlier stages of the project. The following UMBC students have also contributed to the project: Tadewos Bellete, Michael Bishoff, Rose Carignan, Nick Haltemeyer, Nathaniel Lam, Shawn Squire, Tenji Tembo, Nicholay Topin, and John Winder.
Dr. Marie desJardins is a Professor of Computer Science and the Associate Dean for Academic Affairs in the College of Engineering and Information Technology at the University of Maryland, Baltimore County. Prior to joining the faculty at UMBC in 2001, she was a Senior Computer Scientist in the AI Center at SRI International. Her research is in artificial intelligence, focusing on the areas of machine learning, multi-agent systems, planning, interactive AI techniques, information management, reasoning with uncertainty, and decision theory. Current research projects include learning in the context of planning and decision making, intelligent tutoring systems, and analyzing and visualizing uncertainty in machine learning.
As a 2014-15 American Council of Education Fellow, Dr. desJardins spent a semester at Worcester Polytechnic Institute working with President Laurie Leshin studying academic leadership, institutional transformation, and data-driven decision making. She was the 2014-17 UMBC Presidential Teaching Professor and an inaugural Hrabowski Academic Innovation Fellow. She is active in the computer science education community, and leads the CS Matters in Maryland project to develop curriculum and train high school teachers to teach the new AP CS Principles course.
Dr. desJardins has published over 120 scientific papers in journals, conferences, and workshops. She was recently elected as a Member-at-Large-elect for Section T (Information, Computing, and Communication) of the American Association for the Advancement of Science. She is an Associate Editor of the Journal of Artificial Intelligence Research, is a member of the editorial board of AI Magazine, and was the Program Cochair for AAAI-13. She has previously served as AAAI Liaison to the Board of Directors of the Computing Research Association, Vice-Chair of ACM's SIGART, and AAAI Councillor. She is an ACM Distinguished Member, is a AAAI Senior Member, is a member and former chair of UMBC's Honors College Advisory Board, is the former chair of UMBC's Faculty Affairs Committee, and serves on the advisory board of UMBC's Center for Women in Technology.