PhD Defense: Acting, Planning, and Learning Using Hierarchical Operational Models
Sunandita Patra
Remote
Abstract
The most common representation formalisms for planning are descriptive models that abstractly describe what the actions do and are tailored for e fficiently computing the next state(s) in a state-transition system. However, real-world acting requires operational models that describe how to do things, with rich control structures for closed-loop online decision-making in a dynamic environment. Use of a different action model for planning than the one used for acting causes problems with combining acting and planning, in particular for the development and consistency verification of the different models. As an alternative, this dissertation defines and implements an integrated acting-and-planning system in which both planning and acting use the same operational models, which are written in a general-purpose hierarchical task-oriented language offering rich control structures. The acting component, called Reactive Acting Engine (RAE), is inspired by the well-known PRS system, except that instead of being purely reactive, it can get advice from the planner.
The dissertation also describes three planning algorithms which plan by doing several Monte Carlo rollouts in the space of operational models. The best of these three planners, Plan-with-UPOM uses a UCT-like Monte Carlo Tree Search procedure called UPOM (UCT Procedure for Operational Models), whose rollouts are simulated executions of the actor's operational models. The dissertation also presents learning strategies for use with RAE and UPOM that acquire from online acting experiences and/or simulated planning results, a mapping from decision contexts to method instances as well as a heuristic function to guide UPOM. The experimental results show that Plan-with-UPOM and the learning strategies significantly improve the acting e fficiency and robustness of RAE. It can be proved that UPOM converges asymptotically by mapping its search space to an MDP. The dissertation also describes a real-world prototype of RAE and UPOM to defend software-defined networks, a relatively new network management architecture, against incoming attacks.
Examining Committee:
The dissertation also describes three planning algorithms which plan by doing several Monte Carlo rollouts in the space of operational models. The best of these three planners, Plan-with-UPOM uses a UCT-like Monte Carlo Tree Search procedure called UPOM (UCT Procedure for Operational Models), whose rollouts are simulated executions of the actor's operational models. The dissertation also presents learning strategies for use with RAE and UPOM that acquire from online acting experiences and/or simulated planning results, a mapping from decision contexts to method instances as well as a heuristic function to guide UPOM. The experimental results show that Plan-with-UPOM and the learning strategies significantly improve the acting e fficiency and robustness of RAE. It can be proved that UPOM converges asymptotically by mapping its search space to an MDP. The dissertation also describes a real-world prototype of RAE and UPOM to defend software-defined networks, a relatively new network management architecture, against incoming attacks.
Examining Committee:
Chair: Dr. Dana Nau
Dean's rep: Dr. Pamela Abshire
Members: Dr. Jordan Boyd-Graber
Dean's rep: Dr. Pamela Abshire
Members: Dr. Jordan Boyd-Graber
Dr. Tom Goldstein
Dr. Malik Ghallab
Paolo Traverso
Dr. Malik Ghallab
Paolo Traverso
Bio
Sunandita Patra is a PhD candidate at the Department of Computer Science at the University of Maryland, College Park. Her research addresses the challenges faced by hierarchically organized actors (that do continual online planning and acting) in dynamic environments. She has developed integrated algorithms that incorporate all three: acting, planning, and learning using a representation of hierarchical operational models. Prior to this, she worked at Microsoft India Development center at Hyderabad. She did her B. Tech and M. Tech in Computer Science and Engineering from IIT Kharagpur and graduated in the year 2014. For more information, please visit: https://sunanditapatra.
This talk is organized by Tom Hurst