log in  |  register  |  feedback?  |  help  |  web accessibility
PhD Defense: Acting, Planning, and Learning Using Hierarchical Operational Models
Sunandita Patra
Friday, July 10, 2020, 9:00-11:00 am Calendar
  • You are subscribed to this talk through .
  • You are watching this talk through .
  • You are subscribed to this talk. (unsubscribe, watch)
  • You are watching this talk. (unwatch, subscribe)
  • You are not subscribed to this talk. (watch, subscribe)
The most common representation formalisms for planning are descriptive models that abstractly describe what the actions do and are tailored for e fficiently computing the next state(s) in a state-transition system. However, real-world acting requires operational models that describe how to do things, with rich control structures for closed-loop online decision-making in a dynamic environment. Use of a different action model for planning than the one used for acting causes problems with combining acting and planning, in particular for the development and consistency verification of the different models. As an alternative, this dissertation defines and implements an integrated acting-and-planning system in which both planning and acting use the same operational models, which are written in a general-purpose hierarchical task-oriented language offering rich control structures. The acting component, called Reactive Acting Engine (RAE), is inspired by the well-known PRS system, except that instead of being purely reactive, it can get advice from the planner.

The dissertation also describes three planning algorithms which plan by doing several Monte Carlo rollouts in the space of operational models. The best of these three planners, Plan-with-UPOM uses a UCT-like Monte Carlo Tree Search procedure called UPOM (UCT Procedure for Operational Models), whose rollouts are simulated executions of the actor's operational models. The dissertation also presents learning strategies for use with RAE and UPOM that acquire from online acting experiences and/or simulated planning results, a mapping from decision contexts to method instances as well as a heuristic function to guide UPOM. The experimental results show that Plan-with-UPOM and the learning strategies significantly improve the acting e fficiency and robustness of RAE. It can be proved that UPOM converges asymptotically by mapping its search space to an MDP. The dissertation also describes a real-world prototype of RAE and UPOM to defend software-defined networks, a relatively new network management architecture, against incoming attacks.

Examining Committee: 
                           Chair:              Dr. Dana Nau
                           Dean's rep:      Dr.  Pamela Abshire
                          Members:         Dr.  Jordan Boyd-Graber
                                                Dr.  Tom Goldstein
                                                Dr. Malik Ghallab
                                                Paolo Traverso

Sunandita Patra is a PhD candidate at the Department of Computer Science at the University of Maryland, College Park. Her research addresses the challenges faced by hierarchically organized actors (that do continual online planning and acting) in dynamic environments. She has developed integrated algorithms that incorporate all three: acting, planning, and learning using a representation of hierarchical operational models. Prior to this, she worked at Microsoft India Development center at Hyderabad. She did her B. Tech and M. Tech in Computer Science and Engineering from IIT Kharagpur and graduated in the year 2014. For more information, please visit: https://sunanditapatra.wixsite.com/camp

This talk is organized by Tom Hurst