Talks

PhD Defense: A flexible approach for orchestrating adaptive scientific workflows for scalable computing

Swati Singhal

4137 Brendan Iribe Center for Computer Science and Engineering (IRB)

Wednesday, September 14, 2022, 1:00-3:00 pm

You are subscribed to this talk through .
You are watching this talk through .
You are subscribed to this talk. (unsubscribe, watch)
You are watching this talk. (unwatch, subscribe)
You are not subscribed to this talk. (watch, subscribe)

Abstract

Modern scientific workflows are becoming complex with the incorporation of non-traditional computation methods, and advances in technologies enabling on-the-fly analysis. These workflows exhibit unpredictable runtime behaviors and have dynamic requirements. For example, such workflows must maintain overall performance and throughput while dealing with undesired events, adapting to failures, and supporting data-driven adaptive analysis. A fixed, predetermined resource assignment common to HPC machines is inefficient for overall performance, throughput, and data-driven adaptive analysis. While solutions exist to enable elastic resource management, there is no support that can manage the workflows at runtime to determine when the resource assignment and/or the runtime state of tasks (i.e. stopping, starting, changing the task parameters for adapting analysis, or changing how data is sent/received by the workflow tasks) needs to be revised, and perform the feasible changes at runtime accordingly.

This dissertation provides a flexible and portable model, DYFLOW, with strategies to automate the management of scalable and adaptive workflows. The model gathers runtime statistics, tracks the occurrence of important events, and finalizes a plan of action to execute in response to events that occurred, by mediating between suggested actions with respect to the running state of the workflow tasks and resource availability. Further, the model supports a wide range of constructs and tunable parameters that allow users to express events of interest, select prospective responses, and set various preferences to set the service expectation, e.g., throughput, performance, resilience to failures, or quality of results. To showcase that the DYFLOW model supports adaptive functionality desired for emerging workflows, several examples of problematic behavior are demonstrated where DYFLOW accommodates the specific requirements and automates the runtime management process for scientists while delivering the quality of service desired.

Examining Committee:

Chair:
Dean's Representative:
Members:

Dr. Alan Sussman
Dr. Donald Yeung
Dr. Matthew D. Wolf (Oak Ridge National Lab)
Dr. Abhinav Bhatele
Dr. Howard Elman

Bio

Swati Singhal is a Ph.D. student in the Department of Computer Science at the University of Maryland, advised by Dr. Alan Sussman. Her research focuses on high performance computing and explores solutions that deal with the runtime challenges of state-of-the-art scientific workflows.

This talk is organized by Tom Hurst