PhD Proposal: Understanding the Interplay of Fairness, Robustness, and Efficiency in Deep Learning
Vedant Nanda
Zoom: https://umd.zoom.us/j/5422705818
Thursday, September 21, 2023, 3:30-5:30 pm 
Abstract
Deep neural networks (DNNs) are increasingly used in real-world applications, including many high-stakes scenarios where they are used either as autonomous systems (eg facial recognition) or as decision aids (eg medical imaging). This has resulted in concerns about the reliability of these systems, for example, are the decisions made by these models fair for different demographics? Are these decisions reliable under (slight) perturbations to the inputs?
Concerns such as these have led to many exciting works within the very active sub-community of trustworthy machine learning (ML). This community includes many sub-areas, including fairness and robustness. While both fairness and robustness in ML have received plenty of attention as separate fields of inquiry, there is limited work connecting the two areas. Additionally, progress on fairness has also been more focused on classical ML models and there is little understanding of how findings from these models apply to more modern deep learning (DL) models, which represent a use case that's closer to the real world.
In the first part, I will present two works that bridge this gap: (1) We show how fairness constraints introduced in the classical ML setting do not transfer to modern DL models and highlight various additional challenges to applying fairness constraints in such settings, and (2) where we connect the subfields of robustness and fairness by introducing the notion of "robustness bias", that takes into account the disparity in the ease of misclassifying a particular subgroup using adversarial perturbations. These two works expand the scope of trustworthy ML to the context of modern DL models and also connect two subfields of fairness and robustness.
In the second part, I will focus on the nature of invariances of DL models' representations. In recent years, as models have grown bigger and are being trained on increasingly large amounts of data, the common trend is to use these representations and either finetune or use a linear probe for a downstream task. However, most robustness works in DL assume that a model is trained end-to-end on a given task and thus are not representative of how models are actually used. I will present two works that bridge this gap: (1) We look at robustness through the lens of invariances learned by DL models and provide a method to compare it to invariances learned by humans -- this provides a much more realistic measure of robustness of these models, and (2) where we provide a way to compare shared invariances between two different DL models, which provides a measure of "relative robustness", that provides a flexible tool to better understand DL models.
Finally, I will close by presenting some recent intriguing findings that show that even a small subset of neurons in the learned representation of DL models can transfer very well to downstream tasks, thus showing a degree of "diffused redundancy" in pretrained representations. While this presents an exciting opportunity for efficient transfer learning, we show that it can sometimes lead to fairness concerns where performance on certain subgroups hurts more than others. This works presents connections between efficiency and fairness in DL.
Concerns such as these have led to many exciting works within the very active sub-community of trustworthy machine learning (ML). This community includes many sub-areas, including fairness and robustness. While both fairness and robustness in ML have received plenty of attention as separate fields of inquiry, there is limited work connecting the two areas. Additionally, progress on fairness has also been more focused on classical ML models and there is little understanding of how findings from these models apply to more modern deep learning (DL) models, which represent a use case that's closer to the real world.
In the first part, I will present two works that bridge this gap: (1) We show how fairness constraints introduced in the classical ML setting do not transfer to modern DL models and highlight various additional challenges to applying fairness constraints in such settings, and (2) where we connect the subfields of robustness and fairness by introducing the notion of "robustness bias", that takes into account the disparity in the ease of misclassifying a particular subgroup using adversarial perturbations. These two works expand the scope of trustworthy ML to the context of modern DL models and also connect two subfields of fairness and robustness.
In the second part, I will focus on the nature of invariances of DL models' representations. In recent years, as models have grown bigger and are being trained on increasingly large amounts of data, the common trend is to use these representations and either finetune or use a linear probe for a downstream task. However, most robustness works in DL assume that a model is trained end-to-end on a given task and thus are not representative of how models are actually used. I will present two works that bridge this gap: (1) We look at robustness through the lens of invariances learned by DL models and provide a method to compare it to invariances learned by humans -- this provides a much more realistic measure of robustness of these models, and (2) where we provide a way to compare shared invariances between two different DL models, which provides a measure of "relative robustness", that provides a flexible tool to better understand DL models.
Finally, I will close by presenting some recent intriguing findings that show that even a small subset of neurons in the learned representation of DL models can transfer very well to downstream tasks, thus showing a degree of "diffused redundancy" in pretrained representations. While this presents an exciting opportunity for efficient transfer learning, we show that it can sometimes lead to fairness concerns where performance on certain subgroups hurts more than others. This works presents connections between efficiency and fairness in DL.
Examining Committee
Bio
Vedant is a fifth-year PhD student and is part of the Maryland Max Planck joint program. He is advised by Profs. John P. Dickerson (University of Maryland) and Krishna P. Gummadi (MPI-SWS). His research focuses on Trustworthy Deep Learning. Through his research he wishes to better understand the interplay between different facets of trust such as robustness, fairness, and efficiency in modern deep learning systems.
This talk is organized by Tom Hurst