Talks

PhD Proposal: Towards Multimodal and Context-Aware Emotion Perception

Trisha Mittal

3137 Brendan Iribe Center for Computer Science and Engineering (IRB)

Friday, April 29, 2022, 12:00-2:00 pm

You are subscribed to this talk through .
You are watching this talk through .
You are subscribed to this talk. (unsubscribe, watch)
You are watching this talk. (unwatch, subscribe)
You are not subscribed to this talk. (watch, subscribe)

Abstract

Human emotion perception is an integral component of intelligent systems' wide range of applications, including behavior prediction, social robotics, medicine, surveillance, and entertainment. Current literature advocates that humans perceive emotions and behavior from various human modalities and also the situational and background context. Our research focuses on this aspect of emotion perception, as we attempt to build emotion perception models from multiple modalities and contextual cues, as well as use such ideas of perception for various real-world domains of AI applications. We will go over both parts in this talk. In the first part, we will go through two approaches for better and improved emotion perception models. In one approach, we will leverage the idea of using more than one modality to perceive human emotion. In the other approach, we leverage contextual information; background scene, multiple modalities of the human subject and socio-dynamic inter-agent interactions available in the input to predict the perceived emotion. In the second part, we will explore three domains of AI applications; i) video manipulations and deepfake detection, ii) multimedia content analysis, and iii) social media interactions investigation to enrich solutions to them with ideas from emotion perception.

Examining Committee:

Chair:
Department Representative:

Dr. Dinesh Manocha
Dr. Ramani Duraiswami
Dr. Vanessa Frias-Martinez
Dr. Viswanathan Swaminathan
Dr. Aniket Bera

Bio

Trisha Mittal is a fourth-year Ph.D. student in the GAMMA Lab at the University of Maryland, College Park advised by Prof. Dinesh Manocha. She received a master’s degree in Computer Science from the University of Maryland in 2020 and a bachelor’s and master’s degree in Information Technology from the International Institute of Information Technology, Bangalore in 2018. Her research focuses on affective computing which involves building systems that can understand interpret and respond to human emotions. Her work includes developing improved perception models for human emotion using ideas of multimodal learning and context-aware learning. Her work also involves applying ideas of emotion and behavior perception for various real-word applications like multimedia analysis, manipulated videos detection, and social media analysis. She has published her work in top computer vision, artificial intelligence and multimedia conferences and has interned in Adobe Research, San Jose. She was awarded the Adobe Research Fellowship for 2022 and was selected for the Outstanding Research Assistant Award for AY 2021-22 by The Graduate School. She was also selected and invited to participate in the Rising Stars 2021 EECS Workshop and the CRA Grad Cohort for Women 2021. Outside of her research, she enjoys teaching and mentoring younger students through initiatives like AI4ALL and GirlsWhoCode and is a contributing writer for Skynet Today.

This talk is organized by Tom Hurst