Talks

PhD Defense: Towards Multimodal and Context-Aware Emotion Perception

Trisha Mittal

4105 http://umd.zoom.us/my/dmanocha Brendan Iribe Center for Computer Science and Engineering (IRB)

Thursday, April 27, 2023, 11:00 am-1:00 pm

You are subscribed to this talk through .
You are watching this talk through .
You are subscribed to this talk. (unsubscribe, watch)
You are watching this talk. (unwatch, subscribe)
You are not subscribed to this talk. (watch, subscribe)

Abstract

Human emotion perception is an integral component of intelligent systems' wide range of applications, including behavior prediction, social robotics, medicine, surveillance, and entertainment. Current literature advocates that humans perceive emotions and behavior from various human modalities and also the situational and background context. Our research focuses on this aspect of emotion perception, as we attempt to build emotion perception models from multiple modalities and contextual cues, as well as use such ideas of perception for various real-world domains of AI applications. We will go over both parts in this talk. In the first part, we will go through two approaches for better and improved emotion perception models. In one approach, we will leverage the idea of using more than one modality to perceive human emotion. In the other approach, we leverage contextual information; background scene, multiple modalities of the human subject, and socio-dynamic inter-agent interactions available in the input to predict the perceived emotion. In the second part, we will explore three domains of AI applications; i) video manipulations and deepfake detection, ii) multimedia content analysis, and iii) social media interactions investigation to enrich solutions to them with ideas from emotion perception.

Examining Committee

Chair:	Dr. Dinesh Manocha
Dean's Representative:	Dr. Min Wu
Members:	Dr. Ramani Duraiswami
	Dr. Aniket Bera
	Dr. Viswanathan Swaminathan (Adobe Research)

Bio

Trisha Mittal is a Ph.D. candidate in the GAMMA Lab at the University of Maryland, College Park advised by Prof. Dinesh Manocha. Trisha received her master's degree in Computer Science from the University of Maryland in 2020, and her bachelor's and master's degrees in Information Technology from the International Institute of Information Technology, Bangalore in 2018. Trisha's research is focused on affective computing, which involves building systems capable of understanding, interpreting, and responding to human emotions. Her work centers around developing improved perception models for human emotion using ideas of multimodal learning and context-aware learning. Additionally, Trisha applies ideas of emotion and behavior perception to various real-world applications, such as multimedia analysis, manipulated video detection, and social media analysis. Her research has been published at top computer vision, artificial intelligence, and multimedia conferences. Trisha has completed internships at Adobe Research in San Jose and Apple MLR in Cupertino. She was awarded the Adobe Research Fellowship for 2022 and was selected for the Outstanding Research Assistant Award for AY 2021-22 by The Graduate School. Trisha was also chosen and invited to participate in the Rising Stars 2021 EECS Workshop and the CRA Grad Cohort for Women 2021. In her free time, Trisha enjoys teaching and mentoring younger students through initiatives like AI4ALL and GirlsWhoCode, and she is a contributing writer for Skynet Today.

This talk is organized by Tom Hurst