log in  |  register  |  feedback?  |  help  |  web accessibility
PhD Proposal: Towards Human-Level Audio Intelligence: Improving Audio Understanding with Large Audio-Language Models
Sakshi
Remote umd.zoom.us/my/dmanocha
Monday, November 2, 2026, 12:30-2:00 pm
  • You are subscribed to this talk through .
  • You are watching this talk through .
  • You are subscribed to this talk. (unsubscribe, watch)
  • You are watching this talk. (unwatch, subscribe)
  • You are not subscribed to this talk. (watch, subscribe)
Abstract

Audio, comprising speech, sounds, and music, carries rich information about the physical world and is central to human communication. Large Audio-Language Models (LALMs) now perform well on recognition tasks such as speech transcription, sound classification, and captioning. However, they still struggle to reason about how sounds relate to one another, to answer expert-level questions, and to interpret long, spatial, or audio-visual input. Our research addresses these gaps by enhancing audio perception and reasoning in LALMs through better audio representations, targeted training data, and rigorous evaluation.

In this talk, I will present our work on improving the audio understanding and complex reasoning of LALMs, together with an expert-level benchmark that reveals where current models fail. I will also discuss our recent progress in long-form, spatial, and audio-visual understanding, which is key to interpreting real-world scenes. Together, these efforts move LALMs from recognizing individual sounds toward reasoning over complex auditory scenes.

Bio

Sakshi is a PhD student in Computer Science at UMD, advised by Professor Dinesh Manocha and Professor Ramani Duraiswami. Her research focuses on improving audio understanding and reasoning in large audio-language models. Her publications have appeared at ICLR, ICML, EMNLP, NeurIPS, ACL, etc. She has interned at Dolby Laboratories and collaborated on projects with NVIDIA and Adobe.

This talk is organized by Migo Gui