Audio, comprising speech, sounds, and music, carries rich information about the physical world and is central to human communication. Large Audio-Language Models (LALMs) now perform well on recognition tasks such as speech transcription, sound classification, and captioning. However, they still struggle to reason about how sounds relate to one another, to answer expert-level questions, and to interpret long, spatial, or audio-visual input. Our research addresses these gaps by enhancing audio perception and reasoning in LALMs through better audio representations, targeted training data, and rigorous evaluation.
In this talk, I will present our work on improving the audio understanding and complex reasoning of LALMs, together with an expert-level benchmark that reveals where current models fail. I will also discuss our recent progress in long-form, spatial, and audio-visual understanding, which is key to interpreting real-world scenes. Together, these efforts move LALMs from recognizing individual sounds toward reasoning over complex auditory scenes.
Sakshi is a PhD student in Computer Science at UMD, advised by Professor Dinesh Manocha and Professor Ramani Duraiswami. Her research focuses on improving audio understanding and reasoning in large audio-language models. Her publications have appeared at ICLR, ICML, EMNLP, NeurIPS, ACL, etc. She has interned at Dolby Laboratories and collaborated on projects with NVIDIA and Adobe.

