Large language models (LLMs) have utterly transformed the field of natural language processing. However, training LLMs comes at a massive financial and environmental cost, making them out of reach of academic research labs. Meanwhile, these models are costly to update and remain opaque in their reasoning process. In this talk, I will discuss two recent works from my group and share some thoughts on how we could move forward in the age of LLMs. The first half of the talk will introduce a new training approach for language modeling called TRIME, which is designed for training LMs with memory augmentation. Our approach can enable language models to better leverage long-range contexts or retrieve external knowledge at testing time, with negligible compute overhead compared to standard LM training. I will argue that augmenting language models with a large retrieval component can greatly ease the computational and privacy concerns in current LLMs. In the second part of the talk, I will introduce a new framework called NLProofS for deductive reasoning in natural language. I will argue that current LLMs still struggle with generating complex proofs in a single shot, while our step-by-step approach with an independently-trained verifier can generate both valid and relevant steps and achieve much stronger performance.
Danqi Chen is an Assistant Professor of Computer Science at Princeton University and co-leads the Princeton NLP Group. Her recent research focuses on training, adapting, and understanding large language models, and developing scalable and generalizable NLP systems for question answering, information extraction and conversational agents. Before joining Princeton, Danqi worked as a visiting scientist at Facebook AI Research. She received her Ph.D. from Stanford University (2018) and B.E. from Tsinghua University (2012), both in Computer Science. Danqi is a recipient of a 2022 Sloan Fellowship, research awards from Google, Meta, Amazon, Adobe, Apple and Salesforce, and outstanding paper awards from ACL 2016, EMNLP 2017 and ACL 2022.