Statistical methods have revolutionized natural language processing (NLP) and have led to advances in applications such as automatic translation, question answering, and structured search. Fueled by access to large amounts of textual data, researchers have develope increasingly detailed models of language to improve accuracy on these tasks. However, these improvements often come at the cost of vastly increasing the search-space of the underlying prediction problems. Solving these prediction problems optimally is very challenging, leading most large-scale NLP systems to rely heavily on heuristic search algorithms.

This talk presents methods for finding provably optimal predictions for important models in natural language processing. I will focus on two core NLP tasks, syntactic parsing and machine translation. For both of these problems I will show how Lagrangian relaxation, a classical method from combinatorial optimization, can be used to decompose challenging prediction problems into problems that are solvable with efficient combinatorial algorithms. Experiments show that the methods find optimal solutions for the vast majority of examples, with  comparable efficiency to commonly-used heuristic algorithms.