Question answering is one of the most important and challenging tasks for understanding human language. Recently, using raw text as the knowledge source for question answering receives significant attention. Given an input question and a passage (pre-given or retrieved), neural models have high answer accuracy on the benchmark datasets. Nevertheless, recent studies indicate that the questions in the current benchmarks are mostly answerable by single evidence sentence, so neural models do not need strong language understanding capability to have high numbers on those simple questions. This proposal takes a step further to study the multi-evidence question answering problem, in which multiple evidence sentences are necessary to find the answer. We argue that answering multi-evidence questions requires more language understanding, and introduce systems to answer such multi-evidence questions.

One real application for multi-evidence question answering is trivia competitions such as Quizbowl and Jeopardy!, in which complex multi-evidence questions are posted to evaluate human knowledge. Therefore, we first build a dataset QBLink from Quizbowl competitions with around 56000 human authored entity centric questions containing multiple parallel clues. Then we introduce a question answering system DELFT that inherits the structural reasoning ability of knowledge graph question answering approaches with the broader coverage of free-text. DELFT first extracts multiple free-text evidence and constructs a semi-structured evidence graph (i.e., it has the graph structure, but the relations are not pre-defined). Then a graph neural network combines evidence to select an answer. DELFT outperforms both knowledge graph question answering and reading comprehension approaches.

Next we introduce Transformer-XH that focuses on generally modeling semi-structured text. Transformer-XH introduces an extra hop attention in its layers that aggregates representations from different textual pieces following their structure. Transformer-XH outperforms state-of-the-art systems and is generalizable in multiple multi-evidence reasoning tasks.

In the proposed work, we plan to build a unified graph based framework to different types of questions (e.g., single evidence, multiple evidences). This unified system will benefit from the larger training data combined from each task. Under this framework, we propose two work on free-text knowledge graph relation learning and question decomposition.

Examining Committee: 

 

                          Chair:               Dr. Jordan Boyd-Graber   
                          Dept rep:         Dr.  David Jacobs
                          Members:        Dr. Hal Daumé III