Title: Discovering Latent Structure in Task-Oriented Dialogues
Authors: Ke Zhai, Jason D. Williams
Speaker: Ke Zhai (Oral presentation practice at ACL 2014)
Time: 30 minutes

Abstract: A key challenge for computational conver- sation models is to discover latent structure in task-oriented dialogue, since it provides a basis for analysing, evaluating, and building conversational systems. We propose three new unsupervised models to discover latent structures in task-oriented dialogues. We apply them to two real, non-trivial datasets: human-computer spoken dialogues in bus query service, and human-human text-based chats from a live technical support service. We show that our models extract meaningful state representations and dialogue structures consistent with human annotations. Quantitatively, we show our models achieve superior performance on held-out log likelihood evaluation and an ordering task.

Title: Assessing the Reliability and Reusability of an E-Discovery Privilege Test Collection
Authors: Jyothi Vinjumur, Douglas Oard, Jiaul Paik
Speaker: Jyothi Keshavan Vinjumur (Poster presentation practice at SIGIR 2014)
Time: 15 minutes

Abstract: In some jurisdictions, parties to a lawsuit can request documents from each other, but documents subject to a claim of privilege may be withheld. The TREC 2010 Legal Track developed what is presently the only public test collection for evaluating privilege classification. This paper examines the usability and reusability of that collection. For reliability, the key question is the extent to which relevance judgments correctly reflect the opinion of the senior litigator whose judgment is authoritative. For reusability, the key question is the degree to which systems whose results contributed to creation of the test collection can be fairly compared with other systems that use those relevance judgments as in the future. These correspond to measurement error and sampling error, respectively. The results indicate that measurement error is the larger problem.

Title: Political Ideology Detection Using Recursive Neural Networks
Authors: Mohit Iyyer, Peter Enns, Jordan Boyd-Graber, Philip Resnik
Speaker: Mohit Iyyer (Poster presentation practice at ACL 2014)
Time: 15 minutes

Abstract: An individual's words often reveal their political ideology. Existing automated techniques to identify ideology from text focus on bags of words or wordlists, ignoring syntax. Taking inspiration from recent work in sentiment analysis that successfully models the compositional aspect of language, we apply a recursive neural network (RNN) framework to the task of identifying the political position evinced by a sentence. To show the importance of modeling subsentential elements, we crowdsource political annotations at a phrase and sentence level. Our model outperforms existing models on our newly annotated dataset and an existing dataset.