The Text REtrieval Conference (TREC), now in its 27th year, is quite possibly the largest coordinated effort ever created to understand how to improve the state of the art in artificial intelligence. Focused on the field of information retrieval (the technology central to search engines), TREC builds datasets that allow researchers to measure the effectiveness of their search systems. In order to build those datasets, we need to understand how those systems will be used to support people in their everyday tasks. Then, we design metrics and annotate data to support measuring those systems. Dr. Ian Soboroff has been involved with TREC for 17 years and will discuss how to measure IR and AI performance, how to build datasets to produce those measurements, and information on the TREC conference series.
Dr. Ian Soboroff is a computer scientist and leader of the Retrieval Group at the National Institute of Standards and Technology (NIST). The Retrieval Group organizes the Text REtrieval Conference (TREC), the Text Analysis Conference (TAC), and the TREC Video Retrieval Evaluation (TRECVID). These are all large, community-based research workshops that drive the state-of-the-art in information retrieval, video search, web search, information extraction, text summarization and other areas of information access. He has co-authored many publications in information retrieval evaluation, test collection building, text filtering, collaborative filtering, and intelligent software agents. His current research interests include building test collections for social media environments and nontraditional retrieval tasks.