log in  |  register  |  feedback?  |  help  |  web accessibility
Developing Performance Benchmarks for Interactive Analysis Systems
Friday, November 15, 2019, 11:00 am-12:00 pm Calendar
  • You are subscribed to this talk through .
  • You are watching this talk through .
  • You are subscribed to this talk. (unsubscribe, watch)
  • You are watching this talk. (unwatch, subscribe)
  • You are not subscribed to this talk. (watch, subscribe)
Data exploration requires fast queries, especially when coupled with interactive visualization systems. Modern analytics database systems promise to respond quickly to queries even over large data. However, we still lack adequate infrastructure to empirically assess which of these systems provide satisfactory performance. For example, the standard database approach is to run a query benchmark (e.g., TPC-H), and compare the results for each DBMS (e.g., average response time or latency). However, DBMS benchmarks are designed for data warehousing and transactional processing, which bear little resemblance to interactive environments. In contrast, visualization benchmarks, like the Visual Analytics Benchmark Repository [6], focus on human perception and productivity (e.g., how accurately a user can analyze data), but provide limited support for performance measures or comparison of analytical operations (i.e., queries).
We propose that a new benchmark should combine the best of both worlds by blending methodology from the database and the visualization communities. We aim to develop a new benchmark that allows for systematic and repeatable measurement of large-scale, interactive analytics systems. Our benchmark is inspired by the different ways that the database and visualization communities evaluate systems. In this talk, we will present ongoing work to develop a new benchmark to validate the suitability of database systems for some of the most demanding needs of interactive visualization systems: crossfiltering. Our benchmark is meant to allow database designers to check if their system is compatible with low-latency interactions, but also to develop strategies to exploit the particular patterns and pacing of human interactions for further optimization.

Leilani Battle is an Assistant Professor at the University of Maryland, College Park, with a joint appointment in the University of Maryland Institute for Advanced Computer Studies (UMIACS). She is also affiliated with the UMD Human-Computer Interaction Laboratory (HCIL). Her research interests focus on developing interactive data-intensive systems that can aid analysts in performing complex data exploration and analysis. Her current research is anchored in the field of databases, but utilizes research methodology and techniques from HCI and visualization to integrate data processing (databases) with interactive interfaces (HCI, visualization). She is an NSF Graduate Research Fellowship Recipient (2012), and her research is currently supported by an NSF CISE CRII Award (2019-2021) and ORAU Ralph E. Powe Junior Faculty Enhancement Award (2019-2020). In 2017, she completed a postdoc in the UW Interactive Data Lab. She holds an MS (2013) and PhD (2017) in Computer Science from MIT, where she was a member of the MIT Database Group, and a BS in Computer Engineering from UW (2011), where she was a member of the UW database group.

This talk is organized by Ramani Duraiswami