PhD Defense: A Multi-Faceted Approach for Evaluating Visualization Recommendation Algorithms
Zehua Zeng
Monday, October 24, 2022, 10:00 am-12:00 pm 
Abstract
Data visualizations allow analysts to quickly understand data trends, outliers, and patterns. However, designing the "best" visualizations for a given dataset is complicated. Multiple factors need to be considered, such as the data size, data types, target analysis tasks being supported, and even how the visualization needs to be personalized to the audience. In response, many visualization recommendation algorithms are being proposed to reduce user effort by automatically making some or all of these design decisions for analysts. However, existing visualization recommendation algorithms are evaluated in isolation, or the comparisons do not measure user performance. In other words, existing algorithms are not tested in a way that aligns with how they are used in practice. The lack of evaluation approaches makes it impossible to know how functional an algorithm is compared to another across various analysis tasks, hindering our ability to design new algorithms that provide significantly more benefits than the existing ones.
This dissertation contributes a multi-faceted approach for evaluating visualization recommendation algorithms to investigate factors affecting an algorithm's performance and ways to improve it. It first proposes an evaluation-focused framework and then demonstrates how the framework can evaluate strategic behaviors and user performance among a broad range of existing algorithms. The case study results show that newly proposed algorithms might not significantly improve user performance. One way to improve the algorithm performance is by integrating more established theoretical rules or empirical results on how people perceive different visualization designs, i.e., graphical perception knowledge, to guide the recommendation ranking process. Thus, this dissertation next presents a thorough literature review of existing graphical perception knowledge that can inform visualization recommendation algorithms. It contributes a knowledge base that collates existing theoretical and experimental visualization comparison results and summarizes key study outcomes. Further, this dissertation conducts an exploratory analysis to investigate the influence of each piece of graphical perception research in changing a visualization recommendation algorithm's behavior and outputs. Based on the analysis findings, this dissertation opens avenues at the intersection of graphical perception and visualization research, like how to evaluate the utility of new graphical perception work in guiding visualization recommendations.
This dissertation contributes a multi-faceted approach for evaluating visualization recommendation algorithms to investigate factors affecting an algorithm's performance and ways to improve it. It first proposes an evaluation-focused framework and then demonstrates how the framework can evaluate strategic behaviors and user performance among a broad range of existing algorithms. The case study results show that newly proposed algorithms might not significantly improve user performance. One way to improve the algorithm performance is by integrating more established theoretical rules or empirical results on how people perceive different visualization designs, i.e., graphical perception knowledge, to guide the recommendation ranking process. Thus, this dissertation next presents a thorough literature review of existing graphical perception knowledge that can inform visualization recommendation algorithms. It contributes a knowledge base that collates existing theoretical and experimental visualization comparison results and summarizes key study outcomes. Further, this dissertation conducts an exploratory analysis to investigate the influence of each piece of graphical perception research in changing a visualization recommendation algorithm's behavior and outputs. Based on the analysis findings, this dissertation opens avenues at the intersection of graphical perception and visualization research, like how to evaluate the utility of new graphical perception work in guiding visualization recommendations.
Examining Committee
Bio
Zehua is a Ph.D. student at the Department of Computer Science, University of Maryland, College Park. She holds an MSc in Computer Science from the University of Maryland and a BEng in Telecommunication Engineering from the Beijing University of Post and Telecommunication. Her research interests include Visualization Recommendation, Visual Analytics, and general Human-Computer Interaction.
This talk is organized by Tom Hurst