Data visualization is a powerful strategy for using graphics to represent data for effective communication and analysis. Unfortunately, creating effective data visualizations is a challenge for both novice and expert design users. The task often involves an iterative process of trial and error, which by its nature, is time-consuming. Designers frequently seek feedback to ensure their visualizations convey the intended message clearly to their target audience. However, obtaining feedback from peers can be challenging, and alternatives like user studies or crowdsourcing is costly and time-consuming. This suggests the potential for a tool that can provide design feedback for visualizations. To that end, I aim to create a virtual, human vision-inspired system that looks into the visualization design and provides feedback on it using various AI techniques. The goal is not to replicate an exact version of a human eye, but to develop a practical and effective system that delivers design feedback to visualization designers, utilizing advanced AI techniques, such as deep neural networks (DNNs) and large language models (LLMs).
My thesis includes three distinct works, each aimed at developing a virtual system inspired by human vision using AI techniques. Specifically, these works focus on simulation, representation, and automation, collectively progressing toward the aim. First, I develop a methodology to simulate human perception in machines through a virtual eye tracker named A Scanner Deeply. This involves gathering eye gaze data from chart images and training them using a DNN. Second, I focus on effectively and pragmatically representing a virtual human vision-inspired system by creating Perceptual Pat, which includes a suite of perceptually-based filters. Third, I automate the feedback generation process from chart analysis to textual suggestions with Visualizationary, leveraging large language models to enhance the automation. I report on challenges and lessons learned about the key components and design considerations that help visualization designers. Finally, I end the dissertation by discussing future research directions for using AI for augmenting visualization design process.
Sungbok Shin is a 5th year Ph.D. Candidate in the dept. of Computer Science at the University of Maryland, College Park. He is a member of the VisTeam in Human-Computer Interaction Laboratory (HCIL), working with Dr. Niklas Elmqvist. His research interests are in the intersection of Information Visualization, Artificial Intelligence, and Human-Computer Interaction. He is interested in developing systems that leverage AI to augment visualization design. He received his Master's and Bachelor's degree in Computer Science and Engineering from Korea University, South Korea. Upon graduation, he will be continuing research as a postdoctoral scholar with Dr. Jean-Daniel Fekete, in Aviz team at Inria, Université Paris-Saclay, France.