The increasing complexity of expert domains, such as machine learning performance optimization, highlights a critical limitation of current AI systems: their inability to function as true cognitive partners in iterative, multi-stage problem-solving. Experts are forced to act as "human middleware," constantly translating high-level intent into low-level system actions and managing fragmented information, leading to high cognitive load and suboptimal outcomes. This preliminary proposal introduces the Digital Executive Assistant System (DEXAS), an intent-driven framework designed to overcome this challenge by mediating expert cognition. DEXAS is founded on three interconnected pillars: a Unified Semantic Workspace that integrates heterogeneous data sources (e.g., logs, metrics) into a dynamic, machine-readable state to semantically ground user goals; Collaborative Agentic Workflows that leverage a multi-agent architecture to autonomously decompose problems, delegate tasks, and synthesize solutions under flexible expert oversight; and Longitudinal Context and Reasoning that enables the system to learn from prior interactions and refine its strategies over time. To validate these principles, this dissertation will develop and evaluate the ML Performance Debugging Assistant. This instantiation will investigate three key research questions: (RQ1) the methods for translating ambiguous human intent and diverse data into an effective problem representation for decomposition; (RQ2) the design of multi-agent orchestration for generating and refining solutions with expert-in-the-loop control; and (RQ3) the impact of such an agentic system on expert performance, cognitive load, and trust, including the identification of key interaction design principles for transparency. Through this research, DEXAS seeks to establish a novel paradigm for human-AI collaboration, enhancing expert agency and efficiency in complex digital environments.
Yuran Ding is a Ph.D. student in Computer Science at the University of Maryland and a recipient of the NSF Graduate Research Fellowship. Her research focuses on designing intelligent systems that function as cognitive partners, making AI more responsive, intuitive, and expressive for demanding tasks. She develops agentic workflows and multimodal systems that integrate diverse inputs like speech, gesture, visual context, and domain knowledge with large language models to enable adaptive, intent-driven interaction. Her prior projects have spanned haptic illusion systems, augmented presentation tools, and advanced LLM agents, with work receiving honorable mentions at premier HCI venues CHI and UIST.
Her doctoral work introduces the "Digital Executive Assistant System (DEXAS)" framework, aimed at mediating expert cognition in complex, multi-stage problem-solving. DEXAS leverages a Unified Semantic Workspace, Collaborative Agentic Workflows, and Longitudinal Context to bridge the gap between human intent and system execution, which she plans to validate through an ML Workload Performance Debugging Assistant. Looking ahead, Ding aims to explore how systems of intelligent agents can collaboratively interpret and decompose human intent, manage shared memory, and build rich contextual understanding over time, fostering systems that are truly adaptive, interpretable, and aligned with human goals.