log in  |  register  |  feedback?  |  help  |  web accessibility
PhD Proposal: An Intelligent Practice Assistant for Beginner Violin: Closing the Motor-Learning Loop with Typed, Pedagogically Grounded Feedback
Siyuan Peng
Friday, August 14, 2026, 2:00-3:30 pm
  • You are subscribed to this talk through .
  • You are watching this talk through .
  • You are subscribed to this talk. (unsubscribe, watch)
  • You are watching this talk. (unwatch, subscribe)
  • You are not subscribed to this talk. (watch, subscribe)
Abstract

Learning the violin is a fine motor skill driven by a perception-action feedback loop. In weekly lessons, a teacher completes this loop by diagnosing errors and providing targeted interventions. However, during solo home practice, beginners cannot reliably detect or diagnose their own mistakes, which often leads to cementing errors rather than fixing them.


This proposal introduces an intelligent practice assistant designed to close this at-home feedback loop. While such systems can be modeled end-to-end, this work argues for a hybrid architecture that embeds explicit pedagogical structure. The central hypothesis is that under the scarce expert training data typical of music pedagogy, explicit structure acts as a powerful inductive bias. By combining learned multimodal perception with a teacher's diagnostic vocabulary of hypothesized error-to-cause attributions, a structured system can match or exceed end-to-end models in diagnostic accuracy and intervention selection while remaining highly sample-efficient and auditable. Each layer is evaluated against a matched unstructured baseline on identical data, spanning diagnosis, feedback generation, and intervention policy. An early prototype running the full perception-action loop on real recordings already supports this approach. The proposal details ongoing lesson data collection and an 18-month roadmap to evaluate the full decision policy.

Bio

Siyuan Peng is a PhD student in the Department of Computer Science at the University of Maryland, College Park, advised by Prof. Cornelia Fermüller. Peng's research sits at the intersection of machine listening, neuro-symbolic reasoning, and intelligent tutoring systems, with a focus on AI for music education. The work is grounded in a conviction about how intelligent systems should be built: purely data-driven models are black boxes — powerful, but opaque, data-hungry, and unaccountable for their decisions — while purely symbolic systems are transparent but too rigid to capture the richness of real-world signals. Peng believes the interesting territory is the hybrid in between, where the pattern-recognition power of learning meets the interpretability and data-efficiency of explicit structure, and each covers the other's blind spots. Peng's dissertation puts this belief to work in music education: an intelligent practice assistant that helps beginner violinists hear, understand, and fix their own mistakes between lessons.

This talk is organized by Migo Gui