While today’s tools allow us to communicate effectively with others via video and text, they leave out other critical communication channels, such as physical skills and embodied knowledge. These bodily cues are important not only for face-to-face communication but even when communicating motor skills, subjective feelings, and emotions. Unfortunately, the current paradigm of communication is rooted only in symbolic and graphical communication, leaving no space to add these additional haptic and/or somatosensory modalities.
This is precisely the research question I tackle: how can we also communicate our physical experience across users?
In this talk, I introduce how I have engineered wearable devices that allow for sharing physical experiences across users, such as between a physician and a patient, including people with neuromuscular impairments and even children. These custom-built user interfaces include exoskeletons, virtual reality systems, and interactive devices based on electrical muscle stimulation.
I then investigated how we can extend this concept to support interactive activities, such as product design, through the communication of one's bodily cues.
Lastly, I discuss how we can optimize our subjectivity using the psychophysics approach, such as a sense of agency, when our bodies are modified, actuated, or shared with a computer or a human partner.
I conclude my talk by discussing how we can further explore the possibilities enabled by a user interface that communicates more than audio-visual cues and the roadmap for using this approach in new territories, such as understanding how our bodies, perceptions, and somatic interactions contribute to the formation of human embodiment, subjectivity, and behavior.
Jun Nishida is an Assistant Professor in the Department of Computer Science and Immersive Media Design program at the University of Maryland, College Park, where he leads Embodied Dynamics Laboratory (https://emd.cs.umd.edu/). Previously he was a postdoctoral fellow at the University of Chicago, advised by Prof. Pedro Lopes. He received his Ph.D. in Human Informatics from the University of Tsukuba, Japan in 2019. His research interests focus on developing interaction techniques and wearable interfaces where users can communicate their embodied experiences to support each other by means of wearable and human augmentation technologies, with applications in the fields of rehabilitation, education, and design. He has received ACM UIST Best Paper Award, ACM CHI Best Paper Honorable Mention Award, Microsoft Research Asia Fellowship Award, and Forbes 30 Under 30 Award among others.