Building a device that uses ultrasound to translate hand movement into audio feedback, and investigating the resulting experiences of defamiliarization and immersion, through human-computer-interaction.
THE DEVICE
Emitters placed on certain fingers play the modulated ultrasonic audio. When the user moves their fingers, the mics on other fingers pickup the shifting demodulated audio and feed it into the connected headphones. In doing so, the device provides dynamic auditory experiences by translating subtle hand gestures into changes in the ongoing audio-scape.
INSTALLATION USE
STUDY
This study examined whether the device defamiliarizes one’s hand movements and how this contributes to immersion when used as an interactive installation. It also explored participants’ subjective experiences during a week of daily device use. One-time tests with interviews and a questionnaire assessed immediate defamiliarization effects. Extended-use tests over the week included interviews and quantitative mindfulness questionnaires to assess any prolonged impact on the relationship between hand and mind.
RESULTS
The device successfully translates even the most delicate hand gestures into audio-feedback. Participants experienced defamililiarization. The installation facilitated introspection in participants, creating awareness of connections between their mind and body. This link was also found to manifest itself as a connection between the participant and the device itself.
