AxAR
[2024]
AxAR is an Augmented Reality application for independently learning songs on the guitar. This project focuses on providing the user with real-time guidance and feedback while playing the instrument, using a Head Mounted Display with XR capabilities. It provides a more intuitive and seamless visual approach to music learning compared to traditional methods, which rely on shifting between different points of view between the instrument and music instructions. Additionally, it provides real-time performance feedback to the user without a comprehensive setup.
An in-headset showcase of the interface and controls of AxAR in use.
The integration of AR technologies in guitar playing has been a topic of research for more than two decades, using a variety of forms, most of them relying on a digital mirrored image of the guitar player where instructional elements can be added.
The recent advancements in accessibility and performance of headset technologies have democratized a number of virtual user interactions previously impossible. Some of these aspects offer a fertile environment for the development of guitar learning software that surpass the constraints of screen-based devices, namely the focus shift of the player’s point of view.
This thesis concentrates on providing a new user interface appropriate for guitar learning in XR, and methods of evaluating user performance using hand tracking and audio analysis with no additional hardware besides the headset device itself.
A short demo of the audio analysis/hand tracking method for performance detection in AxAR
The existence of similar research was a push towards elaborating on the user experience of such an application, refining the concept of serially overlaying graphics on a guitar fretboard through a novel iteration. Part of this project also dealt with ways that suggested components for AR music notation can be positioned in the surrounding space of the fretboard rather than on top of it, along with conventional GUIs that are essential in the learning of a song on the guitar.
A 10-minute presentation of the project