Undergrad Research Project - Applying Augmented Reality For Engaging Education

Spring 2016

Steffen Holm
Austin Lee
Project description

This project explores the intersection of Education and Augmented Reality (AR) through the development of three distinct virtual educational experiences. Developing AR teaching demos exploring the topics of 3D Calculus, piano playing, and basic electronics will allow us to consider different ways of using AR to meet educational needs while also giving us an opportunity to begin developing software for the upcoming generation of VR and AR devices. The goal is to both work on challenging problems in the areas of both human-computer interaction and education while also answering the question, “Can education be made more effective through the use of Augmented Reality technology?” Initially, the research will focus on producing a proof-of-concept working prototype demo. This demo will be the piano playing demo that aims to teach a student a song on the piano by visually highlighting the correct sequence and timing of key presses. The student would be instructed on how to correctly play the song before they start practicing. In addition, the student would get real time feedback derived from finger positioning and sound data received from the 3D camera and microphones respectively. This feedback would come in the form of visual overlay indicating the correct key and the wrong key that was played. Main parameters we would look at would be the song’s degree of difficulty (beginner, intermediate, advanced) and the musical experience of the student (beginner, intermediate, advanced). This would allow us to see at which combination of those two parameters AR is most useful. Would a beginner student that can’t read notes very well be able to learn a song much above his skill level? Could an expert still find use in the real time coaching? These are questions that we would explore while experimenting with this demo. Overall the goal for this demo would be to investigate an alternative method of teaching a student to play the piano and explore novel ways for the student to actively interact with the musical instrument by the AR system during the learning process. If time permits we would evaluate the demo through preliminary user study then move on to developing more future work or improve the singular demo.

Return to project list