VR platform changes how people work together on music remotely
The University of Birmingham has created a new virtual reality system that could change how artists practice, play, and teach from afar by solving a major technical problem: delay.
The Joint Active Music Sessions (JAMS) platform adds responding models that let musicians play together perfectly in time through VR glasses. This gets rid of the small delays that can make it hard to work together online. The University of Birmingham’s Dr. Massimiliano (Max) Di Luca noted that even minor lag could greatly impact performance, noting that delays as low as 10 milliseconds can disturb a musician’s beat, break focus, or mess with the technical aspects of playing.
The method works by letting artists record themselves playing, which is then used to make personalized models that can interact with other musicians in a virtual world. Dr. Di Luca said that the software makes a character that responds and is perfectly timed with the music partner. For shows, practice, or teaching, musicians only need an iPhone and VR gear to join.
What makes JAMS unique is that it focuses on recording small visual cues that are important for playing with others. The platform carefully records every detail, like how a violinist’s bow moves or how artists look at each other during important times. These models move and react to the VR user’s movements, creating a unique, personalized experience that feels a lot like working together in person.
The Augmented Reality Music Ensemble or ARME initiative is where the technology came from. It brought together experts from six fields, such as computer science, music, sports science, psychology, and more. This teamwork between people from different fields made it possible to make accurate computer models that show both the changes in time and the physical moves that are necessary for natural rhythmic interaction.
Dr. Di Luca said that the goal was to make making music online feel like making music in person. Musicians can change the looks of the avatars they play with or use a maestro’s character to practice and get better.
JAMS can be used for more than just teaching and performing music. It also has a lot of potential as a social tool where artists can meet, work together, and share their work with bigger audiences through virtual events. By making digital twins of musicians, the platform also opens up new business opportunities. This could change the way that music is licensed and shared in virtual worlds.
