The Three Pieces for Solo Clarinet were written in 1919 by Igor Stravinsky as a gift for the patron Werner Reinhart, and is one of the most well-known pieces in the solo clarinet repertoire. However, the piece is performed mostly in auditions, and less in public performance settings. Nikki Pet, clarinetist and animator studying at the Yale School of Music (MM ‘23, DMA ‘29), created a highly synchronized music video to emphasize its Jazz connections and highlight its narrative storytelling, designed for the concert stage. Using technology described paper “Video Accompaniment: Synchronous Live Playback for Score-Aligned Animation” (Sound and Music Computing 2023), Kaitlin Pet and Christopher Raphael transformed this “static” music video to a “video accompaniment” that remains synchronized with a live performer during concert. Our visual accompaniment software uses as a backend Informatics Philharmonic, an automatic accompaniment software developed by Chris Raphael. It uses statistical machine learning techniques such as a Hidden Markov Model for score following and a Baysian Belief network to model the musical progression, allowing both hearing and prediction to adapt to the individual player. We connected Informatics Philharmonic to a Max/MSP-based video playback controller that speeds up and slows down the video to remain in-sync with Nikki as she freely plays. The result is a responsive multimedia performance where the player is completely unencumbered by the technology, one where they are allowed full control over tempo and interpretation without having to worry about staying in time with electronic performance media.
Type of Submission
Our program would be a good fit for Performance 1 or Performance 2. We will provide performers (Nikki Pet).
Technical/Stage Requirements
High Level Overview
Clarinet soloist plays with projected “video accompaniment” controlled by a program run on our MacBook.
Personell
Nikki Pet (author, will come to the conference) will be the sole performer. Kaitlin Pet will setup and start the “video accompaniment” program, as well as monitor it during the performance.
Performance Duration
Total: 10 minutes
5 minute technology/piece introduction, 5 minute solo clarinet playing.
Setup
Ideally, both Nikki (clarinet) and Kaitlin (tech) will be onstage, with Kaitlin starting and monitoring the program connected to a projector / large screen. This presupposes it is possible to control the projection monitor from the stage.
If the projection monitor must be controlled from an external panel or recording booth, Kaitlin and Nikki will start on stage to introduce the piece, then Kaitlin will move to the recording booth to start and monitor the software.
Equipment Needs
A high-quality video projector OR large LED screen. We must be able to connect our laptop to the projector.
Power source (230 Volt) . Laptop running live animation must be plugged in to power (We will bring necessary adaptors to power devices from the United States)
Microphone (mono signal): The microphone must be placed close to the clarinet and be connected to the laptop to allow real-time audio processing.
A music stand
2 chairs
A table. This is to hold clarinet-related equipment (e.g. instrument case, water, reeds, etc. ) and/or a laptop (if we can control the projector from the stage)
A room where Nikki Pet can warm up by herself. This is not strictly necessary but would be preferred.
Additional Notes
We will need access to the performance space and projector setup beforehand to test the setup and correct for any project-related latency in the animation. Ideally we would have at least 30 minutes for this testing.
Program Notes
The Three Pieces for Solo Clarinet were written in 1919 by Igor Stravinsky as a gift for the patron Werner Reinhart, and is one of the most well-known pieces in the solo clarinet repertoire. However, the piece is performed mostly in auditions, and less in public performance settings. Nikki Pet, clarinetist and animator studying at the Yale School of Music (MM ‘23, DMA ‘29), created a highly synchronized music video to emphasize its Jazz connections and highlight its narrative storytelling, designed for the concert stage. Using technology described paper “Video Accompaniment: Synchronous Live Playback for Score-Aligned Animation” (Sound and Music Computing 2023), Kaitlin Pet and Christopher Raphael transformed this “static” music video to a “video accompaniment” that remains synchronized with a live performer during concert. Our visual accompaniment software uses as a backend Informatics Philharmonic, an automatic accompaniment software developed by Chris Raphael. It uses statistical machine learning techniques such as a Hidden Markov Model for score following and a Baysian Belief network to model the musical progression, allowing both hearing and prediction to adapt to the individual player. We connected Informatics Philharmonic to a Max/MSP-based video playback controller that speeds up and slows down the video to remain in-sync with Nikki as she freely plays. The result is a responsive multimedia performance where the player is completely unencumbered by the technology, one where they are allowed full control over tempo and interpretation without having to worry about staying in time with electronic performance media.
Media
Ethics Statement
We do not have conflicts of interest in this work. Since the training of machine learning models is done with the performer’s own music, we believe our work sidesteps a potential ethics issue found in other AI-based music systems.
Creative Commons Attribution 4.0 International License (CC-BY 4.0)
