This piece explores audiovisual composition using hand gestures as the primary means of interaction. It adapts the Sabetan technique of Indonesia's Wayang Kulit shadow puppetry, aiming to demonstrate embodied audio-visual interaction. Sabetan involves expressive gestures by the puppeteer, conforming to the scene's characterization.
The embodied interaction is performative, creating audio and visual elements simultaneously. It draws on Michael Chion's "added-value" concept. A choreographed hand movement, incorporating Sabetan, facilitates real-time interaction with a depth camera and machine learning. This concept is implemented in Ciung Wanara, an interactive music system integrating Wayang Kulit and supervised machine learning.
Utilizing Flucoma, the interactive assisted-ML system enables simultaneous control of sound synthesis, spatialization, and visuals based on hand gesture spatial location adapted from Sabetan techniques.
Accepted submissions will be programmed at AIMC 2024 in one of three performances, please indicate which Performance(s) your submission is suitable for:
• Performance 2, at an Oxford University performance space, will feature a flexible stage and electronics/projected visuals set-up suitable for music with more complex technical requirements. No live performers are provided, but musicians are welcome to perform their own works, or provide their own performers (this should be indicated in the submission)
Laptop table with power source
Multichannel Speakers-System (preferable Octophonic)
Video Projector with HDMI connection
"Ciung Wanara" is an interactive audiovisual composition inspired by Indonesian shadow puppet battles. Using hand gestures as the primary mode of interaction, this piece merges Wayang Kulit traditions with modern technology to weave a captivating narrative of a wise king's triumph over an evil sorcerer.
Through the integration of Sabetan techniques and innovative technologies like Flucoma (supervised machine learning), "Ciung Wanara" enables simultaneous control of sound synthesis, spatialization, and visuals based on gesture input. The rooster avatars that respond dynamically to performer movements, creating a unique immersive experience, bridging between tradition and innovation in audiovisual storytelling.
1 to 5 high-resolution images or photographs in .jpg format:
video or audio documentation of the performance:
• Norman MacGeorge Scholarship (research grant)
• Project 11