Dome Conductor

The foundation of the experience is a recording of the Swedish Radio Symphony Orchestra led by chief conductor Daniel Harding, performing the first 90 seconds of Beethoven's Fifth Symphony - one of the most recognizable and demanding openings in classical music. We filmed the orchestra with a wide-angle camera capturing the full concert hall, producing footage that could be mapped onto the 180° dome and played back at variable speed. Multiple takes were recorded over two sessions, along with additional material for gamification elements where the orchestra reacts to the visitor's success or failure. The orchestra's own sound technicians made a professional audio recording that serves as the soundtrack.

The core challenge was translating a visitor's conducting gestures into a timing signal that controls the orchestra playback. Researchers at KTH developed a machine learning model trained on motion data from multiple conductors - from experienced professionals to students - all conducting the same recording. Each conductor brought their own style and interpretation, giving the model a wide vocabulary of conducting motion to learn from. In the dome, we replaced the motion capture suit used during research with a simple camera-based setup - a single skeleton tracker that reads the visitor's movements in real time. This made it practical for a museum setting: no suit to put on, no calibration, just step up and start conducting. The system reads the visitor's gestures and sends a timing signal to our playback engine, which advances the recording at the corresponding pace. Working together with SMASH, we built the full real-time pipeline connecting gesture input to dome playback.

The visitor stands in the centre of the Wisdome Stockholm dome theatre, surrounded by a 180° projection of the orchestra. The scale is life-sized - the musicians fill the visitor's field of view just as they would from a real podium. When the visitor raises their hands and begins to conduct, the orchestra follows. Move faster and the music accelerates. Pause, and the orchestra holds. The experience lasts around 90 seconds, and includes gamification: if the visitor fails to communicate clearly, the orchestra grows visibly unhappy. Finish successfully, and they applaud. An instruction video gives waiting visitors a crash course in basic conducting patterns before their turn. The installation opened at Tekniska museet in 2025 and has been experienced by hundreds of visitors so far - many of them children encountering a symphony orchestra for the first time.