This short video shows my first test using the Nintendo Wii controller, or “Wii-mote,” to affect the playback of a short passage. The motion and position of the controller tells the computer to select from among pre-recorded samples of musicians playing loud or soft.
The results are promising, but I can easily see that this approach will have limitations as the music gets more complex. For example, when first violins need to be emphasized and then second violins immediately following them, how will the controller “know” which instrument group to modify? We’ll probably have to pair multiple controllers (perhaps Wii-motes) with multiple musicians and computers.