[display_podcast]
This short video shows my first test using the Nintendo Wii controller, or “Wii-mote,” to affect the playback of a short passage. The motion and position of the controller tells the computer to select from among pre-recorded samples of musicians playing loud or soft.
The results are promising, but I can easily see that this approach will have limitations as the music gets more complex. For example, when first violins need to be emphasized and then second violins immediately following them, how will the controller “know” which instrument group to modify? We’ll probably have to pair multiple controllers (perhaps Wii-motes) with multiple musicians and computers.
DeeDee,
I’m working on the tempo control right now, actually. I think the nunchuck would just be distracting, though. The seconds can figure out when to come in on their own.
I’m glad you noticed the grace notes. This is how they were played in Beethoven’s day, according to the critical commentary by Jonathan Del Mar, editor of the 2000 edition of the symphony published by Barenreiter.
Here is what Del Mar says about which interpretration of the grace notes would be “odd” (or “wacky” to use your term):
And, in the third movement of this symphony (m. 186 and 190), the grace notes are always played before the beat. So, from the standpoint of consistency (or, put another way, to avoid being arbitrary), it makes sense that the grace notes in the second movement would be before the beat as well.
Apart from these scholarly considerations, however, I also wanted to see what musical affect before-the-beat grace notes would have. They do change how the melodic line moves forward. By appearing in a metrically weak position, they seem to increase forward momentum. (Whereas, when they are on the beat, in the 20th-century style, they seem to divide the melody into smaller segments.)
One of the great things about working with a digital orchestra is the fact that these points of interpretation can actually be put into sound and judged on their musical merits and not merely debated on musicological grounds.
I don’t think that you would need multiple wii-motes (or at least not more than two) to accomplish what you’re discussing, but rather a very fine tuned set of spatial commands that would interpret where you’re pointing as communication with particular parts of the orchestra, just like a real conductor. Being a musician who almost went into computer science, this is really exciting stuff!
Perhaps you could find a way to make your right wii-mote control tempo, both for dynamics, and left for articulation. For example, have your fonts divided down into forte-marcato, forte-staccato, forte-legato, etc, then piano-marcato, piano-staccato, piano-legato… you get the point.
Incredibly promising work!! Thank you for devoting time to something like this!
I’ve played a little bit with a Wii remote to control modulation, pitch bend, as well as using it as a performance tool (not practical for the latter). This is a good idea since it makes recording expression on the fly very convenient.
Now, did you say that you’re using the Wii as well as the remote? What are you using to get the Wii talking to your sequencing software?
And a side question: Is Jonathan DelMar by chance related to Norman Del Mar?
Bobby
As an expression controller…you could even make it into a MIDI theremin!