Sensory Extensions
I am required to keep a blog for one of my MIT classes. So, I am doing it here.
It should be mirrored to http://collision.mit.edu
For this assignment we were to:
“1. Write a short post (1-3 paragraph) with media/images on the blog documenting your research.
2. Make an object, extension, or interactive work that demonstrates your research. Consider the physical limits of your given sense. Your productions need not be overly literal.”
I am going to discuss audio.
We hear sound as a disturbance wave traveling through a fluid (normally air).
This wave travels through the pinna (the outre part of the ear) where it is reflected into the inner ear and the mechanical wave is converted into electricity to be carried to the brain.
Compared to other animals we have fairly poor hearing with a limited frequency and distance range.
One idea implemented for an extension was a fulfillment to the inquiry of, “What if I were in someone else’s shoes?”. But taken even farther to “What if I were in someone else’s head?”. Using a technique called “binaural audio” I built a system in which the “sensor” person wears microphones inserted into the outer ear. This sound is amplified and sent to the “remote” person which hears everything from the sensor’s perspective.
Another idea implemented took the idea of “extension” to be that of an extension of one sense into another sense. What if we could hear people without them making a sound? I wrote a program in processing / java which used the openCV library for face tracking and OSC to generate a sine wave through SuperCollider. The net effect is that when a webcam is pointed towards a person, they generate a unique sound. A person wearing headphones connected to this system could then visualize the position of people in the room without actually viewing them.
Here we see the visualization of face positions in view of the camera. The micky mouse / bear heads are just a feedback for testing. Each person in the fram is assigned a sine wave. When the person gets closer, the frequency goes up. When they get lower the sound lowers and when they move higher the volume goes up. When the move left / right the sound pans with them. This produces an aural means for visualizing room presence.
