Visual Microphones: The Future is Now

The minutest movements of plant leaves. A glass of water: deceptively still. A bag of chips, lying discarded on the table. One of these things may be slightly less poetic than the others, but they do have one thing in common: scientists from MIT can recover sound from all three.

Calling it “the Visual Microphone” a team of researchers are using visual data to recover sound from videos of everyday objects. These objects are seemingly still to the naked eye, but upon reviewing the video, researchers were able to pinpoint the modes of vibration of these objects.

From their abstract:

“When sound hits an object, it causes small vibrations of the object’s surface. We show how, using only high-speed video of the object, we can extract those minute vibrations and partially recover the sound that produced them, allowing us to turn everyday objects—a glass of water, a potted plant, a box of tissues, or a bag of chips—into visual microphones.”

Comprised of Abe Davis, Michael Rubinstein, Neal Wadhwa, Gautham Mysore, Fredo Durand and William T. Freeman, the team’s website says they’re working on releasing code and data. But so far, they’ve posted sound samples of their work. Check it out here.