Visual Microphones: The Future is Now

The minutest movements of plant leaves. A glass of water: deceptively still. A bag of chips, lying discarded on the table. One of these things may be slightly less poetic than the others, but they do have one thing in common: scientists from MIT can recover sound from all three.

Calling it “the Visual Microphone” a team of researchers are using visual data to recover sound from videos of everyday objects. These objects are seemingly still to the naked eye, but upon reviewing the video, researchers were able to pinpoint the modes of vibration of these objects.

From their abstract:

“When sound hits an object, it causes small vibrations of the object’s surface. We show how, using only high-speed video of the object, we can extract those minute vibrations and partially recover the sound that produced them, allowing us to turn everyday objects—a glass of water, a potted plant, a box of tissues, or a bag of chips—into visual microphones.”

Comprised of Abe Davis, Michael Rubinstein, Neal Wadhwa, Gautham Mysore, Fredo Durand and William T. Freeman, the team’s website says they’re working on releasing code and data. But so far, they’ve posted sound samples of their work. Check it out here.

The Science of Wikidrummer

Video

We’re back from spring break with a new video!

Drummer Julien Audigier and Audio Zéro put together the Wikidrummer video, an exploration of different environments and the effect these spaces had on drum sounds.

In this video, we break down the science behind why all of these spaces sound different. We’ll also show you our own reverb experiment, and how you can incorporate physical spaces into your music, even if it was recorded somewhere else.

Where Music, Education, and Technology Collide

We at the Science of Music are firmly perched on the intersection between music and technology, but we’re at a few other crossroads as well.

With technology changing the way we look at art (and vice versa) it’s only to be expected that it’s changing the way we look at education. We, in particular, are especially interested in the way technology is reshaping how we think of music education.

A new edition to NYU’s faculty in 2013, Alex Ruthmann also serves as the President of the Association of Technology in Music Education. We’re happy to have him at NYU, and especially happy to talk with him about the future of music education, why people should learn music, and what it means to be both an innovator and an educator. For more from Alex, you can find him on Twitter as @alexruthmann or visit him at his website.

Analog Vs. Digital: The Age-Old Question

Video

Can you really hear a difference between analog and digital recording? What about in synthesis? In a personified rap battle between analog and digital, who would win?

NYU students, having nothing better to do at our ivory tower of academia (I mean, it’s not like we live in one of America’s biggest commercial and cultural centers or anything) love contemplating age-old questions such as these. You may remember some freshmen who previously did ponder whether was better to auto-tune or to not auto-tune. And while these questions may prove inconclusive, the best part is, as always, in the debate.

 

 

So our freshmen rose to the occasion once again to bring forth this rousing performance; a call to arms for their fellow rap battlers. And a cry heard throughout the land did ring: Who won? Who lost? You decide!*

*No seriously, leave a comment below.

NAMM, Loudness Wars, and Grammys (but not in the way you think): Weekly Roundup for January 19-26th

We’re trying something new here at Science of Music: from now on we’ll give the low-down on all things that are at once science-y, tech-y, and music-y in the news once a week. Watch this space for more.

NAMM 2014

We’re keeping our eye on the “best of” lists and products coming out of NAMM since we can’t actually be there…which we’re still getting over for more reason than one. (I mean, just for the weather alone, right?) xlr8r has a take that covers the good, bad, and weird while Line 6 has created an unholy guitar-amp-bluetooth-speaker-iOS-integration combo.

But what we really, really want is this right here. Korg has announced a build-it-yourself kit that will let you build your own MS-20 synth. This analog, monophonic synth comes pre-disassembled (which is a little sad, because some of the fun is tearing something apart) giving you the chance to put it together. For the ultimate consumer-tinkering-friendly experience, no soldering or knowledge of schematics are required. The MS-20 kit is expected to be out in March for around 1,400 USD.

Breaking Genre

British Singer Katherine Jenkins says her record sales game is too strong to ignore. She claims her “crossover” pop-classical style of singing has become so popular it’s “becoming its own thing.” In fact, the Telegraph is referring to it as the “crossover” genre. To this, we say: what defines a musical genre, anyway?

A 1981 study by Franco Fabbri defines genre as “a set of musical events (real or possible) whose course is governed by a definite set of socially accepted rules.”

More recently, companies like Echo Nest (which supplies Spotify with data) are mining for these rules with—according to their website—over 35,000 songs and over 1 trillion data points. With user data refining such an enormous machine will the algorithm become the ultimate genre codifier?

A quick borrowing of my sister’s Spotify account showed a biography that called Jenkins “classical” but that her related artists contained singers like Sarah Brightman, Andrea Bocelli, and Charlotte Church, all of whom are known for their pop take on classical singing. Then again, it also had serious classical musician Howard Blake and non-classical songwriter Emmy Rossum.

Crossover as a genre? Maybe.

Peace in Our Time

Hugh Robjohns of Sound on Sound covers the protracted end of the “Loudness Wars” in the magazine’s February issue. Mastering engineer Bob Katz declared an end to the wars at AES in October, but how will we keep the peace?

In the noise of modern society, everyone is clamoring to get heard. In particular, recorded audio has been trying to “out-loud” itself for some time now, which has led to the loss of dynamic range, over-compression, and just bad sound in general. But with new technology that will put a smarter limit on audio, either broadcasted or streamed, the war may be over. The rub? Overly loud mixes will probably end up sounding “feebler” over the new system. So shut up or sound bad.

Dead Musicians’ Society

The Providence Journal ran a commentary piece about the importance of music education in schools while an upstate New York music teacher receives the first-ever Grammy in Music Education.

Of note in the PJ commentary is the idea that musicianship is not a 21st century skill. Of course not: it’s a 23rd century skill. Who else is going to teach our cyborg overlords how to play the violin? They’re going to want to be proper gentlemen, after all.

Got a tip? Send us a message at scienceofmusicnyu@gmail.com , or , or post it to our Facebook page.

A Speaker From a Starbucks Cup?

Part of what we at the Science of Music love about DIY electronics is the tinkering aspect of it. It’s the idea that you can take an everyday object and just…flip the script a little bit to create something new entirely. That’s the spirit with which we present this video project.

We looked at a Starbucks cup and, with just some magnets, a coil of wire, some tape we saw how you could “flip the script” to create a working speaker. Plus, it’s a good excuse to get yourself a frappuccino*.

(*Note: not an action we recommend in January. Unless you’re close to or within the Southern Hemisphere. In which case: Hi, can we come visit?)

Introduction to Resistors

Video

In a continuation of our Introduction to Electronic Components series we present (drum roll please) resistors!

Resistors…resist. It’s what they do. Well, they don’t resist everything. They won’t help you resist the temptations of the dark side, for example. But these little guys are useful components when it comes to regulating the amount of electricity with which you want to work.

Interested in more? Check out our full playlist here.