Meet in Kobe

At the end of October, I traveled to Kobe, Japan for the tenth International Society for Music Information Retrieval conference.  This was my second ISMIR, and to me this conference is different than many of the other academic conferences I attend.  For one, the work is inherently multi-disciplinary, drawing from such disparate fields as machine learning, musicology, information retrieval, acoustics, composition, statistics, and sociology.  But second, I think the crowd at ISMIR is different in a friendlier way.  Perhaps I feel this way simply because ISMIR aligns well with my own interests, or perhaps the composition of the participants and audience is varied enough to capture more than just a single vertically integrated field of expertise.

This year, Paul Lamere invited me to participate on an industrial panel during the conference, along with representatives from other companies working in the music information retrieval world.  I really enjoyed the conversation; representatives from smaller companies like Barcelona Music and Audio Technologies, The Echo Nest, and Last.fm were there to discuss MIR alongside large corporations like Yahoo, NTT, and Gracenote (I represented Microsoft).

I had two main points in the panel (though I’m sure I had opinions on a lot of other topics).  First, I frequently feel at academic conferences that even though much of the work can be technically elegant and solve problems with measurable success, sometimes it is difficult for me to get excited about slight increases in prediction accuracy (or whatever) when practical problems would prevent the research from ever being used in the real world.  For example, often work in recommendation engines measures success in a very closed way, even though that method is somewhat standard and accepted in science.

Usually with IR or machine learning problems researchers carve up their data sets and perform what are known as “hold-out experiments.”  Data from one partition are used to train an algorithm, whereas data from the remaining partition are used to test the output of the algorithm.  This works well for many cases, but in the realm of media recommendations I think the process breaks down.  These experiments fail to account for the effect of users interacting with the system and the time effects of recommendations as a whole.

It can be notoriously difficult to measure the true performance of a recommendation engine if you do not have the luxury of trying the system out on a large enough set of users, so I understand that the hold-out experiment is sometimes the only way one can measure an algorithm given the data set.  But too often I feel that researchers focus on satisfying F-measure curves or the bounds of the experiment and miss the mark when it comes to developing a system that real users love.

Perhaps that sounds too practical, for of course there is merit in science for science’s sake alone.  But the point I wanted to make in the panel is that we still don’t have a good way of measuring recommenders’ performance, and if someone really wants to have an impact with real users, sometimes simpler solutions that perform well and return reasonable results fast will blow away the fanciest learning algorithm.  Greg Linden sometimes discusses this idea on his blog.

The second point I wanted to make is that as far as the music industry goes, the democratization of music technology and the application of MIR-specific technology into mass-market products over the past decade presents a big opportunity for the scientific community and for industry alike.

Whether you love or hate Rock Band and Guitar Hero, they have changed the way we experience and consume music.  The recording industry has had an extremely difficult time adapting to technological advances, and as a result they watched their revenues collapse.  The labels are also desperate to sell you the same content you already have again and again by inventing new formats.  Well, Harmonix was able to do what the record labels could not by transforming the music experience.  By adding a little additional metadata to recordings people already know and by creating an immersive experience, many of us are willing to buy the same songs again and to enjoy them in this new way.

Someone in the audience asked if we on the panel could point to specific examples of MIR technology being applied to products.  The game products are some, and Microsoft Songsmith is another.  Companies like Microsoft, Last.fm, Pandora, and Apple are using MIR technology to drive data-driven music experiences with Zune Smart DJ and Genius.

There have also been huge advances in music creation technology over the past few years.  Apple placed GarageBand on every new Macintosh computer so the masses can create music without having taken a lesson.  Yamaha licensed MIR technology in some of their new music production tools in Cubase and for Vocaloid.  And, there are literally hundreds of brand new iPhone apps using MIR.

There’s an abundance of opportunity for MIR, and as long as people are entertained, enlightened, and satisfied by listening to or making music, we all have a chance to embark on some very interesting, fun work.

Thanks to: Paul Lamere, Justin Donaldson, Malcolm Slaney, Òscar Celma, and Norman Casagrande.

Posted: November 8th, 2009 | Tags: conferences, ismir | 3 Comments »

---