Knowing you better than you know yourself

In his 2005 interview with Giant Robot magazine (.pdf posted here), the artist Ryan McGinness (whose work is featured at the beginning and end of this blog) talks about how the role of the curator is increasingly important as it becomes easier and easier for people to produce creative output.  McGinness says that technology has created many more writers (thanks to blogs), musicians (thanks to cheaper computers and recording software), etc., and that it is "more and more difficult to separate the extraordinary from the average."  That observation has always stuck with me because it struck me as an excellent observation, and it becomes especially interesting when extended beyond creative output to apply to information as a whole.

The strength of the curator lies in finding someone whose tastes you trust - in a way, someone whose tastes are similar to yours.  Many internet sites now provide us with automatic curators in the form of recommendation engines, the algorithms that try to figure out what you might like based on what you do like.  The interesting thing that distinguishes recommendation engines from curators is the absence of another human being's bias; recommendation engines make us curators for ourselves.

For example, the Amazon.com homepage will display products that you might be interested in purchasing based on the products you've looked at in the past.  Usually the connections are pretty obvious here: you'll see books by the same author, movies with the same actor, and so on.  A more sophisticated and impressive recommendation engine powers the internet radio site Pandora, which builds radio stations based around certain artists.  I created a Pixies radio station and Pandora started lining up songs that featured "electric rock instrumentation, punk influences, a vocal-centric aesthetic, minor key tonality, and electric guitar riffs."  A Talking Heads radio station played music with "basic rock song structures, subtle use of vocal harmony, extensive vamping, a vocal-centric aesthetic, and major key tonality."  In each case, the recommendations are only about one step away from a personal preference you've demonstrated to the engine.

Many recommendation engines are perpetually being tweaked in a quest to find the best possible combination of algorithms to get inside your head.  In fact, Netflix is holding a contest to see if anyone can build a recommendation engine that is 10% better than their current engine at guessing how many stars you will give a movie.  The prize: $1,000,000.  The Netflix executives have said that they're not sure how to quantify the financial benefit of a better recommendation engine, but they're positive that it's worth more than a million dollars.

Recommendations don't just help us to find new products to spend money on (and hey, who doesn't need help with that?); they help us to find emotional fulfillment.  Dating sites are starting to evolve beyond simple search algorithms.  eHarmony prompts you to move beyond "traditional" dating by using their patented Compatibility Matching System to pre-screen partners across 29 dimensions.  Whether or not this system facilitates a more satisfied clientele than a rival dating site with a more primitive matching system - like, oh, I don't know, Adult Friend Finder - remains to be proven.