Muse and the Red Hot Chili Peppers
by Ben Perreira
Students of elementary statistics learn very early on that correlation does not equal causation. Just because two things react similarly and predictably it doesn’t mean one is causing the other. The oft-cited (grim) example is that ice cream sales and drowning deaths are highly correlated. One is not causing the other; they both spike in the summer.
Last summer my friend and I were discussing contemporary rock bands. He drew the conclusion that Muse and the Red Hot Chili Peppers must sound similar because a lot of people like both of them. Now, as a huge fan of the latter and not so much the former, I had to object. And as someone with some understanding of statistical interrelation, I also had to object. A lot of people like pizza and burgers, two foods that don’t taste very similar.
But now, I realize he may have been onto something.
Over a decade ago Amazon figured out that there are patterns around which people purchase things. This breakthrough led to a significant enhancement of the recommendation feature that now accounts for a third of Amazon’s sales. As Viktor Mayer-Schönberger writes in Big Data, “the recommendation system didn’t actually need to compare people with other people, a task that was technically cumbersome. All it needed to do was find associations among products themselves.”
The people at Amazon* found that there was no clear causation available for why customers bought these different products, and they didn’t care! Causation became irrelevant to Amazon because the strong correlations provided them the information they needed to enhance their product and increase sales. They knew that someone who bought, say, a Muse album may also be interested in a Red Hot Chili Peppers album. I may owe my friend an apology.
(*Amazon is a unique case in that it does unbelievably high sales and unit volume, providing uncommonly rich data on which to do regression analysis. However, I’m fascinated by how setting causation aside to examine simple correlations between behaviors can be used create growth opportunities. More to come…)