Did you ever wonder how YouTube settled on the algorithm that powers their Suggested Videos feature? Me neither, but thanks to WebProNews, we now know that it is based on something from Amazon.com. Specifically, it’s based on the “old Amazon recommendation engine.” We know this because of a blog post from Greg Linden–the man who created the recommendation engine in question.
Now, considering the recent waves Google has made in calling out the copycat work of Microsoft’s Bing search engine, it might be easy to take this bit of knowledge and use it to beat up Google. In fact, many have done just that. But that wasn’t why Linden wrote the piece. He was simply pointing out how interesting it was that a type of algorithm developed nearly 12 years ago is still the best choice for YouTube’s Suggested Videos today. In fact, so many people assumed Linden was trying to criticize Google that he had to update the post denying it:
“Update: To be clear, this was not intended as an attack on Google in any way. Googlers built on previous work, as they should. What is notable here is that, despite another decade of research on recommender systems, despite all the work in the Netflix Prize, YouTube found that a variant of the old item-to-item collaborative filtering algorithm beat out all others for recommending YouTube videos. That is a very interesting result and one that validates the strengths of that old algorithm.”
This would all be totally and completely intriguing and fascinating if it wasn’t for one glaring problem: YouTube’s suggested videos feature kind of sucks. I’ve said so before, and I’m sure I’ll say so again. I almost never find any actual useful videos among the group YouTube sets aside especially for me. Has it gotten better? Yes… for certain. But is it useful yet? No, not for me… not at all.
But I’m okay with that. For some reason, I’m just fine with the idea that we haven’t yet created the perfect mathematical formula to predict audience enjoyment. I’m not sure we can, actually, as much as the engineers of the world would disagree. Sometimes I don’t even know what kind of video I want until I see it, usually quite by accident. If I don’t know… how can YouTube know?
There is new hope, however, that the Recommended Videos feature is about to improve dramatically–remember YouTube’s little acquisition last week? They picked up Fflick, a movie-recommendation engine. Is it possible that the Fflick purchase was all about overhauling the suggested videos? Yes, I think it is. If YouTube is serious about getting people to spend tons more time watching videos every day, they’re going to need a better recommendation system than what they have in place now.
Regardless, it is fascinating to me that an algorithm more than a decade old is still the current best solution for recommending videos–which speaks either to the original algorithm’s power or the difficulty in creating a good recommended-videos feature, I’m not sure which. But to compare this to the Bing controversy is apples to oranges. One appears to be blatant thievery, while the other is an adapted work.