BellKor Wins Netflix $1 Million By 20 Minutes 104
eldavojohn writes "As we discussed at the time, there was a strange development at the end of Netflix's competition in which The Ensemble passed BellKor's Pragmatic Chaos by 0.01% a mere twenty minutes after BellKor had submitted results past the ten percent mark required to win the million dollars. Unfortunately for The Ensemble, BellKor was declared the victor this morning because of that twenty-minute margin. For those of you following the story, The New York Times reports on how teams merged to form Bellkor's Pragmatic Chaos and take the lead, which sparked an arms race of teams conjoining to merge their algorithms to produce better results. Now the Netflix Prize 2 competition has been announced." The Times blog quotes Greg McAlpin, a software consultant and a leader of the Ensemble: "Having these big collaborations may be great for innovation, but it's very, very difficult. Out of thousands, you have only two that succeeded. The big lesson for me was that most of those collaborations don't work."
Anonymous Coward (Score:1, Insightful)
the topic confuses me
nonsense (Score:5, Insightful)
Setting an arbitrary goal that only .2% of competitors could meet does not mean that most collaborations don't work. If 90% of the teams met the target, you probably wouldn't be so quick to claim that the vast majority of collaborations do work but rather that the goal wasn't high enough.
Funny, I learned a different lesson... (Score:5, Insightful)
The big lesson for me was that big collaborations were the most successful.
In creating solutions for hard problems most of everything fails and is horribly difficult. No big surprise there. Kinda odd that was the quoted lesson...
Well at least.... (Score:3, Insightful)
Re:Anonymous Coward (Score:3, Insightful)
The Objective (Score:2, Insightful)
OK, so somebody won a prize, offered by NetFlix, to do... what exactly?
Re:It was a tie... (Score:4, Insightful)
Most football games didn't start in 2006, so proportionally 20 seconds is far too long. You didn't exaggerate near enough, someone else can do the math though. (I'm real sleepy, but the imaginary football game came down to roughly 45 milliseconds?)
I'm really surprised Netflix didn't offer 2 million dollars to the two winning teams, or at-least some sort of consolation prize, as it was effectively a tie in a culmination of years of work.
These people did so much work even at a million dollars they would have likely earned below minimum wage. Netflix has come a long way since 2006, and this kind of research would have cost many millions, they really can't lose here. Unless the contest took so long the code isn't useful and they have already surpassed 10% in house.
Re:It was a tie... (Score:3, Insightful)
Re:The Rules are the Rules... (Score:3, Insightful)
Re:I think it's a gloss on prizes as innovation-sp (Score:3, Insightful)
The benefits to society in disseminating knowledge of data mining technologies and good datasets largely dwarfs the knowledge of the winning entry (think Metcalfe's law).
You're only considering the benefits to society that result from this particular competition. The argument about prize systems being inefficient has to do with the fact that while they generate huge interest in a particular topic (and yes, generate more returns than simply the winning entry), they also result in an inefficient allocation of resources to that one particular topic.
I.e., some of the entrants would likely have benefited society more by flipping burgers or sweeping sidewalks than by wasting their time on the Netflix prize.
The problem is somewhat reduced if you have a large number of prizes on various topics, because then people can devote their time to areas where they have more of a chance of winning, or if you make the cost of entry non-zero (it can still be very low - anyone with any real interest and talent will not be turned off by a $1 or $5 registration fee, or by a simple test to assess their capabilities).
Re:ratings systems (Score:4, Insightful)
I'm sure that every schmuck with a Netflix account would be willing to adhere to your stupid rules, and saddened by your unwillingness to pontificate on how you'd change human behavior.
Seriously, this is what Netflix would be if it were invented by Stalin.
Re:ratings systems (Score:4, Insightful)
Your proposed solution would only make sense if people were forced to watch a completely random selection of movies. Once you factor in the fact that people are allowed to select which movies they want to watch, it makes sense that their ratings would cluster towards the high end of the spectrum. That is, in fact, the whole point of this ratings prediction system: to tell you, in advance, which movies you will like. If it worked perfectly, you'd never have to rate a movie below average, because you could avoid ever renting a movie which you wouldn't like.