The Gradual Public Awareness of the Might of Algorithms 169
Soylent Mauve writes "The trend toward data- and algorithm-driven tuning of business operations has gotten a lot of attention recently — check out the recent articles in the New York Times and the Economist. It looks like computer scientists, especially those with machine learning training, are getting their day in the sun. From the NYT piece: 'It was the Internet that stripped the word of its innocence. Algorithms, as closely guarded as state secrets, buy and sell stocks and mortgage-backed securities, sometimes with a dispassionate zeal that crashes markets. Algorithms promise to find the news that fits you, and even your perfect mate. You can't visit Amazon without being confronted with a list of books and other products that the Great Algoritmi recommends. Its intuitions, of course, are just calculations -- given enough time they could be carried out with stones. But when so much data is processed so rapidly, the effect is oracular and almost opaque.'"
The joy of algorithms (Score:5, Interesting)
But seriously, a food recipe is an algorithm for all general purposes. All these people are saying is that the machine learning algorithms and match peoples' personalities and buy stock are too complicated for the average Joe Programmer Wannabe and look more or less like a black box. (which if they employ neural networks, instead of say SVN, they are actually black boxes even for the author who wrote it...).
No, I think you were right the first time. (Score:3, Interesting)
As you've demonstrated, the "oracular" part is badly mistaken.
Amazon almost NEVER guesses something I'd buy.
If I buy a new DVD, I am instantly bombarded with ads for EVERY new DVD. I buy the new Terry Pratchett book and I'm bombarded with EVERY book by him or co-authored by him or licensed by him or whatever. I don't want derivatives.
I picked up the "V" comic book (graphic novel) and now I'm bombarded with every comic book they have.
As relates to your post, you can't be the only techo neo-pagan out there. But they just cannot fit you to that group, can they? Although it should be very, very easy to do so.
Software Patent Propaganda. (Score:5, Interesting)
Don't get caught up in the hype here. Algorithms are nothing special on their own. These articles are trying to make them look important, like inventions or physical objects, to further pump up the notion of software patents. It's not algorithms that are evil in GWB's great internet filters, it's the machinery that's been built on top of an otherwise dumb network and free internet that's evil.
Without algorithms, there can be no computing but there's nothing really special about any one in particular. Algorithms are just instructions, and there are many ways of achieving the same result. Algorithms can stand alone or be combined into programs that do things users want. The net result is just another set of instructions that can be considered a larger algorithm. Without modern computing equipment, most of these instructions are useless. Like the article say, "try doing this at home." No problem, if you have a computer but a real pain if you only have pen and paper. Medical imaging devices take advantage of mathematics that was little more than a curiosity when it was first published in 1917. The inventors of the device reinvented the math without knowing it some forty years later but it was not until the 1980s that the devices became practical due to the lower cost of computing.
This article is pumping up the value and utility of business methods. Common sense is a valuable thing, but it's not always an invention and business methods never are.
Re:Heuristics ARE algorithms (Score:5, Interesting)
That's an elegant metaphor, but someone has misled you.
An algorithm is a precise specification of a process whose outcome is defined by the initial conditions. To cite your example, quicksort is an algorithm -- the outcome of the sorting process is well defined, given the inputs.
But typical implementations of quicksort use a heuristic to choose the pivot element -- median of three, media of five, middle element, etc. These are heuristics because their goal is to choose the median value, but they can't make any guarantee that it will find the median. They can't even guarantee that they will find a good value. In fact, they generally don't even consider all of their inputs! They could choose bad values every time... but on average they don't, and quicksort is fast.
Another way of looking at it is that if an algorithm is correct, it will produce a correct answer for all valid inputs. A heuristic might produce incorrect answers for valid inputs, but it's correct often enough so that it might still be worth using -- especially if a correct algorithm is not known.
You may point out that randomized algorithms have a similar property -- but the difference is that with randomized algorithms the probability of error can be made arbitrarily small. With heuristics, there's no telling.
An underclass? (Score:2, Interesting)
Arthur C. Clarke, "Profiles of The Future", 1961 (Clarke's third law)
just how close are we to having this statement be 'real' to a large majority of people on this planet? discounting any second or third world countries, how many people in first world countries would consider the 'oracular' nature of an algorithm to be 'magical'?
the education system spread throughout the world is creating an over and under society incapable of distinguishing high technology from magical sources. yep, this can only bode well for the future of humanity.
'i pray to you lord skynet, pls water my crops on the back 40!!!11 here is a sacrifice to your computations.....'
Re:Boy They're Slow (Score:2, Interesting)
If they're foolish, sure they hope computers can make better decisions. If they aren't complete fools they realize that computers can provide analytical support for decisions. For example, algorithms can evaluate more potential alternatives, generate potentially good alternatives that they haven't thought of, or make predictions. In most cases, algorithms are just formalizing analysis processes. The supposition is that being able to consider more data leads to better decisions. There are cases where it works really well already, e.g., managing lines at theme parks, basic scheduling, etc (see http://www.scienceofbetter.org/ [scienceofbetter.org]). Algorithms are used extensively in portfolio selection.
Data acquisition isn't a bit deal but getting the data into the right format for the algorithm still is though there's progress being made there. The really hard parts are understanding the problem enough to formalize the process and being able to properly interpret the results. Some problems are much easier to formalize than others (portfolio expected value and risk, production rates and material requirements), some can only be done with surrogate measures at this point (water scarcity, consensus and voting, anything with 'value'), and some we may never be able to fully formalize in an acceptable way (human behaviour). Letting the algorithms take care of the easy stuff is often efficient and work is being done to increase the set of 'easy stuff'.