Follow Slashdot blog updates by subscribing to our blog RSS feed

 


Forgot your password?
Close
typodupeerror
×
Software Math The Internet

The Gradual Public Awareness of the Might of Algorithms 169

Posted by Zonk from the interlocking-gears dept.
Soylent Mauve writes "The trend toward data- and algorithm-driven tuning of business operations has gotten a lot of attention recently — check out the recent articles in the New York Times and the Economist. It looks like computer scientists, especially those with machine learning training, are getting their day in the sun. From the NYT piece: 'It was the Internet that stripped the word of its innocence. Algorithms, as closely guarded as state secrets, buy and sell stocks and mortgage-backed securities, sometimes with a dispassionate zeal that crashes markets. Algorithms promise to find the news that fits you, and even your perfect mate. You can't visit Amazon without being confronted with a list of books and other products that the Great Algoritmi recommends. Its intuitions, of course, are just calculations -- given enough time they could be carried out with stones. But when so much data is processed so rapidly, the effect is oracular and almost opaque.'"
This discussion has been archived. No new comments can be posted.

The Gradual Public Awareness of the Might of Algorithms More

The Gradual Public Awareness of the Might of Algorithms

Comments Filter:

  • Oracular, opaque... (Score:2, Insightful)

    by Anonymous Coward
    and often hilarious or silly. People already trust computers too much.

    • Re: (Score:3, Funny)

      by Paradigm_Complex ( 968558 )
      I checked sources online, you're wrong according too... wait, crap.

    • Re: (Score:3, Funny)

      by morgan_greywolf ( 835522 )
      Absolutely. I belong to several Yahoo and Google Groups geared at the neopagan crowd, and because the groups are categorized as 'religious' groups, the advertising always contains advertisements for 'End Times' books and appeals to join the United Methodist Church, etc. Then again, maybe this the algorithms are doing just what they're supposed to do ... :)

      • But when so much data is processed so rapidly, the effect is oracular and almost opaque.

        As you've demonstrated, the "oracular" part is badly mistaken.

        Amazon almost NEVER guesses something I'd buy.

        If I buy a new DVD, I am instantly bombarded with ads for EVERY new DVD. I buy the new Terry Pratchett book and I'm bombarded with EVERY book by him or co-authored by him or licensed by him or whatever. I don't want derivatives.

        I picked up the "V" comic book (graphic novel) and now I'm bombarded with every comic bo

        • Re:No, I think you were right the first time. (Score:5, Informative)

          by Chmcginn ( 201645 ) * on Sunday September 23, 2007 @02:32PM (#20721261) Journal

          I buy the new Terry Pratchett book and I'm bombarded with EVERY book by him or co-authored by him or licensed by him or whatever. I don't want derivatives.

          My favorite is getting Amazon recommendations for books I've already bought... through Amazon.

          I often find myself saying "Ah, yes, I just bought the hardcover version of that book last year, now I should go out and get the paperback, the second edition with a few minor spelling corrections, etc, etc."

          Or something.

        • "Amazon almost NEVER guesses something I'd buy."

          That's because they don't have enough data to profile you. If amazon had camera's in your home, kept track of your entire life, they'd see the pattern of your ego and behaviour and it would get MUCH MUCH better very quickly.

          Algorithms CAN predict things very accurately PROVIDED they are given enough data, and with human beings I would imagine that it is only STATISTICAL prediction, since one cannot yet calculate what you will notice or enter into your awaren
        • I find it unsettling to categorise people in any way. I find that once you've categorised a person it's very difficult for them to exit the box they've been put in, and all other dealings with that person tend to relate only to that categorisation, forcing them back into the box.

          To find categorisation of humans done by algorithim, often a poorly-proven one, is returning IT back to the era of dehumanising machinery in my perception of it (a difficult place to be, since I've been in IT since 1969). Also b

    • People already trust computers too much.

      No. You can always trust the answer a computer gives you to be correct.

      Its the input data [wikipedia.org] that I'm worried about.

  • Slightly O.T. (Score:5, Informative)

    by gardyloo ( 512791 ) on Sunday September 23, 2007 @12:34PM (#20720389)
    I just (a few minutes ago) found this free PDF book about algorithms (written for the undergrad-level student). It's pretty good: http://beust.com/algorithms.pdf [beust.com]

  • Said one computer scientist (Score:5, Funny)

    by Nicholas Bishop ( 1004153 ) on Sunday September 23, 2007 @12:37PM (#20720413)
    Said one computer scientist getting his day in the sun:

    "I'm melting, I'm melting!"

  • This Just In (Score:5, Funny)

    by the eric conspiracy ( 20178 ) on Sunday September 23, 2007 @12:38PM (#20720423)
    Math is a really really powerful tool.

    • Re: (Score:1, Troll)

      by nih ( 411096 )
      Maths is a really really powerful tool.

      there American, fixed it for you.

      • Re: (Score:2)

        by Enonu ( 129798 )
        If you identify one "Math" for me, then I'll identify one snow for you.

        • If you identify one "Math" for me, then I'll identify one snow for you.
          When you identify one single "Mathematic" for me.
        • If you identify one "Math" for me, then I'll identify one snow for you. If you explain to me why you think "maths" derives from "math" and not "mathematics".

      • there American, fixed it for you.

        This is an American site, you silly little British girlie - man.

      • Well, finish the job:

        Maths are a really really powerful tool.
        • Maths are a really really powerful tool. Maths is not a plural of math, it's a different contraction of mathematics, which is a singular noun, so "is" is correct.
          • Worse yet, mathematics, a singular noun has as its root word the Latin neuter plural mathematica.

            What a muddled language the British inflicted on us.

      • Maths are a really really powerful tool.
        Sheesh. When being a pretentious smartass, take care to keep your own grammar correct.

        Oh, and this is an American site. You're wrong in the first place.

      • Re: (Score:2)

        by treeves ( 963993 )
        Just asking, but if "Maths" is plural of something, shouldn't it be "Maths are really powerful tools".

    • Re:This Just In (Score:5, Insightful)

      by GuyMannDude ( 574364 ) on Sunday September 23, 2007 @02:05PM (#20721085) Journal

      Math is a really really powerful tool.

      While that may be obvious for slashdot readers, it's news to the general public. I remember an endless number of conversations, even as recent as a few years ago, in which people would ask "Can you do anything with that degree other than teach?" upon learning that I was a mathematician. I think it's great that the public is starting to realize that math makes the world go around. God forbid, the gradual public awareness of the power of math might even lead to kids wanting to pay attention in class. While there are drawbacks to this (e.g., the deluge of college kids taking business-oriented mathematics programs with the expectation of a six-figure salary once they graduate), I'm generally happy to see math and computer science get their days in the sun.

      GMD

      • with the expectation of a six-figure salary once they graduate
        Aspiring to a six figure salary is an indomitable human right. Ben Bernanke will go down in history as a champion campaigner of human rights, right alongside compatriate economic guru, Robert Mugabe.

      • Re: (Score:2)

        by raddan ( 519638 )
        I wish that there were better math teachers. In high school, my AP Calculus teacher was unable to provide an answer to the question: "What is this good for?" His inability to answer discouraged a lot of people. It just seemed like an arbitrarily hard obstacle for us.

        This kind of thing is unacceptable. Now that I've taken college-level Calculus and beyond, I can say-- mathematics is really cool, and very useful! I can think of hundreds of things that Calculus is useful for. After all, Calculus was i

  • Boy They're Slow (Score:4, Insightful)

    by PingPongBoy ( 303994 ) on Sunday September 23, 2007 @12:42PM (#20720455)
    Whereas algorithms are instantly aware of their own prowess.

    Is management starting to wonder (again) whether a computer can really do a better job making the important decisions? But can it yet? There is so much data that needs to be acquired in order to return a meaningful answer.

    Some of the most powerful organizations are probably making deals to combine as many databases as possible. Interesting to see (if they would let us see) if that will give them the answers they're looking for. As data acquisition becomes more accurate and less expensive, there will be less privacy but more creative computer output, a trade-off in the value of personal information leading to the possible marginalization of humanity.
    • Humanity won't be marginalized. Who will be marginalized are the people with no power, money, or votes. The power of the elite will be enhanced by this. Now I don't need to employ stasi to watch everyone all the time, I can use a computer that doesn't have the brains to be anything but a loyal servant. I don't need to risk soldiers turning their guns on me when sent to quell a protest if my loyal robot drones will efficiently and painfully kill everyone there. The history of technology and warfare is pretty

    • Re: (Score:2, Interesting)

      by c_sd_m ( 995261 )

      Is management starting to wonder (again) whether a computer can really do a better job making the important decisions? But can it yet? There is so much data that needs to be acquired in order to return a meaningful answer.

      If they're foolish, sure they hope computers can make better decisions. If they aren't complete fools they realize that computers can provide analytical support for decisions. For example, algorithms can evaluate more potential alternatives, generate potentially good alternatives that

    • Re: (Score:2)

      by Boronx ( 228853 )
      ...leading to the possible marginalization of humanity.

      The stories of complex biological systems are of increasing power and command of communities of smaller systems at the expense of independence of the smaller systems, to the point where we start thinking of the community as a single unit.

      I don't see any reason why the evolution of human society won't follow the same pattern. Those cultures which demand and receive submission to the public good will far surpass (already have?) those that do not.

      From the

  • The joy of algorithms (Score:5, Interesting)

    by drgonzo59 ( 747139 ) on Sunday September 23, 2007 @12:43PM (#20720465)
    Yes, finally, the algorithms are making a comeback. Up until now we just randomly banged on our keyboards until something came out. Now we have algorithms -- a plan that we follow step by step. Wow.


    But seriously, a food recipe is an algorithm for all general purposes. All these people are saying is that the machine learning algorithms and match peoples' personalities and buy stock are too complicated for the average Joe Programmer Wannabe and look more or less like a black box. (which if they employ neural networks, instead of say SVN, they are actually black boxes even for the author who wrote it...).

    • Re: (Score:3, Informative)

      by Stochastism ( 1040102 )
      Did you mean SVM? I think the quadratic programming optimizer used for SVM training would count as a black-box, even to most of the SVM crowd ;) And don't get me started on Gaussian Processes.

      Machine learning is supposed to *look* like magic. It's supposed to behave like a black box with just one or two knobs on it. When -- and this is unfortunatley almost always -- it doesn't, then it's not the machine learing doing the work, it's the programmer. In this case I can forgive Joe Wannabe for tearing his hair
    • Amen! Used to, if I wanted something from the store, I'd randomly do things and at some point I ended up at the store and had my item (good luck getting back home though). Now that I've learned about algorithms, I can walk there, get the item, then walk home, all in less than an hour. Algorithms have saved me so much time! What has the world done without algorithms all this time?

    • Re: (Score:2)

      by Hatta ( 162192 )
      But seriously, a food recipe is an algorithm for all general purposes.

      Everything is an algorithm. That's kind of the point.

    • But seriously, a food recipe is an algorithm for all general purposes.

      Not really. It's a popular analogy to give to people without a clue, but it also gives them the wrong impression. Food recipes don't have the precision of an algorithm: Cooks expect to vary ingredients slightly, vary proportions slightly, vary cooking times and heat application slightly, compensate for earlier imbalances in seasonings, etc. In other words, a recipe is merely an indication and the important element is the cook, who e

  • Heuristics are not the same as algorithms (Score:3, Insightful)

    by pedantic bore ( 740196 ) on Sunday September 23, 2007 @12:55PM (#20720545)

    Sheesh! Someone needs to spend some time with a dictionary.

    If only we could have a gradual (or sudden) awareness of the power of heuristics and modeling ...

    • maybe not as beautiful as 'clasic' ones, but algorithms indeed. Something like shapes, you know, 'clasic' algorithms (ie: sort) are somewhat like circles (simple formulaes) but real objects (ie: leafs) are extremely complex formulaes only approximated by fractals and with a lot of 'heuristics' in it.

      • Re:Heuristics ARE algorithms (Score:4, Insightful)

        by Chmcginn ( 201645 ) * on Sunday September 23, 2007 @01:41PM (#20720933) Journal
        GP: "Heuristics are not the same as algorithms"

        P: "Heuristics ARE algorithms"

        Both of these statements can be true. (Depending on the exact meaning of the GP.) For instance:

        Humans are not the same as animals.

        Humans are animals.

        A more exact statement than either is that heuristics are a subset of algorithms, as humans are a subset of animals.

      • Re:Heuristics ARE algorithms (Score:5, Interesting)

        by pedantic bore ( 740196 ) on Sunday September 23, 2007 @01:58PM (#20721045)

        That's an elegant metaphor, but someone has misled you.

        An algorithm is a precise specification of a process whose outcome is defined by the initial conditions. To cite your example, quicksort is an algorithm -- the outcome of the sorting process is well defined, given the inputs.

        But typical implementations of quicksort use a heuristic to choose the pivot element -- median of three, media of five, middle element, etc. These are heuristics because their goal is to choose the median value, but they can't make any guarantee that it will find the median. They can't even guarantee that they will find a good value. In fact, they generally don't even consider all of their inputs! They could choose bad values every time... but on average they don't, and quicksort is fast.

        Another way of looking at it is that if an algorithm is correct, it will produce a correct answer for all valid inputs. A heuristic might produce incorrect answers for valid inputs, but it's correct often enough so that it might still be worth using -- especially if a correct algorithm is not known.

        You may point out that randomized algorithms have a similar property -- but the difference is that with randomized algorithms the probability of error can be made arbitrarily small. With heuristics, there's no telling.

        • Heuristics are always algorithms and they always produce accurate results that depend on their input.

          Heuristics are only vague in the sense, that they don't ask the full question, their result is correct answer for algorithm, but not necessarily the answer for the full un-asked question.

          So heuristics are inaccurate question, NOT inaccurate results.
        • A heuristic is still a type of algorithm, which is simply a recipe for achieving a result. You seem to be stating that that an algorithm can only be a particular KIND of recipe, which is simply incorrect. The definition of "heuristics" is limited to particular types of procedures, but the definition of "algorithm" is not.

          Further, several posters here are simply incorrect about something else. Contrary to what has been stated several times, neither algorithms or heuristics are "guaranteed" to produce corr

          • OK, I can't upload HoTCS, and I doubt you have a copy sitt, but the following links might be useful.

            • I read each of your links, and my answer is the same.

              Heuristics is (this is a rough definition, but it serves) a way, or procedure, for how to find an answer to some hypothetical problem. An algorithm, on the other hand, is more of an IMPLEMENTATION of the solution that was found via heuristics; i.e., a fixed set of rules for achieving a specific result.

              I could write an "algorithm" that specifically describes how to find the solution of a problem via a specific set of "heuristics". In which case, the

              • Re: (Score:2)

                by rjh ( 40933 )
                I almost agree with you.

                First, please see some comments I made a couple of levels up, outlining the definition of an algorithm according to Don Knuth, Thomas Cormen, Charles Leiserson, Ron Rivest and Cliff Stein, all of whom are world-class experts in algorithms.

                The performance of an algorithm is determined by the space and time required for its execution. The Traveling Salesman Problem is an example: it requires a very small amount of space but a factorial amount of time. This is absolutely ridiculous.
                • Therefore, just as you state, an heuristic is (can be) an example of an algorithm, but an algorithm is not necessarily a heuristic. Q.E.D.

                  You can't have it both ways.

                  • Re: (Score:2)

                    by rjh ( 40933 )
                    More to the point, whether something is a heuristic is a subjective decision based on the subjective value we assign to its approximation. Whether something is an algorithm is an objective assessment based on mathematically demonstrable facts.

                    Subjective versus objective. The two concepts are orthogonal to each other. As an example of a heuristic that's not an algorithm, if I believe that astrology is an effective tool for deciding what to do, then astrology is a heuristic. It's not an algorithm.
                    • I agree with what you say. A heuristic need not be an algorithm. What I stated earlier was "IF" one could be said to be a subset of another (in some cases that is valid), then the heuristic must be a subset of algorithms, not the other way around.

                      But certainly there are heuristics that are not algorithms. There is a certain amount of overlap but neither entirely contains the other.

        • I'm a graduate student in CS right now. One of the things I'm researching is stochastic approximation heuristics. Without any argument, these are algorithms. They have to be algorithms, or else the Church-Turing Thesis doesn't apply and we wouldn't be able to have computers do them at all.

          An algorithm is, broadly speaking, a terminating sequence of deterministic steps that effectively derives outputs from provided inputs. But don't believe me--after all, I'm just a random guy on Slashdot. But maybe Co

          • So now that we've got a decent definition of "algorithm", one that's approved by five of the brightest lights in computer science
            THERE ARE FOUR LIGHTS

            /So sorry... couldn't help myself...

    • Re: (Score:2)

      by sholden ( 12227 )
      All heuristics are algorithm (assuming the computer science definition of heuristic). Not all algorithms are heuristics though (sometimes we get to have our cake and eat it too).
  • You might as well argue about public awareness of the power of "recipes" or "formulas", because that is all algorithms are. Businesses were run by algorithm long before the advent of the computer.

    This is just silly. Someone is not comparing apples to oranges, but calling apples oranges. That does not make them so.

  • The Importance of a CS Degree (Score:5, Insightful)

    by Enonu ( 129798 ) on Sunday September 23, 2007 @01:32PM (#20720875)
    This is one of the reason why getting a CS degree is important, despite what the ignorant masses say in the IT industry. Sure writing lame CRUD applications will satisfy your average customer's needs, but sophisticated algorithms are what provide value beyond a simple shopping cart.

    If you're still entrenched in the thought that a CS degree "isn't needed for what I do," then let me propose a somewhat common problem. Suppose your client wants the built in reporting in your web application to minimize the amount of noise introduced by users who forget their password and create a new account rather than resetting it. It's up to you to write code to detect these duplicate accounts. How do you begin doing this beyond simple string comparisons?

    • Re: (Score:2, Insightful)

      by neonfreon ( 850801 )
      Why would anyone ever do this? "Excessive noise"? Oh, you mean more orders? Last time I checked, more entries in the database never hurt anyone (not like every user is going to create duplicate accounts to the point where you're running out of resources, user records are tiny anyhow). Writing some 'intelligent' algorithm to detect duplicate accounts will invariably lead to marking legitimately separate user's accounts as duplicates and eliminating business.

      Ahh, but experience matters too..
      • Over the years I've never missed the degree I never got, but I have caught people with CS degrees doing bonehead things nobody who ever tried to do animation on a Commodore 64 would be dumb enough to try. Not knowing about floating point rounding errors comes up all the time, and is especially nasty when the pennies stop adding up right in the business math. One person I know at a large manufacturing concern insists that you should look for people with computer engineering degrees, because they are at lea
        • A CS degree really should teach basic computer architecture. If you are meeting CS grads who dont know "how the machine works" then that is more of a problem with the school that they went to.

          It seems like since CS covers such a broad range of stuff universities are constantly trying to remove material in order to make the degree easier to obtain. If they arent dumbing down the architecture component of the degree they are removing theory, design, or something else that is perceived as being difficult. I

          • Re: (Score:2)

            by dkf ( 304284 )

            It seems like since CS covers such a broad range of stuff some, rubbishy universities are constantly trying to remove material in order to make the degree easier to obtain.
            There, fixed that for you.
            • I dont know about that, I attended a top 5 engineering school and I saw the material in the CS department getting dumbed down considerably. I have friends that have attended other good schools who say the same thing about their CS departments as well. It isnt just "rubbishy" universities that are doing this.
    • Well its the difference between and electrical engineer and an electrician.

      You don't want to have an electrical engineer to come into your store to pull cables and try to remember all the building codes.
      You don't want a regular electrician to design the circuit board that it going to be used as your circuit breaker in your store's back room.

      I mean... You don't need a CS grad developing your web page, but you hope that a CS grad developed the operating system the web page runs on.

    • Re:The Importance of a CS Degree (Score:4, Insightful)

      by CodeBuster ( 516420 ) on Sunday September 23, 2007 @08:23PM (#20723585)
      How do you begin doing this beyond simple string comparisons?

      It is also useful to realize that just because one can does not mean that one should, especially when the cost of an error is high. There is a tendency, sometimes, among the computer scientists towards too much cleverness, particularly in algorithms, when something much simpler and more reliable would have been better. I cannot tell you how many times bad assumptions about automated processes and the algorithms which control them have lead to inappropriate behavior and blown user expectations under the worst possible conditions. The real world is not the same as the CS labs in your algorithms course and the simpler solution often has much to recommend itself over the efficient and elegant, but hopelessly complex and slightly unreliable algorithm that one learns in the AI courses during their university CS education.

      For example, suppose that your online banking application assumes that you really do want that regular payment upon receipt to go through automatically, because that is how it has happened before, when in fact you, the user, know that a one time payment for an unrelated expense, which has not yet been posted but will be shortly, must be made first. The automated agent makes the deduction for the regular payment automatically while the one time payment, which goes through several days later, is unexpected and overdraws the account. The user curses the system for being too "clever" instead of just carrying out his instructions. Cancel or allow?
  • that the Method of Moments was being applied to economic data. I always thought it was an EM simulation tool, but the theory is generally applicable.
  • Yeah, I'm just waiting for "algorithms" to replace "on the ground" as the next overused buzzword in the media. "We asked General Petraeus about his algorithm for winning the war in Iraq" "Algorithmically, Bob, it seems the Steelers are unbeatable for this year's Superbowl" "That's right, Jane, it looks like mid-length skirts are the algorithm for fashion success this year" It's gonna be great!

  • An underclass? (Score:2, Interesting)

    by Anonymous Coward
    Any sufficiently advanced technology is indistinguishable from magic.
    Arthur C. Clarke, "Profiles of The Future", 1961 (Clarke's third law)

    just how close are we to having this statement be 'real' to a large majority of people on this planet? discounting any second or third world countries, how many people in first world countries would consider the 'oracular' nature of an algorithm to be 'magical'?
    the education system spread throughout the world is creating an over and under soci

    • 'i pray to you lord skynet, pls water my crops on the back 40!!!11 here is a sacrifice to your computations.....'
      Brawndo: It's Got What Plants Crave!
    • As opposed to smart guys like you, who, placed alone on an island with sufficient natural resources, could single-handedly recreate all of modern civilization?

      None of us thoroughly understands the world we live in. The amount you could learn in a single lifetime is only a minuscule fraction.

  • Idiotic (Score:2, Funny)

    by sirdisc ( 988740 )
    This is completely idiotic. All logic works based on algorithms, whether it's in your head or in a computer. Only a monkey or someone with an agenda would write such article. "The Gradual Public Awareness of the Might of Algorithms" eh???????????????????
  • I'm getting my day in the sun? What did I do wrong? I'm sorry! I won't do it again, please let me back inside! I'm getting a tan. Help!

  • I applied for an NSF fellowship last year when applying to grad. school. One of the reviewers essentially torpedoed my otherwise well-received application with a 2/5 on "broader impacts" and the following one-line comment:

    Why study algorithms anyway? It can't be applied!

    The reviewer was a particularly clueless example, but it illustrates that even people within the field still judge algorithms of dubious use. It would have probably been much nicer for my own study if this article came out last year, in a

    • Re: (Score:2)

      by gatkinso ( 15975 )
      I find it hard to believe that you could not successfully appeal that.
      • I could have probably tried (I'm not even sure they had an appeal process; I don't believe I was told of one if there was), but I was offered a fellowship by my graduate school anyway. They apparently had a whole theory lab which just closed down the semester I enrolled, so I was more or less pushed into (you guessed it) ML once I was stuck there.

        Clueless reviewers (or, generalizing, clueless gatekeepers) are a fact of life, especially in science. It's best to just get used to them as early as possible.
  • I don't think Amazon et al are too concerned with rigorous Big O notation and nonrandomized input optimization.

    Maybe I am splitting hairs, but I suspect these folks are occaisonally *applying* Cormen, Leiserson, Rivest, and Stein rather than collaborating with them.
  • The point of the art in NYTimes was that computer systems [in which we embed algorithms] have turned out, after enough years of cost reducing hardware, speeding up communications and harnessing that power to consumer-friendly uses, to be more potent and capable of symbiotic intelligence than even Turing might have expected. By themselves, the computers are only peer-level players in human activities when you see them in Sci-Fi.

Slashdot Top Deals

C'est magnifique, mais ce n'est pas l'Informatique. -- Bosquet [on seeing the IBM 4341]

Close