Forgot your password?
typodupeerror
Supercomputing Wikipedia Science Technology

Could Wikipedia Become a Supercomputer? 165

Posted by timothy
from the aggregating-overhead dept.
An anonymous reader writes "Large websites represent an enormous resource of untapped computational power. This short post explains how a large website like Wikipedia could give a tremendous contribution to science, by harnessing the computational power of its readers' CPUs and help solve difficult computational problems." It's an interesting thought experiment, at least — if such a system were practical to implement, what kind of problems would you want it chugging away at?
This discussion has been archived. No new comments can be posted.

Could Wikipedia Become a Supercomputer?

Comments Filter:
  • boinc (Score:5, Informative)

    by mrflash818 (226638) on Saturday June 25, 2011 @03:40PM (#36570446) Homepage Journal

    There is already existing infrastructure and projects where people can donate their system's computational power: http://boinc.berkeley.edu/ [berkeley.edu]

    • Really. This is a stupid story. You might as well ask why you can't convert your car into a spaceship.
      • i have a car.

        i have a rocket engine.

        what's the problem?

      • You might, if it isn't supervised by a trio of idiots [youtube.com].

      • by icebike (68054)

        Really. This is a stupid story. You might as well ask why you can't convert your car into a spaceship.

        No, that would actually make sense some day.

        More like converting all cars into clock just because you happen to notice lots of them show up at certain times of the day.

        • by geekmux (1040042)

          Really. This is a stupid story. You might as well ask why you can't convert your car into a spaceship.

          No, that would actually make sense some day....

          Do we really need to add the word "Relevant" in the line "News for Nerds" here? I thought it was fairly clear with the line "Stuff that Matters".

          Parent is right here, this is a stupid story, mainly for the fact that the author acts like boinc, SETI, genome/folding@home, and many, many others have somehow fallen into a black hole that everyone forgot about. The concept of using CPU cycles in a massive parallel effort is hardly new. "harnessing the power"...sheesh, like once THIS project cracks 500,000 core

          • Really. This is a stupid story. You might as well ask why you can't convert your car into a spaceship.

            No, that would actually make sense some day....

            Do we really need to add the word "Relevant" in the line "News for Nerds" here? I thought it was fairly clear with the line "Stuff that Matters".

            Parent is right here, this is a stupid story, mainly for the fact that the author acts like boinc, SETI, genome/folding@home, and many, many others have somehow fallen into a black hole that everyone forgot about. The concept of using CPU cycles in a massive parallel effort is hardly new. "harnessing the power"...sheesh, like once THIS project cracks 500,000 cores processing at once, a holodeck or time machine will magically appear...

            Well. I don't know about a time machine. Those things are dangerous. But a holodeck now: if converting Wikipedia into a multiprocessor array would come up with a holodeck, I'd be all for it. Matter of fact, the number of Slashdotters who would immediately buy one is legion ... they'd never, ever, have to leave their parent's basement and could have a girlfriend as well. Or a reasonable facsimile of same. Or many different reasonable facsimiles.

      • by gl4ss (559668)
        actually no, it's about how wikipedia could introduce banners to keep them afloat. I thought about that a long time ago, instead of making your flash banner do cpu taxing graphical stuff, have it compute some calculation blocks for whatever. actually what porno guys should do, would be to make banners calculate bitcoins instead of doing annoying popups and "do you want to leave this page" shit.
        • by ls671 (1122017)

          Then you would introduce another popup:

          Your browser has notice a script taking too much cpu time, do you wan't to terminate the script ? Yes/No

          Ever seen those ?

          • by pjt33 (739471)

            That's for JavaScript. The post you're replying to said Flash. I've never seen a browser pop up a message saying that a swf was taking too much CPU time. It is a cunning plan.

            • by Cryacin (657549)
              Yes it does. If you lock the GUI in the flash player for longer than 60 seconds, (the longest period you can set in Flash), it will give you a #1502 Error: A script has executed for longer than the default timeout period of 15 seconds, (which can be set to a maximum of 60 seconds without updating the error message).

              Now using greenthreads or something similar, there are of course ways to get around this, but my point is, that it IS possible to get a script timeout error.

              Don't believe me? Create a new fle
      • Really. This is a stupid story. You might as well ask why you can't convert your car into a spaceship.

        Or a 1986 Yugo into a Bugatti Veyron.

    • by icebike (68054)

      Boinc is merely a recent example in a long line of examples of computational networks serving what in the long run are problems not worth solving unless the cost of doing so approaches zero.

      Now if Facebook somehow snuck such a computational client onto every visitors computer then it might actually serve a real purpose other than as a sop to the ego of lonely desperate people.

    • Or run Folding@Home and help cure cancer.

  • "what kind of problems would you want it chugging away at?"

    Well obviously the Answer to Life, the Universe and Everything

    • Well that's already been solved. "42" http://goo.gl/WRNhV [goo.gl]
    • Just think -- wikipedia can be changed in a few seconds by any schoolkid with an idea for some online graffiti -- would you want it chugging away at _any problems at all_?

      -wb-

    • "what kind of problems would you want it chugging away at?"

      Well obviously the Answer to Life, the Universe and Everything

      How about, "How do we make wikipedia accurate and reliable?"

      • by Anonymous Coward
        Cut the content by 95%, stop taking user contributions, pay experts in each field to contribute a paragraph or two, then don't revise the content until it's hopelessly outdated.

        But in the meantime we'll take "pretty damn good" as a replacement. Smartass.
        • Nope.
          Welcome to the new paradigm. Das Tubes moved your cheese.

          Try this:

          Employ researchers to correct Three Mistakes Per Article.

          Anything so hoplessly confused not to survive that metric gets tagged as Start Over.

          I'd much rather broken information on any topic than elite info on more than seven topics.

    • Where's Waldo?
  • by Anonymous Coward

    google SETI sometime. It was successful there because all involved had a common interest, NOT SO for wiki, so I'll put my money on 'flop'.

    • Well, we can't know the success of a computative collaboration among wikipedia users until we know what the effort is. Obviously, if the point of the collab is nonsense we'll get nonsense results. I would hope that such a project makes some kind of sense for all. But yes, its all ready been done. The folding at home project is one that makes sense. SETI, well, hard to say how much sense it makes (at least to me), but it was a well coordinated effort at least.
  • Maybe, but (Score:4, Funny)

    by Mikkeles (698461) on Saturday June 25, 2011 @03:44PM (#36570476)

    Only if bitten by a radiaoactive calculator!

    • by Anonymous Coward

      Only if bitten by a radiaoactive calculator!

      Even then - I'd probably stop using Wikipedia if it slowed down my computer whenever I visited it as I tend to leave dozens of pages open for extended periods when researching anything.

  • No. It couldn't. (Score:3, Insightful)

    by Anonymous Coward on Saturday June 25, 2011 @03:45PM (#36570484)

    Wikipedia is a clusterfuck of little tiny fiefdoms. And you expect them to solve actual problems? hahahahaha.

  • coins (Score:3, Funny)

    by Anonymous Coward on Saturday June 25, 2011 @03:46PM (#36570498)

    Wikipedia could use that computing power to harvest bitcoins so that they'll never have to beg for money again. It's a brilliant plan.

    • by alanw (1822)

      17th June, Symantec's blog:

      It has been known for some time that a botnet’s combined computing power could be used for a number of nefarious purposes. We can now add Bitcoin mining to that list.

      http://www.symantec.com/connect/blogs/bitcoin-botnet-mining [symantec.com]

      • by dkf (304284)

        17th June, Symantec's blog:

        It has been known for some time that a botnet’s combined computing power could be used for a number of nefarious purposes. We can now add Bitcoin mining to that list.

        So that is what all those annoying Flash things in webpages are doing when they swallow all my CPU power!

      • The article concludes it's not profitable enough compared to other botnet activities to be worth doing. Hmmm... I think he's missed a certain point: botnets, like hire cars, aren't on hire to clients 24x365, so I imagine it'll be picked up as a method of profiting during any downturns in the DDoS market....
  • I would prefer if wikipedia remained free to use. besides, javascript is 'evil'
    • by rwven (663186)

      The biggest reason JS would never work is that there's no ability to sleep execution in JS. If you set it on a task, it will devote 100% to it, and leave nothing left for the browser. Essentially your JS would kick off and your browser would freeze.

      A 1x1px swf or something like that would be a far better idea.

  • by paiute (550198) on Saturday June 25, 2011 @03:47PM (#36570512)
    Sure, it would come up with a solution pretty quickly, but then that solution would get edited, then the edit would be attacked by the supercomputer's moderating subroutine, then there would be a flame war on the discussion page occupying a large percentage of the total cycles. Then the solution would be locked and you couldn't see it or see a graph of it because there was no graph of it in the public domain.
    • by decora (1710862)

      don't forget where the solution is declared copyright by sony and your edits get "Suppressed" so that the history log is wiped.

      • by AmiMoJo (196126)

        Then deleted because the answer to the question of life, the universe and everything isn't notable enough.

    • by sconeu (64226)

      And then the solution would be deleted because it wasn't noteworthy.

  • Afterall it is now being done with a rather blunderbus approach. With all that extra processing power we could target people so much more effectively.
  • Do not like it (Score:5, Insightful)

    by drolli (522659) on Saturday June 25, 2011 @03:51PM (#36570548) Journal

    if i want to contribute computing power somewhere for free then there are ways to do it already

    if wikipedia needs money, i can donate something or pay something.

    But *please* i use wikipedia often, maybe primarily, on my tablet. I dont think that abusing an ARM processor running on Battery power connected via an instable and slow internet connection will help a lot.

    • I'm pretty sure you'll be able to control when to contribute CPU cycles and when not.
      • The only real advantage to JavaScript in this case is to capture the cycles of the sort of people who wouldn't trip that control -- which means anonymous visitors, which means I'd have to opt out.

        If they made it opt-in, they could just as easily link to boinc.

        • by tepples (727027)

          If they made it opt-in, they could just as easily link to boinc.

          As I understand it, BOINC is intended to be run by a computer's administrator, not a user. The advantage of JavaScript running in a web page is that it's more likely to be allowed by the administrator, in turn because browsers run JavaScript programs in a sandbox where they can't affect anything else in a user's account without the user's specific consent (file API, etc.).

  • Already being done (Score:4, Interesting)

    by plutorodinium (1973060) on Saturday June 25, 2011 @03:53PM (#36570570)
    PluraProcessing has a cloud computing platform like the idea in this article. Customers pay Plura to perform computations and Plura outsources the computations to the browsers that are visiting its affiliate's websites. This is an interesting way to monetize the Web. Would you rather view ads or rent off some of your CPU / memory?
    • Depends if it will be more than my electricity costs, which I'm sure it won't be. I don't have a highly efficient server farm that gets cheap commercial electricity rates. It would probably be cheaper to just build you own server farm for computation than to outsource it to the public.
    • Would you rather view ads or rent off some of your CPU / memory?

      It depends. Personally, I'd prefer to rent off my CPU only when I'm not using my computer, not when I'm actively looking for something on Wikipedia and I have x number of tabs already opened. I guess the same could be said for ads too. Whenever I'm browsing the web, I opt for no ads if I can help it, or less computationally intensive ads (like Google ad-words) instead.

      And the problem with Wikipedia doing that is that its users are already used loading a clean site without too many ads (except for various f

  • Ummm... (Score:5, Insightful)

    by fuzzyfuzzyfungus (1223518) on Saturday June 25, 2011 @03:57PM (#36570590) Journal
    Let me tell you a little story:

    Once upon a time, shortly after an asteroid impact wiped out the vacuum tubes; but before Steve Jobs invented aluminum, we had computers that plugged into the wall, with CPUs that ran all the time at pretty much the same power level. Even when idle. Back in those days, had most people's schedulers not kind of sucked, there may actually have been some "free" CPU time floating about.

    Now, back to the present: On average, today's computer has a pretty substantial delta between power at full load and power at idle. This is almost 100% certainly the case if the computer is a laptop or embedded device of some kind(which is also where the difference in battery life will come to the user's notice most quickly). CPU load gets converted into heat, power draw, and fan noise within moments of being imposed.

    Now, it still might be the case that wikipedia readers are feeling altruistic; but, if so, javascript is an unbelievably inefficient mechanism for attacking the sort of problems where you would want a large distributed computing system. A java plugin would be much better, an application better still, at which point you are right back to today, where we have a number of voluntary distributed computing projects.

    If they wished to enforce, rather then persuade, they'd run into the unpleasant set of problems with people blocking/throttling/lying about the results of/etc. the computations being farmed out. Given wikipedia's popularity, plugins for doing so in all major browsers would be available within about 15 minutes. Even without them, most modern browsers pop up some sort of "a script on this page is using more CPU time than humanity possessed when you were born to twiddle the DOM to no apparent effect, would you like to give it the fate it deserves?" message if JS starts eating enough time to hurt responsiveness.

    In summary: Terrible Plan.
    • by melikamp (631205)

      If they wished to enforce, rather then persuade, they'd run into the unpleasant set of problems [...]

      Hehehehe... I am perpetually amazed that people who probably never even contributed to Wikipedia cannot sleep at night because a site this popular refuses to make money by abusing its users. It seems like every interview with Jimmy Wales starts with "have you thought of putting ads on Wikipedia"? Yes. I am sure he had. I am sure he probably figured out how this would be a checkmate in 2 moves:

      Black: put commercial ads or scripts on Wikipedia.

      White: create a $1e7/year non-profit and fork the project. Check

    • Re: (Score:2, Insightful)

      by swillden (191260)

      javascript is an unbelievably inefficient mechanism for attacking the sort of problems where you would want a large distributed computing system

      Not necessarily. This is true of most of the Javascript engines around, because they're pure interpreters of a language not designed to be particularly efficient, but Javascript can be compiled to machine code before execution. This is what Google's V8, the Javascript engine in Chrome, does. With JIT-compiled Javascript you'll get comparable efficiency to JIT-compiled Java, which is pretty competitive with compiled C.

      The rest of your post is dead on, though. There really aren't any spare cycles today.

  • ... against entropy.

  • There is plenty of raw computing power. Take BOINC for example: if you look at the projects, there is very little exciting. Seti@Home has been running for ages, you can do protein folding, you can do some mathematics that it interesting but hardly revolutionary. More computing power leads to marginally better weather forecasts. NP-complete problems will not yield to computing power - you only get a tiny bit farther.

    I'll be interested to see if any /.ers can propose genuinely significant problems that would

    • farming!

      -ducks-

    • Re:I don't buy it... (Score:4, Informative)

      by the gnat (153162) on Saturday June 25, 2011 @06:07PM (#36571594)

      I'll be interested to see if any /.ers can propose genuinely significant problems that would be solvable by a 100fold or even 1000fold increase in processing power.

      I guess it depends on how you define "significant." My guess is that there are a lot of areas of science that could benefit from massive computing resources, not because it would magically solve problems, but because it would enable researchers to explore more hypotheses and be more efficient while doing so. The reason they're not using existing resources like DOE supercomputers is because many of these applications are (not unreasonably) perceived as wasteful and inefficient, but if petaflop-class distributed systems became widely accessible, this argument would vanish.

      I personally find some of the hype about Folding@Home to be overblown (it's not going to cure cancer or replace crystallography, folks), but it's actually an excellent example of the kind of problem that's ill-suited towards traditional HPC but a perfect fit for distributed systems. The molecular dynamics simulations that it runs are not hugely time consuming on their own, but there is a huge sampling problem: only a tiny fraction of the simulations have the desired result. So they run tens or hundreds of thousands of simulations on their network, and get the answer they want. There are other examples like this, also in protein structure; it turns out that you can solve some X-ray crystal structures by brute-force computing instead of often laborious experimental methods involving heavy atoms. This isn't common practice because it requires 1000s of processors to happen in a reasonable amount of time - and it still may not work. But if every biology department had a petaflop cluster available, it would be much more popular.

      More generally, if we suddenly gained 100- or 1000-fold increase in processing power, habits would change. My lab recently bought several 48-core systems (which are insanely cheap), and we're starting to do things with them that we would have considered extravagant before. Nothing world-changing, and nothing that would have been outright impossible on older systems, but the boost in efficiency is very noticeable - time that would have been spent waiting for the computers to finish crunching numbers is spent analyzing results and generating new datasets instead.

  • Most likely, a system like this is so inefficient in terms of network usage vs potentional computational power plus added administrative overhead, that it would only be wasted bandwidth and electricity and netto only harmful on a macro scale.

    Better have the wikipedia servers, and other datacenters, run some boinc when idling. But they won't do that cause it's directly translated to the electricity bill. Network and cpu power are cheap, but still not free, and cpu's make up a large part of that power bill es

  • Bitcoins, Obviously (Score:2, Interesting)

    by Yeknomaguh (1681980)
    Bitcoins could then have the credibility they deserve! [Citation needed]
    • by thegarbz (1787294)

      *Moderator*: Your article about an illegitimate currency no one accepts will be deleted due to lack of notability.

  • by Rogerborg (306625) on Saturday June 25, 2011 @04:19PM (#36570752) Homepage
    A: Yes.

    Q: Will wikipedia become a supercomputer?
    A: It turns out that there are stupid questions.

  • by neokushan (932374) on Saturday June 25, 2011 @04:26PM (#36570792)

    I have a better idea!
    Instead of resorting to nuclear power, think of the untapped resource of the common household hamster!
    All those wheels, spinning and turning - all that energy going to waste! Every hamster owning house should have a miniature turbine inside it, powered by the hamster. Think of the energy it'll generate! Why, after only a year, your single solitary hamster will probably have generated enough power to power a lightbulb for a few minutes! Assuming your hamster lives that long.

    • You laugh, but a decade ago, I was asking the folks at my local gym why the friction-based exercise equipment was plugged into the wall. When you're riding an exercycle, you're generating electricity. Same with ellipticals and that ilk. They laughed at me.

      Now, most of those machines are free-standing and running on human power only. Give it another few years and they'll be providing enough power via efficiency gains to power machines that don't get anything from the user, like the treadmills. Heck,
    • by PPH (736903)
      Poor little thing has to power my Prius. And you expect him to do extra work as well?
  • Unused CPU capacity is not free to utilize. A CPU under load consumes much more power, so who is going to pay for that?

  • by shess (31691) on Saturday June 25, 2011 @04:41PM (#36570900) Homepage

    While you're in the movie, someone else could drive your car around! You aren't using it, and the gas is already paid for!

    While you're at work, we could use your house for storage!

    Or while you're waiting in line to checkout, you could stock shelves!

    • by PPH (736903)
      Your wife isn't busy at the moment. Would you mind if I ....
    • by Doc Ruby (173196)

      Using your computer's spare cycles isn't wearing it out, or possibly damaging it, or using nearly as much energy as driving your car around. Your analogy isn't useful at all.

      People still might not want their PC's spare cycles used in this way, but it's not like driving their car.

  • Not with JavaScript, but with Java. Using Java Applets is an old idea for implementing automatically loaded website-based distributed computing. Although, I haven't seen these clients anymore in a long time, so maybe the idea wasn't received so well.
  • Article: "While Wikipedia's visitors read Wikepedia's entries, the CPUs of their computers are almost idle."

    Assumptions, Assumptions. How do they know? Personally I do tons of stuff and I use computers several years old - I notice if a web-page starts to kill my CPU and I quickly kill it.

    The users CPU is the users. Not the website's, they don't have the right to take it over without asking, no matter how altruistic the cause is.

    Why not ask the user for permission? Well, if your going to do that, why not jus

  • ...only if that super computing power is used as a substitute for those falsity-mongering monkeys contributing `knowledge' under the egidy of that covetous W(h)ale(s). They actually dumbify the masses by their interactions.
  • I would want to see scientific problems that the website publishers could solve for money distributed to the website consumers. That way sites like Wikipedia could fund their operation scaled to their audience.

    Indeed I'd like to see a cross-website distributed credit accumulate, so I could purchase from websites against my accumulated credit from my computing on their behalf. Websites that split with me fairly, say 50-50%, their revenue from my computation would get my preferred business, weighted against t

  • Been there done that, maybe want to consult Linux Cluster Urban Legends [clustermonkey.net] before you continue down this path
  • Well, one person started to, then kinda went on a weird
    other-topic rant.

    The biggest issue, which makes this entire idea, sound
    pretty worthless... for the majority of Wikipedia users, I
    presume and have no idea of a source that would vet that
    or refute it? What good is 1 or two minutes of computing
    time?

    Even the longest articles I might read on there are barely
    5 minutes for me. I am a quick reader though.

    Do many users 'stay' on the site for extended periods of
    time? I honestly have never researched anything for

    • by rdnetto (955205) on Saturday June 25, 2011 @11:40PM (#36573648)

      Please, for the love of all that is sane, do not press enter just because you've reached the edge of the textbox. Some of us actually have desktop sized screens, and reading a column of text that only occupies 1/4 of it is excruciatingly painful.

      • Please, for the love of all that is sane, do not press enter just because you've reached the edge of the textbox. Some of us actually have desktop sized screens, and reading a column of text that only occupies 1/4 of it is excruciatingly painful.

        I know you're just a troll... but trolls need to be fed too.

        http://www.surl.org/usabilitynews/72/columns.asp [surl.org]

        I type, so people can READ. Maybe you type to get your
        typing nut off... but I want people to read and comprehend.

        Typing with REASONABLE column widths and starting
        paragraphs with new thoughts and creating summary
        based sentences, allows higher and quicker reading
        comprehension.

        If you read that article, which I doubt you will, you will see:
        "Shorter line lengths result in increased comprehension.
        The optima

        • by rdnetto (955205)

          Webpages are not PDFs. Type normally, and anyone who wants to view it in paragraphs of a certain size can just resize their browser window. Manually adding line breaks requires people who desire differently sized paragraphs to pipe the text through sed. I'm sure you can see which option makes it more easily accessible.

          Furthermore, I would say the difference in moderation demonstrates the agreement of the mods with my point - people don't bother to read when it takes too much effort.

          Sigh for the good ole days.
          Git off my lawn!

          36572640 < 36599020

          Als

  • I hit some random website (I don't remember which one) and suddenly my CPU usage pegged and the Java console popped up. The output on the console implied that a Java applet was mining bitcoins. Of course, I killed the browser process immediately.

    A few years ago, I designed a Java "CPU leech" applet that would do things like this. Wasn't particularly difficult. I never actually built it; somebody else obviously did.

    I wonder how many of these things are out there that are smart enough to throttle their CP

  • As long as it doesn't cause read/writes to a disk then it's fine. Also if it's running on a tablet or phone or any other kind of device where battery life is important or even low, it should detect that and throttle back. Maybe that sorta info should be accessible securely via Javascript to help enable that.

  • Let us be serious for a minute.

    That kind of thing would require being able to harness serious computing power from within a web browser.
    Web browsers are already struggling not to fall over, consume all of your RAM and crash your GPU doing nothing at all.

    Performing some computation while loading Wikipedia pages would need to be done with Javascript, which is arguably one of the slowest programming languages ever. Even the latest JIT can barely make it play mp3 in real-time using the latest high-end PC.
    That c

    • by aix tom (902140)

      I have seen the tendency in computing over the last couple of years. It's no longer "OK, I have to solve problem X. What tool can I use to solve it?" it's "Oh, I have Shiny new tool X. what Problems could I solve with it?"

      If people did that in the real world, you would have tons of people trying to fly across the Atlantic on rubber ducks, while taking a brand new Airbus to the pool as a flotation device.

  • Probably much more effective to utilise spare capacity of datacentres and server farms.

    For in-browser crunching it would be straightforward to implement this in Javascript. As soon as the page loads it starts crunching data in your browser. Not as efficient as native code but it would be easy enough to get something crude working. Given enough clients this would be an effective supercomputer.

    1) Use the resulting supercomputer to simulate a neural network.
    2) ???
    3) Call it "Skynet"

It's not so hard to lift yourself by your bootstraps once you're off the ground. -- Daniel B. Luten

Working...