Follow Slashdot blog updates by subscribing to our blog RSS feed

 


Forgot your password?
Close
typodupeerror
×
Science

SETI@home: Research on the Research 50

Posted by Hemos from the looking-behind-the-curtain dept.
Officer Larry writes "A professor and a graduate student at the University of South Carolina just posted a study into SETI@home work unit completion times. They came up with some pretty interesting results. It stems off Team Lamb Chop's work, but these guys are interested in how much variation there is in completion times if the same work unit is analyzed over and over again on the same system. It looks like they have other studies in the hopper. Warning: there looks to be quite a bit of statistics in this study..."
This discussion has been archived. No new comments can be posted.

SETI@home: Research on the Research More

SETI@home: Research on the Research

Comments Filter:

  • Linux wussy! (Score:1)

    by Anonymous Coward
    Any time Linux beats any other OS in anything, there are big screaming headlines, WOOHOO! CHECK THIS OUT! Linux best for ____ or whatever. They get beaten, and there is no summary of the results in the story summary! Virtually every other story summary has the main point of the article listed, but when Linux loses, you try to gloss over that fact. Clearly, Linux got its butt kicked and there are 'many statistics.' Hah.

  • TLC Site down - Here is mirror (Score:1)

    by Anonymous Coward
    Mirror is over at http://tlc.hagabard.com/ [hagabard.com] for now. The entire site may not be there yet. Some bits (sadly the benchmarking) are NOT there and I don't know if there is another copy of the data that will be put up on my mirror or not.

    Sadly, /. cannot take credit for killing tlc.com, it's been down for a spell already. Come kill my box instead.

    -Hagbard (/. - Obelisk) Don't have my login at work... doh.

  • Re:Statistics and Analysis? (Score:1)

    by Anonymous Coward
    They did not use RH Linux.
  • The discovery of extraterrestrial life will likely revolutionize life on earth, probably ending war here so we can work together to wage war on aliens. But the aliens will be so much more advanced that we won't even try. I know, governments wage war, not people.
  • The built the DES cracker for less than $300k. But to what end? SETI@Home has excess processing power now - they don't need faster clients. The Ego is the only driving factor to churn out faster SETI units.
  • Since SETI is most likely purely CPU intensive, there is very little the OS can do to make it faster or slower (apart from deliberately getting in the way, of course). On the other hand, the compiler makes a tremendous difference, and so do the compilation options. Since we do not know which compiler was involved, and which options it was given, the benchmark is fairly useless.
  • What a great slashdot parody article! I mean, the irony of complaining about the benchmarking of Windows NT in a configuration where is is still faster than a Linux distribution. Or is the author poking fun at the idea of a Linux person trashing NT because the performace scales up as you add memory?

  • weee
  • Since SETI is most likely purely CPU intensive, there is very little the OS can do to make it faster or slower (apart from deliberately getting in the way, of course). On the other hand, the compiler makes a tremendous difference, and so do the compilation options. Since we do not know which compiler was involved, and which options it was given, the benchmark is fairly useless.

    Actually, the OS's can make a difference in performance, even with the same calculations.
    Memory management is important for speed, because of alignment issues. OS overhead, for task switching and other stuff matters. Disk access, caching and so on are important. How the OS's handle virtual memory can be important.

    But the trials did not have enough samples to fully explore the situation anyway. rbb

  • I wonder... (Score:3)

    by Moonshadow ( 84117 ) on Friday June 01, 2001 @03:05PM (#182735)
    ...how long till we have a SETI@home@home client to do this research on a distributed level...?
  • Argh, slashdot eats angle brackets in subjects. The subject should have read "WinNT needs >128M ram"
  • I was amused by the article's stament that
    Also of interest is the way Windows NT seems to use more memory more efficiently. The WU completion times decreased with the installation of the second 128 MB DIMM. The other operating systems did not show this behavior.
    because I would have come to a different conclusion.

    I would have said "it is interesting that Windows NT cannot run a WU at full speed with less than 128M of RAM. Other operating systems make effective use of 128M of memory, but NT needs 256M."

  • The entire process while working is 13 or 14megs in total. The WU is just 340k, The performance has less to do with the processor speed, but is more dependent on processor cache size and the FSB speed. I have K6-2 400's and K6-2 550's that perform exactly the same because they all have tiny 64K caches. A K6-III 400 out-performs a k6-2 550, the K6-III has a 256K cache. I also have sun ultrasparc systems, one at 167mhz completes within a hour of the 300mhz system. And I have an old HP 735 running at 125mhz, on HP-UX, a huge, bloated operating system (unlike microsoft though, here the hugeness and bloat are the price for HP-UX's flexibility, stability and utility, LVM kicks motherfucking ass!) and it completes a work unit just a little slower than the K6-2's.

  • When will setiathome go open source? Perhaps some compiler optimizations would hep. Who knows what demented troll is compiling your favorite platform's client.

  • PLUS (Score:2)

    by Rogain ( 91755 )
    This wang suggests testing linux by running the winnt commandline code via wine! Sure adding a layer of windows APIs on top of linux, that will really improve performance.

    Debian 2.2 supports udma100, he should have been runing stable, but in anycase he should state exactly what he installed, and should describe what he removed from cron. A fat desktop system will have tons of programs installed. What he is testing is the performance of setiathome, exim, locate, logrotate, suidmanager, syslogd and half a dozen other programs/scripts all at the same time.

  • Stats Question (Score:3)

    by frank249 ( 100528 ) on Friday June 01, 2001 @03:48PM (#182741)
    Do the n's justify the means?

  • my conspiracy is that the SETI crew is pulling the wool over our eyes, getting us to do Carnivore-esque processing en masse.

  • Sorry, really I am... I have a problem... it was just staring at me... so blankly... an empty page without any posts...

    I wasn't tring to get first post... really it just kinda happened... I didn't think, I just acted.. I don't know what I was thinking.

    Please forgive me... I am a sick man

    Oh god.

  • I've always run SETI because i've enjoyed the competion, not because I want to discover ET (I do, of course, but that's not the reason I run the client).

    IIRC SETI is privately funded so don't gripe about your dollars going to waste.

  • Considering the work unit times were 7-8 hours approximately, a standard deviation of 30-80 seconds is not bad!! That shows minimal interference from uncontrolled factors (other programs inconsistantly using processor resources, temperature, etc.) I commend them for detailed explanation of the statistics.

    What they need to do now (as they said) is do more than 5 trials for each configuration so their results are more reliable, THEN trying to decrease their standard deviations by controlling temperature, and other variables.

    Figuring out which factors affect computation time would be interesting. Especially how the environment around the computer (temp, pressure, relative humidity :) affects computation time.

  • SetiSpy and WINE data (Score:3)

    by JDax ( 148242 ) on Saturday June 02, 2001 @04:59AM (#182746)
    Curious that the screensaver version was used to do this study whereas most places doing benchmarking use the command line interface ("CLI" or text client).

    Also one thing that seemed to be missing from this article was reference to Roelof Englebrecht's data, collected over some period of time and posted on his SetiSpy web site here:

    http://pages.tca.net/roelof/setispy/

    Roelof and TLC's Max, are both the keepers of the TLC benchmark database and Roelof checks each and every submission to ensure that the data isn't corrupted somehow due to an unstable overclock.

    In addition, I currently run the text client in WINE on my Red Hat 6.2 as standard practice and have submitted data to the TLC page for it. It runs just fine and in fact runs the WU FASTER (whether using the -win95 switch or the -nt40 switch) than either native winblows version when run on the same system via dual boot.

  • Huh? Did you look at this page? Perhaps you didn't look at the very bottom of the rankings? [burgernet.de]

    When I read the Athlon scores vs the Pentium III scores, they aligned with other benchmarks I've seen which compared the two chips. What's more interesting is that the fastest Athlon was 2 times faster than the fastest Mac. One might be tempted to say mhz do matter until you look at the clock on the PA-Risc. At less than half the clock speed, it blows the fastest Athlon out of the water. Looks like HP knows how to build a cpu.

  • they used an i686 compiled version of the seti client on a thunderbird.

    Don't amd chips just repsond to i586 optimized instructions, and i686 instruction pairings slow it down?
  • also debian's kernel, and other linux utilities that may have been running, is i386 compiled.

    They probably could have fixed it such that linux came out ahead if that was the result they wanted. Though their reasearch does show that reaching for the shelf, and using what is most easily available in a default config can sometimes lead to NT performance wins.
  • actually the article said they "upgraded" the kernel to 2.4.

    If they went into the make file, and changed the optimizations to amd k7 before re-compiling they might have pretended they were way leet, instead of saying "upgraded" the kernel, which suggests they just pointed apt-get to unstable mirrors, and got binaries (which is also alluded to in the next sentence of the article).
  • Actually, since SETI is so CPU-intensive, the fact that Macs ranked low only means Mac users are busier doing important things on their computers, while Windows users aren't doing anything important. We're just busy creating.

    Ahh, I'm just kidding. Who cares. :-) Hey, grow up okay? "Mac users are losers!"

    To me, you seem like your life priorities are a bit mixed up. Who cares! It's just a computer - don't wet your pants. Use what you like, and don't talk about what you don't. At least have a valid point.

    See you.
  • Well I didn't do any scientific invesigation, but I notice my G4 500MHz Mac completes Photoshop filters CONSIDERABLY faster than the 1GHz (well it might be like 900 something MHz, i dunno) Pentium down the hall. It sure does FEEL like twice as fast, and probably is. But this is getting off-topic so I'll stop......

  • Re:Linux wussy! (Score:3)

    by tie_guy_matt ( 176397 ) on Friday June 01, 2001 @05:26PM (#182753)
    Yeah you are right. If linux had won they would have posted it on slashdot or something. Instead we are learning about the study from our psychic friends.
  • Any alien smarter than us is going to find us before we find them

    Isn't that more reason to start searching now? You're right -- aliens may have "found" us long ago. So now it's our turn to find them. Later is better than never, no?

    any alien dumber than us (i suppose thats what we are looking for) may not even respond

    Respond to what?! SETI is a search. It's not a response. Search first. If you find even a "dumb" alien, wasn't that worth the search? As an analogy, do we search for new species on this planet so they can respond to us? Of course not. They're dumber than us, but still worth knowing about.
  • I don't run SETI@home (gasp!), but does the difference between having 128MB of main memory vs 256MB matter that much? I mean, the machine should be otherwise idle and unless the machine starts swapping it isn't going to make much of a difference how much memory you have.

    In fact, the statistics show that SETI isn't that memory hungry, the differences between the two memory configurations where almost noise; less than a standard deviation in some cases.
  • I'm getting 403's from the web site
  • The authors of the article drew a very idiotic conclusion, which is that because there was a difference in performance in NT between 128 and 256 MB, that this mean that NT used memory efficiently. The OPPOSITE is true. It means that memory utilization overall was poorer in 128 MB and it required 256 MB to operate better. Only with 256 MB could the process run more smoothly. An OS which uses memory efficiently will have a lower performance plateau.

    As for the fact that it beat Linux, I would guess that the code is optimized for Windows, then ported to Linux.

    Also, in my experience setiathome on Linux has blown Windows out of the water...when running without the GUI.

    An interesting study, though I'm surprised by their lack of understanding of the memory issue.

  • Statistics and Analysis? (Score:3)

    by mborland ( 209597 ) on Friday June 01, 2001 @05:51PM (#182758)
    Statistics warning? I was surprised by the lack of statistics used. For one, they tried to make claims of efficiency based off of having only two reference points for each modification (speed 1 vs. speed 2; 128MB vs. 256MB). Seems like you'd need more data points to get a useful curve.

    Also, they didn't seem to consider possibilities like the fact that a default install of RH Linux may run updatedb daily, which if using a slow drive with a lot of files could easily describe the variation. Instead, their first guess is 'unstable packages.' Wazzup? Academics...they love bizarre conclusions in favor of putting in that extra effort to find the truth. They spent enough time setting up the survey...why didn't they finish the job? (Answer: they were working on a deadline...and didn't allocate time to research the variation)

  • Our web host decided to take the server down for what has turned out to be a couple of day upgrade. We have put up up our site on a mirror in the meantime. You can check out the main Team Lamb Chop page here: tlc.hagabard.com [hagabard.com] and you can check out our benchmarking pages here: tlc.hagabard.com/bench/index.htm [hagabard.com]

    Thanks for your support!


    -zAmboni

  • I put up a news item over at www.arstechnica.com [arstechnica.com] that explains how they obtained that information. The information was not obtained using a hack...it really was only some script kiddies at work compiling the user_info.sah information that the S@H servers sent back to them

    -zAmboni

  • Uhm, "relied on third party tools written in visual basic" now that's something you made up :-)
    Still, who cares reading those company "success stories", no matter which company or product they are coming from. I can't even take these stories serious if it's a product I like/support.
    --------
  • I am so tied of site@home, when I finally got a monster PC to get some killer stat, they changed the client so that it took even longer and my speed gain was minimal. :-) Why not support something like folding@home that at least has a slight change of doing some good or bad, but atleast something.
    But then again, I can't join since stanford seems to have blocked my entire ISP. heh
    --------
  • RE: SETI fully explained [howstuffworks.com] and some views on alien physiology [howstuffworks.com] -- over 50 extrasolar planets have been discovered as of 2001.

  • ...I Gauss not.
  • I might have missed it in the article, but this test becomes less significant if the WU contain different data.

    However, I doubt the differences in that kind of data are enough to justify the Linux performance by itself.

  • It runs slower on NT and Linux than on 98. Duh. This more an analysis on timeslices (if even that). Hardware platforms should have varied more. PPC Mips. HP. Vax. Wang. Everything. This article was useless.
  • I thought top-end Mac processors were 2x the speed of top-end Intel and climbing?

    If you could afford a top-end machine, of course.

    All I know is Macs sure aren't taxing their CPU's at home playing games...
  • > No, he's not right. In fact, he's almost as
    > wrong as he could be.

    Is he?

    Saying 100 light years wouldn't make us detected is like a caveman saying, "Gee, we can't throw a rock more than 20 Oog-lengths, so no one will find us!" Meanwhile, a satellite with square-inch resolution orbits overhead and spots the campfire at nighttime with no problems whatsoever.

    We probably are found, and the fact we don't see a galaxy filled with immensly powerful artificial signals indicates either we are being directly shielded, or the standard galactic communication is hidden, piggybacked on background radiation ala Contact, or my favorite theory, there's much better ways to communicate than radio waves, so the period of EMF useage by a civilization is very brief.

  • Ehh, my PII 266 takes > 50 hours, while my PIII 450 takes about 15 hours. First has 95, second 98. Is there THAT much difference between the two?

  • Our groups (MVPs) has done a lot of chatting about the variable results and performance in SETI processing.

    While we haven't done anything nearly as scientifically viable as this study, our conclusion has been that the difference in processing speed between a 1Ghz and 866Mhz Intel processors (each with 128M RAM) is very slight - almost imperceptible in some cases.

    Systems with larger L2 caches, however, seemed to have distinct performance advantages. How (or if) you can apply that information to non-SETI applications I will leave to you.

    -Coach-

  • SETI fully explained [howstuffworks.com] and some views on alien physiology [howstuffworks.com]-- over 50 extrasolar planets have been discovered as of 2001.

  • NO MACS ??? (Score:3)

    by 6EQUJ5 ( 446008 ) on Friday June 01, 2001 @03:12PM (#182772) Homepage

    I can't believe they didn't test any Mac platforms. Mac users are some of the most enthusiastic fans of SETI! Team MacAddict is #3 in the Club Competition [berkeley.edu], only behind Ars Technica and Team Art Bell. If fact they're almost tied with Team Art Bell, with only 5,000 users to Art Bell's 13,000 users !!
  • Yes. It's not a statistically robust experiment at all, which makes it a shame they weren't able to pump-up their sample size (n > 35??) and make some mathematically defensible estimates of the true standard deviation. In their defense,each of their "system tests" looks to have taken over a day....

    But I also don't understand why they chose to test the "variance" of two operating systems using the SETI program. What they are comparing is the run-time reliability of two completely different binaries. Even if they're compiled from the same source code, a large part of the variance can presumably be explained by the integrity of the compiler.

    Don't be so hard on academics though... ;)

  • Cant this be implemented in hardware? maybe full-custom or semi-custom CMOS ASIC's would be possible, but its very expensive ofcouse.
  • Hi everyone,

    It's good to see such a strong response from the community and we thank everyone that has given us constructive feedback. However, we feel that many of you have completely missed the purpose of the article.

    The purpose of the article is NOT to compare OSs, get fast SETI@home times, or even to optimize the systems at all. We thought this was clearly stated in the abstract and methodology sections. Either we screwed up and did not make this apparent or people did not read the article. The study of optimizing systems and making SETI@home faster is already handled very well by Team Lamb Chop [teamlambchop.com] (if the site ever decides to come up...).

    What we DID want to accomplish is to study work unit completion time VARIATION . This question was prompted by our reading of the SETI literature which had not dealt with this matter. Our research question was: on a "typical" system, how much does the completion time vary from instance to instance? If I run the same work unit over and over, how much will my completion time vary? I don't know about the rest of you, but when I run SETI@home on my systems, I do not disable all other processes (including cron or anything else). If we did disable anything out of the ordinary, then we did not accomplish our goals. As was clearly stated in the methodology section, we used the default settings for each OS.

    A separate question was the method used for us to arrive at this conclusion: testing several OSs in several configurations on the same platform. At this time, this method seems a useful one for comparing configurations.

    Now that we have a baseline, we are going to continue and try to figure out the questions posed in the article. Such as: why does Linux have such a high variance in comparision to the other OSs? We have collected many great suggestions and I will test all of them and post the results.

    In any case, this article cannot be used to compare OS to OS performance. Not only is that not the purpose of the article, but there are too many confounding variables--many of which have been raised here.

Slashdot Top Deals

"There is such a fine line between genius and stupidity." - David St. Hubbins, "Spinal Tap"

Close