Become a fan of Slashdot on Facebook

 


Forgot your password?
Close
typodupeerror
×
Space Supercomputing Science

Aussie Scientists Build a Cluster To Map the Sky 58

Posted by Soulskill from the thinking-big dept.
Tri writes "Scientists at the Siding Spring Observatory have built a new system to map and record over 1 billion objects in the southern hemisphere sky. They collect 700 GB of data every night, which they then crunch down using some perl scripts and make available to other scientists through a web interface backed on Postgresql. 'Unsurprisingly, the Southern Sky Survey will result in a large volume of raw data — about 470 terabytes ... when complete. ... the bulk of the analysis of the SkyMapper data will be done on a brand new, next generation Sun supercomputer kitted out with 12,000 cores. Due to be fully online by December, the supercomputer will offer a tenfold increase in performance over the facility's current set up of two SGI machines, each with just under 3500 cores in total.'"
This discussion has been archived. No new comments can be posted.

Aussie Scientists Build a Cluster To Map the Sky More

Aussie Scientists Build a Cluster To Map the Sky

Comments Filter:

  • I wonder (Score:2, Interesting)

    by downix ( 84795 )

    I wonder which CPU the supercomputer will be using. Could be Opterons, or SPARC. I could easily imagine 12000 out of a SPARC Niagra or SPARC VIIfx (8 cores per-die) and would use less wattage than the same number of cores in Opteron. Plus, if they're doing dual or quad-precision, the SPARCs will hold their own.

    • Re: (Score:3, Informative)

      by evol262 ( 721773 )
      They're x6275 blade modules, meaning dual quad core Xeons with a max of 96GB of RAM.

      I like the CMT SPARCs as much as anybody else, but they're frankly not competitive for this sort of workload (massive compression).

      • Re: (Score:1)

        by downix ( 84795 )

        That's why I mentioned the dual or quad precision, the only math-area that the SPARCs are still pretty competitive in.

        • Re: (Score:2)

          by evol262 ( 721773 )
          SPARC VII, yes. The Nehalem Xeons are about on par with T2s at substantially lower cost.

          • Re: (Score:1)

            by downix ( 84795 )

            You know, I tried to verify this, but I could not get any Nehalem Xeon setups able to match the T2 setups. Even remotely capable of matching one. The closest I got was an Apple XServe, w/ 3GB of RAM vs the T5130 w/ 4GB of RAM, and the XServe came up only $2k less than the SPARC while also consuming more electicity. But this is comparing with an Apple, so we all know the markups there. Once there are more Nehalem's out there we'll get a better idea as to cost comparison.

            • Re: (Score:2)

              by evol262 ( 721773 )
              I'm going to say you didn't look hard enough. Published SPEC results have a Sun Fire x4270/x2270 walking all over the T5240/T5120 for 1/3rd of the cost or less (with 6GB of RAM).

              They cannot, of course, run nearly as many threads in hardware, but for brute-forcing compression, it's sort of a moot point.

              • Re: (Score:1)

                by downix ( 84795 )

                A ha, the Intel® Xeon® Processor 5500 Series is the Nehalem, did not have the model number handy in my search. TYVM for putting in the missing pieces.

                • Re: (Score:2)

                  by evol262 ( 721773 )
                  I don't pretend to understand Intel's modeling scheme here. A small bump (55xx), with some of them having HyperThreading, some not, some have QPI, some not, etc. It's not exactly intuitive, and I wouldn't have known to look there either if we weren't using some of them.

  • So basically (Score:5, Funny)

    by Norsefire ( 1494323 ) * on Wednesday June 17, 2009 @09:42AM (#28361451) Journal
    it's a skynet?

    • Re: (Score:2)

      by sdpuppy ( 898535 )
      Certainly. They're uploading to the Sun, no?

      They're also uploading to SGI since the Sun only has one core.

      ducks

  • Two Words (Score:3, Funny)

    by ehaggis ( 879721 ) on Wednesday June 17, 2009 @09:44AM (#28361477) Homepage Journal
    Beowulf Cluster

  • The Math! It hurts the brain (Score:3, Insightful)

    by dmomo ( 256005 ) on Wednesday June 17, 2009 @09:56AM (#28361607)

    next generation Sun supercomputer kitted out with 12,000 cores ... will offer a tenfold increase in performance over the .. two SGI machines, each with just under 3500 cores in total

    How is that 10x faster? I imagine because the new v. old cores are not equally comparable. In that case, why talk number of cores at all?

    • Re: (Score:2, Funny)

      by kburk ( 1352309 )
      ya, but can it run crysis?

    • Re: (Score:3, Informative)

      by evol262 ( 721773 )
      They're moving from 1.6Ghz single core Itaniums to dual quad core Xeon blades. I suspect they're talking about cores to emphasize the density gain, and because people like huge numbers.

    • Re:The Math! It hurts the brain (Score:5, Funny)

      by kenp2002 ( 545495 ) on Wednesday June 17, 2009 @10:09AM (#28361777) Homepage Journal

      "Old Man Rant"

      Why do I cring every time I hear people use terms like Tenfold and order of magnatitude....

      From what I gather the whole 10 Fold, 3 Fold, was more about the progressive thickness of cloth in relation to the number of folds back in the war when we made planes out of canvas.

      1mm thick material when increase 3 fold is

      1 -> 2 -> 4 -> 8 mm thick. Ten fold would then be 512 mm thick...

      Why are we talking about folding stuff? Where are the protients... WHa? I DON'T WANT TO TAKE THE PILLS! WHO ARE YOU PEOPLE! LEAVE ME ALONE!

      I'M NOT DOING ANY LAUNDRY! ;)

      • Re: (Score:3, Interesting)

        by Xtifr ( 1323 )

        From what I gather the whole 10 Fold, 3 Fold, was more about the progressive thickness of cloth in relation to the number of folds

        That's an...interesting theory, but I can't find anything to support it. My M-W dictionary says the phrase goes back to the 12th century (so it has nothing to do with making planes of any type), and clearly states that "tenfold" means ten times [merriam-webster.com], so your suggestion that it "really" means 2^10 is simply false. My own guess is that this ancient phrase has more to do with "in the fold" (where you find sheep, or perhaps wolves) than with cloth, but I can't prove that either.

    • it's ~10x faster on specfp_rate than the itanium2 altix.
      specfp turns out to be a good representation of the codes we typically run...

  • Comment removed (Score:4, Funny)

    by account_deleted ( 4530225 ) on Wednesday June 17, 2009 @09:57AM (#28361615)
    Comment removed based on user account deletion

    • Re: (Score:2)

      by nizo ( 81281 ) *

      Maybe they need that kind of power to run the database? (Don't hurt me!)

      Seriously though, can I have their crummy old machines when they are done with them?

  • Perl power!!! (Score:2, Insightful)

    by Murpster ( 1274988 )
    Good to see the coolest language around being put to a(nother) cool use.
    • Yes! Perl is so well suited for ETL processing, it makes me happy when people use it.

      At my current job I want to take our Informatica server and throw it out the window. That way I can use Perl.

  • Hmmm... (Score:4, Funny)

    by kenp2002 ( 545495 ) on Wednesday June 17, 2009 @10:01AM (#28361673) Homepage Journal

    With that much data we might finally have enough information to generate a singular point of reference in space and time so we can retrofit a poorly designed all stainless steel car and travel back in time 200 years and not find out selves drifting in the middle of nowhere since 200 years ago, relative to some unknown non-moving reference point, our planet, solar system, and galaxy is probably no where near where it was 200 years ago!

    FLUX CAPACITOR FTW!

  • MMMMmmm.... (Score:3, Funny)

    by amstrad ( 60839 ) on Wednesday June 17, 2009 @10:01AM (#28361677)

    Imagine a open/globular cluster of those....

    • But does it run Linux?

      Joking aside, this sounds like a wonderful project, both on the astronomy side and on the technology side. I can't wait to see what they find out with it.

      ...laura

      • Joking aside, this sounds like a wonderful project, both on the astronomy side and on the technology side. I can't wait to see what they find out with it.

        My God! It's full of stars!

        Okay...now I am willing to put joking aside.

        • Re: (Score:3, Funny)

          by CyberLord Seven ( 525173 )
          Someone used to have a wonderful sig:

          My dog! It's full of rats!

          2001: A Dyslexic Odyssey

          No, I don't know the name. I almost never look at names, just sigs.

          • Sounds like the old joke about the agnostic dyslexic insomniac.

            Who used to lie awake at night wondering if there really was a Dog.

            ...laura

  • So... (Score:1)

    by Ltap ( 1572175 )
    It's the Skynet? In all seriousness, it sounds awesome and it's good to know that they're using the right tools for the job (perl to organize, postgreSQL to manage large amounts of data).
  • SGI stopped making supers many years ago.

  • How do they store so much data into PostgreSQL? I thought it had a limit of 32TB per table, so are they using some sort of table partitioning scheme?

    • They're probably only storing the refined data in PostgreSQL, with the raw data being stored as flat files on disk and tape. This is how we do it with 300TB of disk and 17PB of tape.

  • Sounds like a great step forward - and I want to be on record saying that I'm pulling for them.

    However - I'm confused, per usual. As Oracle now owns Sun, what's to say that the camel won't stick his nose into the tent, seeing a database opportunity (read: marketing opportunity) and try to pressure, cajole, coerce, or otherwise influence them to drop a working PostgreSQL in favor of an all-Oracle (Sun) solution? I'm not saying that one is better than the other - I'm just concerned that a political motive o

  • Sounds like a job for BOINC [berkeley.edu].

  • "next generation Sun supercomputer kitted out with 12,000 cores." Can it run Crysis with full eye-candy?

  • "Sun supercomputer kitted out with 12,000 cores." " They collect 700 GB of data every night, which they then crunch down using some perl scripts "

    Doesn't that mean it could be run from a home computer running c code instead?

  • It sounds like it almost has enough power to run Windows Vista with options turned on. Too bad you can only run three programs at a time though.

    Imagine a beowulf cluster of these...

  • Perl - One of the best for ETLs (Score:1, Interesting)

    by Anonymous Coward

    Back in 2004 I calculated contract interests for Columbia Pictures in PERL the data 2.4 Terabytes(took a day using 24 crappy pentium IVs). This doesn't surprise me check this out

    look for duplicated words in a line
    perl -0777 -ne 'print "$.: doubled $_\n" while /\b(\w+)\b\s+\b\1\b/gi' foo.txt

    or to cheat in scrabble in Unix

    input: tolkien

    perl -lne'print if /t/ && /o/ && /l/ && /k/ && /i/ && /e/ && /n/ && length($_)==8' /usr/sha

  • The article describes Coonabarabran as being the "central-west NSW location". Now, Coonabarabran certainly isn't very far west [google.com] in NSW. But, then I realised that they mean population-wise. There aren't too many people west of there!!!

Slashdot Top Deals

Help fight continental drift.

Close