Aussie Scientists Build a Cluster To Map the Sky 58
Tri writes "Scientists at the Siding Spring Observatory have built a new system to map and record over 1 billion objects in the southern hemisphere sky. They collect 700 GB of data every night, which they then crunch down using some perl scripts and make available to other scientists through a web interface backed on Postgresql. 'Unsurprisingly, the Southern Sky Survey will result in a large volume of raw data — about 470 terabytes ... when complete. ... the bulk of the analysis of the SkyMapper data will be done on a brand new, next generation Sun supercomputer kitted out with 12,000 cores. Due to be fully online by December, the supercomputer will offer a tenfold increase in performance over the facility's current set up of two SGI machines, each with just under 3500 cores in total.'"
I wonder (Score:2, Interesting)
I wonder which CPU the supercomputer will be using. Could be Opterons, or SPARC. I could easily imagine 12000 out of a SPARC Niagra or SPARC VIIfx (8 cores per-die) and would use less wattage than the same number of cores in Opteron. Plus, if they're doing dual or quad-precision, the SPARCs will hold their own.
Re: (Score:3, Informative)
I like the CMT SPARCs as much as anybody else, but they're frankly not competitive for this sort of workload (massive compression).
Re: (Score:1)
That's why I mentioned the dual or quad precision, the only math-area that the SPARCs are still pretty competitive in.
Re: (Score:2)
Re: (Score:1)
You know, I tried to verify this, but I could not get any Nehalem Xeon setups able to match the T2 setups. Even remotely capable of matching one. The closest I got was an Apple XServe, w/ 3GB of RAM vs the T5130 w/ 4GB of RAM, and the XServe came up only $2k less than the SPARC while also consuming more electicity. But this is comparing with an Apple, so we all know the markups there. Once there are more Nehalem's out there we'll get a better idea as to cost comparison.
Re: (Score:2)
They cannot, of course, run nearly as many threads in hardware, but for brute-forcing compression, it's sort of a moot point.
Re: (Score:1)
A ha, the Intel® Xeon® Processor 5500 Series is the Nehalem, did not have the model number handy in my search. TYVM for putting in the missing pieces.
Re: (Score:2)
Re: (Score:2)
Done. [google.com]
Google never really mapped it though, they just offer old material in a user-friendly way.
Re: (Score:2)
So basically (Score:5, Funny)
Re: (Score:2)
They're also uploading to SGI since the Sun only has one core.
ducks
Two Words (Score:3, Funny)
Re:Two Words Beowolf Cluster... (Score:1)
Can be bay like wolves and sing Men At Work's "Land Down Under"?
http://www.youtube.com/watch?v=DNT7uZf7lew [youtube.com]
But, change the song to Chips At Work's "Skies Up Over", hehehehehe....
The Math! It hurts the brain (Score:3, Insightful)
next generation Sun supercomputer kitted out with 12,000 cores ... will offer a tenfold increase in performance over the .. two SGI machines, each with just under 3500 cores in total
How is that 10x faster? I imagine because the new v. old cores are not equally comparable. In that case, why talk number of cores at all?
Re: (Score:2, Funny)
Re: (Score:3, Informative)
Re:The Math! It hurts the brain (Score:5, Funny)
"Old Man Rant"
Why do I cring every time I hear people use terms like Tenfold and order of magnatitude....
From what I gather the whole 10 Fold, 3 Fold, was more about the progressive thickness of cloth in relation to the number of folds back in the war when we made planes out of canvas.
1mm thick material when increase 3 fold is
1 -> 2 -> 4 -> 8 mm thick. Ten fold would then be 512 mm thick...
Why are we talking about folding stuff? Where are the protients... WHa? I DON'T WANT TO TAKE THE PILLS! WHO ARE YOU PEOPLE! LEAVE ME ALONE!
I'M NOT DOING ANY LAUNDRY! ;)
Re: (Score:3, Interesting)
From what I gather the whole 10 Fold, 3 Fold, was more about the progressive thickness of cloth in relation to the number of folds
That's an...interesting theory, but I can't find anything to support it. My M-W dictionary says the phrase goes back to the 12th century (so it has nothing to do with making planes of any type), and clearly states that "tenfold" means ten times [merriam-webster.com], so your suggestion that it "really" means 2^10 is simply false. My own guess is that this ancient phrase has more to do with "in the fold" (where you find sheep, or perhaps wolves) than with cloth, but I can't prove that either.
Re: (Score:2)
you so totally missed the joke...
Re: (Score:2)
Ah well, wasn't the first time, and certainly won't be the last. :)
Re: (Score:1)
it's ~10x faster on specfp_rate than the itanium2 altix.
specfp turns out to be a good representation of the codes we typically run...
Comment removed (Score:4, Funny)
Re: (Score:2)
Maybe they need that kind of power to run the database? (Don't hurt me!)
Seriously though, can I have their crummy old machines when they are done with them?
Re: (Score:1)
I think it's unlikely that Perl is being used for any of the heavy computations. I would guess that Perl is being used as a glue language that interfaces with the database and programs written in lower-level languages to do the heavy lifting.
Perl power!!! (Score:2, Insightful)
Re: (Score:1)
At my current job I want to take our Informatica server and throw it out the window. That way I can use Perl.
Hmmm... (Score:4, Funny)
With that much data we might finally have enough information to generate a singular point of reference in space and time so we can retrofit a poorly designed all stainless steel car and travel back in time 200 years and not find out selves drifting in the middle of nowhere since 200 years ago, relative to some unknown non-moving reference point, our planet, solar system, and galaxy is probably no where near where it was 200 years ago!
FLUX CAPACITOR FTW!
MMMMmmm.... (Score:3, Funny)
Imagine a open/globular cluster of those....
Re: (Score:2)
But does it run Linux?
Joking aside, this sounds like a wonderful project, both on the astronomy side and on the technology side. I can't wait to see what they find out with it.
...laura
Re: (Score:2)
Joking aside, this sounds like a wonderful project, both on the astronomy side and on the technology side. I can't wait to see what they find out with it.
My God! It's full of stars!
Okay...now I am willing to put joking aside.
Re: (Score:3, Funny)
No, I don't know the name. I almost never look at names, just sigs.
Re: (Score:2)
Sounds like the old joke about the agnostic dyslexic insomniac.
Who used to lie awake at night wondering if there really was a Dog.
...laura
So... (Score:1)
standard GPU board beats an SGI super these days (Score:2, Informative)
Re:standard GPU board beats an SGI super these day (Score:4, Informative)
PostgreSQL capacity? (Score:2)
How do they store so much data into PostgreSQL? I thought it had a limit of 32TB per table, so are they using some sort of table partitioning scheme?
Re: (Score:2)
Per usual - I'm confused (Score:2)
Sounds like a great step forward - and I want to be on record saying that I'm pulling for them.
However - I'm confused, per usual. As Oracle now owns Sun, what's to say that the camel won't stick his nose into the tent, seeing a database opportunity (read: marketing opportunity) and try to pressure, cajole, coerce, or otherwise influence them to drop a working PostgreSQL in favor of an all-Oracle (Sun) solution? I'm not saying that one is better than the other - I'm just concerned that a political motive o
Sounds like a job for... (Score:2)
Sounds like a job for BOINC [berkeley.edu].
Yes, but (Score:1)
Supercomputer power (Score:1)
"Sun supercomputer kitted out with 12,000 cores." " They collect 700 GB of data every night, which they then crunch down using some perl scripts "
Doesn't that mean it could be run from a home computer running c code instead?
Re: (Score:2)
I wanna see some of the cosmos up close before i die.
Look at the WorldWide Telescope from Microsoft Research: http://www.worldwidetelescope.org/ [worldwidetelescope.org] and Sky from Google: http://www.google.com/sky/about.html [google.com]
These fill somewhat different niches and both provide compiled and web-based clients.
Re: (Score:1)
This is what you can get for free at at the moment:
http://sourceforge.net/softwaremap/trove_list.php?form_cat=134 [sourceforge.net]
Almost good enough (Score:2)
It sounds like it almost has enough power to run Windows Vista with options turned on. Too bad you can only run three programs at a time though.
Imagine a beowulf cluster of these...
Perl - One of the best for ETLs (Score:1, Interesting)
Back in 2004 I calculated contract interests for Columbia Pictures in PERL the data 2.4 Terabytes(took a day using 24 crappy pentium IVs). This doesn't surprise me check this out
look for duplicated words in a line /\b(\w+)\b\s+\b\1\b/gi' foo.txt
perl -0777 -ne 'print "$.: doubled $_\n" while
or to cheat in scrabble in Unix
input: tolkien
perl -lne'print if /t/ && /o/ && /l/ && /k/ && /i/ && /e/ && /n/ && length($_)==8' /usr/sha
western NSW??? (Score:1)