Aussie Scientists Build a Cluster To Map the Sky 58
Tri writes "Scientists at the Siding Spring Observatory have built a new system to map and record over 1 billion objects in the southern hemisphere sky. They collect 700 GB of data every night, which they then crunch down using some perl scripts and make available to other scientists through a web interface backed on Postgresql. 'Unsurprisingly, the Southern Sky Survey will result in a large volume of raw data — about 470 terabytes ... when complete. ... the bulk of the analysis of the SkyMapper data will be done on a brand new, next generation Sun supercomputer kitted out with 12,000 cores. Due to be fully online by December, the supercomputer will offer a tenfold increase in performance over the facility's current set up of two SGI machines, each with just under 3500 cores in total.'"
I wonder (Score:2, Interesting)
I wonder which CPU the supercomputer will be using. Could be Opterons, or SPARC. I could easily imagine 12000 out of a SPARC Niagra or SPARC VIIfx (8 cores per-die) and would use less wattage than the same number of cores in Opteron. Plus, if they're doing dual or quad-precision, the SPARCs will hold their own.
Re:The Math! It hurts the brain (Score:3, Interesting)
From what I gather the whole 10 Fold, 3 Fold, was more about the progressive thickness of cloth in relation to the number of folds
That's an...interesting theory, but I can't find anything to support it. My M-W dictionary says the phrase goes back to the 12th century (so it has nothing to do with making planes of any type), and clearly states that "tenfold" means ten times [merriam-webster.com], so your suggestion that it "really" means 2^10 is simply false. My own guess is that this ancient phrase has more to do with "in the fold" (where you find sheep, or perhaps wolves) than with cloth, but I can't prove that either.
Perl - One of the best for ETLs (Score:1, Interesting)
Back in 2004 I calculated contract interests for Columbia Pictures in PERL the data 2.4 Terabytes(took a day using 24 crappy pentium IVs). This doesn't surprise me check this out
look for duplicated words in a line /\b(\w+)\b\s+\b\1\b/gi' foo.txt
perl -0777 -ne 'print "$.: doubled $_\n" while
or to cheat in scrabble in Unix
input: tolkien
perl -lne'print if /t/ && /o/ && /l/ && /k/ && /i/ && /e/ && /n/ && length($_)==8' /usr/share/dict/words
output:
knotlike
townlike
Just try writing that in C,
Perl is actually quite fast if you use a bunch of one-liners -