How a Frozen Neutrino Observatory Grapples With Staggering Amounts of Data (vice.com) 49
citadrianne writes: Deep beneath the Antarctic ice sheet, sensors buried in a billion tons of ice—a cubic kilometer of frozen H2O—are searching for neutrinos. "We collect...one neutrino from the atmosphere every ~10 minutes that we sort of care about, and one neutrino per month that comes from an astrophysical sources that we care about a very great deal," researcher Nathan Whitehorn said. "Each particle interaction takes about 4 microseconds, so we have to sift through data to find the 50 microseconds a year of data we actually care about." Computing facilities manager Gonzalo Merino added, "If the filtered data from the Pole amounts to ~36TB/year, the processed data amounts to near 100TB/year." Because IceCube can't see satellites in geosynchronous orbit from the pole, internet coverage only lasts for six hours a day, Whitehorn explained. The raw data is stored on tape at the pole, and a 400-core cluster makes a first pass at the data to cut it down to around 100GB/day. A 4000-CPU dedicated local cluster crunches the numbers. Their storage system has to handle typical loads of "1-5GB/sec of sustained transfer levels, with thousands of connections in parallel," Merino explained.
Re: (Score:2)
Attended talk by one of the professors (Score:5, Informative)
Perhaps they could buy a station wagon and (Score:5, Funny)
Perhaps they could buy a station wagon, load it up with tapes and send it with the next dogsled. (I kid.)
It's not like they are using real-time data from this thing - it's more like a traditional particle smashing experiment where most of the analysis is done months and years after the data is collected.
Re:Perhaps they could buy a station wagon and (Score:5, Informative)
Yes and no. There is some advantage to getting close to real time data: there's a a Supernova Early Warning System http://snews.bnl.gov/ [bnl.gov]. This isn't a safety issue, but rather an astronomy issue.
Detectors like IceCube can be used to actually detect the neutrinos from a supernova before the supernova's light reaches Earth. This isn't due to the erroneous claim from a few years ago that neutrinos travel faster than light, but rather because when a supernova occurs, the light from the core of the star takes multiple hours to get out of the core because of all the mass in the way, while the neutrinos aren't slowed down by this almost at all. This means that the neutrinos effectively get a few hours head start on the light- since they are traveling so close to the speed of light, they get to keep almost all this head start by the time they reach Earth. In the case of SN 1987A https://en.wikipedia.org/wiki/SN_1987A [wikipedia.org], a supernova in 1987 which was close enough that we could detect the flood of neutrinos, the neutrinos did as predicted arrive a few hours before the light. This means we can if we detect a neutrino burst and can get its directional data (which IceCube can approximately do) then we can point our telescopes at a supernova *before the light arrives at Earth* which means we'll get to see the very beginning of the supernova and hopefully get a much better understanding.
In order to do this you have to do at least some of your processing in at least close to real time as you can. This is especially important because it isn't actually easy to figure out from the neutrino burst what direction the supernova is coming from, and IceCube is one of the few detectors which gets any good directional data at all, so if this happens we want to process the data rapidly enough to get a good idea of where to look.
Re:Perhaps they could buy a station wagon and (Score:5, Interesting)
In the case of SN 1987A https://en.wikipedia.org/wiki/SN_1987A [wikipedia.org], a supernova in 1987 which was close enough that we could detect the flood of neutrinos, the neutrinos did as predicted arrive a few hours before the light.
That claim is disputed (as you can see on the page you link to), because the equipment has not reliably recorded the times of arrival of the neutrino detections. What you say about Supernova models predicting neutrinos escaping before light is true, but the observational proof is yet to come... making IceCube even more important.
Re: (Score:2)
So the real time processing at the pole would have to be done to a point where it can indicate (with some level of confidence) the presence of a candidate supernova. Otherwise the 18 hour latency (just missed the daily 6 hour window) would render the data useless. Given this level of processing, information sufficient to aim telescopes could easily be sent on an HF band via Morse code (celestial azimuth and declination, time of event, number of neutrinos counted, etc.). The detailed information could follow
Re: (Score:2)
It's not like they are using real-time data from this thing - it's more like a traditional particle smashing experiment where most of the analysis is done months and years after the data is collected.
Well, in some cases the detection of a high energy neutrino has triggered a search for a counterpart at other wavelengths (X-ray, optical etc.).
So near real-time detection of a neutrino can be important to determine its astrophysical origin.
http://www.astronomerstelegram... [astronomerstelegram.org]
http://www.astronomerstelegram... [astronomerstelegram.org]
And from the ANTARES neutrino telescope:
http://www.astronomerstelegram... [astronomerstelegram.org]
Re: (Score:2)
Perhaps they could buy a station wagon, load it up with tapes and send it with the next dogsled. (I kid.)
Replace the station wagon with a transport plane and that's not an inaccurate picture.
It's not like they are using real-time data from this thing
There is a need for real time data. First you want to know that your detector is working. Finding out 6 months later when you are doing the detailed analysis that there is something seriously wrong with e.g. the trigger would be a very bad thing. Secondly there are astronomical events which can occur rapidly like Supernovae. If IceCube pick up a SN signal then you want to let the astronomers know quickly, not several mon
Re: (Score:2)
Dogsleds aren't used in the Antarctic, just the Arctic. Likewise, no penguins in the Arctic. The Antarctic is pretty barren and inhospitable.
But, yes, never underestimate the bandwidth of a snowmobile-towed sledge works just fine here.
Re: (Score:2)
Dogsleds aren't used in the Antarctic, just the Arctic. Likewise, no penguins in the Arctic. The Antarctic is pretty barren and inhospitable.
Dog sleds in the Antarctic seemed to work well for Roald Amundsen.
Re: (Score:2)
Re: (Score:1)
"The financial liabilities of my bank are really expensive. I'd hate to pay that bill."
Yet you do if you have a deposit there...
Re: (Score:2)
Trillions of Microseconds of Billions of Neutrinos (Score:2)
Re: (Score:2)
You have all sorts of issues with your calculation. 131,400 events per year would be 1 every 4 minutes. 6 events per minute as you state would be 6 per minute * 60 minutes * 24 hours * 365 days equaling 3,153,600 events per year.
However that doesn't really matter because 6 events don't happen a minute. They state in the summary and article that they detect 1 neutrino about every ten minutes or 6 a
Re: (Score:3)
Re: (Score:2)
Although somewhat notorious at the time for counting as the first major consumer CPU that "required" active cooling, you could still get away with a big-ass heat sink.
Funny, really, how it's taken us 22 years to get back to TDPs in a range that makes passive cooling once again practical.
Twiddles (Score:1)
"We collect [...] one neutrino [...] every ~10 minutes," researcher Nathan Whitehorn said.
How did he pronounce "~"?
Re: (Score:2)
It is pronounced as "approximately".
Re: (Score:2)
Hsooooow. That wasn't the point.
The point is that he didn't say it at all. He wrote it in an email.
Re: (Score:3)
"We collect dot dot dot one neutrino dot dot dot every tilde ten minutes"
For intonation, check here [nydailynews.com].
Re: (Score:2)
Holy shit. This is awesome. The article contains virtually nothing beyond some large statistics and a picture of an ice hole and some cable trays.
No kidding......would like to have seen more pics of the computing hardware. :( Fascinating project, though.
Would hate to be in charge of the tape rotations. I wonder how many tape drives they have and their capacity?
Re: (Score:2)
Re:Energy (Score:4, Interesting)
Fossil fueled Generators.. They use JP-8 which is a military version of Jet 1-A with a lower freezing temperature. JP-8 fueled turbine generators generate electricity. I'm guessing they burn it directly for heat too. The JP-8 is delivered during the "summer" months both over land and by air if necessary though flying fuel in is pretty expensive.
Holy coincidental mixed unit mess batman .. (Score:5, Informative)
A tonne is the SI unit for 1000 kilograms.
A ton (US) is a funny unit of measure for 2000 lbs (907kg)
A ton (Imperial) is a funny unit of emasure for 2,240 lbs (1,016 kg)
Thus a tonne is about 1.1 tons (US), and 0.98 tons (Imperial)
A cubic kilometer of water is 1 billion (1E9) tonnes
But water expands when it is frozen by about 9%
So a cubic kilometer of ice would be about 1E9 tons (US)
Thus the statement in TFS
a billion tons of ice—a cubic kilometer of frozen H2O
while numerically about correct is a hell of a mess of mixed units.
Re: (Score:2)
The tonne is also known as the metric ton. It's entirely possible "a billion tons" refers to metric tons.
Why not send the unprocessed data back? (Score:1)
Why are they processing it in place if the processed data requires more bandwidth and they have a transmission bandwidth problem?
What did I miss?
David
Re: (Score:1)
I read the article. The summary is out of sequence wrt the article. They transmit ~36TB/yr back to Wisconsin whereupon processing expands it threefold. The bandwidth issue is with the 36TB.
FPGAs and GPUs (Score:2)
Re: (Score:2)
I don't get why we need to look at frozen neutrino (Score:2)
Aren't roioom temperature neutrinos good enough?
Re: (Score:2)