neutron_p writes "When the LHC Computer Grid starts operating in 2007, it will be the most data-intensive physics instrument on the planet. Today eight major computing centers successfully completed a challenge to sustain a continuous data flow of 600 megabytes per second on average for 10 days from CERN in Geneva, Switzerland to seven sites in Europe and the US. The total amount of data transmitted during this challenge -- 500 terabytes -- would take about 250 years to download using a typical 512 kilobit per second household broadband connection."
Ha, wonder if the RIAA is going bezerk as they always do, similar to the internet2 controvorsy.
"ONE MUSIC CD UNCOMPRESSED IN ONE SECOND"
"ONE DVD UNCOMPRESSED IN EIGHT SECONDS"
<SARCASM> After all, the only content that goes over networks like this is obviously RIAA pap^Wvcontent, and 600MB/s is a full CD every second! </SARCASM>
More to the point, the time it would take to get the data onto and off the tapes is left out of your argument. The bandwidth of a truck full of tapes is an old argument, but they're just so damn slow at both endpoints, they're not that useful after all:-(
When the data arrives through a network pipe, it's on disk ready to be crunched through whatever program you're running...
8 or 9 years ago, I used to work in the post-production industry in Soho, London. There's a network called 'Sohonet' where lots of t
When the data arrives through a network pipe, it's on disk ready to be crunched through whatever program you're running...
600 Megs a second. I'd be interested in seeing what sort of disk technology can handle that level of throughput. They must have some amount of buffering going on, hand in hand with the bonus that they're probably able to just stream the data to arrays of disks without really being too concerned about placement (I'm assuming the data transfer is essentially a sequential stream of data, not sodding great numbers of small files, of course).
Haha, I'd hate to have the job of transferring the data to tape and then shipping it simultaneously to 200 places. Good luck keeping up.
From TFA: "When the LHC starts operating in 2007, it will be the most data-intensive physics instrument on the planet, producing more than 1500 megabytes of data every second for over a decade."
"Scientists working at over two hundred other computing facilities...will access the data via the Grid."
But to address your point, yes, tape can be slow. However, the best tape drive money can buy right now (a title claimed by HP's Ultrium 960 [hp.com]) is faster than most hard drives -- 160MB/sec according to the specs. It's not going to be that bad. Expensive, yes, but not slow.
Just a thought experiment: sending a terabyte of data via this tape solution would require (1,000,000 megs / 160 megs per sec) 6,250 seconds, or 104 minutes to write to tape. This assumes 2:1 compression of course, but the actual compressability is unknown.
Sending 500 terabytes in this fashion would require 866 hours (36 days) to write and that same amount to read back onto disk. 72 days sounds like a lot, but this could be shrunk down to as little as 104 minutes if you're willing to employ 500 simultaneously-operating Ultrium 960 tape drives. Expensive, yes, but this is a fun thought experiment where dollars don't matter. Let's assume you use ten drives in an array on both ends (ship the drives with the media to save buying double drives), shrinking your backup/restore times to 86.6 hours (3.6 days) each.
7.2 days plus FedEx Priority Overnight transit time of about 16 hours yields a total transfer time of 7.87 days (7 days, 20 hours, 52 minutes, 48 seconds), or about 680,400 seconds to transfer 500,000,000 megabytes. This gives us a sustained transfer rate of 734MB/sec. This is 22% better performance than the link in the article. The time could be shrunk to as little as one day (the vast majority of it FedEx transit time) if you have 500 tape drives operating all at once.
Total expenditure for such an enterprise would be 10 Ultrium 960 drives (10x$6,190 each = $61,900) and 625 tape cartridges (625x$129 each = $80,625), for a total hardware cost of $142,525. FedEx International Priority shipping costs for a box of tapes like this would be $603, bringing the grand total to $143,128.
Just for giggles, a 500-drive array would cost you $3,095,000 in drive hardware but still take only $80,625 in tapes. With shipping it's a mere $3,176,228.
I'm willing to bet the LHC network costs considerably more than that to operate. What's more, the "tape" network hardware costs need be borne only once. The only operating costs are FedEx shipping costs and replacement tapes if and when needed. It's actually a very efficient way to send huge sums of data from place to place when you think about it.
Note: I've done all this math off the cuff while doing about ten other things, so if my figures are off, don't try to have me drawn and quartered. It was a joke, and it's supposed to be mildly entertaining.
At least things can transfer alot faster within US, if we actually lit the dark fibre underground. We planted so many during the.com eras, yet so many are still unlit due to unwillingness to the hire more techies for maintainance.
Well going outside the US is a different story. I really don't know how we connect to Europe etc.
Come on. The actual fiber is dirt cheap. So what if there's miles and miles of it underground, you still need to hook your little section into the larger network, buy a whole bunch of hardware, AND "hire more techies for maintainance." If it were economically feasible and lucrative, they'd be at it right now. But they're not, seeing as the demand for ultra-high bandwith pipes lie with (a) pirates who don't want to pay for stuff anyway, and (b) CERN. And wuddya know, CERN already has their own ultra-hig
Huh? The cost of laying the fiber pales, as you partly mentioned, when compared to the cost of fiber transceiving equipment. There are tons of 'dark' copper lines as well, that could sustain much greater data throughput than they currently do, but its all about how the data is handled at either end. The same goes for intercontinental runs. You think they are dropping cables one at a time? There is enough bundled up capacity for a long time, but it's 'dark' until it's cost effective to put the gear at either end to use it.
This isn't really for consumer use, it's for scientists that need to transfer data from very sensitive, high-resolution, hi frequency instruments. The kind that haven't been invented yet, but will put out that much data. someday. maybe.
You'd be surprised at the amounts of data captured during experiments in high-energy physics - and keep in mind that this was 500 TB in a *week*, which is longer than you usually want to wait for your data transfer to complete.
your 5mbit downstream pales in comparison to comcasts/ool's 10, or fios's 25. But none of that downstream will help you send files, which is what the 512k was referencing (your rr is probably 316 or somewhere around there)
As for what to send, Physics info works great, as thats what the story is about. For residential use, its all about p2p. When 10/10 is standard(VERY capped fiber) expect a nice p2p mounted filesystem, where instead of the traditional p2p process (search, trim results, download file, wait,
Thats great and all but none of us will be on anything like that for years. If Time Warner had that here they would charge one child a month. You would need 12 wives just to cover your internet bill.
Interesting to note, the Internet2 just had a string of lawsuits pertaining to students using the service for illegal filesharing.
Will this allow you to fileshare so fast that no one can even track it?? Now that would be interesting!
Seriously though, after reading the article and the miscellaneous links. The numbers were astounding! In comparison to my own broadband, I can get 5 or 6 gigs downloaded in a VERY good day at most. Whereas this network enabled traffic of up to 50 terabytes a DAY! Woot woot! When can I hook up for it?
Is that 512kb typical household broadband speed upload or download? I guess for upload that makes sense since most broad band connections are not symmetrical. Download is a different story. I have about 3.5 on a dsl and that is fairly typical for the cable guys as well.
500 terabytes -- would take about 250 years to download using a typical 512 kilobit per second household broadband connection
Well, I've got a 3 megabit connection! It'd only take...uh...well, 42 years or so...but I'd upgrade to that 1 gigabit connection they have in Asia before it finished...
What is with these non-standard terms like "Terabytes" and "Megabytes"? Please re-state the bandwidth and the amount of data tranferred in LoCs (Libraries of Congress) and KLoCs (Kilo Libraries of Congress) so that the rest of the world can understand the magnitude of this achievement.
OK... they lit up the equivalent of two OC48's worth of bandwidth. That's half of an OC192 or a 10G Ethernet. There have been long haul OC192's for a number of years now. If I hook up a hardware-based traffic generator and run at 100% over an OC192 for a few weeks will I get a slashdot article, too?
OK... they lit up the equivalent of two OC48's worth of bandwidth. That's half of an OC192 or a 10G Ethernet. There have been long haul OC192's for a number of years now. If I hook up a hardware-based traffic generator and run at 100% over an OC192 for a few weeks will I get a slashdot article, too?
The accomplishment is not in the data rate, it's in the ability of the participating organisation's to get a stable network going. One that is close enough to the one that the real scientists will be using in a couple of years.
Consider that there's a large number of institutes, universities, etc. that all have their own IT departments, plus all the physicists that have to be involved because it's their grant money funding all this. It's thousands of people coordinating. And I would be surprised if they hadn't set up different service classes, priorisation schemes and what not.
Setting up a trivial network between a couple of sites that are all under your control is close to trivial: you just need to talk to you telco and buy the lines, and hook up the routers. But establishing a working network between these many institutions takes a lot more.
On a related note, CERN is now being sued by the MPAA & RIAA. A spokesmen was commented, saying, "Obviously with 500 terabytes of data being transmitted on the internet, at least some of it had to be copyrighted materials represented by the RIAA and the MPAA. As we know, the internet and communication grids serve no real purpose other then to pirate movies and music." The lawsuit is expected to destroy CERN and any sort of decent networking research anybody was even thinking about doing for the next 50 year.
Time required by my home internet connection, 3mbps, to transfer this data: 41.6666666... years. Rounded to one sig fig, since 500TB is: 42 years. It really IS the answer to life, the universe, and everything.
... the most data-intensive physics instrument on the planet. Today eight major computing centers successfully completed a challenge to sustain a continuous data flow of 600 megabytes per second...
I don't know how fair the comparison is, but I think the Lofar [lofar.org] project will be a heavy contestant for the claim of the experiment with the highest data rate. It is basically a array of some 10000 radio antennas, spread over the northern part of the Netherlands and Germany. It will be operated as one huge phased array.
The data rate [lofar.org] might even be bigger than at Cern: 20 terrabit/sec straight after the A/D converters and still a mighty 0.4 terrabit/sec after the initial data reduction (DSPs + FPGAs). All the remaining data will be transfered over a dedicated fiber network to a central computer. To reduce all this data they need a big fat supercomputer, this will be a IBM Blue Gene [ibm.com] with serial number 2, to be handed over tomorrow [zdnet.com]. For the moment it will be the fastest computer in Europe and ranking somewhere in the top 10 of the world.
Seth Lord and RIAA Chief Mitch Bainwol, felt a sudden disturbance in the force. It was a like a thousand music producers and label execs suddenly cried out in grief and dispair.
It would only take 247.73274987316083206494165398275 years. I wish these articles would check their facts. 247.73274987316083206494165398275 I could live with. I think at 250 I'd start to get impatient...
Slightly offtopic, but as of now they are only recording stuff. This is not interactive at all. I am really surprised that there is no company that does real cool Anime pron with interactivity. One would think there would be a million dollar company out there getting it. They started out with softporn beachvolleyball when the Xbox didn't take off, but let's let id-software develop the engine and think about something real neat. I am supposed to live in the century of Cyberpunk and we don't get 3D, interacti
But will.... (Score:5, Funny)
Re:But will.... (Score:2)
That's all nice, but (Score:5, Funny)
Re:That's all nice, but (Score:4, Funny)
<SARCASM>
After all, the only content that goes over networks like this is obviously RIAA pap^Wvcontent, and 600MB/s is a full CD every second!
</SARCASM>
Parent
Great! (Score:5, Insightful)
Re:Great! (Score:5, Funny)
Only on
Parent
Library of Congresses? (Score:3, Funny)
On a side note, I tried to find out what the real data size of the LOC is, but I could not.
Re:Library of Congresses? (Score:2, Funny)
http://64.233.179.104/search?q=cache:asL7GGh_Js
Re:Library of Congresses? (Score:3, Informative)
Server Error
The server encountered an error and could not complete your request.
If the problem persists, please mail error@google.com and mention this error message and the query that caused it.
Of course, that's because of /.'s lame space-in-url-idiocy. Anyways, the LoC is actually approx. 20TB.
This pales in comparison to... (Score:5, Funny)
Of course, the latency for this gargantuan data pipeline is a bit on the high side...
Re:This pales in comparison to... (Score:5, Funny)
Parent
Re:This pales in comparison to... (Score:4, Funny)
Me too, but it's the frame rates that are killing me (and getting me killed).
Parent
Not really. (Score:3, Insightful)
More to the point, the time it would take to get the data onto and off the tapes is left out of your argument. The bandwidth of a truck full of tapes is an old argument, but they're just so damn slow at both endpoints, they're not that useful after all
When the data arrives through a network pipe, it's on disk ready to be crunched through whatever program you're running...
8 or 9 years ago, I used to work in the post-production industry in Soho, London. There's a network called 'Sohonet' where lots of t
Re:Not really. (Score:4, Interesting)
600 Megs a second. I'd be interested in seeing what sort of disk technology can handle that level of throughput. They must have some amount of buffering going on, hand in hand with the bonus that they're probably able to just stream the data to arrays of disks without really being too concerned about placement (I'm assuming the data transfer is essentially a sequential stream of data, not sodding great numbers of small files, of course).
Parent
That's easy. (Score:3, Funny)
Re:This pales in comparison to... (Score:3, Interesting)
From TFA:
"When the LHC starts operating in 2007, it will be the most data-intensive physics instrument on the planet, producing more than 1500 megabytes of data every second for over a decade."
"Scientists working at over two hundred other computing facilities...will access the data via the Grid."
and now for something completely different.... (Score:3, Funny)
Re:This pales in comparison to... (Score:5, Interesting)
But to address your point, yes, tape can be slow. However, the best tape drive money can buy right now (a title claimed by HP's Ultrium 960 [hp.com]) is faster than most hard drives -- 160MB/sec according to the specs. It's not going to be that bad. Expensive, yes, but not slow.
Just a thought experiment: sending a terabyte of data via this tape solution would require (1,000,000 megs / 160 megs per sec) 6,250 seconds, or 104 minutes to write to tape. This assumes 2:1 compression of course, but the actual compressability is unknown.
Sending 500 terabytes in this fashion would require 866 hours (36 days) to write and that same amount to read back onto disk. 72 days sounds like a lot, but this could be shrunk down to as little as 104 minutes if you're willing to employ 500 simultaneously-operating Ultrium 960 tape drives. Expensive, yes, but this is a fun thought experiment where dollars don't matter. Let's assume you use ten drives in an array on both ends (ship the drives with the media to save buying double drives), shrinking your backup/restore times to 86.6 hours (3.6 days) each.
7.2 days plus FedEx Priority Overnight transit time of about 16 hours yields a total transfer time of 7.87 days (7 days, 20 hours, 52 minutes, 48 seconds), or about 680,400 seconds to transfer 500,000,000 megabytes. This gives us a sustained transfer rate of 734MB/sec. This is 22% better performance than the link in the article. The time could be shrunk to as little as one day (the vast majority of it FedEx transit time) if you have 500 tape drives operating all at once.
Total expenditure for such an enterprise would be 10 Ultrium 960 drives (10x$6,190 each = $61,900) and 625 tape cartridges (625x$129 each = $80,625), for a total hardware cost of $142,525. FedEx International Priority shipping costs for a box of tapes like this would be $603, bringing the grand total to $143,128.
Just for giggles, a 500-drive array would cost you $3,095,000 in drive hardware but still take only $80,625 in tapes. With shipping it's a mere $3,176,228.
I'm willing to bet the LHC network costs considerably more than that to operate. What's more, the "tape" network hardware costs need be borne only once. The only operating costs are FedEx shipping costs and replacement tapes if and when needed. It's actually a very efficient way to send huge sums of data from place to place when you think about it.
Note: I've done all this math off the cuff while doing about ten other things, so if my figures are off, don't try to have me drawn and quartered. It was a joke, and it's supposed to be mildly entertaining.
Parent
Dark Fibre (Score:5, Interesting)
Well going outside the US is a different story. I really don't know how we connect to Europe etc.
Re:Dark Fibre (Score:3, Interesting)
Re:Dark Fibre (Score:5, Interesting)
Parent
Dark Fibre (Score:3, Funny)
The Metamucil of choice by all Lord Siths
rr (Score:3, Interesting)
But seriously. What do you transfer then? I mean, how many Libraries of Congress do you need sitting around on disk.
Re:rr (Score:2, Interesting)
Eh, nevermind...it's a pissing contest.
Re:rr (Score:5, Insightful)
Parent
Re:rr (Score:5, Funny)
Parent
Re:rr (Score:3, Interesting)
As for what to send, Physics info works great, as thats what the story is about. For residential use, its all about p2p.
When 10/10 is standard(VERY capped fiber) expect a nice p2p mounted filesystem, where instead of the traditional p2p process (search, trim results, download file, wait,
42 (Score:5, Funny)
Cost (Score:5, Funny)
Re:Cost (Score:5, Funny)
Parent
Re:Cost (Score:3, Interesting)
At last! (Score:5, Funny)
Where is the torrent ? (Score:4, Funny)
BROADER band? (Score:3, Interesting)
Will this allow you to fileshare so fast that no one can even track it?? Now that would be interesting!
Seriously though, after reading the article and the miscellaneous links. The numbers were astounding! In comparison to my own broadband, I can get 5 or 6 gigs downloaded in a VERY good day at most. Whereas this network enabled traffic of up to 50 terabytes a DAY! Woot woot! When can I hook up for it?
RIAA (Score:5, Funny)
512 kb? (Score:3, Interesting)
640 MB/sec (Score:5, Funny)
"640MB/sec ought to be enough for anybody." --Me,
Ha... (Score:3, Funny)
Well, I've got a 3 megabit connection! It'd only take...uh...well, 42 years or so...but I'd upgrade to that 1 gigabit connection they have in Asia before it finished...
What was Really Transmitted.. (Score:2, Funny)
Standard terms (Score:5, Funny)
Not sure why this is completely notable (Score:5, Insightful)
OK... they lit up the equivalent of two OC48's worth of bandwidth. That's half of an OC192 or a 10G Ethernet. There have been long haul OC192's for a number of years now. If I hook up a hardware-based traffic generator and run at 100% over an OC192 for a few weeks will I get a slashdot article, too?
Re:Not sure why this is completely notable (Score:5, Interesting)
The accomplishment is not in the data rate, it's in the ability of the participating organisation's to get a stable network going. One that is close enough to the one that the real scientists will be using in a couple of years.
Consider that there's a large number of institutes, universities, etc. that all have their own IT departments, plus all the physicists that have to be involved because it's their grant money funding all this. It's thousands of people coordinating. And I would be surprised if they hadn't set up different service classes, priorisation schemes and what not.
Setting up a trivial network between a couple of sites that are all under your control is close to trivial: you just need to talk to you telco and buy the lines, and hook up the routers. But establishing a working network between these many institutions takes a lot more.
Parent
MPAA & RIAA (Score:5, Funny)
The lawsuit is expected to destroy CERN and any sort of decent networking research anybody was even thinking about doing for the next 50 year.
It's the answer! (Score:3, Funny)
Lofar project (Score:4, Interesting)
The data rate [lofar.org] might even be bigger than at Cern: 20 terrabit/sec straight after the A/D converters and still a mighty 0.4 terrabit/sec after the initial data reduction (DSPs + FPGAs). All the remaining data will be transfered over a dedicated fiber network to a central computer. To reduce all this data they need a big fat supercomputer, this will be a IBM Blue Gene [ibm.com] with serial number 2, to be handed over tomorrow [zdnet.com]. For the moment it will be the fastest computer in Europe and ranking somewhere in the top 10 of the world.
Somewhere on Naboo.. (Score:5, Funny)
sri
250 Years? Bah... (Score:4, Funny)
Re:250 Years? Bah... (Score:3, Funny)
Yeah, and if you factor leap years into your calculation, the number is 247.56318604710117372677263163232
Re:That's a lot (Score:2)
Re:woooow.... (Score:3, Interesting)