Virtual Grid Supercomputer Goes (Partly) Online 257
hotsauce writes "The BBC is reporting that CERN (the guys who invented the Web) are working on a virtual supercomputer called the Grid. The Grid taps computing power from 12 countries to process data from a new supercollider that will simulate parts of the Big Bang. Phase One of the Grid just went online."
Over-hyped (Score:4, Insightful)
Re:Over-hyped (Score:5, Interesting)
I'm doing my PhD on it at the moment. While there is a lot of hype there is a lot more substance to it than there was 2 years ago.
For example we now have a single sign on system spread over more than a dozen countries (never easy when the gov wants to know why other countries scientists are using their machines.)
Loads of work has been done on integrating data resources into the network, large scale systems configuration, data discover etc.
As ever the BBC is a bit behind on the news - this is just a new releases of the software. And there had better be substance when LHC comes on line in 2007/8 as it will be generating 15, 000 TB of data to be stored a year and we need some way to deal with this.
15,000 TB/year (Score:2)
Re:Over-hyped (Score:5, Interesting)
People working in half a dozen different languages (natural languages, that is, not programming languages!), a complete lack of chain-of-command, software being signed off on because a document detailing critical bugs had been produced (even though the critical bugs weren't fixed!), progress reports being sent back for revision because "they weren't positive enough"...
Take the worst elements of decentralized open-source programming, combine them with the worst elements of government beaurocracy -- and then replace half of the programmers with physicists who have less programming experience than most second year CS majors -- and you'll get somewhere close to what I've been told about the EU Datagrid.
Of course, this is all second-hand information. Maybe the person who was telling me all of this is just a burnt-out coder who would be unhappy with whatever project he was working on. But I doubt it.
Re:Over-hyped (Score:5, Interesting)
Welcome to reality. Reality for most of the rest of the world, that is.
In Europe we speak lots of languages. That's the reality of the situation. Of course this isn't as efficient as just speaking one language, but it is the reality. I know, as an American you're probably thinking "well why don't you all just speak English?"
I had a funny situation on the outskirts of San Francisco once. I asked a guy sweeping up outside a MacDonalds some directions. I replied in very stilted English that he couldn't understand me because he didn't speak English. I noticed his accent so I asked him my question in Spanish. The look of suprise and joy on his face was comical. He told me that most Americans can't speak another language and expect you to learn English, but not speaking another language themselves don't realise that learning another language isn't easy. And they treat you like crap and think you're stupid if you can't speak English. I don't know if what he said is true or not, but unfortunately I don't find it hard to believe.
Re:Over-hyped (Score:5, Insightful)
Any company which was running a project like this would have picked one location, hired a bunch of people, and had them all working in the same building, speaking the same language. It's only because of EU politics -- the requirement that the EU Datagrid be a multinational consortium -- that there are so many languages and locations involved.
Re:Over-hyped (Score:4, Insightful)
And then tried to launch the grid project at multiple corporate sites, whereupon they discover the nice little problem that corporate politics and economics result in them not being able to run that project anywhere but at the site where the project was developed - your project, we're not contributing a cents worth of spare capacity, go buy your own machines.
If you want a project like this to have the faintest chance to succeed you get all the people who are expected to contribute anything out of their budget involved. Or you'll get sunk. Even in a corporation. And for a multinational, that means you have people with different native languages. This is usually not a problem in a multinational because the people involved in such projects usually have at least one language in common, and often more.
One of the problems with us Americans learning.... (Score:2)
Because of this, circumstances do sort of require that if you're
Re:Over-hyped (Score:2)
i'm expected to speak 3 languages(finnish, english and swedish, which i don't speak too well.. but can understand somewhat still). the language i use when dealing with people who don't speak finnish is english(that includes one of the guys who i share the bathroom and kitchen with who is chinese, and the german and polish guys before him)
well, i do know for a fact that it's not that uncommon in southern europe that
Re:Over-hyped (Score:2)
but unfortunately I don't find it hard to believe.
Which reveals your biases. Almost all high schools and jr. highs in America require 3-5 years of F.L study. I myself am fairly fluent in spanish (which is by far the largest second language in America).
But this is also a cultural difference between America and Europe. Simply put, we have far far too many ethnic minorities here to use anything other then one or two languages in general use, and 10-12 in court use (a friend of mine is a translator at dist
Re:Over-hyped (Score:2)
My attitude is that if English is considered the business language of the world, that most new developments are done in English, that if you move to the US, you better learn the damn language.
Re:Learn the language, dummy! (Score:3, Informative)
Yes! I bet all those US troops in Iraq are currently making a big effort to learn Arabic and Kurdish...
Re:Over-hyped (Score:2)
I'm an American and I've lived in two smallish cities in Germany. In those two places hardly anybody spoke English except if they were university educated. Even in Berlin, I've had problems where nobody spoke English. However, the thing with Germans is that usually you would be addressed in German first. If you look confused and ask them if they sp
Re:Over-hyped (Score:2)
What a waste of time.
My mother was German. Also as waste of time becuase, get this, only Germans speak German! Outside of german school I never heard of word of German.
Under-hyped (Score:5, Insightful)
Over here in Europe we're crap at it, or rather, it's not seen as being so important - which I believe is a mistake.
I was once knew a marketing person at CERN who said that as they had invented the web, they were thinking of putting up some good web pages about it to let the world know what they had achieved. Later I found out that she had been told that it had been decided that it wasn't a priority and so no budget was allocated to it. So outside of techy fields most people in the world don't know that the web was invented in Europe. (Yes, I know the Internet network was an American invention).
This is true of many other achievements in Europe. For instance, everyone in the world knows lots about NASA and yet very little about the achievements of the European Space Agency, which has also done some really impressive things.
The USA also has a good way of claiming innovations as its own, by "Americanising" everything. So, for instance, most people - Americans and non-Americans - think that most technological innovation during the last century has been American, whereas in actual fact much of it was just "absorbed" by America. Example - landing on the moon is seen as a purely American achievement but was actually significantly based on German work. Another example - Einstein was of course European but many people believe he was American.
Don't get me wrong. I'm not really criticising the USA, I'm actually criticising Europe for not promoting its achievements or those of its citizens. And of course one of the great things about the USA is that it has the money and drive to make things happen, which is why many projects initiated in other places end up taking off in the USA. But it is frustrating to meet Americans that believe that everything is invented in America.
Re:Under-hyped (Score:1)
Re:Under-hyped (Score:2)
Your Evidence?
The way I heard it was that Webster decided to "fix" spellings when publishing his dictionary.
Re:Under-hyped (Score:2, Interesting)
But more seriously, NASA gets headlines and the ESF doesn't because NASA was there first with the things that impress the general public. It might be more scientifically sound to be conservative and focus on what benefits research, but you get credit for sticking some gu
Re:Under-hyped (Score:3, Insightful)
For instance, Einstein did nearly all his significant work and published most of his important papers whilst he was in Europe. He only fled to the US in around 1931, after he had published nearly all of his most significant work. And yet you seem to believe that Einstein and others like him "only achieved greatness because they fled to a land where they were allowed to develop their ideas." That's just rubbish, and your misunderstandin
Re:Under-hyped (Score:2)
(Of course then, you have to wonder why the US lags behind some other countries in science education....)
Re:Under-hyped (Score:3, Insightful)
You have to be really careful to say exactly what you mean when you make general statements like this. While it is true that US K-12 (or in particular 8-12) science education falls behind much of Europe, that is not true for college education or other types of education not normally considered (thinktanks, museums, libraries, private education, etc.,). You can't always compare one countrie
Ermmm... (Score:2)
You can compare countries by giving students the same test. That's exactly what the The Third International Math and Science Study (TIMMS) [bc.edu] did.
To get around your objection that some countries segregate students after 8th grade, they tested 4th and 8th graders in all the participating countries. Bottom line was that if you are an American who scored in th
Re:Under-hyped (Score:2)
Re:Under-hyped (Score:2, Funny)
It's not just technology... Pizza is a prime example. Some (fortunately not that many) Americans think it's an American thing.
Re:Under-hyped (Score:2)
Pizza is a prime example. Some (fortunately not that many) Americans think it's an American thing.
If it's got meat and cheese on it, it is.
The original Pizza ("Tomato Pie") is tomato paste and bread.
Re:Under-hyped (Score:2)
Europe has a space agency?
Re:Under-hyped (Score:2, Insightful)
I was once knew a marketing person at CERN who said that as they had invented the web, they were thinking of putting up some good web pages about it to let the world know what they had achieved. Later I found out that she had been told that it had been decided that it wasn't a priority and so no budget was allocated to it. So outside of techy fields most people in the world don't know that the web was invented
Re:Under-hyped (Score:2)
a) Actually, America invented the web.
b) The European space agency wouldn't have achieved anything if it weren't for America.
c) Americans invented the car, rockets, flight, the internet, and atomic energy (and those Americans would take it as an insult to be called Europeans).
d) Europe has a bad attitude to immigration.
e) Einstein was American.
f) The German work that the Americans used to land on the moon was actually American.
I can se
Re:Under-hyped (Score:2)
[American's] think that most technological innovation during the last century has been American, whereas in actual fact much of it was just "absorbed" by America.
My point is that each of the five big technological advances of the last century started here. Mass produced Cars (and the combustion engine), Planes (lightweight engines as well), Atomic Energy (both bombs and other wise), the Internet, and Rockets all were created here. If you want to go even further back, Take a look at Thoma
Re:Under-hyped (Score:2)
Your point is simply not valid.
You're right! All major technologies since America was founded originated in America by Americans, and were not based in any way on any work done outside of America, or by non-Americans.
Now I cannot think of a single thing that someone outside of America has invented. Oh wait! Didn't the French invent waffles?
Re:Under-hyped (Score:2)
Atomic Energy (both bombs and other wise)
Many of the atomic scientists working in the US on this during World War two came from Eurpoe, by way of the UK, and they were in the US because the US and the UK chose to combine efforts on researching the atom bomb after the US joined the war. It was mutually agreed that it made more sense to carry out the research in the US (and move the scientists out of Britian to join with the US effort) because the US could hide them away from public eyes easier - there wa
Re:Under-hyped (Score:2)
> This is a fact that "American's get" and
> European's tend not to. We don't care where you
> were born, or what your past life was.
Of course, those people not here to verify your conjecture on that, but you seem to be saying that just because somebody lives in America they are American.
That does actually explain to me a lot about how Americans think. The whole "melting pot" mentality (racist lunatics or various flavours notwithstanding) basically c
Re:Under-hyped (Score:2)
This is true in TV as well. I've been seeing ads for this "new" show "Coupling" on NBC, so I decided to set my ReplayTV to record it. (Studios are smart: they put ads for new shows at the end of the commercial break, so when I 30-second skip too far, I end up 7-second instant replaying back a bit too far as well; and if the commercial looks interesting, I'll rewind to the beginning of it.)
Since it was a f
Re:Under-hyped (Score:5, Informative)
Einstein was born in 1879 and moved to America in 1931 at the age of 52.
Re:Under-hyped (Score:2)
Yes, the first time to become Swiss. Which is in Europe.
Re:Under-hyped (Score:4, Insightful)
I think the responses I am getting to my post just demonstrate the point I am trying to make.
Einstein moved to America when he was 52. He had done nearly all his most important work by that time.
You say that this is because Americans have a different attitude to imigration. However, when Americans move abroad you don't seem to consider them no longer American.
I understand Madonna now lives in London. Should I now start referring to her as British? Are all her achievements now British ones? Is Gore Vidal Italian? He's been living there for years. Are his books now Italian ones?
You're just proving the point I am making.
Re:Under-hyped (Score:2)
We do have a different (and IMHO better) attitude about immegration.
It's great that the EU is working on a new Grid program. Sun has been trying for years, and never got it to work well. I think it's mostly hype, but we will see.
Re:Over-hyped (Score:2)
Well when I was working on the Web at CERN they already had comprehensive facilities for distributing computing jobs over large numbers of workstations. Most physics code is pretty easy to parallelise at the event level - whether it be simulations or analysis. So no, it is very clear that the grid will be extensively u
When will we do this ourselves? (Score:4, Insightful)
Sure it doesn't work for any type of calculation, but there's still a huge potential.
Distributed backups is another thing I'd like to have now, rather than tomorrow...
Re:When will we do this ourselves? (Score:1)
Re:When will we do this ourselves? (Score:2)
The question is not at all stupid. SETI@Home does this, as does Folding@Home, Distributed.net etc...
But in all of these cases the calculations being done are determined by the server. I'd like to see a system where the calculations could be determined by the users. Providing everyone doesn't need to calculate huge stuff all the time it should work.
The major issue is data security. The program has to be
Re:When will we do this ourselves? (Score:1)
Re:When will we do this ourselves? (Score:3, Insightful)
Re:When will we do this ourselves? (Score:1)
I think he means a generalised piece of software - i.e. you c
Re:When will we do this ourselves? (Score:2, Interesting)
No, since as part of the Grid, everyone will have proof of who authorized the calculation; you'll be able to say to the Feds "I can prove it was John Q. Terrorist, Kabul."
Of course, the Feds will ask you why you were selling any compute time to JQT given that the Grid doesn't mean that you have to sell to all-comers (no more than you have to give Unix
Re:When will we do this ourselves? (Score:5, Insightful)
No, seti is a distributed application - 3 million instances of the same program. The grid is a distributed environment - an operating system if you like, which programmers can write their own applications to run on top of.
Like the OS for your desktop the Grid's middleware software deals with things like I/O, resource allocation, security etc.
So, seti@home could run on the grid, but is not a grid in its own right.
Re:When will we do this ourselves? (Score:2)
Gah!
(Curls into fetal postion)
Re:When will we do this ourselves? (Score:1)
How about this? [bash.org]
Re:When will we do this ourselves? (Score:2, Informative)
Analytical Spectroscopy Research Group [uky.edu]
evolution@home [evolutiona...search.org]
eOn [washington.edu]
Climate Prediction [climateprediction.net]
Distributed Particle Accelerator Design [stephenbrooks.org]
LifeMapper [lifemapper.org]
etc...
Re:When will we do this ourselves? (Score:1)
Cunning OpenMosix plan (Score:2)
I know about OpenMosix, but what I just wondered about was this...
Why not run a VPN for a worldwide network of "trusted" OpenMosix machines? OpenMosix is tunable, so you could tell it if you're on a slow-as dial-up link, so it only gives you long running jobs. You could use a chain of trust approach if required, or just sign up via the OpenMosix website for example.
That might even actually work.
Re:When will we do this ourselves? (Score:4, Informative)
You can get it here [sun.com] along with some case studies of how it's used in production.
Distributed backups is another thing I'd like to have now, rather than tomorrow...
Uuencode, split, and post to Usenet...
Re:When will we do this ourselves? (Score:3, Informative)
The TeraGrid [teragrid.org] is the NSF flagship for grid computing - be it good or bad.
The Grid.org [grid.org] people are some of the former SETI@home people gone more general purpose.
And of course, there is The Global Grid Forum [ggf.org] which is meeting in Chicago in a week or so. GGF is the standards behind the Globus enabled grid.
We could ask why CERN/etal couldn't have come up with a slightly more imaginary name?
We can also ask why NSF [nsf.gov] are such suckers for the last 20 years of hype from the pe
Re:When will we do this ourselves? (Score:2)
Three steps to a distributed backup:
Step one: Zip and encrypt your data into 650mb size chunks.
Step two: Name the chucks 'Matrix Reloaded' and 'Britney and Madonna Snog and Shag'
Step three: Share on Kazaa.
Re:When will we do this ourselves? (Score:2)
Ask and you shall receive...
Check out HiveCache [hivecache.com] for grid-based backups for your enterprise. There are lots of great distributed systems out there that do more than just provide a cheap supercomputer replacement. Some of them can actually save your biz money and eliminate some of the more unpleasant tasks your IT staff has to deal with.
Re:When will we do this ourselves? (Score:2)
Thanks for the info, but unfortunately this solution lacks two things I'm looking for: Open sourceness and platform independency.
The idea is right on target though, but I'm still waiting for a solution I can run together with my friends and over the Internet. This also implies public key encryption and an efficient distribution system though
Re:Distributed computing has been here for years (Score:2)
Care to elaborate? Making claims is easy, but I'd like to know what you mean
Re:Distributed computing has been here for years (Score:2)
Being able to perform Google searches is within my abilities, but I fail to see why you seem to wish to hold on to a piece of information which is freely available.
I'm sure I could find the references you originally implied, but I would be unable to determine if what I find is indeed what you were referring to in the first place. Your comment was far too ambiguous to make it clear what you meant.
I'm sure
Buzzwords (Score:2)
I could tell you all about network queueing systems though, home grown, freely available open source stuff and commercially available systems.
As I said, "Grid computing" is nothing new, it's been in use for decades by those of us who know better than to waste 95% of the computing power they've purchased.
Distrib
Re:Distributed computing has been here for years (Score:3, Funny)
I'm 76, you insensitive clod!
but according to ICDCS [ohio-state.edu], it is older than my lesbian granddaughter.
Datagrid homepage (Score:4, Informative)
Re:Datagrid homepage (Score:4, Funny)
Here's their "intranet", which is publically accessible:
http://eu-datagrid.web.cern.ch/eu-datagrid/intrane t_home.htm
You can't access all the reports, but you can get lots of design documents and stuff. It's done in FrontPage and riddled with spelling errors. Not promising.
Re:Datagrid homepage (Score:3, Informative)
CERN didn't invent the web.. (Score:2, Funny)
Al Gore financed the early web (Score:2)
Re:CERN didn't invent the web.. (Score:1)
Re:CERN didn't invent the web.. (Score:2)
The Grid? (Score:2, Funny)
Jason
ProfQuotes [profquotes.com]
Re:The Grid? (Score:2, Flamebait)
You mean like "It has no substance"?
Don't diss it (Score:3, Interesting)
A friend of mine is working on part of the grid for his PHD in the uk
and once you have watched him transfer 700 MB files from the uk to
Switzerland in under a minute you realise that they aren't joking.
I guess their next step is running quake over it!
Re:Don't diss it (Score:1)
They already did it... [eweek.com]
Guess what is coming next? [butterfly.net]
But I am wait ing for this one [devnull.net].
Re:Don't diss it (Score:2)
Power to the Grid! (Score:4, Funny)
And electrical power from how many?
Re:Power to the Grid! (Score:2)
Re:Power to the Grid! (Score:2)
Not really, but then again I only do it part-time, as a volounteer.
Imagine... (Score:1)
Nobody can be told what the Grid is... (Score:2, Funny)
From Reference.com [reference.com]:
Entry: matrix
Function: noun
Definition: origin
Synonyms: cast, forge, form, grid, model, mold, origin, pattern, source, womb
Concept: origin/source
Source: Roget's Interactive Thesaurus, First Edition (v 1.0.0).
What they do at CERN (Score:5, Insightful)
Sorry, got to disagree with you there... (Score:2)
If you were to mention particle physics then Mr. Average would probably still be none the wiser - to him it would still be "men in white coats with massive dome-shaped heads" stuff. However, if you mention the world wide web, the chances are that he'll know what you're
Re:Sorry, got to disagree with you there... (Score:2)
Well, I guess I better clean up aound here and experiment on something.
Re:What they do at CERN (Score:4, Insightful)
Urban Legend [snopes.com]
CERN's site is down (Score:1)
"The equivalent of" (Score:2)
Re:"The equivalent of" (Score:2)
Because they're still niche terms, just like gigabyte was a few years ago. They're not meaningful to mainstream readership yet (though terabyte likely will be soon, as consumer demand for digital video storage continues to increase).
You will also be disappointed if you expected USA Today articles to mention megaparsecs and kiloTesla anytime soon.
Re:"The equivalent of" (Score:3, Informative)
For example - I bought an 80GB drive for a server the other day.....
Secretary : "80GB? how big is that?"
Me: "Well...... if you presume that there are:
80 characters across a page by 66 lines down, you get approx 4000 characters per page of (dense) text.
So, 80 thousand million divided by 4000 gives you 20 million pages of text."
Secretary (mildly impressed): "That's a lot!"
Now, she has no real idea how muc
Mmm, supercollider (Score:4, Funny)
Re:Mmm, supercollider (Score:2, Funny)
The Future For Technology (Score:3, Insightful)
This would mean that silly frivilous things like email apps etc would literally jump to attention regardless as to the size of the file I am working on, rather than as now everything slowly to treacle...
Question is paying for it. It would have to be cheap, built into the operating system and require fast broadband to transfer the data to the servers... Few year yet probably...
Re:The Future For Technology (Score:2)
..as a user of GRID (Score:2, Informative)
Correction (Score:2, Funny)
I thought Al Gore invented the Web as part of that Internet invention thingy of his.
Which parts? (Score:3, Funny)
Which parts? The "BA..." or the "...NG!" or the "What the f**k was that?" 8-)
Angels & Demons (Score:2)
It wasn't as good as "The DaVinci Code" [danbrown.com] though. (Possibly one of the most interesting books I've ever read.)
NOTE: If you want to read the books, don't look at the pictures on the websit
Oh yeah... (Score:2)
What's their logo [web.cern.ch] all about? 2 rings and 5 lines going out from it?? Rotate the logo and you'll see three 6's!!! SATANISTS!!!
I used to work there... (Score:3, Informative)
Now it seems they want to buy floor space at other institutions around the world.
More fun facts - at the time I left they had 5 STK Powderhorn silos, holding their current data. Prediction for LHC requirements (including better tape storage densities) was that they would need another 40 silos. If you've seen an STK Powderhorn, then you know just how ig the things are. So another building was to be built just for these silos.
Oh, and as someone pointed out, the 15,000 TBs a year is just the data that gets kept - the live data from the detectors is preprocessed in the computer centre and "thinned out". The data rates coming into the computer centre are truly mind-boggling.
Re:I used to work there... (Score:2, Informative)
Obligatory Reference, Please Mod Down (Score:2)
Re:Cue skynet has ben launched jokes... (Score:2)
Re:question (Score:2, Interesting)
the experiment to create artifical black holes is another story, but yes, it will take place in this Collider (LHC).
Anyway, you don't have to be afraid, black holes which doesn't reach the critical mass will "eat up" themselves (radiating out their energy to the environment), so they'll eventually evaporate ( according to Prof. Hawking. The experiment will only create very-very-very small black holes, which will "live" only for some nanosecundum or less
)
you should check out http://www.hawkin
Re:question (Score:2)