Slashdot Log In
Tracking Near-Earth Meteors With a 1.1 Petabyte Database
Posted by
Soulskill
on Fri Aug 08, 2008 07:05 PM
from the outer-space-and-computing-make-for-big-numbers dept.
from the outer-space-and-computing-make-for-big-numbers dept.
Lucas123 writes "The latest and most ambitious attempt to detect 'near-Earth objects' (NEOs) is the Panoramic Survey Telescope and Rapid Response System, or Pan-STARRS. When it's fully operational several years from now, it will have four telescopes, each with a 1.4-gigapixel camera. The system is expected to be able to track virtually all NEOs larger than 300 meters in diameter as well as many smaller ones. Rather than turning to an expensive supercomputer equipped with hundreds or thousands of processors, Pan-STARRS will use a cluster of 50 PC servers connected to 1.1 petabytes of disk storage via fast Infiniband networking gear."
Related Stories
This discussion has been archived.
No new comments can be posted.
The Fine Print: The following comments are owned by whoever posted them. We are not responsible for them in any way.
Full
Abbreviated
Hidden
Loading... please wait.
That's nothing (Score:5, Funny)
Back on my C64 I used to track incoming meteors and asteroids, and then blast them into smaller polygons that were much safer, but still a nuisance.
Re: (Score:2)
Re: (Score:1, Funny)
NASA likes to call them weather balloons.
Re: (Score:2)
Oh they are indeed weather balloons... weather balloons from Betelgeuse carrying a payload suspended from the balloons, which is piloted by grey aliens.
Re: (Score:2)
Did you ever spot any flying saucers?
not yet [sonotaco.com]
Re: (Score:2)
Wouldn't call them safer. =)
As long as they where large, all you had to do was fly around them.
It was the myriad of fragments that killed you in the end.
Personally, I preferred the Vectrex [wikipedia.org] version over the C64 one though. =)
good (Score:4, Funny)
Re: (Score:2, Funny)
Near-Earth Meteors ? (Score:5, Informative)
Meteors have to hit the Earth's atmosphere, so they are by definition near-Earth. The article is talking about near-Earth asteroids, most of which are big enough that we'll have big problems if they ever turn into meteors.
Re:Near-Earth Meteors ? (Score:4, Interesting)
Larger than a meteoroid, the object is an asteroid; smaller than that, it is interplanetary dust.
And since this is an article dealing with NEOs...
The NEO definition includes larger objects, up to 50 m in diameter, to this category.
Parent
Re: (Score:2)
Good homework, as far as it goes. Unfortunately, you missed the bit at the top of the page that said
Re: (Score:2)
Indeed [geosociety.org]
CC.
In case of Armageddon (Score:2, Funny)
Re: (Score:2)
Don't forget the animal crackers!
Re: (Score:1)
Do animals have good security?
Re: (Score:2, Insightful)
"...Near-Earth Meteors..." (Score:3, Informative)
SQL eh? (Score:4, Funny)
Re: (Score:3, Funny)
No THX-1138 XKCD); DROP TABLE asteroids;
Re: (Score:2)
Asteroid 17234 has the temporary name of 2000 EL11, so maybe for a suitable donation to the discoverer you could get your wish.
Hang on (Score:3, Interesting)
Re:Hang on (Score:5, Informative)
Actually, Pan-STARRS is the el cheapo quick and dirty version of the near-Earth survey. Look up the LSST (the Large Synoptic Survey Telescope), currently under construction. http://www.lsst.org/ [lsst.org]
Google volunteered to be involved in the data handling, and Bill Gates and Charles Simonyi have contributed 30 million dollars to the construction. During the initial design, the astronomers actually said, "By the time the LSST goes online (2014) we expect that Moore's Law will allow us to process the data stream."
Parent
Re: LSST will just be the latest in a line. (Score:2)
Sure, but by the time LSST goes online (2014), PS1 (the prototype on Maui, assuming that the full PS4 system is never even built) will have tiled the entire visible sky (90N to I dunno, -60 to -75S, I'd imagine) something like 120 times... so LSST can just look for whatever's left. :)
And it's not like these are the first, of course, NASA JPL's NEAT (Near-Earth Asteroid Tracking) system has been running for years and years.
One thing to consider, though - automated survey systems like these don't just find th
This is so misguided (Score:5, Funny)
I'm sorry, but with 1.1 Petabytes of storage and 1.4-gigapixel cameras, they should be focussing on porn-stars, not Pan-STARRS.
Re: (Score:1)
Porn is pretty much not a huge fan of HD. The human body doesn't look all that good up-close and in high-res. Or at least, not as good as the fantasy that fills in over lower res.
Re: (Score:3, Insightful)
Re: (Score:2)
I'm sorry, but with 1.1 Petabytes of storage and 1.4-gigapixel cameras, they should be focussing on porn-stars, not Pan-STARRS.
Most of them, I'd rather not. In high definition, all the blemishes come out such as zits, scars, lousy makeup and so on. There are exceptions but I'd say the number of potential porn actresses drops by about 90% to only include the really, really stunningly beautiful girls. The cheap porn flick I'd rather pay to not see in HD...
Re: (Score:2)
cheap porn flick I'd rather pay to not see in HD...
Where all the participants (men and women) look and
sound like Carl from Aqua Teen Hunger Force.
Now in 1080p!
Re: (Score:3, Insightful)
Doesn't have to be HD. Think massively widescreen orgy.
Individual boxen? (Score:3, Insightful)
Re: (Score:2)
It's in the article: they got the SQL-server on an academic-license and 50 computers for the same shoe-string butget.
1.1 Petabytes! (Score:2)
Great Related Sci-Fi (Score:2)
If the topic interests you at all, check out Michael Flynn's Firestar series. Near-future science fiction. I contend that it's the best sci-fi series ever written. No, these aren't affiliate links.
http://www.amazon.com/Firestar-Michael-Flynn/dp/0812530063/ref=sr_1_1?ie=UTF8&s=books&qid=1218256538&sr=8-1 [amazon.com]
http://www.amazon.com/Rogue-Star-Firestar-Michael-Flynn/dp/0812542991/ref=pd_sim_b_1 [amazon.com]
http://www.amazon.com/Lodestar-Firestar-Michael-Flynn/dp/0812542967/ref=pd_sim_b_1 [amazon.com]
http://www.amazon.com/Falli [amazon.com]
S&T article on these guys (Score:2)
Wasted equipment..... (Score:2)
I used to use an Atari 2600. Pretty simple set up. Small, compact and plugged right into my TV. Plus, with the addition of a cheap little switch, I could blow up incoming asteroids during commercial breaks. .....sigh..... How I miss my Atari.
Real men don't need controllers with a dozen buttons to beat games. They only need a joystick and a button.
Re: (Score:2)
Haven't you seen the Jakks Atari classics 10 games in a joystick? They have miniaturised the console electronics so that you get a whole Atari 2600 system plus 10 games (Gravitar, Asteroids, Real Sports Volleyball, Centipede, Adventure, Pong, Missile Command, Breakout, Yars' Revenge and Circus Atari) all in a single joystick [amazon.com].
Re: (Score:2)
Better still, Atari Flashback 2 [amazon.com].
It totally amazed me that these console systems are still on sale after 30 years, but make great entertainment for a 80's themed party.
The GrayWulf cluster from Microsoft (Score:2)
From TFA:
A new feature allowing big distributed databases with MS Server 2008. This setup is called a GrayWulf cluster.
I wonder if this was the idea of the new PR company MS hired after the marketing failure of Vista?
Re: (Score:2)
I can't really see the phrase 'Imagine a GrayWulf cluster of these!' catching on on Slashdot. Nice try, Microsoft.
Mil Sats? Govt. Censorship? (Score:2)
So anything that can detect, track and determine the orbits of NEOs efficiently has got to be good at picking up all sorts of things in space. Like military satellites.
Will (any) governments be able to "edit" the database so that things they don't want noticed will remain so? On the other hand, will they be providing this project with satellite data to prevent false positives? Also, will the database be available to the public/amateur astronomers so that things like the recent "ghost" image can be found
Re:A 50 PC cluster that processes 1.1Pb of data? (Score:4, Interesting)
So what is worth tracking that is 1.1Pb of data? Are there really that many NEO that are to be concerned about? 1.1Pb is a LOT of data to manage, even with a cluster of "50 PC's". Will this data be used for modeling or just for tracking or a combination of both? I'm interested in the technical explanation for needing that large of storage.
Parent
Re: (Score:2)
It probably has to do with storing massive amounts of 1.4 gigapixel images in an uncompressed format. It should roughly be about 2-4GB per image depending on what's in the image and if they do any non-lossless post compression, and considering that they're going to be receiving data from 4 telescopes, that's a lot of storage that they'll be using up in just one night's worth of images, let alone years worth of data.
Re:A 50 PC cluster that processes 1.1Pb of data? (Score:5, Informative)
Never mind, should have read the website [hawaii.edu]
Each raw image from a single Pan-STARRS camera will contain 2 Gbytes (2 bytes per pixel). In full survey mode, typical exposures last 30 seconds, so the raw data rate is several terabytes per night for the full telescope. The amount of data produced by Pan-STARRS is so large that it will not be practical to archive every image. Software techniques are therefore being developed to extract the important information from the images, while allowing less crucial information to be discarded.
Parent
Re: (Score:2)
Well. This actually is kind of obvious:
If you want to track something, you have to know that there _is_ something.
If you already know the asteroid, and have the trajectory calculated for a comparison, fine. But then you wouldnt need the survey. Its to find new/changed stuff.
So they need to have at least 1 full reference copy of the sky of the survey on storage to compare against, to look for changed "star" positions or new "points".
Most likely they would need a lot more than that, because of various reasons
Re: (Score:2)
I'm with you on that one AND isn't it a bit old hat this week to refer to mere 1.1PB storage arrays as big? Don't we have news of actual databases that big or bigger?
When we have 1.5 TB drives being made ready for market isn't a PB storage system rather mundane, or at least getting there? Simple RAID-5 Terabyte arrays are common. When you split the storage space across 50 systems, meh, it doesn't really sound like that much. Sure, it's a lot of hardware, and I bet it makes some fan noise but is it really n
Re: (Score:2, Informative)
They are storing (almost) raw data to analyze. You can probably spot the position of some rocks in one "frame." To find their speed since the last frame, you need at least another frame. To predict their future position, a shitload more frames (remember that we can't solve meteoroid orbital equations precisely, we need many more points so the orbits can be fitted to a bunch of aproximations with lots of empirical constants).
Since now we are talking probabilities, you need a large number of measurements to i
Data Analysis (Score:2)
So what is worth tracking that is 1.1Pb of data? Are there really that many NEO that are to be concerned about? 1.1Pb is a LOT of data to manage, even with a cluster of "50 PC's". Will this data be used for modeling or just for tracking or a combination of both? I'm interested in the technical explanation for needing that large of storage.
My dad is currently in Hawaii, working on the data analysis software for Pan-STARRS, so I have a vague idea of what it is about.
It's not the NEO that use 1.1Pb of data, it's storage of the raw (high resolution) data from the telescope. In fact, I'm not even sure they CAN archive all the images.
The point of having it is finding NEO, computing their orbits, so we can be able to predict our doom (and maybe do something about it before it happens).
Actually I think the system also has civilian space-flight appl
Re: (Score:3, Insightful)
So, it will be able to track those objects of such size or greater that would , unavoidably, sterilize our planet ... yet be unable to track those ( dia 300m ) whose paths we actually might be able to deflect ...
but it is a start and is to be applauded....
Who says we can't deflect a 1km object? The point is, you can't do it Armageddon-style at the last minute. But you can give it a small push in some direction 10 orbits (or 30 years) before it hits us. That's why orbit predictions need to be 50 years ahead.
Re: (Score:2)
True, yet the odds are enormously larger that the number of objects smaller than 300m is considerably greater than the number that can be detected. This, logically, makes it relatively pointless to search for 'the biggies' when the risk of a cataclysmic sterilizing event secondary to a smaller, less than 300m , object has to be considerably higher.
A 300m object will make a big BOOM, but it won't kill many people unless it lands on Manhattan (which is unlikely). If we do a little game theory and use the number of human lifes lost as cost, we have to multiply the probability of an event by the number of people it will kill.
If the big asteroids (of the kind that made dinosaurs go extinct) happen once every 70 million years on average, and we assume they kill all of us 7 billions people when they hit, then the big asteroids kill 100 people a year on aver
Re: (Score:2)
"The Earth is currently closed for business due to a reboot, we will be back on-line in...ohh let's see.. another 4.5 billion years. Thank you for waiting!"