Slashdot Log In
Computer Date Glitch May Limit Next Shuttle Launch
Posted by
ScuttleMonkey
on Mon Nov 06, 2006 10:28 PM
from the 1999-called-they-want-their-date-bugs-back dept.
from the 1999-called-they-want-their-date-bugs-back dept.
n3hat writes "Reuters reports that the next Space Shuttle mission may have to be deferred if it gets too close to the New Year because the onboard computers do not handle the changing of the date in the same way as the ground computers. From the article: '"The shuttle computers were never envisioned to fly through a year-end changeover," space shuttle program manager Wayne Hale told a briefing. The problem, according to Hale, is that the shuttle's computers do not reset to day one, as ground-based systems that support shuttle navigation do. Instead, after December 31, the 365th day of the year, shuttle computers figure January 1 is just day 366."
Related Stories
[+]
NASA Avoids "Happy New Year" On Shuttle 181 comments
ClickOnThis noted that NASA is actually avoiding a Shuttle in Space over New Years. It says
"The worry is that shuttle computers aren't designed to make the change from the 365th day of the old year to the first day of the new year while in flight. NASA has never had a shuttle in space December 31 or January 1. 'We've just never had the computers up and going when we've transitioned from one year to another,' said Discovery astronaut Joan Higginbotham. 'We're not really sure how they're going to operate.'" You may notice some deja vu while reading this story. Sorry. Not much happens on Sundays :)
This discussion has been archived.
No new comments can be posted.
The Fine Print: The following comments are owned by whoever posted them. We are not responsible for them in any way.
Full
Abbreviated
Hidden
Loading... please wait.
wtf? (Score:5, Funny)
Re:wtf? (Score:5, Funny)
It was built by the government.
Parent
Re: (Score:3, Insightful)
Re:wtf? (Score:5, Insightful)
Parent
Re:wtf? (Score:4, Insightful)
Parent
Re: (Score:3, Interesting)
You'd think someone at NASA could just do the five-second fix and be done with it though. It's, what, three lines of code?
Big hint, NASA:
while ($day >= 366)
{
$year++;
$day -= 365;
}
Re: (Score:3, Informative)
It would be far safer to just delay the launch a few days.
Re:wtf? (Score:5, Insightful)
Parent
You're clearly not a very good SA (Score:5, Insightful)
^^and here's the key -- it's something you don't know about^^
Now, you make your little 5-second fix, and send seven astronauts into space.
New Year's Eve rolls around, and suddenly the mission started on day 360 and it's now day 1. Holy crap, says the scrubber, we have to scrub as though it's a 359-day mission, instead of a lousy 6.
Scrubbers go into overtime, and break. (Or, scrubber math is done in eight bits, and they think the shuttle's still on the ground and not ready to launch for another ~100 days due to integer roll-over. Or any other set of unforseen possibilities.)
Next, astronauts die of CO2 poisoning because the scrubber subsystem has been compromised.
Great fix, mister five-second-coder.
Parent
Xmas at home (Score:5, Funny)
Imagine you are a member of the shuttle design team and you can make a choice (for the next 20 years) to either know for sure that you're with the kids at home on X-mas and New Year .... or you can suggest a software feature that could result in your New Year's Eve being spoiled down the road because you have to be for days in a dumb control room. Hey, what would you do??
And I still remember, when I was a kid, that we had that Apollo flight during X-mas. I think it was the one that would for the first time go behind the moon. Someone in the control room that year made it into an important enough person on the Shuttle program so that this WOULD NEVER HAPPEN AGAIN. :-)
Parent
Re: (Score:3)
That said, I'll agree that NASA's software is certainly a heck of a lot more stable than Debian. After all, this is rocket science.
Re: (Score:3, Informative)
If they ran their shuttles on something like Debian stable it would be a rock solid platform and probably end up saving them lots of money. Or am I missing something here.
Yes, you are. Primarily the fact that Debian Stable isn't even Carrier Grade, and certainly not qualified for life support. Ie, "doesn't crash in the midst of re-entry."
Keep in mind that the shuttle computers are highly redundant (I believe there are three main computers, and three backups of each component?), monitor a HUGE number
Re:wtf? (Score:5, Interesting)
Parent
Re:wtf? (Score:4, Interesting)
Here is a decent source of HAL/S examples:
http://www.hq.nasa.gov/office/pao/History/compute
Now, look at the procedure called 'read_accel', about 1/4 down the page.
Midway through, there is a ton of gunk. That's the HAL/S maths for you:
the program is allowed to use three lines to express mathematical code, to 'mimic' math
in code. Now, this is the '70s; it's of little wonder that they weren't worried about the
date switch so much as making sure that:
1) the compiler produced code that could be checked & double checked to be '100%' failure proof and at least be resilient to problems.
2) It had to deal with the beast of being machine independent & easily understandable to the PL/I & FORTRAN programmers of the day
3) It also had to make sure that tasks were scheduled properly & run when specific interrupts happened. This is the Norm for Ada/SPARK now, but HAL/S was pretty much the pioneer here within the Aerospace field.
Parent
Not to nit-pick, but... (Score:4, Interesting)
I can't believe this was moderated as +5, Insightful.
The shuttle was designed WELL OVER a quarter century ago. A quarter century ago, they had done some much design and testing, they were able to have the maiden flight (STS-1, Columbia launched in April 1981). Shuttle design and specification requirements analysis began in October of 1968. VMS, CP/M, PC-DOS, and 4BSD did not exist when the Shuttle was designed.
You must be thinking of Multics, which was the closest thing to a modern operating system that existed in the 1960s.
Seriously, you have no idea how old the Shuttle design is. I have no idea why they keep using it after the great work done 20 years ago by Richard Feynman [wikipedia.org] who showed that NASA's shuttle design was about 1/100 flights unreliable. For the record, we've sent up 200 missions and had 2 shuttles blown up. The Space Shuttle is a piece of garbage, and NASA has wasted billions exploring low Earth orbit, rather than do something more useful.
Parent
Actually, the numbers differ (Score:5, Informative)
And, for the record, there have been 117 launches, according to wiki, which I will take as accurate enough for this discussion (far less than 200).
*yes, IWAAE (I was an aerospace engineer) working for NASA, and was involved with shuttle payloads and structural reliability analyses.
Parent
Re: (Score:3, Funny)
Right. It only seems like Debian stable releases are a quarter century apart.
Re:wtf? (Score:5, Insightful)
Standard parts don't like being bombarded with radiation. Standard operating systems aren't fault-tolerant.
Parent
Re:wtf? (Score:5, Interesting)
While I'm not thoroughly educated on this particular subject, I would say that it's a pretty good chance that the flight computers on the shuttles are based on technology that's at least 15 years old (all shuttles underwent a "glass cockpit" update in the mid-late 90s). You don't see NASA cutting a purchase order to cdwg.com when the newest AMD or Intel offering is announced and stuffing that into the shuttles. This stuff is designed, planned, coded for and integrated over a number of years and is very static. No changes. If there has to be changes, they're done under a quality control methods so strict that, yes, Duke Nukem 3D might see the light of day first.
And that's just the hardware part.
On the software side, I'd say you're probably looking at stuff written in any assortment of "classic" languages such as ADA, COBOL, or worse. Due to the nature of the metric f*k ton of sensors, mechanical servos, data inputs, and other such esoteric (and dated) hardware on the shuttles, the software must control, query, parse and monitor, the software is pretty darn married to the platform it runs on.
So, before blurting "D0odz, just instahl leenux n yr shuttlz (deeban stble rox wif glox!)" Give it some deeper thought. There's likely a darn good reason why things are they way they are (bugs not withstanding) when it comes to large flying contraptions that are designed to safely get 7 people 300 miles up, keep them there for a week (or two) and get them home. Sometimes simple things (to you and I) such as a year roll-over are outside the scope when it comes to designing systems to do what the shuttle does.
Parent
Well.... (Score:4, Insightful)
*ducks and runs for cover*
Seriously though- they never "envisioned" a mission occuring over the end-of-year? Let me guess: a defense (space) contractor designed the systems.
Yay! (Score:3, Funny)
Well, of course. (Score:4, Funny)
Do you know how many eligible 35 year old computer bachelors there are out there? Ill tell you: none. Of course the shuttle computers can't get a date.
I'm enjoying a little schadenfreude... (Score:5, Informative)
Granted, the work they do is very impressive and the process is very exacting. But come on...they haven't been able to fix a simple year rollover event in 30 years?!?
From the Fast Company article:
I would say that requiring a reboot every year on December 31 is a pretty huge error. In this case, it is forcing NASA to launch earlier than they otherwise would wish. And this isn't the first time this type of problem has caused problems. The New Scientist has a similar article [newscientistspace.com] that goes into more detail:
At least they know it's a problem... (Score:3, Insightful)
...which is more than many software development processes would reveal. Chances are that this known restriction is on a check-list which every shuttle mission has to be checked against, and the list would exist precisely because the software development and verification process is so solid and conservative.
That's pronounced "design decision" (Score:4, Insightful)
I would say that requiring a reboot every year on December 31 is a pretty huge error.
I wouldn't. When you're designing something like Shuttle software that has to work absolutely flawlessly 100% of the time, you don't put in any frills. And on something that is only ever in space for 10-15 consecutive days at most, year-end handling is most certainly a frill. (If you are a professional software developer, it ought to be obvious just how many things could break by adding a feature like that. If the original design calls for a monotonically increasing day number, for example, there's very likely to be some code that relies on that, so you have to go through the entire system, checking everything that even touches the day counter to ensure it can handle a reset from 365 to 1--and then check everything that uses those routines, and so on and so on.)
I suspect this is routine to NASA, and the reporter just blew it out of proportion. After all, Windows can handle end-of-year rollover, so if the Shuttle can't then it's broken, right?
Parent
GP is right. NASA programmers screwed up. (Score:3, Insightful)
do a rollover, it's that the ground computers do a rollover, and the shuttle computers don't.
If it ain't broke (Score:4, Interesting)
Seriously, though, it's worked fine. The software has not killed anyone. They can either fix it and modify a very critical system on an enormously complex vehicle, or they can move the launch date around a few days, which they seem to do for every launch anyway. B is probably safer and more predictable.
Problem seems obvious (Score:3, Funny)
Date/Time Formats (Score:5, Informative)
The end-of-year rollover depends on the leap year and leap second (if any), and has traditionally been a source of problems.
Could it be... dates are hard? (Score:4, Insightful)
Every date math equation I've seen has all sorts of wierd magic numbers in them where it isn't clear how those numbers were obtained. This may work just fine in day to day computations, but oddball bugs in date calculations can lead to some very wierd errors. Look at the C library sometime for the date functions. It's quite impressive.
Perhaps when the shuttles were designed, the inability to schedule across the new year was acceptable to avoid introducing odd bugs in the program to keep the software provably correct. Ground systems, which can be repaired in the middle of a mission easily, can be a little less bug-free, since a miscalculation won't cause the Earth to suddenly veer off course.
Hold up, everybody (Score:5, Insightful)
You people with your "WTF NASA SUXORS THIS IS EASY FIX!!!11!!1!one!!" need to stop and think for a second. This is a space application that carries HUMAN BEINGS! Think about how hard it will be to get this "easy fix" qualified, proven, documented, etc. Its not an easy task. A formal qualification test on the systems I work on (military land- and air-, but not space-based navigation software) can take months, and require all sorts of tests and documentation. Anything that isn't formally tested (i.e. run in a van, on a plane, etc) must be shown to not fail in any way; all exceptions handled, no bad data can cause an undesireable state, etc. I would hate to see the type of scrutiny that the Shuttle software goes through (although I could probably call somebody in our Space division across the street and find out).
Second, I don't know exact specifics, but based on the information provided, I think this "glitch" will have to do with the data/time difference between ground stations and the Shuttle computers. Things like message time stamping between the Earth and the Shuttle, etc, will be wrong, and things could be garbled or just dropped all together. The navigation systems themselves should not be terribly impacted since the date will just roll to the next day. Inertial instrument samples will continue to flow in and be correctly time stamped, be it the 366th or 400th or 500th day.
Hard To Believe (Score:3)
http://en.wikipedia.org/wiki/List_of_space_shuttl
The closest I could find was STS-103, the HST servicing mission in '99. Launched December 19th and lasted 7d 23h.
Actually most error free software in the world (Score:4, Informative)
How many other groups can deliver a half million lines of code with only 1 error (and no, not this issue. And as far as this being an error or bug, it really isn't. It's a know design restriction on a system that just works. Do you really want to go redesign a large chunk and possibly introduce life threatening bugs, or work within the known design window for the system.
A [sorta] Rocket Scientist Replies... (Score:4, Insightful)
It's like this: A clock rollover (such as at midnight or the last day of the month or year) always sets something back to zero. That resetting is a risk: Is there something somewhere that doesn't take the rollover into account? It may be an obvious bug, or not so obvious - what if the problem is dynamic? For example, what if system A sends some data and rolls over, and system B rolls over and receives the data? Then it looks like stale data, but isn't. How do you test for dynamic conditions like this?
Dodging this bullet is far, far cheaper than testing for it.
The only time I know of that a shuttle flight software bug affected a flight was uh...STS 2 or 3 or thereabouts. The shuttle often flies an updated load on one or two of its computers before the load is installed on all of them. On this mission, a new load on one GPC dumped (crashed) at T -9 seconds or so, causing everything to shut down automatically. The shuttle launched a day or two later, after the new load was rolled back.
Funny thing was, the same bug had occurred in the training simulators before launch, but was written off as a lack of fidelity of the simulator itself, not a bug in the flight software.
After that, the astronauts really began to appreciate running the real GPCs with the real flight software in the simulators.
PS: Although I work at NASA, this message is my own expression, and not that of NASA or my employer. I am a programmer only, not anyone with any kind of authority or insight except for my experiences here.
No CMMI comments? Are there real developers here? (Score:5, Informative)
Here is a list of participants in this program. [cmu.edu]
And here [cmu.edu] is a general overview of what CMMI is.
And just to put it into perspective, when I was last working with CMMI, there were only 3 companies certfied at level 5. Nasa, Motorola, and another one I can't remember. I am sure that has changed but nonetheless, it's a big deal and shows a serious effort to do things in a controlled, measureable, testable, way.
I only bring this up to counter the ridiculous "solutions" that some have proposed on this site.
"I can fix that in 3 lines of code".
Well, great. That might work at YOUR company. But please don't do that at NASA. Despite what many think here, NASA is a top-notch software development house. And I would expect nothing less given what is at stake.
Re: (Score:3, Informative)
Re: (Score:3, Insightful)
Re: (Score:3, Interesting)
They write the right stuff [fastcompany.com]
???
This story is obviously bogus... (Score:3, Interesting)
Re:lame (Score:5, Funny)
Oh, shit! You mean we're not supposed to be following intergalactic star dates?? No wonder those programs I wrote have so many date bugs...
Parent
Re:lame (Score:5, Funny)
Nah, everyone knows geeks are useless at dates because they never get any. Predictable failure, that one.
Parent
Re: (Score:3, Insightful)
Do you even remember why the Y2K thing happened? People saved space back in the day by using a 2 digit year. Hell, in the 1970's, people were using a one digit year to save even more memory and storage space. The Space Shuttle uses very old technology for its computer systems (read: 1970's level technology), and doesn't have much memory. That extra 10 lines of code could make it oversized.
Additionally, making a change to space (or even
Re:How Many Times? (Score:5, Informative)
Parent
Re:How Many Times? (Score:5, Funny)
They put computers in bra straps now? Sheesh, I was just getting used to the old ones, and now this?
Parent
Re:How Many Times? (Score:4, Funny)
Parent
Re: (Score:3, Funny)
Re: (Score:3, Insightful)
Re:How Many Times? (Score:5, Interesting)
Space shuttle's a little different:
http://www.fastcompany.com/online/06/writestuff.h
Here we're talking *six* 9s of *bug-free code* (1 error in 420,000 lines of code in the previous version). Not uptime -- bugs. Mistakes. For the simple reason that if you make a mistake on the Shuttle, people die.
You won't get a Java implementation that bug-free without a crack team of developers working for decades, by which point Java will be just as outdated then as the Shuttle code is now.
Remember -- if it ain't broke, don't fix it!
Parent
Re:Bites me (Score:5, Funny)
That's why every new year all the soldiers climb out the trenches, swap chocolates, cigarettes etc, and shake hands before climbing back in and resuming war the next day.
Parent
Re: (Score:3, Informative)
Or maybe not. The Apollo Guidance Computer software had at least one serious issue in flight: the early versions allowed you to run the launch setup program in space and that would reinitialise the Inertial Measurement Unit so it forgot where you were. I believe someone did that by mistake on Apollo 8, fortunately NASA were able to send them the data to load into the computer to tell i