ISS Computer Failure 289
A number of readers wrote us with news of the computer problems on the International Space Station. Space.com has one of the better writeups on the failure of Russian computers that control the ISS's attitude and some life-support systems. Two out of six computers in a redundant system cannot be rebooted. The space shuttle Atlantis may have its mission extended until the problem is fixed. A NASA spokesman was optimistic that the problem can be resolved; worst-case scenario would be for the shuttle to evacuate everyone onboard the ISS. Engineers are working on the theory (among others) that the failure may have been triggered by new solar panels installed earlier in Atlantis's mission.
Yakov Smirnoff says: (Score:3, Funny)
You need the russian guy from armageddon (Score:5, Funny)
Re:You need the russian guy from armageddon (Score:5, Funny)
Patch Tuesday (Score:2, Funny)
I have to be misreading that (Score:5, Funny)
So the ISS is throwing a temper tantrum? Just put it in time out
Re: (Score:2, Funny)
definition of attitude (Score:5, Informative)
Maybe the new solar panels are a new input to the attitude program - "I am a new solar panel, I need to be pointed this way so that my 1 axis motor can track the sun"
Re: (Score:2)
So the ISS is throwing a temper tantrum? Just put it in time out
How very quaintly 20th century. Pharmaceuticals are the answer these days.
Re: (Score:2)
I thought the same thing. Of course the Russian computers have an attitude problem.
"Beep boop... It's so fucking cold out here... I wish I were back in Siberia, at least there it's only cold 11 months out of the year. We're going to die out here, aren't we?"
Just send up some vodka already, problem fixed. Or, at least addressed.
That's the problem right there (Score:3, Funny)
Two out of six computers in a reduntant system cannot be rebooted.
NASA should have invested in a redundant system, rather than buying a cheap grey-market knockoff.
Re: (Score:2)
It's like an increment of 0.5 degrees Kelvin.
I can sympathize. (Score:5, Funny)
DFMEA (Score:5, Interesting)
Hopefully they're starting with their DFMEA documentation... "guessing" at the problem and having "theories" is probably not a good way to go. Also, it's apparently a common-mode failure, which you shouldn't have in a safety-critical system; generally this is avoided by having different computer hardware and/or completely different code to do the same tasks.
Quite unfortunate that it seems like systems engineering is lacking in more and more disciplines recently, although I suppose it makes good systems engineers more valuable.
My list for this would be something like: "Computer doesn't boot." Possible reasons: "No Power", "Insufficient power", "Corrupt memory", "Broken circuits", etc. Then you go down that tree further and find the root cause. The most disturbing thing is that they had such a major common-mode failure...whatever happened to the "no single points of failure" mantra?
* sigh *
Re: (Score:3, Insightful)
Re: (Score:3, Informative)
Hrm, the summary is different than the article; the article stated that "two of the six computers are running" which means 4 are down, not 2. Whichever is correct, any time more than one computer goes down, you have to look for common-cause failures.
Also, according to the article the US computers don't control attitude thrusters and that particular life support system, so the state of the US computers doesn't matter.
(Note for the anonymous poster above, but I didn't want to post twice: "common cause" mean
Re:DFMEA (Score:4, Informative)
The US computers do however control the CMG's - the backups for the attitude thrusters[1], and the life support for the US side of the station. So even the loss of all the Russian computers wouldn't leave the station in trouble. (Unless CMG desaturation was required - which doesn't happen all that often.)
[1] Next year, IIRC, a second set of CMG's goes active and then the CMG's become primary with the attitude thrusters going into the backup role.
Re:DFMEA (Score:5, Interesting)
These problems are not easy to diagnose when you have hands on capability leave alone 200 miles above Earth.
I do hope that it is sorted out swiftly and the ISS and its occupants remain safe.
Re: (Score:2)
Re: (Score:3, Informative)
"guessing" at the problem and having "theories" is probably not a good way to go.
Welcome to the real world of problem solving. Any solution always starts out as a guess. It's pretty much impossible to solve any problem without eliminating a whole bunch of possibilities (i.e. guessing and having theories). It's likely 10 times harder when you don't have the tools necessary to diagnose this particular problem. (i.e. they need an oscilloscope to look for strange power fluctuations from the new solar array)
System Wide Reboot? (Score:3, Funny)
Re: (Score:3, Interesting)
Re: (Score:2)
Re: (Score:2)
Jurassic ISS? (Score:2)
Hey, here is a crazy idea (Score:3, Interesting)
No, no--I know is sounds crazy. But hear me out. Maybe we could actually pursue something NEW--you know, dare to violate that 30-year-old sacrosanct NASA policy of just repeating themselves over and over again and wasting trillions of $ on contractors and grandiose promises which never amount to squat.
Just a thought.
Re:Hey, here is a crazy idea (Score:5, Interesting)
It's kinda like finding out your house you're current building will cost twice as much as normal.
Do you just leave it half finished and abandon it or do you keep pumping money into it?
Re: (Score:2)
Re:Hey, here is a crazy idea (Score:5, Informative)
The investment in time, money, and energy has already been made. To abandon it now, no matter how dysfunctional it is, would be a bigger waste. If the initiatives to return to the Moon and move on to Mars are going to go forward (and given Congress' past performance in this regard, I highly doubt it), then ISS is a necessary platform to span the gap between the Earth and the Moon. MInd you, when the United States was first thinking of going to the Moon, Werner von Braun put forward the plan to build a space station first, then use it as the assembly point for the journey to the Moon. Then, the platform would already have been established, and the Space Shuttle would have been the next natural extension after the end of Apollo. But the idea was shelved in order to get to the Moon by 1970, and as a result we have the current situation. So, we have done it backwards, but to abandon it now would be truly a giant step in reverse.
Re: (Score:2)
And even if it could, what is the point? We've got to get the payload up there either way--and why not go with the PROVEN, much simpler technology that got us there THE FIRST TIME? There is no need to send up a
Re: (Score:2)
The ISS wasn't built to be a launch platform for the moon or Mars. It was built as an overpriced space laboratory.
I thought the actual reason it was built was "to give the Shuttle somewhere to go"? And its orbit was chosen for Russia's convenience. You would probably want a less inclined orbit for a stop-over to Luna or Mars.
Stopping rule (Score:3, Insightful)
Re: (Score:2)
Half the problem is that the ISS was designed by Congress and the President of the United States (Reagan, Bush I, Clinton), not by NASA, vis-a-vis the NASA budget dance. The White House and Congress were continually changing and reshaping its mission, its design, and eventually decided that we had to partner with other countries if we were ever going to get it built. Ask any old NASA engineer and they'll tell you -- this is not your father's space station.
That said, it is modular, and could be reconfigure
Re: (Score:3, Insightful)
If anybody had ever come up with any mission for which a "way station" served any purpose whatsoever.
But they haven't.
I actually have asked a few NASA engineers I know, and their (private) opinion is unanimous: drop the pointless money-suck into the ocean, ASAP.
Re: (Score:3, Insightful)
Meeting up with a reentry capsule in earth orbit is an excellent idea. The ISS doesn't do anything useful here though. You can just meet the capsule.
Re: (Score:3, Informative)
"One of the intentions was allegedy for it to be used for construction in space."
I doubt that intention was seriously advanced by engineers who know anything about it. To build something in space needs the parts (as pre-assembled on the ground as possible) and someone to bolt them together. A different big thing to build first isn't a great idea.
"Clearly there are advantages to being able to build spacecraft and such not intended for travel through atmosphere."
Next to the advantages of getting to do the bu
Re: (Score:2)
The investment in time, money, and energy has already been made. To abandon it now, no matter how dysfunctional it is, would be a bigger waste.
The sunk cost fallacy [wikipedia.org]. Just because the international community has pissed away however many $billions on this is not a good reason to go on doing so. There are something like 15 shuttle missions still planned, which could be scrapped, saving countless more $billions and probably even lives (and I don't just mean if one of the remaining shuttles happens to blow
Re: (Score:2, Insightful)
Yeah, 'trillians' of $$$...
Oh wait, no. Department budgets for 2007 [whitehouse.gov]:
Sort your fucking country out. Just a thought.
Re: (Score:2)
Re: (Score:2)
Re: (Score:2)
On the other hand the Hubble cost A LOT less, and even though Nasa was/has(?) abandoned it, it' still provids a lot of valuable information. Even with some components broken, the Hubble has really kicked
Re: (Score:2)
Except that once we abandon ISS, start planning again,
it will be 20~ years gone, then we will start putting
up the successor device. And it will be a 30 year
old sacrosanct obsolete POS. With calls to evacuate
it and start new.
Re: (Score:2)
Alternatively, we could sell it to a private concern, like Bigelow [bigelowaerospace.com] or Virgin [virgingalactic.com] or a private consortium who happens to have a few (ok, many, many) billions lying around.
Then again, the buyers might require that the ISS be boosted into a more usable, higher orbit before they take posession.
Re: (Score:2)
How bad a worst case? (Score:3, Interesting)
The stated worst case scenario is that the ISS will need to be evacuated, but if the remaining gyros are being overwhelmed, might the station enter an unrecoverable spin state before the problem is resolved?
Re: (Score:2, Funny)
Re: (Score:2, Informative)
If they need to evacuate, there are sufficient Soyuz escape modules (tried and tested as the standard re-entry module used by Cosmonauts for the last 40+ years with an almost unchanged design) for all of the current crew capacity on the ISS. Well, I hope so for there sake, or we might have a spaceborn version of what happened to the unfortunate inhabitants of the S.S. Titanic, where passengers vastly outnumbered available spaces on the lifeboats of the supposedly unsinkable ship.
Re:How bad a worst case? (Score:5, Interesting)
Evacuating ISS is always a last resort, because should something happen to it while unoccupied, it'd be a total loss. We won't have another shuttle ready for a month or so, and I believe the Russians just recently did a Soyuz exchange, so there'd be no quick return, even if the problems were fixed. With attitude control in question, it could become too unstable for even a shuttle or Soyuz docking to occur.
Does the ethnicity matter? (Score:5, Insightful)
It's not like the Russians would send crappy stuff up to the ISS anyways, they would put all their best into it. And the Russians have a history of having some excellent mathematicians.
Re: (Score:3, Funny)
Re: (Score:2)
Re: (Score:3, Informative)
This [amazon.com] is an interesting read on this subject. The answer to your question is that the fact that the computers are Russian probably does matter.
It's not that the Russian mathematicians ar
Re: (Score:2)
The answer to your question is that the fact that the computers are Russian probably does matter.
Could you elaborate please on what you mean by it "probably does matter"? For the audience that doesn't have time to read Olberg today?Re: (Score:3, Funny)
After all Redmond is in USA.
*ducks*
Re: (Score:2)
eventually the plan is to have a fully redundant self sustaining russian AND united states attitude control, life support, communications etc etc. but the module with the
Re: (Score:3, Insightful)
The truth is sadly quite the opposite. For example, they installed an unmodified Elektron oxygen generation system - despite the fact that it had a long and less than stellar record when installed on Mir.
Re: (Score:2)
Re: (Score:3, Insightful)
Re: (Score:2)
though a little engineering/science competitiveness certainly beats lobbing missiles at each other, right?
I suspect the current mood also has something to do with the months of reading about the 'failure' of American space shuttles, and the saving grace of Russian resupply missions. Next year it'll be something about the failure of the German-built communications systems, and thank God for the British-built semaphore flags.
Re: (Score:2)
Russian parts, American parts, (Score:5, Funny)
(un)cooperation (Score:2, Insightful)
What does that mean ? That NASA doesn't relay communication to the russians so that they can start working on the problem right away ? Then they have more serious issues than a software error. The whole thing sounds like there is no real trust between the two agency. I understand that you want to give work to everybody and maybe keep some technology
Re: (Score:2)
Organizing your priorities... (Score:5, Funny)
"The lights, the fans and, thank God, the potty, all those things are working," Suffredini said.
Well at least he has his priorities in order. God knows you don't want anyone looking into the Hubble to see the ISS going by with your ass hanging out of the window.
Absolut Terror (Score:4, Funny)
From the article: The computer failures have left the station without the use of its Russian attitude control
I guess the liquor cabinet door in the ISS is computer controlled.
NASA has a problem alright, but not with the ISS (Score:5, Insightful)
When the shuttle launched last week, the headline quoting NASA was 'perfect launch'.
Then, we heard this: "NASA says shuttle damage is not serious"
Huh? I thought it was 'perfect'...?
'NASA studies gap in shuttle's shields' - "not appearing to be an urgent problem" - "Other than that, the vehicle is very clean. NASA's Shannon said." http://msnbcmedia.msn.com/j/msnbc/Components/Phot
"The first shuttle launch of the year helped put NASA back on track after a run of bad luck and scandal on the ground during the first half of the year."
Next, we get this: "NASA checks into potential hit on shuttle"
"Sensors on the shuttle Atlantis have recorded hits on the leading edges of the wings, around the area where Columbia suffered fatal damage four years ago, NASA officials said Tuesday. However, they emphasized that the hits probably did no damage to Atlantis."
"What we have seen does not indicate that we have been hit by anything," NASA's Shannon said."
Huh? Do we have a hit or not...? Shannon has quite the golden tongue.
My point is that NASA always says "perfect launch", even when they are sitting on data that suggests damage or problems. And - here we go again.
NASA does everything they can to shine up their process and actions to avoid even hints of trouble. They are more worried about bad press and how the public views their capabilities than they are for the short term. This story about a computer glitch on the ISS is a smokescreen to cover their asses while they try to fix whatever is wrong on the Shuttle. Hit or no hit, something is amiss.
Sooner or later... Always
The mindset-climate at NASA has always been the same and always will be the same. Hubris.
Just for the record (Score:5, Interesting)
The first piece of the space station was Zarya, the Russian control module that was launched into orbit November 20, 1998. A few weeks later, on December 4, 1998, the U.S. module Unity was launched into space. On December 7, 1998, the two modules were connected.
That makes the ISS just over 8 years in service.
How old is Atlantis?
Space Shuttle Atlantis has completed 27 flights, spent 220.40-days in space, completed 3468 orbits, and flown 89908732 miles in total, as of September 2006. Atlantis visited visited MIR in 1997!
Atlantis is 23 years old as of last April. 21 years in service. More than twice as old as the ISS.
Now, tell again - which is the real bucket of bolts? ISS or Atlantis?
Re:Just for the record (Score:5, Insightful)
The ISS has spent more time in continuous orbit and more time in space.
So I guess that means...nothing really.
Re: (Score:2)
Re: (Score:2)
Re:NASA has a problem alright, but not with the IS (Score:2)
First of all, every shuttle mission since Columbia has had a bunch of little problems reported by NASA. Remember when they did a spacewalk to pull out gap fillers? Remember freeze-frames showing foam impacts?
So your first "point", that NASA always claims a perfect launch, is simply false. But besides that, the ISS problem is WAY more serious than your supposed shuttle wing impact conspiracy. If the ISS is abandoned due to this, and they can't fix this proble
you've conspiracy on the mind timmie (Score:2)
You're saying NASA faked the photo of the hole in the wing [msn.com]??? Oh...my bad
Lord - If you're right, timmie, this could be the biggest thing since Bush had his USD$50 watch lifted by an Albanian pickpocket!!!
Look, bud - you can call me a trans-gendered duck for all I care, but 'labeling' your target is weak debating at best, so unless you've got something more, you are only mildly entertaining - good for a few laughs. Very few.
Thanks for taking a run at me Mr.
Re: (Score:3, Informative)
Re:NASA has a problem alright, but not with the IS (Score:2)
B) When they do they are talking about the launch specific details. Firing of the rockets, attitude control, good roll, and ship control, to name the main points.
So the problem here is you have no idea what they are talking about so you filled your ignorance in with wild ass theories handed to you by the monkeys living in your ass.
Re: (Score:2)
I'm not sure the English/Russian speaking engineer that I heard earlier during the first false smoke alarm due to a softwa
Correction: 4 out of 6 computers down (Score:2, Insightful)
Re: (Score:2)
Two out of six computers in a reduntant system cannot be rebooted.
From TFA:
The station's Russian segment has a network of six primary computers, three for guidance and navigation and three for command and control, any one of which can handle the duties of its counterparts, Suffredini said, adding that only two were online early Wednesday.
Big difference!
I interpret this to mean the following:
tech support in space .. (Score:4, Funny)
Tech Support in Space [nasa.gov]
did they do an upgrade recently .. (Score:2)
Filed under: Desktops
The International Space Station crew is doing some spring cleaning this week to get ready for an upcoming computer upgrade. Related and unrelated novelties include 10 times faster networking and a brand new window and camera combo which was installed last week
http://www.engadget.com/2007/03/19/iss-getting-rea dy-for-a-new-computer-system/ [engadget.com]
http://www.spacescan.org/entry/international-space -station-may-soon-get-computer-upgradations/ [spacescan.org]
Graphite failure (Score:2)
Re:Graphite failure (Score:5, Informative)
NASA uses 30-year old UNIX derivative (Score:3, Interesting)
Re: (Score:3, Informative)
VxWorks isn't a UNIX, it is a real time operating system from Wind River [windriver.com]. Its has POSIX compliance in a decent number of areas so writing a thread / task is similar to programming for UNIX, but it can be quite a different beast when it comes to actually running the software.
And now they're working again... (Score:2)
http://www.cnn.com/2007/TECH/space/06/14/space.sh
Worst case scenario (Score:2)
Update: Computers down again (Score:4, Informative)
If disconnecting the power doesn't fix them problem, then the situation is even worse... the station will have to be evacuated next Wednesday, and would no longer have attitude control. It is likely that it would tumble out of control before any new mission could be made, making it impossible to dock the the ISS and probably resulting in its eventual re-entry.
Things are not looking good.
Just curious.. (Score:2)
And hey, I'm all for NASA and having an orbital station. But a little common sense could go a long way. (Yeah, I know there were probably lots of engineers who wanted a test bed, but an administrator/politician
Re: (Score:2)
So, do you suppose that somewhere, someone as a test environment on Earth, where they can test how components will interact on ISS before we spend millions of dollars to send it up there? A little investment on the front end could save us money on the back end with issues like this, I would think.
:-/
And hey, I'm all for NASA and having an orbital station. But a little common sense could go a long way. (Yeah, I know there were probably lots of engineers who wanted a test bed, but an administrator/politician nix'd it.)
To be fair, it might be a situation that is difficult to recreate on Earth. I'm not saying that's the case, I'm just saying that's not outlandish.
Re:OS? (Score:5, Funny)
Re: (Score:2)
Re:OS? (Score:5, Funny)
Re:OS? (Score:5, Interesting)
No.
On NASA's manned space equipment you will find no software that is not controlled by NASA. These folks don't just run a few tests. They spend thousands of dollars per SLOC in testing. They actually mathematically prove their software's correctness. Perhaps the Russian agency's quality isn't quite as high, but I still doubt their (or anyone else's) systems onboard the ISS have any OS at all. Most likely they are all custom embedded systems.
I'd council against jumping to conclusions about the cause of this solely based on the Russian origin of these systems. I remember a lot of people did that with the early Ariane crash [embedded.com] based on it being written in Ada, and ended up looking pretty silly when the problem turned out to be some ported code that wasn't rewritten properly for the new platform.
Re: (Score:2, Informative)
First about the "all software is NASA controlled" assertion.. Well.. While I was watching NASA TV, I caught a glimpse of one of the astronauts obviously attempting to retrieve some e-mail from his laptop.. And then complaining over the com that he was getting a "you can only have one instance of Outlook running" - ground control advised for a laptop reboot, but the guy upstairs wasn't too keen on doing that (apparently, to him, this meant it was a server problem !) - the capcom person at that
Re:OS? (Score:5, Informative)
The personal communication laptops the astronauts have are windows machines. The machines that run both ISS and Shuttle are **not**. They are derivatives of UNIX, and, as grandparent said, have many eyes and many thousands of dollars poured into each line of code. There was a good article not too long ago in Fast Company [fastcompany.com] about the shuttle coding team.
From the article: the last three versions of the program -- each 420,000 lines long-had just one error each. The last 11 versions of this software had a total of 17 errors. Commercial programs of equivalent complexity would have 5,000 errors. That's impressive. The same care went into the ISS computers, at least from the US's side. I can't speak for Russia as I don't have that level of familiarity with them.
Last, everyone is talking about the 'russian' computers.. Well, this guy last night in the press conference did state these were actually "western style" *european* computers !
The Russian computers failed. The US computers have 'taken over' temporarily. Why? Because we have this nice little satellite network called TDRSS (Tracking and Data Relay Satellite System) which lets us relay communications with shuttle over the vast majority of the orbit. Russia does not. They can only communicate over line of sight, which is a few times each day for about 8 or so minutes.
Re: (Score:3, Informative)
First about the "all software is NASA controlled" assertion.. Well.. While I was watching NASA TV, I caught a glimpse of one of the astronauts obviously attempting to retrieve some e-mail from his laptop.. And then complaining over the com that he was getting a "you can only have one instance of Outlook running" - ground control advised for a laptop reboot, but the guy upstairs wasn't too keen on doing that (apparently, to him, this meant it was a server problem !) - the capcom person at that time then seemed to be taking the diplomatic side and answered : "lemme check" !
There's a difference between "flight control software" and "laptop." I've seen some missions where mission-specific software is running on laptops but there's nothing going on with the laptop where the safety of the shuttle is put in jeopardy when the laptop crashes. This whole same conversation came up weeks ago on a battlestar thread here. The military HAS been known to do things inexplicably stupid like run a destroyer on NT. A computer crash there disabled the ship's engines. However, this is known as
Re: (Score:2)
I blame Radio Shack.
Re:Computer Failure... (Score:4, Funny)
Not quite...
"I'm sorry comrade, In space, gyroscopes, turn you"
sigh, life is balanced again.
Re: (Score:2)
Re: (Score:2)
Re: (Score:2)
http://www.snopes.com/business/genius/spacepen.as
Re: (Score:2)