NASA Declares Starliner Mishap a 'High Visibility Close Call' (arstechnica.com) 71
After pondering the totality of issues that arose during a December test flight of Boeing's Starliner spacecraft this week, NASA chief of human spaceflight Doug Loverro said Friday that he decided to escalate the incident. From a report: So he designated Starliner's uncrewed mission, during which the spacecraft flew a shortened profile and did not attempt to dock with the International Space Station, as a "high visibility close call." This relatively rare designation for NASA's human spaceflight program falls short of "loss of mission" but is nonetheless fairly rare. It was last used by NASA after a spacewalk in 2013 when water began to dangerously pool in the helmet of astronaut Luca Parmitano. Asked to explain during a conference call with reporters why he did this, Loverro said, "We could have lost a spacecraft twice during this mission."
In this, Loverro referred to two software errors that occurred during the two-day flight. The first problem occurred when Starliner captured the wrong "mission elapsed time" from its Atlas V launch vehicle -- it was supposed to pick up this time during the terminal phase of the countdown, but instead it grabbed data 11 hours off of the correct time. This led to a delayed push to reach orbit. The second error, caught and fixed just a few hours before the vehicle returned to Earth through the atmosphere, was due to a software mapping error that would have caused thrusters on Starliner's service module to fire in the wrong manner. NASA and Boeing officials held Friday's teleconference to announce the conclusion of a report from an Independent Review Team established after December's flight. These reviewers made 60 recommendations to NASA and Boeing for corrective actions that ranged from fixing these software issues to ferreting out others that may still exist in the spacecraft's flight code.
In this, Loverro referred to two software errors that occurred during the two-day flight. The first problem occurred when Starliner captured the wrong "mission elapsed time" from its Atlas V launch vehicle -- it was supposed to pick up this time during the terminal phase of the countdown, but instead it grabbed data 11 hours off of the correct time. This led to a delayed push to reach orbit. The second error, caught and fixed just a few hours before the vehicle returned to Earth through the atmosphere, was due to a software mapping error that would have caused thrusters on Starliner's service module to fire in the wrong manner. NASA and Boeing officials held Friday's teleconference to announce the conclusion of a report from an Independent Review Team established after December's flight. These reviewers made 60 recommendations to NASA and Boeing for corrective actions that ranged from fixing these software issues to ferreting out others that may still exist in the spacecraft's flight code.
Re: Euphemism (Score:5, Insightful)
More like "Boeing royally screwed up but I'm not going to be saying anything because I dont want to jeopardize that sweet job I'm going to get from them after I retire from NASA."
Re: (Score:2)
I'm not sure where you get that from. It sounds like this about the most serious rating he could have given, short of "loss of mission" which it clearly wasn't.
Boeing is likely to be unhappy.
Re: Euphemism (Score:3)
The fact that NASA also basically said "this was a major failure, but no need to redo the test".
Re: (Score:2)
I guess Loverro forgot to read the press release you did. Better add that oversight to the list of things the NASA safety office will now investigate.
Re: (Score:3)
Nobody at NASA has ever said that there's no need to redo the test. They've consistently said that they can't yet say if there's a need to redo the test. NASA processes move at a glacial pace, that decision is months away. I'm 99% sure the decision they make will be that Starliner has to repeat the test.
Re: Euphemism (Score:3)
Given the recent experience with 787 Max 8, Boeing should be doing everything to make the fixes and even offer to do another test flight, without being asked.
It will be expensive, but Boeing needs to show that it cares about safety and that it also understands the culture of software.
Re: (Score:2)
Maybe Elon could offer em a cheap launch :)
Re: (Score:2)
Oh snap!
Bravo sir. You made my day already.
I golf clap in your general direction
Re: (Score:2)
You win today's "This is /., we don't RTFA!" award.
Blame NASA Budge Cuts (Score:5, Funny)
Boeing royally screwed up
No, clearly the problem was that due to NASA budget cuts they could not afford Boeing's "optional" safety features.
Re: (Score:2)
Boeing royally screwed up
No, clearly the problem was that due to NASA budge
They could not even afford a "t".
t cuts they could not afford Boeing's "optional" safety features.
Safety is optional in all of Boeing's products but they throw in the software bugs at no additional charge.
Re: (Score:2)
Re: (Score:3)
Not really. A flight where you recover the vehicle and contents intact is considerably more encouraging than the average rocketry failure.
In the glory days of spaceflight, these particularly faults could not even have happened because the systems in question would be controlled by an army of nerds sitting in front of consoles at Mission Control. There was very little software to fail. Although some of the custom computers they did to control things like Saturn V stages were quite interesting [youtu.be], there's no
Re: (Score:2)
Re:Euphemism (Score:4, Interesting)
That would be true if it's the rocket you're testing. The rocket here was well-proven and not in need of testing, the test was to dock the capsule with the ISS. It failed to reach the proper orbit and so failed to dock, as well as almost failing in worse ways.
Shuttle Columbia failed to land, but have any capsules ever failed to land from orbit?
Re: (Score:1)
Multiple times, actually, both from orbit and from deep space. The first one was probably Soyuz-1 in 1967, later notable ones include the Genesis sample-return mission that smashed into a desert.
Re: (Score:2)
Re: (Score:2)
Soyuz 1 and Soyuz 11 are fair examples of capsule failure, but Genesis was a completely non-comparable sort of craft -- tiny, experimental and not required to be safe since it had no relation to any human spaceflight plan. Genesis is more equivalent to failed Mars landers.
Re: (Score:2)
Shuttle Columbia failed to land, but have any capsules ever failed to land from orbit?
Capsules have not failed to land, but some capsules have landed less intact and with fatalities. Soyuz 11 landed successfully but with all three cosmonauts dead https://en.wikipedia.org/wiki/Soyuz_11 [wikipedia.org], and before that Soyuz 1 resulted in the death of Vladimir Komarov, its only cosmonaut, when it hit the ground at very high speed https://en.wikipedia.org/wiki/Soyuz_1 [wikipedia.org]. However, both of these incidents were in the very early era of space flight, and it is fair to say that no similar incident with a capsule syst
Is this Boeing speak for "Close Enough"? (Score:2)
Wasn't this the Boeing mission that they considered "Close Enough" to count as successful and bypass actually doing it on their way to using the craft for a manned mission to the station?
Re: (Score:1)
Wasn't this the Boeing mission that they considered "Close Enough" to count as successful and bypass actually doing it on their way to using the craft for a manned mission to the station?
The initial response by NASA said they would review whether another uncrewed test would be required as part of their contract (NASA is nothing if not politic, especially when contractual issues are involved), but the more they looked, the more they found (both at Boeing, and NASA itself) that suggested lack of discipline at both Boeing, and NASA itself.
The reality is that NASA has long ago moved to being extremely risk adverse. Ask the astronauts, and they mostly understand that they might die at any tim
In other words.... (Score:3, Informative)
Re: (Score:3)
Re:In other words.... (Score:4, Insightful)
NASA seems to favor Boeing over SpaceX for whatever reason.
No real surprise there. Boeing has always been an import part of the entire NASA story (they were there as contractors from the beginning of the space race), and ULA (owned by Boeing and Lockheed Martin) has strong ties to NASA and the DOD to this day.
Re: (Score:2)
And Boeing "owns" the souls of too many powerful politicians who can force NASA to their (Boeings) will. Whether Boeing will think this worth the price is still to be seen.
Re: (Score:2)
Re: (Score:1)
Re: (Score:2)
The SpaceX Crew Dragon blew-up because a one-way fuel valve failed to behave as expected. SpaceX learnt new Physics from the investigation. In fact, engineers learn most from failures and is why SpaceX likes to test to destruction.
Re: (Score:2)
Congress favors Boeing, and Congresscritteres hold the purse strings.
Another article here (Score:5, Informative)
Friendly reminder (Score:3, Informative)
Just a friendly reminder to Boeing, This [giphy.com] is not a flight sequencer.
Re: (Score:2)
Yikes, change of plans (Score:3)
"...when water began to dangerously pool in the helmet of astronaut Luca Parmitano."
Holy shit, to drown in outer space. That's a new horror I never imagined.
"...but instead it grabbed data 11 hours off of the correct time. This led to a delayed push to reach orbit."
Okay, yeah that sounds bad. Real bad. Ultra bad. Mega bad.
"...a software mapping error that would have caused thrusters on Starliner's service module to fire in the wrong manner."
God damn, by the time you had any inkling of this it would have been, "Well, it was nice knowin' you guys..."
So screw it- I'm canceling my plans to become an astronaut and I'm just gonna enjoy another cold one in my easy chair instead....
Re: (Score:2)
I'm sure drowning in a space suit has resulted in death in at least one space horror film but I can't for the life of me remember which... Was it "Life"?
Re: (Score:2)
Actually, from what I understand, it didn't "grabbed data 11 hours off of the correct time", it actually used its system uptime clock instead of the Mission Elapsed Time clock. The capsule had been powered up 11 hours before launch. The problem was apparently that it never "grabbed" the time at all.
And then, in true Boeing style, they used the clock time as the sole means to decide that it was already on orbit and thus course corrections should use the RCS thrusters instead of the main engines. And because
Re: (Score:2)
So screw it- I'm canceling my plans to become an astronaut and I'm just gonna enjoy another cold one in my easy chair instead....
I wonder if Virgin Galactic will offer beer on their suborbital flights?
I wonder what the nitrogen bubbles in Guinness will do in space?
I wonder when this antibiotic I'm on will let me quit wondering about weird stuff.
So many questions. (Score:2)
...Starliner captured the wrong "mission elapsed time" from its Atlas V launch vehicle -- it was supposed to pick up this time during the terminal phase of the countdown, but instead it grabbed data 11 hours off of the correct time.
mr spock please explain
First, why would the orbital vehicle rely on a clock located in the launch section? It should have its own clock carefully synced before T-minus-zero.
If they did have to copy the Atlas clock to the Starliner clock, why wouldn't the huge difference be noted then and an automatic remedial function kick in then?
I assume the needed burn didn't take place 11 hours off schedule. So they did it by manual command?
Re: (Score:2)
Well, it was syncing (theoretically) its clock to launch time by getting the launch time from the launch vehicle. That way it'll not do something stupid like fire its engine to separate from the launcher while the launcher is still boosting (after, say, a three minute delay in launch due to whatever - a flock of geese, an idiot in a Cesna...).
Why it was 11
Re: (Score:1)
Different time zones programmed.
Re: (Score:1)
Different time zones programmed.
Na, the guys in the control rooms at Nasa working for Boeing just didn't get the correct way to set it up a networked time sync directly from the coders who were the original source of the sync problems. [microsoft.com]
Re: (Score:2)
Really? These days almost all software libraries use a Date function/object that models time as a count (usually ms) from the 1970 epoch -- Zulu time. The time zone factor is only used when translating the date/time to human readable format. They probably don't use node.js or JSEE in their embedded software but I can't imagine why whatever they use would do it any different.
So for the time zone configuration to affect the transfer, it means that they converted it to human-readable (using TZ #1) and
Re: (Score:2)
These days almost all software libraries use a Date function/object that models time as a count (usually ms) from the 1970 epoch -- Zulu time.
"these days"? Engineers have been doing that since, well, just about 1970. Which is why it was silly to have worried about planes dropping out of the sky on 1/1/2000. Engineering programmers are much smarter than financial programmers.
However, I will not be booking any flights on 19th January 2038.
Re: (Score:2)
OMG, just realised we are closer to 2038 than to Y2K. So
a) where is my flying car?
and
b) get off my lawn!
Re: (Score:2)
"these days"? Engineers have been doing that since, well, just about 1970. Which is why it was silly to have worried about planes dropping out of the sky on 1/1/2000. Engineering programmers are much smarter than financial programmers.
Keep in mind though, that more and more engineering decisions are being made by financial people.
Re: (Score:2)
Also Internet Network Time Protocol (NTP) rolls over in 2036 which will prevent systems getting the correct UNIX time if the system starts-up with the 1970 Epoch time and date.
Re: (Score:2)
Why it was 11 hours off is the tough one.
Easy. It was the uptime clock, from when the capsule had been powered up. It's basically a "blinking 12:00" problem, they never set the clock.
Re: (Score:2)
Why it was 11 hours off is the tough one.
Someone up above said it was using the uptime of the computer, and failed to grab the mission time from the rocket. Which seems just as stupid, if not stupider. If your mission success depends on the capsule having the right mission time in its systems, is not one of the pre-launch checks to see that it does? That's not even a "hindsight is 20-20" sort of thing. That's on par with making sure it has fuel and weighs the expected amount.
60 corrective actions (Score:1)
So will this work like a Boeing/Air Force deal? NASA will have to pay Boeing to make the corrections?
"Sorry NASA all those changes will have to be paid for by you."
"Why?"
"Pilot error!"
"There was no pilot."
"And that is why you have to pay for this. Clearly you're in error not putting one of your pilots in that computerised death trap!"
"OK. Here is my credit card number."
Re: (Score:2)
It's a "fixed-price" contract, which means Boeing bid for and got a specific amount of money to build the system and deliver the 12 or so missions. SpaceX wrote off their exploded capsule on their own dime, and they only charged NASA about 60% of Boeing's bid.
But I'm sure that someone will still find a way to give Boeing money for a re-launch despite it being a fixed-price contract.
Anyone get the feeling (Score:2)
as a "high visibility close call" (Score:4, Funny)
"We could have lost a spacecraft twice during this mission." The first problem [was] 11 hours off of the correct time. The second error, caught and fixed just a few hours before
Hey, it's all good. Nothing broke, and the OTA updates fixed things before anything bad happened. (We can do OTAs just like Tesla does. See? We're just like them. Really!) But, uh, everything's perfectly all right now. We're fine. We're all fine here, now, thank you. How are you?
Now: don't you want to sign for this ongoing maintenance contract. That's a nice spacecraft you've got there; shame if something were to happen to it.
Re: (Score:2)
Can you do Over The Air updates when there's no air?
Re: (Score:2)
Can you do Over The Air updates when there's no air?
Once in orbit, it is "over the air". Over most of it any way. Still be the odd trace of it here and there.
Re: (Score:2)
>Hey, it's all good. Nothing broke, and the OTA updates fixed things before anything bad happened.
and in fairness to Boeing, better than 9 out of 10 737 Max flights landed successfully . . .
hawk
Re: (Score:2)
Han, is that you?
Transfer funds to SpaceX (Score:2)
Make Boeing refund all the money and deposit it into Elon Muskâ(TM)s bank account. The fire their executive team, disband the Starliner project and hire everyone with an IQ above 75 (if thereâ(TM)s any) from the Starliner project into the SpaceX team.
Re: (Score:2)
Don't believe that if SpaceX eliminated all competition from the market, that it would not, over time, become infected with the same complacency and ineptitude. There is simply too much easy money to be made from graft vs innovation when you have a monopoly position, and the forces that drive this are fundamental human behaviours.
Boeing is capable of fixing these problems. They have gotten a craft into orbit which is no minor feat, so it's not like they are completely useless. Perhaps they do need a good cl
Re: (Score:2)
SpaceX actually was worried about issues and thus did further extensive testing on the capsule above and beyond what NASA required, and, "lo and behold," a system that NASA engineers had already signed off on three times exploded under those circumstances.
The exact
Boeing hires its programmers on Fiverr (Score:1)
Boeing Software??? (Score:2)
737 Max - software problem, Starliner - software problem. Boeing seems to have problem with software.
In short, do another test flight (Score:2)
I think Boeing needs to do another test flight of CST-100 at minimum to "work out the bugs."
Quality is not Job 1 (Score:1)
"fixing these software issues to ferreting out others that may still exist in the spacecraft's flight code"
I think they meant to say do still exist.
Boeing's priorities are:
1) Schedule
2) Budget
3) Quality