Follow Slashdot stories on Twitter

 


Forgot your password?
Close
typodupeerror
×
Space Science

Shuttle Assessment Tool was Inferior 30

Posted by michael from the you-can-do-a-lot-with-a-spreadsheet dept.
An anonymous reader writes " Shuttle report in Houston Chronicle: 'The computer program Boeing engineers used to predict that a debris-damaged Columbia could land safely wasn't much more than a simple chart of past foam damage, accident investigators said Tuesday.'"
This discussion has been archived. No new comments can be posted.

Shuttle Assessment Tool was Inferior More

Shuttle Assessment Tool was Inferior

Comments Filter:

  • The Tufte version (Score:4, Informative)

    by RobotWisdom ( 25776 ) on Wednesday April 09, 2003 @10:35AM (#5693056) Homepage
    Edward Tufte is a demigod in the world of information-design, and he made an interesting case [edwardtufte.com] recently that bad PowerPoint design in Boeing's report contributed to the misinterpretation of the analysis. Eg, the way the ppt-slide was laid out almost completely concealed the fact that the test was on a small cube of foam.
    • "The reports provided the rationale for NASA officials to curtail further research (such as photographing the Columbia with spy cameras) on the tiles during the flight. Here is a close analysis of an important slide from a Boeing report. This discussion was prepared for a chapter on the cognitive style of PowerPoint in my new book Beautiful Evidence..."

      It would be interesting to see the following headline in the NY Times:

      "PowerPoint Killed Shuttle"

  • Yeah (Score:3, Funny)

    by 0x0d0a ( 568518 ) on Wednesday April 09, 2003 @10:44AM (#5693123) Journal
    Shuttle report in Houston Chronicle: 'The computer program Boeing engineers used to predict that a debris-damaged Columbia could land safely wasn't much more than a simple chart of past foam damage, accident investigators said Tuesday.'

    In other news, the Houston-based ContractorCorp announced its new, $20 million-dollar-a-license aerospace disaster analysis software...

    Also today, President Bush vowed that "no cost would be spared" to identify that shuttle problem that "struck such a tragic blow to our nation's future"...

  • "Just a Spreadsheet" doesn't much matter (Score:5, Insightful)

    by 0x69 ( 580798 ) on Wednesday April 09, 2003 @11:39AM (#5693447) Journal
    They had only guesses as to what kind of material (foam, ice, ice-loaded foam, etc.) hit the wing. Only crude estimates as to how much hit, & where. They'd NEVER done real inspections (ultrasound, X-ray, etc.) of those carbon-carbon composite leading edges (to look for delamination, fractures, internal erosion from oxygen entering through surface pinholes, etc.) I haven't heard that they had ANY real test data from larger hits.

    In this context, it doesn't much matter whether the "program" is half a million lines of gigaflop-sucking Fortran or a Buck Rogers Secret Decoder Ring. They were (fairly contentedly) starved for meaningful input.

    GIGO.
    • In this context, it doesn't much matter whether the "program" is half a million lines of gigaflop-sucking Fortran or a Buck Rogers Secret Decoder Ring. They were (fairly contentedly) starved for meaningful input.
      You are absolutely correct in that they had no real data at the time. The problem is that they used an unsophisticated tool to perform a task for which was was never intended, and then (upon liking the results) stopped looking for more data or a better tool.
      NASA had ordered telescopic images of
      • Isn't hindsight a great thing? If they had known at the time that their analysis wasn't good enough do you really think they would have used only a spreadsheet? They much have thought that it was ok. Not to mention that even if they HAD said "it's dead" there would have been NO POSSIBLE WAY to save the crew. No matter how the numbers came out, once the foam hit the shuttle and caused the damage, it was naver comming home, and it's crew was going to suffer that fate with it. It was a tragic accident, le
        • Not to mention that even if they HAD said "it's dead" there would have been NO POSSIBLE WAY to save the crew

          Reports on released emails say that NASA engineers would seem to disagree [space.com]. Maybe you're better at spaceflight than they are...
          • Notice the one thing missing from that artice, a clear, workable plan. Even if they HAD decided to rescue them:
            1) There were no other shuttles ready to launch, and none could have been preped and launched in time
            2) The only other space agency in the world that had put a man in space and returned him safly is the russians. They COULD have had a launch ready in time, however they can carry only 3 people back. They would have needed at least 2, probably 3 to make it work. That they couldn't have done. N
            • We already know your 1-4 were not options, so did NASA, which is why they weren't discussing them. Also, the shuttle didn't have enough deltaV to reach the ISS so that wasn't an option anyway.

              The point is that while the media and other commentators were saying "there was nothing they could have done", the NASA guys had discussed things they could have done, had the damage been thought serious enough.

              The mindset that got Apollo 13 back doesn't seem to have disappeared, as some have suggested, it was just n
              • They can't change it too much, because they don't want to bounce off the atmosphere. You go too steep you burn up, to shallow and you bounce off. :) Also the shuttle can't "fly" it's like a brick with really stubby wings, even assuming that you could come down in such a way that heat wasn't the problem, you still need to be able to land.

                I do see your point. There may have been SOME options, but with the clock ticking, and very few materials to work with, it probably wouldn't have saved them. :(
                • [...]you still need to be able to land.

                  Not so - if it is decided the shuttle won't make landing then you can go for bail-out instead. This was discussed in one of the released NASA emails in the context of landing-gear door burn-through leading to gear failure, and needing to decide on belly-landing vs. bail-out.

                  The re-entry trajectories are calculated with the target of the glide path to landing. If you know (or think you know) that those trajectories will result in loss of a wing, then you aren't landi

  • What choice did they have? (Score:3, Insightful)

    by GypC ( 7592 ) on Wednesday April 09, 2003 @03:15PM (#5694776) Homepage Journal

    Seems like a moot point to me. From what I understand they had no alternative but to attempt a landing. Maybe if they had somehow scraped together another shuttle launch right after the first one they could have all ridden home in the second one? Or maybe fixed the damage to the first one? I doubt it.


    • Yeah. Just like the way they gave up on Apollo 13 after it exploded in space. Oh wait, they got it back safely.

      Don't for a second think that NASA engineers take their job lightly. The ones I've known are very committed. Look at the number of emails and memos that were passed around about the foam hitting the wing.

      Hindsight says they should have taken the warnings more seriously, but they get warnings on every flight. If they grounded the shuttle until it was as safe as a passenger plane, we just wou

      • Re:What choice did they have? (Score:5, Insightful)

        by sl3xd ( 111641 ) on Wednesday April 09, 2003 @07:22PM (#5697314) Journal
        Just like the way they gave up on Apollo 13 after it exploded in space. Oh wait, they got it back safely.

        That's a pretty cynical attitude. It also greatly underscores the fact that we nearly lost Apollo 13's crew, for several reasons. In fact, Houston had every reason to expect that Odyssey would break up during re-entry as well; it is a miracle that it didn't. In fact, NASA did as much giving up then as they did with the shuttle. In both cases, it was unrealistic to 'mount a rescue'; spaceflight isn't that commonplace, which is a fact many seem to forget. Launching payloads into orbit isn't a daily or weekly thing; manned spaceflight is even less common; maybe a monthly thing, if that. NASA basically had the choice: Re-enter now, and take our chances, or re-enter later, and take the same chance. With Apollo 13, NASA gave up the moment they told the crew of the Odyssey to do the final retro-burn to re-enter Earth's atmosphere. They had no choice -- if they didn't bring Odyssey down, the crew would die of asphyxiation, or the extreme temperatures of space.

        The same choice presented itself with Colombia; a 'rescue' is only slightly more plausable now than it was during Apollo; which is to say, one step above utterly implausable.

        The fact of the matter is that in both cases, the craft had to come down if the crew were to have any chance of survival.

        For Columbia, docking with Station Alpha was impossible-- wrong orbit, not enough fuel. There was an oxygen reserve for a few days to a week, depending on who you talk to. It's a moot point, since those few days are insufficient to launch any kind of rescue; it takes weeks to get a scheduled shuttle launch going; hell, it takes weeks to get any rocket ready for launch. And it's not like Boeing, Lockheed, Arianne, or Russia have a spare launch vehicle laying around prepped and ready to go for a rainy day; they certainly don't have two, which is the number of Soyuz craft it would take to return the crew back to Earth. It costs serious $$$ to keep a rocket in a 'prepped and ready to go' state; enough so to make it impractical.

        There have been many, many cases where the heat shields of a spacecraft were damaged, or uncertain: Friendship 7, Apollo 13, a couple of the Gemini missions, and at least 20% of the shuttle flights. (There were entirely missing tiles when Columbia made its maiden voyage, and this has repeated itself several times on every one of the shuttles that have flown.). All of them turned out well.

        So, there was a choice: Die slowly of asphyxiation and/or dehydration, hoping that the (extremely long) odds of survival until a 'rescue' could be mounted would favor you, or take the much more comfortable odds that you will die during re-entry, when death would take a few microseconds?

        No matter what option was taken, the crew would still have to go through re-entry; the only difference would be the craft it happens with.
        • ...For Columbia, docking with Station Alpha was impossible-- wrong orbit, not enough fuel.

          How about they couldn't dock because they didn't have the correct docking mechanism.

          If they had enough fuel to get to the Space Station(SS) about the only choice they had to get the crew onto it was to use the 2 space suits to ferry folks from Columbia to the SS. I'm assuming that there is an airlock on the SS that can be used for EVA's.

          This might have given folks enough time on the ground to get 4 (need a traine
        • Or, if they had known about the situation earlier (inspection via telescope) they could have come up with something.

          It was a 10 day flight. Knowing about the problem a couple of days into it would have given MUCH better options, rather than just winging it on the reentry.

          Stretch the rations/air for a few more days. "hey...there's a Progress resupply going to the ISS next week. Can we divert that and give them another week or so?"

          Who knows? But I do know that having 2 weeks to figure something out is p
          • Or, if they had known about the situation earlier (inspection via telescope) they could have come up with something.

            How much do you not know about optics? That argument is a fairly bad joke, in my view. There aren't many spy-sattelites sitting on the ground, pointing upward -- espescially ones on a motor-driven mount that is smooth enough to provide a clear picture, is there? And don't give me anything about astronomical telescopes, which are the only other thing that can see clearly enough -- there i
            • And don't give me anything about astronomical telescopes, which are the only other thing that can see clearly enough -- there is a *big* difference in the slew speed between keeping a star targeted, and keeping an orbital spacecraft targeted.

              But they *did* take an image while it was up there. It did not show the area of concern because the shuttle was facing the wrong direction at the time, but it looked clear enough to be able to show a dark hole about a foot or so in diameter to me, perhaps even 6 inc
            • Or, if they had known about the situation earlier (inspection via telescope) they could have come up with something.

              How much do you not know about optics? That argument is a fairly bad joke, in my view.

              Not to dredge up an old post, but apparently someone on the CAIB [www.caib.us] thinks we can and should do it.

              "Recommendation Two: Prior to return to flight, NASA should modify its Memorandum of Agreement with National Imagery and Mapping Agency (NIMA) to make on-orbit imaging for each Shuttle flight a standard requ
    • Seems like a moot point to me. From what I understand they had no alternative but to attempt a landing.

      That is not quite true. There are some suggestions that later popped up that could have reduced the chance of outright failure.

      One was to turn the damaged section away from the sun for a few hours to cool it down a bit.

      Second, they could eject some non-essential items to make the shuttle lighter, reducing re-entry drag-time.

  • Boeing's Analysis (Score:3, Informative)

    by ASCIIMan ( 47627 ) on Wednesday April 09, 2003 @03:23PM (#5694841)
    For anyone that's interested in the actual Boeing presentation materials, NASA put copies up on their accident investigation website about a month and a half ago.

    Oh, and here are some previous TPS Reports [nasa.gov] thrown in for good measure.

  • Feynman (Score:5, Informative)

    by Henry V .009 ( 518000 ) on Wednesday April 09, 2003 @03:37PM (#5694979) Journal
    When I first heard of the foam analysis, it immediately reminded me of something that Feynman wrote in his Challenger report. This story seems to confirm the connection. I've typed out an excerpt from Feynman's report. It's worth reading. Feynman's brutalization of one of NASA's mathematical safety models in the third paragraph is the really relevant part.

    The phenomenon of accepting for flight seals that had shown erosion and blow-by in previous flights is very clear. The Challenger flight is an excellent example. There are several references to flights that had gone before. The acceptance and success of these flights is taken as evidence of safety. But erosion and blow-by are not what the design expected. They are warnings that something is wrong. The equipment is not operating as expected, and therefore there is a danger that is can operate with even wider deviations in this unexpected and not thoroughly understood way. The fact that this danger did not lead to a catastrophe before is no guarantee that it will not the next time, unless it is completely understood. When playing Russian roulette the fact that the first shot got off safely is little comfort for the next. The origin and consequences of the erosion and blow-by were not understood. They did not occur equally on all flights and all joints; sometimes more, and sometimes less. Why not sometime, when whatever conditions determined it were right, still more, leading to catastrophe?

    In spite of these variations from case to case, officials behaved as if they understood it, giving apparently logical arguments to each other often depending on the "success" of previous flights. For example, in determining if flight 51-L was safe to fly in the face of ring erosion in flight 51-C, it was noted that the erosion depth was only one-third of the radius. It had been noted in an experiment cutting the ring that cutting it as deep as one radius was necessary before the ring failed. Instead of being very concerned that variations of poorly understood conditions might reasonably create a deeper erosion this time, it was asserted, there was "a safety factor of three." This is a strange use of the engineer's term "safety factor." If a bridge is built to withstand a certain load without the beams permanently deforming, cracking, or breaking, it may be designed for the materials used to actually stand up under three times the load. This "safety factor" is to allow for uncertain excesses of load, or unknown extra loads, or weaknesses in the material that might have unexpected flaws, etc. If now the expected load comes on to the new bridge and a crack appears in a beam, this is a failure of the design. There was no safety factor at all; even though the bridge did not actually collapse because the crack only went one-third of the way through the beam. The O-rings of the Solid Rocket Boosters were not designed to erode. Erosion was a clue that something was wrong. Erosion was not something from which safety can be inferred.

    There was no way, without full understanding, that one could have confidence that conditions the next time might not produce erosion three times more severe than the time before. Nevertheless, officials fooled themselves into thinking they had such understanding and confidence, in spite of the peculiar variations from case to case. A mathematical model was made to calculate erosion. This was a model based not on physical understanding but on empirical curve fitting. To be more detailed, it was supposed a stream of hot gas impinged on the O-ring material, and the heat was determined at the point of stagnation (so far, with reasonable physical, thermodynamic laws). But to determine how much rubber eroded it was assumed this depended only on this heat by a formula suggested by data on a similar material. A logarithmic plot suggeste3d a straight line, so it was supposed that the erosion varied as the .58 power of the heat, the .58 being determined by a nearest fit.

  • First, I am an engineer. Alot of you probably are too. What I'm going to say will probably be modded flamebait, etc, but I'm fighting my own battles at work in regards to problems that no one else saw... or care
    Alot of the analysis has been attacking the engineers for not asking enough questions. Thats fine and dandy in a 100% hindsight problem- we have a failed shuttle- lets' find out why. Alot of the reviews have been talking about data presentation- thats good too- I went to school for engineering, not marketing, and therefor don't know what a marketer does as to how to present information without getting bogged down in details.
    But when it comes straight down to it, it's money, pure and simple. Do you think CAT scans of tiles are inexpenisve? Probably a couple $k each. Do this for every tile. Want to understand turbulence completely (and people that say you can model a chaotic system- just watch the weather channel to know how EASY that is)- that costs money and time. Quite a bit of both, too.
    So now you've got budget concerns on projects that aren't funded and you can only skunk work it too much (note- skunk work is done on the side, unpaid overtime/salary, and 'hiding' the cost of equipment time/usage under a variety of things. It's amazing what you can do sometimes).
    Now and then you get lucky and management comes around... funds your project, everyone gets paid with a little back in the jar for the next skunk project... then again, what does management usually know? zip. Just those bottom line numbers
    Now obviously there was a bit of scaleup issue. I'm not comfortable with a 5x scaleup on some jobs, much less a 640x prediction- thats me personally. And the analysis that reads safety as a failure, instead of safety as a problem is dead on (1/3 the O-ring). But don't go too hard on the engineers- many comments are headed that way. Just remember under-funding answers the important questions, and may lop a bunch of details under assumptions... and every now and then you get bit in the ass... hard.

Slashdot Top Deals

If a thing's worth doing, it is worth doing badly. -- G.K. Chesterton

Close