Behind the Scenes At NASA's Mission Control Center 38
willith writes "I was recently given the opportunity to spend several hours on the floor of Historic Mission Operations Control Room #2, at the Johnson Space Center in Houston. MOCR2 was used to control almost manned Gemini and Apollo mission, including Apollo 11 & 13. More, my tour guide was none other than famous Apollo mission controller Sy Liebergot, one of the fellows behind the solution that saved Apollo 13. I go in-depth on the role of the flight controller during Apollo, and focus on how and why Mission Control functioned, and I spend a lot of time talking about the consoles and how they worked. The feature includes a ton of anecdotes and stories from Mr. Liebergot about mission control in general, and about what he did during Apollo 12 & 13 specifically. I also put together a supplemental report that goes through each and every station and describes their Apollo-era layout."
My supervisor loves showing off te consoles (Score:5, Interesting)
He designed the P-Tube controller boards, any time we're involved with tourist he shows those boards off.
0.005$ is not enough (Score:5, Insightful)
Big daring projects like the Apollo program and Mars Curiosity, is the reason why I love NASA so much. It's one of the few Federal programs I would actually like to see expanded. They do so much with so little. I hope whoever wins this next election finds a way to expand their budget.
“We tend to hear much more about the splendors returned than the ships that brought them or the shipwrights. It has always been that way. Even those history books enamored of the voyages of Christopher Columbus do not tell much about the builders of the Nina the Pinta and the Santa Maria or about the principle of the caravel. These spacecraft their designers builders navigators and controllers are examples of what science and engineering set free for well-defined peaceful purposes can accomplish. Those scientists and engineers should be role models for an America seeking excellence and international competitiveness. They should be on our stamps.” Carl Sagan,
Re:0.005$ is not enough (Score:5, Insightful)
It's one of the few Federal programs I would actually like to see expanded.
They would have more to work with if they could do away with the bloated govt contractor process.
Re: (Score:3)
Re: (Score:1)
They would have more to work with if they could do away with the bloated govt contractor process.
Total cost of the MSL project is about US$2.5 billion
Total cost of Iraq War is about. $4 trillion
Source - Wikipedia
Only $2.5B to flawlessly land a 1 ton nuclear powered rover on the surface of Mars. Versus $4T? NASA can take my tax dollars any day. And just imagine what NASA could have done with $4T? Manned Mars mission? Thorium reactors to process the plutonium needed. Fusion reactor to produce the hydrogen fuel for the return trip. More kids interested in science and engineering. A better economy. Not t
Re: (Score:2)
Re:0.005$ is not enough (Score:4, Insightful)
Only $2.5B to flawlessly land a 1 ton nuclear powered rover on the surface of Mars.
I see your logic. Point out really bad spending in order to make something that costs less look justified. The problem is that it doesn't.
"Only $2.5B" is the entire income of 55,000 Americans at the median. The space shuttle, including first launch and re-entry, cost quite a bit less.
1 ton is 32000 ounces. So the MSL has so far cost $80,000 per ounce.
Dont even think about bringing up the fuel, because the space shuttle was 2000 times as massive and could be put into polar orbit for about 20% of the cost of the MSL.
No sir, while it's nice to do science stuff, the cost was not justified. This was NASA shoveling money at private corporations, enabled by runaway government deficits and a lack of any semblance of responsible oversight. Did you miss the article last week about the set of weather satellites that have cost $12.5B so far, and arent even launch-able for several more years?
NASA's primary mission is to shovel money at corporations. The science stuff is an excuse to do so, and you can't justify the gross inefficiency with logical fallacies about the Iraq war. It's nice to do science stuff, but don't shut your brain off just because science is being done.
Re:0.005$ is not enough (Score:4, Insightful)
It's one of the few Federal programs I would actually like to see expanded.
They would have more to work with if they could do away with the bloated govt contractor process.
I know we pay them public money, and they can be bought off for the purposes of industrial espionage or for just plain espionage by corporations or foreign powers, but it's still disrespectful to call them "contractors" instead of "Senators".
Re: (Score:3)
It's one of the few Federal programs I would actually like to see expanded.
They would have more to work with if they could do away with the bloated govt contractor process.
They could do away with the bloated govt contractor process if they weren't bound by congressional authorization acts.
There's a reason we like to refer to SLS as the Senate Launch System, and it's not because that's what we'd like to use it for. Sadly, it doesn't look like our politicians are going to pull their heads out about matters of general interest, so I see no hope for them viewing an (unfortunately) niche interest organization like NASA as anything other than a jobs program any time soon.
Kind of s
Re: (Score:2)
Sorry, "ARE why". I didn't get any sleep last night and forgot Slashdot still lacks an edit button.
Re: (Score:1)
They do so much with so little?
My neighbor has been hired into Goddard as a contractor twice now to get two different programs back on track because they were over budget and behind schedule.
There is A LOT of dead wood at NASA. For example, he took instruments to the JHU Applied Physics Lab for shake and vibe testing because APL could do the testing in a day, while the NASA union guys would take a week for the same job.
Anybody know off the top of their heads.. (Score:5, Interesting)
Re: (Score:3)
MOCR 2 was used for every manned Gemini flight except for Gemini 3, and every manned Apollo flight except Apollo 7. MOCR 1 was used for ASTP and all Skylab flights.
Re: (Score:2)
Pretty cool history there... (Score:1)
I got to tour the same historic mission control room myself a few years ago. Didn't get to meet any of the original controllers then though, just had a regular tour guide, but if all goes as planned, I'll be dining with Paul "IronFlight" Dye, shuttle flight director, this Saturday evening. I met Paul this summer at Oshkosh. We both fly homebuilt Vans RV airplanes.
I missed out on getting to meet Gene Krantz at a fly-in in Hondo, TX a few years ago.at the final SWRFI in 2007.
Thank you so much! (Score:5, Interesting)
This is wonderful information -- I'm so glad to finally know it all. Thank you for the thorough documentation! I grew up at 1927 Richvale Lane, just a short bike ride from building 30. We were the first 25 houses there when it was all pastures, wild animals and bayous, and the Manned Spacecraft Center was brand new. My dad worked on Mercury, Gemini, Apollo, and then shuttle. We had dinners with the guys who worked there, as well as with the astronauts and their families. Of course, I was just a kid and thought is was all cool but no big deal. In retrospect, I can now see how magical it was. Everyone in the entire community was working together for something so optimistic and positive during a pretty lousy time otherwise, with assassinations, the war, and violent protests and riots.
But we were immune to it because everyone was 100% focused on getting a man to the moon and returning him safely to earth before the end of the decade. I guess that's why I like startups so much -- it's the same focus on a single objective with everyone pulling together. We don't change the world as the 100,000 engineers did on the 60s, but it's all we have left these days.
Re: (Score:2)
Nerd crack (Score:2)
Thanks for posting this!
thanks! (Score:4, Informative)
I always thought one of the most amazing things about rocket engines like the shuttles main engine was the fact that liquid o2 had to first run through holes in the nozzle to warm it up while cooling the nozzle so it didn't melt. It's such an elegant solution that is both simple and incredibly complex at the same time.
Krantz's book, "Failure is not an Option" (Score:5, Interesting)
Krantz's book, "Failure is not an Option", covers much of this material, with more information about the people at those consoles and what they did.
Those Philco-Ford console systems showed up in a number of other places, including NORAD HQ in Cheyenne Mountain and the USAF Satellite Control Center in Sunnyvale. Those screens are TV screens on a cable TV system, with a TV tuner in the console. All video generation is elsewhere. Anyone on the system could tune in anyone's screen. Military command and control centers are often set up with that capability even today. It makes coordinated teamwork possible without people having to physically hang around the console where the action is.
Re: (Score:3)
"Failure is not an Option" is a good book, but Christopher C. Kraft's book "Flight" covers many of the same events better. I closely followed the Mercury, Gemini, and Apollo missions at the time they occurred and find Kraft's discussion of the engineering problems and solutions clearer, and he is not afraid to be critical of men (like John Glenn) when he believes they were wrong.
Kraft also originated the concept of the Mission Control Center.
Must-read link from the ars article (Score:2)
Apollo 13, We Have a Solution [ieee.org]
"Almost Manned"??? (Score:3)
Re: (Score:2)
Highlights from TFA, and Apollo 13 details (Score:5, Informative)
According to TFA, the control room featured is mostly Shuttle era equipment, with SOME Apollo era consoles and equipment.
Interesting things I learned from the article:
- Bare information from the mainframe (IBM System/360) was combined with an automated slide overlay to make it more readable with column headings and threshold levels etc.
- Each person manning a console had a small team of people in another room helping them and communicating with them.
- These people were the real deal, and were hand picked for their positions. People who couldn't deal with the weight of the position were washed out within a year before they ever made it to a live mission.
- From TFA " Sy recommends the IEEE's three-part article, "Houston, We Have a Solution" as the most complete and accurate retelling of the entire Apollo 13 explosion and its aftermath."
That is HERE: http://spectrum.ieee.org/aerospace/space-flight/apollo-13-we-have-a-solution [ieee.org] and it is a FANTASTIC read. The lessons learned from Apollo 13 were fantastic, and upgrades were applied before Apollo 14. One major one was that if something went out of spec, a light would come one advising that there was an anomaly. But, if it went away, so did the light. So an intermittent problem had to be caught red handed. By Apollo 14, they'd changed it so that the light *stayed* on until dismissed. Seems obvious now, but back then it wasn't. These people were true pioneers.
Re:Highlights from TFA, and Apollo 13 details (Score:5, Informative)
The early shuttle era equipment IS Apollo era equipment.
Every console had to be be more or less rewired between every Apollo mission to accommodate the new/different equipment on the back end and to meet the unique objectives of every mission. MOCR1 and MOCR2 were wired identically, the idea was if you smoked a console or lost a whole room the controller(s) could move to the other control room and continue working.
During the shuttle era the consoles were no longer completely rewired each mission as they more or less standardized on equipment. Eventually all of the hard-wired consoles were replaced with PC's running UNIX (they used Alpha's for a long time, now Linux - Red Hat unfortunately) and of course all the equipment in the server rooms interface with those. The serial networks are now being slowly replaced with IP based equipment. Now that the shuttle is gone the elimination of the serial equipment is increasing.
Another note, the video monitors on the old consoles were HDTV's of their day. Sure they were black and white, but they had more scan lines, nobody that worked on those is on shift at the moment for me to ask but I think they were in the 800 lines range, that's part of why the original Apollo landings were broadcast from a news camera pointed at a monitor instead of from a direct feed - commercial television equipment wasn't designed to work with the video signals we used then.
There is still just a little bit of Apollo era equipment in use today. The drawings of the projection system hint at the mirrors that bounce the projector images to the screen. In MOCR 2 the original mirrors are still used, but they're mostly for tourist and occasional use, the ones in FCR1 were replaced with Mylar mirrors a couple of years ago. The Apollo era mirrors were incredibly thick and NOT safety glass. One of the workers that was sent to remove the ones for FCR1 was one of the same workers sent to install them 40 something years earlier. The telecom frames from the Apollo era are still in use and actively maintained, there's just not as much frame now as their used to be.
More trivia - the communications keysets on those old consoles belonged to the VIS system - which stood for Voice Intercommunications System. That was replaced by DVIS which was Digital Voice Intercommunications system. DVIS in now all but replaced by DVICE, Digital Voice Interface Communications Equipment. They still call all of them keysets [nasaspaceflight.com]. I showed my daughter one of the VIS keysets and asked if she knew what that round thing was for. She had no clue. I explained to her that's how they used to make phone calls.
Re:Highlights from TFA, and Apollo 13 details (Score:4, Interesting)
Sy Liebergot also said those handles on side of monitor are referred as "security handles" for situations when the controller gets worried when things are going wrong. He can grab on to those handles like a scared child holding onto mommy.
I had fortunate opportunity to have Sy for our featured speaker at an Engineers Week banquet in 1996. Sy said Hollywood built a replicate MOCR because need to remove certain consoles for particular camera angles. But this MOCR replica was made so well, it was spooky. Everything was made to detail including ashtrays full of cigarette butts and all those documents in binders and hanging on clipboards behind the controllers were all reprints [or at least as much as they can get] of Apollo 13 flight plan. Speaking of documentation, much of that was thrown out. Some of the controllers saved material and they knew those flights were historic but there's just so much that can be saved. And everything was re-written for the next mission. Liebergot said Skylab was the worst in terms of paperwork, there was tons of it (maybe too much). He also said Skylab was the most demanding, i.e. rotating shifts. It was during this period when that had highest divorce rate for those working in mission control.
Sy was not alone with his EECOM, he had a few guys in backroom with more extensive displays including stripchart recorders. He had no idea what happened when Apollo CSM O2 tank exploded, his first reaction is it was an instrumentation problem because so many systems went overrange like what you would see if PCM data stream gets corrupted. Obviously shortly later they realized an explosion occurred. After his shift he went back to the EECOM support room and saw the stripchart of O2 tank temperature. Trace is level then begins climbing at switch closure of tank stir, trace slopes up until it drops vertically, then rails to top of chart (or bottom, I forgot which but I made a custom coffee mug with this O2 temperature trace and titled, "It's gotta be instrumentation!"
No smoking is allowed in federal buildings these days. Gene Kranz at a AIAA meeting in 1990s said, "us old guys need to smuggle in some 'victory cigars.'"
Another excellent book as recommended by both Liebergot and Kranz is "Apollo: Race to the Moon" by Charles Murray and Catherine Cox. It describes much detail of the program including "mission control" that consists of MOCR, SPAN, and MER. I was most impressed with Mission Evaluation Room headed by Don "Mad Don" Arabian. This room consisted of design development engineers, tables, documentation, blueprints, and telephones for these guys to call respective contractors i.e. North American, Grumman, etc. whenever problems occurred. Liebergot said MOCR deals with realtime, MER deals with fix-its (i.e. "we have funny readings on this instrument." MER says "we'll look into it and get back with you later on." Don Arabian said of his room, "We don't need any damn fancy consoles or anything!" He also said, "When something goes wrong those guys in MOCR ain't got the foggiest idea what to do." Another quote of NASA HQ in Washington DC, "Hubcaps, useless ornamentation." Liebergot said of Arabian that yes he was mad and wild, a slash-and-burn type of guy. However, if you want someone for a lively speaker who still remembers much of the details of Apollo systems, that is Don. Sy Liebergot says he is able to remember much details because people keep asking him same questions.
The big shame... (Score:3)
The big shame is that the consoles aren't complete and are now just dark and inanimate. It's a bit like going around an aircraft museum full of dead aircraft, while it's interesting - I'd much rather see this stuff powered up and running like it used to, just like I would far rather see a B-17 in the air than in a museum never to fly again.
Needs some background research (Score:4, Interesting)
From TFA:
"In the movies, a spacecraft launch is often accompanied by bombastic music and the seat-juddering bass of rockets thundering, with shots of flight controllers frantically mashing buttons intercut with shaky-cam special effects of the launch vehicle clawing its way skyward. I asked Sy about what a person actually experienced while sitting at a console during launch, and it turns out that reality, again, is a very different place from fiction."
The author should learn what he's talking about - the room usually shown during launch (particularly during Apollo 13) is the LCC, not MOCR. The LCC is located adjacent to the VAB at Cape Kennedy and controls the testing, checkout, launch, and flight of the vehicle until it clears the tower, at which point the MOCR takes over.
He's also seems unaware that there's any media other than mini-series and fiction... If you're really interested in the MOCR, and wish for a less slack-jawed account, try and find a copy of Murray & Cox's Apollo (sometimes subtitled "Race to the Moon"). (Hard copies are expensive and collectible sadly.) In 1994 was Apollo along with Lovell's Apollo 13 that first actually discussed the MOCR in detail and kicked off the modern wave of more serious and less starry-eyed books about the Apollo Program.
Re: (Score:2)
From TFA:
"In the movies, a spacecraft launch is often accompanied by bombastic music and the seat-juddering bass of rockets thundering, with shots of flight controllers frantically mashing buttons intercut with shaky-cam special effects of the launch vehicle clawing its way skyward. I asked Sy about what a person actually experienced while sitting at a console during launch, and it turns out that reality, again, is a very different place from fiction."
The author should learn what he's talking about - the room usually shown during launch (particularly during Apollo 13) is the LCC, not MOCR. The LCC is located adjacent to the VAB at Cape Kennedy and controls the testing, checkout, launch, and flight of the vehicle until it clears the tower, at which point the MOCR takes over.
He's also seems unaware that there's any media other than mini-series and fiction... If you're really interested in the MOCR, and wish for a less slack-jawed account, try and find a copy of Murray & Cox's Apollo (sometimes subtitled "Race to the Moon"). (Hard copies are expensive and collectible sadly.) In 1994 was Apollo along with Lovell's Apollo 13 that first actually discussed the MOCR in detail and kicked off the modern wave of more serious and less starry-eyed books about the Apollo Program.
Great article (Score:1)
Listen To The Apollo 11 Lunar Landing (Score:1)