NASA Fixes Galileo, Starts Recovering Data

linuxwrangler writes "After radiation damaged the recorders on Galileo it was feared that the data from the November flyby of Amalthea would be lost. Today NASA announced that they have repaired the recorder and are busy downloading the data. Meanwhile they also contacted pioneer 10 (64 bytes from pioneer10.nasa.gov: icmp_seq=1 ttl=255 time=80700000 ms)"

Rocket scientists

[MacAndrew]

I asked a friend about the Galileo problem, and the heck do you fix something from thousands or millions of miles away? It's very difficult he replied, and if aero/astro people are like him in general, these are bright folks.

Most of his experience had been with trying to figure out why solar arrays in orbit weren't doing their job, where the problem turned out to be not a loose wire but defective engineering (not his :).

Re:Rocket scientists

[Simon Field]

High energy protons had damaged an LED.

By running current through the LED for hours, they annealed it enough to get the job done.

Guy Webster 818-354-6278 Jet Propulsion Laboratory, Pasadena, Calif. December 17, 2002

News Release: 2002-231

Galileo Millennium Mission Status

NASA's Galileo spacecraft has begun transmitting high-priority scientific information that was collected and stored on its tape recorder during the orbiter's early-November dash by Jupiter, which brought it closer to the planet than ever before.

Damage from naturally strong radiation near Jupiter had left the tape recorder inoperable for weeks. Galileo's flight team traced the problem to a light-emitting diode in the electronics controlling the motor drive, and then gradually and carefully completed a successful long-distance repair job.

"We're delighted playback has begun. There was no guarantee we could get to this point," said Dr. Eilene Theilig, Galileo project manager at NASA's Jet Propulsion Laboratory, Pasadena, Calif. "It's a real tribute to the dedication and creativity of the team, and it continues the tradition set by the larger Galileo teams earlier in the mission who overcame challenging setbacks to keep the mission a success."

Playback and transmission began late last week, after tape movements earlier in the week positioned the tape to start the download with the data of greatest scientific interest. That information is from a period starting on Nov. 4, when Galileo was beginning to get closer to Jupiter than the moon Io's orbit. It ends when the spacecraft shut down its instruments on Nov. 5 as part of a precautionary standby reaction to other radiation-induced problems with the spacecraft's computer shortly after passing the inner moon Amalthea.

"We hope this will be the best data set that's ever been collected about the inner region of Jupiter's magnetic environment," said JPL's Dr. Torrence Johnson, Galileo project scientist. The tape playback may also provide recorded information about dust particles that form Jupiter's faint "gossamer" ring, which Galileo flew through on Nov. 5.

The recovery was achieved by running a current through the damaged diode to anneal, or repair, radiation-caused damage. The first annealing attempt of six hours produced barely discernible improvement. Three additional treatments, for a total of 83 more hours of annealing treatment, produced progressive improvements, to the point that the tape recorder can run for about an hour at a time. A fifth treatment produced no additional gain. However, normal playback runs the tape recorder for only a few minutes at a time, so the improvement appears sufficient, said JPL engineer Greg Levanas, who helped plot the recovery strategy.

The flight team plans to continue playback until mid-January, the scheduled end of Galileo's mission operations.

Galileo's tape recorder became critical to the mission's strategy for handling science data after the spacecraft's main antenna failed to open fully during the journey from launch in 1989 to arrival at Jupiter in 1995. Most information from onboard science instruments has been recorded onto tape during the busy days when the spacecraft has flown near one of Jupiter's moons. There have been over 30 such flybys since 1995. The recorded data has been played back slowly for transmission to Earth via Galileo's smaller secondary antenna during the weeks and months before each subsequent flyby.

Galileo has operated five years longer than its original prime mission and has received more than four times the cumulative dose of harmful radiation it was designed to withstand. Last month's encounter took the orbiter nearly twice as close to Jupiter as it had ever been before, and exposed it to the strongest radiation it has yet experienced. In particular, the spacecraft entered the most intense region of a trapped proton belt and was exposed to 40 times the proton radiation of any single previous passage close to Jupiter, probably more than the rest of the mission combined.

The diode that radiation apparently damaged in the tape recorder is a gallium-arsenide semiconductor component that emits light. The motor-drive control has three of them. Light from them shines through windows in a rotating wheel onto detectors on the other side of the wheel. That setup senses the turning of the wheel and feeds digital logic that controls drive signals for the motor.

The damage apparently came from high-energy protons from Jupiter's radiation belt displacing atoms in the semiconductor's crystalline molecular lattice. Passing a current through the diode for hours serves as a way for electron flow to cause some of the displaced atoms to shift back to their original lattice positions.

Galileo has nearly depleted its supply of the propellant needed for pointing its antenna toward Earth and controlling its flight path. While still controllable, it has been put on a course for impact into Jupiter next September. The maneuver prevents the risk of Galileo drifting to an unwanted impact with the moon Europa, where it has discovered evidence of a subsurface ocean that is of interest as a possible habitat for extraterrestrial life.

Additional information about Galileo and the discoveries is available at
http://galileo.jpl.nasa.gov

Re:Rocket scientists

[MacAndrew]

Wow. You are (1) quick and (2) relevant. Why aren't there more like you?

I really did look for this on my own; like a dolt I went to the NASA site, and here I practically grew up in Pasadena. I forwarded this to my rocket friend, thanks.

How they fixed it.

[EccentricAnomaly]

Here's some details [nasa.gov] about the fix. They isolated the problem to a bad LED, and ran current through it to melt away the damage... pretty cool.

Technology

[Noodlenose]

I have to admit that I have immense respect for these alpha nerds from NASA:

imagine trying to repair a 20 year old computer with continuous hardware and mobo probs from a gazillion miles away with only a console and a lag of a couple of hours....

My hat's off to you, guys...

Dirk

Re:Technology

[MacAndrew]

About Galileo, some tales [yarchive.net] from several years ago, mentioning the current tape problem [space.com].

I would like to hear what exactly the engineers did. I have a feeling it was the interplanetary version of whacking your TV set to stop the whine.

Not all twiddle-the-computer exercises work out well. NASA is not one to dwell on failure, but they'll hand-deliver a press release to your door for great news. E.g., I read that contact with one of the Viking landers was lost years ago after someone sent bad data to its antenna tracking system. The lander was very late in its lifespan, but would you like to have been the guy who did it? We've found reasons to keep in touch with even the Voyagers (or should I say V'gers?), as well as the nearly 4x too old Galileo.

The Web is so cool: Galileo's current position [nasa.gov]

And Galileo tour guide [nasa.gov] -- the Galileo stuff at the NASA site is a little dusty. :)

Should we have a moment of silence with spunky Galileo burns up? Do you think the Jovians will retaliate?

Re:Technology

[david.given]

I remember hearing, but can't verify, a particularly inspired hack on, IIRC, a Pioneer: something needed adjustment, possibly an antenna arm. So they turned on all the heaters around the part in question. The resin softened. Then they fired the thrusters to jolt the spacecraft, and switched off the heaters. The arm bent into the correct position, and then hardened.

I suspect that it's a myth, but damn, it ought to be true.

Incidentally, did you know that the Pioneer computers were so simple they didn't have any jump instructions? They just executed all the instructions in memory one sweep after another. Conditionals were done by masking out blocks of code using condition codes. Slow, yes; but the processor could be implemented in a handful of radiation hardened transistors, and if the computer ever reset spontaneously due to, e.g., passing through Jupiter's magnetosphere, noone cared.

And they're still going...

Re:Weird Technology

[MacAndrew]

A lie so good it must be true?

I heard about a guy who timed his code to the speed of rotating drum memory so that "impossible" loops would work. It made the code a tad difficult for others to maintain after he retired.

Re:Weird Technology

[Tolchz]

I believe you're talking about Mel. The story is in the Jargon file I believe.

Re:Weird Technology

[MacAndrew]

No, I thought it was the Story of O.

Wait, that's something else.

You're right. ;-)

Re:Technology

[Hadlock]

somthing like this?

Oh, the five problems? The high-gain antenna didn't deploy. The tape in
the tape recorder tends to stick. The thrusters can explode if fired in
long steady burns, which is why Galileo always fires them in pulses, and
need to be burped regularly to avoid slow propellant decomposition --
TVSat 2 fortunately used the same thrusters, was launched before Galileo,
and had a solar array fail to deploy, so its thrusters were fired long and
hard during attempts to shake the array loose.

got it from the link in your parent's post. i'm assuming 5 is pioneer 5. i dunno for sure, though.

Re:Technology

[tqft]

It was in the article I read - a diode was stuffed so they ran current backward through a large number of times until it reannealed "problem to a light-emitting diode in the electronics controlling the motor drive, " " The recovery was achieved by running a current through the damaged diode to anneal, or repair, radiation-caused damage. The first annealing attempt of six hours produced barely discernible improvement. Three additional treatments, for a total of 83 more hours of annealing treatment, produced progressive improvements, to the point that the tape recorder can run for about an hour at a time. A fifth treatment produced no additional gain. However, normal playback runs the tape recorder for only a few minutes at a time, so the improvement appears sufficient, said JPL engineer Greg Levanas, who helped plot the recovery strategy. " http://www.jpl.nasa.gov/galileo/news/release/press 021217.html

Bunch of damn show offs.

[BoomerSooner]

Nothing worse than reading this stuff and then going back to my projects at work and no longer feeling as elite!

Oh well, would those NASA geniuses be able to improve patient healthcare? (Yes if they worked with me, damnit foiled again!) I guess helping people improve their quality of life is enough for me. It's too bad politics is 99% (coughHIPAAcough) of the problem instead of the solution.

Don't feel bad

[marcus]

Very few of these guys are rookies. They are culled from vast fields of applicants. Work hard, study hard, think hard, and apply for jobs at astro/aero/comm companies. You might just turn into one of these "elites".

Started with pilot lessons, then my eyes went bad. Even though I could no longer meet the mil spec for pilots (much less astronauts), I still got a Navy ROTC scholarship. I became an expert at writing embedded real time code. I got a job at Motorola. My code(not my body, but part of my mind) flies in space today. ;-)

Just as open source goes to show, pride is the greatest motivator when you want inspired, ultimate quality work. Even though my rate today is more than twice what I made back then, I have never, ever put so much effort and thought into a project as that one.

Cheer up, if you really, really try, chances are you will succeed.

Ohh no we will slashdot pioneer10.nasa.gov !!!!

[Nicolay77]

Poor host!!

Re:Ohh no we will slashdot pioneer10.nasa.gov !!!!

[program21]

With a ping like that, you'd think it already was.