Slashdot Log In
The Software Behind the Mars Phoenix Lander
Posted by
timothy
on Thu Jul 10, 2008 04:04 PM
from the distant-robot-friends dept.
from the distant-robot-friends dept.
chromatic writes "Imagine managing a million lines of code to send over seven hundred pounds of equipment millions of miles through space to land safely on Mars and perform dozens of experiments. You have C, 128 MB of RAM, and very few opportunities to retry if you get it wrong. O'Reilly News interviewed Peter Gluck, project software engineer for NASA's Mars Phoenix Lander, about the process of writing software and managing these constraints — and why you're unlikely to see the source code to the project any time soon."
Related Stories
This discussion has been archived.
No new comments can be posted.
The Fine Print: The following comments are owned by whoever posted them. We are not responsible for them in any way.
Full
Abbreviated
Hidden
Loading... please wait.
Great software! (Score:2, Interesting)
Re:Great software! (Score:5, Informative)
Nope. VxWorks.
These questions and more answered in TFA.
Parent
Re:Great software! (Score:5, Funny)
you mean...
Answers to these questions and more, rendered inaccessible by /.
Parent
Nope, its written in C. (Score:4, Funny)
But you didnt read the article, you were more just hoping for a slashdot linux rally cry or something, werent you.
But if someone crys in a dark basement creepily lit by a monitor, does anyone here it?
Damn, i guess I did.
Parent
Re: (Score:2, Redundant)
Does it run on linux?
Well I'm glad it didn't run Windows for Workgroups 3.11, because then they wouldn't get any support anymore over there on Mars!
wow, long article, here's the answer to the teaser (Score:5, Interesting)
basically, its because the code is part of a space vehicle regulated by international arms and trafficking laws. That means Joe Blow doesnt get it.
Sorry dude, you're Joe Blow. Unless you're reading this from a JPL/NASA'ish sort of place. Then you're just smirking.
===================
FTA:
Sort of on a different topic, I have a quote here. One of our editors talked to Frank Hecker from the Mozilla Foundation the other day.
Okay.
In that talk, he suggested that all software developed by the Federal Government should be released to the public domain or a very, very liberal open-source license. That's not even a copyleft license. Does the American public have any access to the source code currently on the Phoenix? Are there plans to make some of the source code available?
Well, no. There are no plans to make that available. And one of the issues that we have is that our spacecraft are designated as subject to international trafficking and arms regulations. So even --
Crypto regulations in exporting and such?
Yeah. Yeah. I mean even though these are not military spacecraft, the technology used in them is space technology. And so the State Department does not allow us to release anything that we've done in terms of technical details to foreign scrutiny. Now, in fact as I said, we have a team of Canadians. The Canadians delivered our meteorology instruments, and we had to be very careful about our relationship with them and how much we could disclose to them.
Really?
Yeah. Yeah.
I can see that in applying control software, but how about the payload software?
Even the payload software -- in this particular case, remember that the payload software operates within the confines of the RAD 6000 that contains the spacecraft software. And although the newer versions of real-time operating systems allow you to compartmentalize better, the older ones are just global name space. So there really wasn't any way to allow them to provide software for the MET instruments. So we had to define an interface and build the software at JPL, and then do our integration testing. And we worked closely with the Canadians in terms of the integration testing and making sure that the software was going to do what they needed it to do.
Right.
But we could not actually release the source code to them.
Re:wow, long article, here's the answer to the tea (Score:5, Insightful)
This is basically the reason why space technology is so primitive. The science has been stifled for years by government regulations.
Parent
Re:wow, long article, here's the answer to the tea (Score:5, Insightful)
It's a tricky balance though. Nuclear missile launch codes are also -- technically -- public property, yet I am not sure it'd be a good idea to release that in the public domain.
I think the way things are handled right now is the best we are going to get: basic science is open, applied scientific results are secret.
Parent
Re:wow, long article, here's the answer to the tea (Score:5, Interesting)
Neither the basic science, nor the applied science (aka engineering) is open.
The only reason any of us know the rocket equation is because it was invented before these laws were.
Parent
Re: (Score:2)
I seem to be getting a lot of recent hits on http://scholar.google.com/scholar?q=rocket+equation [google.com]
Re: (Score:2)
Nuclear missile launch codes are also -- technically -- public property, yet I am not sure it'd be a good idea to release that in the public domain.
Why not? I would think they'd be pretty useless without the rest of the stuff in the football and clearance through whatever other security protocols there are.
Your statement is flawed. (Score:5, Insightful)
Parent
Re: (Score:2)
Umm.. not talking about software.. talking about space travel in general.
Re: (Score:2, Funny)
Well. They really can make the software as complex + bug ridden as they want.
They only have to make sure that the code that runs the robot that presses the reset button is bug free :)
Re:Your statement is flawed. (Score:5, Informative)
I know exactly how to do that. I program Crestron systems remotely as well as some ham radio embedded stuff. Cresron gear across the country cant be reset without paying someone to do so, and an embedded PC in the norther wild of michigan on a tower that takes a weekend to go and fix are very much the same thing.
You check, recheck, and test on the copy system. you also build in fail safes so that if your upload fails, it reverts to the old code or fails to the loader so you can upload again.
Parent
Re: (Score:3, Informative)
Yes I can, if I go to the tower to connect to it. Although last time I disconnected all those wires because lightning uses those 200 foot wires coming down as antennas and destroys the computer running the digi. so right now you have to drive to the location, climb the tower, get the box, climb down, work on it, test it, climb up reinstall it and then climb down.
i'd rather make sure I get everything right and upload via 6 meters at 1200bps overnight from 30 miles away.
Re: (Score:3, Insightful)
Beyond that, it has to be tested time and again to make sure there are NO errors.
Well, no, not really. It's far more economical to ensure that the system can recover from serious errors and that errors are fixable, rather than try to assure zero errors. Mainly because the former is possible and the latter isn't.
If you computer at home freezes, you hit reset. Trying pushing the reset button on a Mars rover--let me know how that works out for you.
This is addressed in TFA -- the watchdog timer takes care of it; they're ubiquitous in high-reliability embedded systems.
Re: (Score:3, Interesting)
Well, the shuttle software has zero bugs - or seemingly as close to it as to be indistinguishable from zero. The software for the [nuclear tipped missile] fire control system I used to work on in th
Re: (Score:3, Informative)
If you computer at home freezes, you hit reset. Trying pushing the reset button on a Mars rover--let me know how that works out for you.
Actually, in the article, he basically stated that they essentially do have a reset button. The software pings the hardware every 64 seconds. If the hardware doesn't receive the ping on time, it resets the software to potentially resolve the error/lockup.
Now in the event that that doesn't work, we have a whole second set of avionics onboard. So the hardware will try to boot to the same side, and if the same side doesn't come up and start stroking the watch-stop timer, then it will swap to the other side and boot the first side.
Re: (Score:3, Informative)
Also, people often forget that by the time a probe or rover makes the news, many years have passed since the "feature freeze".
Years of design, testing, waiting for a launch possibility, travel.
Nah, space technology is just as junky as us. (Score:3, Insightful)
Dude, the lunar lander program crashed repeatedly on Neil Armstrong and Buzz Aldrin right when they were trying to land on the moon. It was so bad, that mission control basically told them to ignore it and Neil went ahead and landed the thing really by the seat of his own pants. You really can't have a bug much more worse than that!
When you think about it, space software is probably the most unreliable software there is. I mean, it is a classic cathedral design, has only a handful of users, and so, yeah,
Re:Reset button on a lander (Score:5, Informative)
It's called a dead man's switch and is implemented in rail locomotives, for example. A horn sounds a tone at (probably) random intervals and you must press a button within a certain amount of time or the engine is throttled to idle. Previously, you had to keep your foot on a pedal at all times but it was defeated by just putting a brick on it.
The idea is that if you're dead, you can't hit the switch, so the train you're supposed to be controlling will stop rather than plow through a stop signal at some later time and hit something or go off the track.
Parent
Re:Reset button on a lander (Score:4, Informative)
It's called a dead man's switch and is implemented in rail locomotives, for example.
Dead man's switch for humans, watchdog timer for computers. TFA mentions the phoenix watchdog going off every 64 seconds.
Parent
Re: (Score:2, Troll)
Funny but I would say that Phoenix is anything but primitive.
Re: (Score:2)
Hehe, by today's standards. Compared to the entire future history of space technology, we're still in the primitive phase.
Re: (Score:3, Informative)
I was told that the reason space craft run on very old perating systems is not because of the ideas you're thinking, but because old operating systems have basically no unknown quirks. They are having anything unknown pop out.
the actions the have to undertake are just fine on old tech, because OLD = PROVEN in alot of cases.
Not like the olden days (Score:5, Interesting)
I'm curious how many old kinds of code we're still communicating with. FTA, Cassini is ADA-based. I know the Voyager craft are in FORTH (my first programming love).
Re: (Score:2)
Don't forget the Oldest Kind of Code [wikipedia.org]
Re:Not like the olden days (Score:5, Informative)
Parent
Why unlikely to see the source? (Score:2)
Maybe because it's freaking NASA?
Re:Why unlikely to see the source? (Score:4, Informative)
Parent
Re: (Score:3, Informative)
I stand at least partly corrected then... Although still not sure they would release code like what is running the Mars lander. On the other hand, what would anyone do with that code if it was available? I suppose there could be some homebrew interplanetary lander projects out there... :)
And for anyone else initially confused and unwilling to click links, the reference to SPICE in parent's post isn't about the circuit simulator!
Huh? (Score:5, Informative)
What's that have anything to do with it?
NASA has an OSI approved license:
It could probably be easier to find NASA software, and I doubt this particular software would ever be released, but there's lots of NASA software that's been released:
There's issues because much of NASA stuff is done as part of grants, and so it's officially owned by the academic / research institution that won the grant ... as such, there might be other NASA funded code that's out there, that you don't know is NASA code... at least one program (AISRP [nasa.gov]) has started a place to collect software by grantees.
I've been to NASA workshops where there's plenty of code that's being written where people would LOVE to have their software find a broader audience. At the last one, we had an hour debate on if we were allowed to release code as GPL, as that'd place restrictions on the use of the code (that derivative copies have to be open), which should not be done as the software was developed w/ federal money and as such citizens should be free to do whatever they want with it. I think someone was assigned to talk to NASA's legal department and find out what we had to do to release our code.
Parent
How hard can it be? (Score:4, Funny)
Canadians! (Score:3, Funny)
In soviet canada, mars probe software discloses you?
My eyes, they burn! (Score:5, Insightful)
What a horrible interview.
WHen interviewing someone, you don't tell them what they know, you ask them.
An example:
"That's not a really beefy embedded board actually. It's what, thirty-three megahertz?
Yeah. That's â" yeah.
About 128 megabytes of RAM?
That's right.
I imagine that produces some interesting challenges, getting all of that software to run together on that board while also having it land on the planet successfully.
"
Painful.
Re: (Score:3, Insightful)
But just think how smart and well-informed the interviewer sounds!
I stopped reading halfway through. Useless interviewer leads to useless interview.
hello.c (Score:5, Funny)
Misleading summary (Score:3, Interesting)
So Peter? (Score:3, Funny)
You're the project software engineer. I noticed that was singular. Is there just one product software engineer? Like sort of the managing engineer for the project?
Yeah. Project Software Systems Engineer is the title. And our software was developed -- the flight system software was developed in three different locations. Lockheed Martin developed the spacecraft software, and then we had payload software developed by both the University of Arizona and the Jet Propulsion Laboratory.
So Peter what is it that you do here?
I hand specifications to engineers that write the flight software.
So you take the papers to them?
Well no my secretary does that...
Re:128MB? (Score:5, Funny)
| would you like me to: |
| o Help me come up with new jokes |
| o Help me bash Bill Gates |
| o Help me spell Microdollarsignoft |
| o No thanks, let me keep making |
| a redundant ass of myself |
\ P.S.: **** open sores /
\
\
\ ____
\ / __ \
\ O| |O|
|| | |
|| | |
|| |
|___/
--
cpu0: Microsoft Clippium ("GenuineClippy" ChromedMetal-Class). Paperbinding, lockpicking, fish-hook-hack support.
template greedily stolen from this guy: http://slashdot.org/~ClippySay
Parent
Re: (Score:2, Funny)
Micro$soft?
Good job.
Re: (Score:3, Funny)
Micro$soft?
Good job.
Help me spell Microdollarsignoft == Micro$oft... Fail is you.
Re:Millions of lines? (Score:5, Interesting)
Well, of course, the proper response to your query is "it doesn't work like that" or "neither are a good metric" or something, but that's a big boring, so let's consider an empirical result.
liblink-grammar.so.4.3.5 is 616129 bytes. It is built from 23289 lines of code. So that's about 26.4 bytes of code per line.So 128 MB of RAM can hold about 5,084,005 lines of code :)
Parent
Re:Millions of lines? (Score:4, Insightful)
For legacy reasons that have just sort of stuck, the maximum line size in C is often no more than 80 characters long, although plenty of people ignore this unwritten "rule" these days.
I'd say you can safely assume that each line is around 80 characters, though, as a lot of lines will use very few.
But it's all irrelevant as I doubt they'd bother transmitting the entire source code to Mars when they can just compile it into a good ol' binary that's probably a hundredth of the original source code's size, if not more.
Parent
You know (Score:5, Funny)
if they wrote it in perl, it would only be 1 line.
Parent
Re:You know (Score:5, Funny)
Yes, but it would use the unholy letters of the dark lord himself, and bring death and destruction upon any man who gazed upon its ghastly source.
Parent
Re: (Score:3, Funny)
Yes, but it would use the unholy letters of the dark lord himself, and bring death and destruction upon any man who gazed upon its ghastly source.
Ah! I see you know APL...
Re: (Score:2)
Re: (Score:3, Interesting)
How many lines of code can 128 MB of RAM hold and what is the average 'line' for C?
I don't know, but 15 years ago, I would have killed for 128 MB of RAM or even a 128 MB HDD. My first "PC" had 4 MB RAM and a 102 MB HDD. It ran DOS 6.2, Windows 3.1 and a host of crappy DOS games. (Actually, I don't think the DOS games used more than the first MB)
Strip the GUI and even the CLI, and you'll find that 128MB is quite a bit if your main concern is code. Data could take quite a chunk of that, but if you're just talking about text files with data and configuration, a few MB could handle it wit