Want to read Slashdot from your mobile device? Point it at m.slashdot.org and keep reading!

 


Forgot your password?
Close
typodupeerror
×
Mars NASA Programming Software Space IT Technology

The Software Behind the Mars Phoenix Lander 152

Posted by timothy from the distant-robot-friends dept.
chromatic writes "Imagine managing a million lines of code to send over seven hundred pounds of equipment millions of miles through space to land safely on Mars and perform dozens of experiments. You have C, 128 MB of RAM, and very few opportunities to retry if you get it wrong. O'Reilly News interviewed Peter Gluck, project software engineer for NASA's Mars Phoenix Lander, about the process of writing software and managing these constraints — and why you're unlikely to see the source code to the project any time soon."
This discussion has been archived. No new comments can be posted.

The Software Behind the Mars Phoenix Lander More

The Software Behind the Mars Phoenix Lander

Comments Filter:

  • Great software! (Score:2, Interesting)

    by brunokummel ( 664267 )
    Does it run on linux?

  • wow, long article, here's the answer to the teaser (Score:5, Interesting)

    by deft ( 253558 ) on Thursday July 10, 2008 @04:16PM (#24143275) Homepage

    basically, its because the code is part of a space vehicle regulated by international arms and trafficking laws. That means Joe Blow doesnt get it.

    Sorry dude, you're Joe Blow. Unless you're reading this from a JPL/NASA'ish sort of place. Then you're just smirking.

    ===================

    FTA:
    Sort of on a different topic, I have a quote here. One of our editors talked to Frank Hecker from the Mozilla Foundation the other day.

    Okay.

    In that talk, he suggested that all software developed by the Federal Government should be released to the public domain or a very, very liberal open-source license. That's not even a copyleft license. Does the American public have any access to the source code currently on the Phoenix? Are there plans to make some of the source code available?

    Well, no. There are no plans to make that available. And one of the issues that we have is that our spacecraft are designated as subject to international trafficking and arms regulations. So even --

    Crypto regulations in exporting and such?

    Yeah. Yeah. I mean even though these are not military spacecraft, the technology used in them is space technology. And so the State Department does not allow us to release anything that we've done in terms of technical details to foreign scrutiny. Now, in fact as I said, we have a team of Canadians. The Canadians delivered our meteorology instruments, and we had to be very careful about our relationship with them and how much we could disclose to them.

    Really?

    Yeah. Yeah.

    I can see that in applying control software, but how about the payload software?

    Even the payload software -- in this particular case, remember that the payload software operates within the confines of the RAD 6000 that contains the spacecraft software. And although the newer versions of real-time operating systems allow you to compartmentalize better, the older ones are just global name space. So there really wasn't any way to allow them to provide software for the MET instruments. So we had to define an interface and build the software at JPL, and then do our integration testing. And we worked closely with the Canadians in terms of the integration testing and making sure that the software was going to do what they needed it to do.

    Right.

    But we could not actually release the source code to them.

    • This is basically the reason why space technology is so primitive. The science has been stifled for years by government regulations.

      • Re:wow, long article, here's the answer to the tea (Score:5, Insightful)

        by megaditto ( 982598 ) on Thursday July 10, 2008 @04:28PM (#24143517)

        It's a tricky balance though. Nuclear missile launch codes are also -- technically -- public property, yet I am not sure it'd be a good idea to release that in the public domain.

        I think the way things are handled right now is the best we are going to get: basic science is open, applied scientific results are secret.

      • Your statement is flawed. (Score:5, Insightful)

        by flattop100 ( 624647 ) on Thursday July 10, 2008 @04:30PM (#24143545)
        Space technology is not "flawed." It is rigorously tested to survive A)Lift off B)Months and years of dormancy C)Descent D)Operation on another planet millions of miles away, with minutes-long latency. Beyond that, it has to be tested time and again to make sure there are NO errors. If you computer at home freezes, you hit reset. Trying pushing the reset button on a Mars rover--let me know how that works out for you. Space technology is not primitive. It may seem simplistic, but that's to guarantee functionality. Read the definition of "mission-critical" and think about what you typed there. It's a little different that "recreational software development."

        • Re: (Score:2)

          by QuantumG ( 50515 ) *

          Umm.. not talking about software.. talking about space travel in general.

        • Re: (Score:2, Funny)

          by mystik ( 38627 )

          Well. They really can make the software as complex + bug ridden as they want.

          They only have to make sure that the code that runs the robot that presses the reset button is bug free :)

        • Re:Your statement is flawed. (Score:5, Informative)

          by Lumpy ( 12016 ) on Thursday July 10, 2008 @04:42PM (#24143773) Homepage

          I know exactly how to do that. I program Crestron systems remotely as well as some ham radio embedded stuff. Cresron gear across the country cant be reset without paying someone to do so, and an embedded PC in the norther wild of michigan on a tower that takes a weekend to go and fix are very much the same thing.

          You check, recheck, and test on the copy system. you also build in fail safes so that if your upload fails, it reverts to the old code or fails to the loader so you can upload again.

          • You can't use out of band management on your ham gear? Or the PC/embedded system driving it?

            • Re: (Score:3, Informative)

              by Lumpy ( 12016 )

              Yes I can, if I go to the tower to connect to it. Although last time I disconnected all those wires because lightning uses those 200 foot wires coming down as antennas and destroys the computer running the digi. so right now you have to drive to the location, climb the tower, get the box, climb down, work on it, test it, climb up reinstall it and then climb down.

              i'd rather make sure I get everything right and upload via 6 meters at 1200bps overnight from 30 miles away.

        • it's trivial to implement. Just create a device that temporarily cuts the power periodically unless instructed to do otherwise by the computer. Voilà you got an automatic reset switch.

          • Re:Reset button on a lander (Score:5, Informative)

            by Buran ( 150348 ) on Thursday July 10, 2008 @06:25PM (#24145255)

            It's called a dead man's switch and is implemented in rail locomotives, for example. A horn sounds a tone at (probably) random intervals and you must press a button within a certain amount of time or the engine is throttled to idle. Previously, you had to keep your foot on a pedal at all times but it was defeated by just putting a brick on it.

            The idea is that if you're dead, you can't hit the switch, so the train you're supposed to be controlling will stop rather than plow through a stop signal at some later time and hit something or go off the track.

          • Re: (Score:2)

            by 4D6963 ( 933028 )

            it's trivial to implement. Just create a device that temporarily cuts the power periodically unless instructed to do otherwise by the computer. Voilà you got an automatic reset switch.

            Or how about you just RTFA instead of trying to reinvent the wheel? It says that they have this external stop watch system thing that expects a ping from the main computer thing every 64 seconds and that if it doesn't get it it resets the computer. Well of course there's a bit more to it but just RTFA.

        • Re: (Score:3, Insightful)

          by Kerkyon ( 584936 )

          Beyond that, it has to be tested time and again to make sure there are NO errors.

          Well, no, not really. It's far more economical to ensure that the system can recover from serious errors and that errors are fixable, rather than try to assure zero errors. Mainly because the former is possible and the latter isn't.

          If you computer at home freezes, you hit reset. Trying pushing the reset button on a Mars rover--let me know how that works out for you.

          This is addressed in TFA -- the watchdog timer takes care of it; they're ubiquitous in high-reliability embedded systems.

          • Re: (Score:3, Interesting)

            by DerekLyons ( 302214 )

            Beyond that, it has to be tested time and again to make sure there are NO errors.

            Well, no, not really. It's far more economical to ensure that the system can recover from serious errors and that errors are fixable, rather than try to assure zero errors. Mainly because the former is possible and the latter isn't.

            Well, the shuttle software has zero bugs - or seemingly as close to it as to be indistinguishable from zero. The software for the [nuclear tipped missile] fire control system I used to work on in th

            • Well, the shuttle software has zero bugs - or seemingly as close to it as to be indistinguishable from zero.

              numbers [fastcompany.com]:

              the last three versions of the program -- each 420,000 lines long-had just one error each. The last 11 versions of this software had a total of 17 errors. Commercial programs of equivalent complexity would have 5,000 errors.

              So we could expect the Mars code to have about 3 bugs if it's managed as well. Being that it's a short project, doesn't carry humans, and doesn't have as long a history, I

          • Or haven't you noticed how often you windows desktop "flickers". ie. It got hit by a watchdog restart.

        • Re: (Score:3, Informative)

          by coop0030 ( 263345 )

          If you computer at home freezes, you hit reset. Trying pushing the reset button on a Mars rover--let me know how that works out for you.

          Actually, in the article, he basically stated that they essentially do have a reset button. The software pings the hardware every 64 seconds. If the hardware doesn't receive the ping on time, it resets the software to potentially resolve the error/lockup.

          Now in the event that that doesn't work, we have a whole second set of avionics onboard. So the hardware will try to boot to the same side, and if the same side doesn't come up and start stroking the watch-stop timer, then it will swap to the other side and boot the first side.

        • Re: (Score:3, Informative)

          by imsabbel ( 611519 )

          Also, people often forget that by the time a probe or rover makes the news, many years have passed since the "feature freeze".
          Years of design, testing, waiting for a launch possibility, travel.

        • Dude, the lunar lander program crashed repeatedly on Neil Armstrong and Buzz Aldrin right when they were trying to land on the moon. It was so bad, that mission control basically told them to ignore it and Neil went ahead and landed the thing really by the seat of his own pants. You really can't have a bug much more worse than that!

          When you think about it, space software is probably the most unreliable software there is. I mean, it is a classic cathedral design, has only a handful of users, and so, yeah,

          • Those crashes was because the computer was getting more data input than expected. More precisely, the programmers had only foreseen one of the radars (landing, or rendezvous) to be on at any given time. This is written down into the flight plan. But on approach, the astronauts felt that in the case of a screwup they wanted the rendezvous radar already on so they could meet up faster. Something of a bug, but perhaps more of a mis-communication of requirements.

        • Space technology is not "flawed." It is rigorously tested to survive A)Lift off B)Months and years of dormancy C)Descent D)Operation on another planet millions of miles away, with minutes-long latency.

          If it were open source, maybe someone a few years ago would have noticed that feet aren't meters.

      • Re: (Score:2, Troll)

        by LWATCDR ( 28044 )

        Funny but I would say that Phoenix is anything but primitive.

        • Re: (Score:2)

          by QuantumG ( 50515 ) *

          Hehe, by today's standards. Compared to the entire future history of space technology, we're still in the primitive phase.

          • Re: (Score:2)

            by 4D6963 ( 933028 )

            Hehe, by today's standards. Compared to the entire future history of space technology, we're still in the primitive phase.

            That whole "OMG an IBM POWER running at 33 MHz == primitive" thing is just silly. Yeah, your telephone is faster than that, so what? It doesn't make it better than the Apollo program's on-board computer, no matter how "primitive" it is. It does everything it has to do and that's what matters. What would be the interest of having a 2 GHz CPU on a space probe rather than a 33 MHz CPU? Not that much, considering that the CPU speed is hardly a bottleneck. A faster CPU won't give more pixels to your images, it w

            • Re: (Score:2)

              by QuantumG ( 50515 ) *

              You're like the 4th idiot who has replied to me and moaned about me saying the computing platform was primitive.

              I was trying to say that. I was trying to say that all of science related to space and rockets and launch, etc, is primitive because it has been so suppressed.

              I make this clear just a little further on in the thread, but Slashdot's thread system encourages people to not read the entire thread, so we get misunderstandings like this.

      • Re: (Score:3, Informative)

        by deft ( 253558 )

        I was told that the reason space craft run on very old perating systems is not because of the ideas you're thinking, but because old operating systems have basically no unknown quirks. They are having anything unknown pop out.

        the actions the have to undertake are just fine on old tech, because OLD = PROVEN in alot of cases.

      • This is basically the reason why space technology is so primitive. The science has been stifled for years by government regulations.

        If being primitive means stable, predictable, and rock solid dependable - by all means bring it on!

        Seriously, sometimes you simply don't need more processing power as it doesn't buy you anything. The missile fire control system I worked on in the Navy only had a clock speed of 1MHZ, but it still spend a fair amount of time waiting for the hardware to catch up. (Physic

    • Effectively, the US gov't is saying that information/human knowledge (or at least, some information) is a weapon. That would make Stallman's position that everyone deserves access to all human knowledge somewhat analgous to the Fourth Amendment, no?

      Weird!

    • basically, its because the code is part of a space vehicle regulated by international arms and trafficking laws. That means Joe Blow doesnt get it.

      Even if you did get it, what the hell would you do with it? It isn't like you could borrow a snippet here for the video codec you are writing or post a snippet on the forums there to help someone with the chat program they are writing. Even just reading it straight out is going to be like studying hieroglyphics because you don't have all the hardware specs for

      • Actually the source code would probably not be very instructive, even if was released, due to strict rules set down in the JPL code requirements (no use of dynamic memory for example) that would make the programs largely unsuitable for re-use in other projects or, at the very least, a poor example of how to write efficient code (no malloc, pointers, or other associated language features).

    • Here's an interesting web-page on the processors used in various space probes over the years.

      http://www.cpushack.net/space-craft-cpu.html [cpushack.net]

      It seems Viking was the first to use micro-processors. Before that they used TTL, which is sort of a roll-your-own CPU based on bunches of simpler logic chips (NAND gates, multiplexers, etc.).
         

    • basically, its because the code is part of a space vehicle regulated by international arms and trafficking laws. That means Joe Blow doesnt get it.

      Sorry dude, you're Joe Blow. Unless you're reading this from a JPL/NASA'ish sort of place. Then you're just smirking.

      Maybe he's a rocket scientist. What do you know about Mr. Blow's job?

  • a related story? http://developers.slashdot.org/article.pl?sid=08/06/06/2333206

  • Not like the olden days (Score:5, Interesting)

    by Average ( 648 ) on Thursday July 10, 2008 @04:18PM (#24143343)

    I'm curious how many old kinds of code we're still communicating with. FTA, Cassini is ADA-based. I know the Voyager craft are in FORTH (my first programming love).

    • Don't forget the Oldest Kind of Code [wikipedia.org]

    • Re:Not like the olden days (Score:5, Informative)

      by frieko ( 855745 ) on Thursday July 10, 2008 @06:04PM (#24144999)
      I wouldn't call Ada obsolete. C is only good because a lot of people know C and lots of COTS parts are C. But you can't beat Ada if you need a language that was built from the ground up for mission-critical reliability.

  • Maybe because it's freaking NASA?

    • I believe its that this is also due to the fact this is often the contractor's propriety code.

      Even if it wasn't ITAR restricted, you wouldn't see the contractors releasing the code so that they can re-use, re-sell, and re-profit.

    • Re:Why unlikely to see the source? (Score:4, Informative)

      by rk ( 6314 ) * on Thursday July 10, 2008 @04:47PM (#24143861) Journal
      NASA releases all kinds of code. As an example, many people in the space science community rely on SPICE from JPL's Navigation and Ancillary Information Facility, and you can play from home [nasa.gov]. I think the newest version of Celestia [shatters.net] has a CSPICE interface to get extremely accurate planetary positions and spacecraft pointings into it.

      • Re: (Score:3, Informative)

        by exley ( 221867 )

        I stand at least partly corrected then... Although still not sure they would release code like what is running the Mars lander. On the other hand, what would anyone do with that code if it was available? I suppose there could be some homebrew interplanetary lander projects out there... :)

        And for anyone else initially confused and unwilling to click links, the reference to SPICE in parent's post isn't about the circuit simulator!

        • Re: (Score:2)

          by smaddox ( 928261 )

          Howabout building a homebrew transmitter, using the code to reverse engineer the communication protocal, then DOS'ing the lander.

    • Huh? (Score:5, Informative)

      by oneiros27 ( 46144 ) on Thursday July 10, 2008 @05:07PM (#24144201) Homepage

      Maybe because it's freaking NASA?

      What's that have anything to do with it?

      NASA has an OSI approved license:

      It could probably be easier to find NASA software, and I doubt this particular software would ever be released, but there's lots of NASA software that's been released:

      There's issues because much of NASA stuff is done as part of grants, and so it's officially owned by the academic / research institution that won the grant ... as such, there might be other NASA funded code that's out there, that you don't know is NASA code... at least one program (AISRP [nasa.gov]) has started a place to collect software by grantees.

      I've been to NASA workshops where there's plenty of code that's being written where people would LOVE to have their software find a broader audience. At the last one, we had an hour debate on if we were allowed to release code as GPL, as that'd place restrictions on the use of the code (that derivative copies have to be open), which should not be done as the software was developed w/ federal money and as such citizens should be free to do whatever they want with it. I think someone was assigned to talk to NASA's legal department and find out what we had to do to release our code.

      • Re: (Score:2)

        by AMuse ( 121806 )

        I'm super excited to see NASA's opensource work getting exposure, so don't take my nitpick the wrong way -- just wanted to put the fact out in the open that the NAS (NASA Advanced Supercomputing) is actually at Ames. It has its own tertiary level DNS space as a major resource but the facilities are in the same place.

      • Re: (Score:2)

        by rah1420 ( 234198 )

        In addition there's a NASA initiative called COSMIC [openchanne...dation.org] that I remember subscribing to Back In The Day. According to the links [nasa.gov] I've found, it looks like COSMIC still has some legs.

    • NASA releases lots of items. For example, they were the ones that did the original network card drivers for Linux.

      And to be honest, even the NSA releases a fair amount of OSS. Keep in mind that they have 2 missions.

  • How hard can it be? (Score:4, Funny)

    by Dex5791 ( 973984 ) on Thursday July 10, 2008 @04:23PM (#24143445)
    Just open the existing code base for the previous lander and cut&paste.

  • Canadians! (Score:3, Funny)

    by vimm ( 1300813 ) on Thursday July 10, 2008 @04:32PM (#24143603)
    FTFA

    We have a team of Canadians... we had to be very careful about our relationship with them... how much we could disclose to them

    In soviet canada, mars probe software discloses you?

  • My eyes, they burn! (Score:5, Insightful)

    by geekoid ( 135745 ) <{moc.oohay} {ta} {dnaltropnidad}> on Thursday July 10, 2008 @05:15PM (#24144327) Homepage Journal

    What a horrible interview.

    WHen interviewing someone, you don't tell them what they know, you ask them.

    An example:
    "That's not a really beefy embedded board actually. It's what, thirty-three megahertz?

    Yeah. That's â" yeah.


    About 128 megabytes of RAM?


    That's right.

    I imagine that produces some interesting challenges, getting all of that software to run together on that board while also having it land on the planet successfully.
    "

    Painful.

    • Re: (Score:3, Insightful)

      by rho ( 6063 )

      But just think how smart and well-informed the interviewer sounds!

      I stopped reading halfway through. Useless interviewer leads to useless interview.

    • Actually, these questions, to me, made it seem like a good interview.

      For one, it shows the interviewer actually did his homework. He knows a lot about what he is talking about. How very different from reporters who go to interviews with almost zero knowledge, then try to make a story out of what they were told, and get it wrong. I've seen that happen far too many times.

      For another, it puts the interviewee at ease. Some people get really nervous about interviews. If you ask them a few easy questions, they'll

    • Re: (Score:2)

      by syousef ( 465911 )

      I agree with you. My favourite part was when the interviewer cut off the interviewee. Brilliant technique I thought. Why he has the makings of a slashdot troll.

  • Is the reason they hide it is to conceal the fact that it was originally open source code and they just copped it to use for this since they were not up to the task. Might sound conspiratorial but I have seen my work stolen to be incorporated into computers for the military and they even left my back doors in there. That is not some fantasy of mine. I have one of the battlefield computers with my code in it and it is because they went to mil. surplus and not destroyed like they were supposed to be. A milita
  • While in a lot of places ITAR can actually hurt us, this is actually helping us. A log of this software would be useful to countries to help them develop precision guidance systems for their missles. As it is, Iran uses GPS to help with their system, and a another dead reckoning system. Apparently, Iran is looking to add in Beidou as it becomes available (overall, clinton pretty good, but his china policy was a disaster). Even China would use this code.

  • hello.c (Score:5, Funny)

    by xx_chris ( 524347 ) on Thursday July 10, 2008 @05:35PM (#24144633)
    #include void main() { printf("Hello Mars\n"); }
  • I throw away hardware better than that! (For the greenies, please read "recycle" instead of "throw away".)

    • Re: (Score:2)

      by 4D6963 ( 933028 )

      I throw away hardware better than that!

      It depends what you call better. If by 'better' you mean 'bigger' I take it you'd think that this [abc.net.au] is better than this [americanmusclecars.net].

      And 128 MB is plenty. Sure in the wonderful of bloated modern PC software where the simplest e-mail client hogs up 40 MB of RAM it might not seem like much but there's little you can't do with 128 MB by designing software properly that can't be done with 2 GB. Just to put in perspective how 128 MB is a lot, the Playstation 2 only had 32 MB of RAM, which was obviously enough to run Grand Thef

  • Misleading summary (Score:3, Interesting)

    by techno-vampire ( 666512 ) on Thursday July 10, 2008 @06:01PM (#24144959) Homepage
    I suppose it's inevitable that the summary of a Slashdot article is inaccurate, but in this case it's highly misleading. The code in the Phoenix Lander has nothing whatsoever to do with getting it to Mars. The Spaceprobe Navigation Package (Are they still using MOPS and TRAM, I wonder? After all, they were good enough for Voyager I and II.) run on mainframes at JPL, in Pasadena, and course corrections are sent from their to the space craft. This is because the same programs doing the navigation for Phoenix can be used at the same time for other missions, instead of wasting valuable memory (and the energy needed to run them) on putting a separate copy of the program on every, single probe.

  • No million line API's to conform to. No overlooked, hidden RFC on page 3000 of the latest standard revision from Sony. No implementing everything twice. Peter Gluck, software manager, must be a huge asset with the C language obstacle. Software management triumphs again.

  • So Peter? (Score:3, Funny)

    by codepunk ( 167897 ) on Thursday July 10, 2008 @10:16PM (#24147541)

    You're the project software engineer. I noticed that was singular. Is there just one product software engineer? Like sort of the managing engineer for the project?

    Yeah. Project Software Systems Engineer is the title. And our software was developed -- the flight system software was developed in three different locations. Lockheed Martin developed the spacecraft software, and then we had payload software developed by both the University of Arizona and the Jet Propulsion Laboratory.

    So Peter what is it that you do here?

    I hand specifications to engineers that write the flight software.

    So you take the papers to them?

    Well no my secretary does that...

Slashdot Top Deals

Solutions are obvious if one only has the optical power to observe them over the horizon. -- K.A. Arsdall

Close