Forgot your password?
typodupeerror
NASA Mars Science

Upgrading Software From 350 Million Miles Away 228

Posted by samzenpus
from the distant-support dept.
CWmike writes "Picture doing a remote software upgrade. Now picture doing it when the machine you're upgrading is a robotic rover sitting 350 million miles away, on the surface of Mars. That's what a team of programmers and engineers at NASA are dealing with as they get ready to download a new version of the flight software on the Mars rover Curiosity, which landed safely on the Red Planet earlier this week. 'We need to take a whole series of steps to make that software active. You have to imagine that if something goes wrong with this, it could be the last time you hear from the rover,' said Steve Scandore, a senior flight software engineer at NASA's Jet Propulsion Laboratory. 'It has to work,' he told Computerworld. 'You don't' want to be known as the guy doing the last activity on the rover before you lose contact.'"
This discussion has been archived. No new comments can be posted.

Upgrading Software From 350 Million Miles Away

Comments Filter:
  • by ronhip (465417) on Friday August 10, 2012 @03:10AM (#40942711)

    The spacecraft TRAVELLED 350 million miles to get there, but as of tonight, Mars is only about 157.5 million miles from Earth.

  • Re:Failsafe (Score:5, Informative)

    by fatphil (181876) on Friday August 10, 2012 @03:37AM (#40942915) Homepage
    Exactly. That's how it's done in the telecomms world (infrastructure, not terminals). Typically the new software is given three attempts to boot, and if it doesn't acknowledge that it's fully booted after three attempts, the bootloader falls back to the previous version of the software. Of course, things get tricker if you need to update the bootloader, but those should be very rare situations. However, they in turn can be handled a similar way (typically there's a 3-stage boot, the initial being a ROM bootstrap, then your bootloader, then the OS which you'll want to change).
  • Re:Failsafe (Score:5, Informative)

    by Anonymous Coward on Friday August 10, 2012 @03:42AM (#40942949)

    Computers: The two identical on-board rover computers, called "Rover Compute Element" (RCE), contain radiation hardened memory to tolerate the extreme radiation from space and to safeguard against power-off cycles. Each computer's memory includes 256 kB of EEPROM, 256 MB of DRAM, and 2 GB of flash memory.[22] This compares to 3 MB of EEPROM, 128 MB of DRAM, and 256 MB of flash memory used in the Mars Exploration Rovers.[23]
    The RCE computers use the RAD750 CPU, which is a successor to the RAD6000 CPU used in the Mars Exploration Rovers.[24][25] The RAD750 CPU is capable of up to 400 MIPS, while the RAD6000 CPU is capable of up to 35 MIPS.[26][27] Of the two on-board computers, one is configured as backup, and will take over in the event of problems with the main computer.[22]

    http://en.wikipedia.org/wiki/Curiosity_rover#Specifications [wikipedia.org]

    Data transfer speeds between Curiosity and each orbiter may reach 2 Mbit/s and 256 kbit/s, respectively, but each orbiter is only able to communicate with Curiosity for about eight minutes per day

    When you have little bandwidth, better get it right the first time.

  • Re:it can fly? (Score:5, Informative)

    by Bonobo_Unknown (925651) on Friday August 10, 2012 @03:45AM (#40942967)
    The point of the exercise is to replace the no longer needed flight software with software it can use to better perform it's tasks while on Mars.
  • "flight software"? (Score:1, Informative)

    by Barnett (550375) on Friday August 10, 2012 @03:46AM (#40942977) Homepage
    Why are they updating the "flight software"? I thought they were done with the flying bit?
  • by Anonymous Coward on Friday August 10, 2012 @03:50AM (#40943013)

    99% of brickings are the result of people doing stuff that the manufacturer did not intend for you to do, on devices where important design details were hidden for commercial reasons.

    This is unlikely (one would hope) to be the case here.

  • by kasperd (592156) on Friday August 10, 2012 @04:42AM (#40943329) Homepage Journal

    they have made mistakes in updating from Earth to Mars before.

    Sounds like it was not just a software update gone wrong but rather some mechanical problem which they were trying to work around. It was nothing like the usual bricking problem, where a firmware update overwrites code which is needed to perform future firmware updates.

    The rovers have several mechanisms to make it safer to update firmware remotely. But ultimately a combination of multiple unfortunate events can still lead to the loss of a rover. And one of those events may have been human error. From the description it sounds like mechanical problems with the solar panel, combined with two cases of human error in coordination of updates, another case of human error trying to correct the previous human errors, an unfortunate condition triggering a latent problem introduced by previous errors, and finally ending up in a position causing the battery to overheat, and loss of power being the ultimate reason it was impossible to adjust the previous mistakes.

  • by gagol (583737) on Friday August 10, 2012 @04:49AM (#40943357)
    That is probably why a team of 100 software engineers issues about 1000 commands per day for the rover. My guess is a lot of the work is triple checking everything before they upload an update. There is just no room for error in this situation.
  • Re:Failsafe (Score:5, Informative)

    by jkflying (2190798) on Friday August 10, 2012 @05:45AM (#40943603)

    The radiation this thing emits is NOTHING compared to the solar and cosmic radiation it would experience both in transit and on Mars. Putting everything in a metal box only helps so much, you still need specifically designed electronics which can handle the odd bit of radiation without dying. Even with a thick metal box you can't run an i7 on Mars, or not for very long at least. Your standard DDR3 isn't going to work either, or your standard EEPROM.

    The other thing to remember is that although this project is extremely important, they're still not going to throw more capabilities in than they need, because that is more that can go wrong. For a remote sensing platform, the amount of EEPROM isn't that important - you just need enough to hold your communication protocols, some basic reaction-to-obstacle algorithms and the motor control code. You aren't going to be pulling massive libraries in. The emphasis is on making it as simple as possible, so that there is less chance for bugs to creep in. Those extra MIPS will come in handy for the navigation and onboard image processing, and the flash for storing interesting info until you can upload, so those are what they have upgraded the most.

  • Re:Failsafe (Score:4, Informative)

    by serviscope_minor (664417) on Friday August 10, 2012 @08:44AM (#40944531) Journal

    Not really. That might have been true 10 years ago.

    No.

    All I'm saying is: you can bet the hardware is in a well-shielded heavy metal box, and today all it takes is about 1/4 of a cubic inch to squeeze in another GB of RAM or flash.

    I wonder why they didn't think about that. A nice thick, heavy metal box. Easy! Perhaps you should go and work for NASA?

    Let's ignore the earth's magnetosphere for the moment and make some massive assumptions.

    The pressure on the ground is about 10^5 Pa. That means there's 10^4 Kg of stuff above you to absorb radiation from space. That equates to 10m of water, 1.25m of steel ot about 90cm of lead. Quite a lot.

    Mars is about 1.5 Au from the sun, so receives about 0.4 times the radiation.cos

    The atmosphere is about 600Pa, by comparison.

    Radiation hardening is a very well established field. Using some degree of shielding is just one of the many techniques in use. On Mars, it is simply not enough on its own.

    It is very, very difficult to make a rad-hard processor, and then very thoroughly test it. Yo can't just keep shrinking the feature size, because is it goes down, the effect of radiation increases. Not only that but as the amount of crystal per transistor shrinks, the chance of unrecoverable lattice damage increases, due to the lack of redundancy.

    There are faster Rad-hardened DSPs, but those are, well, DSPs and only actually really fast for DSP like tasks.

    There also are almost certainly faster ones available now. But it's been in transit for a year, and they certainly weren't building it with a brand-new untested processor for which thay had to write all the software on the way after they launched it.

    So, given the constraints, it's a pretty great CPU to have on board.

  • by necro81 (917438) on Friday August 10, 2012 @08:57AM (#40944629) Journal
    In some cases, the software loaded on the device is not suited to the task the engineers want it to do. TFA mentions that the software on the device now is geared towards interplanetary cruise, EDL, and some very basic on-the-surface tasks. If they actually want the rover to do what they've sent it there to do, they need to perform the upgrade. Why not have the entire suite of mission software on the rover when it launches? Perhaps they hadn't gotten around to coding/testing the on-the-surface software yet. Probably the limiting factor is the program storage space on the rover. According to this JPL website [nasa.gov]:

    The computer contains special memory to tolerate the extreme radiation environment from space and to safeguard against power-off cycles so the programs and data will remain and will not accidentally erase when the rover shuts down at night. On-board memory includes 256MB of DRAM and 2 GB of Flash Memory both with error detection and correction and 256kB of EEPROM

    Think you'd be able to code everything the rover is ever meant to do, in a single unchanging program image, into just a few hundred kB?

    In other cases, upgraded software provides new capabilities that weren't envisioned during the original design. Spirit and Opportunity, for instance, were given lots of new capabilities over their mission life: like the ability to autonomously navigate based on Simultaneous Locating And Mapping (SLAM [wikipedia.org]) using the various cameras. These are capabilities that were just in development in academia when the rovers were originally programmed, but became proven during the MER mission. As a result of having that autonomous navigation capability, Spirit and Opportunity were able to travel much further distances than they would have if every single wheel revolution needed to be commanded from Earth.

  • by fisted (2295862) on Friday August 10, 2012 @10:00AM (#40945319)
    It's not a linear relationship since you need additional propellant to move the additional propellant you needed for the extra payload
  • by Frans Faase (648933) on Friday August 10, 2012 @10:14AM (#40945499) Homepage
    If you would inform yourself, you would know that we are not talking about a general PC with 4Gbytes of memory here, but about a much smaller (but reliable and radiation hardend) PowerPC compatible system with limited RAM. The reason that they planned this update is because they want to remove the flight software for the trip to mars and replace it by software needed to drive and control the rover. It is true that they spend improving the software during the time that the spacecraft was flying to mars. That would be more than logical to do. Please note that the software for the Spirit and Opportunity rover also have been updated several times. It would not surprise me, that when they know the Curiosity Rover better, they will perform another software update.
  • by jpmorgan (517966) on Friday August 10, 2012 @05:24PM (#40951667) Homepage

    No, it follows from the Tsiolkovsky rocket equation, and it is linear. The amount of fuel required is exponential in the delta-V required, but linear in the payload mass. m_1 = m_0 e^{- \Delta v / v_e}

"Marriage is low down, but you spend the rest of your life paying for it." -- Baskins

Working...