Become a fan of Slashdot on Facebook

 


Forgot your password?
Close
typodupeerror
×
Space Earth Technology

Galileo Satellites Are Experiencing Multiple Clock Failures (bbc.com) 195

Posted by BeauHD from the on-the-button dept.
elgatozorbas writes: According to a BBC article, the onboard atomic clocks that drive the satellite-navigation signals on Europe's Galileo network have been failing at an alarming rate. From the report: "Across the 18 satellites now in orbit, nine clocks have stopped operating. Three are traditional rubidium devices; six are the more precise hydrogen maser instruments that were designed to give Galileo superior performance to the American GPS network. Each Galileo satellite carries two rubidium and two hydrogen maser clocks. The multiple installation enables a satellite to keep working after an initial failure. All 18 spacecraft currently in space continue to operate, but one of them is now down to just two clocks. Most of the maser failures (5) have occurred on the satellites that were originally sent into orbit to validate the system, whereas all three rubidium stoppages are on the spacecraft that were subsequently launched to fill out the network. Esa staff at its technical centre, ESTEC, in the Netherlands are trying to isolate the cause the of failures - with the assistance of the clock (Spectratime of Switzerland) and satellite manufacturers (Airbus and Thales Alenia Space; OHB and SSTL). It is understood engineers have managed to restart another hydrogen clock that had stopped. It appears the rubidium failures 'all seem to have a consistent signature, linked to probable short circuits, and possibly a particular test procedure performed on the ground.'"
This discussion has been archived. No new comments can be posted.

Galileo Satellites Are Experiencing Multiple Clock Failures More

Galileo Satellites Are Experiencing Multiple Clock Failures

Comments Filter:

  • " It appears the rubidium failures 'all seem to have a consistent signature, linked to probable short circuits, and possibly a particular test procedure performed on the ground."

    Dependent upon what the what the test was designed to do, it either passed or failed. It's probably best not to test it again.

    • Re:Just a guess.. (Score:5, Funny)

      by I'm New Around Here ( 1154723 ) on Friday January 20, 2017 @02:21AM (#53701605)

      Quality Test #87: Can you bend the circuit card more than 10 degrees? If so, to what degree?

      • Re: (Score:3, Funny)

        by davester666 ( 731373 )

        Answer: Each time, I can bend it a little more. I ended the test when I reached 180 degrees. I was unable to go past 180 degrees due to a physical limitation.

    • Re: (Score:2)

      by klui ( 457783 )

      My swag is solder whiskers from RoHS directives. Anyone familiar with these devices that know if they're on the list of exemptions?

      • Re: (Score:2, Informative)

        by K. S. Kyosuke ( 729550 )
        Why would RoHS directives apply to spaceflight? That would be silly.

        • Why would RoHS directives apply to spaceflight? That would be silly.

          Yes it would. Of course, RoHS is silly to begin with. Even more so with more and more electronics being recycled.

          Years ago, I came across an old soldering booklet put out by IIRC Kester. Of interest was the contaminants section, with failures and defects. Many of the defects shown were by the presence of the components of modern day lead free solder.

          • Re: (Score:3)

            by Megol ( 3135005 )

            Bullshit! RoHS isn't silly, it wouldn't apply to spacecraft and you "bible" doesn't negate the actual research that is done on solder alloys after it was printed!

            You obviously don't understand alloys and that proportions of different components plus the choice of the components makes a huge difference. You don't understand the reasoning (and research) behind the RoHS and you think an old book triumphs modern research...

            It is easy to list things that changes the properties of solders, one example is that bis

            • Bullshit! RoHS isn't silly, it wouldn't apply to spacecraft and you "bible" doesn't negate the actual research that is done on solder alloys after it was printed!

              You obviously don't understand alloys and that proportions of different components plus the choice of the components makes a huge difference. You don't understand the reasoning (and research) behind the RoHS and you think an old book triumphs modern research...

              It is easy to list things that changes the properties of solders, one example is that bismuth shouldn't be used in standard leaded solder as it can drastically reduce melt temperatures, in a lead free formulation there are not such problems (though an excess of bismuth will make joints fragile).

              Bullshit! RoHS isn't silly, it wouldn't apply to spacecraft and you "bible" doesn't negate the actual research that is done on solder alloys after it was printed!

              Chill out anger person. The pamphlet showed photograps and photomicrographs of various solders. Lead free solder is not something just thought up then designed by teams of scientists after tin/lead was declared bad. Analysis has gone on for many decades, as pointed out in a cite below. This thing had dozens of images, and the lead free solders of the time looked remarkably like lead free solder of today.

              You obviously don't understand alloys and that proportions of different components plus the choice of the components makes a huge difference. You don't understand the reasoning (and research) behind the RoHS and you think an old book triumphs modern research...

              Oh anger person, I think maybe less coffee or working out whatever has caused your rage might be in order

      • +1 Came here to post this.

      • Lots of stuff is exempt from RoHS. It mostly applies to things you're selling on the open market, i.e. products. You can still buy full-fat lead solder perfectly legally.

        I'm actual rather partial to the tin/lead/silver blend for rework. It's quite a bit more expensive than normal eutectic solder, but melts at a lower temperature, which is really handy.

        • Lots of stuff is exempt from RoHS. It mostly applies to things you're selling on the open market, i.e. products. You can still buy full-fat lead solder perfectly legally.

          I'm actual rather partial to the tin/lead/silver blend for rework. It's quite a bit more expensive than normal eutectic solder, but melts at a lower temperature, which is really handy.

          I have multiple types of solder, modern substandard lead free solder, and Tin lead which I use for most things, depending on what the original solder was. Plus a number of silver containing We don't want to mix the types for certain. It is amazing the number of different compositions we have to choose from. I even worked on a couple ancient Tektronix scopes that used special solder that they included inside the case - the components were soldered onto ceramic bars with metal plated notches. You were screwe

          • We don't want to mix the types for certain.

            A cretain amount of mixing is OK. Mixing standard RoHS and Sn/Pb is fine, since tin and lead are completely miscable. Not sure about the silver one, but I've reworked SnPb and RoHS with it (it's designed for that) no problem.

            I wouldn't go any further and mix more exotic types though. There are some awfully exotic ones there.

            You were screwed if you used any other solder.

            Screwed how? Was it calibrated for the thermocouple effect with that solder or something?

            But the

            • Screwed how? Was it calibrated for the thermocouple effect with that solder or something?

              No, what happened was that the lead in regular solder would dissolve the metal plated onto the notch, and it would destroy the connection point. Here is the interior of one of those old scopes http://www.barrytech.com/tektr... [barrytech.com] You can see the ceramic strips around the middle and top of the scope with the components mounted in between. Seriously weird stuff.

              • Oh I see. How amazing! Any idea what the composition of the solder or contact point was? I've never worked on anything remotely like that.

                • Oh I see. How amazing! Any idea what the composition of the solder or contact point was? I've never worked on anything remotely like that.

                  They wanted us to use a solder with 3 percent silver. Sn62Pb36Ag2 The silver was to help against dissolution of the silver from the contact, and had a little bit lower of a melting point.

                  The process to make the strips was probably similar to that used for plated through holes. There must have been some reason that they used silver instead of copper however, I suspect it is mechanical, since the plated area isn't annular, and the stresses will be different. To my surprise - silver ceramic terminals are s

                  • How remarkable. I'm currently in electronics professionally a bit, but it's all modern, mainstream stuff. It's amazing to see something totally off the wall. The silver could be a conductivity thing? It's about 10% better than copper I think, though I'd be surprised if there was enough in that to make it worthwhile. I guess it might be adhesion to the ceramic too, or possibly they didn't develop copper plating techniques as well back when the scope was made.

                    I'd never heard of the problem of Sn/Pb dissolvin

                    • It's amazing to see something totally off the wall. The silver could be a conductivity thing? It's about 10% better than copper I think,

                      You think that you can get better than 10% cross-sectional area consistency when soldering something by hand? Without having to do an individual test on every component made, and re-work on ... well, rework on any of them would probably destroy the cost saving from whatever solution your peculiar solder was trying to achieve. There's a reason that, for example, you build vo

          • Re: (Score:2)

            by Agripa ( 139780 )

            I even worked on a couple ancient Tektronix scopes that used special solder that they included inside the case - the components were soldered onto ceramic bars with metal plated notches. You were screwed if you used any other solder.

            The situation was not quite that bad. Tektronix included a small spool of the silver solder they used for assembly because normal tin/lead solder can dissolve enough of the silver off of the ceramic strips to cause the solder joint to separate.

      • Their is a specific exemption Spacecraft in the RoHS directive.

      • Re: (Score:2, Interesting)

        by Anonymous Coward

        Actually, surprisingly enough. RoHS does not apply to things manufactured within the EU, just items imported into the EU.
        It is more a protectionist regulation than an environmental one.
        The Chinese now have a much more restrictive RoHS policy than the EU does on paper

      • IIRC Aerospace gets an exemption.

        The problem is that having both leaded and lead-free inventory is a PITA. A small ammount of lead contamination leaking into stuff that is supposed to be lead-free can lead to expensive problems both legally and technically.

        The result is lots of stuff that doesn't legally need to be lead-free nevertheless gets made on lead-free processes. Even if the final assembly is done with leaded solder it is very likely that things like component surface platings will be done with lead

      • Re: (Score:3)

        by Thud457 ( 234763 )
        "Tin Whiskers" is Optimus Prime's cat's name.

      • Re: (Score:2)

        by Megol ( 3135005 )

        You can't be serious?!? First whiskers is much less of a problem than those ranting about non-leaded solder, second it is possible to use techniques to reduce* the risk of whiskers etc. that can be done for some extra expense and third critical infrastructure etc. like spacecraft are exempt from the RoHs demands!

        (* using leaded solder also only reduce risks of whisker formation)

    • Re:Just a guess.. (Score:4, Funny)

      by arglebargle_xiv ( 2212710 ) on Friday January 20, 2017 @04:20AM (#53701953)
      These failures would never have happened with an H5 chronometer. Maybe they should launch with one of those as an additional backup.

      • Had to search for what that was - the Board of Longitude really did screw him over, didn't they.

      • Like others I had to look that up but the makers name sounded familar and then I got the this text on wiki [wikipedia.org]:

        One of the controversial claims of his last years was that of being able to build a land clock more accurate than any competing design. Specifically, he claimed to have designed a clock capable of keeping accurate to within one second over a span of 100 days.

        And I remembered reading about it when someone did finally build his clock and it turned out he was right.

    • Re:Just a guess.. (Score:5, Interesting)

      by tomhath ( 637240 ) on Friday January 20, 2017 @07:47AM (#53702461)
      NASA had a similar problem years ago. They were testing the component by exposing it to voltage spikes. Passing the test also damaged the component.

  • CIA inside job (Score:3, Interesting)

    by Anonymous Coward on Friday January 20, 2017 @02:13AM (#53701579)
    To make sure there is no concurrence from another global measuring system. CT ? Possibly. But is that far fetched ?

    • Re:CIA inside job (Score:5, Insightful)

      by anti-pop-frustration ( 814358 ) on Friday January 20, 2017 @03:07AM (#53701755) Journal
      Well I couldn't help thinking about the Iranian civilian program and the 'alarming rate of failure' of their centrifuges.

      Are the contractors who worked on Galileo going to discover some kind of Stuxnet variant on their network?

      Technically, it's possible. Thanks to work of security researchers we know it happened before, Stuxnet is well-documented. And thanks to Edward Snowden and the journalists who reported on the documents he leaked, we know the NSA/TAO [wikipedia.org] does in fact hack allies.

      I wish these kind of doubts could be instantly discarded as conspiracy theories, unfortunately, this is not the world we live in. The most technically capable nations (USA/China/Russia/Isreal) have made the choice of using hacking as a weapon rather than helping secure the systems used by their citizens.
      • Replying to myself but yes, 'Israel' not 'Isreal'.

        Time for that second cup of coffee....

        • Or the ESA just fails at engineering. For example they've done many Mars missions and none has been successful, and for what purpose would the CIA sabotage that?


      • As soon as I read the subject I thought this may be a stuxnet-like derivative. -which may turn out to be wrong...

        The reality is that the hacking tools available are far scarier than we have yet found proof of if you believe the reactions of some agency leakers about Stuxnet. (how they address the notion as a minor part of a far more sophisticate project)

        If it didn't happen already, electricity itself can be used as an attack vector to infiltrate systems. I don't mean shutting the power down I mean using

      • Re:CIA inside job (Score:5, Interesting)

        by enriquevagu ( 1026480 ) on Friday January 20, 2017 @05:25AM (#53702119)

        I do not work for Galileo, but I know some people involved in the Project.

        Interestingly (and unfortunately) the entities most interested in a failure and subsequent delay are neither millitary enemies nor allies (for whatever "ally" means, when you consider hacking your allies). The largest interest in a project delay comes from the many corporations directly involved in its development. This project has been feeding many millios of taxpayer euros during many years to a lot of European tech companies, and the cost uprising has been actually benefitial for them because there were no substantial economic sanctions from these delays (probably an example of a wrongly managed project). Successfully completing the project would kill this guaranteed job & income. I do not claim that delays and increased costs have been intentional, but they have been definitely benefitial for them.

        • ESA contracts are fixed price. Even if there are no penalties, you do not gain any benefit by being late: actually, you do lose money as you spend more for the same payout.

      • Proof it's NOT a CIA inside job (Score:4, Insightful)

        by cmseagle ( 1195671 ) on Friday January 20, 2017 @08:54AM (#53702765)
        The proof that these clock failures aren't the result of some CIA meddling is that they haven't had any actual impact. A foreign government would want to take enough of these clocks offline to make the constellation fail, and they would want to make this happen at a time of their choosing. Ho do a few sporadic clock failures, some of which have already been fixed, benefit anyone?

    • Re:CIA inside job (Score:5, Informative)

      by GuB-42 ( 2483988 ) on Friday January 20, 2017 @05:07AM (#53702055)

      GLONASS and Beidou work fine.
      I suspect that these would be the most likely targets should the CIA want to disable competing systems. Europe is supposedly an ally, Russia and China, not so much.

      • And keeping the Galileo military-grade GPS an unreliable entity will insure that GLONASS and Beidou work fine and used exclusively until the USG sends them the back-doored kill signal at the opportune time; or at least that's the way I would do it if I were a Spy Movie script-writer.

  • It's aliens. And if they like Donald Trump I'd say that's an asset, not a liability.

    • has to be aliens. their fiendish plan is to destroy the world's clocks, and then invade when we all sleep in.

  • They are using lead solder in these things I hope? If not, they could be shorting out because of tin whiskers. NASA even has a site devoted to this : https://nepp.nasa.gov/whisker/... [nasa.gov]

    • Seeing as a lot of space equipment gets built in the EU that doesn't exhibit these failures, whatever went wrong in this particular series of hardware units quite likely has nothing to do with the solder.

    • Solder with lead is *far* less prone to tin whiskers than lead-free solder - which is mostly tin. Most aerospace applications have special dispensation from the RoHS rules concerning lead because it's a critical component in eliminating tin whiskers.

      • Re:Tin Whiskers? (Score:4, Informative)

        by AmiMoJo ( 196126 ) on Friday January 20, 2017 @03:39AM (#53701839) Homepage Journal

        Since the Indians are not experiencing the same failure rate on identical hardware, it seems likely that it's something about the environment in the EU's satellites that is causing the problem. Maybe power supply issues, temperature control issues, or vibration issues.

        As you say, lots of things (like test equipment and of course aerospace) are exempt from RoHS so it's unlikely to be that. They will have used the most suitable materials.

    • Pure tin solder is prohibited on most space programs.

  • Not alarming (Score:4, Insightful)

    by hackwrench ( 573697 ) <hackwrench@hotmail.com> on Friday January 20, 2017 @02:38AM (#53701665) Homepage Journal
    The satellites have backup clocks for the very reason that something like this would happen. It just wasn't expected to happen on this scale, but despite that all satellites apparently have at least one rubidium and one hydrogen clock running. They have successfully gotten one hydrogen clock that wasn't running to run again, and there is apparently no reason to expect either way that the other clocks will or will not get restarted. Somebody in earlier comments mentioned a test that was run that may have induced the failure, so it is a bad test. Add it to the things to not do in the future. But the article seemed a bit light on what the things that can be done are, or what got that one clock working.

    • Re:Not alarming (Score:5, Funny)

      by Hognoxious ( 631665 ) on Friday January 20, 2017 @02:54AM (#53701711) Homepage Journal

      I blame systemd.

    • Re: (Score:1)

      by Anonymous Coward

      4 clocks can only tolerate 1 failure (if you don't understand this, read https://en.wikipedia.org/wiki/Byzantine_fault_tolerance#Early_solutions), so 2 failures means they don't know what time it is, which means they have failed to have accurate time keeping.

      • 4 clocks can only tolerate 1 failure

        That's only true if they fail simultaneously. Since they know one is bad, they can eliminate it before the next one fails. Now if one more fails, they are fine (the other two are still in agreement).

        If they have a way of externally checking the correctness then they can have three fail and still be operational, because they know which one is correct.

        • That's only true if they fail simultaneously. Since they know one is bad, they can eliminate it before the next one fails. Now if one more fails, they are fine (the other two are still in agreement).

          We don't know what happens in case of failure - does the clock give inaccurate time, or does it give no time at all. You could have a design where the clock either works or doesn't work, and you don't need to check them for disagreement at all.

          As I read it, they put four clocks on every satellite because these clocks break sometimes, and having for maximises the chances of having at least one that is working.

      • Re: (Score:2, Informative)

        by Anonymous Coward

        This is not true for frequency references, which are the function of the on-board clocks. And they are externally disciplined by the two ground stations, which have an array of Cesium and Active Hydrogen Maser clocks.

        The Galileo satellites can operate with only a single atomic clock, at reduced performance. They can operate with two clocks of different types at full performance.

    • Re: (Score:2)

      by jez9999 ( 618189 )

      That's like saying "only 3 of the 4 jet engines have blown up, we're still fine"...

    • The earliest satellite in the constellation was launched only 5 years ago and is less than half way through it's minimum design life.

      Yes failures have so far not knocked any satellites off line but only because of redundancy. There is no information as to if these failures are random or systematic. In either case the redundancy comes into play to prevent well controlled random errors most likely to occur as satellites approach end of life.

      A 12.5% failure rate in the core components of a system that has been

    • It is totally alarming. Redundancies are not supposed to kick in so early in the program. Such a failure rate is off scale compared to the reliability targets of a space equipment.

  • Wow (Score:3)

    by nospam007 ( 722110 ) * on Friday January 20, 2017 @02:50AM (#53701703)

    My local Mom and Pop newspaper told me that a couple of days ago, I guess it also has news for nerds, and faster than /.

  • Were the capacitors manufactured about ten years ago? ;-)

    • You wouldn't use electrolytic capacitors in space because the low pressure would cause the electrolyte to boil away fairly rapidly. Yes, even faster than normal.

      As Mark Watney discovers in The Martian when his laptop goes "phut" when he takes it outside of the dome, consumer electronics, even units approved for usage on NASA missions, aren't designed to withstand environments outside what's normal on Earth.

      • Your post brings to mind a question I was pondering the other day, which may have useful implications.

        Other than the battery, would low pressure damage the components typically used in smartphones? I'm thinking around 2 Kpa or so.

      • Were the capacitors manufactured about ten years ago? ;-)

        You wouldn't use electrolytic capacitors in space because ...

        Years ago there were some bad production runs of capacitors that found there way into various electronic devices. Blaming the capacitors for hardware failures became a bit overused, a sort of joke, at the time. :-)

  • Zoze Zwiss (Score:4, Funny)

    by codeButcher ( 223668 ) on Friday January 20, 2017 @03:33AM (#53701815)

    ... with the assistance of the clock (Spectratime of Switzerland) ... manufacturer

    Seems to be a big blow to the Swiss clock makers' reputation for accuracy and reliability. But rejoice. Maybe those Swiss watches will start to sell for more realistic prices...

    Disclaimer: This post is intended to produce whooshing.

    • Cuckoo clocks don't work well in the vacuum...

      • That's why the world needs cuckoo spacesuits!!!!

        (I believe the cuckoo clock is more a Swabian (https://en.wikipedia.org/wiki/Swabia) thing. Black Forest and all you know. Even if Swabian German and Swiss German sounds very much alike to non-Swiss non-Swabians.)

      • Well the bird doesn't cookoo in a vacuum but being in free-fall doesn't help a weight driven clock do anything. Most people would be perfectly fine with a bird that doesn't cookoo in a cookoo clock. Most of the people would be ecstatic if their cookoo clock didn't cookoo from when the kids went to bed to until the adults got their first cup of coffee!!

    • EU to the Swiss timekeeping sector: Folks, this is not what we meant by a grand complication!

  • . . . .is what sort of shirt were the clock engineers wearing ??

  • They should have used unobtanium clocks, FTFY.

  • Reversing the polarity of the neutron flow always seems to fix the TARDIS, should work here, too.

  • Time will tell. (Score:3)

    by UnixUnix ( 1149659 ) on Friday January 20, 2017 @09:10AM (#53702835) Homepage
    Years ago working on a NASA spacecraft a hitch developed and we couldn't read the system clock. We informed the mission manager. "-- Great, I'll just go tell the Director we can't do this mission because we don't know what time it is". Never lose your sense of humor.

Slashdot Top Deals

A physicist is an atom's way of knowing about atoms. -- George Wald

Close