Want to read Slashdot from your mobile device? Point it at m.slashdot.org and keep reading!


Forgot your password?
Math Transportation

How Data Science Powered the Search for MH370 (hpe.com) 133

"In the absence of physical evidence, scientists are employing powerful computational tools to attempt to solve the greatest aviation mystery of our time: the disappearance of flight MH370." Slashdot reader Esther Schindler shared this article from HPE Insights: Satellite communications provider Inmarsat announced it had found recorded signals in its archives that MH370 had sent for another six hours after it disappeared. The plane had been aloft and flying for that whole time -- but where had it gone? As Inmarsat scientists examined the signals, they saw that what they had was not data such as text messages or location information. Rather, the signals contained metadata: information about the signal itself. This was recorded as the satellite automatically contacted the plane's communications system every hour to see if it was still logged on. Bafflingly, whoever had taken the plane hadn't used the satcom system to communicate with the outside world, but had switched it off and then on again, leaving it able to exchange hourly "pings" with the satellite. Some of the metadata related to extremely subtle variations in the frequency of the signal. "We're talking about changes as big as one part in a billion," says Inmarsat scientist Chris Ashton.

Nobody had tried to use this kind of data to try to locate an airplane before. At first, Ashton's team didn't know if the attempt would work. But painstakingly, over the course of weeks, the team figured out how the movement of the plane, the orbital wobble of the satellite, and the electronics within the satcom system all interacted to create the data values that had been received. "We had to create the model from scratch," Ashton says. Their work revealed that the plane had flown into the remote southern Indian Ocean. They didn't know where exactly. But since there are no islands in that part of the world, it was impossible that anyone could have survived. For the first time in history, hundreds of people were declared legally dead based on mathematics alone.

Then mathematician Dr. Neil Gordon led a team from the Defense Science and Technology Group "to extract a path from a subset of the Inmarsat data called the Burst Timing Offset. This measured how quickly the aircraft responded each time the satellite pinged it, and was used to determine the distance between the satellite and the plane." They ultimately generate "a probabilistic 'heat map' of the plane's most likely resting places using a technique called Bayesian analysis. These calculations allowed the DSTG team to draw a box 400 miles long and 70 miles across, which contained about 90 percent of the total probability distribution.
This discussion has been archived. No new comments can be posted.

How Data Science Powered the Search for MH370

Comments Filter:
  • Very userful (Score:5, Insightful)

    by 110010001000 ( 697113 ) on Sunday October 29, 2017 @09:36PM (#55455409) Homepage Journal
    "Their work revealed that the plane had flown into the remote southern Indian Ocean. They didn't know where exactly."

    Amazing stuff.
    • Re:Very userful (Score:5, Interesting)

      by fermion ( 181285 ) on Sunday October 29, 2017 @09:44PM (#55455439) Homepage Journal
      The problem with fake science is that reporters state supposition as fact even when the data suggests it is not, and scientists hang onto to their 90% confidence level even when data suggests there is no such thing.

      This is an old story, and I think the analysis has shown it to be more conjecture that a real model. There is simply too many assumptions that need to be made, the variables are too complex.

      The analysis was interesting, and if it had worked would have a great feat of data analysis. But over a year ago drift analysis of the wreckage indicated that any probably location was nowhere near the IMSAT estimated locations.

      • Re:Very userful (Score:5, Insightful)

        by Solandri ( 704621 ) on Monday October 30, 2017 @12:21AM (#55455909)
        Drift analysis relies on the average value of year's worth of estimated currents and winds. The satellite data analysis relied on precisely known satellite positions and the speed of light (which is also precisely known). My money is on the satellite analysis being more accurate.

        But until the plane is actually found, there's no point arguing which is correct. We can't draw any conclusions until the plane is found. And it probably will never be found. Even if the search area indicated by the satellite signals is accurate, finding it there was always going to be a long shot (after the pingers stopped after 30 days). Given the relative sizes of the plane and the search area, finding a needle in a haystack is child's play by comparison. This is like trying to find a needle in field of haystacks.

        If they wanted to test the accuracy of their satellite analysis, they should be running it on planes on regular flights. They can calculate a plane's position at certain times based on similar satellite ping times, then check it against the plane's actual flight path. Do it enough times and you can figure out just how accurate the methodology is.
        • by Jesus H Rolle ( 4603733 ) on Monday October 30, 2017 @04:26AM (#55456277)

          This is like trying to find a needle in field of haystacks.

          Use a really big magnet.

          Technology makes things easier.

        • If they wanted to test the accuracy of their satellite analysis, they should be running it on planes on regular flights. They can calculate a plane's position at certain times based on similar satellite ping times, then check it against the plane's actual flight path. Do it enough times and you can figure out just how accurate the methodology is.

          You shouldn't need to test the satellite's accuracy until you ensure that you have an accurate way to find the location. In other words, what you should be questioning is the accuracy of data analysis (look at the quote from TFA below). The satellite collected only the time difference when it pinged the plane. That would give you a uniformed error distance (if the data is inaccurate) each ping. Then you should calculate for the satellite's location when it pinged and received the signal. Sadly, these inform

    • I have attended three long-form talks by Dr Neil Gordon, and some others involved. He is always the most compelling speaker. I have followed up in one-on-one discussions, at least twice. This effort has been a hugely consuming effort for most, and there is an 'answer' to where MH370 is: It is MOST likely to be in the next-most-probable statistical area, currently to the north-east of the last-most-likely area. Read the 'Full' report and get the picture. https://www.atsb.gov.au/public... [atsb.gov.au]
    • I want to know if the team ran any baseline data. Can the use this technique to locate a plane that isn't lost? There might be some surprises in that result.
  • Well... (Score:5, Informative)

    by hcs_$reboot ( 1536101 ) on Sunday October 29, 2017 @09:37PM (#55455419)
    all that science worked well!
  • by QuadEddie ( 459328 ) on Sunday October 29, 2017 @09:46PM (#55455451)
    If Shia Labeouf were on the plane, 4chan could have found it in less than 24 hours.
    • Re: (Score:2, Interesting)

      by Anonymous Coward

      I think /pol/ is getting bored of ruining Shia's life. A week ago he put the flag on top of some music hall in France and all they've done so far is launch a flaming drone at it in a half-hearted attempt to set it on fire.

      • I think it's less of a challenge. They know exactly where it is. They could probably get it with a little effort, but why bother? There's not a lot of bragging rights for that.
  • by Anonymous Coward

    Perhaps you shouldn't gloat about how great your Data Science is, if you haven't actually achieved your one goal.

  • by eclectro ( 227083 ) on Sunday October 29, 2017 @10:33PM (#55455619)

    Everybody would not be needing to comb over minute pieces of data and vast esoteric computations if service providers had behaved better.

    The satellite service was capable of gathering the gps data [extremetech.com] from the plane instantaneously and throughout its flight path. But the satellite company was charging for it, and Malaysian authorities did not want to pay for it presumably because it cost too much.

    If the gps location service had been available for this flight, one can't help but wonder if there was a possible intervention that could have been undertaken when the plane would have been discovered wildly off course, and even though it appears the crash was not survivable, the quick crash site discovery and possible apprehension of possible criminals involved (if there are any).

    As it is, everybody was chintzy all the way around at the expense of the safety of the flying public.

    • As it is, everybody was chintzy all the way around at the expense of the safety of the flying public.

      No lie. What year is it? It's really not acceptable that we don't know where all airlines are, all the time.

    • GPS data isn't available throughout an entire flight and isn't completely accurate. And it doesn't get reported to authorities.
    • As it is, everybody was chintzy all the way around at the expense of the safety of the flying public.

      At some point I would hope that people stop throwing money at the already ludicrously safe experience of travelling via aircraft and instead spend those savings on maybe making my car drive to the airport safer given I'm far more likely to die there on the road than in a aircraft crash.

      And that goes double, triple and then some for driving or just living in a city like Kuala Lumpur.

  • Where were the editors? HPE's not Slashdot's.

    Then mathematician Dr. Neil Gordon led a team from the Defence Science and Technology Group...

    Given that this is a proper noun the article's spelling is incorrect even in the US. The rest of the world is constantly making allowance for US spelling but it seems that the favour is not being returned.

  • While the theoretical model has been carefully studied (See for example, http://epubs.siam.org/doi/pdf/... [siam.org] ), I'm not aware if any entity ever validated the model by actually flying an aircraft along one of the potential flight paths and comparing the ping times and doppler offsets from the theoretical model with an actual flight path. Does anyone know if that was ever done? Second best would be to compare the metadata from some other known flight with that flight's actual path.

    • by Anonymous Coward

      They used the data from planes in the air at the time of all of this to compare the metadata. That is why they feel they have it correct. They were able to reproduce the results across a variety of aircraft and figure out arcs for them, using this method.

  • by kenwd0elq ( 985465 ) <kenwd0elq@engineer.com> on Monday October 30, 2017 @12:39AM (#55455941)

    I'd be a whole lot more impressed about the performance of "Big Data" if the submarine ROVs had found any trace of the aircraft. Right now, what they have is a big fat NOTHING. Some control surfaces washed up on islands a thousand miles away are not indicative of the performance of any sort of data analysis.

    • by Anonymous Coward

      Given that the black box has enough battery to ping away for 30 days, a nuclear sub would take less than a week to get there and pin point it exactly. Maybe not get to the depth of the sunken plane, but would be able to sit right there above it and say it is exactly below us and have the coordinates.
      Notice that no media has mentioned anything about manned submarines finding it. No sub found it as it was not there to be found. There are military subs all over the world and within 30 days, one would have foun

      • Re: (Score:3, Informative)

        by Anonymous Coward

        ... because nuclear submarines, which are expensive and built to 24/7 stand alert, are just willy nillly sent off looking for a missing civil airliner instead of hunting enemy submarines and putting nuclear missiles off the enemy's coast.

        Oh by the way, water affects sonar the same way for sonobouys, dippers, towed sonar arays and submarines. You still have to be relatively close.


    • Now that Big Data works, maybe we should build better underwater detection equipment?
    • I think back to ancients of Samos figuring out the circumference and diameter of the Earth, and the size of the moon in relation to the Earth, and I must agree with you! Blathering on, without evidence, is simply an advertisement pitch. On the other hand, I go with my previous analyses of following the money, and the facts that the Blackstone Group (major owner) and the Carlyle Group (minor owner) owned Freescale Semiconductors, whose foundries were responsible for the FPGAs aboard that aircraft, which a
  • Solves nothing.

  • Umm, hold up, folks.

    Don't declare victory until the airframe is found.


  • by slashmydots ( 2189826 ) on Monday October 30, 2017 @02:08AM (#55456093)
    Anyone keep reading these headlines and thinking it's the new Intel chipset?
    • Huh? It's not?! I have been boasting my knowledge of the new intel chipset discovery to my colleagues all morning.

  • Too early (Score:5, Insightful)

    by mseeger ( 40923 ) on Monday October 30, 2017 @03:17AM (#55456187)

    This post comes too early. First find the damn thing, then boast about how this or that method helped finding it.

  • Dead by mathematics? (Score:4, Interesting)

    by thegarbz ( 1787294 ) on Monday October 30, 2017 @05:28AM (#55456405)

    How about declared dead after they couldn't find the plane for over 9 months and no one had established contact?

    This article is a load of crap. It's an example of how these data models have failed to achieve anything useful. Firstly after almost 2 years they announced that they were looking in the wrong place: http://www.aljazeera.com/news/... [aljazeera.com] and also that they were confident that after spending $200m the plane was not in the search area they established. https://www.theguardian.com/wo... [theguardian.com]

    Good work big data!

  • by gweihir ( 88907 ) on Monday October 30, 2017 @07:27AM (#55456591)

    That is merely a bit more special RF signal analysis engineering and not so much different from other radio-location tasks, although you usually have more data. Calling this "Data Science" is nonsense.

  • by Anonymous Coward

    There was a similar aircraft where the oxygen bottle for the pilots broke its valve top and shot out the side of the aircraft like a rocket, putting a decent-sized hole in the fuselage (A foot or two in diameter). Turning off all the electronics breakers is consistent with fighting an electrical fire. So the theory would be - the pilots' oxygen bottle, in the radio bay below them, starts leaking, producing an oxygen-rich environment underneath them in the electrical hold - an electrical spark then results

  • I'm so impressed. With all this sophisticated data science they know exactly where it is. The only problem is that they haven't found the airplane. They know where it is. They really do. It's just that they haven't found it. Another :"WIN" for science!

  • Having worked a lot in aviation (even specifically with Inmarsat) I really doubt these results, even beyond the statement that they created this model from scratch (and thus unverified/untested). Navigation isn't even very accurate in planes, so much so that you have things like NAT tracks ( https://en.wikipedia.org/wiki/... [wikipedia.org] ) which establish fixed routes across the Atlantic where there is little radar coverage, GPS is not complete ( see RAIM https://en.wikipedia.org/wiki/... [wikipedia.org] ) and often inaccurate. They ne

Top Ten Things Overheard At The ANSI C Draft Committee Meetings: (10) Sorry, but that's too useful.