Follow Slashdot blog updates by subscribing to our blog RSS feed

 



Forgot your password?
typodupeerror
Check out the new SourceForge HTML5 internet speed test! No Flash necessary and runs on all devices. ×
Biotech Crime Patents The Courts Technology

Source Code On Trial In DNA Matching Case (post-gazette.com) 117

An anonymous reader writes: While computer analysis by other programs was inconclusive in matching DNA evidence to a suspect, one program, TrueAllele, gave a match. As reported in the Pittsburgh Post-Gazette, an expert witness for the defense wants access to the 170,000 lines of source code to determine whether the match is scientifically valid. Not surprisingly, the software creator is resisting. From the article: "TrueAllele, created by Dr. Perlin and in its current version since 2009, is the only computer software system of its kind that interprets DNA evidence using a statistical model. It can single out individuals in a complex DNA mixture by determining how much more probable a match is versus mere coincidence. Complex mixtures can involve multiple people, as well as degraded or small DNA samples. ... Although the technology is patented, the source code itself is not disclosed by any patent and cannot be derived from any publicly disclosed source. The source code has never been revealed, he said, and it would cause irreparable harm to the company if it were. In his declaration, Dr. Perlin said that reading the source code is unnecessary to validate the program, and that a review could be done in his office or online."
This discussion has been archived. No new comments can be posted.

Source Code On Trial In DNA Matching Case

Comments Filter:
  • Wrong industry? (Score:5, Insightful)

    by Type44Q ( 1233630 ) on Saturday October 10, 2015 @10:39AM (#50698851)
    Guess if he didn't want his code audited, this guy shouldn't have marketed his software to this particular industry. Dumbass.
    • Re: (Score:2, Interesting)

      by retroworks ( 652802 )
      Huh. Interesting comment, and on point. The right to defend oneself legally seems to trump copyright law. One the other hand, if I wanted to see someone's copyrighted code, could I simply write bad code (producing a different result) and thereby get access to another programmer's code in any court case? Say for example I want to see automobile code, I find a vehicular homicide case, show a result on my program where the driver was not at fault because automobile code was badly written, and demand to see
      • Re: (Score:3, Informative)

        by Eunuchswear ( 210685 )

        Nothing to do with copyright law.

      • by tkrotchko ( 124118 ) on Saturday October 10, 2015 @11:17AM (#50699025) Homepage

        As the other poster said, he's not saying it's a patent or copyright issue, he's effectively saying it's a trade secret.

        So the issue is really pretty clear isn't it? If he refuses to show his code to an expert witness and explain it, then the evidence can't be used.

        We'll see what the judge has to say.

        • by mspohr ( 589790 ) on Saturday October 10, 2015 @03:46PM (#50700103)

          Essentially this guy is going to jail based on a secret algorithm that can't be verified.
          I can't imagine how this could be legal.

          • by almechist ( 1366403 ) on Saturday October 10, 2015 @05:11PM (#50700407)

            Essentially this guy is going to jail based on a secret algorithm that can't be verified. I can't imagine how this could be legal.

            Actually, it's worse than that, he's facing a death sentence. It's inconceivable to me - but, sadly, unsurprising in this day and age - that someone might be legally executed based in part on the results of a proprietary algorithm that the defense is not allowed to examine. I can only hope the judge recognizes the seriousness of the situation. This case definitely bears watching

            • Really they should not be relying on him at all to prosecute.

              If other analysis systems can't match him well, the defense should be partying and ready to call it a day, regardless of how one particular system responds. That is easily reasonable doubt.

            • It seems like the software is accusing the defendant, and the defendant should have the right to face and question his accuser, which would mean reading the source code.
        • If he refuses to show his code to an expert witness and explain it, then the evidence can't be used.

          Not true...

          As I understand it, he should be able to get his program (or a modification of it) to produce as an output:
          - The computation of the probabilities
          - The data used to compute them, with annotation giving a trace back to its source.
          - The assumptions behind the computation.

          The issue of HOW IT IDENTIFIED this individual is separate from WHAT IT IDENTIFIED ABOUT HIM. The former i

          • Re:Not true. (Score:4, Insightful)

            by sjames ( 1099 ) on Sunday October 11, 2015 @02:48AM (#50702519) Homepage Journal

            Honestly, the 50,000 foot view of the methodology sounds a bit dodgy to me. I would like to know what peer reviewed experiments have demonstrated that the methodologies in use can identify a single person out of a mix of DNA that actually owned the item. Were they replicated? Then there is a need to show that the software actually performed that methodology without error. Perhaps the prosecution would care to have a third party run the methodology by hand in a blind test?

            If those 2 sticking points cannot be satisfied, then the "evidence" is bunk.

          • by Agripa ( 139780 )

            Of course the defence is going to do their darndest to monkeywrench the prosecution, and threatening the tool builder with disclosure of his trade secrets is a good move tactically. It's up to the judge (and possibly the appeals judge) to call them on it if it's just an irrelevant thrash.

            As if the prosecution did not pick or encourage a testing method which would prevent cross examination.

      • by SLi ( 132609 ) on Saturday October 10, 2015 @11:18AM (#50699027)

        Well, not really. If the relevant facts are roughly as stated in the summary, it's indeed quite possible that the company will be forced to produce the source code or not rely on the evidence. However the only thing this means is that the defendant's paid experts get access to the source code under a strict protective order. They will then produce an expert report, which is the only thing anybody else will have access to, and even that may be sealed in whole or part if it would reveal, in the opinion of the judge (and often anyway unless the defendants object) significant trade secrets.

        I think the two most realistic reasons to oppose are the costs of production and the possible loss of reputation if the evidence due to the inevitable criticism by opposing experts.

        • You are right of course (and while a copyright may be issued prior to a license, it's a licensing and not copyright issue). But what if it is a class action suit, all accused parties in vehicular homicides demand access to auto computer codes for prosecution? I agree it's thin (would require cooperation of expensive defense lawyers with little direct benefit to the client), but as a supporter of RightToRepair I thought it was a question worth asking.
        • I think the two most realistic reasons to oppose are the costs of production and the possible loss of reputation if the evidence due to the inevitable criticism by opposing experts.

          The costs of producing the source code: So damned near $0 it doesn't bear mentioning. If you can build it, you can produce it.

          The costs of possible loss of reputation: can be solved by sealing part of the court records, if necessary. But nobody has a right to a certain reputation.

          • by HiThere ( 15173 )

            Actually, that's not always true. I've heard of companies that used software they only had in binary. I suppose you could turn that into assembler easily enough, though you might end up with some of your data being rendered as code.

            (The case I heard of was back in the 1970's and the programmer who originally built the software fixed it with binary patches, so the code didn't mean anything...but it had been lost anyway by this point.

            They used this software as a part of how they figured their profits, which

            • by msobkow ( 48369 )

              You write your own software in binary?

              I haven't seen that since the age of panel switches and lights for bootstrapping old, old, old computers...

              • by HiThere ( 15173 )

                No. But the guy who was maintaining the software originally wrote it in assembler, and then fixed bugs by doing binary patches. Not me, I never worked for the company, or met the guy who wrote the software. I understand the company was a shoe seller, but I don't even know whether it was a manufacturer or a vendor.

                Yes, however, this was on an OLD computer. But the software was kept as a deck of punched cards, not panel switches. (It's not THAT old.)

            • by sjames ( 1099 )

              If they only have it in binary, that would mean that they are falsely testifying to the validity of code they haven't examined.

        • by mspohr ( 589790 )

          The problem is that I have my expert and they have their expert... sounds like a standoff.
          If I am going to jail, I need to know that it's based on real science (public, audited, peer reviewed, verified) not some guy's secret algorithm.

          • Dr. Perlin created the method, the company and uses the software. There are no articles that independently validate the software. The software is a only one of its kind type item. In this case the artificial patent monopoly may work against Dr. Perlin and his company.

            From the article

            According to a court filing made by Dr. Perlin in the case, his company, Cybergenetics, “has invested millions of dollars over two decades to develop its TrueAllele system, the company’s flagship product. Althoug

      • Re: (Score:2, Insightful)

        by Anonymous Coward

        You don't get to use the code. It is opened for analysis only for few selected professionals and most certainly not shown to someone who has competing code. There is no problem with copyright here. The copyright is still with the one (company) who wrote the code.

      • This isn't about copyright, it is a patent in question. Which makes me think a bit. In a patent you have to describe in detail how the device works so that after the patent expires others can easily emulate it. Shouldn't that mean the source code itself is required to be contained in the patent?
    • by Layzej ( 1976930 )
      Best practice is to discard the minority report. No need to review the source code since there is probably no practical way to prove it correct anyway.
      • Re:Wrong industry? (Score:5, Interesting)

        by Layzej ( 1976930 ) on Saturday October 10, 2015 @11:07AM (#50698975)

        For instance, for any mission critical component NASA may have three different programs, each written in a different language and running on a unique platform. If at any time one of the programs gives an answer that is not consistent with the other two then the minority report is discarded and the other two are presumed to be correct. No need to halt the proceedings and debug at that point.

        In this case there is only one program that finds a match. It should be considered unreliable and discarded.

        • Not really in this case. This is not a situation where multiple programs do the same thing. Others match DNA identifiers to find a match where the one in question matches the statistical probably of identifiers being a match. Its kind of like the difference between determination of a pipeline diameter by measuring the pipeline verses measuring the flow rate and working back. So while the objective is the same, the approach is different enough to be separated from each other.

        • For instance, for any mission critical component NASA may have three different programs, each written in a different language and running on a unique platform.

          Nope, the record (which was the Shuttle's control system) is two different programs running on identical hardware. They weren't even identical programs, the first had all mission features, the second had just enough to reach orbit and to return from orbit to earth. But even that was highly unusual - the norm is two identical computers running ident

      • by pepty ( 1976012 )
        Not really. In this case the best practice for the defense is to consider any match, even from the majority, to be statistically flawed and ask that the full sequences be compared directly. When you are fishing in a database it makes sense to only look for a few datapoints. If you want to prove the match is a false positive there's no need to look at the software results again, these days you can send the trace evidence and the suspect's DNA to a company that does full sequencing of genomes and do the fu
        • That's not the limitation.

          Of course you can sequence every base pair of the suspect. The DNA evidence is limiting, the problem is the evidence.

          Just for example, pretend we know beyond a doubt that a killer and five other people used the same pen to sign in in a hotel. Let's pretend that the pen fell down a couple times and it was a rainy day (mud on the floor).

          There is a mixture of 5 people's DNA plus bacterial DNA of 100 species and the bacterial enzymes which are busy degrading the DNA.

          This mess is
          • by sjames ( 1099 )

            That sounds to me like an inconclusive result, not something you should use to send someone to death.

            • Well, I agree that for a conviction that this is probably insufficient alone, especially when you consider that, the other 5 people, even if you know who they are, may refuse to have their entire genome sequenced as a comparator (I would refuse and I studied genetics), so the comparisons are even weaker.

              That is why this computer program needs to be examined even more thoroughly than most people realize.

              Whether anyone should be put to death on the basis on any evidence, no matter how sound it is, is a ques
              • by sjames ( 1099 )

                It's even worse. The methodology that the software might or might not correctly implement is itself unproven.

                • You're absolutely correct, for a non-programmer like myself, most of us confuse the algorithm and the code implementing the algorithm and just call them the program (I know that's wrong but it's how I think).
    • by hey! ( 33014 )

      The source code shouldn't matter; it's the method used by the source code. If that method cannot be reproduced without the source code, then the output of the program is worthless. If it can be reproduced without the source code, then the output of the program may have value, if the method used stands up to scientific scrutiny.

      As it stands all the prosecution has amounts to a black box with a red and green light on top and a slot in the side into which a couple of samples are dropped. If the light subseq

      • The source code shouldn't matter; it's the method used by the source code.

        Okay, now go forth and prove that a stated method was used without referring to the code.

        As it stands all the prosecution has amounts to a black box with a red and green light on top and a slot in the side into which a couple of samples are dropped. If the light subsequently turns red, then the prosecutor wants the jury to believe the samples match. But they have no reason to believe that other than the prosecutor telling them to trust the box.

        Right, without a code analysis they have no way to know if the box contains anything of value.

        • by hey! ( 33014 )

          It doesn't matter what the code produces. You use an independently developed and open system to confirm that the code in question conforms to the method. Then it's a matter of showing the method is valid, which of course is the important question. Patents don't mean something accomplishes what its inventor purports it does.

    • Re:Wrong industry? (Score:5, Insightful)

      by alzoron ( 210577 ) on Saturday October 10, 2015 @11:37AM (#50699113) Journal

      Exactly right.

      This is basically the same as asking an expert witness how they determined that the defendant was involved in a crime and the witness refusing to answer the question because "It's a secret."

    • Guess if he didn't want his code audited, this guy shouldn't have marketed his software to this particular industry. Dumbass.

      Moreover, what if his source code was a sham, and in fact, it was a human, looking at the evidence via a microscope or other clairvoyant instrumentation that decides on the life/death of an individual.

      As a minimum, the source code should be shown, should be compiled and tested with the compiled version. Statistical sampling can be prone to definite errors, particularly if the sample sizes analyzed are too small. What were the sample sizes? Under 100, or under 2000? The former would be dangerous and probab

  • Seriously at what point is the general public going to stop accepting that bullshit lie?

    My cold fusion work only when I run the experiment in my shed with no one watch too ...

    • by gerddie ( 173963 )

      My cold fusion work only when I run the experiment in my shed with no one watch too ...

      Obviously, it's quantum physics, by monitoring the experiment you change the outcome ;)

  • Reasonable Doubt (Score:3, Insightful)

    by Anonymous Coward on Saturday October 10, 2015 @10:46AM (#50698883)

    From the perspective of the burden of proof placed on the Prosecution, they have to disclose how they arrived at this derived 'evidence' of a match via TrueAllele. Criminal justice can't be served using a "Black-Box" as an input.

    Disclose the software and its methods to a legally-sworn-to-secrecy-expert-witness, or toss the evidence as inadmissible.

    • by godrik ( 1287354 )

      I do not know the US legal system that much. But it seems that if you can not get a court-nominated expert witness to vouch for the result, the evidence should be discarded.
      Now, I would even prefer if an independent lab could reproduce that result.

    • by tomhath ( 637240 )

      They already have an expert witness - the author of the program. He is willing to testify how his program reached the conclusion it did. At some point you need to accept whether or not an expert is indeed an expert, otherwise you get into an infinite loop of "my expert needs to verify your expert's expertise"

      In this case the defense is on a fishing trip to find a bug or two in the code, which they will then use to discredit the entire program even if the bug has nothing to do with the conclusion.

      All that sa

      • His testimony simply saying 'well it works" isn't usable because of his vested interest.

        Unless someone else can duplicate the results, it's not really scientific data and shouldn't be admissible.

        Consider VW diesel engines. You can't trust software unless you can validate it.

      • by NoKaOi ( 1415755 )

        They already have an expert witness - the author of the program. He is willing to testify how his program reached the conclusion it did. At some point you need to accept whether or not an expert is indeed an expert, otherwise you get into an infinite loop of "my expert needs to verify your expert's expertise"

        Said expert is not impartial, for 2 reasons: 1. It's his company, so he has a major financial stake in testifying that his software is perfect. 2. The prosecution is effectively his customer, and the any good business person know the customer is always right.

        As somebody above stated, if the author of the software wasn't willing to submit to a code review, then he picked the wrong damn market. If your life and freedom were at stake, would you want to take the word of the author of the software, or would yo

      • by sjames ( 1099 )

        However, the defense DOES have a right to bring in their own expert witness. That witness has to be allowed to examine all of the evidence and methodology used by the prosecution's expert.

    • From the perspective of the burden of proof placed on the Prosecution, they have to disclose how they arrived at this derived 'evidence' of a match via TrueAllele.

      IMHO: Unless there is an issue with whether the database TrueAllele searched was obtained illegally (making any results of searching it for suspects "fruit of the poisoned tree"), they DON'T have to show how the match was found.

      They just have to show that the match IS a match. This can be done with the data involved in the match standing on its

  • Trust us (Score:3, Informative)

    by Anonymous Coward on Saturday October 10, 2015 @10:47AM (#50698891)

    Perhaps it's time for a more open process and open source code backing these types devices before their results are accepted as forensic evidence.

    CSI is a lie:
    http://www.theatlantic.com/politics/archive/2015/04/csi-is-a-lie/390897/

    Crime lab major errors:
    http://www.mprnews.org/story/2013/02/14/news/saint-paul-crime-lab-major-errors-found

    • Pretty eye opening. Worth the 5 minutes to scan through it.

    • by aix tom ( 902140 )

      And another example on how "DNA evidence" sometimes isn't:

      https://en.wikipedia.org/wiki/... [wikipedia.org]

    • by NoKaOi ( 1415755 )

      Perhaps it's time for a more open process and open source code backing these types devices before their results are accepted as forensic evidence.

      Agreed, and it doesn't even have to be free to be open source. When someone's freedom is at stake, the burden of proof is on the prosecution, which in these cases means the burden of proof is on the software to show that it works. How can they possibly show that it works, beyond a reasonable doubt, without code review?

  • Thrown it out (Score:2, Insightful)

    by Anonymous Coward

    Guess if we can't see the code that shows that the DNA sample is valid then thrown out the results, pretty simple.

    People need to learn if they are creating software that needs to hold up in court that they can't hide it. Trowing out the evidence would be the first step to make sure no one wants to pay for software that can't be audited in a investigation. Then no one will buy his software anymore and he can be happy that it is still safe from prying eyes.

  • by Eunuchswear ( 210685 ) on Saturday October 10, 2015 @11:00AM (#50698935) Journal

    If his method is patented he has no need to keep his code secret.

    Unless it's shit, of course.

    • Forcing the source code to be revealed might set a precedent that could be used against companies like Google or other cloud service providers. While TrueAllelle appears to be a standalone system rather than a cloud service, the company behind it could claim a similar "trade secret" defense against the petition.
      • I see two reasonable options here:
        1. reveal the source code. does this mean the court gets to force a business to reveal it? I don't think that's right.
        2. remove the evidence from the case.

        If TrueAllele sold their software to prosecution or a forensics department and are unwilling to provide source code, then that should be a civil case for misrepresenting their software as appropriate for working with evidence.

        (My statements assume the world is fair and that courts are interested in facts and truth. I real

  • If your going to use a computer to generate evidence then yes you must allow the defence to look at the technique that means source code. You must never be required to merely observe at somebody else's lab especially when there companys continued business relies on the test succeeding. Realy anything that another lab that is not associated to the first can not do should not be admissible.

    This gets even more important when the tests are destructive so it can only be done once. Validating the means used etc etc elsewise it becomes a black box to provide evidence against whoever they want.

  • by Anonymous Coward

    Have a third party selected by the court audit the code. There are tons of firms which have the expertise to do this, be it any of the big four or even security consultants, and a court selecting one would make it independent from both parties in the case.

  • You mean RAND() is not a perfectly valid methodology for determining guilt? Come on, he looks guilty.

  • by bfwebster ( 90513 ) on Saturday October 10, 2015 @11:45AM (#50699149) Homepage

    ...I think the defense has the better argument. I have used software tools (both third party and ones I have developed personally) to do source code comparisons and analysis, but they only serve to point me to likely areas of investigation; I have never directly reported and relied upon the output from one of my custom tools in my expert reports.

    A key aspect of expert testimony is that your analysis should, in theory, be repeatable by any other qualified expert using the same methodology (which needs to be spelled out in your report). If Perlin is relying directly upon his custom program for his conclusions, he needs to thoroughly expose his methodology -- which, in effect, means either allowing his source code to be reviewed or producing a detailed summary of his methodology that would allow someone else to reproduce it. Trying to claim trade secret status (which is what he's doing, in effect) for a expert methodology is an oxymoron.

  • by OrangeTide ( 124937 ) on Saturday October 10, 2015 @11:57AM (#50699205) Homepage Journal

    Then it is not science, and it should not be admissible as evidence. It doesn't really matter that it can't be reproduced because the software vendor won't share their techniques as they believe the software to be a trade secret. If it is not possible to confirm results, it's not science! Yes, TrueAllele is a toy and not only am I skeptical of anyone using it as the basis of their scientific research, use of TrueAllele om court ought to throw any conviction into question as well.

    • Not to mention the sheer audacity of the claim that "[TrueAllele] is the only computer software system of its kind that interprets DNA evidence using a statistical model."

      There's nothing unique or interesting about this guy's software except the specific application.

  • Although the technology is patented, the source code itself is not disclosed by any patent and cannot be derived from any publicly disclosed source.

    If the patent doesn't disclose the invention in sufficient detail [wikipedia.org] for it to be reproduced by someone skilled in the arts, is the patent valid?

  • by FrozenGeek ( 1219968 ) on Saturday October 10, 2015 @01:27PM (#50699563)
    So, if I understand this correctly, his product uses a very different methodology to match DNA samples than do his competitors. In this case, his product gave a completely different result from the results generated by his competitors. Has anyone done an independent, double-blind study comparing the results of his product and those of his competitors? And I'm not talking about a handful of tests. I'm talking about thousands, or tens of thousands, of tests.

    I can understand why the manufacturers don't want to do that. It may well show that, oh dear, the best product is wrong 5% of the time (not good). Or that product X is head and shoulders above the rest.

    But seriously, what industry-wide testing has been done? We're staking peoples' lives to the efficacy of this technology. How effective is it?

  • This has already been ruled on for traffic cameras. I think that is a pretty good precedent.
  • by tlambert ( 566799 ) on Saturday October 10, 2015 @04:18PM (#50700203)

    The main problem is finding a disposable expert.

    The people who originally clean-roomed the IBM BIOS for Compaq were split into two teams, with a Chinese wall between them: the analysis team, and the implementation team. The analysis team analyzed the IBM BIOS, wrote a specification, and then the implementation team implemented a BIOS to that specification. At which point the analysis team were effectively "burned", as in being forever barred from ever working on an implementation team in the future. They were highly paid for this, but they were disposable.

    As with clean-room engineering, this expert would not be permitted to work on any software covered by the trade secret in the future. In an expert witness situation, you might be able to get away with disposing of the expert, if all they did was witnessing, rather than actually coding in the field of expertise themselves. However, how likely is it that you can find someone like that who also qualifies as an expert?

    Further complication: Having testified (presumably in favor of the prosecution, in this case), would the expert witness be permitted to testify on similar goal programs in the future, given what the [now] knows about the process and techniques of the one they testify about today? Would exposure to multiple, competing trade secrets, damage their ability to perform an unbiased analysis, given what they knew from earlier experience? In general, I think you [as the defense] could argue that it, in fact, did damage their impartiality in their analysis.

  • by gweihir ( 88907 ) on Saturday October 10, 2015 @05:34PM (#50700473)

    The typical reason to keep code secret from everybody is because it is of abysmally bad quality or there are other severe problems hidden in there. Reasonable-quality code gets inspected and audited by 3rd parties all the time under NDA. In this particular case, it may also well be that the code does not do what its creator claims and the patent is bogus. If the expert finds this, the code becomes worthless and the creator may even become a target for litigation.

    • The typical reason to keep code secret from everybody is because it is of abysmally bad quality or there are other severe problems hidden in there.

      That's not the reason.

      A lot of code violates copyright, patents, and license agreements like the GPL. You would *not* believe what some of the ATI and nVidia code looks like, and you would *not* believe the number of USB keyboards running firmware that one manufacturer pretty much copied wholesale from another, and you would not believe the number of companies that sell "sanitized" open source software as proprietary code to third parties.

      • by gweihir ( 88907 )

        And code violating copyrights is not a "severe problem hidden in there"? I would think it is.

        • And code violating copyrights is not a "severe problem hidden in there"? I would think it is.

          It's not a "severe problem" for everyone, only for the copyright holder.

          I would only class something as a "severe problem" if it impacted the correct function of the software.

          Legal problems do not impact function.

          • by gweihir ( 88907 )

            You have a problem with language semantics. Obviously, the definition of "severe problem" you use here is something you dreamed up, and incompatible with general use.

            And, incidentally, if discovered, it becomes a severe problem for those that wrote and own the software and possible those that use it. Fro example, it could then become subject to criminal penalties (i.e. personal ones) to continue to use the software.

            • You have a problem with language semantics. Obviously, the definition of "severe problem" you use here is something you dreamed up, and incompatible with general use.

              severe /svir/
              1. harsh; unnecessarily extreme: severe criticism; severe laws.
              2. serious or stern in manner or appearance: a severe face.
              3. grave; critical: a severe illness.

              Yes, copyright laws are indeed severe. However, violation of copyright does not lead to death, like the severe flaws in the Toyota ECM software, so in that sense, unlike a severe illness, a violation of copyright is not severe in the same way the Toyota ECM software or an illness can be considered severe.

              And, incidentally, if discovered, it becomes a severe problem for those that wrote and own the software and possible those that use it. Fro example, it could then become subject to criminal penalties (i.e. personal ones) to continue to use the software.

              China regularly ignores patent an

              • by gweihir ( 88907 )

                Many words, no relevant content. Misdirection to cover up your display of ignorance would be my guess.

  • Rather than providing source code, you would want this kind of software certified. So, construct some tests with a number of samples and a number of reference samples and check whether the software can find one or more "golden eggs", that is a priori known matches. Also, do it double blind, etc.

We can found no scientific discipline, nor a healthy profession on the technical mistakes of the Department of Defense and IBM. -- Edsger Dijkstra

Working...