This Impenetrable Program Is Transforming How Courts Treat DNA Evidence (wired.com) 186
mirandakatz writes: Probabilistic genotyping is a type of DNA testing that's becoming increasingly popular in courtrooms: It uses complex mathematical formulas to examine the statistical likelihood that a certain genotype comes from one individual over another, and it can work with the subtlest traces of DNA. At Backchannel, Jessica Pishko looks at one company that's caught criminal justice advocates' attention: Cybergenetics, which sells a probabilistic genotyping program called TrueAllele -- and that refuses to reveal its source code. As Pishko notes, some legal experts are arguing that Trueallele revealing its source code 'is necessary in order to properly evaluate the technology. In fact, they say, justice from an unknown algorithm is no justice at all.'
I WANT THE TRUTH! (Score:2)
about the code!
Re:I WANT THE TRUTH! (Score:4, Insightful)
You can't handle the truth [insert full quote here]
but anyway this should not be admissable in court until the source code (with needed toolchain) has been vetted by the Oldest and Crankiest Qualified persons they can find.
(does this have a bias as to which "race" it comes up with?? will it pop certain trait markers more often??)
Re: (Score:3)
Whether it does or not doesn't matter as long as it isn't reviewed, because it will certainly be challenged on these grounds or at the very least you'll see the relevant pressure groups cry foul, whether or not it is.
This alone means that this MUST be reviewed before even thinking about considering it as admissible evidence.
Re: (Score:2)
Having access to the source code would no doubt make it much easier to verify, but if they refuse, you can still run a series of studies aimed at testing whether or not the program actually does what it claims. If you have some known DNA samples, you can have the program analyze them in order to produce estimates and see how cl
Re: (Score:2)
This. Essentially black box testing [softwarete...entals.com].
Re: (Score:3)
Computers cannot lie, unless programmed to do so.
Re: (Score:3)
All computers are programmed to lie, either through mistakes or intentional. You will never get away from this because computers are programmed by humans and our creations will always suffer from our own errors and misdeeds.
Re: (Score:2)
A lie is an intentional deception. If you have a computer that can lie to you, it must have passed the Turing test a long time ago and may even be self-aware by now.
Computers process data, and operate on the principle of: Garbage In, Garbage Out.
Re: I WANT THE TRUTH! (Score:2)
I can answer this (Score:5, Insightful)
As a programmer I can assure you that I am infallible and perfect. My superiority is the reason I am a programmer and most people are not.
Re: (Score:2)
...and here's the proof. [adequacy.org]
Re: (Score:2)
I wouldn't so much question the programmer as I'd question the definition of "correct"...
Re: (Score:2)
double random() // chosen by 5 roll of a fair D10.
{
return 0.82740;
}
Re: (Score:2)
Re: (Score:2)
If only you could randomly generate a number between 0 and 1 of arbitrary precision by using a d10 die to determine each individual digit.
Re: (Score:2)
Give Americans more credit. It's more likely to be
Revealing != open source (Score:5, Informative)
We have seen issues with red light cameras, we have seen issues with labs doing drug testing on hair, we have seen child abuse panics from psychology "experts". Both methods and experts have to be open for independent, impartial validation. Otherwise they are no better than a duck test.
Re: (Score:2)
Otherwise they are no better than a duck test.
I say we go back to the duck test! I keep protests of "this is a witch hunt" and yet we don't even test to see if they float like a duck! How are we supposed to know if we've found a witch if not for the duck test? ;)
Re:Revealing != open source (Score:4, Insightful)
I don't think even NDA access is appropriate. How versed are you in probabilistic medical programming? How many people would you say are?
Hell, even the most experienced developer will need some time to acclimatize to any sufficiently complex codebase, now throw in the specialties on top of it. It's beyond unreasonable to expect any "expert" to have limited access ( both physical and temporal ) to the codebase then be expected to give expert testimony in court on it.
The only way things like this get properly vetted is via "many eyes", and even that's no guarantee.
Speaking of experts; let's pretend your some poor schmuck ( literally ), using a tool like this in a case where you can't afford an expert witness ( and it would be beyond pricy I'd expect. I know I'd charge a shitload ) only guarantees a compromised defense.
No, I can't see any reason why the code shouldn't be publicly available if the tool will be used to help convict people.
Re:Revealing != open source (Score:4, Insightful)
All software used in criminal convictions should be open source as a requirement of using it.
Speed traps, DNA processing, fingerprint matching, gait detection, photo enhancement software, even the firmware in the cameras used at the crime scene.
Re: (Score:3)
I'm not even going with "open source", merely publicly available.
Perhaps a new license which protects a company's investment in it's intellectual property ( legally prevents others from using the publicly available code ), but still requires publishing of said property.
Re: (Score:3)
You mean something like oh I don't know copyright!
Re: (Score:3)
Something that we agree on! I never thought I'd see the day.
Ideally, they'd sign all of their output with a hash of their firmware too. Having source code "similar" to what is actually running on the device is not enough.
Re: (Score:2)
No. That is entirely unreasonable.
If it's being used as evidence in a public court, it is only reasonable to provide public access to the source code, and specifically to provide access to the methods & computations that produced a particular result in a particular case.
We are not supposed to have secret courts in the US (FISA notwithstanding).
Re: (Score:2)
And *that's* the point. I think I might be willing to accept that the results were probably correct if it produced the same result on the same information when it was included as a part of a large double-blinded sample, where most of the answers had known values and the evaluator was totally independent of all interested parties. I'd even accept the results as grounds for stating the probability of the result. But the evaluator needs to be independent and the test needs to be double blind with both known
"evidence" (Score:5, Insightful)
As Terry Pratchett wrote somewhere: "Evidence means 'that what is seen'". Nuff said.
Paaia
Is this different than a human "expert witness"? (Score:3, Insightful)
A lot of expert witness testimony comes down to a judgement call -- "In your opinion, as someone who has been working in this field for 20 years, how confident are you that these signatures / bullet marks / fingerprints / DNA match?" That's the result of an algorithm that you can't examine either, and has at least as much opportunity for being corrupted by unconscious prejudice or outright bribery as a piece of software.
Re:Is this different than a human "expert witness" (Score:5, Insightful)
You have the right to face your accuser, which includes examining the evidence against you. This is secret evidence. It amounts to "because we say so", and should not be tolerated.
A software bug you're not permitted to look for could send you to jail. At least with a human expert witness you can cross-examine them.
Re: (Score:3)
Expert judgement can be countered by other experts. Here we are being presented with something as a "Fact". There is no way to dispute it and there is no way to verify it which is what people are having a problem with.
Re:Is this different than a human "expert witness" (Score:5, Insightful)
Expert judgement can be countered by other experts. Here we are being presented with something as a "Fact". There is no way to dispute it and there is no way to verify it which is what people are having a problem with.
Questioning expert's qualifications is fair game in trials. If you can demonstrate that expert is not impartial, you can largely mitigate their testimony.
How do you question algorithm like if (1) = Guilty; other than code review?
Re:Is this different than a human "expert witness" (Score:5, Interesting)
Well it shouldn't be accepted as fact. Ideally the courts would instruct the jury to treat the software's output as similar to a human being saying, "This is my expert opinion." You can submit your own software's "opinion" as evidence as much as you can get your own expert human to testify on your behalf.
It is true that you can't cross-examine it; but ideally, that should make the software less reliable. If you had an expert who, upon cross-examination, always responded, "I don't know, it just seems that way", then he wouldn't have much credibility. Ideally, software that can't justify its "opinion" should be treated the same way.
I have said "ideally" here several times, recognizing that it may well be the case that this isn't how people actually think. But I think a more constructive response to this misplaced trust is to help inform courts and defense lawyers more clearly (who should in turn inform the juries).
Re:Is this different than a human "expert witness" (Score:4, Interesting)
Well it shouldn't be accepted as fact. Ideally the courts would instruct the jury to treat the software's output as similar to a human being saying, "This is my expert opinion." You can submit your own software's "opinion" as evidence as much as you can get your own expert human to testify on your behalf.
One of the requirements for presenting expert testimony is that you have to provide all of the materials that the expert used in forming their opinion. If the results of some software were treated as an expert opinion, the "materials relied upon" would almost certainly include the source code. It may even make the programmers, as the source of those materials, subject to being deposed about how they developed the software.
Re: (Score:2)
US Constitution, sixth amendment: "In all criminal prosecutions, the accused shall enjoy the right...to be confronted with the witnesses against him; to have compulsory process for obtaining witnesses in his favor....". It seems to me that a device that announces something should have some humans, i.e. witnesses, testifying in its favor, but the courts may not agree.
Re: Is this different than a human "expert witness (Score:2)
Re:Is this different than a human "expert witness" (Score:4, Insightful)
Re: (Score:3)
What you're describing is exactly why experts frequently have their credibility challenged and why they need to provide the means by which to verify their credentials. The problem here is that they're providing no means by which to establish or confirm the credibility of the algorithm, and they know that doing so doesn't harm them as it would with an expert witness.
Imagine if the prosecution put an "expert" on the stand who testified how the prosecution wanted, but when the defense attorney asked where the
Re: (Score:3)
You brought up a good point.
So the counter would be to write a program that accepted the same physical evidence data and simply returned whatever answer the defense wants.
Re: (Score:3)
Re: (Score:2)
Hell, PBS Frontline did a special about the horrors of modern "forensics", titled 'The Real CSI'
It's an eye-opener.
profoundly failing to understand justice (Score:2)
A successful conviction may be legitimately tipped by accurate checked evidence, in this case DNA
But justice is not a matter of technical facticity. It is withholding something from a party that they deserve.
The evidence may help identify discrepancies between the two, but it is a major conflation to substitute that with justice.
Re: (Score:2)
"Justice" is probably in regard to dispute of "the courts find this blackbox satisfactory"
as in "the courts find this box sufficient means for achieving their goals"
as in "the courts find this accomplishes their job"
as in "the courts find this accomplishes justice"
There needs to be testing and validation... (Score:5, Insightful)
Jurors and judges need to know what the probabilities are. Remember, in a criminal trial, the standard for evidence is "beyond a reasonable doubt." Sending people to prison for life or even to death row based on flimsy evidence is unacceptable.
This isn't to say that it hasn't happened before -- Cameron Todd Willingham was executed in Texas on the testimony of an "arson expert" with no formal training in the field.
The code should be evaluated or the tool should be banned from court. The company doesn't like it? Too bad. They don't have to sell to the forensic lab/law enforcement market.
Re: (Score:2)
The code should be evaluated or the tool should be banned from court. The company doesn't like it? Too bad. They don't have to sell to the forensic lab/law enforcement market.
Arguably, the program can be evaluated without the source code. Simply use known samples and examine the output. Do the results of the analysis match what was known about the samples?
This testing would have to be performed by a neutral third party of course.
Re:There needs to be testing and validation... (Score:5, Informative)
Re: (Score:3)
I agree with Thelasko: We don't need to see the source code, we just need to see the results. Evaluate it as a black box -- feed it known samples and see if it produces the correct results. If so, it's reasonable to admit the machine's evaluation.
What if there's some flaw that affects a small fraction of cases? It's possible, but it's also possible that code inspection wouldn't find the problem either. There's always another test, another set of eyes, another *something* that could be done. At some point
Re: (Score:2)
That's how it's SUPPOSED to be, but in practice you won't find a cop or anyone else testifying about the internal state of the breathalyzer or why a sample might cause it to read 0.035. It'll just be 0.02 is the legal limit, he came up 0.035, case closed!
Re: (Score:2)
Apparently part of the output is the probabilities that a particular sample is from the suspect or from someone else. How can you feed the program "known samples" that can evaluate that it produced correct results of the probabilities for the particular samples used in the case? (One sample was reported as having "a one
Re: (Score:2)
How can you feed the program "known samples" that can evaluate that it produced correct results of the probabilities for the particular samples used in the case? (One sample was reported as having "a one in 211 quintillion chance that it originated from someone else".)
I am not an expert in the field. However, I propose the following tests:
These tests should be repeated many times to form a statistical profile, and then compared to the output of the software.
Re: (Score:2)
Yes, sometimes with side effects. I get a rash from sulfa, and I get the lisinoprol cough. I've never seen "five years in prison" listed as a possible side effect on medical data sheets, and if there's the chance of a serious adverse reaction I'm informed in advance and the doctors and nurses take precautions. I've watched for some odd nasty side effects that didn't in fact happen, and was told what to do if, for example, my Achil
Re: (Score:2)
Leveson notes that a lesson to be drawn from the [THERAC-25] incident is to not assume that reused software is safe: "A naive assumption is often made that reusing software or using commercial off-the-shelf software will increase safety because the software will have been exercised extensively. Reusing software modules does not guarantee safety in the new system to which they are transferred..."
I'm sure Thelasko meant well, we hold faith in science driven by observation, but we're not speculating about string theory, we're declaring that someone did, by this court's finding, commit X and deserves to Y. The declaration is different than musing "we consistently observe Z so protons would be Q." in a report, which doesn't actually assert Q is fact.
Re: (Score:2)
So can "tin whiskers". We need to be allowed to inspect the hardware too.
No we don't. If the software is open, then the defense can run it on their own hardware, and should get an identical result.
Re: (Score:2)
That doesn't necessarily work. With a Breathalyzer, there's no way to run that same breath through a defense machine. We need to have such devices inspected.
Re: (Score:2)
The program is useless if you have to predetermine the correct answer for every possible input, and the evaluation is useless if you do anything less.
Re: (Score:2)
The program is useless if you have to predetermine the correct answer for every possible input, and the evaluation is useless if you do anything less.
Bullcrap. A pocket calculator has a predetermined correct answer for every possible input. That does not make it "useless".
Not good enough (Score:2)
Arguably, the program can be evaluated without the source code.
At considerable expense. But even then it still is a problem because you would have to do it for every single case. Otherwise you have no way to know if something is different or wrong with the analysis in a case where no verification was conducted.
Simply use known samples and examine the output. Do the results of the analysis match what was known about the samples?
You're talking about using controls and/or independent testing methods. Not really good enough because if there is a discrepancy you run into Segal's Law [wikipedia.org] (a man with a watch knows the time and a man with two is never sure). You have no way to know which test
Re: (Score:2)
Without the source, it would be hard to assure complete coverage in the sample data, The test would have to be exhaustive.
That's fine if the company wants to go that way, but of course, exhaustive testing will cost plenty more and make it far less likely to ever be funded. Until that testing happens, the whole technique should have the same legal standing as the magic 8 ball.
Re: (Score:2)
The code should be evaluated or the tool should be banned from court. The company doesn't like it? Too bad. They don't have to sell to the forensic lab/law enforcement market.
Arguably, the program can be evaluated without the source code. Simply use known samples and examine the output. Do the results of the analysis match what was known about the samples?
This testing would have to be performed by a neutral third party of course.
Oh, like Volkswagen's Dieselgate?
Re: (Score:2)
And they need to know what the probabilities are pooled from. One of the early problems with DNA suspect testing was that early DNA databases were collected predominately from FBI agents.
Who happened to be mostly white.
Which means their gene expression probabilities were different than for blacks. Which meant that the probability of a false positive DNA match on black suspect with a DNA sample from another black person was a couple of
Re: (Score:2)
Even if the "expert" had received training, it would not have made any difference, because no one had ever done a scientific evaluation of the way a house burns. All the opinions of arson experts were bullshit, full of confirmation bias.
Comment removed (Score:5, Interesting)
Re: (Score:2)
One thing is having access to the source code and a completely different story is properly analysing it. When dealing with something as complex as (probabilistic!) DNA sequencing, it seems quite clear that the most sensible way to validate the program is actually using it. Set up a proper benchmark with a relevant number of samples and confirm whether this (+ any other) program works exactly as expected. This would also be an excellent way to objectively assess its accuracy.
Exactly this. My kingdom for modpoints.
You don't test software by looking at the code. You test the software by testing it. If it ain't broken, you're not testing hard enough.
While I'm very pro-OSS, I'm anti forcing private companies to disclose their source code. It is their work, their intellectual property. It's up to the judge to admit the closed-source evidence and up to the jury to weigh it.
Re: (Score:2)
Secret code = Justice denied (Score:2)
You don't test software by looking at the code. You test the software by testing it. If it ain't broken, you're not testing hard enough.
Doing a black box analysis of software when the code should be available for review by a defendant is so wrong headed I barely know where to start There is NO place for secret code when it comes to convicting people of crimes. The defendant should be able to question any and all methods being used to accuse them of a crime.
While I'm very pro-OSS, I'm anti forcing private companies to disclose their source code.
Tell me that when you are facing a life sentence and you aren't allowed to examine the code being used to send you to jail. If we're talking about a word processor, who cares but when
Re: (Score:2)
I'm anti forcing private companies to disclose their source code.
They don't have to disclose their source code. They can choose instead to have it not be usable in court.
Freedom of choice does not mean freedom from consequences.
Re: (Score:2)
Freedom of choice does not mean freedom from consequences.
That is what it means actually. Well, more specifically it's about being free from consequences that you don't wish to be subjected to. If you don't have freedom from consequences that you don't want then it's meaningless because you're really talking about free will, not societal freedom. "If you do drugs, we'll throw you in jail as a consequence! We support your freedom of choice in doing drugs!" That isn't useful.
Re: (Score:2)
If my choices have no consequences, why bother? If my choices can have consequences I like, then they can have consequences I don't like, if only by comparison. This applies when discussing free will or societal freedom. Freedom from consequences I don't want is perforce ineffectuality.
Re: (Score:2)
You don't test software by looking at the code. You test the software by testing it. If it ain't broken, you're not testing hard enough.
But you use the code to find interesting boundary cases that need additional scrutiny in testing!
To properly test software *requires* access to source. Otherwise all you're doing is poking it with a stick to find vulnerabilities.
Re: (Score:2)
Re: (Score:2)
So the article claims a false positive rate of 1 in 211 quintillion for a particular trial. To test that with a 95% confidence interval we would need at least 600 quintillion samples. Now we're a bit short on people on this planet. I don't think Earth could support this many people so we need to colonize other planets. To make things simple, lets assume the average planet
Re: (Score:2)
Re: (Score:2)
Actually, GP is correct if we're resorting to empirical testing. We would want about six hundred quintillion samples to test against to verify that. To say that the chance is one in 211 quintillion rather than one in 211 quadrillion, which is three orders of magnitude difference, we'd have to have enough testing to show that the error rate was less than one in 211 quadrillion, which means that we'd have to have enough samples so that the failures were significantly less than one in 211 quadrillion. That
Re: (Score:2)
Re: (Score:2)
Good idea. If we assume these are independent trials then it's much more feasible :) We can even do more than two people. An
experiment could be you got the perps DNA and a mix of 5 other
samples. Now can you detect whether or not the perp is in the mix.
Also I'm not worried ab
Re: (Score:2)
Sorry I couldn't help myself. I figured you didn't read the article, and the ridiculous claims TrueAllele made. Human error for DNA testing has been measured to be around 1 in 200, so these tiny probabilities are just dangerous theatrics. Still it's an interesting challenge to estimate extreme probability values. I was half hoping you'd shut me up with some nice technical way around the problem...
As for empirical testing, it makes sense as part of a larger system of evaluation. Looks like they have
Re: (Score:2)
Re: (Score:2)
Re: (Score:2)
Re: (Score:2)
That depends heavily on how the software is written. The software can be written to match the algorithm so it's verifiable. It usually isn't, of course, but it would be nice if that were required for forensic software. After all, if we're using this in a court of law, we should b
Re: (Score:2)
Code quality and data provenance (Score:2, Informative)
I've done some genomic work, during the Human Genome Project. I had to step away from the work due to my concerns about the lack of quality. The analysis software of the data, to assemble longer genesic fragements for testing and verification, was so very very poor that all the scientists learned to ignore the analysis and order longer sequence manually, by eyeballing it with their personal experience. It was hideously expensive to do this constantly, especially with the amount of sequences to sample and te
Surprise (Score:2)
Re: (Score:2)
The voting tabulators my state uses can be not available for public examination, because we check against hand counts.
If killer bots are used in warfare, the public doesn't need the details. If they're used in police work, it darn well does.
It's worked for Breathalyzers for years (Score:2)
3 things, pick 1 (Score:3)
Re: (Score:3)
For example, if you can place the suspect in the vicinity of the crime and a motive for the crime along with DNA evidence that says >=95%, then that's a solid conviction.
However, if you are trawling 23andme.com or Ancestry.com databases for matches, and then just grabbing and prosecuting the first match, you'
Re: (Score:2)
I disagree. I'm fine with using DNA to eliminate a suspect ("This DNA can't possibly be that person") and with furnishing the police with a list of interesting people to further investigate.
I don't believe though that DNA should ever be used to convict someone. There may be very few false positives but there will be some, and that's too many for it to be reliable in court.
The whole forensic process is flawed, the DNA analysis is rarely uncontaminated, the subset of DNA markers is too small, the risks of fal
Re: (Score:2)
I think we are saying the same thing more or less, in different ways. DNA evidence should be used as a diagnostic funnel. But 95% certainty is usually sufficient in court to secure a conviction by a jury or a judge - whether or not you or I agree with those odds.
Re: (Score:2)
Absolute proof is not required for conviction; the standard is proof beyond a reasonable doubt. DNA can be used as part of the evidence for that. I personally would never vote to convict based on it alone, but then I'm the sort of guy who gets to be a peremptory strike from the jury box.
No way (Score:2)
If I have a small partial DNA sample there will be multiple people in the world that it will match.
No way. Does that mean there are multiple evil twins in the world I've never met?
Re:Computers and computer modeling is infallible (Score:5, Insightful)
Please point to this model and indicate where it shows your precise location will be under 10' of water by now. Either that or admit you are full of shit and a liar.
A lack of evidence of this "model" will indicate you are a liar whether you respond or not.
Re: (Score:2)
Re:Computers and computer modeling is infallible (Score:4, Informative)
Well it's only be ~10 years since the Inconvenient Truth. I'm not sure what "near future" mean. If that means 1 year to you or 50 years to the rest of us. (I think his film was aiming at the year 2100, but I don't recall exactly).
Here's an example, some islands are completely covered by water at high tide. http://theconversation.com/sea... [theconversation.com]
The most up to date information has projections range from 0.2 meters to 2.0 meters (0.66 to 6.6 feet) of sea level rise in the next 100 years. [Melillo et al., 2014]. And that's the thing about science, you'll find that it is never 100% accurate and if you look back to previous theories and predictions can be embarrassingly inaccurate. But the scientific method generally leads to better answers through many iterations of models, research and theories.
Al Gore's 20 feet rise greatly exceeds the most conservative models, as you've already noted. On the other hand if all the ice covering Antarctica, Greenland, and in mountain glaciers around the world were to melt, sea level would rise about 70 meters (230 feet). That's the far extreme of what could be done with the matter available on Earth, it's not at all likely. (maybe if the Earth's axis tilted to expose the poles? Or maybe if 10's of thousands of years went by and we acquired an atmosphere like Venus that make air temperature nearly uniform across the planet, including the poles?)
Re: (Score:2)
How is this for you?? Al Gore claimed that coastal cities would be 20 feet under water in the near future in his movie 'An Inconvenient Truth'
A politician spouting off his opinion is not a "scientific model".
Can you point to any climate model, peer reviewed and published, that predicted a 10 foot ocean rise by 2017? No? How about a one foot rise by 2020? No? Anything?
Re: (Score:2)
The truth [wordpress.com]
So every one of your "73 predictions" used 1977 as a baseline? Either you found a whole lot of 40 year old "models", or you are full of bullcrap. It is hard to tell since of the 73, exactly this many are actually named or cited: 0.
Re: (Score:2)
Scientific models are scientific. If they turn out to be wrong, that's important information that will be reflected in the next version. Remember that all models are wrong, but some are useful.
Scientists are judged on what they say, since we don't get enough information to discern their intent.
Re: (Score:2)
You are aware, I presume, that the rate of sea level rise in the future could be different from 1.7mm/year? There are good reasons to think that it will increase. Nobody exactly knows how by how much, of course. Current models don't predict anywhere near that rise in that time frame, but it's easy to come up with possible ways for it to happen.
Re: (Score:2)
Re: (Score:2)
Re: (Score:2)
Do you seriously believe every dope dealer, thief, rapist, etc. is entitled to that?
Yes. More importantly, I also believe that everybody accused of being a dope dealer, thief, rapist, etc. is entitled to that.
Take the plea, do the time, pay the fine
The whole plea system in the US is a horrific barrier to justice, particularly when matched to the cost of a capable legal defence.
Re: (Score:2)
It's not possible to tell from source code that a procedure works reliably. It is possible to tell that a procedure doesn't work reliably, or, at least, that there's plenty of reasonable doubt about it.