United States v. Lewis

ANN D. MONTGOMERY U.S. DISTRICT COURT

MEMORANDUM OPINION AND ORDER

Thomas Calhoun-Lopez, Assistant United States Attorney, United States Attorney's Office, Minneapolis, MN, on behalf of Plaintiff. Kevin C. Riach, Esq., Fredrikson & Byron, P.A., Minneapolis, MN, on behalf of Defendant.

I. INTRODUCTION

This matter is before the undersigned United States District Judge for a ruling on Defendant Kenneth Davon Lewis' ("Lewis") Objection [Docket No. 119] to Magistrate Judge David T. Schultz's January 6, 2020 Report and Recommendation [Docket No. 115] ("R&R"). In the R&R, Judge Schultz recommends denying in part and granting in part Lewis' Motion to Exclude DNA Evidence [Docket No. 27]. For the reasons stated below, the Objection is overruled and the R&R is adopted.

II. BACKGROUND

The full background of this motion is thoroughly set forth in the R&R and is incorporated by reference. Briefly, Lewis faces trial after being federally indicted on one count of being an armed career criminal in possession of a firearm. See Superseding Indictment [Docket No. 126]. Lewis was arrested on April 18, 2018 after scuffling with police officers and a landlord in the stairwell of an apartment building. R&R at 2-3. A Smith & Wesson 9mm gun was recovered at the scene. R&R at 1, 3-4.

After Lewis' arrest, the gun was sent to Midwest Regional Forensic Laboratory ("MRFL") for DNA testing. The MRFL lab analyzed three DNA swabs from the gun using a probabilistic genotyping software program called STRmix. Gov't Ex. 1 at 2; Gov't Ex. 19. Each of the analyzed swabs contained a large amount of "good quality" DNA. Tr. Vol. III [Docket No. 83] at 456.

The lab received five DNA swabs from the gun, but was able to analyze only three of the swabs because two did not include sufficient DNA to permit testing. R&R at 46; Gov't Ex. 1 at 2.

Unless otherwise specified, all Exhibits referenced in this Order were introduced during the three days of hearings on the Daubert motion.

Based on the STRmix analysis, the MRFL lab determined the DNA on the gun was a mixture from four persons. Gov't Ex. 1 at 2; Def. Ex. 6 at 1. The STRmix results showed that the DNA mixture in each of the three swabs is "greater than one billion times more likely if it originated from [Lewis] and three unknown unrelated individuals than if it originated from four unknown unrelated individuals." Gov't Ex. 1 at 2. This statistic, called a likelihood ratio ("LR"), is interpreted as providing "extremely strong support for inclusion" of Lewis as a contributor to the DNA mixture found on the gun. Id. Lewis was determined to be the highest contributor and was estimated to have contributed 56% to the DNA mixture. Def. Ex. 6 at 1; Tr. Vol. II [Docket No. 60] at 166-67. The lowest contributor was estimated to have contributed 6% to the mixture. Def. Ex. 6 at 1.

The STRmix results also showed that all involved police officers and the landlord were excluded as contributors to the DNA mixture on the gun. Gov't Ex. 1 at 2. No likelihood ratio was provided for this result. See id.

Lewis challenges the admissibility of the DNA evidence, arguing the use of STRmix to analyze the DNA samples at issue is not sufficiently reliable to be admissible under Daubert v. Merrell Dow Pharms., Inc., 509 U.S. 579 (1993). Judge Schultz conducted three days of evidentiary hearings and heard testimony from several experts, including Dr. John Buckleton. Dr. Buckleton is the co-developer of STRmix, which was created in 2011 through a joint venture that included New Zealand's Institute of Environmental Science and Research ("ESR"), where Dr. Buckleton serves as lead scientist. Tr. Vol. I [Docket No. 53] at 7, 12-13. Testimony was also heard from Anne Ciecko, the DNA technical leader at the MRFL lab. Tr. Vol. II at 127. Three defense experts also testified: Nathaniel Adams, a software engineer who reviews and analyzes forensic DNA testing results; Dr. Dan E. Krane, Ph.D., professor of biological sciences at Wright State University and president of Forensic Bioinformatics, Inc.; and Dr. Mats Heimdahl, professor of computer science at the University of Minnesota. Id. at 299-300; Tr. Vol. III at 378-79, 555-56; Def. Exs. 18, 25, 33.

Additionally, Judge Schultz appointed Dr. William Thompson as a special master to advise the Court on the issues of scientific reliability. See Order [Docket No. 77]. Dr. Thompson is a professor emeritus in the University of California-Irvine's Department of Criminology, Law & Society. Thomspon CV [Docket No. 114]. He also chairs the Human Factors Committee and is a member of the Forensic Science Standards Board of the Organization of Scientific Area Committees, which is sponsored by the National Institute of Standards and Technology. Id. He has written numerous articles on the topic of forensic DNA evidence. Id. Dr. Thompson reviewed the transcripts and exhibits from the first two days of testimony and personally attended the third day, during which he had the opportunity to ask questions of the witnesses. Dr. Thompson provided a Special Master's Report [Docket No. 113] to the Court on October 31, 2019. The Report was shared with the parties prior to their submission of post-hearing briefs. R&R at 7.

On January 6, 2020, Judge Schultz issued a 63-page R&R finding that STRmix meets Daubert admissibility standards as to the DNA evidence of inclusion—that is, the evidence regarding the likelihood that Lewis is a potential contributor to the DNA mixture on the gun. As to the DNA evidence of exclusion—that is, the evidence excluding the police officers and landlord as potential contributors to the DNA mixture—Judge Schultz determined this DNA evidence falls below STRmix's established threshold for reliability. The R&R thus recommends denying Lewis' Daubert motion as to the evidence of inclusion, and granting the motion as to the evidence of exclusion. Lewis objects to the determination that the DNA evidence of inclusion is admissible. There were no objections to the conclusion that the evidence of exclusion is inadmissible.

III. DISCUSSION A. Standard of Review

In reviewing a magistrate judge's report and recommendation, the district court "shall make a de novo determination of those portions of the report or specified proposed findings or recommendations to which objection is made." 28 U.S.C. § 636(b)(1)(C); see also D. Minn. L.R. 72.2(b). A district judge "may accept, reject, or modify, in whole or in part, the findings or recommendations made by the magistrate judge." 28 U.S.C. § 636(b)(1)(C). B. Daubert Standard

The admission of expert testimony is governed by Rule 702 of the Federal Rules of Evidence, which provides:

A witness who is qualified as an expert by knowledge, skill, experience, training, or education may testify in the form of an opinion or otherwise if: (a) the expert's scientific, technical, or other specialized knowledge will help the trier of fact to understand the evidence or to determine a fact in issue; (b) the testimony is based on sufficient facts or data; (c) the testimony is the product of reliable principles and methods; and (d) the expert has reliably applied the principles and methods to the facts of the case.

When evaluating the admissibility of expert testimony, a trial court serves as the gatekeeper to "ensure that any and all scientific testimony or evidence admitted is not only relevant, but reliable." Daubert v. Merrell Dow Pharms., Inc., 509 U.S. 579, 589 (1993).

"In a case involving scientific evidence, evidentiary reliability will be based on scientific validity." Id. at 590 n.9 (emphasis in original). A trial court may consider one or more of the following non-exclusive factors in assessing scientific validity: "(1) whether the theory or technique can be (and has been) tested; (2) whether the theory or technique has been subjected to peer review and publication; (3) the known or potential rate of error; and (4) whether the theory has been generally accepted [in the relevant scientific community]." Lauzon v. Senco Prods., Inc., 270 F.3d 681, 687 (8th Cir. 2001) (citing Daubert, 509 U.S. at 593-94). A district court possesses broad discretion in making its reliability determination. Kumho Tire Co. v. Carmichael, 526 U.S. 137, 142 (1999). The proponent of the expert testimony bears the burden of showing by a preponderance of the evidence that the testimony is admissible. Lauzon, 270 F.3d at 686.

"As a general rule, the factual basis of an expert opinion goes to the credibility of the testimony, not the admissibility, and it is up to the opposing party to examine the factual basis for the opinion in cross-examination." United States v. Finch, 630 F.3d 1057, 1062 (8th Cir. 2011) (internal quotations and alterations omitted). Although Rule 702 favors admissibility rather than exclusion, the district court must ensure that an expert's testimony "rests on a reliable foundation." Daubert, 509 U.S. at 597.

C. Lewis' Objections

Lewis raises three objections to the R&R's conclusion that STRMix evidence of Lewis' inclusion as a potential contributor to the DNA mixtures meets Daubert's admissibility standard. Lewis argues: 1) STRMix has not demonstrated foundational validity; 2) STRMix has no known error rate; and 3) STRMix did not follow minimum software industry practices to ensure its software performed reliably.

1. Foundational Validity

Lewis argues STRmix has not been shown to be a sufficiently reliable method for analyzing the type of complex DNA mixtures at issue here. Lewis contends that validation studies show the range of reliability for STRmix does not extend beyond DNA mixtures involving more than three contributors in which the minor contributor constitutes less than 20% . The DNA mixtures at issue here each involve four contributors with the minor contributor constituting 6%. Def. Ex. 6 at 1. Lewis thus contends these mixtures exceed the bounds of reliability which have been validated for STRmix.

Lewis largely relies on a 2016 report by the President's Council and Advisors on Science and Technology ("PCAST") that addressed the validity and reliability of probabilistic genotyping software programs such as STRmix. Def. Ex. 2. The PCAST Report examined the published evidence and concluded:

The two most widely used methods (STRmix and TrueAllele) appear to be reliable within a certain range, based upon the available evidence and the inherent difficulty of the problem. Specifically, these methods appear to be reliable for three-person mixtures in which the minor contributor constitutes at least 20 percent of the intact DNA in the mixture and in which the DNA amount exceeds the minimum level required for the method.

Id. at 80 (footnote omitted). The PCAST Report further stated that "[t]he range in which foundational validity has been established is likely to grow as adequate evidence for more complex mixtures is obtained and published." Id. at 82. An addendum to the PCAST Report was issued in 2017. Addendum PCAST Report [Docket No. 48, Attach. 2]. The addendum clarifies that the concerns related to the minor contributor arise when the person of interest contributes less than 20% of the DNA in the mixture. Id. at 8; Tr. Vol. I at 89.

In response to the PCAST Report, a study was conducted and published by STRmix co-developer Dr. Buckleton and his colleagues at ESR. Gov't Ex. 16. The study, titled Internal Validation of STRmix—a Multi-Laboratory Response to PCAST (the "PCAST Response Study"), examined 2,825 DNA mixtures compiled from 31 laboratories. Id. at 12. Mixtures of three, four, five, and six contributors were specifically targeted. Id. The mixtures were interpreted by staff at ESR using STRmix Version 2.5.02. Id. As stated in Dr. Thompson's Special Master Report, "[w]hen the mixtures were compared with the DNA profiles of thousands of known contributors and millions of non-contributors, STRmix was able to distinguish the contributors from non-contributor[s] with a high level of accuracy." Special Master Report at 31. The study "show[s] persuasively that STRmix is capable of producing accurate results with extremely low error rates: STRmix not only works, it seems to work extremely well, at least when used in the manner it was used in these studies." Id. at 30-31.

In the R&R, Judge Schultz determined that the PCAST Response Study "satisfies the criteria set forth by PCAST for expanding [STRmix's] foundational validity to mixtures of four and five persons, assuming sufficient DNA material is present and that a sufficient percentage of that DNA was contributed by the most minor contributor." R&R at 46. Judge Schultz thus concluded that STRmix is a reliable principle and method for analyzing complex DNA mixtures including the DNA mixture in this case.

Lewis objects to this conclusion, arguing the PCAST Response Study used Version 2.5.02 of the STRmix software, whereas the MRFL lab used STRmix Version 2.4.05 to process the DNA mixture in this case. Lewis contends several changes were made to STRmix between Version 2.4.05 and Version 2.5.02, and that these changes directly affect the LR calculations performed by the different software versions. Lewis thus argues the PCAST Response Study's validation testing on the later version of STRmix did not validate the earlier version that was used here.

The R&R anticipated this objection and concluded the argument lacked merit because the newer STRmix version "changed nothing fundamental in the program that would be likely to affect its accuracy." R&R at 33. In reaching this conclusion, the R&R examined four miscodes in Version 2.4.05 that were corrected in the later version, and determined that none of the miscodes in Version 2.4.05 had a material impact on the LR calculations. Id. at 33-34. The R&R also noted that ESR had performed diagnostics on Version 2.4.05 and confirmed it was valid. Id. at 33 (citing Tr. Vol. III at 515). The MRFL lab also performed its own internal validation study of STRmix Version 2.4.05 and determined it was valid. Id. at 34 (citing Def. Ex. 14).

The R&R examined Miscode Numbers 4, 5.1, 5.2, and 5.3 listed in Defendant's Exhibit 28. See R&R at 33-34.

Dr. Buckleton testified that ESR performs regression testing "on all versions [of STRmix] against the new one. So 2.4.05 has been tested against 2.5." Tr. Vol. III at 515.

Lewis nevertheless argues that three additional changes were made between Versions 2.4.05 and 2.5.02 that were not discussed in the R&R. Lewis contends that these changes affected LR calculations, and that the combined effect of the changes on LRs is unknown. This argument ignores that the ESR performed diagnostics on Version 2.4.05 to confirm its validity, and that the MRFL lab conducted an internal validation study of Version 2.4.05 for use in the lab. Given these safeguards, the Court agrees with the conclusion in the R&R that the changes from Version 2.4.05 to 2.5.02 did not fundamentally change STRmix's accuracy.

Because the updates from Version 2.4.05 to Version 2.5.02 did not materially alter the STRmix software's accuracy, the PCAST Response Study's validation testing on the later version was sufficient to establish the foundational validity and reliability of Version 2.4.05.

2. Error Rate

Lewis next argues that STRmix is unreliable because it does not have a known error rate. This argument was also addressed in the R&R, which concluded that the "error rate for false inclusion is known and is acceptably small." R&R at 42. In reaching this conclusion, the R&R relied on Dr. Buckleton's testimony that the error rate for false inclusion is "immeasurably small," and the Special Master Report stating that "[w]hile there were a few instances in which STRmix produced results that falsely linked non-contributors to the [DNA] mixtures, these misleading results were rare." Tr. Vol. I at 64; Special Master Report at 8.

A false inclusion occurs when a person who did not contribute to the DNA mixture is falsely linked to the mixture. Special Master Report at 8. --------

Lewis argues that "despite these claims that the error rate is 'small' or 'rare,' no numerical error rate or range of error rates for STRmix false inclusions has ever been stated on the record [or] in the documentary evidence." Obj. at 9. However, Daubert does not require that an error rate be numerically identified for scientific evidence to be found sufficiently reliable. Rather, the known or potential error rate is one of several non-exclusive factors that courts consider when assessing the scientific validity of a theory or technique. Daubert, 509 U.S. at 593-94.

Here, the R&R found that the error rate for STRmix "can be and has been estimated by checking how often the program assigns highly incriminating likelihood ratios to the profiles of known non-contributors." R&R at 42. The PCAST Response Study shows that such errors are rare and occur no more often than would be expected by chance due to an "adventitious match," meaning two individuals having the same or extremely close DNA. Special Master Report at 8, Tr. Vol. I at 42-43. "In other words, the rate of false inclusions was approximately what would be expected if STRmix performed its function flawlessly." R&R at 42 (quoting Special Master Report at 8). Based on this evidence, the R&R correctly concluded that the error rate for STRmix is acceptably small, and that the absence of a precisely calculated error rate does not alter the reliability and validity of STRmix.

Lewis also argues that the R&R only addressed error rates for false inclusion, and did not account for the possibility that STRmix might generate inaccurate LRs. As an example, Lewis argues STRmix might calculate a DNA sample as being a billion times more likely to have originated from the defendant, when in actuality the LR is only 20 times more likely. Lewis contends there are no published studies showing how often STRmix miscalculates the magnitude of a LR.

Judge Schultz considered the accuracy of the LR generated by STRmix and found that "[t]he variation in the precise LR numbers generated by STRmix from one run to the next is very small. The STRmix validation studies have demonstrated that the variance in LRs is consistently less than an order of magnitude." R&R at 39 (citing Tr. Vol. II at 151, 170); see also Def. Ex. 14 at 28-29 (discussing LR precision and noting that LR does not vary more than one order of magnitude). Thus, there is no basis to assume that STRmix would generate an inaccurate LR of the magnitude provided in Lewis' example. To the contrary, Dr. Thompson has stated that "[s]tatistical analyses suggest that, in the aggregate, the LRs produced by STRmix are properly calibrated and do not overstate the value of incriminating evidence." Special Master Report at 36. Thus, the absence of a specified error rate does not render STRmix unreliable.

3. Software Industry Standards

Finally, Lewis argues STRmix is not sufficiently validated from a software engineering perspective because it does not satisfy industry practices for the development and testing of new software. This argument lacks merit. As the R&R recognized, STRmix complies with all published guidance documents specifically directed to software validation for probabilistic genotyping systems. Those guidelines include standards published by the Scientific Working Group on DNA Analysis Methods, the Forensic Science Regulator, and the International Society for Forensic Genetics.

The R&R also found that STRmix "very nearly" complies with the safety-critical software developments published by the Institute of Electrical and Electronics Engineers ("IEEE"), and that strict compliance with IEEE standards is not required because STRmix has been rigorously tested and shown to be reliable. R&R at 37-38.

Resisting this conclusion, Lewis relies on the following testimony of Dr. Mats Heimdahl, Dean of the University of Minnesota Computer Science and Engineering Department, to argue that the STRmix software has not been adequately validated:

[The STRmix software] seems to be a plethora of configuration parameters that may or may not have been explored effectively during the validation testing efforts, but this is where the verification testing would come in and start twiddling all of those in all kinds of different ways to see if they interact in unexpected ways, if it always works and if it fails when it's expected to fail, and so on and so forth.

Tr. Vol. III at 591. This testimony does not overcome the R&R's thorough and well reasoned analysis leading to the conclusion that the STRmix software has been sufficiently validated.

IV. CONCLUSION

Based upon the foregoing, and all the files, records, and proceedings herein, IT IS HEREBY ORDERED that:

1. Defendant Kenneth Davon Lewis' Objection [Docket No. 119] to Magistrate Judge David T. Schultz's January 6, 2020 Report and Recommendation is OVERRULED;

2. The Report and Recommendation [Docket No. 115] is ADOPTED; and

3. Lewis' Motion to Suppress DNA Evidence [Docket No. 27] is GRANTED IN PART and DENIED IN PART as follows:

a. The evidence as to Lewis' inclusion as a potential contributor to the DNA mixtures found on the gun is admissible at trial.

b. The evidence as to the exclusion of the relevant police officers and the landlord as potential contributors to the DNA mixtures found on the gun is not admissible at trial.

BY THE COURT:

s/Ann D. Montgomery

ANN D. MONTGOMERY

U.S. DISTRICT COURT Dated: March 3, 2020