Putting statistics into forensic firearms identification

The new approach also seeks to transform firearm identification from a subjective method that depends on an examiner’s experience and judgement to one that is based on objective measurements. A landmark 2009 report from the National Academy of Sciencesa nd a 2016 report from the President’s Council of Advisors on Science and Technology both called for research that would bring about this transformation.

The theory behind forensic ballistics
When a gun is fired, and the bullet blasts down the barrel, it encounters ridges and grooves that cause it to spin, increasing the accuracy of the shot. Those ridges dig into the soft metal of the bullet, leaving striations. At the same time that the bullet explodes forward, the cartridge case explodes backward with equal force against the mechanism that absorbs the recoil, called the breech face. This stamps an impression of the breech face into the soft metal at the base of the cartridge case, which is then ejected from the gun.

The theory behind firearm identification is that microscopic striations and impressions left on bullets and cartridge cases are unique, reproducible, and therefore, like “ballistic fingerprints” that can be used to identify a gun. If investigators recover bullets or cartridge cases from a crime scene, forensic examiners can test-fire a suspect’s gun to see if it produces ballistic fingerprints that match the evidence.

But bullets and cartridge cases that are fired from different guns might have similar markings, especially if the guns were consecutively manufactured. This raises the possibility of a false positive match, which can have serious consequences for the accused.

A Statistical approach
In 2013, Song and his NIST colleagues developed an algorithm that compares three-dimensional surface scans of the breech face impressions on cartridge cases. Their method, called Congruent Matching Cells, or CMC, divides one of the scanned surfaces into a grid of cells, then searches the other surface for matching cells. The greater the number of matching cells, the more similar the two surfaces, and the more likely they are to have come from the same gun.

In their recent study, the researchers scanned 135 cartridge cases that were fired from 21 different 9-millimeter pistols. This produced 433 matching image pairs and 4,812 non-matching pairs. To make the test even more difficult, most of the pistols were consecutively manufactured.

The CMC algorithm classified all the pairs correctly. Furthermore, almost all the non-matching pairs had zero matching cells, with a handful having one or two due to random effects. All the matching pairs, on the other hand, had at least 18 matching cells. In other words, the matching and non-matching pairs fell into highly separated distributions based on the number of matching cells.

“That separation indicates that the probability of random effects causing a false positive match using the CMC method is very low,” said co-author and physicist Ted Vorburger.

A better way to testify
Using well-established statistical methods, the authors built a model for estimating the likelihood that random effects would cause a false positive match. Using this method, a firearms expert would be able to testify about how closely the two cartridges match based on the number of matching cells, and also the probability of a random match, similar to the way forensic experts testify about DNA.

Although this study did not include enough test-fires to calculate realistic error rates for actual casework, the study has demonstrated the concept. “The next step is to scale up with much larger and more diverse datasets,” said Johannes Soons, a NIST mechanical engineer and co-author of the study.

With more diverse datasets, researchers will be able to create separate models for different types of guns and ammunition. That would make it possible to estimate random match rates for the various combinations that might be used in a crime.

NIST notes that other groups of researchers are working on ways to express the strength of evidence numerically, not only for firearms but also fingerprints and other types of pattern evidence. Many of those efforts use machine learning and artificial intelligence-based algorithms to compare patterns in the evidence. But it can be difficult to explain how machine-learning algorithms work.

“The CMC method can be easily explained to a jury,” Song said. “It also appears to produce very low false positive error rates.”

— Read more in J. Song et al., “Estimating error rates for firearm evidence identifications in forensic science,” Forensic Science International 284 (March 2018) (DOI: 10.1016/j.forsciint.2017.12.013)