Defense attorneys have been saying for years; breathalyzers are flawed. A new report that was never meant to be seen was and it confirms the defense attorneys’ suspicions.
Unsurprisingly, our story begins with a DUI case which arose out of Washington state. The DUI defense attorney in the case sought the assistance of two experts to analyze the breathalyzer’s source code accuracy in producing blood alcohol content results.
The experts determined that the source code used by the breathalyzer in the case, which happens to be the same breathalyzer used in other states including California, was fundamentally flawed by producing elevated blood alcohol content readings for reasons I’m not smart enough to understand. The experts put their findings into a preliminary report.
In 2017 at a DUI defense attorney conference, the experts shared their findings and the report with the attendees. This prompted the manufacturer of the breathalyzer to initiate legal action for violation of a protective order regarding the breathalyzer’s source code. The manufacturer and the experts settled out of court.
The experts are now tightlipped about the settlement and have walked back the contents of their report later claiming that it was incomplete, unfinished, and should not be relied upon by those few people who were in possession of it.
ZDNet, however, obtained at copy of the report and published an article detailing its contents and the implications it could have for thousands of DUI cases in a number of states, including California.
Researchers say a breathalyzer has flaws, casting doubt on countless convictions
May 10, 2018, ZDNet – The source code behind a police breathalyzer widely used in multiple states — and millions of drunk driving arrests — is under fire.
It’s the latest case of technology and the real world colliding — one that revolves around source code, calibration of equipment, two researchers and legal maneuvering, state law enforcement agencies, and Draeger, the breathalyzer’s manufacturer.
This most recent skirmish began a decade ago when Washington state police sought to replace its aging fleet of breathalyzers. When the Washington police opened solicitations, the only bidder, Draeger, a German medical technology maker, won the contract to sell its flagship device, the Alcotest 9510, across the state.
But defense attorneys have long believed the breathalyzer is faulty.
Jason Lantz, a Washington-based defense lawyer, enlisted a software engineer and a security researcher to examine its source code. The two experts wrote in a preliminary report that they found flaws capable of producing incorrect breath test results. The defense hailed the results as a breakthrough, believing the findings could cast doubt on countless drunk-driving prosecutions.
The two distributed their early findings to attendees at a conference for defense lawyers, which Draeger said was in violation of a court-signed protective order the experts had agreed to, and the company threatened to sue.
Their research was left unfinished, and a final report was never completed.
Draeger said in a statement the company was protecting its source code and intellectual property, not muzzling research.
“Pursuant to a protective order, Draeger provided the source code to both of the defense experts in Snohomish County,” said Marion Varec, a spokesperson for Draeger. “That source code is highly proprietary and it was important to Draeger that the protective order limit its use to the purposes of the litigation at issue.” Draeger says it believes that one of the experts entrusted to examine the source code was using it in violation of the protective order, so Draeger sent the expert a cease and desist letter. Draeger says it “worked with the expert to resolve the issue.”
Of the law firms we spoke to that were at the conference and received the report, none knew of Draeger’s threat to launch legal action. A person with a copy of the report allowed ZDNet to read it.
The breathalyzer has become a staple in law enforcement, with more than a million Americans arrested each year for driving under the influence of alcohol — an offense known as a DUI. Drunk driving has its own economy: A multi-billion dollar business for lawyers, state governments, and the breathalyzer manufacturers — all of which have a commercial stake at play.
Yet, the case in Washington is only the latest in several legal battles where the breathalyzer has faced scrutiny about the technology used to secure convictions.
TRIAL BY MACHINE
When one Washington state driver accused of drunk-driving in 2015 disputed the reading, his defense counsel petitioned the court to obtain the device’s source code from Draeger.
Lantz, who was leading the legal effort to review the Alcotest 9510 in the state, hired two software engineers, Falcon Momot, a security consultant, and Robert Walker, a software engineer and decade-long Microsoft veteran, who were tasked with examining the code. The code was obtained under a court-signed protective order, putting strict controls on Momot and Walker to protect the source code, though the order permitted the researchers to report their findings, with some limitations. Although the researchers were not given a device, the researchers were given a binary file containing the state’s configuration set by Washington State Patrol.
Although their findings had yet to be verified against one of the breathalyzers, their preliminary report outlined several issues in the code that they said could impact the outcome of an alcohol breath test.
In order to produce a result, the Alcotest 9510 uses two sensors to measure alcohol content in a breath sample: An infrared beam that measures how much light goes through the breath, and a fuel cell that measures the electrical current of the sample. The results should be about the same and within a small margin of error — usually within a thousandth of a decimal point. If the results are too far apart, the test will be rejected.
But the report said that under some conditions the breathalyzer can return an inflated reading — a result that could also push a person over the legal limit.
One attorney, who read the report, said they believed the report showed the breathalyzer “tipped the scales” in favor of prosecutors, and against drivers.
One section in the report raised issue with a lack of adjustment of a person’s breath temperature.
Breath temperature can fluctuate throughout the day, but, according to the report, can also wildly change the results of an alcohol breath test. Without correction, a single digit over a normal breath temperature of 34 degrees centigrade can inflate the results by six percent — enough to push a person over the limit.
The quadratic formula set by the Washington State Patrol should correct the breath temperature to prevent false results. The quadratic formula corrects warmer breath downward, said the report, but the code doesn’t explain how the corrections are made. The corrections “may be insufficient” if the formula is faulty, the report added.
Issues with the code notwithstanding, Washington chose not to install a component to measure breath temperature, according to testimony in a 2015 hearing, and later confirmed by Draeger.
Kyle Moore, a spokesperson for Washington State Patrol said the police department “tested and approved the instrument that best fit our business needs,” and believes the device can produce accurate results without the breath temperature sensor.
The code is also meant to check to ensure the device is operating within a certain temperature range set by Draeger, because the device can produce incorrect results if it’s too hot or too cold.
But the report said a check meant to measure the ambient temperature was disabled in the state configuration.
“The unit could record a result even when outside of its operational requirements,” said the report. If the breathalyzer was too warm, the printed-out results would give no indication the test might be invalid, the report said.
Draeger disputed this finding. A spokesperson said the Washington devices check their temperature, the check is enabled, and that the devices will not produce a reading while the device is outside its operational temperature range.
When asked, a Washington State Patrol spokesperson would not say if the breathalyzer was configured to allow breath tests outside its operational temperature range, saying only that the device “has been tested and validated in various ambient temperatures.”
The report also scrutinized the other sensor — the fuel cell — used to measure a person’s alcohol levels. Any fuel cell will degrade over time — more so when the breathalyzer is used often. This decay can alter the accuracy of test results. The code is meant to adjust the results to balance out the fuel cell’s decline, but the report said the correction is flawed. Breathalyzers should be re-calibrated every year, but the state’s configuration limits those adjustments only to the first six months, the report added.
“We also note that the calibration age does not account for the use frequency of conditions; a unit that has been used hundreds of times per day would have the same correction as one used only once or twice in several months,” the report said.
Concluding the nine-page report, the researchers say they are “skeptical” that the Alcotest 9510 can produce a reliable measurement of breath alcohol.
“Although the apparatus states its output in very absolute terms, we recommend interpreting the results with extreme caution,” the report said.
LEGAL BATTLES
Although Momot and Walker’s code review was limited to devices in Washington, similar concerns dragged other states into protracted legal battles, forcing prosecutors to defend not only the breathalyzer but also how it’s configured.
But the line between Draeger’s source code and each state’s configuration is blurry, making it difficult to know who is responsible for incorrect results.
Draeger said in an email that the “calibration and adjustment procedures depend on the instrument, additional equipment and materials, and the persons performing these procedures.” When asked about the guardrails put in place to prevent calibration errors, the company said, “only trained and certified personnel perform special instrument certification procedures.”
Washington State Patrol said the device produces accurate results, even without certain sensors installed.
If source code gets into the wrong hands, the damage would be incalculable.
Draeger’s breathalyzer is widely used across the US, including in California, Connecticut, Massachusetts, New Jersey, and New York. It’s often the only breathalyzer used in the states where they were bought.
In both New Jersey and Massachusetts, defense lawyers raised concerns. By acquiring the devices used by the states, lawyers commissioned engineers to analyze the code who say they found flaws that they say could produce incorrect results.
But defense teams in both states largely failed to stop their state governments from using the devices, public records show.
New Jersey’s top court found in 2008 that a similar Alcotest breathalyzer — said to use the same underlying algorithms as the Alcotest 9510 — was “generally scientifically reliable” and can be used with some configuration changes. One such change was to adjust the breathalyzer’s results for women over age 60 — who often aren’t able to produce the minimum breath volume of 1.5 liters required for a test. But defense lawyers argued that these changes were never put into place.
The same court ruled five years later that the breathalyzer “remains scientifically reliable, and generates results that are admissible” in court.
In nearby Massachusetts, a scandal that blew up in 2017 involving alleged failings in the breathalyzer threw thousands of prosecutions into disarray, because “all but two of the 392 machines” examined in the state had not been properly calibrated.
A district judge ruled that breath test results from miscalibrated devices for two years prior to September 2014 were “presumptively unreliable,” said Joe Bernard, a defense attorney who led the case against the Alcotest 9510 in Massachusetts.
Bernard, and his colleague Tom Workman, a computer forensic expert who later trained as a lawyer and consulted on the case, obtained the state’s source code and produced a report.
In a phone call, Workman criticized the Draeger breathalyzer, arguing that it can produce widely inflated results. One section of his report claimed the device had a litany of programming errors, including code that — like in Washington — apparently fails to correct for fuel cell fatigue.
But the court rejected the findings and found the source code still produced sound scientific results.
“THROW CAUTION TO THE WIND”
While legal battles were ongoing, Washington waited to push ahead with its deployment, but the ruling in New Jersey case in 2008 was seen as a vote of confidence.
Almost a year later, Washington State Patrol’s toxicologist said in an email seen by ZDNet that the police department should “throw caution to the wind” to deploy the device to police officers across the state without commissioning an independent source code evaluation — though she recommended confirming with the chief of police.
When asked whether an independent evaluation was ever commissioned, a Washington State Patrol spokesperson would not comment further and referred back to the legal filings in the case.
A later email in 2015 confirmed that the Washington State Patrol “never commissioned” an independent evaluation.
Moses Garcia, a former Washington state prosecutor who now works for a non-profit providing local governments in the state with legal advice, said in an email that the earlier breathalyzer in the New Jersey case had already been deemed admissible, and that the newer Alcotest 9510 uses the “same basic algorithms and formulas” as its predecessor.
The former prosecutor criticized the defense’s discovery effort as “speculation.”
“In adopting and approving the [Alcotest 9510], the Washington breath alcohol program exceeds, by far, the scientific standards accepted in the scientific community for breath test instrument validation,” he said.
Five years after the contract was signed, Washington State Patrol began deploying hundreds of Draeger breathalyzers in 2014 — sparking interest from defense attorneys in the state.
Not long after, defense attorneys in the state sought access to the devices.
Lantz was granted access to the source code used for Momot and Walker’s code review by a local county court. In one of several recent phone calls with ZDNet, he recounted how he set out to see if there were problems with the state’s device.
“We thought we would find something but nothing like this,” he said.
SETTLEMENTS AND SETBACKS
Hundreds of DUI lawyers descended on Las Vegas in mid-2017 for their annual gathering.
At the event, the two researchers shared their findings, which claimed the Alcotest 9510 having a “defective design.”
Word spread quickly. Draeger sent the researchers a cease and desist letter claiming defamation and alleging the two violated a protective order, designed to protect the source code from leaking.
Draeger and the researchers settled before a case was filed in court, avoiding any protracted legal battle. A legal case disputing the fine print of the order could have taken years to resolve.
Draeger said it “remains willing to provide the source code for use in other litigation in Washington, so long as a proper protective order is in place.”
Beyond a tweet by Walker pointing to a settlement statement on his site, there was little to indicate there had been any legal action against the pair.
The statement said that the two experts “never intended to violate the protective order” and denied any wrongdoing. But the two sides “agree” the draft report was based on incomplete data and not finished — and that “no one in possession of the report should rely on it for any purpose.”
We reached out to Walker with questions, but he referred only to the settlement statement on his company’s website, and he declined to comment further.
Draeger would not say why the settlement did not include a retraction on the report’s findings.
“There has not been an evidentiary hearing in Washington. If and when there is one, Draeger will cooperate fully,” a spokesperson said.
But Lantz paints a different picture. The defense attorney said he believes there “really was no technical violation of the protective order,” because the report didn’t disclose any source code.
“I do believe that [Draeger] is trying to interpret the protective order to be something that it’s not,” he said. “If we could go back in time, I would’ve asked that the report was not handed out — just because of the optics of it.”
Lantz said the protective order is vague, but contends it was framed to prevent the researchers from using the source code or their findings for commercial gain — effectively preventing Momot and Walker from using their knowledge to build their own competing devices. He believes the order gives Draeger near complete control over the code and anything the company deems “protected” information.
That’s when Draeger “began developing a strategy on how to block” the researchers’ report, said Lantz, because the company didn’t want the “pervasive exposure of these flaws.”
“I believe that interest of Draeger’s to protect their bottom line overlaps with the state’s interest to keep juries from hearing this information about the problems,” he said.
Draeger maintained that it is protecting its intellectual property. The company said in response that it “takes very seriously the proprietary nature of its source code,” and “protects proprietary information as a sound business practice,” which can include various types of communications or agreements for a particular matter.
Momot and Walker are no longer involved with the case, but Sam Felton, a Washington-based software engineer, is set to conduct another review of the Alcotest 9510 code. When contacted, Felton would not speak in specifics about his findings to date, citing his own protective order, except that he found things in the code that caused him “to have concerns.”
And Lantz, now at a new law firm, is working on starting discovery proceedings in neighboring King County, home of Seattle, the largest city in the state.
