November 1, 2019 Vol. 2, No. 11

Sifting the Ashes

The government’s star witness had a rough cross-examination.

(NY) His response to some tough questioning was truly embarrassing.

Bad science results in dropped arson charges for 2, overturned convictions for 2 others

(CT) Some non-reproducible chemical analysis put a big hole in the case.

Judge dismisses Warren County man’s home arson case

(OH) Independent testing paid for by the court refutes insurance company’s expert.

Guest Column:

Forensic science isn’t ‘reliable’ or ‘unreliable’ – it depends on the questions you’re trying to answer

Claude Roux, Distinguished Professor of Forensic Science, University of Technology Sydney

Disclosure statement

Claude Roux receives funding from the Australian Research Council including Linkage grants with the Australian Federal Police, the Victoria Police, the NSW Health Pathology and Rofin Australia Pty Ltd, and from the US National Institute of Justice. He is the President of the International Association of Forensic Sciences (2017-2020), current Vice-President of the Australian Academy of Forensic Sciences, the immediate Past-President of the Australian & New Zealand Forensic Science Society (ANZFSS) and a Fellow of the Royal Society of New South Wales. He also serves on the Scientific Advisory Board of the International Criminal Court.

After recent criticism in the US and the UK, forensic science is now coming under attack in Australia. Several recent reports have detailed concerns that innocent people have been jailed because of flawed forensic techniques.

Among the various cases presented, it is surprising that the most prominent recent miscarriage of justice in Victoria did not rate a mention: the wrongful conviction of Farah Jama, who was found guilty of rape in 2008 before the verdict was overturned in 2009.

This omission is not entirely unexpected. The forensic evidence in the case against Jama was DNA. Despite this fact, the recent media comments have re-emphasised the view that DNA is the gold standard when it comes to forensic techniques. Justice Chris Maxwell, president of the Victorian Court of Appeal, said:

with the exception of DNA, no other area of forensic science has been shown to be able reliably to connect a particular sample with a particular crime scene or perpetrator.

How can the same technique simultaneously be the forensic gold standard and contribute to such a dramatic miscarriage of justice? Is forensic science so unreliable that none of it should be admissible in our courts? Of course not, otherwise the criminal justice system would be left relying on much less reliable evidence, such as witness statements and confessions.

Read more: Get real, forensic scientists: the CSI effect is waning

Evidence in context

It makes no sense to assess the reliability of any forensic technique in the abstract. A forensic method is only “reliable” as far as it helps answer the particular questions asked in the context of a particular case. Asking the wrong questions will undoubtedly deliver the wrong answers, even if the best and most fully validated forensic method is applied.

Conversely, some forensic methods are perceived by some commentators to have less intrinsic value or even questionable reliability. But these methods might yield the answer to a crucially relevant question.

A typical example would be an incomplete shoe mark of poor quality left at a crime scene. It might not be possible to assign this mark to a specific shoe, but it might be enough to exclude a particular shoe or to identify the direction in which the perpetrator walked.

Forensic science is much more than merely applying methods or conducting tests – success also depends on the ability to identify and answer a relevant question.

A forensic science system is not like a clinical laboratory, processing samples and producing results for prescribed tests. Rather, good forensic science requires collaboration between investigators, scientists and other stakeholders. The focus should be resolving judicial questions using a scientific approach.

What matters most is the detection, recognition and understanding of the traces left by individuals during an alleged crime. This a much more complex issue than simply deciding whether or not a particular forensic method is deemed “reliable”.

Complex process

Forensic science is much less cut-and-dried than television dramas might suggest. When a DNA swab or a shoe mark lands on a forensic scientist’s lab bench, it has already gone through many steps, each with their own uncertainties.

These uncertainties are unavoidable, because forensic traces typically represent the aftermath of a chaotic event. The only option is to manage these uncertainties through a better understanding of how these traces are generated, persist, degrade, interact with each other, and how the information they hold can be interpreted.

The debate about the reliability of forensic science is not new. It illustrates a more fundamental issue: the lack of understanding of forensic science among the general public (who are potential jurors), and even among highly reputable law practitioners and non-forensic scientists.

Legacy of reform

The high-profile 2009 US National Academy of Sciences report and the 2016 Obama Administration report, both of which criticised some uses of forensic evidence, prompted an international reaction and several reviews of forensic practices.

They justified more empirical research to support some forensic conclusions. These improvements have been occurring in Australia for some years under the leadership of the National Institute of Forensic Science and through several academic research programs. And the recent UK House of Lords enquiry into the state of forensic science in England and Wales identified the Australian forensic science model as a leading example.

However, these reports excluded crime scene management from the scientific domain. They provided limited guidance about the challenging topic of interpretation of forensic evidence. This is disturbing because these are the two areas that require most attention if we are serious about improving forensic science outcomes. As the recent media coverage has shown, evidence interpretation remains a sore point between the legal and scientific communities. Where is the boundary of the responsibility of science versus the law? The fact that the legal community poorly understands forensic evidence is undoubtedly a shared responsibility. Shifting the blame onto forensic science will only exacerbate the problem.

If we think this is all too hard with traditional physical evidence, how does the criminal justice system expect to cope with our rapidly evolving digital society? Digital evidence is typically harder to assess than physical evidence in terms of volume, variety, rapidity, and privacy issues.

Better education, research and collaboration will form a large part of the answer. They will induce a better understanding of forensic science and its fundamental principles, so it can serve justice with confidence.

Reprinted with permission from The Conversation,

Case Study of the Month

United States v. John Michael Weber1

The Weber family lived in a mobile home in Hebron, Indiana. John Michael Weber was a short-haul truck driver, generally making trips of four or five hours in each direction. He was not home on June 30, 1994, when his house caught fire and his entire family was killed. The fire occurred at 4:30 in the morning, and at 7:00 a.m., he was seen more than 200 miles away, making a delivery of ethanol. His truck was seen the night before in the same town in Wisconsin where he made the delivery. He was not anywhere near his home in Hebron.

The fatal fire was investigated by the Indiana State Fire Marshal’s Office, which declared, after observing “flammable liquid pour patterns” throughout the fully involved mobile home, that the fire had been intentionally set. The report stated:

All accidental causes were eliminated. There are liquid accelerant burn patterns in all rooms except the three bedrooms. The pattern starts at the south end bathroom. In this room the linoleum floor covering is burned away and the wood flooring is charred. Found in this room was melted copper piping indicating intense heat at least 1980–2000 degrees…

The aluminum strip, which divides the kitchen and north hallway has melted, indicating at least 1220 degrees at the floor level. Also near this strip there is a hole burned through the floor….

Many samples were taken and have been sent to the lab to determine what accelerant may have been used.

All 14 samples, some of which were selected with the aid of a canine, tested negative for the presence of detectable amounts of ignitable liquid residues.

The author became aware of the case two years after the fire, when contacted by a reporter from the local newspaper. She was inquiring about arson fires and asked if I would look at some photographs and autopsy reports. Two of the photographs are shown in Figures 1 and 2. After reviewing this information, I advised the reporter that I could not determine the cause of the fire from this amount of data but that there was no evidence to indicate that the fire was anything other than accidental. I advised her that I had seen several cases where fire investigators had erroneously interpreted radiant heat damage as evidence of ignitable liquids. The mobile home had certainly sustained damage from full room involvement, rendering a visual interpretation of “pour patterns” invalid. An article appeared in the Vidette Times newspaper and that should have been the end of it.2 The reporter told me that the father wanted someone prosecuted. He believed the investigators who called the fire incendiary and was upset with them for failing to identify a suspect. My advice to him was to calm down and stop criticizing the police, lest he find himself charged with setting the fire.

Figure 1. Photograph of the Weber mobile home in Hebron, Indiana, immediately after the fire.

After another two years had passed, I received a call from an ATF agent (not a certified fire investigator), requesting a copy of my file on the case, and I told her I did not have

Figure 2. Another view of the Weber residence.

such a file. Later, however, I did remember that I had six photographs and the autopsy reports, and advised the agent of this, to which she replied that she would send a subpoena. I would have been more than happy to mail the contents of the file up to Indiana, but when the subpoena arrived, it required me to personally deliver the documents to the courthouse. A day or two later, I was contacted by attorney John Maksimovich, who advised me that he was representing Michael Weber in a federal arson trial. It seems that the agent had decided that Weber was an abusive husband and had concocted a theory that required him to steal a car or somehow otherwise transport himself back home in the middle of the night, set the fire, and then be back in Wisconsin the next morning to deliver the truckload of ethanol. The agent had convinced an Assistant U.S. Attorney to base a prosecution on this implausible version of events, despite the fact that two senior ATF-certified fire investigators had told both the agent and the AUSA that there was no credible way that they could prove that this was a set fire. Hence, some five years after the fire, John Michael Weber was accused of setting it. (He had previously passed a polygraph exam administered by the Indiana State Police.)

I was advised by defense counsel that my appearance to deliver the subpoenaed documents, which the prosecutor already had in her possession, was unnecessary, and he retained me as his expert. I received a voice mail from the AUSA on the morning of the hearing, however, telling me that I would be held in contempt for not showing up, but when counsel moved to have me found in contempt, the judge advised her that it would be necessary to file a brief. Before the week was over, the U.S. Attorney (who has since become a Federal Judge) learned from defense counsel that the AUSA had been advised not to proceed by two respected government certified fire investigators but had chosen to ignore that advice. In fact, the AUSA had told one of the CFIs that she was “not interested” in hearing his opinion. Seeing the quality of the evidence his assistant proposed to present, the U.S. Attorney filed a joint motion with defense counsel to dismiss the case. The AUSA, when asked to comment on the criticism from the ATF agents, stated, “They are not saying it was not arson. They’re simply questioning the way the conclusion was reached.”3

Error Analysis

Most of the errors in this case occurred because people who were unqualified to conduct fire investigations assumed control of the case. The AUSA chose to ignore the inconsistent data in the form of alibi evidence. Presented with a critical review from colleagues of the agent who “made” the case, she chose to ignore their sound advice. The original investigators misinterpreted irrelevant data in the form of the radiant burn patterns.


This case illustrates the awesome power that our criminal justice system provides to prosecutors in the form of “discretion.” “The prosecutor has more control over life, liberty, and reputation than any other person in America.”4 Even when faced with negative reports from two senior law enforcement officers who were experts at fire investigation, the prosecutor in this case used that “discretion” to bring an evidence-free charge against Mr. Weber.

Even though the case against him was dismissed, he had no way to recover the significant legal expenses that this misguided prosecutor caused him to incur. Even after his release, Michael Weber still believed the fire was incendiary.5

In December 2018, I reviewed the case of Pennsylvania v. Michele Black. (Sifting the Ashes, Volume 1 #6). This was another case of a prosecutor abusing the civil rights of an innocent citizen. The prosecutor in that case had offered to allow Mrs. Black to plead to a “summary offense,” involving no jail time, no fine, and no probation. When she refused to admit to doing something that she didn't do, the prosecutor responded by charging her with six felonies. It's almost like he did not want her to exercise her right to a trial.

The prosecutor in the Weber case also thought it was perfectly fine to issue a subpoena to me to bring her copies of material she already had, and travel to Hammond, Indiana on my own dime. I felt a little put out by her demands, and especially her threat to find me in contempt of court, but I was assured by the DOJ’s Office of Professional Responsibility in Washington that such conduct was perfectly within the prosecutor's “discretion.” Not only are prosecutors granted this wide-ranging power, there is next to no accountability for it. Prosecutors who have been found to have caused wrongful convictions are immune from being sued by their victims even if they engaged in deliberate misconduct. The case of Imbler v. Pachtman, 424 U.S. 409 (1976) provides prosecutors with absolute immunity from civil suits alleging misconduct.6 (Fire investigators, however, enjoy no such immunity.)

Fortunately, the vast majority of prosecutors understand that it is their duty to do justice rather than to just get convictions, and fortunately most of them do not act like the attorney that handled the Weber case. “The primary duty of the prosecutor is to seek justice within the bounds of the law, not merely to convict.”7


1 United States of America, Plaintiff v. John Michael Weber, Sr., Defendant, United States District Court, Northern District of Indiana, Hammond Division, Case No.: 2:98 CR 195.

2 Brown, S. and Walsh, S., What killed the Webers: Arson or Accident?, The Vidette Times, Valparaiso, IN, June 30, 1996, 1.

3 Brown, S., Judge Frees Father in Fatal Arson Case, The Vidette Times, Valparaiso, IN, February 23, 1999, 1.

4 Robert H. Jackson, Attorney General of the United States, address to the second annual conference of United States Attorneys , April 1, 1940 in the Great Hall of the Department of Justice Building in Washington, D. C.

5 Kosky, K., Weber Family: Mike Weber's life unraveled in the 15 years since his wife and four children perished in a Hebron arson fire, The Vidette Times, Valparaiso, IN, August 17, 2009.

7 American Bar Association, Criminal Justice Standards for the Prosecution Function,

Can you withstand a 1033 challenge?

Many certified fire investigators are being challenged not on their opinions or methodology, but on their qualifications. Cases are being dismissed or settled for far less than they are worth because adverse counsel have learned to use NFPA 1033 do design a simple quiz. If you fail that quiz, sponsoring counsel may decide to settle for less, or worse, a judge may decide that despite your many years of experience or your certification from NAFI or IAAI. You have failed to demonstrate that you are a qualified expert. This article by Attorney John Reis in Claims Magazine lays out the problem, and

includes sample quiz questions.

If you find these test questions problematic, the solution is a refresher course on the 16 subjects required by NFPA 1033, all of which are covered in depth in Scientific Protocols for Fire Investigation. I have distilled the book into a three-day presentation with over 2000 slides that I can bring to your venue whenever you are ready. For more information on getting up to speed, go to

Subscribe to Our Newsletter

firstname :