Despite earlier studies that showed a 50 percent reduction in deaths by using such checklists for surgeries, these researchers found no significant difference in the rate of deaths and complications before and after the checklists were introduced.
The checklists were designed to get the surgical team to work together and to take the time to be sure of the patient's identity, the procedure to be done and what equipment will be needed. Also included was an accounting of all the surgical instruments after the operation.
"We couldn't identify a measurable improvement with checklists," said lead researcher Dr. David Urbach, a professor of surgery and health policy at the University of Toronto.
One problem might be mandating the use of a checklist, as was done in Canada, without getting hospitals to buy into the program, he said.
"It's one thing if hospitals develop these things on their own, where people are motivated and engaged," Urbach said. "That can transform a culture and make it more safety-conscious. If it is mandated, it may not achieve the same result."
It could also be that the first studies done on the benefits of checklists painted too rosy a picture of their benefit, Urbach said. "It may not be possible to achieve the results detected in those studies," he said.
Those first studies suggested that checklists could cut the risk of dying in half. "That's a really big effect," Urbach said. "Very few things reduce something by 50 percent."
Urbach said he still believes in the value of checklists as a way of bringing the surgical team together to talk about the patient. "There is probably a lot of value in checklists, even if it can't be measured in terms of reducing the risk of bad outcomes," he said.
The report was published in the March 13 issue of the New England Journal of Medicine.
Dr. Lucian Leape, an adjunct professor of health policy at the Harvard School of Public Health, wrote an editorial accompanying the study in the journal. "The surgical checklist is a powerful tool for reducing complications and deaths, but it only works if you use it right," Leape said.
Implementing the use of a checklist is not easy, he said. "It requires people to change a lot of the ways they do things. It's difficult to do, and it takes time to see the effects," he said.
The checklist most commonly used is one created by the World Health Organization. Its list is divided into three parts.
The first part includes steps to be taken before anesthesia, including verifying the patient's identity, medical condition and the procedure.
The second part, called a "timeout," takes place before the first incision. During this time, everybody introduces themselves. The surgeon and others confirm what they are going to do and check that all necessary medications and any special patient needs are in place.
Finally, before the patient leaves the operating room, all instruments and sponges are accounted for, any blood for lab tests is labeled and any problems are discussed. The doctor reviews any concerns about the patient's recovery and management.
Leape said the WHO checklist is a guide, and each hospital needs to adjust it to make it work for them. "The point is to make it your own," he said. "In the process of making it your own, you really develop a lot more teamwork, and that's what I think really makes a difference."
The problem in all areas of patient safety is how to get hospitals to improve their performance, Leape said.
"We are still looking for the answer for that," he said. "It's a culture change, and culture change has to be driven by the leadership."
He said some combination of incentives and sanctions might be needed.
For the study, Urbach and his colleagues surveyed all hospitals that perform surgery in Canada's Ontario Province. They took data on deaths and surgical complications, comparing the three months before and the three months after checklists were introduced.
During the three months before checklists were used, there were about 109,000 procedures. In the three months after checklists were put in place, there were about 106,000.
The researchers found that before the use of checklists, 0.71 percent of surgical patients died in the hospital or in the month after their operation. After the checklists were introduced, 0.65 percent of patients died.
The rate of complications was 3.86 percent before the use of checklists and 3.82 percent after, they said.
Visit the U.S. National Library of Medicine for more on patient safety.