Health Data Collection Is Often Substandard

By Sondra Forsyth

Most clinical registries that collect statistics on patient outcomes are substandard and lacking in features that could make them useful for patients, doctors and policy makers, according to a study.

The investigation, by researchers from the Johns Hopkins University School of Medicine, was published in the Journal of Healthcare Quality. The study revealed poor data monitoring and reporting, which hurt efforts to study the disease, guide patients in their choice of treatments, and formulate effective health policies.

“Our results highlight the acute need to improve the way clinical outcomes data are collected and reported,” said senior investigator Marty Makary, M.D., M.P.H., professor of surgery at Johns Hopkins. “Failure to measure and accurately track patient outcomes remains one of the greatest problems in modern health care, curtailing our ability to understand disease, evaluate treatments and make the health-care industry a value-driven marketplace.”

Clinical registries are databases of patient outcomes developed and maintained by medical organizations and medical specialty groups.

To evaluate the quality of clinical registries, Makary and colleagues say they created “a registry of registries” to study the way the health care industry measures its performance. “We found it’s the Wild West,” Makary said. “With a few notable exceptions, most registries are underdeveloped, underfunded and often are not based on sound scientific methodology.”

According to a news release from Johns Hopkins, the investigators looked at 153 U.S. clinical registries containing health service and disease outcomes data. On average, a registry contained information on more than 160,000 patients treated across more than 1,600 hospitals.

Less than one-quarter of registries adjusted their results for differences in disease complexity — information statistically reflective of disparities in illness severity and socio-economic status among patients treated across hospitals. Unadjusted data, the researchers say, could be misleading and should be interpreted with great caution.

Less than one-fifth of registries contained independently entered data — information entered by clinicians other than the ones involved in care. Independently entered data is an important principle in eliminating the well-established bias of self-reported data, the researchers say.

The investigators also said, though, that several organizations had excellent registries, with carefully analyzed statistics that were reported and audited in a meaningful way. These organizations included the United Network for Organ Sharing; the National Surgical Quality Improvement Program, maintained by the American College of Surgeons; and the National Cardiovascular Data Registry of the American College of Cardiology.

“These organizations’ databases illustrate the power and potential of clinical registries to improve patient outcomes and inform best practices,” said study lead author Heather Lyu, a research fellow at Hopkins. “And if we really want to get serious about measuring and improving performance, we need to develop criteria that will help others run similarly successful registries.”

The hallmarks of a good registry, the authors say, include:

Data accounting for differences in patient case complexity across hospitals that allows for meaningful comparisons of outcomes

Broad hospital participation

Measurement of complications that matter to patients and affect their quality of life

Independent data collection that eliminates the bias inherent in self-reporting

Public reporting and open access to hospital performance for taxpayer-funded registries