Evaluability of the Effect of Oncology Center Certification

An editorial to accompany the article: “Initial Cancer Treatment in Certified Versus Non-Certified Hospitals – Results of the WiZen Comparative Cohort Study” by Jochen Schmitt, Monika Klinkhammer-Schalke, et al. in this issue of Deutsches Ärzteblatt International

Based on the evaluation of health insurance data and data from selected clinical cancer registers, Schmitt and Klinhammer-Schalke et al. (1) report that the certification of oncological centers in Germany is associated with a more or less strong survival benefit for eleven cancer entities. This is a very extensive non-interventional study in which the authors have attempted to take some confounding variables into consideration. For ethical and pragmatic reasons, it would not have been feasible to undertake a randomized controlled study. In this respect, the WiZen project is an important and commendable study.

Weak evidence base

The evidence base to date on the importance of center certification for outcomes of care is thin. The Cochrane Library contains only one systematic review from 2016, which produced only very weak evidence for any benefit (2). On the other hand, there are numerous systematic reviews and meta-analyses that, despite some shortcomings, suggest that the outcome of the surgical care of cancer patients is positively associated with hospital case volume and surgeon volume (3, 4). The results produced by Schmitt and Klinhammer-Schalke et al. therefore appear understandable, that is: Even though certification is not necessarily to be equated with treatment volume, the WiZen study does produce a significant correlation between hospital size (number of beds) and certification status.

Risk of systematic bias

Unlike a randomized controlled study, observational studies are considerably more susceptible to systematic errors (bias), especially with regard to clinical issues, due for example to

• selection and information bias
• low resolution of routine data from the statutory health insurance system (SHI)
• confounding due to unmeasured confounders (for example, the exact stage of the disease, histology, unexpected perioperative risk)
• very poorly measured confounders.

The authors interpret their results with much enthusiasm. Even though the results appear plausible, the critical reader cannot fully appreciate the almost unreserved optimism associated with expressions used by the authors, such as: “very high degree of robustness”, “reliable and valid”, “high degree of external validity”, and “fundamentally applicable”. SHI data contain neither information about histology (morphology and grading) nor about TNM or UICC stages (UICC, Union internationale contre le cancer) of cancer diseases. These variables are, however, the most established prognostic factors. If these factors are unequally distributed between patients treated at certified centers (Z1) and those treated at non-certified centers (Z0), then confounding will develop. The authors were not even able to adjust for the variables morphology and grading in the SHI data.

Examples of confounding

The example of primary neoplasms of the pancreas can be used to illustrate the problem of failing to take into consideration morphology in North Rhine-Westphalia (NRW): About 7% of all primary neoplasms of the pancreas are neuroendocrine neoplasms (NENs). The relative 5-year survival probability in the affected patient group is 65%. However, it is only 11% for patients with adenocarcinoma (5). NENs require diagnostic clarification by positron emission/computed tomography (PET/CT) using specific radiopharmaceutical markers. These imaging modalities are much more likely to be available at certified centers. Treatment is also complex and more commonly undertaken at certified centers. Certified centers therefore treat NENs at a markedly higher percentage than non-certified centers. According to our own extrapolations for NRW for a quantitative bias analysis (6), the proportion of pancreatic NENs in stage Z1 is about 28%, whereas this proportion is only 8% for stage Z0, so this selection bias alone could explain the reported positive center effect for pancreatic cancer. Unfortunately, adjustments were not made for the variable morphology in the analyses based on data of clinical cancer registers (CCR) either.

Evaluation of the AOK data only took distant metastasis into account (ICD-10: C78–C79)—provided that it was coded. On the other hand, adjustments were made for UICC stage in the analysis of CCR data. However, no adjustments were made here for the variable morphology. Furthermore, with the exception of gynecological tumors, the percentage of missing data for the UICC stage in the CCR data was at least twice as high for Z0 patients with all other entities as for Z1 patients (for example, colon/rectum, Z0: 17.3%, Z1: 6.8%; lungs: Z0: 13.9%, Z1: 3.4%). This makes any statistical control for anticipated confounding due to the unequal distribution of the UICC stages more difficult.

Structural inequality

The example of malignant neoplasms of the pancreas shows the enormous structural inequality between Z0 and Z1 patients: In the WiZen study, 60% of the Z0 patients were treated in very small hospitals (1–299 beds), but only 2% of the Z1 patients. The proportion of over 80-year-olds was 29% amongst the Z0 patients and 23% amongst the Z1 patients (7). Although the authors made adjustments for these structural inequalities, they are only the tip of the iceberg when prognostically important variables such as severity level of comorbidities, perioperative risk, ECOG Performance Status are considered, which could not be, or were only inadequately, adjusted.

The consequence of a change of the cancer patient’s health care provider is also uncertain. What is the impact of a patient with cancer undergoing only diagnostics or also primary therapy in a certified center and subsequently being treated further in a non-certified facility for adjuvant and follow-up therapies? It is quite possible that such changes led to an underestimation of the true effect of certification.

Open questions

The extent to which the bias potentials mentioned can change, or even nullify, the observed effects of certification on the survival of patients with cancer cannot be answered. Of course, it is quite possible that there is still a center or certification effect. These methodological issues would be reason enough to be more cautious about drawing political conclusions. In this respect, the political tone of the publication, which is unusual for a scientific article, is surprising. It is reflected, among other things, in the reference to the positive evaluation of the WiZen study by the Innovation Committee of the Federal Joint Committee (G-BA), in the extrapolation of the potential years of life saved, and in the prospect that the government commission might consider the results of the WiZen study.

Conflict of interest statement
The author declares that he no conflict of interest exists.

Manuscript received on 2 August 2023, revised version accepted on 2 August 2023.

Translated from the original German by Dr Grahame Larkin MD

Prof. Dr. med. Andreas Stang, MPH
Institute of Medical Informatics, Biometrics and Epidemiology, University Hospital Essen
Hufelandstraße 55, 45147 Essen

Cite this as:
Stang A: Evaluability of the effect of oncology center certification.
Dtsch Arztebl Int 2023; 120: 645–6. DOI: 10.3238/arztebl.m2023.0184

References

1. Schmitt J, Klinkhammer-Schalke M, Bierbaum V, Gerken M, Bobeth C, Rössler M, Dröge P, Ruhnke T, Günster C, Kleihues-van Tol K, Schoffer O, on behalf of the WiZen Study Group: Initial cancer treatment in certified versus non-certified hospitals: results of the WiZen comparative cohort study. Dtsch Arztebl Int 2023; 120: 647–54. DOI: 10.3238/arztebl.m2023.016

2. Flodgren G, Goncalves-Bradley DC, Pomey MP: External inspection of compliance with standards for improved healthcare outcomes. Cochrane Database Syst Rev 2016; 12: CD008992

3. Gruen RL, Pitt V, Green S, Parkhill A, Campbell D, Jolley D: The effect of provider case volume on cancer mortality: systematic review and meta-analysis. CA Cancer J Clin 2009; 59: 192–211

4. Morche J, Mathes T, Pieper D: Relationship between surgeon volume and outcomes: a systematic review of systematic reviews. Syst Rev 2016; 5: 204

5. Stang A, Wellmann I, Holleczek B, et al.: Incidence and relative survival of pancreatic adenocarcinoma and pancreatic neuroendocrine neoplasms in Germany, 2009–2018. An in-depth analysis of two population-based cancer registries. Cancer Epidemiol 2022; 79: 102204

6. Fox MP, MacLehose RF, Lash TL: Applying quantitative bias analysis to epidemiologic data. 2nd ed. Cham: Springer; 2021

7. Schoffer O, Rößler M, Bierbaum V, et al.: Ergebnisbericht: Wirksamkeit der Versorgung in onkologischen Zentren. www.innovationsfonds.g-ba.de/downloads/beschluss-dokumente/268/2022–10–17_WiZen_Ergebnisbericht.pdf: Technische Universität Dresden 2023.