A Complex Intervention to Prevent Medication-Related Hospital Admissions

Results of the Stepped-wedge Cluster Randomized Trial KiDSafe in Pediatrics

Background: Children are often treated off-label and are at a disadvantage in pharmacotherapy. The aim of this study was to implement and evaluate a quality assurance measure (PaedPharm) for pediatric pharmacotherapy whose purpose is to reduce medication-related hospitalizations among children and adolescents.

Methods: PaedPharm consisted of the digital pediatric drug information system PaedAMIS, pediatric pharmaceutical quality circles (PaedZirk), and an adverse drug event (ADE) reporting system (PaedReport). The intervention was implemented in a cluster-randomized trial (DRKS 00013924) in 12 regions, with a pediatric and adolescent medicine clinic in each and a total of 152 surrounding private practitioners, in 6 sequences over 8 quarters. In addition to the proportion of ADE-related hospital admissions (primary endpoint), comprehensive process evaluation included other endpoints such as coverage, user acceptance, and relevance to practice.

Results: 41 829 inpatient admissions were recorded, of which 5101 were patients of physicians who participated in our study. 4.1% of admissions were ADE-related under control conditions, and 3.1% under intervention conditions (95% CI: [2.3; 5.9] and [1.8; 4.5], respectively). A model-based comparison yielded an intervention effect of 0.73 (population-based odds ratio; [0.39; 1.37]; p = 0.33). PaedAMIS achieved moderate user acceptance and PaedZirk achieved high user acceptance.

Conclusion: The introduction of PaedPharm was associated with a decrease in medication-related hospitalizations that did not reach statistical significance. The process evaluation revealed broad acceptance of the intervention in outpatient pediatrics and adolescent medicine.

When it comes to drug therapy, children are at a disadvantage in terms of safety and proof of efficacy. Between 42 and 90 percent of children and adolescents are treated with drugs off-label in the inpatient setting and between 46 and 64 percent in the outpatient setting (1, 2, 3, 4, 5, 6). Reasons for this include a lack of clinical studies due to methodical and ethical challenges in the pediatric subpopulation and the limited economic interest of pharmaceutical companies (7, 8, 9). Furthermore, the off-label use of drugs is associated with an increased risk of adverse drug events (ADEs) (10, 11, 12, 13, 14).

About three to five percent of all hospitalizations involving children are the direct result of an ADE (11, 12, 15). A study from the UK shows that 22.1% of these were avoidable and should therefore be regarded as medication errors (MEs) (11).

Children are excluded from the monitoring system for approved medications when it comes to off-label use. The continuous spontaneous reporting of ADEs by attending doctors is therefore an important instrument to generate drug safety signals, especially in pediatrics.

Despite the increase in drugs approved specifically for children as a result of the requirements of the EU Pediatric Regulation, off-label use remains a challenge for pediatric pharmacotherapy, now and in the future. Not every off-label use is associated with the same risks (16, 17). That is why the ready availability of existing evidence and knowledge of the special features of pediatric pharmacotherapy are particularly important.

The aim of the project “KiDSafe—Improving drug therapy in children and adolescents by increasing medication safety” was to lower the proportion of ADE-related hospitalizations by introducing a structured quality assurance measure (PaedPharm), comprising a pediatric drug information system (PaedAMIS), amongst other things, and to improve pediatric medication safety.

The present article summarizes the key aspects of the study. Additional available information and results may be found in the publicly accessible outcome report (18).

Methods

PaedPharm

PaedPharm is a model of care comprising three modules (PaedAMIS, PaedZirk, and PaedReport) to provide and train healthcare professionals with evidence-based information about drug therapy in children and adolescents (Box).

Box

The components of the intervention PaedPharm

Study design and implementation

PaedPharm was developed and evaluated during the present study (DRKS—German Clinical Trials Register DRKS 00013924) in accordance with current international recommendations of the Medical Research Council UK for complex interventions (19, 20). Approvals were obtained from the responsible ethics committees prior to the start of the study.

The present publication is oriented on the CONSORT guideline for stepped wedge cluster randomized trials (cRCT; [21]). The intervention was implemented throughout Germany in 12 regions (clusters) together with specialist doctors in private practice for pediatrics and adolescent medicine, child psychiatry, and general medicine, in a stepped wedge cluster randomized trial using a cross-sectional design. Each cluster consists of a pediatric and adolescent medical clinic (university hospital or municipal hospital) and their regionally referring participating specialists in private practice.

PaedPharm was introduced in a stepwise manner in six different sequences (Figure 1). The twelve clusters were randomly allocated to the six sequences (eMethods section). After a three-month training phase (T), two clusters each changed on a quarterly basis from the control phase (C) to the intervention phase (I). Thus, clusters which had not yet been introduced to the intervention served as control regions for those clusters where introduction of the intervention had already taken place. At the start of the intervention phase, all participating doctors received exclusive access to PaedAMIS via a username and a password and were invited to regular quality circles.

Figure 1

Study design – time-based allocation of the various intervention phases in the sequences

Evaluation methods, endpoints, and sample size

Data to assess the benefits of PaedPharm were collected in the participating clinics. The primary endpoint was the proportion (%) of ADE-related hospitalizations in all non-elective hospital admissions where assignment to participants was possible. The intention was to reduce this proportion by one third (from 3% to 2%) (12). Calculated for 12 clusters, six sequences, a duration of seven quarters, and a power of at least 80%, it was planned to recruit 240 doctors (eMethods). As part of a quantitative and qualitative process evaluation, a secondary aim was to observe the implementation process and direct intervention aspects, for example, user acceptance.

Data acquisition and instruments for assessment

Between 01 July, 2018, and 30 June, 2020, each non-elective admission to the participating clinics which fulfilled the inclusion criteria was identified (Table 1), and age on admission, sex, medication history, and diagnoses (according to ICD-10) were documented.

Table 1

Inclusion and exclusion criteria at clinic and case levels

On establishing a positive medication history, a check was made for the presence of an adverse drug event (ADE). For this purpose, an algorithm designed specifically for the study to detect cases of ADE was used (22). Once an ADE was suspected and after obtaining consent from the parents, the case was transferred in pseudonymized form to the ADE coordination unit (ADE-C). The region of origin and, thus also, the intervention status were rendered unrecognizable to the coordination unit.

Suspected cases of an ADE without parental consent were forwarded to the Drug Commission of the German Medical Association (AkdÄ) after an initial causality check by the local study team (presence of at least a temporal relationship according to the WHO-UMC system [23]). Suspected ADEs were examined by both assessor authorities (ADE-C and AkdÄ) using established algorithms and definitively assessed (ADE suspicion “confirmed”/“not confirmed”; [23]). Training and regular exchange between the two assessor authorities ensured that evaluation of suspected ADEs was comparable between both authorities. The result was recorded on a local level (Figure 2). Hospital admissions due to medication errors (ME) were always rated as drug related. Cases were communicated to the evaluating institution on a monthly basis in pseudonymized, anonymized or aggregated form, depending on whether or not consent had been given.

Figure 2

Flow diagram of data acquisition process conducted in the clinics

Statistical analysis

For the primary endpoint, the proportion of ADE-associated admissions in all hospitalizations by the referring and participating private practitioners under control and intervention conditions was compared. A logistic model for correlated data was fitted to these data, taking into account the cluster-randomized stepped-wedge design (eMethods). The intervention effect is reported as an odds ratio with a 95% confidence interval. Missing data were replaced using a multiple imputation procedure (eMethods).

Process evaluation

Questionnaires asking about satisfaction with the quality circles (PaedZirk) and the number of spontaneous reports from participating practices (PaedReport) were used to describe the quality of implementation. At the end of the evaluation period, a standardized questionnaire was sent to all participating physicians asking about acceptance, satisfaction, relevance to practice, and sustainability of PaedPharm (eFigure 1).

eFigure 1

Results

The CONSORT flow diagram (eFigure 2) provides an overview of inclusion and continuation of all participating physicians and of all hospitalizations.

eFigure 2

CONSORT flow diagram of participating physicians and hospitalizations

Participating physicians

Out of 1386 physicians contacted, 160 (240 were planned) were prepared to participate in the trial (136 from pediatric and adolescent medicine, ten general medicine, 14 pediatric and adolescent psychiatry and psychotherapy). Eight doctors stopped participating even before introduction of the intervention, allowing cases from 152 doctors to be included (eFigure 3).

eFigure 3

Characteristics of the participating physicians and overview of the recorded patient cases

Hospital admissions

Altogether, 41 829 patient cases were included (eFigure 3), of which 5101 admissions were assigned to participating physicians. It was not possible to compile a participation status in 209 cases (control phase n = 106, training and/or intervention phase n = 103). In 55 cases, a definitive ADE assessment by the ADE assessor authorities (control phase n = 40, training and/or intervention phase n = 15) was not possible. These missing data (n = 264) were imputed for the regression analysis. Table 2 shows the characteristics of this identified patient population.

Table 2

Characteristics of the patient population (data from 10 of 12 clinics*)

Primary endpoint

Altogether 77/2497, 36/775, and 66/1829 ADE-related hospitalizations from participating physicians in control, training, and other intervention phases were identified, respectively. Thus, according to period-corrected estimates of the population-based mean and under control conditions, 4.1 % of admissions were ADE-related. This figure was 3.1% (95% confidence interval: [2.3; 5.9] and [1.8; 4.5], respectively) under intervention conditions. The model-based comparison resulted in a population-based odds ratio (= intervention effect) of 0.73 ([0.39; 1.37]; p value 0.33). A second post-hoc analysis of the primary endpoint was conducted to compensate for the too low number of participating physicians and to use the collected data more efficiently. All recorded non-elective hospitalizations were taken from control quarters and used for this analysis, which therefore also included those from non-participating doctors. This allowed 57% of all recorded admissions to be included in this analysis. After these adjustments and in the population-based mean, 3.7% of admissions were ADE related under control conditions, and 3.0% under intervention conditions. The model-based comparison resulted in a population-based odds ratio comparable to the main analysis (= intervention effect) of 0.80 ([0.52; 1.24]; p value 0.31).

Process evaluation—Pediatric drug information system—PaedAMIS

Altogether, 96 physicians participated in the final survey. Of these, 70% stated they were “partially” (n = 22/92) or “completely” (n = 42/92) satisfied with PaedAMIS (on a five-point Likert scale). Regarding relevance to practice, 40% (“fully or partially relevant”, n = 12 + 25/92) stated that PaedAMIS was a great help in daily work, while 63% of respondents (n = 57/91) said they consulted PaedAMIS less than once a week. Three participants (3/91) used PaedAMIS several times a day. Thirty-three of 81 (41%) respondents had subsequently altered their prescriptions in five to 10 percent of cases. For the majority of respondents, the information in PaedAMIS had resulted in at least one alteration in their prescribing pattern (n = 66/81).

Process evaluation—Quality circles—PaedZirk

During the whole project period, 45 of 72 scheduled quality circles were held in presence and, due to the pandemic, one as a live webinar. The mean number of participants was 7.3 person/invitation. Twenty-five scheduled quality circles had to be canceled because of the corona virus pandemic of 2020. The webinar invitation was offered for the final sequence because this one was particularly affected by contact restrictions (eFigure 4). The quality circles were considered relevant to practice by 81% (78/96) of the participants. Participation in the seminars changed their confidence in drug prescription for 67% (64/96). The quality circles were rated as “good” to “very good” in every category. On average, the participants also gave a grade 1.3 to the category “relevance to practice”.

eFigure 4

Overview of the participant numbers per quality circle by cluster values (n = 335)

Process evaluation—ADE reports—PaedReport

A total of 28 spontaneous reports were recorded during the evaluation period. With altogether 160 participating physicians, that was an annual reporting rate of 0.088 spontaneous reports per physician.

Process evaluation—Sustainability

Of 96 respondents, 72 would still use PaedAMIS and 73 would attend the quality circles after the end of the project. Half of them (n = 48) reported that something had changed sustainably with regard to processes of prescribing medications. Forty-five percent (n = 48) said that structures and/or processes pertaining to reporting ADEs had changed. Sixty percent (n = 58) said that PaedPharm had achieved a sustainable change in their awareness for ADEs. Sixty-six percent (n = 63) intended to report suspected cases of ADE more often in the future. Eleven percent (n = 11) of the respondents reported no change.

Discussion

The present study examined for the first time in a large patient population in Germany the benefit of a multimodal intervention for pediatric medication safety. A little more than four percent of all non-elective hospitalizations were due to ADEs, confirming the results of international studies (11). With regard to the benefit of PaedPharm, there was a relative risk reduction of −26%, which was close to the assumption of −33% made during sample size calculation.

The lack of statistical significance is primarily due to the small number of participating private practitioners (240 planned versus 152 effective participants). Only 11.5% of the approached physicians were prepared to participate in the healthcare study. The actual participation rate was similarly low to that of other studies (24). At 13%, the proportion of admissions from participants lagged behind the planning assumption of 40%.

Altogether 25 of 72 scheduled quality circles (35%) had to be canceled because of the coronavirus pandemic and the associated lockdown measures. The quality circles were an important component of the intervention, and this was possibly a reason for the somewhat weaker observed intervention effect (−26 %) in comparison with what had been assumed (−33 %). However, sensitivity analyses showed that the data of the last period contributed little to the overall outcome.

The process evaluation provided information indicating acceptance and benefit of the intervention. More than two thirds of participants confirmed that they intended to continue using Paed AMIS in the future. Paed AMIS was still in the development phase when the KiDSafe study was started, and only around 100 monographs were available. This number had already risen to 401 by the end of the project (June 2021). This included monographs which had been prioritized following discussions with the physicians in the quality circles. In the meantime, the pediatric drug information system is freely available to all healthcare professions under the name Kinderformularium.DE (www.kinderformularium.de) and is constantly being expanded and updated.

The available quality circles (PaedZirk) were regarded as extremely relevant to practice and were well attended. The personal participation of experts in the field of pediatric and adolescent pharmacotherapy was particularly appreciated. The results reveal the considerable importance of expertise in developmental pharmacology for appropriate pharmacotherapy. Regular further education courses together with appropriate expertise could therefore continue to play an important role in the future.

The spontaneous reporting of ADEs is of particular importance in pediatrics, given the common off-label medication use and associated lack of systematic monitoring following marketing authorization. It must be assumed that the high degree of off-label use together with its associated insecurity amongst doctors and anticipated litigation/legal consequences in the event of ADEs have a negative impact on the reporting rate. In Germany in 2008, the reporting rate for children was only 45% of that for adults (reports per million population; [25]). According to the annual report of the AkdÄ, 231 spontaneous reports for the age group children and adolescents up to the age of 18 years were registered in 2018 (26). During the same year, there were 5776 specialists for pediatric and adolescent medicine practising in Germany and 996 specialists for pediatric and adolescent psychiatry (27). Based on these figures, this results in an estimated annual reporting rate of 0.018 notifications per private specialist. The reporting rate during the KiDSafe study was 0.088, indicating an almost fivefold increase in the rate. That could demonstrate that PaedPharm succeeded in creating an increased awareness and attention for ADEs.

Strengths and weaknesses

It can be assumed that the study design and the selected primary endpoint resulted in a high internal validity of the results. The selected design allowed for adjustments for period effects. Thus, the anticipated increase in cases of suspected ADE once data acquisition had started could also be taken into consideration.

The identification of potential ADE-related hospitalizations was standardized by a clinical algorithm, and validation of suspected cases was achieved by independent, blinded experts (UAE-C) and the AkdÄ. The strict separation between implementation in private practices and the fully decoupled evaluation in clinics for pediatric and adolescent medicine also promised a high degree of validity, as did the institutional distinction between implementation and evaluation.

The lower-than-anticipated participation rate of physicians in private practice and the low consent rate for a high-quality assessment of suspected ADEs in a reference center were possibly obstacles for an empirical proof of efficacy of the intervention. Another factor was that restrictions imposed by the coronavirus pandemic prevented implementation of the project to the planned extent.

Summary and conclusions

With PaedPharm, a multimodal intervention to improve medication safety in children and adolescents was implemented and evaluated for the first time in Germany. The intervention resulted in a statistically not significant reduction of drug-related hospitalizations.

The process evaluation revealed the potential for PaedPharm to raise awareness for the correct use of drugs in children and adolescents and to improve the reporting rate of ADEs.

Other authors

Dorothée Malonga Makosi, Philipp Mildenberger, Marcel Romanos, Astrid Bertsche, Thilo Bertsche, Peter Dahlem, Karin Egberts, Bernhard Erdlenbruch, Stefanie Fekete, Ulrike Haug, Gerd Horneff, Axel Hübler, Wieland Kiess, Martina P. Neininger, Tim Niehues, Bernhardt Sachs, Karl-Florian Schettler, Filippa Schreeck, Tim Steimle, Tobias Wenzl, Stefan Wirth, Fred Zepp, Matthias Schwab

Affiliation of the other authors

eBox

Collaborators

Funding

The study “KiDSafe – Improving drug therapy in children and adolescents by increasing medication safety” was supported with funds from the Innovation Fund of the Federal Joint Committee (funding code: 01NVF16021).

The development of the drug database PaedAMIS was sponsored by the Federal Ministry of Health.

MS and FS were supported by the Robert Bosch Foundation, Stuttgart.

Data Sharing

The primary data cannot be made available for data protection reasons. After consulting the authors, it might be possible to access aggregated data.

Conflict of interest statement
The authors declare that no conflict of interest exists.

Manuscript received on 28 November 2022, revised version accepted on: 08 May 2023.

Translated from the original German by Dr Grahame Larkin MD

Corresponding author
Prof. Dr. rer. nat. Antje Neubert

Department of Pediatric and Adolescent Medicine, University Hospital Erlangen

Loschgestrasse 15

91054 Erlangen

Cite this as:
Neubert A, Toni I, König J, Malonga Makosi D, Mildenberger P, Romanos M, Bertsche A, Bertsche T, Dahlem P, Egberts K, Erdlenbruch B, Fekete S, Haug U, Horneff G, Hübler A, Kiess W, Neininger MP, Niehues T, Sachs B, Schettler KF, Schreeck F, Steimle T, Wenzl T, Wirth S, Zepp F, Schwab M, Urschitz MS, Rascher W, on behalf of the KiDSafe Consortium: A complex intervention to prevent medication-related hospital admissions—results of the stepped-wedge cluster randomized trial KiDSafe in pediatrics. Dtsch Arztebl Int 2023; 120: 425–31. DOI: 10.3238/arztebl.m2023.0123

►Supplementary material
eReferences, eMethods section, eTables, eFigures, eBox:
www.aerzteblatt-international.de/m2023.0123

References

1. Allen HC, Garbe MC, Lees J, et al.: Off-label medication use in children, more common than we think: a systematic review of the literature. J Okla State Med Assoc 2018; 111: 776–83

2. Kimland E, Odlind V: Off-label drug use in pediatric patients. Clin Pharmacol Ther 2012; 91: 796–801.

3. Knopf H, Wolf IK, Sarganas G, Zhuang W, Rascher W, Neubert A: Off-label medicine use in children and adolescents: results of a population-based study in Germany. BMC public health 2013; 13: 631.

4. Magalhaes J, Rodrigues AT, Roque F, Figueiras A, Falcao A, Herdeiro MT: Use of off-label and unlicenced drugs in hospitalised paediatric patients: a systematic review. Eur J Clin Pharmacol 2015; 71: 1–13.

5. Andrade SRA, Santos P, Andrade PHS, da Silva WB: Unlicensed and off-label prescription of drugs to children in primary health care: a systematic review. J Evid Based Med 2020; 13: 292–300.

6. Meng M, Lv M, Wang L, et al.: Off-label use of drugs in pediatrics: a scoping review. Eur J Pediatr 2022; 181: 3259–69

7. Guidi B, Parziale A, Nocco L, et al.: Regulating pediatric off-label uses of medicines in the EU and USA: challenges and potential solutions: comparative regulation framework of off label prescriptions in pediatrics: a review. Int J Clin Pharm 2022; 44: 264–9.

8. Neubert A, Planner C, Cranswick N: The new European regulation on pediatric medicines: safety and ethics perspectives. Paediatr Drugs 2008; 10: 147–9.

9. Wimmer S, Neubert A, Rascher W: The safety of drug therapy in children. Dtsch Arztebl Int 2015; 112: 781–7.

10. Bellis JR, Kirkham JJ, Nunn AJ, Pirmohamed M: Adverse drug reactions and off-label and unlicensed medicines in children: a prospective cohort study of unplanned admissions to a paediatric hospital. Br J Clin Pharmacol 2014; 77: 545–53.

11. Smyth RL, Peak M, Turner MA, et al.: Programme Grants for Applied Research. ADRIC: Adverse Drug Reactions In Children – a programme of research using mixed methods. 2014: Southampton (UK): NIHR Journals Library.

12. Smyth RM, Gargon E, Kirkham J, et al.: Adverse drug reactions in children—a systematic review. PloS One 2012; 7: e24061.

13. Eguale T, Buckeridge DL, Verma A, et al.: Association of off-label drug use and adverse drug events in an adult population. JAMA Intern Med 2016; 176: 55–63.

14. Bellis JR, Kirkham JJ, Thiesen S, et al.: Adverse drug reactions and off-label and unlicensed medicines in children: a nested case-control study of inpatients in a pediatric hospital. BMC Med 2013; 11: 238.

15. Gallagher RM, Mason JR, Bird KA, et al.: Adverse drug reactions causing admission to a paediatric hospital. PloS One 2012; 7: e50127.

16. van der Zanden TM, Smeets NJL, de Hoop-Sommen M, et al.: Off-label, but on-evidence? A review of the level of evidence for pediatric pharmacotherapy. Clin Pharmacol Ther 2022; 112: 1243–53.

17. Meng M, Zhou Q, Lei W, et al.: Recommendations on off-label drug use in pediatric guidelines. Front Pharmacol 2022; 13: 892574

18. Neubert A, Rascher W, Toni I: KiDSafe – Verbesserung der Versorgung von Kindern und Jugendlichen mit Arzneimitteln durch Erhöhung der Arzneimitteltherapiesicherheit. Ergebnisbericht. www.innovationsfonds.g-ba.de/downloads/beschluss-dokumente/242/2022-08-19_KiDSafe_Ergebnisbericht.pdf (last accessed on 11 May 2023).

19. Craig P, Dieppe P, Macintyre S, Michie S, Nazareth I, Petticrew M: Developing and evaluating complex interventions: the new medical research council guidance. BMJ 2008; 337: a1655.

20. Moore GF, Audrey S, Barker M, et al.: Process evaluation of complex interventions: medical research council guidance. BMJ 2015; 350: h1258.

21. Hemming K, Taljaard M, McKenzie JE, et al.: Reporting of stepped wedge cluster randomised trials: extension of the CONSORT 2010 statement with explanation and elaboration. BMJ 2018; 363: k1614.

22. Schulze C, Toni I, Moritz K, Eberl S, Rascher W, Neubert A: Development and adjustment of an algorithm for identifying drug-related hospital admissions in pediatrics. J Patient Saf 2022; 18: 421–9.

23. Uppsala Monitoring Centre: The use of the WHO-UMC system for standardised case causality assessment. https://who-umc.org/media/164200/who-umc-causality-assessment_new-logo.pdf (last accessed on 26 May 2023).

24. Güthlin C, Beyer M, Erler A, et al.: Rekrutierung von Hausarztpraxen für Forschungsprojekte, Erfahrungen aus fünf allgemeinmedizinischen Studien. Z Allg Med 2012; 88: 173–81.

25. Oberle D, Pönisch C, Paeschke N, Lehmann B, Mentzer D: Pharmakovigilanz bei Kindern. Bulletin zur Arzneimittelsicherheit 2010; 10: 6–18.

26. Stammschulte T, Köberle U, Prause L, Pitzer M, Gundert-Remy U: Spontanmeldungen von Nebenwirkungen an die AkdÄ im Jahr 2018. Arzneiverordnung in der Praxis (AVP) 2019; 46: 180–89.

27. Bundesärztekammer: Ärztestatistik 2018 . www.bundesaerztekammer.de/ueber-uns/aerztestatistik/aerztestatistik-2018/ambulant-taetige-aerzte/ (last accessed on 26 May 2023).

e1. Hughes J, Hakhu NR, Voldal E, Xia F: Package swCRTdesign: Stepped Wedge Cluster Randomized Trial (SW CRT) Design. R package version 3.3, www.CRAN.R-project.org/package=swCRTdesign 2022 (last accessed on 6 June 2023).

e2. Mancl LA, DeRouen TA: A covariance estimator for GEE with improved small-sample properties. Biometrics 2001; 57: 126–34.

e3. Little RJ, Rubin DB: Statistical analysis with missing data. Vol. 793. 2019: John Wiley & Sons.