Electronic Alerts for Acute Kidney Injury

A Systematic Review

Background: Acute kidney injury (AKI) often takes a complicated course if diagnosed late and undertreated. Electronic alerts that provide an early warning of AKI are intended to support treating physicians in making the diagnosis of AKI and treating it appropriately. The available evidence on the effects of such alert systems is inconsistent.

Methods: We employed the PRISMA recommendations for systematic literature reviews to identify relevant articles in the PubMed, Scopus, and Web of Science databases. All of the studies that were retrieved were independently assessed by two of the authors with respect to the methods of computer-assisted electronic alert systems and their effects on process indicators and clinical endpoints.

Results: 16 studies with a total of 32 842 patients were identified. 8.5% of admitted patients had community-acquired or hospital-acquired AKI, with an in-hospital mortality of 22.8%. Fifteen electronic alert systems were in use throughout the participating hospitals. In 13 of 15 studies, alarm activation was accompanied by concrete treatment recommendations. A randomized controlled trial in which no such recommendations were given did not reveal any benefit of the alert system for the patients. In controlled but non-randomized trials, however, the provision of concrete treatment recommendations when the alert was activated led to more frequent implementation of diagnostic or therapeutic measures, less loss of renal function, lower in-hospital mortality, and lower mortality after discharge compared to control groups without an electronic alert for AKI.

Conclusion: Non-randomized controlled trials of electronic alerts for AKI that were coupled with treatment recommendations have yielded evidence of improved care processes and treatment outcomes for patients with AKI. This review is limited by the low number of randomized trials and the wide variety of endpoints used in the studies that were evaluated.

About one in 10 patients receiving inpatient treatment will develop acute kidney injury (AKI) (1, 2). Sawhney et al (1) even reported that 17.6% of patients with pre-existing chronic kidney disease developed AKI. Acute kidney injury reduces the therapeutic results for the specialist department that provides the primary treatment and is an independent risk factor for in-hospital mortality that is raised by several orders of magnitude (hazard ratio 1.4–15.4; 13–41% of cases) (1–4). A typical and serious long-term consequence is the development or progression of chronic renal failure in 10–20% of cases (4, 5). Acute kidney injury has a greater incidence and a higher rate of complications than myocardial infarction (6).

The diagnosis is made on the basis of (7):

• A defined rise in the serum creatinine concentration (>50% from the previous measurement within a maximum of 7 days, or >0.3 mg/dL [>26.4 µmol/L] within a maximum of 2 days or to >4.0 mg/dL [>354 µmol/L]);
• And/or a reduction in diuresis (<0.5 mL/kg/BW/h over 6 hours);
• And/or initiation of acute renal replacement therapy.

The stages of acute kidney injury are described in Table 1. The most common triggers of AKI are sepsis, complex surgical procedures, nephrotoxins, hypovolemia, cardiac decompensation, and urinary retention (8). Recommended effective countermeasures are early diagnosis and the initiation of rapid multifactorial measures (Table 1), in order to identify as early as possible trigger factors and factors that support and maintain renal injury, and thereby create optimal conditions for complete or extensive renal recovery (7).

Table 1

Stages of acute kidney injury and measures (7)

The duration of AKI crucially determines patients’ survival (AKI stage 1 for <2 days: mortality 13.7/100 person years versus AKI stage 3 for >7 days: mortality 43.8/100 person years) (9). If the diagnosis is delayed and insufficient therapeutic measures are initiated, this constitutes an independent risk factor (odds ratio 1.45; 95% confidence interval: [1.04; 2.039]) for higher in-hospital mortality (10). Optimized therapeutic care reduces the development of higher stages of AKI by some 50% and in-hospital mortality by 20% (11).

Electronic alerts or early warning systems are intended to enable earlier detection of acute kidney injury. Figure 1 shows the principle underlying an AKI early warning system. Some individual publications or narrative reviews found patient-relevant benefits (12, 13), and some others didn’t (14). What is not clear is whether consistent alert triggers were used and what the extent was to which the alarm signal targeted the recipients and provided concrete treatment recommendations.

Figure 1

Principle of an electronic early warning alerting system for acute kidney injury

On this background, we conducted a systematic literature search of the current level of knowledge regarding AKI early warning systems. We focused mainly on the characteristics of AKI alerting systems, including trigger, type, message, and recipient of the alerting process and on effects of AKI alerting systems on process indicators and patient-relevant endpoints.

Methods

Study design

To find answers to these questions, we summarized in the present review article the results of a systematic literature search according to the recommendations of the PRISMA statement (15). The study protocol was registered (www.crd.york.ac.uk/prospero, CRD42016041510, search term: “alert”). Two of the authors (CA/MH) independently identified studies and conducted screening, selection, and data extraction. In case of disagreement, this was resolved by discussion in a consensus decision or by the decision of a further author (A H-F).

Literature search

The Box shows a summary of the search strategy, search terms, extracted data, and endpoints (see eBoxes 1 and 2 for more detail). To identify appropriate studies we used the databases Medline, Scopus, and Web of Science, independently of the publication type and status and without any limits imposed on the time periods covered. Furthermore we regularly screened medical journals that were relevant for the subject matter of our review article, such as the New England Journal of Medicine, Lancet, Journal of the American Medical Association, (Clinical) Journal of the American Society of Nephrology, Clinical Kidney Journal, and conference abstracts; we searched study registries (clinical trials.gov, German Clinical Trials Register) for unpublished studies and took up reading recommendations from experts in the subject (effective date: 20 May 2016).

Box

Summary of search criteria, extracted data, and endpoints*

eBox 1

Literature databases, search terms, and literature search

eBox 2

Extracted data and endpoints

Study selection

The inclusion criteria for our study were:

• Patient population: Hospital inpatients
• Intervention: Electronic alerting system for identifying patients with acute kidney injury (acquired on an inpatient or outpatient basis)
• Reported endpoints: Characteristics of the alerting systems, including the trigger for the AKI alert, the type and targeted recipient of the alerting process, and the potential linking of the alert to treatment recommendations, as well as effects of the AKI alerting system on process indicators and patient-relevant endpoints.
• Study design: (Pseudo) randomized studies, cohort studies.

We did not include studies that did not use electronic alerts for acute kidney injury.

Data extraction, endpoints, study quality, and study bias

Alert characteristics, process indicators, and patient-relevant endpoints from the studies were extracted by using a standardized study documentation sheet that had been developed a priori. For each study we extracted the dates and endpoints listed in eBox 2. Non-randomized controlled studies were assessed regarding the representativeness of the patient population, comparability of study groups, and quality of endpoint collection (Newcastle-Ottawa scale [16]), and regarding the risk of bias in the study results by using the ACROBAT-NRSi tool (A Cochrane-Risk-Of-Bias Assessment-Tool of non-randomised studies of interventions, http://methods.cochrane.org/bias/assessing-risk-bias-included-studies). We assessed randomized studies in terms of their reported approach to randomization and blinding and the description of the dropout rate (Jadad scale [17]). The current consensus is that a point value of <3 points on the Jadad scale indicates a notably reduced study quality (Two authors CA, MH) collected the scale point scores independently of one another. Where disagreement arose regarding the point score, this was resolved by discussion and consensus or by the decision of a third author (A H-F). The study quality and the risk of bias in the study results were not used as exclusion criteria. We described the results in a descriptive analysis. We designed a subgroup analysis of the effects of electronic alerts for the controlled studies, which linked the alert with concrete treatment recommendations or co-treatment by specialists.

Results

By applying the search strategy, we identified 958 potentially relevant publications, of which 16 primary publications (a list of the excluded publications is available from the authors) were included in the data extraction and analysis (11, 12, 14, 18–30), after deduplication of the records and after screening titles, abstracts, and full text publications according to our inclusion and exclusion criteria (Figure 2). Eleven of the included studies had control groups (11, 12, 14, 18, 19, 21–23, 25, 29, 30). Nine of these studies were not randomized, two were randomized (14, 22). The remaining 5 studies were observational studies. eTable 1a lists for each of these publications the relevant data on patients, study design and quality, and the risk of bias in the reported study. None of the studies had received private funding. The included studies reported data relating to 32 842 patients with AKI (of whom 49% were women), which had been collected by means of an electronic alerting system. Patients requiring chronic dialysis were excluded.

Figure 2

Flowchart showing study selection process

eTable 1a

Patients, study design, study quality, and study bias

Table 2 summarizes the characteristics of patients with AKI. The population included elderly patients who were subject to substantial in-hospital mortality or follow-up mortality (about 23%) (11, 12, 14, 20, 21, 23, 26–28, 30), whereas for hospital inpatients admitted during the period under comparison, mortality was 2% (28). The incidence of AKI was about 9%; almost half of the patients had developed a moderate to severe AKI stage (11, 20, 21, 23–28, 30). Consultant nephrologist support had been requested in the setting of routine clinical treatment in 12% of patients with AKI (11, 14, 21). Recovery of renal function was reported for three quarters of cases (20, 21, 23, 26). The severity grade of the acute kidney injury was associated with the duration of the inpatient stay (11, 14, 21, 25, 27, 28, 30). In direct analogy, the stage-related in-hospital mortality or mortality at follow-up rose in linear fashion with the AKI stage (12, 20, 21, 26–28).

Table 2

Characteristics of patients with acute kidney injury

Characteristics of electronic alerting systems for acute kidney injury

eTable 1b lists study specific results for the functionality and effects of the reported AKI early warning alerts. Results regarding functionality are summarized in Figure 3. All identified studies used a defined and mostly consensus-supported increase in serum creatinine to trigger the alarm (7, 31, 32). One study (23) recorded reductions in diuresis in addition to creatinine increases to detect AKI. Fifteen AKI electronic alerts operated hospital-wide; one was restricted to an intensive care ward (23). The alarm was triggered mostly in a fully automated way (11, 12, 14, 18, 20, 23, 25–27, 29, 30), without interrupting the work of the treating ward physicians (non-disruptive) (12, 14, 18, 20–22, 25, 27, 29)—for example, by inserting a text alert in the laboratory program or by email, or while linking concrete treatment recommendations or initiating specialist support (11, 12, 18, 19, 21–23, 25–30) (Figure 3). The alarm signal was passed to the treating physicians and, in some cases, also to the hospital/ward pharmacists (14, 22) or doctors specializing in AKI treatment, such as nephrologists (25) or specially trained specialists in internal medicine (29).

Figure 3

Typical characteristics of AKI alerting systems

eTable 1b

Functionality and effects of the reported AKI alerting systems

Process indicators and patient-relevant effects of electronic alerts in controlled studies

Both randomized studies (14, 22) had a point score of 3.5 (3.0–4.0) out of a maximum of 5 points on the Jadad scale (17). One randomized study did not report any patient-relevant endpoints, merely process indicators, such as adjustment of medications in AKI, for which no differences between groups had been observed (22). The other randomized study did not provide treatment recommendations to ward physicians, did not affect the care status of patients with AKI, and did not find any differences for patients undergoing acute kidney replacement therapy and for in-hospital mortality or follow-up mortality, or other patient-relevant endpoints, such as the length of inpatient stay (14).

Non-randomized studies had a point score of 6 (4–7) of a maximum of 9 points on the Newcastle-Ottawa scale (16). The risk for bias was moderate in the non-randomized studies, except in the one reported by Gulliford (critical, [29]) and Kolhe et al (low, [30]). In 10 out of 11 controlled studies, the AKI alert was linked to concrete treatment recommendations for the ward physicians or specialist co-treatment was provided (11, 12, 18, 19, 21–23, 25, 29, 30). In 7 out of 8 controlled studies of AKI electronic alerts and linked treatment recommendations, which reported process indicators (11, 12, 18, 19, 23, 29, 30), the alert group underwent more renal ultrasonography investigations than the control group, the administration of nephrotoxic medications was stopped earlier, or patients’ fluid status was optimized (Figure 4). Furthermore, AKI alerts linked to concrete treatment recommendations in the alert groups led to better renal function in all studies that reported this particular endpoint (11, 18, 23, 30), although the definition of improved renal function was subject to substantial variability and the rate of kidney replacement therapy was lower in one study only (12), whereas it remained unchanged in 3 studies (23, 25, 30). In-hospital mortality or mortality at follow-up was reduced in the AKI alert group with concrete treatment recommendations in 4 studies compared with the control group (11, 21, 29, 30)—in 3 studies this difference reached significance (11, 21, 30), in 1 study it fell from 44% to 25% without any reporting of statistical significance (30)—and in 3 studies it remained unchanged (18, 23, 25). Furthermore, one study reported lower in-hospital mortality in the alert group compared with the control group if the Critical Care and Outreach team was called to the patient’s bedside due to threatening changes to the vital parameters within a maximum of 24 hours after AKI alert (12). Of the two identified randomized studies (14, 22) only one (14) collected patient-relevant endpoints and showed—without any suggested treatment recommendations—no patient-relevant benefits for an alerting system for patients with AKI.

Figure 4

Effect of AKI alerting systems with linked concrete treatment recommendations on process indicators in controlled, non-randomized studies

Discussion

We conducted a systematic literature search and identified 16 studies that investigated electronic alerts for detecting acute kidney injury. These early warning systems captured 32 842 patients with AKI. They were mostly fully automated and non-disruptive and used as the trigger for the alert a defined rise in serum creatinine. The controlled non-randomized studies were often of alerts linked to concrete treatment recommendations to the treating ward physicians or with the introduction of specialist (consultant) support. These studies provided indications of an increase in the initiation of nephroprotective measures, a milder course of the acute kidney injury, and lower in-hospital mortality or follow-up mortality compared with the control group. The reliability of the results of the non-randomized controlled studies we included was low to moderate, mainly because the reporting was of limited quality.

Patients who died during an inpatient stay with prior severe acute kidney injury received adequate care in less than 50% of cases as far as laboratory tests and imaging exams to identify the causes and the initiation of therapeutic measures are concerned (33). In another study, renal ultrasonography was undertaken in 7% of patients with AKI, and in almost all cases, medication therapy using nephrotoxins—such as non-steroidal anti-inflammatory drugs, contrast medium, or aminoglycosides—was continued (14). A cross-sectional study of more than 2 million patients showed that AKI was identified and treated in 25% of those affected; a delay in the diagnosis was found to be an independent risk factor for in-hospital mortality (10). Specialist co-treatment was described as a protective factor (10).

Thus far, early warning systems have been used in patients with kidney disorders primarily in order to detect medication problems (34); the acceptance of medication warning systems is limited if they are not linked to concrete treatment recommendations (35). Thomas et al. reported (25) that specialist recommendations for medication intervention existed in 229 out of 251 treated inpatients in whom an electronic alert system detected AKI. The benefit of a medication early warning system to avoid adverse effects was greatest when appropriate measures were concretely named (36). One measure to help avoid alert fatigue might be for the pharmacologist or pharmacist to check the causality of a certain drug in the setting of a renal event (36). The use and benefit of electronic alerting systems should be checked regularly and feedback given to all parties involved. Since 2015, British hospitals have used electronic alerts for acute kidney injury. Conclusive results of this nationwide intervention in patient care are not yet available. In principle, however, action seems urgently required in terms of counteracting the development of chronic renal failure (40% of patients with undetected acute kidney injury versus 15% of patients with known acute kidney injury [37]).

This review article summarized data on the epidemiology and care provision of hospital inpatients with acute kidney injury, primarily from the United Kingdom, but also from the United States and Belgium. It provides an overview of the current state of affairs, data, and study quality regarding AKI alert systems. The data show that the patients are older and 90% of them do not receive specialist care. The implementation of electronic alert systems is subject to great variance (trigger threshold, previous value, exceeding a minimum value, recipient, invasiveness of alert transmission, detailed reference to treatment measures) and gaps in the quality standard, bias control, and evidence levels of the reporting studies. The recommendation in a recent consensus paper of the Acute Dialysis Quality Initiative, to link AKI alerting systems to context specific treatment recommendations (38), is supported by the results of our study. Our critical evaluation of the quality of existing studies on electronic alerts and explanation of study results in terms of background, technical details, treatment recommendations, and endpoints may be useful in planning further studies and assessing generalizability and local implementation of AKI alerting systems.

Limitations

The validity of this study is limited because of small case numbers in some of the identified individual studies, the small number of randomized studies, result bias, and the wide variation in the reported endpoints. The fact that individual studies are restricted mainly to serum creatinine as the alert trigger is based on its clinical use as a diagnostic criterion for acute kidney injury, but using new renal biomarkers in the setting of an AKI alerting system seems a possibility. None of the identified studies provided instructions or recommendations for the frequency of creatinine measurements. The studies entailed investigations under real-life conditions. We have no solid information to indicate limited generalizability of the results of our review to the German situation. An individual randomized design for investigating the effects of AKI alerting systems is hampered by potential transfer effects between the intervention and control groups. “Before and after” studies with well planned characterization of patients and measures, and especially cluster randomized studies would enable robust conclusions. In planning such studies, the role of chronic renal failure as a risk factor for acute kidney injury will have to be considered.

On the basis of the data described and of our study findings, the implementation of electronic alerts for AKI is feasible and promising. Their cost–benefit effect will need to be reviewed.

Conflict of interest statement

Dr. Haase-Fielitz has received third-part funding from the B. Braun Foundation. The remaining authors declare that no conflict of interest exists.

Manuscript received on 2 July 2016, revised version accepted on 10 October 2016.

Translated from the original German by Birte Twisselmann, PhD.

Corresponding author
Dr. rer. medic. Anja Haase-Fielitz
Institut für Sozialmedizin und Gesundheitsökonomie
Otto-von-Guericke Universität Magdeburg
Leipziger Str. 44, 39120 Magdeburg, Germany

@Supplementary material
eBoxes, eTables:
www.aerzteblatt-international.de/17m0001

References

1. Sawhney S, Marks A, Fluck N, Levin A, Prescott G, Black C: Intermediate and long-term outcomes of survivors of acute kidney injury episodes: a large population-based cohort study. Am J Kidney Dis. 2016 Aug 20. pii: S0272–6386(16)30256–6. doi:10.1053/j.ajkd.2016.05.018. [Epub ahead of print

2. Wang HE, Muntner P, Chertow GM, Warnock DG: Acute kidney injury and mortality in hospitalized patients. Am J Nephrol 2012; 35: 349–55

3. Kindgen-Milles D: Epidemiologie und prognostische Relevanz des akuten Nierenversagens. In: Jörres A (ed.) Akutes Nierenversagen bei Intensivpatienten: Praxis und Therapie. Köln: Deutscher Ärzteverlag 2010: 1–14.

4. Chawla LS, Amdur RL, Amodeo S, Kimmel PL, Palant CE: The severity of acute kidney injury predicts progression to chronic kidney disease. Kidney Int 2011; 79: 1361–9

5. Coca SG, Singanamala S, Parikh CR: Chronic kidney disease after acute kidney injury: a systematic review and meta-analysis. Kidney Int 2012; 81: 442–8

6. Chawla LS, Amdur RL, Shaw AD, Faselis C, Palant CE, Kimmel PL: Association between AKI and long-term renal and cardiovascular outcomes in United States veterans. Clin J Am Soc Nephrol 2014; 9: 448–56

7. KDIGO Clinical practice guideline for acute kidney injury. Kidney Int Suppl 2012; 2 (1): 19–36.

8. Uchino S, Kellum JA, Bellomo R, et al.: Acute renal failure in critically ill patients: a multinational, multicenter study. JAMA 2005; 294: 813–8

9. Coca SG, King JT Jr, Rosenthal RA, Perkal MF, Parikh CR: The duration of postoperative acute kidney injury is an additional parameter predicting long-term survival in diabetic veterans. Kidney Int 2010; 78: 926–33

10. Yang L, Xing G, Wang L, et al.: ISN AKF 0by25 China Consortiums. Acute kidney injury in China: a cross-sectional survey. Lancet 2015; 386: 1465–71

11. Kolhe NV, Staples D, Reilly T, et al.: Impact of compliance with a care bundle on acute kidney injury outcomes: a prospective observational study. PLoS One 2015; 10: e0132279

12. Prendecki M, Blacker E, Sadeghi-Alavijeh O, et al.: Improving out-comes in patients with acute kidney injury: the impact of hospital based automated AKI alerts. Postgrad Med J 2016; 92: 9–13

13. Selby NM: Electronic alerts for acute kidney injury. Curr Opin Nephrol Hypertens 2013; 22: 637–42

14. Wilson FP, Shashaty M, Testani J, et al.: Automated, electronic alerts for acute kidney injury: a single-blind, parallel-group, randomised controlled trial. Lancet 2015; 385: 1966–74

15. Moher D, Liberati A, Tetzlaff J, Altman DG: Preferred reporting items for sys- tematic reviews and meta-analyses: the PRISMA statement. The PRISMA Group. PLoS Med 2009; 6: e1000097

16. Wells GA, Shea B, O’Connell D, et al.: The Newcastle-Ottawa Scale (NOS) for assessing the quality of nonrandomised studies in meta-analyses. www.ohri.ca/programs/clinical_epidemiology/oxford.asp (last accessed on 25 June 2016).

17. Jadad AR, Moore RA, Dawn C, et al.: Assessing the quality of reports of ran- domized clinical trials: is blinding necessary? Controlled Clinical Trials 1996; 17: 1–12

18. Rind DM, Safran C, Phillips RS, et al.: Effect of computer-based alerts on the treatment and outcomes of hospitalized patients. Arch Intern Med 1994; 154: 1511–7

19. McCoy AB, Waitman LR, Gadd CS, et al.: A computerized provider order entry intervention for medication safety during acute kidney injury: a quality improvement report. Am J Kidney Dis 2010; 56: 832–41

20. Thomas M, Sitch A, Dowswell G: The initial development and assessment of an automatic alert warning of acute kidney injury. Nephrol Dial Transplant 2011; 26: 2161–8

21. Selby NM, Crowley L, Fluck RJ, et al.: Use of electronic results reporting to diagnose and monitor AKI in hospitalized patients. Clin J Am Soc Nephrol 2012; 7: 533–40

22. McCoy AB, Cox ZL, Neal EB, et al.: Real-time pharmacy surveillance and clinical decision support to reduce adverse drug events in acute kidney injury: a ran- domized, controlled trial. Appl Clin Inform 2012; 3: 221–38

23. Colpaert K, Hoste EA, Steurbaut K, et al.: Impact of real-time electronic alerting of acute kidney injury on therapeutic intervention and progression of RIFLE class. Crit Care Med 2012; 40: 1164–70

24. Ahmed S, Curtis S, Hill C, Hine T: Population-based estimated reference creatinine values: a novel method of a robust electronic acute kidney injury alert system. Nephron Clin Pract 2014; 128: 166–70

25. Thomas ME, Sitch A, Baharani J, Dowswell G: Earlier intervention for acute kidney injury: evaluation of an outreach service and a long-term follow-up. Nephrol Dial Transplant 2015; 30: 239–44

26. Flynn N, Dawnay A: A simple electronic alert for acute kidney injury. Ann Clin Biochem 2015; 52: 206–12

27. Porter CJ, Juurlink I, Bisset LH, Bavakunji R, Mehta RL, Devonald MA: A real- time electronic alert to improve detection of acute kidney injury in a large teaching hospital. Nephrol Dial Transplant 2014; 29: 1888–93

28. Wallace K, Mallard AS, Stratton JD, Johnston PA, Dickinson S, Parry RG: Use of an electronic alert to identify patients with acute kidney injury. Clin Med 2014; 14: 22–6

29. Gulliford S, Sloan J: Acute physicians should take the lead in developing acute kidney injury services in district general hospitals. Acute Medicine 2014; 13: 48–50.

30. Kolhe NV, Reilly T, Leung J, et al.: A simple care bundle for use in acute kidney injury: a propensity score matched cohort study. Nephrol Dial Transplant. 2016 Nov; 31 (11): 1846–1854. Epub 2016 May 4

31. Bellomo R, Ronco C, Kellum JA, Mehta RL, Palevsky P: Acute Dialysis Quality Initiative Workgroup: acute renal failure—definition, outcome measures, animal models, fluid therapy and information technology needs: The Second Interna- tional Consensus Conference of the Acute Dialysis Quality Initiative (ADQI) Group. Crit Care 2004; 8: R204–12

32. Mehta RL, Kellum JA, Shah SV, et al.: Acute kidney injury network: report of an initiative to improve outcomes in acute kidney injury. Crit Care 2007; 11: R31

33. Adding insult to injury: a review of the care of patients who died in hospital with a primary diagnosis of acute kidney injury (acute renal failure). A report by the National Confidential Enquiry into Patient Outcome and Death 2009. www.ncepod.org.uk/2009report1/Downloads/AKI_report.pdf (last accessed on 25 November 2016).

34. Czock D, Konias M, Seidling HM, et al.: Tailoring of alerts substantially reduces the alert burden in computerized clinical decision support for drugs that should be avoided in patients with renal disease. J Am Med Inform Assoc 2015; 22: 881–7

35. Awdishu L, Coates CR, Lyddane A, et al.: The impact of real-time alerting on appropriate prescribing in kidney disease: a cluster randomized controlled trial. J Am Med Inform Assoc 2016; 23: 609–16

36. Leung AA, Schiff G, Keohane C, et al.: Impact of vendor computerized physician order entry on patients with renal impairment in community hospitals. J Hosp Med 2013; 8: 545–52

37. Meran S, Wonnacott A, Amphlett B, Phillips A: How good are we at managing acute kidney injury in hospital? Clin Kidney J 2014; 7: 144–50

38. Hoste EA, Kashani K, Gibney N, et al.: Impact of electronic-alerting of acute kidney injury: workgroup statements from the 15(th) ADQI Consensus Conference. Can J Kidney Health Dis 2016; 3: 10.