Validation of Cardiovascular Risk Prediction by the Arriba Instrument

An Analysis Based on Data From the Study of Health in Pomerania

Background: It is recommended in cardiovascular prevention guidelines that treatment should be based on overall cardiovascular risk. The arriba instrument has been widely used for this purpose in Germany. The aim of this study is to validate risk prediction by arriba with the aid of morbidity and mortality data from the population-based Study of Health in Pomerania.

Methods: In a longitudinal analysis, the arriba instrument was used to calculate the 10-year overall cardiovascular risk at baseline for subjects who had not sustained any prior cardiovascular event. Cardiovascular event rates were determined from follow-up data, and discrimination and calibration measures for the risk determination algorithm were calculated.

Results: Data from 1973 subjects (mean age 51 ± 13 years, 48% men) were included in the analysis. After a median follow-up of 10.9 years, cardiovascular events had occurred in 196 subjects, or 9.8%. The ratio of predicted to observed event rate was 0.8 (95% confidence interval: [0.5; 1.1]), 1.3 [1.0; 1.8], and 1.1 [0.8; 1.4] for subjects at low, intermediate, and high cardiovascular risk, respectively. Arriba underestimated cardiovascular event rates in women and overestimated them in persons aged 30–44 and 45–59. The area under curve was 0.84 [95% CI 0.81; 0.86].

Conclusion: The discrimination scores of the arriba instrument resemble those of SCORE-Germany and PROCAM, but a better adjustment to the target population would be desirable. The results support the recommendation of the German Guideline for Cardiovascular Risk Counseling in General Practice for the use of the arriba instrument. An unresolved problem is the failure to consider intervention effects, resulting in an overall mild overestimation of risk.

Determining a person’s cardiovascular risk and counseling them accordingly are crucial components of cardiovascular primary prevention. The guidelines on cardiovascular prevention from the German College of General Practitioners and Family Physicians (DEGAM) as well as the German cardiac Society (DGK) and the European Society of Cardiology (ESC) recommend assessing the overall cardiovascular risk on the basis of multivariate functions (1, 2). In Germany the risk prediction instruments in use are SCORE2 (previously SCORE-Germany, Systematic Coronary Risk Evaluation), PROCAM (Prospective Cardiovascular Münster), and arriba (Aufgabe gemeinsam definieren, Risiko subjektiv, Risiko objektiv, Information über Präventionsmöglichkeiten, Bewertung der Möglichkeiten, Absprache über weiteres Vorgehen [define the task jointly, subjective risk, objective risk, information about prevention options, assessing the options, agreeing on how to proceed]) (3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13). They predict the 10-year overall risk for a cardiovascular event (myocardial infarction, stroke) in persons who had not experienced a prior cardiovascular event.

To predict the risk, information on individual cardiovascular risk factors is collected and the overall risk is calculated from those (Table 1) (3, 14, 15). The SCORE-2-, PROCAM, and arriba instruments predict the 10-year overall risk for fatal and non-fatal cardiovascular events. The arriba instrument was developed from the Framingham risk score (4, 16). It was adapted for European data by adjusting using a calibration factor on the basis of the British Regional Heart Study (17). Additionally the instrument was extended for predicting stroke and for glycemic control on the basis of HbA1c in patients with diabetes (4). The mentioned risk prediction instruments are available free of charge online (Table 1). The arriba instrument is often used for the purpose of cardiovascular risk counseling in primary care/general practice because of the option to depict the effects of treatment options.

Table 1

Risk predictors for the risk prediction instruments arriba, PROCAM, and SCORE2

This study aims to validate the arriba instrument regarding the agreement between predicted and observed cardiovascular 10-year event rate (calibration) and to identify persons with cardiovascular events (discrimination) on the basis of morbidity and mortality data of the population-based Study of Health in Pomerania (SHIP) (18).

Methods

Basis of our data

The analysis is based on data from the population-based cohort study SHIP (source population 214 057 subjects from the north east of Germany in 1996) (18). We used data from the baseline examination (study period 1997–2002, 4308 participants, response rate at baseline 68.8%) and the first and second follow-up examination (SHIP-1: 3300 participants, SHIP-2: 2333 participants, response rate 71%). We used self-reported data on diagnoses of hypertension and diabetes, myocardial infarction, stroke, medication intake, laboratory parameters, results from blood pressure and somatometric measurements, data from the medication review, and data from the mortality follow-up (18).

Inclusion criteria

We included in our analysis all subjects without a prior self-reported cardiovascular event (myocardial infarction, stroke) aged 30–80 years who participated in the baseline examination and both follow-ups, or who died from a cardiovascular event during the study period of 10.9 years to the second follow-up (median follow-up period). We included only subjects with complete data in the calculation of the overall cardiovascular risk with the arriba instrument (Figure 1).

Figure 1

Composition of the study population

Evaluation

Our study was a longitudinal analysis. For all included subjects we used the arriba instrument to calculate cardiovascular 10-year risk based on cardiovascular risk factors (predicted cardiovascular event rate) (for definitions, see the eBox and eTable 1). Non-fatal events were defined on the basis of self-reported myocardial infarction and stroke from the data of both follow-up examinations (response rate of the second follow-up to baseline 54%). Fatal cardiovascular events were defined on the basis of the mortality follow-up if a main cause of death had been coded to the diagnostic groups ICD-GM I10-I15, I20-I25, I26-I28, I30-I52, I60-I69, I70-I79 (eTable 2).

eBox

Definition of cardiovascular events

eTable 1

Overview of measurements and definitions of laboratory-chemical and clinical variables

eTable 2

Numbers of coded main causes of death in subjects with a fatal cardiovascular event

Study participants who met the inclusion criteria at the time of the baseline examination formed the baseline cohort. Missing values for the endpoints under consideration were the result of study drop-out or “lost to follow-up” or concurrent events (death from non-cardiovascular causes). Study participants without missing endpoint values formed the complete case cohort. Assuming “missing at random” we used a logistic regression model to model the probability of participation by study participants depending on relevant variables of the baseline examination (sex, age, cigarette smoking, total and HDL cholesterol concentrations, systolic blood pressure, intake of antihypertensive medication, body mass index, presence of diabetes mellitus, number of medications taken in the preceding seven days, number of doctors’ visits within the preceding four weeks). The cases in the complete case cohort were weighted by using inverse probability weights for participation in the follow-up examinations. Applying these weights means that persons with a low probability of participating are given a greater weight so as to reduce the influence of bias as a result of selective participation.

Calibration (agreement between predicted and observed events) and discrimination (differentiating between persons who developed a cardiovascular event and those who did not) were assessed as criteria for the validity of the arriba instrument. We reported the ratio between predicted and observed cardiovascular events (predicted over observed ratio, P/O ratio) as the parameter for calibration for the total cohort and defined subgroups by age, sex, and predicted risk (low [<10%], moderate [10–20%], and high [≥20%]. A P/O ratio >1 indicates an overestimate and a P/O ratio <1 an underestimate of the cardiovascular risk. For the central analysis we calculated event rates and P/O ratios on the basis of the weighted cases of the complete case cohort. Additionally we report unweighted event rates for the baseline and complete case cohorts. We used receiver operating characteristic (ROC) curves and AUC (area under the curve) values to determine discrimination. No established methods to consider inverse probability weights are known for this analysis, therefore, unweighted data from the complete case cohort was used. We used SAS 9.3 (SAS Institute Inc., Cary, NC, USA) to analyze our data.

Research ethics and data protection

SHIP received ethics approval and consent from the participants, as well as a framework concept for data protection and storage that was agreed with the state data protection commissioner (18).

Results

At baseline, n=3446 subjects met the inclusion criteria (baseline cohort). The complete case cohort consisted of the n=1973 subjects who had participated in both follow-up examinations and those who had died from a cardiovascular event by the time of the second follow-up (Figure 1). Table 2 shows the characteristics of both cohorts. The mean age in the complete case cohort was 51 (standard deviation 13) years; 48% were men. Of the subjects, 69% (1356/1973) had a low cardiovascular risk, 30% (388/1973) a moderate risk, and 12% (229/1973) a high risk (Table 3). By the second follow-up, 80 subjects (41.%, 80/1973) in the complete case cohort had experienced a non-fatal cardiovascular event and 116 subjects (5.9%, 119/1973) a fatal events.

Table 2

Characteristics of the study population (data from baseline examination)

Table 3

Predicted and observed 10 year event rates for the arriba instrument

The unweighted predicted event rate in the baseline cohort was 9.80% and in the complete case cohort, 8.27% (eTable 3). Differences were also seen for the subgroups; these are indications that weighting for missing endpoints was important. The weighted predicted event rate in the complete case cohort was near identical to that in the baseline cohort (9.79 versus 9.80). The results were also similar for the individual subgroups, which can be seen as an indication that weighting adjusts for missing values.

eTable 3

Predicted and observed 10-year event rates for the arriba instrument

The P/O ratio based on the weighted cases of the complete case cohort for the total cohort was 1.05 (Table 3). For subjects with a low, moderate, and high cardiovascular risk, the P/O ratio was 0.8% each (95% confidence interval [0.5; 1.1]), 1.3 [1.0; 1.8], and 1.1 [0/8; 1.4]. The P/O ratio in women and in the group aged 60–80 years was below 1, and in men and in the age groups 30–44 and 45–59 years in some cases notably above 1 (Table 3). The AUC for discrimination of persons with a cardiovascular event was 0.84 [0.81; 0.86] (Figure 2).

Figure 2

ROC (receiver operating characteristic) curve with AUC (area under curve) value

Discussion

Summary of the main findings

Overall, the arriba instrument shows good discrimination for persons with a cardiovascular event on the basis of the AUC value. The event risk in persons with a low cardiovascular risk is slightly underestimated by the arriba instrument and the event rate especially in persons with a moderate risk is slightly overestimated.

Prognostic quality

Analyses with a high number of observations and a sufficiently high number of events are the gold standard for validating risk prediction instruments (19, 20). For the primary care setting in Germany, no comprehensive data exists, hence population-based cohort studies provide an alternative option for external validation.

As far as we are aware, thus far only the DETECT study has reported findings for the calibration or discrimination of the arriba instrument (21). In contrast to our analysis, this study reported a risk overestimate even in the risk range <10% and a notably higher overestimate in the risk range >20%. The study had a median follow-up period of only four years, and an older paper version (point score) of arriba was used to calculate the risk. The validity of the paper version is considerably limited, since included predictors are categorized. Patients with diabetes were excluded, whereas the arriba instrument explicitly allows for a risk calculation for diabetes patients. Because of the methodological limitations of the study, its importance regarding the validity of arriba is limited. The same authors reported indications of a risk overestimate for persons with a moderate or high risk for two investigated versions of PROCAM (21). The limitations mentioned above partly apply here too.

Data on the external validation of SCORE-2 for Germany on the basis of different cohorts have been published recently and show an AUC between 0.68 and 0.78 (5). German data on calibration have been published for the SCORE instrument, but not for SCORE2 (data from SHIP Study, study period 1997–2001 and KORA Study ([KORA, Kooperative Gesundheitsforschung in der Region Augsburg—cooperative health services research in the Augsburg region]; study period 1999–2001, age standardized state-related mortality data) [9–11]. Depending on age group, the calculated P/O ratios for SHIP were 10.5–9.28 for men and 0.26–3.15 for women. The results show an underestimate for women aged 40–54 years, and an overestimate for women aged 55–64 years as well as for men aged 45–65. The overestimate was greater the older the age group and was greater for the SHIP than for the KORA study populations. Because of divergent categories and methods, these data are not directly comparable with ours. It is not known whether SCORE2 can be assumed to have similar characteristics.

Three review articles compared a selection of cardiovascular risk prediction instruments in the time period to 2016 regarding measures of discrimination (14, 15, 22). For SCORE, AUC values (externally validated) of 0.65–0.87 and P/O ratios of 0.28–1.50 were reported in different international populations; for the PROCAM instrument the AUC values were 0.61 and 0.64. The AUC value for the arriba instrument of 0.84 [0.81; 0.86) in our study is comparable with the values published for SCORE and SCORE2 and higher than that for the PROCAM instrument.

A limitation regarding the quality of the prediction for risk prediction instruments is the need for continuous recalibration because of the decrease in cardiovascular events over time and because of the changes in medication treatments and their effect on cardiovascular risk factors and events (14, 23, 24). These challenges have so far not been resolved for any of the published instruments (25).

Suitability for clinical use

In addition to validity, the selection of appropriate endpoints and risk factors, the practical applicability and the evidence of patient-relevant effects are criteria for the good clinical applicability of a risk prediction instrument (14).

Of the risk prediction instruments in use in Germany, the endpoints of the arriba, PROCAM, and SCORE2 instruments (cardiovascular morbidity or mortality) enable the implementation of the regulations of the Federal Joint Committee regarding the prescription of lipid lowering drugs from an overall cardiovascular 10-year risk of ≥20 % (1, 5, 12, 26). The arriba instrument considers—in contrast to SCORE (SCORE2) diabetes and HbA1c as risk predictors; the differentiation of the cardiovascular risk in persons with type 2 diabetes was, however, improved in SCORE2 (1, 2, 27).

With regard to applicability in routine clinical practice, an intervention study showed that arriba Herz risk counseling can be integrated in the Disease Management Program for coronary heart disease, and an observational study showed that less than 10% of doctors applying the model reported an unacceptably longer consultation time if they administered the computer based arriba instrument (28, 29).

An effect on endpoints such as overall mortality or cardiovascular mortality has thus far not been confirmed for any of the cardiovascular risk prediction instruments (30). Collins et al showed in a meta-analysis a slight reduction of systolic blood pressure, total and LDL cholesterol, and a slight increase in abstinence from smoking. All effects were categorized as uncertain because of unsatisfactory study quality and short follow-up periods (3). For the arriba instrument, an intervention study showed an effect of physician counseling on the prescription rate of antiplatelet drugs in the follow-up consultations, but not on prescription rates of statins or antihypertensive medications (31). A single consultation with the arriba instrument did not lead to change in smoking behavior in another intervention study (28).

The principle of joint decision making of doctors and patients, however, is in contrast to the expectation that using risk prediction instruments results in better risk constellations and a corresponding reduction in disease events. Those decisions of patients that do not result in the optimization of cardiovascular risk factors also have to be accepted.

Regarding patient relevant effects, a randomized controlled trial for the paper-based version of the arriba instrument showed greater satisfaction for patients in the intervention group as well as a subjectively experiences better participation in decision making during the consultation (32). An observational study also found high satisfaction among patients for the consultation using the electronic version; 81% of patients reported that they had implemented the decision made as a result of the consultation (33).

Study results have shown that 15–20% of patients overestimate their own cardiovascular risk and 13–40% underestimate it (34). In contrast to the SCORE and PROCAM instruments, the absolute cardiovascular risk can be depicted by age, so that individual categorization is possible.

For the SCORE and PROCAM instruments, we are not aware of any studies investigating their applicability in clinical practice using German data.

Strengths and limitations

To our knowledge, our analysis was the first to study the predictive validity of the arriba instrument over an observation period of 10 years. We used data from a population-based cohort study with a quality assured comprehensive data collection of cardiovascular risk factors. We used inverse probability weights to reduce the effect of bias as a result of selective participation. This was feasible for the parameter calibration but not for discrimination. Confounding as a result of selective participation is possible. Overall, the validity of the results is limited because of the limited number of available observations and cardiovascular events (19). Difference between the time period of the risk prediction (=10 years) and individual inconsistent follow-up periods, as well as a missing date for the timing of the cardiovascular event can lead to overestimates or underestimates. Lacking quality in documenting fatal cardiovascular events from the coded main cause of death are known from other studies (false positive results documented in up to 30% of coded cases) (35, 36). In our study, no precise main cause of death was given for 25 persons (4%, 25/567) of all those deceased in the observation period, with the result that assuming a missing cardiovascular event in these persons may have prompted an underestimate in the event rate. Family history as a cardiovascular risk factor was defined as negative for all included persons since the age of the relatives at onset of the cardiovascular disorder had not been recorded. In this context, the overall cardiovascular risk may be subject to an overestimate or underestimate. No untreated cohort was analyzed in our study. If in patient groups at higher risk the observed risk is lower than the predicted risk, this may be the result of an overestimate by the score or the result of effective prevention (37).

Conclusions

Guidelines for cardiovascular prevention consistently recommend predicting the overall cardiovascular risk. Our analyses show that the validity of the arriba instrument is comparable to other risk prediction instruments used in Germany. The analyses support the recommendation of the German guideline for risk counseling for cardiovascular prevention in primary care in favor of the arriba instrument. Guideline recommendations for cardiovascular risk prediction instruments should—in addition to risk prediction—also consider aspects of clinical applicability.

Conflict of interest statement
Professor Chenot received financial funding from the Zentralinstitut für die kassenärztliche Versorgung [the central institute for healthcare delivered by statutory health insurance physicians] for a research project on the subject of this article, which he initiated.

Dr Donner-Bantzhoff holds share in Novartis and Alcon. He is involved in developing arriba, joint managing director or the charitable Society for patient Centered Communication, and chairman of the arriba cooperative.

Dr Haasenritter is a member of the guideline committee of DEGAM, joint author and coordinator of the DEGAM guideline on “general practitioner advice on cardiovascular risk prevention.”

Dr Angelow, Prof Donner-Banzhoff, and Prof Chenot were coauthors on the DEGAM guideline.

The remaining authors declare that no conflict of interest exists.

Manuscript received on 21 December 2021, revised version accepted on 4 May 2022.

Translated from the original German by Birte Twisselmann, PhD.

Corresponding author
Dr. med. Aniela Angelow
Universitätsmedizin Greifswald
Abteilung Allgemeinmedizin
Institut für Community Medicine
Fleischmannstraße 6, 17475 Greifswald

Cite this as:
Angelow A, Klötzer C, Donner-Banzhoff N, Haasenritter J, Schmidt CO, Dörr M, Chenot JF: Validation of cardiovascular risk prediction by the arriba instrument—an analysis based on data from the Study of Health in Pomerania. Dtsch Arztebl Int 2022; 119: 476–82. DOI: 10.3238/arztebl.m2022.0220

►Supplementary material

eReferences, eBox, eTables:
www.aerzteblatt-international.de/m2022.0220

References

1. Deutsche Gesellschaft für Allgemeinmedizin und Familienmedizin: www.degam.de/degam-leitlinien-379.html (last accessed on 8 December 2021).

2. Deutsche Gesellschaft für Kardiologie: http://leitlinien.dgk.org/2022/pocket-leitlinie-praevention-von-herz-kreislauf-erkrankungen-version-2021/leitlinien.dgk.org/2021/2021-esc-guidelines-on-cardiovascular-disease-prevention-in-clinical-practice/) (last accessed on 13 April 2022).

3. arriba: Kardiovaskuläre Prävention. http://arriba-hausarzt.de/module/kardiovaskuläre-prävention (last accessed on 8 December 2021).

4. Donner-Banzhoff N, Keller H, Sadowski E-M, et al.: Individualisierte Risikoprognose für die Herz-Kreislauf-Prävention: ein Werkstattbericht. In: Kunz R, Ollenschläger G, Raspe H, Jonitz G (eds.): Lehrbuch Evidenzbasierte Medizin in Klinik und Praxis. Bern: Verlag Hans Huber 2007; 295–304.

5. SCORE2 working group and ESC cardiovascular risk collaboration, SCORE2 risk prediction algorithms: new models to estimate 10-year risk of cardiovascular disease in Europe. Eur Heart J 2021; 42: 2439–54.

6. SCORE2-OP working group and ESC cardiovascular risk collaboration, SCORE2-OP risk prediction algorithms: estimating incident cardiovascular event risk in older persons in four geographical risk regions. Eur Heart J 2021; 42: 2455–67

7. European Society of Cardiology ESC). Heart Score. www.heartscore.org/en_GB/ (last accessed on 2 June 2022).

8. Keil U, Fitzgerald AP, Gohlke H, Wellmann J, Hense HW: Risikoabschätzung tödlicher Herz-Kreislauf-Erkrankungen: die neuen SCORE-Deutschland-Tabellen für die Primärprävention. Dtsch Arztebl 2005; 102: A 1808–12

9. Hense HW, Koesters E, Wellmann J, Meisinger C, Völzke H, Keil U: Evaluation of a recalibrated systematic coronary risk evaluation cardiovascular risk chart: results from systematic coronary risk evaluation Germany. Eur J Cardiovasc Prev Rehabil 2008; 15: 409–15

10. Rücker V, Keil U, Fitzgerald AP, et al.: Predicting 10-year risk of fatal cardiovascular disease in Germany: an update based on the SCORE-Deutschland risk charts. PLoS One 2016; 11: e0162188

11. Holle R, Happich M, Löwel H, Wichmann HE: MONICA/KORA study group. KORA—a research platform for population based health research. Gesundheitswesen 2005; 67 Suppl 1: S19–25

12. Assmann-Stiftung für Prävention 2018: Procam Gesundheitstests. https://www.assmann-stiftung.de/procam-tests/ (last accessed on 8 December 2021).

13. Assmann G, Schulte H, Cullen P, Seedorf U: Assessing risk of myocardial infarction and stroke: new data from the Prospective Cardiovascular Münster (PROCAM) study. Eur J Clin Invest 2007; 37: 925–32

14. Gorenoi V, Hagen A: Overview of risk-estimation tools for primary prevention of cardiovascular diseases in European populations. Cent Eur J Public Health 2015; 23: 91–9

15. Cooney MT, Dudina A, D’Agostino R, Graham IM: Cardiovascular risk-estimation systems in primary prevention: do they differ? Do they make a difference? Can we see the future? Circulation 2010; 122: 300–10

16. Sullivan LM, Massaro JM, D’Agostino RB Sr.: Presentation of multivariate data for clinical use: the framingham study risk score functions. Stat Med 2004; 23: 1631–60

17. Brindle P, Emberson J, Lampe F, et al.: Predictive accuracy of the framingham coronary risk score in British men: prospective cohort study. BMJ 2003; 327: 1267

18. Völzke H, Alte D, Schmidt CO, et al.: Cohort profile: the study of health in Pomerania. Int J Epidemiol 2011; 40: 294–307

19. Van Calster B, Nieboer D, Vergouwe Y, De Cock B, Pencina MJ, Steyerberg EW: A calibration hierarchy for risk models was defined: from utopia to empirical data. J Clin Epidemiol 2016; 74: 167–76

20. Collins GS, Altman DG: An independent and external validation of QRISK2 cardiovascular disease risk score: a prospective open cohort study. BMJ 2010; 340: c2442

21. Grammer TB, Dressel A, Gergei I, et al.: Cardiovascular risk algorithms in primary care: results from the DETECT study. Sci Rep 2019; 9: 1101

22. Damen JAAG, Hooft L, Schuit E, et al.: Prediction models for cardiovascular disease risk in the general population: systematic review. BMJ 2016; 353: i2416

23. Liew SM, Doust J, Glasziou P: Cardiovascular risk scores do not account for the effect of treatment: a review. Heart 2011; 97: 689–97

24. Mehta S, Jackson R, Wells S, Harrison J, Exeter DJ, Kerr AJ: Cardiovascular medication changes over 5 years in a national data linkage study: implications for risk prediction models.Clin Epidemiol 2018; 10: 133–141

25. Rospleszcz S, Thorand B, de Las Heras Gala T, et al.: Temporal trends in cardiovascular risk factors and performance of the framingham risk score and the pooled cohort equations. J Epidemiol Community Health 2019 ;73: 19–25

26. Gemeinsamer Bundesausschuss (GBA): Anlage III – Übersicht über Verordnungseinschränkungen und -ausschlüsse in der Arzneimittelversorgung durch die Arzneimittel-Richtlinie und aufgrund anderer Vorschriften (§ 34 Absatz 1 Satz 6 und Absatz 3 SGB V). 2016. www.g-ba.de/downloads/83-691-714/AM-RL-III-Verordnungseinschraenkung_2022-01-18.pdf (last accessed on 2 June 2022).

27. Bulugahapitiya U, Siyambalapitiya S, Sithole J, et al.: Is diabetes a coronary risk equivalent? Systematic review and meta-analysis. Diabet med 2009; 26: 142–8

28. Bleckwenn M, Münster S, Weckbecker K: Motivation of high-risk patients to stop smoking in the DMPM. Gesundheitswesen 2015; 77: 885–7

29. Hirsch O, Keller H, Krones T, Donner-Banzhoff N: Arriba-lib: evaluation of an electronic library of decision aids in primary care physicians. BMC Med Inform Decis Mak 2012; 12: 48

30. Collins DR, Tompson AC, Onakpoya IJ, Roberts N, Ward AM, Heneghan CJ: Global cardiovascular risk assessment in the primary prevention of cardiovascular disease in adults: systematic review of systematic reviews. BMJ Open 2017; 7: e013650

31. Keller H, Krones T, Becker A, et al.: Arriba: effects of an educational intervention on prescribing behaviour in prevention of CVD in general practice. Eur J Prev Cardiol 2012; 19: 322–9

32. Krones T, Keller H, Sönnichsen A, et al.: Absolute cardiovascular disease risk and shared decision making in primary care: a randomized controlled trial. Ann Fam Med 2008; 6: 218–27

33. Hirsch O, Keller H, Krones T, Donner-Banzhoff N: Acceptance of shared decision making with reference to an electronic library of decision aids (arriba-lib) and its association to decision making in patients: an evaluation study. Implement Sci 2011; 6: 70

34. van der Weijden T, Bos LB, Koelewijn-van Loon MS: Primary care patients’ recognition of their own risk for cardiovascular disease: implications for risk communication in practice. Curr Opin Cardiol 2008; 23: 471–6

35. Modelmog D, Rahlenbeck S, Trichopoulos D: Accuracy of death certificates: a population-based, complete-coverage, one-year autopsy study in East Germany. Cancer Causes Control 1992; 3: 541–6

36. Pagidipati NJ, Gaziano TA: Estimating deaths from cardiovascular disease: a review of global methodologies of mortality measurement. Circulation 2013; 127: 749–56

37. Pajouheshnia R, Peelen LM, Moons KGM, Reitsma JB, Groenwold RHH: Accounting for treatment use when validating a prognostic model: a simulation study. BMC Med Res Methodol 2017; 17: 103

e1. European Guidelines for Cardiovascular Disease Prevention, Eur Heart J 2007; 28: 2375–414.

e2. The Expert Committee on the diagnosis and classification of diabetes mellitus. Follow-up report on the diagnosis of diabetes mellitus. Diabetes Care 2003; 26: 3160–67

e3. Latza U, Hoffmann W, Terschüren C, et al.: Erhebung, Quantifizierung und Analyse der Rauchexposition in epidemiologischen Studien. Berlin: Robert Koch-Institut; 2007.