Continuing Medical Education in Deutsches Ärzteblatt – an Analysis

Two weeks ago, Deutsches Ärzteblatt published its 46th CME article, an ophthalmology article entitled "Basic Knowledge of Refractive Surgery." The participation rate observed thus far indicates that, as with our previous offerings, 15 000 to 20 000 physicians will have taken part in the educational activity by the end of the six-week valid credit period. The great interest in our CME activities has thus continued unabated since we began offering this series to physicians in September 2004. Based on our readers' answers to the feedback queries that accompany each article, we would like to present an interim report on the CME activities in Deutsches Ärzteblatt, now in their fourth year.

Participation statistics
Between 1 September 2004 and 2 March 2008, readers took part in our CME activities a total of 861 770 times. This means that an average of 19 000 readers participated in each activity. The total number of participants amounted to 82 189, which represents approximately one-quarter of all practicing physicians in Germany. On the average, each participant took part in 10 different activities (figure 1 gif ppt). These figures indicate that the CME program offered by Deutsches Ärzteblatt is the most successful of its kind among the many German-language journals that publish such articles. The Ärztekammer Nordrhein (North Rhine Medical Association) alone has certified a total of 64 journals to offer CME activities (Prof. Reinhard Griebenow, Ärztekammer Nordrhein, Feb. 2008, personal communication).

Looking at demographic characteristics reveals several differences between our CME participants and the general population of physicians in Germany. Whereas women represent 40% of practicing physicians in this country, they made up 44% of those who took part in our CME activities. With regard to age, 34% of practicing physicians in Germany are between 40 and 50 years old, but among our CME participants a full 42% were in this age group. In contrast, physicians between 60 and 69 years of age, who comprise 11% of all practicing physicians in Germany, were underrepresented in our sample, at only 4%.

Of the physicians who participated in our CME activities, 38% were in private practice, compared to 41% nationwide. A total of 47% of our CME participants worked in a hospital, compared to 48% of practicing physicians in Germany. Our data also show that many of those who participated in our CME activities were not required to take part in further education programs, but did so of their own volition: thirty-two percent of our CME participants were residents (Assistenzärzte), many of whom were presumably not yet specialists (Fachärzte). Moreover, 4% of the participating physicians were not practicing at the time of participation, and one out of 200 was in retirement.

As could already be observed in our last survey (1), looking at participating physicians according to their specialty revealed an above-average proportion of anesthetists: indeed, one out of seven physicians who took part in our CME activities belonged to this group, although they represent only 6% of all physicians in Germany. The other most frequently represented groups among our participants were internists (17% of participants versus 13% of all physicians), general practitioners (14% of participants versus 13% of all physicians), and surgeons (8% of participants versus 9% of all physicians).

Thus, whereas the other specialists were represented among our CME participants roughly according to their proportion in the general population of physicians in Germany, the percentage of anesthetists among our CME participants has remained consistently more than twice as high – a phenomenon for which we have yet to find an explanation.

The geographic distribution of participants was very even; hardly any differences could be observed. Corresponding to the size of their respective Medical Associations, most participants came from Bavaria (15% versus 16.2% of all physicians in Germany), Baden-Württemberg (14% versus 12.9%), and from the North Rhine Medical Association (14% versus 12.2%).

Reader feedback
Feedback from our readers shows a high degree of satisfaction with our CME activities, with more than three-quarters of participants rating the articles as "good" or "excellent" (figure 2 gif ppt). Similarly, three out of four participants indicated that the topic in question was "very well presented" from the perspective of everyday practice. One out of 10 participants noted that they would like the articles to contain more detailed information.

Our reader feedback also mirrors the fact that Deutsches Ärzteblatt, in contrast to the specialty journals, is aimed at a broad readership. Although 54% of our participants indicated that they had "open questions" about the respective topic or "no set strategy" prior to the CME activity, a full 16% described the topic under discussion as "irrelevant" (figure 3 gif ppt). Here, it is likely that we are coming up against limiting factors related to our diverse readership. After all, it is difficult, if not impossible, to find a topic that will be relevant to each and every reader of Deutsches Ärzteblatt.

In light of our broad readership, ensuring that our articles are written in a clear and comprehensible way is paramount. Judging from reader feedback, it appears that our authors – all of whom are specialists in their respective field – have achieved this goal: almost all participants (96%) indicated that the articles were suitable for all physicians, whereas only 4% described the articles as "suitable only for specialists." Here, of course, we need to take into account the possibility that readers who did not understand a particular article may have been less likely to take part in the evaluation and provide feedback.

The vast majority of participants (92%) were able to answer the test questions correctly after reading the article; only a minority of participants (8%) needed to refer to additional, outside literature to find a solution.

The test questions and their answers
Naturally, the questions for the articles are designed so that they can be answered without referring to outside sources. After all, CME activities are not university examinations, but rather further training for physicians who have already obtained their credentials. As specified by the Medical Associations that accredit these types of activities, the questions are meant only to test whether a physician has read a particular article and understood it correctly. Indeed, in many CME courses that require attendance there are no test questions at all, and when tests are administered, the granting of CME credits is not conditional upon the results.

On the average, our readers were able to answer approximately 95% of all questions correctly. This corresponds to the results seen in other journals: in Der Internist, for example, 93% of all questions were answered correctly (2), and among the journals published by Springer Verlag that offer CME credits, correct answers were given in 83% to 97% of all cases. Among journals published by Thieme-Verlag, the pass rate was 92%, which also indicates a high proportion of correct answers. Similar figures can be seen in CME activities offered by the specialty journals published by Schattauer-Verlag, which have a pass rate of 86% (personal communication, Feb. 2008, Dres. P. Herrmann, Springer, V. Hirschel, Thieme, J. Hueber, Schattauer).

Multiple-choice questions are not ideal for testing a participant's comprehension. Many of our readers will have less-than-fond memories of these from their days at university. Nonetheless, we are dependent on them, as they represent the only way to conduct an evaluation in print journals. However, although multiple-choice questions can be found everywhere in the world of medicine, phrasing them correctly still represents a challenge. Studies of German-language medical journals, including Deutsches Ärzteblatt, have discovered formal errors in the CME questions (2, 3), although it should be pointed out that even negatively phrased questions ("Which of the following options is incorrect…") are considered errors in form. A study by Stagnaro-Green und Downing (4) has found formal errors in the CME questions published by the New England Journal of Medicine. In other words, these shortcomings do not render the questions worthless; indeed, even the Institute for Medical and Pharmaceutical Examination Questions (Institut für Medizinische und Pharmazeutische Prüfungsfragen) in Mainz has been known to make an occasional mistake.

We have received a broad range of feedback on our CME questions. For example, one reader writes, "The questions for this article, which deals with a very important medical topic, are highly sophisticated – too academic." In contrast, others have written, "Usually the questions are closely correlated with the CME article and can be answered after studying the article carefully"; or, "the CME articles offered by Deutsches Ärzteblatt are almost always of practical relevance. […] Whether I'm granted a CME credit or not is really just a question of competitive spirit."

An analysis of 370 questions posed by Deutsches Ärzteblatt revealed a surprisingly low number of differences between subgroups of participants with respect to the proportion of correct answers. When looking at different groups of participants in terms of age, specialty, Medical Association membership, and other variables, most of the differences were significant, but nevertheless so small that they had no practical relevance. Although a gender comparison shows that women participants performed significantly better than men, there was only a 0.01-question difference between the two groups, i.e. an average of 9.55 (standard deviation 0.81) versus 9.54 (SD 0.83) correct answers. This marginal difference may be due to the large proportion of correct answers (i.e. the ceiling effect). However, this difference between the groups remained inconsequential even when we considered only those multiple-choice questions that were answered correctly by fewer than 90% of participants.

Overall, our data show that there is great demand for the CME activities offered by Deutsches Ärzteblatt, and that these activities are rated positively by those who have taken part in them. The data also show that there is room for improvement. We will continue to place great store in the independence of Deutsches Ärzteblatt and its content, and will rely in the future, as we have in the past, on our rigorous review process to ensure the high scientific quality of the submitted manuscripts. We are delighted that so many readers have chosen to take part in our CME activities, and would like to take this opportunity to wish all of you an enjoyable and rewarding learning experience.

Conflict of interest statement
The authors declare that no conflict of interest exists according to the guidelines of the International Committee of Medical Journal Editors.


PD Dr. med. Christopher Baethge
Chief Scientific Editor
Catrin Marx
Editor, Section CME

Dtsch Arztebl Int 2008; 105(11): 204–6
DOI: 10.3238/arztebl.2008.0204

References

1. Marx C: CME im Deutschen Ärzteblatt: 43 000 Teilnehmer – sehr gute Bewertung. Dtsch Arztebl 2005; 102(37): A 2453–6.

2. Kühne-Eversmann L, Nussbaum C, Reincke M, Fischer MR: CME-Fortbildungsangebote in medizinischen Fachzeitschriften: Strukturqualität der MC-Fragen als Erfolgskontrollen. Med Klin 2007; 102 (12): 993–1001.

3. Rotthoff T, Fahron U, Baehring T, Scherbaum WA: Die Qualität von CME-Fragen in der ärztlichen Fortbildung – eine empirische Studie. Z ärztl Fortbild Qual Gesundh wes 2008; 101: 667–74.

4. Stagnaro-Green AS, Downing SM: Use of flawed multiple-choice items by the New England Journal of Medicine for continuing medical education. Medical Teacher 2006; 28: 566–8.