Could you outline your views on the measurement of outcomes?
The first thing is to understand it’s a very complex process. It’s complex but it can be looked at in a relatively uncomplex way, if I can say it that way. If you look at how a physician or any clinician would progress through an educational activity in developing his or her skill, that if you look at it in that way it’s probably a little bit more readily understood. It’s important first for people who plan educational activities to focus on what an individual should know how to do, should know what to do, how to do it and when to do it. Then once that’s accomplished there ought to be practice involved so that that individual can get feedback from someone who knows or who is an expert in the area. Providing practice over the course of an activity is very important; it’s one of the things that is missing from our current approach to doing continuing education. By practice I mean that cases are developed where the learner can be challenged at several different points during the management of a case and then provided feedback on whether or not he or she has made the right decision. That can be done, obviously individually would be best, but it can be done these days with some of the technology that exists in terms of in some places they’re called audience response systems. So you can stop the case at a certain point and poll the participants in terms of what they would do given the information that’s just provided.
In terms of measuring the outcomes I think that there are a couple of ways to do that. In many cases the people who are organising a continuing education activity don’t have access to the data that resides in electronic health records or in patient charts. Therefore, it’s important to try and develop a simulated setting in which you can create those tests, or those cases rather, and use the audience response system in the manner that I just described as a surrogate for the actual objective data that would be available in healthcare records.
How would you measure the response within e-learning?
You could take advantage of the interactive capability of the web and its multimedia capability so it might even be a little bit stronger in the sense of being able to provide some of the pictures of patients or charts or some of the tests, radiographs, the imagery and so forth that goes along with that. But the audience response system could still be used and if there’s access to data then it can be more individualised in terms of creation of patients that look like the patients that an individual doctor takes care of.
What were the key take-home messages from the ECME meeting?
I still think that physicians and other health professionals are motivated by wanting to improve their performance. So if what they’re provided through that time, even though there’s no credit, is meaningful to them in terms of helping them address an issue that they’ve identified, they’ll participate. Using money or other kinds of incentive approaches that are more or less gifts are not very effective and I would not recommend that at all.
Do you see any other ways of incentivising feedback?
The struggles that CME providers are having are significant but it reminds me of the conversations we were having in the US several years ago, maybe even back to the early ‘90s. I don’t mean that as a negative or critical comment, it’s just an observation that, for whatever reason, Europe is at that point in the development of their CME/CPD efforts. What needs to be done more than anything is that there needs to be regulation, there’s no question about that, and what the issue is now is whether or not it should be accreditation of programmes of activities or accreditation of activities. What should be done is a careful examination of what the customers of this process need and what would be best for them. So you’ve got providers on the one hand and you’ve got physicians on the other, what makes sense in terms of them, their needs. Certainly there is a need for quality and physicians want quality education so how do those three things come together in a way that makes sense is probably the important learning that I got out of the couple of days that I’ve been here.
What are the key things that Europe can learn from the US?
The key feature has been the conversation that’s occurred between professional groups of educators and physicians with the accreditation body. The accreditation body in the US has been very responsive in terms of soliciting comments from the stakeholders to be sure that what they’re proposing in terms of the next level of accreditation is appropriate. So there needs to be that sort of collegial conversation that occurs over time that an effective accreditation system will develop. It has to be owned by the people that it affects, I believe.
