Effectiveness of education in evidence-based healthcare: the current state of outcome assessments and a framework for future evaluations

Background? A discipline which critically looks at the evidence for practice should itself be critically examined. Credible evidence for the effectiveness of training in evidence-based healthcare (EBHC) is essential. We attempted to summarise the current knowledge on evaluating the effectiveness of training in EBHC while identifying the gaps. Methods? A working group of EBHC teachers developed a conceptual framework of key areas of EBHC teaching and practice in need of evidence mapped to appropriate methods and outcomes. A literature search was conducted to review the current state of research in these key areas. Studies of training interventions that evaluated effectiveness by considering learner, patient or health system outcomes in terms of knowledge, skills, attitude, judgement, competence, decision-making, patient satisfaction, quality of life, clinical indicators or cost were included. There was no language restriction. Results? Of 55 articles reviewed, 15 met the inclusion criteria: six systematic reviews, three randomised controlled trials and six before-after studies. We found weak indications that undergraduate training in EBHC improves knowledge but not skills, and that clinically integrated postgraduate teaching improves both knowledge and skills. Two randomised controlled trials reported no impact on attitudes or behaviour. One before-after study found a positive impact on decision-making, while another suggested change in learners' behaviour and improved patient outcome. We found no studies assessing the impact of EBHC training on patient satisfaction, health-related quality of life, cost or population-level indicators of health. Conclusions? Literature evaluating the effectiveness of training in EBHC has focused on short-term acquisition of knowledge and skills. Evaluation designs were methodologically weak, controlled trials appeared inadequately powered and systematic reviews could not provide conclusive evidence owing to weakness of study designs.