Dilemmas and solutions- experiences of a national Family Medicine applied knowledge licensing test during a pandemic

ABSTRACT: Background: The unprecedented COVID-19 pandemic brought significant challenges to all of medicine, including primary care training and examinations. The MRCGP AKT is high-stakes licensing 200-item MCQ for UK trainee family physicians and is part of an assessment tripos that, up to the onset of the pandemic, included a Clinical Skills Assessment using Simulated Patients and workplace based assessment. The AKT is blueprinted onto a curriculum content specification and computer delivered three times a year at test centres across the UK. It tests the knowledge base underpinning independent general practice within the context of the UK National Health Service. We report on the challenges and dilemmas faced during the pandemic, decisions taken, and lessons learned. Rapid exam changes needed to be made, and communicated effectively to candidates, whilst maintaining standards and fairness to candidates.

Summary of Work: Challenges included lockdown travel restrictions, reduced capacity, social distancing and shielding candidates being unable to leave home. The April 2020 AKT was cancelled and prioritisation measures implemented to ensure candidates at the end of their training could enter the (stressed) workforce. We engaged with a wide range of stakeholders, carefully looked at remote testing, made contingency plans prioritised for those unable to sit exams and changed exam regulations to ensure fairness to candidates. In this emergency, we delivered a previously published exam which some candidates were unaware they had sat previously, and assessed how these candidates performed. We compared cohort performance before and during the pandemic.

Summary of Results: We summarise why we did not remote test, how we obtained key worker status, and adapted contingency plans. Analysis of candidates who had previously sat the same exam showed they performed less well. Despite wide-ranging changes in training and workplace experience, there was no significant difference in cohort performance overall pre-and peri-pandemic.

Discussion and Conclusions: COVID-19 constraints changed trainees clinical exposure, restricted training and supervisor support. However, exam preparedness did not appear adversely affected when measured by overall pass rates. Unexpectedly, candidates who sat an identical exam did not benefit from previous exposure. Take-home Messages: Involving stakeholders in key decisions and regular communications are essential. Test security and standards were not compromised.