‘Success’ in policy piloting: process, programmes and politics

Research has demonstrated that pilots contain multiple shifting purposes, not all of which relate to simple policy testing or refinement. Judging the success of policy pilots is therefore complex, requiring more than a simple judgement against declared goals. Marsh and McConnell provide a framework against which policy success can be judged, distinguishing programme success from process and political success. We adapt Boven's modification of this framework and apply it to policy pilots, arguing that pilot process, outcomes and longer term effects can all be judged in both programme and political terms. We test this new framework in a pilot programme in the English National Health Service, the Vanguard programme, showing how consideration of these different aspects of success sheds light on the programme and its aftermath. We consider the implications of the framework for the comprehensive and multi-faceted evaluation of policy pilots.