Reconciling multiple counterfactuals when evaluating biodiversity conservation impact in social-ecological systems

When evaluating the impact of a biodiversity conservation intervention, a ‘counterfactual’ is needed, as true experimental controls are typically unavailable. Counterfactuals are possible alternative system trajectories in the absence of an intervention and comparing observed outcomes against the chosen counterfactual allows the impact (change attributable to the intervention) to be determined. Since counterfactuals are hypothetical scenarios, and by definition never occur, they must be estimated. Sometimes there may be many plausible counterfactuals, given that they can include multiple drivers of biodiversity change, and be defined on a range of spatial or temporal scales. Here we posit that, by definition, conservation interventions always take place in social-ecological systems (SES; ecological systems integrated with human actors). Evaluating the impact of an intervention within an SES therefore means taking into account the counterfactuals assumed by different human actors. Use of different counterfactuals by different actors will give rise to perceived differences in the impacts of interventions, which may lead to disagreement about its success or the effectiveness of the underlying approach. Despite that there are biophysical biodiversity trends, it is often true that no single counterfactual is definitively the ‘right one’ for conservation assessment, so multiple evaluations of intervention efficacy could be considered justifiable. Therefore, we propose the need to calculate a quantity termed the sum of perceived differences, which captures the range of impact estimates associated with different actors within a given SES. The sum of perceived differences gives some indication how closely actors within an SES agree on the impacts of an intervention. We illustrate the concept of perceived differences using a set of global, national and regional case studies. We discuss options for minimising the sum, drawing upon literatures from conservation science, psychology, behavioural economics, management and finance.