Dr Alexandra (Alex) Penn FRSA

From 2016 to 2018 Defra worked with the CECAN Case Study Team, led by Newcastle and York Universities, to help inform and improve their approach to evaluating rural development policy in England. The complexity of the policy presented particular challenges for evaluation. The case study aimed to help Defra meet these challenges through the testing and development of evaluation models and building capacity to incorporate complexity thinking throughout the policy cycle.

This briefing explains what complexity science and systems thinking means for people developing and delivering policy. It also introduces a common language and set of symbols to help frame thinking, conversations and action on complexity.

This is an Evaluation Policy and Practice Note that explores the application of dependency models in complex policy evaluation.

The energy trilemma describes the interaction in the energy system between sustainability and emissions, affordability and prices, and security of supply. The sheer number of programmes and policies with close interaction and overlap in this area has led to a crowded and complex policy landscape with a range of potentially complementary and conflicting aims. In this case study, CECAN and the Department for Business, Energy and Industrial Strategy (BEIS) worked together to build a richer understanding of this complex area by developing a participatory systems map of the energy trilemma.

The field of industrial ecology applies ecosystem theory to industrial production, human consumption and societies. This article presents a case study of the development of the bio-based economy in the area surrounding the Humber estuary in the North-East of England. The study developed an agent-based model to simulate the evolution of the industrial system. We explain how the qualitative research process led to the development of a toy model that has successively been specified

Understanding and manipulating bacterial biofilms is crucial in medicine, ecology and agriculture and has potential applications in bioproduction, bioremediation and bioenergy. Biofilms often resist standard therapies and the need to develop new means of intervention provides an opportunity to fundamentally rethink our strategies. Conventional approaches to working with biological systems are, for the most part, “brute force”, attempting to effect control in an input and effort intensive manner and are often insufficient when dealing with the inherent non-linearity and complexity of living systems. Biological systems, by their very nature, are dynamic, adaptive and resilient and require management tools that interact with dynamic processes rather than inert artefacts. I present an overview of a novel engineering philosophy which aims to exploit rather than fight those properties, and hence provide a more efficient and robust alternative. Based on a combination of evolutionary theory and whole-systems design, its essence is what I will call systems aikido; the basic principle of aikido being to interact with the momentum of an attacker and redirect it with minimal energy expenditure, using the opponent’s energy rather than one’s own. In more conventional terms, this translates to a philosophy of equilibrium engineering, manipulating systems’ own self-organisation and evolution so that the evolutionarily or dynamically stable state corresponds to a function which we require. I illustrate these ideas with a description of a proposed manipulation of environmental conditions to alter the stability of co-operation in the context of Pseudomonas aeruginosa biofilm infection of the cystic fibrosis lung.

The use of complexity science in evaluation has received growing attention over the last 20 years. We present the use of a novel complexity-appropriate method – Participatory Systems Mapping – in two real-world evaluation contexts and consider how this method can be applied more widely in evaluation. Participatory Systems Mapping involves the production of a causal map of a system by a diverse set of stakeholders. The map, once refined and validated, can be analysed and used in a variety of ways in an evaluation or in evaluation planning. The analysis approach combines network analysis with subjective information from stakeholders. We suggest Participatory Systems Mapping shows great potential to offer value to evaluators due to the unique insights it offers, the relative ease of its use, and its complementarity with existing evaluation approaches and methods.

Central government guidance seeks to ensure and enhance the quality of practice and decision-making across – and sometimes beyond – government. The Magenta Book, published by HM Treasury, is the key UK Government resource on policy evaluation, setting out central government guidance on how to evaluate policies, projects and programmes. The UK Centre for the Evaluation of Complexity Across the Nexus was invited to contribute its expertise to the UK Government’s 2020 update of the Magenta Book by developing an accompanying guide on policy evaluation and ‘complexity’. A small multidisciplinary team worked together to produce a set of guidance, going through multiple stages of work and drawing on a variety of sources including academic and practitioner literature and experts and stakeholders in the fields of evaluation, policy and complexity. It also drew on Centre for the Evaluation of Complexity Across the Nexus’ own work developing and testing evaluation methods for dealing with complexity in evaluation. The resulting Magenta Book 2020 Supplementary Guide: Handling Complexity in Policy Evaluation explores the implications of complexity for policy and evaluation and how evaluation can help to navigate complexity. This article, designed primarily for practitioners who might be interested in this guidance and how it was developed, describes the processes involved, particularly related to the interdisciplinary dialogue and consultation with other key stakeholders that this involved. It also briefly outlines the content and key messages in the guidance, with reflections on the experiences of the authors in developing the guide – including the challenges and insights that arose during the process, particularly around the challenges of communicating complexity to a broad audience of readers.

Selection on the level of loosely associated groups has been suggested as a route towards the evolution of cooperation between individuals and the subsequent formation of higherlevel biological entities. Such group selection explanations remain problematic, however, due to the narrow range of parameters under which they can overturn within-group selection that favours selfish behaviour. In principle, individual selection could act on such parameters so as to strengthen the force of between-group selection and hence increase cooperation and individual fitness, as illustrated in our previous work. However, such a process cannot operate in parameter regions where group selection effects are totally absent, since there would be no selective gradient to follow. One key parameter, which when increased often rapidly causes group selection effects to tend to zero, is initial group size, for when groups are formed randomly then even moderately sized groups lack significant variance in their composition. However, the consequent restriction of any group selection effect to small sized groups is derived from models that assume selfish types will competitively exclude their more cooperative counterparts at within-group equilibrium. In such cases, diversity in the migrant pool can tend to zero and accordingly variance in group composition cannot be generated. In contrast, we show that if within-group dynamics lead to a stable coexistence of selfish and cooperative types, then the range of group sizes showing some effect of group selection is much larger. © 2007 alifexi.org.

The value of complexity science and related approaches in policy evaluation have been widely discussed over the last 20 years, not least in this journal. We are now at a crossroads; this Special Issue argues that the use of complexity science in evaluation could deepen and broaden rendering evaluations more practical and rigorous. The risk is that the drive to better evaluate policies from a complexity perspective could falter. This special issue is the culmination of 4 years’ work at this crossroads in the UK Centre for the Evaluation of Complexity Across the Nexus. It includes two papers which consider the cultural and organisational operating context for the use of complexity in evaluation and four methodological papers on developments and applications. Together, with a strong input from practitioners, these papers aim to make complexity actionable and expand the use of complexity ideas in evaluation and policy practice.

Theory of Change diagrams are commonly used within evaluation. Due to their popularity and flexibility, Theories of Change can vary greatly, from the nuanced and nested, through to simplified and linear. We present a methodology for building genuinely holistic, complexity-appropriate, system-based Theory of Change diagrams, using Participatory Systems Mapping as a starting point. Participatory System Maps provide a general-purpose resource that can be used in many ways; however, knowing how to turn their complex view of a system into something actionable for evaluation purposes is difficult. The methodology outlined in this article gives this starting point and plots a path through from systems mapping to a Theory of Change evaluators can use. It allows evaluators to develop practical Theories of Change that take into account feedbacks, wider context and potential negative or unexpected outcomes. We use the example of the energy trilemma map presented elsewhere in this special issue to demonstrate.

This review concerns the methodological challenges that industrial ecology faces in integrating natural and social sciences. Network analysis can be seen as the most promising method to mediate between industrial ecology's overall systems approach and the complex structures found in society. It is a well established method across scientific disciplines, including the social sciences. It has been successfully applied in industrial ecology, in which localized phenomena of industrial symbiosis have been a key focus, and where metrics from both the social and natural sciences are used to understand socio-metabolic structures. In this paper we classify such studies as Social-Material Network Analyses and we discuss the body of work, drawing on network analyses from various disciplines. A challenge is the hierarchical nature of industrial networks and how it can be addressed socially. We discuss the opportunities and limitations of metric-driven network analysis and offer a review of methodological options for Social-Material Network Analyses. © 2014 The Authors.

In a drive to achieve net zero emissions, U.K. transport decarbonisation policies are predominantly focussed on measures to promote the uptake and use of electric vehicles (EVs). This is reflected in the COP26 Transport Declaration signed by 38 national governments, alongside city region governments, vehicle manufacturers and investors. However, emerging evidence suggests that EVs present multiple challenges for air quality, mobility and health, including risks from non-exhaust emissions (NEEs) and increasing reliance on vehicles for short trips. Understanding the interconnected links between electric mobility, human health and the environment, including synergies and trade-offs, requires a whole systems approach to transport policymaking. In the present paper, we describe the use of Participatory Systems Mapping (PSM) in which a diverse group of stakeholders collaboratively constructed a causal model of the U.K. surface transport system through a series of interactive online workshops. We present the map and its analysis, with our findings illustrating how unintended consequences of EV-focussed transport policies may have an impact on air quality, human health and important social functions of the transport system. We conclude by considering how online participatory causal modelling techniques could be effectively integrated with empirical metrics to facilitate effective policy design and appraisal in the transport sector.

Fuzzy cognitive mapping is commonly used as a participatory modelling technique whereby stakeholders create a semi-quantitative model of a system of interest. This model is often turned into an iterative map, which should (ideally) have a unique stable fixed point. Several methods of doing this have been used in the literature but little attention has been paid to differences in output such different approaches produce, or whether there is indeed a unique stable fixed point. In this paper, we seek to highlight and address some of these issues. In particular we state conditions under which the ordering of the variables at stable fixed points of the linear fuzzy cognitive map (iterated to) is unique. Also, we state a condition (and an explicit bound on a parameter) under which a sigmoidal fuzzy cognitive map is guaranteed to have a unique fixed point, which is stable. These generic results suggest ways to refine the methodology of fuzzy cognitive mapping. We highlight how they were used in an ongoing case study of the shift towards a bio-based economy in the Humber region of the UK. © 2013 Elsevier B.V. All rights reserved.