Getting research ready for the messy world of systems change

Strategic Lead for Evaluation

Director of Research

Senior Researcher (Data Specialist)

Place-based and Complex Evaluation Lead

As we began the year, we set out our strategic priorities, with a core focus on the early years, mental health and safety for children, young people and families. Our work is increasingly concerned with systems change; working with partners to challenge and disrupt the conditions, structures and power that uphold inequalities in outcomes and experiences for children and young people.

Attempts to change these conditions – also called systems change approaches – need to match this complexity by being ambitious, flexible, multi-disciplinary, and iterative. Our systemic change and design teams work with partners to build these capabilities in practical, applied ways. Whether evaluating complex interventions or programmes aiming for wholesale systems change, our mixed methods seek to understand not just if but how change occurs.

“Alongside growing interest in systems change, there are calls for complexity-sensitive, systems change evaluation, and guidance for evaluators is growing. But real-life examples are few and far between. We are working to fill that gap. ”

In this blog, our Senior Research and Evaluation team – Julie Harris, Finlay Green, Charlotte Woodhead and Sean Manzi explore some of the progressive methods we are using to engage with the complexity of systemic change.

Evaluating Systemic Change

This year, we’re drawing to a conclusion our summative evaluation of the Lambeth Early Action Partnership (LEAP). This is one of five A Better Start sites that received £38 million from the National Lottery Community Fund to improve the life chances of babies, young children, and families, from 2015 to 2025. Working alongside our team of community researchers, we’re combining realist evaluation, contribution analysis and theories of change to help LEAP understand not just whether they’ve helped to tackle inequalities in their community but how and why the context surrounding Lambeth, LEAP, and families has shaped their contribution. We’ll soon be hosting learning events later this summer and launching a final report and podcast where you can hear more about how we’re working with others to put this kind of research into practice.

We are also embarking on a range of exciting developmental evaluations of place-based systemic change — approaches that seek to generate actionable learning as initiatives unfold.

One of our newest projects is a five-year partnership with Impact on Urban Health, who have funded Old Kent Road Family Zone to work alongside a team of community researchers as they aim to strengthen governance arrangements and understanding of local needs to tackle drivers of inequalities and help children thrive. Our developmental evaluation on this will bolster learning and support equitable approaches to decision-making to affect change.

In one of our projects that aims to address the root causes of young people’s mental health in Northern Devon and Newham, we are shaping the design and integrating learning from UCL and Exeter University's developmental and realist evaluation of Kailo. This involves exploring the ‘ripple effects’ of how our work impacts the contexts that affect young people’s mental health and wellbeing. 

Our evaluation and research aims

Through our evaluation research, we aim to build shared knowledge and understanding of issues by centring the voices of experts through every level of the system – be they policymakers, service planners and commissioners, managers, professionals bringing their practice wisdom, or young people and families voicing their lived experience of services. For example, we’ve been commissioned by the Department of Education and Youth Endowment Fund (YEF) – in partnership with Rand Europe — to undertake research that explores how young people are impacted by serious youth violence and their interactions with systems of support. We will be supported in the design and delivery of this project by the Young Persons’ Research Advisory Panel (YRAP) at the University of Bedfordshire’s ‘Safer Young Lives Research Centre’, ensuring young people’s experiences and perspectives are centred throughout. Finally, we’ve also been commissioned by the Youth Futures Foundation – alongside our collaborators at the Centre for Evidence and Implementation – to produce guidance on theory-based evaluation. This is part of our work to help build the field and advance nuanced and practically applied research methodologies. 

Research, evaluation and the AI revolution

We have long been at the fore of progressive applications of research. It’s clear that artificial intelligence (AI) will fundamentally change the way that we develop, deliver and evaluate services for children and young people – bringing a wealth of opportunities and challenges to navigate. 

AI will transform how we predict needs and the influences on children and families, personalise services, and model the redesign of systems of support. It will also help expedite and analyse vast bodies of research and data. This is an exciting prospect for those exploring long-term change, developmental evaluations or piloting programmes.

These advances are full of potential, but they also come with their challenges. We need an AI-informed and capable workforce who are confident in the use and appraisal of AI and its outputs. To support them, we need strong guiding ethical principles to ensure that our use of AI promotes equity, wellbeing, and social justice. Open, transparent and replicable practices will help us to develop and maintain the highest standards of AI use by ensuring accountability and responsibility.

We are starting to develop our own Open Science and AI practices and policies. You can follow our progress here. For more information on applying and using AI in your context and practical suggestions about how to get your organisation AI ready, check out our Senior Researcher and Data Specialist Sean Manzi’s recent keynote at the 2024 Children and Young People Now Evaluation, Evidence and Impact Conference. 

This is just a window into some of the broader discussions we’re having as part of our research and evaluation approaches. We’ll continue to share real-life examples of place-based and systems change evaluation, as well as our critical reflections and learning about AI advances in research and evaluation. We want our work to support meaningful and ethical change that tackles ‘wicked’ problems in all their messiness and complexity.

If you are currently exploring what an evaluation of a complex or place-based systems change initiative might look like in your context or are interested in partnering with us in the future, feel free to get in touch.