Handling Complexity in Evidence from systematic reviews and meta-analyses of Public Health Interventions (CEPHI)

This proposal aims to develop methods for assessing and enhancing the generalisability of evidence from complex public health interventions. We will explore how this can be achieved through new approaches to analysing existing meta-analytic evidence, in combination with secondary data analysis and additional synthesis.

Background

Many public health challenges appear intractable, with substantial investments made into interventions that have little impact. Childhood obesity is one such example, where levels of childhood obesity have broadly stabilised, but with no apparent declines in prevalence despite substantial investment. The development of evidence-based strategies has been widely recommended.

Despite their high regard, the ability to utilise evidence from meta-analyses and systematic reviews is hampered by the lack of connection between the contexts in which interventions were conducted and the context in which the evidence is to be applied. In this proposal we seek to develop methods of exploring and enhancing the generalisability of meta-analysis using secondary data analysis and additional synthesis.

Aims and objectives

This proposal has three main objectives: (1) to identify – in collaboration with stakeholders – the study features that should be examined in considering the generalisability of evidence; (2) to explore the contribution of adjunct analyses in exploring and enhancing the generalisability of meta-analytic evidence; and (3) to produce a deployable approach to making review evidence more relevant to local usage.

Methods

The proposed methodological research uses childhood obesity as a case example and is spread over three work packages (WPs):

WP1: We will co-produce a systems-based logic model with stakeholders that helps to identify contextual features of interest and how they interact with a potential intervention, to guide later stages.

WP2: This work package explores the utility of two approaches in examining the generalisability of evidence, drawing on WP1. First we will assess how using existing observational data and employing statistical methods (namely reweighting of effect sizes and latent class analysis) in novel ways can help to create an overall measure of effect from meta-analysis that is more applicable to a defined population.

Next we will explore the utility of Qualitative Comparative Analysis (QCA) in examining the influence of context. Drawing on set-theory, QCA will be used to examine configurations of different contextual features that align with more successful interventions.

WP3: Given that the primary motivation of this project is that end-users are not utilising review evidence because of its disconnect with their particular local circumstances, we need to explore the utility of the proposed enhancements to meta-analytic evidence. We will conduct two workshops and further meetings to evaluate the utility of the approaches trialled above and develop guidance around their usage.

Timelines for delivery

This work takes place between April-December 2021, with the first outputs planned for an interactive webinar in November. Anticipated impact and dissemination

The impact of this work will (i) identify and evaluate new methods for exploring the generalisability of meta-analytic evidence; and (ii) allow systematic reviewers and/or knowledge translators to replicate the approaches taken through issuing guidance and case studies.