Map Map


What is known about the effects of relevant implementation strategies?

The process of finding and appraising evidence of the effects of strategies for implementing policy options is similar to the process described earlier for finding and appraising evidence of the impacts of the policy options. This entails finding, selecting, and assessing the reliability of systematic reviews. Based on the evidence from these, judgements are then made about the effects of implementation strategies and about how much confidence to place in those estimates.

A systematic review is the ideal starting point for finding out what is known about the effects of implementation strategies. Many systematic reviews of strategies for changing the behaviours of recipients and providers of care are now available; however, there are fewer that focus on strategies for addressing health system, social, and political constraints. Health Systems Evidence (www.healthsystemsevidence.org) is a good place to begin searching for systematic reviews that address the effects of implementation strategies. Other sources that can be searched for systematic reviews of the impacts of health system arrangements and implementation strategies include CADTH Rx for Change (www.cadth.ca/en/resources/rx-for-change), The Cochrane Library, and PubMed.  SUPPORT has prepared concise summaries of the best available evidence of the effects of health systems interventions, including implementation strategies, for low and middle-income countries. These summaries are provided in the Libraries section of these guides. 

Strategies for finding systematic reviews of implementation strategies are available in the ‘Additional resources’ section of this guide.

Reviews of implementation strategies commonly address overlapping questions from different perspectives. For example, reviews of strategies for improving professional practice may address the effects of a type of intervention across different practices (e.g. educational meetings), or the effects of different types of interventions to address a single problem or condition (e.g. diabetes), or a type of behaviour or practice across different conditions (e.g. prescribing). Sometimes reviews are restricted to a specific setting (e.g. primary care in low- and middle-income countries), or to a specific type of health worker (e.g. traditional birth attendants). There may be good reasons for undertaking reviews from these different perspectives, but reviews with an overly narrow focus may be misleading. Such reviews may, for example, draw spurious conclusions about the effects of a type of intervention. If there is a paucity of studies of a particular intervention that meets the selection criteria, a review of the effects of the intervention across different conditions and settings may provide a better estimate of its potential effects than a review of its effects in very specific circumstances. Therefore, when searching for and selecting reviews of the effects of implementation strategies, it is prudent to search for broadly-focused reviews and to be cautious when interpreting the results of more narrowly-focused reviews.

Once a relevant systematic review is found, a decision is then needed about how much confidence to place in it. A SURE checklist for making judgements about how much confidence to place in a systematic review is included in the ‘Additional resources’ section of SURE Guide 4, which can also guide judgements about the reliability of reviews of implementation strategies.

If a systematic review without important limitations cannot be found, searching for individual studies may be necessary, either to supplement the information in a review or in place of a systematic review. If this is done, attention should be paid to the same processes used when selecting studies for inclusion in a systematic review. In other words, as far as possible, systematic and transparent (explicit) methods should be used to find, select, and critically appraise studies, as well as to synthesise the results of relevant studies. Ideally, the methods used to do this should be described in an appendix to the policy brief.

Judgements about the potential effects of implementation strategies are similar to the judgements that must be made about the potential impacts of the policy options being implemented. The GRADE framework provides the structured and transparent approach needed to make these judgements, including those related to how much confidence to place in estimates of the likely effects of implementation strategies. The SURE worksheet for preparing a summary of findings using GRADE, is appended to SURE Guide 4, and can be used to guide judgements about the estimates of effect of implementation strategies.



This page was last updated November 2011.