Are there important uncertainties that should be addressed prior to making a decision?

While some options may be promising, they may require large investments or may be associated with important risks of adverse effects. Therefore it may be prudent to undertake an evaluation before fully implementing a policy. Substantial caution is required, for example, if interventions require large investments of scarce resources that cannot be retrieved later.² When there is important uncertainty about the benefits of interventions, a rigorous evaluation (such as a pilot study) can potentially prevent resources being wasted that could otherwise have been put to better use elsewhere. This may appear to introduce unnecessary delays, but as Julio Frenk, the former Minister of Health of Mexico notes: “Both politically, in terms of being accountable to those who fund the system, and also ethically, in terms of making sure that you make the best use possible of available resources, evaluation is absolutely critical.”³

Consideration should be given to undertaking an evaluation prior to fully implementing a policy whenever judgements about the effects of policies are based on theories, surrogate outcomes, limited observational studies, or inadequate randomised impact evaluations. The advantages and disadvantages of undertaking an evaluation prior to the full implementation of a policy are summarised in Table 6.1. In general, whenever there is a substantial risk of the benefits of a policy not outweighing the harms and costs, or a potential need to make important modifications to a policy, the advantages of an evaluation are likely to outweigh the disadvantages. However, this depends on whether an evaluation can be completed within an acceptable timeframe, and whether the evaluation is feasible and affordable.

In a policy brief there are several ways to outline options that have important uncertainties which should be addressed prior to full implementation. Depending on how compelling the arguments are for undertaking an evaluation, these options can include:

• Flagging the uncertainties and ensuring that consideration is given to undertaking an evaluation before the full implementation
• Describing the full implementation and undertaking an evaluation as two separate options (see the example link below)
• Describing the option with an evaluation being undertaken first as part of the option (in other words, the option should not be fully implemented without first undertaking an evaluation)

In an example of a policy brief on integrating mental health into primary care in Zambia, two options were offered: one was full implementation and the other was incremental implementation with an initial evaluation. Further details about this study can be found in the ‘Additional Resources’ section of this guide.

This page was last updated November 2011.