|
|
|
A ‘balance sheet’ or ‘summary of findings table’ is a simple but powerful way to present the advantages and disadvantages of the different policy options considered.8,10 Presenting these tables together with brief texts that qualitatively summarise the key messages aids understanding of the potential impacts of the options.
The aim of a summary of findings is to help decision makers develop an accurate understanding of the important consequences of the options being compared. A summary of findings helps to achieve this in four key ways. Firstly, it condenses the most important information and thus enables efficient consideration. Secondly, it focuses attention on the most important outcomes. This increases the likelihood that decision makers will gain an accurate perception of what is known about the impacts of the options being considered and the important consequences. Thirdly, building a summary of findings is a helpful way to organise thoughts, structure evidence analysis, and focus debate. Fourthly, a summary of findings can help to develop more explicit judgements about what the most important consequences of options are, the underlying evidence for this, and subsequent judgements about the balance between the relative advantages and disadvantages of the options presented in a policy brief. This helps decision makers to form their own judgements about the trade-offs between desirable and undesirable consequences.
Although a there is no single optimal format for a summary of findings table, sometimes even for all the options within a single policy brief document, the template in Table 4.1 illustrates the key information that should be included. The roman numerals correspond to the key in Box 4.3.
Box 4.3 Key Information for Summary of Findings table |
i. A title indicating the comparison summarised in the table ii. The characteristics of the evidence, including the types of participants (patients or populations), types of settings (e.g. countries) where the studies were done, the intervention and what the intervention was compared to iii. The most important outcomes, including the intended benefits, possible harms and costs iv. The estimated impact of the intervention on each outcome (preferably provided quantitatively) v. The amount of information upon which the information is based, such as the number of participants or units (e.g. facilities), as well as the number of studies vi. The quality of the evidence for each outcome (based on the considerations summarised above and those found in the SURE Worksheet for preparing a summary of findings using the GRADE framework) |
Examples of
completed summaries of findings tables can be found in the
‘Additional resources’
section of this guide. Further examples can be found in the
SUPPORT summaries in the Libraries
section of these guides. Workshop
materials and a
PowerPoint presentation on summarising findings about the
likely impacts of policies are provided in the ‘Additional resources’ section of
this guide.
Health system interventions, like clinical interventions, can have
unintended harmful effects. It is important to consider these as
well as the desired effects of policy options when summarising the
potential impacts of different policy options. For example,
although paying for performance or results-based financing is
widely advocated, such approaches may have both desirable and
undesirable impacts. These could include: motivating
unintended behaviours, distortions (ignoring important tasks that
are not rewarded with incentives), gaming (improving or adjusting
reporting instead of improving performance), cherry picking
(selecting or avoiding patients depending on the ease or difficulty
of achieving performance targets), widening of the resource gap
between the rich and poor, and a greater dependence on financial
incentives.11 Further
guidance is offered in the SUPPORT Tool on
using research evidence in balancing the pros and cons of
policies.
The local context should also be considered when summarising the potential impacts of different policy options. Descriptions of what is already happening locally, how interventions may need to be tailored, and the applicability of the evidence should be included. A worksheet, Workshop materials and a presentation on finding and using local evidence are provided in the ‘Additional resources’ of this guide. A SUPPORT tool providing guidance on how to find and use evidence about local conditions is also provided.
There are two important limitations that should be considered when using a summary of findings to make decisions. Firstly, when there are complicated trade-offs between multiple outcomes, judgements by policymakers may require a high level of information processing. Secondly, the value judgements employed by policymakers as they weigh different outcomes could remain implicit. Underlying assumptions (including value judgements) can be made more explicit by including in the summary of findings the results of economic analyses (see Box 4.4) if they are available or possible to undertake.8, 9 In addition, economic analyses enable the use of sensitivity analyses to explore the effects of both uncertainties and varying assumptions on the results of an evaluation.
Box 4.4 Economic analyses |
|
Formal economic models, such as cost-effectiveness analysis and cost-utility analysis (see below), can help to inform judgements about the balance between the desirable and undesirable consequences of an option. Economic models can be valuable for complex decision making and for testing how sensitive a decision is to key estimates or assumptions. A model, though, is only as good as the data on which it is based. When estimates of benefits, harms, or resource use come from low-quality evidence, the results will necessarily be highly speculative. Moreover, published cost-effectiveness analyses are specific to a particular setting and this may differ in important ways from the setting of interest in the policy brief. Guides to using economic analyses (validity and results) are included in the ‘Additional resources’ section of this guide and some of the terms used in economic analyses are defined below. Terminology Cost-effectiveness analysis – An economic evaluation in which the costs and consequences of options are expressed as a cost per unit of health outcome (e.g. cost per death averted). Cost-utility analysis – An economic evaluation in which the costs and consequences of options are expressed as a cost per quality-adjusted life-year (QALY) or disability adjusted life-year (DALY). Cost-utility analyses are a type of cost-effectiveness analysis and are sometimes called cost-effectiveness analyses. Cost-benefit analysis – An economic evaluation in which both the costs and the consequences (including health outcomes) are expressed in monetary terms. Cost-minimisation analysis – An economic evaluation conducted in situations in which the consequences of the alternatives are the same and the only issue is their relative costs. Sensitivity analysis – A test of the stability of the conclusions of an evaluation over a range of probability estimates, value judgements, and assumptions. This may involve repeated analyses in which one or more of the parameters of interest are varied. |