Timedora

Synthesizing and Communicating Evaluation Findings

A practical guide to combining evaluation findings, comparing options fairly, making evidence-based recommendations, and communicating uncertainty and limitations.

From Evidence to a Fair Comparison

Start with the decision

An evaluation is useful when it brings evidence together to inform a specific choice, rather than presenting a sequence of separate studies or methods. Begin by identifying the decision and the questions that matter to it—for example, effectiveness, implementation, cost, or effects on different groups. Organize evidence around those questions, and combine findings from different methods when they speak to the same issue.

When findings converge, they can strengthen confidence. When they conflict, do not assume that one study must be wrong. Differences in the people studied, setting, implementation, outcome measures, or study quality may help explain the disagreement; the intervention may also work differently in different circumstances. Before drawing a conclusion, ask whether the evidence answers the decision question, whether the comparison is credible, whether the outcomes matter, and how uncertain the estimates are. A precise estimate from a weak or poorly matched study may be less useful than qualitative evidence that explains why an intervention succeeded or failed. Make clear when conclusions rely on few studies, indirect evidence, or assumptions.

A hypothetical tutoring comparison
OptionReading gainCost per studentMain trade-off
Continue current provisionSmallLowLeast disruption, but little expected improvement
Add online tutoringModerateMediumFlexible and easier to expand, but requires reliable internet and device access
Provide in-person tutoring at schoolsLargerHighMore direct support, but requires more staff and scheduling
Illustrative example. Hypothetical tutoring comparison from the evaluation example. The options are described qualitatively; the table does not supply measured estimates.

Compare options consistently

Compare each viable option with the same benchmark, such as current practice or a do-minimum alternative. Use consistent outcomes, time horizons, and cost perspectives so the comparison is meaningful. Consider expected benefits, costs, feasibility, risks, public cost, and effects that cannot reasonably be expressed in money.

The tutoring comparison shows why a simple ranking is not enough: in-person tutoring has the largest estimated reading gain, while online tutoring costs less per student. Which is preferable depends on priorities and implementation conditions, such as whether the aim is the largest gain, broader reach, or lower cost. A benefit–cost ratio or other may help, but it cannot by itself show who receives benefits, which harms may be unmeasured, or how uncertain the estimate is. Report the estimates and uncertainty, not just an option ranking. If options were not tested under comparable conditions, apparent differences may not reflect true differences in impact.

Distribution, Trade-offs, and Recommendations

Look beyond the average

An overall average can conceal who benefits, who pays, who faces risks, or who may be excluded. Examine groups and places relevant to the decision—for example, by income, age, disability, geography, or access to services—and check whether there is enough evidence to compare their outcomes reliably. Keep the analysis proportionate to the likely scale of effects, but do not let important differences disappear inside an average.

In the tutoring example, online provision may be less accessible to students without dependable internet or a quiet place to study. In-person provision may serve those students better, but reach fewer people if staffing capacity is limited. Possible responses include lending devices, offering a school-based option, or targeting additional support. These are ways to address potential access differences, not proof that one approach will work in every setting. If subgroup estimates are uncertain or based on small samples, describe them as suggestive rather than conclusive.

Make the judgment explicit

A reasoned recommendation connects the decision and its priority objectives to comparative evidence about what each option achieves, at what cost, and with what risks. Then explain what is gained or sacrificed, how effects are shared, and why the preferred balance fits the objectives. Evidence informs the choice, but does not determine it: choosing greater average effectiveness over lower cost, for example, also reflects values placed on effectiveness, affordability, fairness, and risk.

If no option performs best on every criterion, say so. Explain why the selected balance is preferred and identify any safeguards, adaptations, or further tests needed. A single summary measure or option ranking cannot substitute for this reasoning.

Communicating Conclusions and Limits

Explain what the evidence can establish

State what the evaluation can and cannot establish, and explain how relevant limitations could affect the recommendation. Short follow-up may miss whether an effect lasts; incomplete or inconsistent data may distort estimates; uncertain causal attribution weakens confidence that the intervention caused an observed change; and small subgroup samples can make group comparisons imprecise. Differences between study and intended settings make it less certain that results will transfer, while unmeasured effects may leave important benefits or harms out of the comparison.

Statistical significance alone is not a complete account of uncertainty. When estimates are available, explain a confidence interval or plausible range in accessible terms. A confidence interval gives a range of estimates compatible with the data and the method used; a wider interval generally indicates less precision. Explain what outcome the range concerns and whether the uncertainty could change the comparison or recommendation.

For example, a conclusion might favor in-person tutoring for reading gains during the first school year while noting that the evaluation covered only three schools and did not measure longer-term outcomes. The limited number of schools makes it uncertain whether the result generalizes to other districts, and the missing long-term evidence means the evaluation cannot establish whether the advantage will persist. These caveats qualify the conclusion; they should not obscure what the evidence indicates.

Adapt the communication, not the conclusion

Keep the underlying conclusions consistent while adjusting format and detail for the audience. Decision-makers may need a concise recommendation and comparison table; practitioners may need implementation details; the public may need a plain-language summary. A brief report can lead with the decision, main evidence, distributional effects, trade-offs, and limitations, with methods and supporting analyses available separately. Plan outputs for their users: a final report alone may not be the most useful way to communicate findings.

  • : Bringing together evidence from different methods that addresses the same decision question; converging findings can strengthen confidence.

  • : Examining how benefits, costs, risks, and access are shared across groups and places.