Assessing Evidence and Uncertainty
A practical guide to asking focused evidence questions, assessing sources, synthesizing findings, and communicating what is known and uncertain.
Frame the Question and Search Transparently
Start with a question that can guide the search
A useful identifies four things: who or where the question concerns, what action or condition is being considered, what it is compared with, and which outcomes matter. For example: For students who are behind in reading, does small-group tutoring, compared with usual instruction, improve reading ability over one school year? This specifies a population, an action, a comparison, an outcome, and a time frame.
Decide in advance which outcomes matter, including possible trade-offs. The tutoring example could involve both reading scores and the staff time required. Looking only at scores could miss an important consequence of the intervention. A precise question also helps prevent evidence about a different group, setting, or outcome from being treated as if it directly answered your question.
Search systematically and show your work
Search more than one suitable source. Depending on the question, evidence might include research studies, systematic reviews, official statistics, administrative records, program evaluations, and relevant expertise from people affected by the issue. Research findings can be considered alongside local context and experiences of practitioners or communities.
Keep a brief search record: the question and search terms; where and when you searched; what kinds of evidence you included or excluded; and important gaps in what you located. This makes the process easier for others to understand and shows where the evidence base may be incomplete.
A uses explicit methods to find, assess, and combine relevant studies. A rapid evidence assessment can be useful when time is limited, but it generally narrows the scope or depth of the search. Describe those limits: a rapid assessment should not be presented as exhaustive if it was not. The suitable sources and search approach depend on the question.
Judge Evidence and Separate What Is Known
Judge both relevance and quality
Do not judge evidence by its headline, the number of sources, or apparent certainty alone. Ask two related questions: Does this evidence address my question? and Do its methods support the conclusion being drawn?
Check the evidence in several ways:
Design and bias: Could participant selection, measurement, or follow-up distort the result? Could other factors explain an observed difference?
Measurement: Were outcomes measured consistently and appropriately?
Directness: Does the evidence concern the population, setting, action, comparison, and outcomes you care about?
Consistency and precision: Do findings align across studies? How wide is the range of plausible effects?
Completeness: Might studies or results be missing because they were not published or reported?
A study can be useful and still have limitations. Risk of bias can differ by outcome. And several studies do not automatically make a conclusion trustworthy if their methods are weak or results are selectively reported.
Keep findings, assumptions, uncertainty, and gaps distinct
These categories describe different things:
Finding: An observation supported by a specified source or dataset. In the tutoring example, students offered tutoring had higher average reading scores at year’s end.
: A proposition used in analysis or planning but not established by the evidence. For example, the program can be delivered with the same quality in every school.
Uncertainty: A limit on how confidently a finding or estimate can be stated. For example, the estimated improvement is positive, but the range of plausible effects is wide.
Missing information: Evidence needed to answer the question that has not been collected, found, or reported. For example, the evaluation did not report whether results differed for students who began farthest behind.
The distinction matters: an is not a result, and missing information is not evidence that an outcome did not occur. Make assumptions visible, especially when they are both uncertain and consequential. Such assumptions may be worth testing before relying on them in a decision.
Synthesize Findings Without
Bring findings together by what matters
Synthesis means using the evidence together to answer the question—not just listing studies. Organize findings by the outcomes, groups, or settings that matter. Explain where studies agree and where they differ, and consider whether differences in methods or context could help account for disagreement.
For example, when considering whether small-group tutoring improves reading, keep the reading outcome central while also attending to the relevant student groups and settings. A result from a different setting may still be informative, but it may not directly predict what will happen locally.
Weigh evidence; do not count votes
Avoid deciding that an approach works simply because more studies report positive than negative results. This kind of treats studies as if they contributed equally. But study size, quality, precision, and relevance matter: a small, weak study need not count as much as a strong study that closely matches the question.
When quantitative combination is appropriate, a can estimate an overall effect. It does not make unlike studies equivalent or remove their limitations. Combining results is not a substitute for judging the studies and their fit to the question.
Report important outcomes even when the results are inconclusive or not statistically significant. Leaving them out can make the evidence appear more decisive or complete than it is.
Make the conclusion and its limits visible
A compact summary can put the question, best available finding, confidence and limitations, and assumptions or gaps side by side. For the tutoring question, a source-grounded summary is: studies reviewed suggest reading may improve; findings vary and effects in local schools may differ; local implementation quality and subgroup outcomes are not yet known.
This format separates the current finding from its limits and from what remains unknown. If a gap concerns a consequential , further data collection or a small-scale test may be more useful than treating the evidence as settled.
Communicate Effects and Uncertainty
Describe the estimate, not just its label
When reporting an estimated effect, state its direction and size, as well as its uncertainty. A statement that an estimate is positive tells the direction, but not how large it is or how precisely it has been estimated. Avoid reducing the result to a simple “significant” or “not significant” label.
A can be explained in plain language as a range of values compatible with the data and statistical model. A wide range signals that the estimate leaves substantial uncertainty about the effect’s size. The interval does not capture every possible limitation: bias, weak measurement, or a poor fit between the studied setting and the setting under consideration can add uncertainty too.
Sensitivity checks test whether conclusions change under reasonable alternative analytical choices. If the conclusion changes, that indicates it depends on those choices; if it remains similar, that can indicate greater robustness. Such checks do not remove other limitations in the evidence.
Do not confuse “not established” with “absent”
means the available evidence has not established an effect. One reason could be an imprecise estimate. That does not prove the effect is absent.
Evidence of no meaningful effect is a stronger conclusion. It is better supported when the plausible range is narrow enough to rule out effects large enough to matter. Thus, an inconclusive result and a well-supported finding of little or no meaningful effect are not interchangeable. When communicating either conclusion, state what the evidence can support and what uncertainty remains.
: The extent to which evidence directly concerns the population or setting, action, comparison, and outcomes in the question.