2 Evaluating Trade-Offs
Learn how to compare options by making their benefits, harms, costs, values, and uncertainties explicit.
Set up a fair comparison
A sound comparison begins by defining the decision, the people affected, and the available options. Set a realistic —the likely outcome if no new option is chosen—and compare each option with it over the same time period and population.
Choose outcomes that matter to the decision. Distinguish observed evidence from assumptions and forecasts, and report how certain each estimate is. Certainty can differ across outcomes: evidence about benefits may be stronger than evidence about harms or costs. A precise-looking number is not necessarily well supported.
Where possible, report the expected between an option and the , rather than only a relative percentage. For example, about fewer cases per people may be easier to interpret than a reduction when the starting rate is not given.
Compare benefits, harms, and costs
List important consequences separately before deciding how to balance them. This makes it possible to see what each option may achieve, what it may cost, and who may benefit or be harmed.
Benefits: Identify which outcomes improve, by how much, for whom, and when. Consider both the size and importance of each improvement.
Harms: Consider possible adverse effects, burdens, or risks. Assess their severity, likelihood, duration, and distribution, not only their average.
Costs and resources: Include money, staff time, equipment, infrastructure, and the effort needed to deliver an option. A resource used for one option cannot be used elsewhere, creating an .
Keep unlike outcomes distinct unless there is a defensible way to combine them. A cheaper option is not automatically better if it produces much less benefit or shifts burdens onto people with fewer alternatives. Benefits to many people do not necessarily resolve a serious harm to a smaller group.
Make values explicit
People may value the same outcomes differently. Preferences can prioritize affordability, speed, privacy, convenience, fairness, or protection for those at greatest risk. Consider whose preferences are represented and whether differences between people’s preferences could change the decision.
Consider equity, acceptability, and feasibility alongside benefits, harms, and resources. These factors can influence both which option is chosen and whether it works in practice.
can help structure a comparison when important criteria cannot all be expressed in money. It involves defining criteria and rating options against them; if appropriate, weights can express the criteria’s relative importance. Weights are value judgments, not facts. State who set them, explain them, and test whether different reasonable weights change the ranking. A composite score should not disguise a serious harm or disagreement.
Make uncertainty visible
Distinguish uncertainty about what will happen from disagreement about what matters most. For uncertain estimates, show plausible ranges or scenarios and identify the assumptions that drive them. Test whether the preferred option changes when uncertain benefits, harms, costs, or criterion weights are varied.
If small changes reverse the result, the decision is sensitive and should be presented cautiously. If one option remains preferable across plausible assumptions, the decision is more robust.
When uncertainty matters, a decision-maker may favor a reversible choice, a limited trial, or gathering more information before committing. These approaches are especially relevant when waiting is feasible and the consequences of a mistaken choice are substantial. Further information is valuable insofar as it could realistically change the decision.
Apply the comparison to transport options
Suppose a city compares adding fixed-route buses with offering on-demand vans. Buses may carry more passengers for their annual budget, while vans may provide more convenient access for people who cannot easily reach a bus stop.
A fair comparison would report expected trips, costs, travel times, and effects on access for relevant groups over the same period. It would also identify uncertainties in demand and operating costs. The city would need to make explicit how it weighs total ridership against accessibility and distributional fairness; the evidence alone cannot determine those priorities.
Bring the trade-offs together
There is often no single option that is best on every dimension. Compare options against a clear , examine benefits, harms, and full resource costs separately, and consider how consequences are distributed. Make values and uncertainty explicit, use structured scoring only as a transparent aid, and test whether the decision holds under reasonable alternative assumptions.