Assessing public projects’ value for money: An empirical study of the usefulness of cost–benefit analyses in decision-making

Article highlight

The paper presents a detailed look at the use of cost–benefit analyses in determining value for money in publicly-funded projects in Norway.

Keywords

• Project value
• Project appraisal and evaluation
• Cost–benefit analysis
• Business case
• Quality assurance

What does the paper cover?

Value for money, as measured by cost–benefit analyses, is a crucial part of the business case for major public investment projects. The paper presents an empirical study of the practice of cost–benefit analyses in Norway, a country that has made considerable efforts to promote quality and accountability in cost–benefit analyses of public projects. In particular the country has introduced mandatory external quality assurance reviews of the business case for the largest public projects. The paper offers a set of practical recommendations to increase the usefulness of cost–benefit analyses further.

Methodology

The author used a qualitative method, looking at the practice of cost–benefit analysis in 58 projects within the Norwegian quality assurance scheme. The research included a document review, interviews, and a review of the decisions made by the Norwegian government (the Cabinet).

Research findings

Cost–benefit analyses are comprehensive and partly standardised
Most of the cost–benefit analyses were relatively comprehensive, and appraisals of similar types of projects generally included the same impact categories.

Inconsistent handling of non-monetised impacts
Non-financial impacts were often essential in the cost–benefit analyses. However, their interpretation was sometimes unclear and arbitrary.

Uncertainty thoroughly assessed for capital cost, but to a lesser extent for other impacts
The research included an assessment of major uncertainties relating to costs and benefits, and how these were assessed and presented. Generally, the studied cost–benefit analyses were more concerned about risks to the capital cost than risks to benefits and other long-term impacts.

Other considerations were not clearly distinguished from value for money
Overall, nearly half of cost-benefit analyses included other decision criteria (such as goal achievement, sustainability, and distributional impacts) but these considerations were not clearly distinguished from the cost–benefit analysis. Generally, the discussion of distributional impacts was not comprehensive enough. Immediate effects were discussed more often than long-term distributional effects.

Appraisal optimism was avoided for net present value estimation, but may have influenced the cost–benefit analysis in other ways
Although not always openly stated, there was commonly a preferred project alternative from the ministry/agency’s perspective. This raises the question of whether the conceptual appraisal documents were biased in favour of a preferred alternative. The paper finds that external quality assurance has been essential in reducing the risk of appraisal optimism for the monetized costs and benefits. However, the CBAs may still be biased in terms of the non-monetized impacts, and there may be a risk of excluding or systematically downgrading the simplest and less costly alternatives.

Transparency and communication acceptable, but could be improved
Transparency and clear communication are crucial to ensuring the usefulness of a cost–benefit analysis. Overall, the researcher judged the level of transparency as acceptable in about 80 per cent of cases, meaning that they were documented in sufficient detail, either in the main report or in an appendix. However, many reports could have been improved.

Decision-makers found cost–benefit analyses more useful when approved by an independent party
The ultimate test of whether decision-makers find cost–benefit analyses useful is the extent to which they follow the recommendations in the reports. In about 80 per cent of cases, the Cabinet chose to go ahead with either one conceptual alternative or, in a few cases, several conceptual alternatives to be developed further into a major construction project.

Decision-makers find that the external quality assurance is useful. One interviewee said, ‘The existence of two CBAs that come to the same conclusion is a strong indicator of quality.’

Conclusions

• The use of external quality assurance normally reduces the risk of appraisal optimism.
• The risk of inconsistent, incomplete and/or inaccurate estimates should be limited, given the time and resources spent on the analyses and the considerable expertise involved.
• Cost–benefit analyses are heeded by decision-makers. The Cabinet almost always approved a project proposal if it was recommended as good value for money by the ministry/agency, and endorsed by the quality assurer – but not otherwise.
• Impacts other than financial ones need a clearer definition and more systematic treatment, distinguished from considerations beyond the project’s value for money.

Significance of the research

1. The research has provided the basis for a set of practical recommendations to increase the usefulness of cost–benefit analyses in both the public and the private sector. These are summarised below.
2. A number of perspectives beyond value for money may be relevant to decision-makers.
3. An important purpose of a cost–benefit analysis is to assess a number of alternative solutions to the problem at hand.
4. Completeness and consistency are important quality criteria.
5. Possible errors and uncertainties need to be identified and presented as part of the cost–benefit analysis.
6. The non-monetised impacts are as relevant as the monetised ones.
7. Measures should be taken to prevent optimism bias on the part of project promoters.
8. Analyst competence and qualifications are key.
9. Clear communication, such as the use of simple language and a readily available summary, are important for transparency in reports, and relevant to decision-makers who are not experts in cost–benefit analysis.