Go back

Cost-benefit analysis can inform choices on science infrastructure

Estimating the value of spillovers and the public’s appetite for discovery can give society a voice in capital spending decisions, say Massimo Florio and Chiara Pancotti.

European research infrastructures can cost huge amounts of money. For example, the Large Hadron Collider at the European particle physics laboratory Cern is expected to cost €13.5 billion to build and operate over its lifetime, excluding scientists’ salaries. Science’s internal quality-control mechanisms usually determine how such infrastructures are assessed and selected. Typically, a peer-review process will appraise a project’s scientific merit, sometimes complemented by a business case.

This approach is usually efficient and fair, as far as it goes, but it leaves out wider society and says little about the overall socioeconomic impact of these large and costly projects. In the UK, the National Audit Office recently criticised the government for the poor standard of evidence used in allocating funding for science facilities. In Sweden, businesses have expressed concern about the cost of the European Spallation Source under construction in Lund. With a growing number of capital-intensive research infrastructures seeking public support, we need ways to examine whether this spending is worth it.

For EU-funded projects in other areas, the procedure is more structured. The European Commission requires a cost-benefit analysis for all infrastructure projects costing more than €50 million that seek co-funding from European Structural and Investment Funds. By systematically comparing costs with benefits, this gives an estimate of the net welfare change a project will bring. This can be expressed using simple numbers, usually as a figure that economists call net present value. This allows competing or alternative projects to be compared and ranked.

Research and innovation funding has seen little effort to use cost-benefit analysis systematically. The uncertainty of discovery and the intangible value of much scientific knowledge have been seen as formidable hurdles. The result is that there is little understanding of how to assess the net impact of research infrastructures on society.

In a bid to remedy this, nearly three years ago the European Investment Bank Institute opened a competitive call to develop and test a conceptual framework for research, development and innovation infrastructures. Our team, made up of economists and physicists from the University of Milan and the Centre for Industrial Studies, also based in Milan, won the bid and we have recently completed the project. One early result has been a new chapter on research in the latest edition of the Commission’s Guide to Cost-Benefit Analysis of Investment Projects for Cohesion Policy.

So far, we have tested our model on two facilities: the Large Hadron Collider at Cern and the National Centre for Oncological Treatment (CNAO) in Pavia. These are both particle accelerators, but with very different remits. The LHC is a global facility devoted to basic science, whereas the CNAO caters mainly to Italian clientele, for cancer treatment and biomedical research.

For the LHC, our analyses suggest that, between the start of its construction in 1993 and its expected decommissioning in 2025, the accelerator will yield an expected net benefit of nearly €3bn. For the CNAO, which has approximate total costs of €512 million, we calculated an expected €1.6bn net benefit over its lifespan.

As much as possible, we sought to use and adapt the basic principles of cost-benefit analysis. In line with these fundamentals, our model makes use of: shadow prices, which put a monetary value on social costs and benefits that are not generally captured by market prices; a counterfactual scenario, comparing the consequences of going ahead with a project or not; and discounting, which converts past and future values into their present equivalents.

However, our model includes some novel features to address the specific challenges that research projects pose, such as their uncertainty and intangible nature.

Among these novelties is a probabilistic approach: whenever possible, we express all the critical values as a distribution of probabilities rather than as a single, unrealistically precise figure. This is a way of dealing with uncertainty and the risk of optimism bias. In effect, this approach gives an estimate of the risk of a particular project, as well as its expected returns.

Our model quantifies uncertainty where it can be measured but it leaves aside what is intrinsically unmeasurable. One important difference between investing in a laboratory and, say, a bridge is that experiments and tests could generate discoveries and innovations that cannot be anticipated when the funding decision is made. Additional sets of services and possible uses could arise during a facility’s lifespan.

These are some of the inherently unpredictable effects of research. Forecasting their probability or magnitude is usually impossible. The impact of the World Wide Web, invented at Cern, is a classic example.

Hence, we took a conservative approach, omitting such unpredictable effects from our analysis, assuming that they would have a positive value. In our analysis of the LHC, for example, we have not tried to put a value on the discovery of the Higgs boson.

Our second novelty was to assign the core beneficiaries of research infrastructures to standard economic categories: firms, consumers, employees, taxpayers. One economic peculiarity of research infrastructures is that the producers and the beneficiaries of their services are often one and the same. For instance, scientists both create knowledge and use it in their subsequent work.

Failing to account for this duality generates confusion in economic analyses aiming to determine and separate the producers and consumers of research, development and innovation infrastructures. This leads to errors in valuation, the most striking being the inclusion of scientists’ salaries among a project’s benefits rather than its costs.

Linked to this, the third novelty is the breakdown of benefits into two broad classes: use and non-use. The first of these covers those with direct or indirect links to a facility. This might include scientists, students, firms benefitting from technological spillovers, consumers gaining innovative services and products, public visitors and those involved in outreach activities.

Calculating the benefits of use depends on the scientific specifics of each project and the different groups that stand to benefit. Identifying beneficiaries and related benefits is the easy part. Much trickier is putting a value on the scale of these benefits over the project’s lifespan.

Some benefits are common to many kinds of infrastructure and are well understood. These include human capital formation—for research, one example includes any salary premium going to a facility’s PhD students—recreational benefits for visitors, and health benefits for patients.

Other benefits are more unusual and harder to calculate. These include the benefits that accrue to the firms working with and on a research infrastructure, such as the development of new or improved products, patents, knowledge spillovers and hands-on experience, known to economists as learning by doing. Calculating the value of these benefits requires financial records and supply-chain data that are not usually in the public domain. Cern, however, gave our team access to procurement information that allowed us to study the long-term effect of learning by doing on its suppliers’ profits. The data confirmed the positive effect of technological learning on the profits of Cern’s hi-tech contractors.

We also estimated the socioeconomic value of outputs such as scientific articles. This is the value of producing new information combined with the value attributable to the influence of such outputs on other scientists. We valued the former by estimating the marginal production cost of scientific papers, using the average scientist’s hourly wage as a proxy. The latter is reflected in the number of people citing a work, and can be valued by estimating the opportunity cost of the time needed to download, understand and cite a piece of work.

The second category, non-use benefits, is a monetary expression of the value that society places on potential discoveries, regardless of a facility’s present or future use. Our project’s final major novelty lies in our attempt to estimate the non-use value of discoveries; that is, society’s willingness to pay for the possibility that something may be found even if its use cannot be predicted.

Here, we adapted techniques used to value environmental and cultural goods. Many people express a willingness to pay to preserve the Grand Canyon or the Taj Mahal, for example, even if they do not plan to visit them. Similarly, we surveyed the public to estimate willingness to pay for the potential discoveries from the LHC.

For this facility, we estimated that a third of the total benefits come from technological spillovers, a third from human-capital formation and the remainder from knowledge output, cultural effects and non-use value. For the CNAO, which specialises in cancer therapy, the major benefits are from improvements to patient health.

Even though this conceptual framework is rooted in the tradition of cost-benefit analysis, its application is in its infancy. We would encourage further studies to enlarge the knowledge base for applying cost-benefit analysis to research infrastructures and, in turn, improving and expanding these techniques. It would be particularly valuable to fine-tune the model so that it could be applied to infrastructures beyond particle physics facilities.

Following our cost-benefit analysis of the LHC, Cern has commissioned our team to assess the socioeconomic impact of its next-generation Future Circular Collider, due to begin operation in 2040. The European Space Agency and European Strategic Forum on Research Infrastructures have also shown an interest in our methods.

The history of fields such as environmental and cultural economics suggests that it takes several years of real-world tests before practitioners and policymakers accept novel ideas. The interest in using cost-benefit analysis for research infrastructures is an acknowledgement that the way that these decisions are made needs to change. But this approach should be seen as a complement to, not a substitute for, the status quo. Evaluation of scientific merit, financial issues and strategic considerations will remain a crucial part of the process.

Massimo Florio is professor of public economics at the University of Milan. Chiara Pancotti is an economist at Centre for Industrial Studies in Milan. See www.eiburs.unimi.it.

More to say? Email comment@ResearchResearch.com

This article also appeared in Research Europe