Introduction
Project appraisal and project selection are key functions in the planning and allocation stages of public investment. Effective appraisal supports decision making for optimization of project design and impact and is critical in selecting projects that yield the highest social and economic returns. Project appraisal and selection have vital roles for infrastructure governance because they serve a gatekeeping function, ensuring in principle that only socially and economically viable projects reach the implementation stage. Moreover, good appraisal and selection methods increase the probability of maximizing net benefits to society, including by scrutinizing investment and operational costs to avoid unfinished projects or inefficient operations (Rajaram and others 2014).
Good project appraisal and selection requires institutions capable of designing appraisal methodologies and having procedures in place for sound project selection. Inaccurate and unrealistic appraisal may lead to inefficiency and wastage of resources through cost overruns during implementation, or even to incomplete projects. Furthermore, poor project selection can result in overprogramming of projects or wasteful white elephant projects with limited social and economic value (Rajaram and others 2010). Examples include grandiose presidential palaces, vast university campuses, or unnecessarily large airports (Mauro 1997).
The IMF’s Public Investment Management Assessment (PIMA) framework assesses whether countries have a project appraisal system that ensures major project proposals are subject to rigorous appraisal using standard methodology and taking account of potential risks, systematic vetting processes for project selection based on published standard criteria, and inclusion in a pipeline of approved projects (see Figure 13.1; IMF 2018). Often, this is not the case: project appraisal and selection generally score quite poorly in institutional design (de jure) and effectiveness (de facto) in comparison with the other public investment management practices covered in PIMA, particularly in low-income developing countries and emerging market economies. Given the critical importance of project appraisal and selection in infrastructure governance, this is an issue of great concern.
Average PIMA Scores for Project Appraisal and Selection
Source: World Bank and IMF staff calculations using the PIMA data set.Note: PIMA = Public Investment Management Assessment. Dispersion of PIMA scores for project appraisal and selection effectiveness by income level. Values displayed for the minimum, average, and maximum scores by income level; median displayed with an ×.
- Download Figure
Download figure as PowerPoint slide
Average PIMA Scores for Project Appraisal and Selection
Source: World Bank and IMF staff calculations using the PIMA data set.Note: PIMA = Public Investment Management Assessment. Dispersion of PIMA scores for project appraisal and selection effectiveness by income level. Values displayed for the minimum, average, and maximum scores by income level; median displayed with an ×.
- Download Figure
Download figure as PowerPoint slide
Average PIMA Scores for Project Appraisal and Selection
Source: World Bank and IMF staff calculations using the PIMA data set.Note: PIMA = Public Investment Management Assessment. Dispersion of PIMA scores for project appraisal and selection effectiveness by income level. Values displayed for the minimum, average, and maximum scores by income level; median displayed with an ×.
- Download Figure
Download figure as PowerPoint slide
This chapter assesses good practices in project appraisal and selection. It discusses the defining characteristics of an effective project appraisal function, including types of evaluation, appraisal methodologies, and safeguards against undue political interference. Then it examines how to link project appraisal and project selection to the budget cycle. The chapter finds that a clear, well-supported appraisal methodology and published project selection criteria with well-defined processes for project selection are critical for good infrastructure governance. Undue political influence is an issue in many countries and should be mitigated through rigorous analysis, scrutiny by a central ministry using clear and transparent procedures, and an independent review of projects before they are included in the budget. In low-capacity countries, outsourcing of project appraisal could be considered but should be balanced with the need for in-house capacity building and the development of practical know-how.
Characteristics of the Project Appraisal Functiont
Types of Evaluation and Their Application
Four main types of evaluations are used in project appraisal (Box 13.1). In general, all countries rely to a great extent on cost-benefit analysis, although they also complement it in specific cases with other methodologies. In the United Kingdom, cost-benefit analysis is the default, whereas in Korea, which uses multicriteria analysis, cost-benefit analysis is the most heavily weighted component of the analysis. In Ireland, the methodological choice depends on the type, scale, and complexity of the project, with cost-benefit analysis used wherever possible. It is worth noting that in some countries, such as the United Kingdom, cost-benefit analysis is being used more today than in the past because there have been concerns about inefficiencies resulting from the use of methods in which economic efficiency has less weight.1
Main Types of Evaluation Used for Project Appraisal
Cost-Benefit Analysis
This technique is used to compare the total costs of a project with its total benefits. It provides the net cost or benefit associated with a given project. Alternatives are appraised and compared to select the best approach, the one that yields the most benefits relative to the costs (Kaplan 2014a). The intellectual and conceptual underpinnings of cost-benefit analysis are robust as it is based on principles of applied welfare economics, which provide a clear and rigorous framework for assessing the “social” (or economic) value of projects. However, it requires capacity in advanced economics, and some variables (such as more intangible benefits or costs) can be difficult to estimate.
Cost-Effectiveness Analysis
This is an alternative to cost-benefit analysis that compares the relative costs of two or more courses of action with their related outcomes. Cost-effectiveness analysis is more commonly used when it is not possible to carry out cost-benefit analysis, in instances when quantifying the benefits is difficult (Kaplan 2014b), or when outputs are standardized. The benefit of cost-effectiveness analysis is its simpler methodology.
Multicriteria Analysis
This uses weighting and scoring of the most important project impacts. It is often used when quantification of costs and benefits is not pursued. Multicriteria analysis can be used to compare alternative actions based on the aggregation of criteria, which can be qualitative or quantitative. Multicriteria analysis can be easy to apply. However, it lacks theoretical or conceptual underpinnings for investment appraisal. The approach relies on decision makers having a high degree of discretion and creates the risk of preferences driving the analysis.
Simplified Methodologies
Simplified methodologies attempt to evaluate a project using techniques that are simpler in scope. These are used for back-of-the-envelope analysis of low-cost investments and could include simplified templates for cost-benefit analysis or cost-effectiveness analysis, or simplified multicriteria analysis using a few weights with a basic rating scale. These approaches can be used when more rigorous methods are infeasible or too costly.
Source: Authors.
The main phases of evaluation encompass the following:
Prefeasibility (also known as options appraisal in some systems), under which a study is prepared presenting the relevant alternatives to solve a given problem, risks are identified, and preliminary estimates of costs and benefits are provided; and
Feasibility, which expands on the prefeasibility phase by refining data collection, providing detailed estimates of costs and benefits for the selected alternative, performing a detailed assessment of risks, and assessing environmental and social impacts (Rajaram and others 2010).
In some countries, detailed project designs and tender documents are also subject to evaluation. Reappraisal may be carried out if project assumptions change after approval, or at the end of the construction phase or during operation of the project, mainly for monitoring purposes.2,3
Conditions for application of the appraisal system vary widely between countries. Decision rules around cost thresholds and exemptions govern whether project proposals are subject to appraisals and, if so, which types of appraisals are to be used. In principle, all projects should be covered by an economic appraisal. However, in practice, given resource and capacity constraints, countries limit how and under which conditions different appraisal techniques are applied. In some countries, such as Norway, Canada, and Korea, only larger projects are subject to rigorous cost-benefit analysis. In other countries, such as Ireland, the appraisal methodology depends on the type, scale, and complexity of the project (Box 13.2).
The share of the public investment budget that is subject to project appraisal is determined largely by the thresholds applied in the appraisal system. Yet, no established best practice exists when it comes to using such thresholds, which have a large range, even if normalized by GDP (Table 13.1). This indicates that appraisal is operationalized very differently across countries. Thresholds are usually determined through considerations about capacity, assessment of the risk of poor project proposals, and the size of the budget. Many low-income developing countries would find it practical to set a higher initial threshold when capacity is low and gradually lower it as the capacity of the system matures.
Thresholds for Requiring Application of the Appraisal System
Examples illustrate the wide variation between countries in conditions for application of the appraisal system in terms of threshold values and exempted sectors or areas.
In Chile, all investment initiatives financed by the government, regardless of the amount, are subject to technical and economic analysis. This includes municipal projects financed with capital transfers from the central government, provided the transfers cover more than 50 percent of project costs (Ministry of Social Development and Ministry of Finance 2018).
In Korea, the threshold is W50 billion ($43 million) for central government projects, and W30 billion ($26 million) for subnational government projects or projects with private participation receiving a central subsidy equal to or greater than that amount (Kim 2012).
In Canada (Québec), the threshold is Can$50 million ($38 million) for all projects, except for those related to the maintenance or improvement of transport infrastructure, in which case the threshold is Can$100 million ($76 million), according to Samset and others (2016).
In Norway, the threshold for central government projects is NKr750 million ($84 million), according to Samset and others (2016).
In Ireland, the appraisal methodology to be used depends on the type, scale, and complexity of the project:
For project proposals below €10 million ($11 million), approving authorities should engage with sponsoring agencies as to whether an economic appraisal is required and what type of economic appraisal is appropriate.
For all other project proposals, approving authorities and sponsoring agencies should engage on the choice of the appropriate appraisal methodology in line with sectoral guidance. Wherever possible, cost-benefit analysis should be used. In cases where this may not be possible or desirable, cost-effectiveness or multicriteria analysis may be used.
Cost-benefit analysis is used for all projects more than €100 million ($111 million) (Department of Public Expenditure and Reform 2019).
Source: Authors.
Table 13.1.
Appraisal Thresholds Normalized by GDP
Source: World Bank staff calculations using GDP estimates (current US dollars) from the World Development Indicators.
Table 13.1.
Appraisal Thresholds Normalized by GDP
| Country | 2018 GDP ($ billion) | Project Cost Threshold ($ million) | Threshold/GDP (x 1,000) | Normalized Threshold/GDP (Relative to Ireland) |
|---|---|---|---|---|
| Canada | 1,713 | 38 | 0.022 | 0.77 |
| Ireland | 382 | 11 | 0.029 | 1.00 |
| Korea | 1,619 | 43 | 0.027 | 0.92 |
| Norway | 434 | 84 | 0.193 | 6.73 |
Source: World Bank staff calculations using GDP estimates (current US dollars) from the World Development Indicators.
Table 13.1.
Appraisal Thresholds Normalized by GDP
| Country | 2018 GDP ($ billion) | Project Cost Threshold ($ million) | Threshold/GDP (x 1,000) | Normalized Threshold/GDP (Relative to Ireland) |
|---|---|---|---|---|
| Canada | 1,713 | 38 | 0.022 | 0.77 |
| Ireland | 382 | 11 | 0.029 | 1.00 |
| Korea | 1,619 | 43 | 0.027 | 0.92 |
| Norway | 434 | 84 | 0.193 | 6.73 |
Source: World Bank staff calculations using GDP estimates (current US dollars) from the World Development Indicators.
In practice, no single appraisal system covers all public investment. For example, a central project appraisal system might not cover subnational spending or spending by state-owned enterprises. Also, some sectors could be exempted.4 Again, there is no best practice on decision rules for jurisdictional and sectoral coverage, but they should be based on an assessment of the risk of low- or negative-value projects by sector, appraisal capacity, and the overall size of the portfolio. In general, countries should aim to expand coverage of their appraisal system. In some countries, line ministries also play a technical oversight role in relation to subnational investment and can provide technical support for their project preparation (Box 13.3).
In principle, all investment projects should undergo the same appraisal process, regardless of funding and procurement modalities, although in many countries that is not always the case. The decision on whether to realize a project through budget funding, donor funding, or a public-private partnership should be taken after the project has been determined to be a government priority and appraised.
Decision Rules for Project Appraisal
Project appraisals consider many dimensions related to project proposals, from policy relevance and economic rationale to social and environmental impacts. They delve into the technical design and engineering of the proposed solutions and assess whether financial arrangements are sound and affordable, including whether to use commercial opportunities. They also consider whether projects are achievable from a project management perspective. Ideally, they also assess whether potential risks have been adequately identified and mitigated, or at least minimized (see Chapter 11 of this book).
Strengthening Subnational Project Preparation and Appraisal in Colombia
In Colombia, subnational governments—especially small municipalities—have limited capacity to prepare and appraise projects. Analyses have shown that public investment portfolios at the subnational level are fragmented and not aligned with strategic planning. In some cases, municipalities propose small projects only because of their lack of capacity to identify and prepare bigger projects. The general adjusted methodology for project preparation and appraisal does not differentiate between projects with different levels of complexity or risks. Moreover, projects financed by royalties from nonrenewable natural resources often do not consider operations and maintenance costs, which undermines the long-term sustainability of investments. The National Planning Department has designed a series of toolkits and technical assistance mechanisms to help municipal governments improve project preparation and appraisal. Colombia is also advancing a series of policy reforms aimed at strengthening collaboration among subnational governments in public investment and promoting development of high-impact regional investments.
Source: Authors.
Evaluation of proposed solutions to a problem should explore a variety of alternatives, such as whether to use new, refurbished, or used equipment; or whether to rent, purchase, or build an asset. It should consider variations in scale and timing, the output to be produced, and the intended service quality. Combinations of recurrent and capital inputs should also be considered, along with whether project services should be outsourced, the alternatives for location and sites, and regulatory issues.
These myriad considerations, when contextualized in specific countries, result in a variety of approaches to decision making. For example, key process responsibilities at the prefeasibility stage vary. In Chile, the line ministry is in charge of proposing and appraising projects, while reviewing is the responsibility of a central evaluation unit within the Ministry of Social Development. The ministry provides a recommendation, and its decision stands unless it is overruled by the president.
In Korea, the line ministry submits a list of projects that are candidates for a Preliminary Feasibility Study to the Ministry of Economy and Finance. The preliminary study helps the economy and finance ministry assess the validity of public sector projects (Korea Development Institute 2016b). The ministry selects these projects on the basis of rationale, relevance, and affordability, among other things. The study is the responsibility of the economy and finance ministry but is undertaken by the Public and Private Infrastructure Investment Management Center, an independent professional entity within the Korea Development Institute.5
The United Kingdom and Ireland offer a contrast to Korea and Chile in that their central evaluation units play less of a gatekeeper function. Their role is more advisory than regulatory, and the line ministry has more authority. For example, in Ireland the central unit does not do appraisals but only reviews cost-benefit analysis for the largest projects before they go to the cabinet. Furthermore, it has less control over the methodological framework.
Appraisal Methodologies
The methodological foundations of project appraisal in welfare economics are broad and deep. Most project appraisal methodologies in use today reflect this academic pedigree (for example, Harberger 1972; Jenkins, Kuo, and Harberger 2011). The following are three notable good-practice country experiences:
Chile’s methodological approach, the General Methodology for the Preparation and Evaluation of Projects, is one of the most comprehensive and transparent in use (see Ministry of Social Development 2013). The methodology, as well as nationally applied parameters (such as the economic cost of foreign exchange) and sectoral conversion factors (such as the economic cost of unskilled labor) calculated from it, are well developed, and most of the methodological work (and the conversion factors) are available online.6 Chile’s system also applies social cost-benefit analysis as the default mode of analysis for public investment.
Korea’s multicriteria analysis uses a decision-making technique that treats economic analysis (largely cost-benefit analysis) as a core factor but also considers others, including policy analysis and regional development analy-sis.7 These three main factors are weighted according to government priorities and have been revised over time. Historically, economic analysis has been weighted at 40 percent to 50 percent, making it the most significant factor in the overall analysis.
The United Kingdom’s system, which is laid out in the HM Treasury Green Book (2018), is perhaps the most widely emulated methodology and one of the longest in use (Box 13.4). The Green Book recommends social cost-benefit analysis as the approach to detailed comparison of the short-list of options, while social cost-effectiveness analysis is also used in some circ*mstances. The Green Book no longer recommends the use of multicriteria analysis.
Appraisal and Evaluation in the United Kingdom
The Green Book is the United Kingdom’s central government guidance on how to appraise and evaluate policies, programs, and projects. Developed by the Treasury, it applies to all proposals about public spending in the country. It provides approved guidance and methods, recommended tools for developing options, and standard values for use across government. The aim of the Green Book is to help officials develop objective advice to support decisions across government. It is geared to a variety of users, from policy officials to analysts.
The Green Book provides a high-level overview of appraisal and evaluation and describes how appraisal fits within the government decision-making processes. For practitioners, it provides more detailed information on how to generate options and undertake long-list appraisal, followed by how to undertake social cost-benefit analysis of a short-list of options. It sets out the approach to valuation of costs and benefits and outlines how to present appraisal results. Finally, the Green Book sets out the approach for monitoring and evaluation, including different types of evaluation and uses before, during, and after evaluation. It contains a variety of annexes with further technical information and values for use in appraisal across government.
Source: Authors, based on HM Treasury 2018.
Robust appraisal methodologies provide for centrally calculated national economic parameters, including shadow prices. Key parameters include the economic (or social) discount rate and conversion factors for labor and foreign exchange, as well as other input costs (energy, transportation, and so on), the social value of time, the statistical value of life, and the social price of carbon emissions. It is important for a central oversight agency, such as a finance ministry or a planning agency through a specialized unit, to provide these parameters for all stakeholders in the public investment system. Chile provides and publishes many of these parameters (Ministry of Social Development 2018). More recently, the Ugandan Ministry of Finance has published national economic parameters and a commodity-specific database of economic conversion factors, which is innovative because it allows users to adjust or update for market distortions (regarding tax and subsidy rates) easily as needed (Ministry of Finance, Planning and Economic Development 2018). This approach allows the appraisal system to be managed more sustainably because it keeps down the costs of updating the large set of conversion factors (Jenkins, Kuo, and Harberger 2011).
To be useful, the general methodology must also be complemented by specific sectoral guidelines or applications. The specific aspects of applying an appraisal methodology to different sectors will vary, even among subsectors. Good-practice systems produce detailed guidance on how to apply the general methodology; for example, how to calculate economic (social) benefits by sector. In Chile and Korea the sectoral guidelines are prepared by the central oversight agency, whereas in the United Kingdom and Ireland they are prepared by spending units, consistent with the general methodology (and central units providing advice and guidance as needed). Chile has published more than 20 sector-specific methodologies, including for water, transport, energy, communications, education, health, justice, sports, and public buildings (Ministry of Social Development 2019b). Korea has about a dozen sector-specific methodologies, including for airports, ports, information technology, roads and railways, social welfare, health, and industrial complexes (Korea Development Institute 2016a).
A core element of a well-developed appraisal methodology should be the requirement to conduct risk analysis (Chapter 11). The main techniques for managing uncertainty in project appraisal are sensitivity analysis, which identifies key risk variables through determining their impact on project outcomes; scenario analysis, in which multiple variables are altered simultaneously to demonstrate the combined impacts of particular scenarios (for example, best case and worst case); and the Monte Carlo analysis, in which risk variables (identified from the sensitivity analysis) are modeled as probability distributions, which generate project outcomes as expected values (Jenkins, Kuo, and Harberger 2011). Risk analysis is required by the guidelines in Chile, Colombia, Ireland, Korea, and the United Kingdom, among others.
More recently, research on optimism bias, which shows that project costs and completion times tend to be systematically higher and longer, respectively, than initially projected, has motivated some governments to adopt methods to control ex ante for such biases (Flyvbjerg 2006). The United Kingdom’s Green Book recommends applying adjustments for optimism bias and provides adjustment factors for different generic categories of spending (for example, for capital costs, adjustment factors for buildings, civil engineering, equipment, and so on). The Green Book also recommends reviewing the optimism bias adjustment at different stages of appraisal. Procedures for this include the Gateway Review process (Box 13.5). One unsettled aspect of appraisal methodologies is how to treat equity effects. In cost-benefit analysis, costs and benefits are typically aggregated across individuals, without taking into consideration who receives the benefit or who pays the cost.8 The methodological foundation of distributive analysis in project appraisal is to acknowledge that any economic externalities from a project accrue to different stakeholders, whether consumers (that is, project beneficiaries), producers, labor, or government (Jenkins, Kuo, and Harberger 2011). Some countries, such as the United Kingdom and New Zealand, require distributional analysis, in which, at a minimum, appraisers quantify how project costs and benefits accrue to different socioeconomic groups. These systems also recommend that distributional weights (for example, that benefits for low-income groups receive higher weight) should be used where possible. The use of distributional weights, however, is not widely accepted because of the potential for inefficiencies to be generated and greater discretion in decision making to be introduced.
The United Kingdom’s Gateway Review Process
The Gateway Review process, which was introduced by the Office of Government Commerce, requires examination of a program or project at key decision points in its life cycle to provide assurance that it can move successfully from one stage to the next. The process is mandatory for procurement, IT-enabled, and construction programs and projects in the United Kingdom. The reviews are structured as “peer reviews,” in which independent practitioners examine the progress and likelihood of successful delivery of the program or project.
The reviews provide valuable perspective to internal teams and also serve as an external challenge to the robustness of plans and processes. They help to bring realism to estimated completion times and cost targets. In the case of projects, the process examines five delivery areas beyond project appraisal: (1) business justification, (2) the delivery strategy, (3) the investment decision, (4) readiness for service, and (5) an operational review and benefits realization.
Source: Authors, based on Office of Government Commerce (2007).
Undue Political Influence in Project Appraisal and Selection
Political influence is a defining factor in the allocation of public resources at both the appraisal and selection stages of public investment management. Political considerations are important for determining investment priorities and the types of projects that fit national, regional, and sectoral plans. However, political influence can sometimes also be used to override the technical appraisal, and this can generate significant inefficiencies. In many cases, political decisions are opaque, which limits accountability and likely results in inefficiencies that include decisions to undertake white elephant projects.
Countries have adopted different approaches to factor in political priorities in decision making. In Chile, the government aims to maintain the technical purity of its rigorous cost-benefit analysis approach, but it makes a formal provision for the president to override the appraisal to account for political priorities. Projects can be designated as Presidential Priorities, with the ability to veto appraisal results vested in the president, although this practice is becoming less common. The advantage of the system is that it is formalized, transparent, and can contribute to accountability as the president is associated with those projects. Korea takes a different approach by formally incorporating variables for policy priorities and equitable territorial development directly into the multicriteria analysis.9 One of the political pressure points in Korea is the impetus for more equitable regional development, so the Korean system attempts to quantify a project’s ability to address regional needs. This factor is then weighted and, along with the cost-benefit analysis, incorporated into the formal appraisal. Challenges with this approach include the generation of pressures to increase the weight of this factor in decision making. Moreover, most low-income developing countries would not have the technical capacity needed to implement the approach. That said, the Chilean and Korean examples show how transparent decision rules can improve outcomes.
Capacity Development Approaches
It is not surprising that the best-performing systems—in terms of analytically rigorous methods and their consistent implementation—have taken systematic approaches to building public sector capacity through significant investment over several decades. Both Chile and Korea are cases in point. Chile, for example, has been providing training since the mid-1970s and the responsible ministry continues to offer basic, intermediate, and advanced diploma courses in social project evaluation, as well as specialized courses. Hundreds of officials are trained each year.10 Korea has also invested significantly in developing its methodologies and training its officials.
In low-capacity countries, external consultants could play an important technical role in project appraisal. The disadvantage of this approach is that it engenders possible conflicts of interest (consultants may have incentives to provide project sponsors with the results they would like to see) and a lack of incentives to develop capacity in house. Outsourcing appraisal functions may also mean that public officials do not develop the skills needed to be intelligent consumers of consultants’ reports. Taking time and dedicating the resources to developing capacity to design and implement a project appraisal system is a critical and very likely high-return investment.
Considerations for Project Selection
The decision to proceed with a project11 can be quite contentious politically. As such, it is important to clarify institutional roles and establish clear processes for project selection. Several European countries have introduced tools such as models, criteria, or scoring grids to strengthen their project selection functions.12
The IMF has identified several practices and procedures for strong project selection, which are outlined in the revised PIMA framework. All major projects, regardless of whether they are financed by the government’s own resources or whether they are donor funded or are public-private partnerships, should be reviewed by a central ministry. Ideally, independent experts or organizations provide input into the process before the decision to include a project in the budget. The revised PIMA framework also highlights the importance of governments publishing standard criteria for project selection, outlining a process for the selection of projects, and making the selection through the given process (IMF 2018).
The PIMA framework also suggests that the government should maintain a pipeline of appraised investment projects, which should be used for selecting projects that will be included in the budget (IMF 2018). In Chile, for example, the government has developed a pipeline of appraised and approved projects that are eligible for budget funding (Rajaram and others 2014). In Ireland, the government recently expanded the functionality of its Capital Tracker database to improve the inventory of the pipeline of capital investment projects with a medium-term horizon (Department of Public Expenditure and Reform 2018).13
Colombia is a good example of a country with a database that supports project selection. Sponsoring agencies and line ministries conduct a formal project review, which is then subjected to independent review from the National Planning Department (World Bank 2018). The department determines which projects are feasible and can be preselected for inclusion in the Bank of National Investment Programs and Projects, the country’s project database. Line ministries propose which projects should be financed from the national budget from those that have been included in the database. Together with the National Planning Department, they decide which ones to include in the Public Investment Program (IMF 2017). Projects financed by royalties go through a different process.
Australia has gone a step further and has developed an Infrastructure Priority List, a publicly available list of nationally significant infrastructure investments that the country needs over the next 15 years. The Priority List is updated regularly, includes projects with a full business case that have been assessed by the independent Infrastructure Australia Board, and guides decisions on how best to allocate resources. An infrastructure priority map available at Infrastructure Australia’s website presents information about projects and initiatives (early-stage solutions without a full business case) that have received a positive evaluation (Infrastructure Australia 2019).
Project selection criteria should be clear and transparent (IMF 2015). A good example of project selection criteria comes from the Slovak Republic, which uses a scorecard with 23 criteria for prioritizing projects organized across three principles. The projects are reviewed by the Ministry of Finance, which scores them according to the criteria. The three principles include an assessment of the strategic relevance of the project, a review of the economic appraisal and fiscal afford-ability of the project, and an assessment of the maturity of the project and its related implementation plan (IMF 2019).
Linking Project Appraisal and Selection to the Budget Cycle
Decisions on which projects to pursue should be an integral part of the budget process, and strong infrastructure governance systems link project appraisal and selection to the budget cycle. Looking at the institutional arrangements, this implies having a well-defined process for project appraisal and transparent criteria for project selection. Project appraisal and selection should perform a gate-keeping function, ensuring that only projects that have gone through the process (and have been independently reviewed) are selected for funding in the budget. The budget preparation process should also adequately integrate projects’ recurrent and capital expenditures (that is, should consider projects’ capital outlays and the funds needed for operations and maintenance). A key consideration is the affordability of projects, ensuring that sufficient budget funding is available for the selected projects (Rajaram and others 2014); or, to put it another way, which projects to select, given the budget envelope. If too many projects vie for finance through the budget, prioritization should be based on the projects’ net present value, pursuing those with the highest value given the budget constraint.14 Kazakhstan has a three-stage upstream public investment management process for project proposal, appraisal, and selection, which is conducted through the annual budget. A key feature of the process is that to be eligible for inclusion in the draft budget, projects must have completed a feasibility report and received a positive appraisal and a positive decision from the relevant budget committee. The criteria for project proposal, appraisal, and selection is clearly outlined in the country’s budget code (Kim and others 2020).
Conclusions
A clear, well-supported appraisal methodology and published project selection criteria with well-defined processes for project selection are the foundations of a good infrastructure governance system. This includes having a clear methodology with national and sectoral guidelines for project appraisal. A management or research unit employing robust methodology is also important. Where it should be housed (the finance or planning ministries, or an affiliated think tank) would depend on the country setting.
In practice, the appraisal and selection process cannot be reduced to a purely technical exercise. Political influence exists in developing, emerging, and advanced systems and this affects (or even determines) how projects are ultimately decided. This is the main limitation of the technical work. Advanced systems are designed and negotiated to channel politics transparently and in a structured manner. Political influence can be an issue in project appraisal and selection in both weak and strong infrastructure governance systems, but it can be tempered through rigorous analysis, scrutiny by a central ministry using clear and transparent procedures, and an independent review of projects before they are included in the budget. Capacity development, as illustrated by the Chilean and Korean experiences, is an expensive, long-term undertaking. In low-capacity settings (low-income developing countries and fragile and conflict-affected states), outsourcing or using consultants for core functions are possible alternatives, but they must be balanced with sustained investments in capacity building and the development of practical know-how. History shows that all project appraisal and selection systems adapt over time, and while some improve, some also deteriorate. As such, there is a need for constant vigilance to ensure that systems adopt relevant new techniques while also preventing backsliding.
References
Burduja, Sebastian Ioan, Florian Gaman, Victor Giosan, Graham Glenday, Eric Nolin Huddleston, Marcel Ionescu-Heroiu, Elena Iorga, and others. 2014. Identification of Project Selection Models for the Regional Operational Program 2014–2020. Washington, DC: World Bank.
- Search Google Scholar
- Export Citation
Department of the Prime Minister and Cabinet. 2016. Cost-Benefit Analysis. Canberra: Department of the Prime Minister and Cabinet.
- Search Google Scholar
- Export Citation
Department of Public Expenditure and Reform. 2018. “Minister Donohoe Updates Cabinet on the Ongoing Delivery of Project Ireland 2040.” Press release, Gov.ie, September 19.
- Search Google Scholar
- Export Citation
Department of Public Expenditure and Reform. 2019. Public Spending Code: Guide to Evaluating, Planning and Managing Public Investment. Dublin: Department of Public Expenditure and Reform.
- Search Google Scholar
- Export Citation
Florio, Massimo, and Silvia Vignetti. 2013. The Use of Ex Post Cost-Benefit Analysis to Assess the Long-Term Effects of Major Infrastructure Projects. London: Centre for Industrial Studies.
- Search Google Scholar
- Export Citation
Flyvbjerg, Bent. 2006. “From Nobel Prize to Project Management: Getting Risks Right.” Project Management Journal 37 (3): 5–15.
- Search Google Scholar
- Export Citation
Harberger, Arnold C. 1972. Project Evaluation: Collected Papers. London: Palgrave Macmillan UK.
- Search Google Scholar
- Export Citation
HM Treasury. 2018. The Green Book: Central Government Guidance on Appraisal and Evaluation. London: HM Treasury.
- Search Google Scholar
- Export Citation
Infrastructure Australia. 2019. Infrastructure Priority List. Sydney: Infrastructure Australia.
- Search Google Scholar
- Export Citation
International Monetary Fund (IMF). 2015. Making Public Investment More Efficient. Washington, DC: IMF.
- Search Google Scholar
- Export Citation
IMF. 2017. “Ireland: Technical Assistance Report—Public Investment Management Assessment.” International Monetary Fund, Washington, DC.
- Search Google Scholar
- Export Citation
IMF. 2018. “Public Investment Management Assessment—Review and Update.” International Monetary Fund, Washington, DC.
- Search Google Scholar
- Export Citation
IMF. 2019. “Slovak Republic: Technical Assistance Report—Public Investment Management Assessment.” International Monetary Fund, Washington, DC.
- Search Google Scholar
- Export Citation
Jenkins, Glenn P., Chun-Yan Kuo, and Arnold C. Harberger. 2011. “Cost-Benefit Analysis for Investment Decisions.” Development Discussion Papers, 2011–5.
- Search Google Scholar
- Export Citation
Kaplan, Josiah. 2014a. Cost-Benefit Analysis. Melbourne, Australia: BetterEvaluation.
- Search Google Scholar
- Export Citation
Kaplan, Josiah. 2014b. Cost-Effectiveness Analysis. Melbourne, Australia: BetterEvaluation.
- Search Google Scholar
- Export Citation
Kim, Jay-Hyung. 2012. The Republic of Korea: PIM Reform after the Financial Crisis. Washington, DC: World Bank.
- Search Google Scholar
- Export Citation
Kim, Jay-Hyung, Jonas Alp Fallov, Simon Groom, and Marin Darcy. 2020. Public Investment Management Reference Guide. Washington, DC: World Bank.
- Search Google Scholar
- Export Citation
Korea Development Institute Public and Private Infrastructure Investment Management Center (KDI PIMAC). 2019. 2018 PIMAC Annual Report. Sejong: Korea Development Institute.
- Search Google Scholar
- Export Citation
Korea Development Institute. 2008. General Guidelines for Preliminary Feasibility Studies, 5th edition. Sejong: Korea Development Institute.
- Search Google Scholar
- Export Citation
Korea Development Institute. 2016a. Preliminary Feasibility Study Guidelines. Sejong: Korea Development Institute.
- Search Google Scholar
- Export Citation
Korea Development Institute. 2016b. Public Institution Evaluation. Sejong: Korea Development Institute.
- Search Google Scholar
- Export Citation
Mauro, Paolo. 1997. “Why Worry about Corruption?” Economic Issues, International Monetary Fund, Washington, DC.
- Search Google Scholar
- Export Citation
Ministry of Finance, Planning and Economic Development. 2018. Commodity-Specific Economic Conversion Factors Database for the Republic of Uganda. Ministry of Finance, Planning and Economic Development, Kampala, Uganda.
- Search Google Scholar
- Export Citation
Ministry of Social Development. 2013. General Methodology of Project Preparation and Evaluation. Santiago, Chile: Ministry of Social Development.
- Search Google Scholar
- Export Citation
Ministry of Social Development. 2018. Social Prices 2018. Santiago, Chile: Ministry of Social Development.
- Search Google Scholar
- Export Citation
Ministry of Social Development. 2019a. Training for the National Investment System. Santiago, Chile: Ministry of Social Development.
- Search Google Scholar
- Export Citation
Ministry of Social Development. 2019b. Requirements by Sector for Project Formulation—Current Classification. Santiago, Chile: Ministry of Social Development.
- Search Google Scholar
- Export Citation
Ministry of Social Development and Ministry of Finance. 2018. Norms, Instructions, and Procedures for the Public Investment Process (NIP). Santiago, Chile: Ministry of Social Development, Ministry of Finance.
- Search Google Scholar
- Export Citation
Office of Government Commerce. 2007. OGC Gateway Process: Review 0–Strategic Assessment. London: Office of Government Commerce.
- Search Google Scholar
- Export Citation
Office of Management and Budget. 1992. “Guidelines and Discount Rates for Benefit-Cost Analysis of Federal Programs.” Circular A-94. Washington, DC: Office of Management and Budget.
- Search Google Scholar
- Export Citation
Rajaram, Anand, Kai Kaiser, Tuan Minh Le, Jay-Hyung Kim, and Jonas Frank. 2014. The Power of Public Investment Management: Transforming Resources into Assets for Growth. Washington, DC: World Bank.
- Search Google Scholar
- Export Citation
Rajaram, Anand, Tuan Minh Le, Nataliya Biletska, and Jim Brumby. 2010. A Diagnostic Framework for Assessing Public Investment Management. Washington, DC: World Bank.
- Search Google Scholar
- Export Citation
Samset, Knut F., Gro Holst Volden, Nils Olsson, and Eirik Vårdal Kvalheim. 2016. Governance Schemes for Major Public Investment Projects: A Comparative Study of Principles and Practices in Six Countries. Trondheim: Norwegian University of Science and Technology.
- Search Google Scholar
- Export Citation
World Bank. 2018. Public Investment Management in Colombia. Washington, DC: World Bank.
- Search Google Scholar
- Export Citation
1
See the new Green Book (HM Treasury 2018).
2
A unique characteristic of project appraisal in Korea is the use of a Reassessment Study of Feasibility, which is triggered when cost overruns exceed 20 percent of planned costs (for certain types of projects). Project costs are monitored through a management system. In a few instances, projects were canceled because the Reassessment Study was applied (Kim 2012).
3
Another form of evaluation is retrospective analysis, which is conducted at the end of the project (Florio and Vignetti 2013). This type of evaluation compares the outputs and outcomes of a project with the objectives envisioned at the design stage (Rajaram and others 2014). Technically, analysis after the fact is not part of project appraisal, but it provides a feedback loop with lessons learned from concluded projects that ideally would feed into the design and appraisal of new projects.
4
In Chile, the appraisal system excludes large items such as housing subsidies and municipal investment, with some exceptions (see Box 13.2), and state-owned enterprises are covered only if the public sector has a capital contribution greater than 50 percent of state-owned enterprises’ social capital. Moreover, defense and spending on natural disaster–related emergencies and reconstruction are exempted. So are public infrastructure conservation projects, but they must be recorded in the country’s Integrated Project Bank (Ministry of Social Development and Ministry of Finance 2018).
5
A key metric to determine whether the appraisal function of a country is robust is the rejection rate (that is, what percentage of projects are accepted versus rejected). In Korea, during 1999–2018, 64.2 percent of projects were deemed feasible (KDI PIMAC 2019). This means that more than one-third of project proposals were rejected. In many developing countries, the rejection rate is not tracked.
6
Conversion factors convert market prices to economic prices, eliminating distortions and accounting for externalities.
7
Policy analysis considers aspects such as consistency with policy and risk factors in pursuing the project. Regional development analysis reviews the level of regional development and ripple effects on the regional economy. See Korea Development Institute (2008) for more details.
8
See Department of the Prime Minister and Cabinet (2016) for guidance from Australia on how to account for equity within cost-benefit analysis.
9
As described previously, Korea’s multicriteria analysis combines quantitative and qualitative criteria for decision making. Economic analysis (largely cost-benefit analysis) is the most heavily weighted component. Projects are also evaluated from policy analysis and regional development perspectives, which are given different weights.
10
In Chile, about 550 public sector officials are trained per year in formal courses and 600 in other project training as needed (Ministry of Social Development 2019a).
11
A decision to proceed would not in itself guarantee that funding will be obtained. Securing funding for a project occurs through the budgetary process (Kim and others 2020).
12
See Burduja and others (2014) for an overview of project selection models used in Estonia, Germany, Ireland, Italy, Lithuania, Poland, Slovenia, and the United Kingdom related to the use of funds from the European Union’s Regional Operational Program.
13
According to the 2017 Ireland PIMA, a pipeline of approved projects was available at the department level, but not at the national level.
14
Projects with a positive net present value increase social welfare and are generally preferred over those with a negative net present value, which should be avoided (Office of Management and Budget 1992).
