Appraisal methodology

Overview

The diagram below describes the steps that the Project Team must take for an Economic Appraisal. These are explained further in the following sections.

Steps required for Economic Appraisal

The Project Team may also elect to undertake an Economic Impact Assessment (EIA). An EIA differs from the Economic Appraisal in that it estimates the gross impact that the project is expected to have on the economy in terms of employment and economic output, whereas an Economic Appraisal compares the net benefits and costs of a project to the community.

An EIA is not mandatory for any project, Program or Precinct under the Capital Framework. The EIA complements, rather than replaces, the Economic Appraisal. For example, the Project Team can use the outputs of the Economic Appraisal to derive inputs for the EIA. However, the Economic Appraisal remains the primary tool for investment decision making by Government (including the CBA and MCA).

Step 1: Define appraisal framework (Tier 1, Tier 2 and Tier 3)

The Project Team is required to undertake Step 1 for all Tiers. Step 1 involves the Project Team defining the Base Case and at least one (preferably two or three) Project Case option(s), and identifying benefits (and economic costs) and project costs relevant to the project.

Base Case

The Base Case for an Economic Appraisal is the ‘Business-as-Usual’ or ‘do minimum’ option that would be expected to take place should the project not go ahead. Importantly, the Base Case is not necessarily the same as a ‘do nothing’ scenario. It should represent the scenario under which there is no change in planned expenditure or policy, other than any expenditure required to continue to provide the current or Government’s previously agreed level of service, such as ongoing repairs, maintenance and lifecycle replacements.

The Project Team should define a Base Case for projects at all Tiers. For Tier 1 and Tier 2 projects, the Project Team should provide additional detail that will allow a full CBA to be conducted, with assumptions and the level of service provided under the Base Case clearly defined.

The Base Case should include the following:

• Any known (i.e. already committed or funded) changes to infrastructure or services that will occur regardless of whether the project, or other investment options, go ahead
• Operating and maintenance costs required to ensure that existing assets can continue to provide a minimum level of service into the future.

For the Base Case, the Project Team should specify the following in relation to services or outcomes:

• The existing service(s) that are required to be delivered to align to current policy and that will be maintained into the future under the Base Case
• Likely outcomes for users from the service provided under the Base Case
• Current and future expected maintenance requirements, including staffing costs
• Timing of ongoing costs, where available
• Other future developments that will affect the demand for the service(s), such as one-off events and land use changes that are unrelated to the project (e.g. land releases that will affect the demand for the service).

An appropriately defined Base Case is critical for an effective Economic Appraisal, as it forms the starting point from which benefits and economic costs are measured for each Project Case. An accurate definition of the Base Case ensures that the incremental benefits (and economic costs) and project costs of each project option are measured correctly and provides an appropriate appraisal of the project’s benefits (and economic costs) and project costs.

Project Case options

The Project Team should define the Project Case options, which should match the shortlisted project options already identified through the Options Analysis process. For Tier 1 and Tier 2 projects, the Project Team is expected to provide more clearly defined project options that will allow the team to conduct a full CBA. For more information on defining options, see the Guidelines on undertaking an Options Analysis.

Where a Base Case is a ‘do minimum’ option, each Project Case should be seen as a ‘doing more’ or ‘doing extra’ option, where an additional amount of Government investment over and above that assumed for the Base Case is expected to generate additional benefits. Each Project Case should reflect a well-defined intervention such as a capital investment, or an upgrade of an existing asset or set of assets.

The Project Team should articulate the problem or opportunity, or need, that each Project Case is intended to address. This statement should provide a rationale for intervention and describe the reasons why an intervention is required to address the problem or opportunities and meet a particular need. The Project Team may refer to the ILM and/or Needs Analysis.

For each Project Case, the Project Team should specify:

• Costs required to implement the proposed solution, including both direct and indirect costs (e.g. direct costs of infrastructure upgrades and associated indirect costs including project management, procurement and ongoing management costs)
• The expected benefits (and economic costs) and project costs that will be generated through implementation of the Project Case, over and above those expected under the Base Case and including avoided costs or expenditure that would have occurred under the Base Case
• Timing of the proposed intervention(s) (e.g. the expected date that an infrastructure upgrade would be expected to be completed, and how long it will be in operation).

Identification of benefits and economic costs

The Project Team is required to identify and describe the set of benefits and economic costs that are expected to flow from a project, regardless of whether or not they are easy to quantify. For Tier 1 and Tier 2 projects, this list will form the set of benefits and economic costs to be measured through the CBA and MCA. For more information on the examples of types of benefits and economic costs, refer to the note on benefits and economic costs.

It may be difficult or impossible for the Project Team to calculate the monetary value of some benefits and economic costs, particularly those that focus on broader social or environmental outcomes generated by the project. The Project Team should still identify these benefits and economic costs in order to undertake MCA to assess them as part of the overall appraisal.

All identified benefits and economic costs should be able to be measured, even when measuring is complex. If the Project Team is unable to determine how a benefit or economic cost could be measured, it should question whether the benefit or economic cost exists.

The Project Team should refer to the ILM for the project when identifying benefits and economic costs to ensure that the Economic Appraisal will capture all of the benefits and economic costs identified in the ILM. The Project Team should frame benefits and economic costs in terms of the outcomes they will deliver due to the project’s implementation (e.g. travel time savings for public transport users or decreased wait times for services). Where possible, as part of the Economic Appraisal, the Project Team should quantify and then calculate the monetary value of these benefits and economic costs to generate an estimate of the value of the project’s benefits and economic costs.

Where there are multiple project options or Project Cases, the Project Team should state the extent to which each benefit and cost item is relevant to each option. The Project Team should provide clear evidence supporting the inclusion of each benefit and cost in the Economic Appraisal.

The following benefits may be relevant:

• Direct benefits to users of a project: for example, in relation to safety, health, productivity, etc.
• Indirect benefits arising from the project: such as environmental benefits, improved safety or community amenity outcomes. This should include the benefits to users of adjacent assets due to shifting demand, such as benefits from land use or transport network changes
• Avoided or reduced costs: incremental costs that are unavoidable if nothing is done, but may be avoided or reduced if action is taken. These costs should include indirect avoided costs, such as reductions in adverse environmental impacts (including those related to greenhouse gas emissions)
• Revenues: incremental revenues from introduction of the project such as user charges or lease revenues (the Project Team should consult closely with ICA in Treasury on these potential benefits, to avoid any double counting)
• Residual value of assets: at the end of the appraisal period (if any, and contingent upon depreciation allowance for lifecycle costs).

The Project Team must also consider the economic costs that result from the project. Economic costs are the opposite of benefits in that they are the negative effects related to the benefits described above.

Additional distinctive benefits and economic costs are likely to be relevant based on the project’s specific nature.

Identification of project costs

The Project Team should include costs of the project required to achieve any benefit identified in the Economic Appraisal, including the project’s design, construction, operation and maintenance costs.

The Project Team should provide an estimate of the costs associated with each Project Case, as well as costs within the Base Case no longer expected to be incurred as a result of the project. This estimate is required for projects at all Tiers. However, for Tier 1 and Tier 2 projects, the Project Team is expected to be able to provide more detailed and refined cost estimates suitable for inclusion in the CBA (including year-by-year profiles of costs).

All Economic Appraisals should be based on incremental benefits (and economic costs) and project costs; that is, the net benefits (and economic costs) and project costs associated with a particular project. The Project Team should exclude any changes that would have occurred anyway under the Base Case. The Project Team should make clear the assumptions underlying all capital and recurrent cost estimates in the evaluation (additional information on standard assumptions is provided at Step 2 below).

The degree of accuracy desirable will vary with the significance of the project, data availability and cost of obtaining missing data. Best estimates are often sufficient but, if there is doubt as to whether these will be acceptable, the Project Team should seek advice from ICA and FABG.

Costs should include, where relevant:

• Capital expenditure (capex) costs: including costs associated with planning,⁠ (Footnote: The Project Team may need to consult with FABG in relation to the accounting treatment of planning costs. ) procurement, and delivery of a proposed solution
• Operational expenditure (opex) costs: including ongoing operations, staffing and maintenance costs, and costs associated with periodic repair or upgrade work required
• Environmental costs: including greenhouse gas emissions generated through the construction and operational phases of the investment, where the Project Team is able to identify these costs. The Project Team should refer to EPSDD guidance on calculating the costs of carbon emissions
• Lifecycle costs: including costs of periodic replacement of assets related to the project (such as IT infrastructure and systems, vehicles, machinery or other equipment).

Within the cost estimate, the Project Team should also include an appropriate allowance for contingency – generally, the Project Team should use a P50 contingency estimate in the Economic Appraisal. More information on the development of contingency allowances is provided in the Financial Analysis Guidelines and Risk Analysis Guidelines.

The Project Team should present costs in real terms in the Economic Appraisal section, noting that costs will therefore differ from the costs included in the Financial Analysis section of the Business Case.

Step 2: Conduct Economic Appraisal (Tier 1 and Tier 2)

Key parameters and assumptions

The Project Team should set out benefits (and economic costs) and project costs in real terms over the appraisal period for the Economic Appraisal: that is, benefits (and economic costs) and project costs should not include increases in values or prices due to inflation. However, the Project Team should account for real escalation (for example, the expected increase in real wage costs or real construction material costs over time) in all benefits (and economic costs) and project costs.

The Project Team should then apply an appropriate real discount rate (refer to Key parameters for Economic Appraisal table below) to the stream of benefits (and economic costs) and project costs.

The discounting process considers the fact that initial investment costs are borne up-front, while benefits (and economic costs) and project costs (e.g. operating costs) may extend far into the future. Discounting the value of future benefits (and economic costs) and project costs brings these cashflows back to a common time dimension – present value – for the purpose of comparison. The process of discounting real benefit (and economic cost) and project cost values reflects the concepts of time preference for money and the opportunity cost of capital.⁠ (Footnote: The concept of time preference for money recognises that current consumption is valued over future consumption. Refer to the Infrastructure Australia, Assessment Framework – Guide to economic appraisal, 2021 for a more detailed discussion of present values. The concept of opportunity cost of capital recognises that since capital is limited, any capital investment occurs at the expense of an alternative investment. The discount rate therefore incorporates an assumed return on an alternative investment that is foregone. )

The table below lists the key parameters required to conduct an Economic Appraisal and suggested values for each. The Project Team can alter these parameters on a project-by-project basis following discussion with ICA.

The Project Team should clearly detail the assumptions that it has made in conducting the Economic Appraisal when presenting the results in the Business Case. Assumptions may include:

• Parameter values used (see table above) and the sources and rationales for the values used
• Inputs used in forecasting future demand for a service, asset or other initiative that affects the quantification of benefits (and economic costs) and project costs related to a project (for both the Base Case and each Project Case).

The Project Team should provide an appendix as part of the Business Case that details the inputs, parameters, assumptions and methodology used to estimate the monetary value of each benefit and cost item included in the analysis. The appendix should include sufficient detail for an experienced third party to be able to reproduce the appraisal results based on the parameters, inputs and assumptions used.

Quantifying benefits and economic costs

The process of quantifying benefits and economic costs involves taking the list of benefits and economic costs identified at Step 1, determining for each benefit and economic cost whether a monetary value can be calculated, and applying an appropriate methodology to determine a value for each benefit and economic cost.

Where supporting data are available, the Project Team should make every effort to put a value on the monetary or quantifiable benefits, economic costs and impacts. However, these Guidelines recognise that, in some cases, quantifying and monetising all identified benefits and economic costs appropriately is difficult, due to data limitations or where an appropriate methodology to quantify and calculate the monetary value of benefits is not available. Where a monetary value is unable to be calculated for benefits and economic costs, the Project Team must undertake MCA. See the information below for more information.

When undertaking a detailed CBA, the Project Team may wish to refer to detailed technical guidelines specific to the type of project being considered under the analysis. A list of selected guidelines is provided at Appendix A, covering various topics including transport projects, land use impacts and general guidance for estimating and monetising a large range of benefit items. The Project Team should refer to other guidelines not included in Appendix A if appropriate for the project.

Wider Economic Benefits (WEBs)

Analysis of WEBs is complex and requires specialist experience. The assumptions that the Project Team makes can significantly influence the estimation of the value of WEBs. Therefore, if WEBs are to be included in the Economic Appraisal, the Project Team should contact the Economic and Financial Group (EFG) within the Treasury Stream of the Chief Minister, Treasury and Economic Development Directorate (CMTEDD) and also ICA to ensure that the WEBs are measured in a consistent manner across the ACT.

WEBs are improvements in economic welfare that typically have not been captured in traditional CBA. This concept is relatively new in the Economic Appraisal of projects. Although the inclusion of WEBs is not widely accepted in a traditional CBA, both the ACT Government and Infrastructure Australia endorse their use.

WEBs include impacts on the community that are broader than the conventional benefit items generally included in an Economic Appraisal. They include:

• Agglomeration benefits: increased productivity and opportunities for collaboration due to increased density of firms and workers
• Imperfect competition benefits: increase in output in sectors with price cost margins because of an improvement of the project
• Labour supply benefits: increased labour supply resulting from a reduction or removal of barriers for new workers entering the workforce as a result of the project.

WEBs will only apply to larger, Tier 1 projects and, even then, may not apply to some projects. The Project Team should include WEBs in the appraisal for Tier 1 projects if they are likely to apply and the cost of undertaking this analysis is reasonably affordable.

The Project Team should report the results of the Economic Appraisal both with and without WEBs, so that the value of WEBs can be easily separated out from the value of other, more direct benefits and economic costs.

Appendix A includes links to detailed guidelines on WEBs that the Project Team should use to estimate these benefits. In cases where the Project Team has chosen to quantify WEBs in the Economic Appraisal, the Project Team should clearly include the sources used to quantify the WEBs, the underlying parameters and any uncertainties relating to the source.

Quantifying project costs

The Project Team should present costs on a year-by-year basis, including capital expenditure (capex), ongoing operational expenditure (opex), lifecycle costs and contingency. Similarly to the estimation of benefits and economic costs, the Project Team should present costs in real terms and discount future costs by the appropriate real discount rate to give a present value of project costs.

In contrast to a Financial Analysis, the Economic Appraisal requires the estimation of benefits (and economic costs) and project costs in real terms.

The Project Team should include an appropriate allocation for contingency within the cost estimate used in the Economic Appraisal. For Tier 1 projects, the Project Team is expected to provide a probabilistic P50 cost estimate (P50 is used for the Economic Appraisal, with P90 generally used for estimating financial project costs). For Tier 2 projects, a less complex estimation method for contingency may be used, but it is recommended that the Project Team develop a P50 cost estimate for use in the Economic Appraisal where possible.

Greenhouse Gas Emissions

The quantitative Economic Appraisal needs to take into account the impacts of both the Base Case and the Project Case on Greenhouse Gas (GHG) emissions, using an agreed Social Cost of Carbon. These impacts include both emissions from operations of existing assets (Base Case) and new assets (Project Case), plus emissions from the construction of the new assets (Project Case). The Climate Change and Energy Policy branch in EPSDD has developed guidance on calculating GHG emissions and on the Social Cost of Carbon.

Where the project is likely to have an impact on the consumption of transport fuels or natural gas relative to the Base Case of a project, then the Economic Appraisal should include an assessment of the impact of the project on the ACT’s emissions reduction targets set in the Climate Change and Greenhouse Gas Reduction Act 2010.

Non-monetised benefits and economic costs

Where possible, the Project Team should aim to quantify and calculate the monetary value of all benefits, economic costs and impacts from a project so that the outputs of the Economic Appraisal reflect as many of the project’s benefits and economic costs as possible. However, it can be difficult or impossible to calculate the monetary value of some project benefits and economic costs (e.g. some of the benefits considered under the Wellbeing Impact Assessment), despite these benefits being key to a project’s impact on the community.

In recognition of the fact that a monetary value cannot be calculated for all benefits and economic costs, the Project Team should include an MCA of non-monetised benefits and economic costs in the Economic Appraisal. This analysis should take the form of a concise qualitative discussion of benefits and economic costs that the project is expected to generate and that may be used for comparative purposes within the Options Analysis. The Project Team is not expected to conduct an extensive quantitative exercise with weightings and scores as part of this MCA. However, the Project Team should describe these benefits and economic costs for which a monetary value cannot be calculated in the most accurate, concise and objective terms possible. The MCA should be informed by the findings of the Wellbeing Impact Assessment.

This MCA analysis allows the Project Team to demonstrate the project’s benefits and economic costs without needing to determine a way to calculate the monetary value of a benefit or economic cost, where calculation may be impractical or impossible.

Other considerations

The Project Team should also be aware of other considerations in relation to Economic Appraisal methodology when conducting an Economic Appraisal. Although the technical guidelines referenced in Appendix A provide information on best practice methodology, two important points are summarised below for ease of reference.

Resource and market prices

The Project Team is expected to undertake the CBA based on market prices. There are sometimes differences between market prices and underlying resource costs, as individuals place higher value on something than the actual resource cost of making it (due to taxes). In this case, the Project Team may need to include resource cost corrections to capture the impact on revenue streams. Examples of taxes that may result in the need for resource cost corrections in an Economic Appraisal include tolls, stamp duty, GST, land tax, income and payroll taxes and fuel excise duty.

In the case of fuel excise duty, the market price of fuel exceeds its resource cost by the amount of excise duty and GST. If an intervention causes an increase (or reduction) in fuel consumption, the impact on consumers will be measured as the market cost of the fuel, but there will also be an increase (or reduction) in fuel excise duty received by government (in this case, the Commonwealth Government). The Project Team should capture the impact on such revenue streams in the CBA as a resource cost correction.

A detailed theoretical explanation of resource cost correction is included in the Transport for NSW Principles and Guidelines for Economic Appraisal of Transport Investment and Initiatives (Appendix 9). The Victorian Department of Treasury and Finance guidelines also includes an appendix on the use of ‘shadow prices’ to correct for distortion in market prices (see Appendix D, Economic Evaluation for Business Cases – Technical Guidelines).

Rule of a half

The rule of a half can be applied to benefits (or economic costs) captured within the CBA that are as a result of new or lost users.

Any intervention attributable to the project is likely to have demand impacts. For any new or lost users of the services or infrastructure, the Project Team should only capture half of the benefits or economic costs. This rule reflects the fact that the size of the benefits (or economic costs) delivered to a new user (or lost user) varies between users, with not all new or lost users expected to feel the full impact of the benefit (or economic cost). A more detailed explanation of the theoretical basis for the rule of a half is included in the Transport for NSW Principles and Guidelines for Economic Appraisal of Transport Investment and Initiatives (page 116 and Appendix 9). The rule of a half is also referred to in the Infrastructure Australia, Assessment Framework – Guide to economic appraisal in relation to valuing induced demand (page 41).

Alignment with Commonwealth Government requirements (where applicable)

Where the Project Team is considering Commonwealth Government funding, the Project Team should ensure that the Economic Appraisal methodology and outputs also align with Commonwealth Government requirements. These Guidelines have been developed in a way that is broadly consistent with Commonwealth Government guidelines on Economic Appraisals. However, the Project Team should also consider the links to several relevant documents and the Commonwealth Government guidelines that are provided in Appendix A.

Step 3: Report economic results (Tier 1 and Tier 2)

BCR and NPV

The benefit to cost ratio (BCR) and net present value (NPV) are the key measures of the Economic Appraisal that compare benefits with costs. The Project Team should present a BCR and NPV for each Project Case in the Economic Appraisal.

The BCR is the ratio of the present value of benefits to the present value of costs. It is calculated as follows:

A BCR equal to or greater than 1 indicates that the project is likely to generate benefits (in monetary terms) that exceed its costs over the appraisal period.

The NPV is the difference between the present value of total benefits and the present value of total costs. It is calculated as follows:

The Project Team should present the NPV in real terms in the base year for the appraisal, generally in $ millions. A positive NPV indicates that the project is likely to generate benefits (in monetary terms) that exceed its costs over the appraisal period. The Project Team should present the BCR and NPV both with and without WEBs (if calculated).

The above notwithstanding, any conclusion about the economic viability of a project will also need to consider any non-monetised benefits and economic costs.

MCA

The Project Team should use the outputs of the Wellbeing Impact Assessment to inform the MCA.

For further information on what an MCA involves and how to undertake one, refer to the MCA Guidelines.

Distributional analysis

The distributional analysis is a discussion of the relative impacts of the project’s benefits and economic costs on different groups in the community. Where possible, the distributional analysis should quantify the impacts of the project’s benefits and economic costs on these different groups. The Project Team should also outline the key ‘winners and losers’ from the project. The different groups of the community may include consumers, businesses, workers and governments, as well as individuals or households by income bands, locations, gender, ethnicity, etc.

The Project Team should also indicate whether the Economic Appraisal results, reflected in the BCR, NPV and MCA, are likely to underestimate or overestimate the impact of the project on any particular group (for example, by not quantifying and monetising a certain benefit or cost that disproportionately affects a particular group in the community).

The distributional analysis should form part of the decision-making process for weighing up various project options, and may also be used to inform decision making for other investments that may complement or support the project being assessed.

Sensitivity analysis

Sensitivity testing of the results of the CBA is a key element of the Economic Appraisal. The purpose of the sensitivity analysis is to acknowledge that there is always a degree of uncertainty and ultimately risk surrounding the CBA results for a project, and to test the impacts of a number of scenarios on the core result. Typically, there are five sources of uncertainty surrounding a project:

• Capital costs
• Construction duration and therefore the date for commencement of operations
• Operating (including maintenance) costs and any revenues
• Inputs, parameters and assumptions underpinning key benefits (and economic costs) and project costs (demand for a service, values placed on the service, the social cost of carbon emissions, etc.)
• Uncertainty around the key appraisal assumptions (including the discount rate and appraisal period).

The Project Team should undertake sensitivity testing on benefit and cost items that reflect the sources of uncertainty above, as well as on key parameters or assumptions that are used in the analysis (including the discount rate). The Project Team may utilise the probability distribution of costs and benefits, where available, to determine the sensitivities that it will analyse.

The Project Team should report the results of the sensitivity tests in the Business Case along with the core results, with commentary on the extent to which the sensitivities affect the core results.