Multi-Criteria Analysis

Purpose of this section

The purpose of this section is to provide guidance to the Project Team on:

  • Why Multi-Criteria Analysis (MCA) should be used
  • When to use MCA within the Capital Framework
  • How to apply MCA effectively
  • How to sufficiently document the results of MCA.

Objectives of Multi-Criteria Analysis

MCA is a decision-making tool that uses a ‘non-monetary’ evaluation to compare options in a structured and transparent manner. The purpose of MCA is to differentiate and evaluate options systematically using a set of agreed assessment (or evaluation) criteria. MCA often is used to support Cost Benefit Analysis (CBA) when some or all of the project benefits are not able to be monetised.

Using Multi-Criteria Analysis in the Capital Framework

Within the Capital Framework, the Project Team should use MCA to:

The Project Team may also elect to use MCA to undertake an initial comparison or filtering of a longlist of project options to identify a shortlist of project options.

The Project Team may refer to the supplementary technical guidelines in Appendix A for further information on undertaking MCA.

MCA is also a useful tool in evaluating tender responses under Integrated delivery models, which are complex and include multiple components that contribute to their overall value for money. This is explained further in the Guidelines for Public Private Partnerships.

Risks

MCA provides an adaptable, cost-effective and relatively simple tool for comparing alternative options or tender responses on a like-for-like basis in a transparent manner. In addition, use of an MCA within an Economic Appraisal ensures that important social, wellbeing and environmental benefits that are difficult to monetise can be considered in decision-making.

However, there are a few risks that the Project Team should consider and manage. These are:

  • Subjectivity: MCA relies on the informed judgement of the person, team or organisation applying the tool in identifying evaluation criteria, deciding on their relative importance and, where criteria are unable to be rated quantitatively, rating options qualitatively. Therefore, there is a level of subjectivity within MCA. To mitigate this subjectivity, the Project Team should:
    • Involve key stakeholders who have the required knowledge and experiences, to agree and confirm the evaluation criteria and the MCA evaluation process
    • Ensure there is robust reasoning and documentation of the MCA process
    • Select evaluation criteria that are as specific as possible and are linked directly to the project objectives or vision
    • Rate evaluation criteria using evidence-based and quantifiable measures, where possible, or using enough qualitative analysis so that each option can be ranked appropriately and consistently.
  • Assessing viability: It is difficult for a Project Team to consider adequately the scale of benefits relative to costs within MCA, though this is fundamental to the viability of any project. To mitigate this difficulty, the Project Team should consider evaluation criteria that evaluate both the benefits and the costs of the project options, quantifying them objectively when possible.
  • Double counting: There is a risk that the evaluation criteria are not mutually exclusive, resulting in ‘double counting’ (evaluation of a characteristic of a project option under more than one criterion). Double counting may lead to an inflated or deflated ranking of options. To manage double counting, the Project Team should avoid using similar data to inform the rating of differing evaluation criteria. Additionally, where the MCA is supplementing a CBA, the Project Team should also be careful not to include evaluation criteria that may be assessed through the CBA, so as not to double-count benefits.

The Project Team may refer to the supplementary technical guidelines in Appendix A for further information on MCA’s strengths, limitations and risks.

Footnotes: