Road Network Operations
& Intelligent Transport Systems
A guide for practitioners!
Road Network Operations
& Intelligent Transport Systems
A guide for practitioners!
Indicators for measuring the success of an ITS scheme are directly linked to the objectives that it is aiming to achieve. ITS, like any other investment, should contribute towards solving a problem or delivering a vision. The measure of success is how far the problem is solved or the vision delivered within the available budget. It is important to define indicators which can be used to measure whether the ITS scheme has met its objectives.
The template above for Critical Factors Analysis shows how measures (or indicators) are linked to goals. The same approach can be used for establishing indicators to evaluate the extent to which objectives are met. It provides a systematic way of expressing the outcomes being sought and the visible results or measures expected – if the ITS has the desired effect. These outcomes will not necessarily be entirely within the public sector, as the example above – of efficiency measures, shows. An evaluation may include indicators measuring the extent to which private sector objectives are met, as well as the extent to which public sector objectives are met.
This approach can be applied to the full range of objectives for ITS. In addition to those illustrated above, safety improvements might include benefits – such as, perceptions of personal safety. Environmental benefits will often include noise reduction and might include community benefits – such as a more pleasant neighbourhood. Efficiency benefits can go beyond the supply-side measure of traffic flow or throughput – to efficiency gains for industry from access to information, knowledge and control over fleet movements. In each of these cases, it is possible to define a ‘goal area’ or ‘outcome’ and to agree the ‘measures’ by which progress can be monitored.
At the more detailed evaluation planning stage, this sort of table can also be used to summarise the sources of data which will be used to provide evidence on the indictors. (See Monitoring)
Where possible, indicators should be based on commonly used measures, so that evaluation results can be compared between schemes. For example network performance may be measured using indicators such as change in journey times at peak hours and change in variability of travel time. If there are agreed definitions of these at a national level – or within a road authority – it will be easy to compare the outcomes of different schemes aimed at improving network performance.
Indicators must be defined in quantifiable terms. In evaluating ITS, many of the indicators used are quantified measures. For example, an indicator for the impact on safety is the number of road accident fatalities – which can be measured from accident records. Another important indicator may be the impact on users’ perceptions of safety. This is a qualitative measure but it can be quantified in various ways. Examples include: asking users to compare safety before and after using a 5-point/7-point rating system – and measuring the proportion of users who say they think a system is safe, or measuring the proportion who say they think safety has improved.
It is helpful to review other evaluation projects when deciding on the indicators to be used. They may help identify common indicators that have been used nationally or even internationally in comparing scheme results. They will also help inform which indicators are most suitable for measuring performance against different types of objective – and potential sources of data for each indicator.
A critique of an evaluation of transport performance in Canada provides an example of some of the pitfalls involved in defining and using indicators. Of the 23 indicators used, a few were found to be appropriate and widely used, some were found to be ambiguous and biased – and others were found to be illogical. The ways in which comparisons were made between areas was considered inappropriate due to geographic, demographic and economic differences.
Evaluation guidelines are also a useful source of indicators:
Appendix 1 of these guidelines proposes performance indicators. Those considered particularly useful are highlighted – and suggestions for indicators aimed primarily at private sector and public sector objectives are identified. See EU ITS Portal
Indicators for evaluation include:
This project developed performance indicators for evaluating traffic efficiency, traffic safety, emissions reduction, social inclusion and land use. The indicators are targeted at urban schemes and were developed in consultation with 16 cities across Europe. The project report discusses the principles behind performance measures and the process of developing them. It presents an evaluation framework and provides guidelines for applying it.
The UK Department for Transport’ guidance on transport analysis includes detailed information on indicators for scheme costs and user and provider impacts.
The benefits and costs databases provide information on the indicators which have been used to evaluate a wide range of ITS deployments .
The case studies in the UK’s Department for Transport’s ITS Toolkit include examples of indicators used to evaluate schemes.
Print