Monitoring Performance

Monitoring in conjunction with evaluation is used to learn lessons and improve future performance. It addresses issues such as: does the ITS meet the objectives and are the outcomes as intended? It can identify the range of benefits achieved which can then be quantified with data from internal monitoring. Careful monitoring of costs helps identify the operational and maintenance costs for a specific service – which can be separated from wider organisational costs.

The performance of the relevant elements of the transport system should be monitored to provide the benchmark against which the added value of the ITS scheme is measured. Performance monitoring can be used to improve operation of the ITS, provide data on the impacts and benefits and demonstrate whether the anticipated benefits and impacts have been realised.

The focus of the monitoring activity is likely to vary with the scale and maturity of ITS deployment. For example, for an area where ITS is relatively new and the number of applications is limited – such as cooperative driving based on Vehicle-to-Vehicle (V2V) communications – monitoring is likely to focus in depth on the performance of individual applications. Where deployment is extensive – such as electronic payment or non-stop tolling – the main focus is likely to be on monitoring the overall presence and performance of ITS in the transport system at a strategic level – in addition to the individual applications. These different approaches have different evaluation requirements.

For example, the US Federal Highway Administration (FHWA) has taken a strategic view of progress with ITS deployment through the use of “a few good measures” that are monitored each year. This is illustrated by its “Intelligent Transportation Systems Benefits": 1999 Update”.

The performance criteria defined in the evaluation plan (See Evaluation Plan) should be monitored both before and after implementation. Depending on the ITS implementation, it may be necessary to monitor ‘control’ sites as well as the implementation site.

Performance monitoring covers the services delivered as well as the technologies used. It involves:

• measuring technical performance against the technical and functional specification to assess whether the supplier has delivered the ITS specified in the contract
• optimising operation of the system
• providing the data to measure the effectiveness of the ITS in achieving the objectives established by the organisations and operators before implementation (an evaluation input)
• providing the data to identify positive and negative impacts, including aspects that were not part of the scope of the scheme – which is also used as an evaluation input

Indicators should be selected which are easy to understand and measure, and are clearly linked to the performance requirements and objectives of the scheme. Where possible they should be common indicators, which can be used across a variety of schemes. (See Indicators)

Where possible, the ITS scheme should be designed so that it automatically provides data for performance monitoring. For example:

• road network operators have network monitoring technologies and applications which support long term monitoring of the network’s performance. ITS can help the network operator to assess how well the network serves the road user. For example, aggregations of individual journey times for road sections are an indication of levels of congestion (See Network Monitoring and Operational Activities)
• fleet operators benefit from in-vehicle and roadside systems to improve operational performance. For example, the automatic vehicle location data in a real time bus information system in Izmir in Turkey is used for fuel monitoring (See World Bank Case Study: Izmir, Turkey)