The Evaluation Cycle

The evaluation cycle uses the principles and values that underpin the development of any community – those of Learning, Evaluation and Planning (LEAP), which are often summarised as: analyse – plan – do – review. The evaluation process is cyclical – positive results can be built on, and less positive results can be analysed to determine what went wrong, what can be done to remedy the situation – and what can be done to improve the results of similar applications in the future. This cycle is illustrated below and shows that evaluation is a key part of the process of implementing ITS. It is not an optional ‘extra’. The diagram shows how monitoring and evaluation can provide both short-term feedback to improve current operations and also results that will feed into the ITS development and investment strategy.

After deployment, post-implementation monitoring and evaluation of an ITS scheme is used to assess whether the system – as installed and operated – is meeting project objectives, delivering the expected performance and matching user requirements (which themselves may change over time). ‘User needs’ are not necessarily those of the driver of a vehicle. They might be the needs of the road authority or network operator to improve system performance and deliver better safety or traffic flow, the needs of the environment, the needs of the wider integrated transport network – or the needs of the communities affected by other people’s travel.

The results of post-implementation analysis should be fed into the evaluation cycle, improving operations and monitoring – and influencing ITS strategies for the future. The feedback provided by the evaluation informs future investments and their design. If the ITS is not performing as expected, feedback can help in understanding how to adjust or adapt the scheme.

A carefully constructed evaluation plan is key to this cyclical process and is an essential part of any programme for developing ITS. (See Evaluation Plan)

Significance

Evaluation is significant for any ITS deployment, but especially for a scheme which is innovative or deployed into a new situation. It shows the contribution that the scheme can make to meeting transport policy and everyday operational objectives, and can be used to improve and fine tune ITS operations –providing feedback for subsequent rounds of deployment. Without an adequate evaluation, organisations cannot be certain that they have obtained the properly functioning scheme for which they have paid. Evaluation results can also be used to inform future ITS policy and strategy. (See Improving Performance)

Pre-implementation analysis

Another use of feedback from ITS evaluation is to improve the process of appraisal prior to deployment of a future scheme. (See Project Appraisal) The results of real life evaluation impacts – which include the costs of building, maintaining and operating ITS schemes and the benefits achieved – can be used in future investment decisions, to make sure that systems selected meet specific user requirements.

Role of Road Network operators

Road network operators have an important role throughout the evaluation cycle. They are involved in developing the operating strategies for their networks, identifying ITS project objectives, pre-implementation analysis to inform investment decisions, scheme selection and analysis of user requirements. As scheme ‘owners’ they gather monitoring data during implementations of ITS. Increasingly, ITS includes features that provide the basis for monitoring patterns of traffic demand and the behaviour of road users. (See Network Monitoring) Road network monitoring data is used by operators for pre-implementation analysis and for monitoring traffic, safety, incidents and other performance criteria on the network – after an ITS implementation has been deployed. It provides vital input to the evaluation. Road network operators will often commission independent evaluation and use the results to feed back into future strategies and investment decisions. (See Operational Activities)