The evaluation cycle uses the principles and values that underpin the development of any community – those of Learning, Evaluation and Planning (LEAP), which are often summarised as: analyse – plan – do – review. The evaluation process is cyclical – positive results can be built on, and less positive results can be analysed to determine what went wrong, what can be done to remedy the situation – and what can be done to improve the results of similar applications in the future. This cycle is illustrated below and shows that evaluation is a key part of the process of implementing ITS. It is not an optional ‘extra’. The diagram shows how monitoring and evaluation can provide both short-term feedback to improve current operations and also results that will feed into the ITS development and investment strategy.
After deployment, post-implementation monitoring and evaluation of an ITS scheme is used to assess whether the system – as installed and operated – is meeting project objectives, delivering the expected performance and matching user requirements (which themselves may change over time). ‘User needs’ are not necessarily those of the driver of a vehicle. They might be the needs of the road authority or network operator to improve system performance and deliver better safety or traffic flow, the needs of the environment, the needs of the wider integrated transport network – or the needs of the communities affected by other people’s travel.
The results of post-implementation analysis should be fed into the evaluation cycle, improving operations and monitoring – and influencing ITS strategies for the future. The feedback provided by the evaluation informs future investments and their design. If the ITS is not performing as expected, feedback can help in understanding how to adjust or adapt the scheme.
Automatic Fare Collection, Turkey
In Izmir, Turkey, the information generated by the automatic fare collection and real time bus information system is being used to re-design the route network, with planned interchanges and improved service quality. It has led to significant increases in passenger numbers. See World Bank Case Study: Izmir, Turkey
Evaluation is significant for any ITS deployment, but especially for a scheme which is innovative or deployed into a new situation. It shows the contribution that the scheme can make to meeting transport policy and everyday operational objectives, and can be used to improve and fine tune ITS operations –providing feedback for subsequent rounds of deployment. Without an adequate evaluation, organisations cannot be certain that they have obtained the properly functioning scheme for which they have paid. Evaluation results can also be used to inform future ITS policy and strategy. (See Improving Performance)
Another use of feedback from ITS evaluation is to improve the process of appraisal prior to deployment of a future scheme. (See Project Appraisal) The results of real life evaluation impacts – which include the costs of building, maintaining and operating ITS schemes and the benefits achieved – can be used in future investment decisions, to make sure that systems selected meet specific user requirements.
Road network operators have an important role throughout the evaluation cycle. They are involved in developing the operating strategies for their networks, identifying ITS project objectives, pre-implementation analysis to inform investment decisions, scheme selection and analysis of user requirements. As scheme ‘owners’ they gather monitoring data during implementations of ITS. Increasingly, ITS includes features that provide the basis for monitoring patterns of traffic demand and the behaviour of road users. (See Network Monitoring) Road network monitoring data is used by operators for pre-implementation analysis and for monitoring traffic, safety, incidents and other performance criteria on the network – after an ITS implementation has been deployed. It provides vital input to the evaluation. Road network operators will often commission independent evaluation and use the results to feed back into future strategies and investment decisions. (See Operational Activities)