The term ‘evaluation of ITS’ is an assessment of the extent to which an ITS scheme has met its objectives. It provides lessons on improving performance in the future. The main issues to consider are:
The road network operator can play an important role in the evaluation of ITS deployments aimed at supporting Road Network Operations. Positive and negative impacts on network operations should be assessed for any ITS applications that are in the research and development phase or are part of a large-scale field trial. Evaluations of routine deployments of established ITS technologies are also important to establish good practice. For example an evaluation of the use of speed cameras in a completely new context should establish the degree of compliance by road users and whether this has a beneficial effect on traffic efficiency and accidents.
The evaluation is a planned and structured assessment of the impacts of an ITS scheme and the extent to which it has met its objectives. The impacts assessed include the financial costs and negative consequences as well as the benefits. (See ITS Benefits) Evaluation takes place after deployment has been completed, but it is important to plan the evaluation before the deployment takes place and schedule the resources to carry it out. Evaluation is often undertaken by an independent organisation so that the results are seen as a true and unbiased assessment of the scheme. This is particularly so for an ITS scheme that is highly innovative or which has a high public profile, such as the congestion charge scheme in Stockholm.
Monitoring and evaluation are not carried out for their own sake – they are not ends in themselves. ITS projects cover a wide diversity and involve considerable investment of financial and other resources by stakeholder organisations. A formal evaluation is important in order to check that the expected value of an investment has been realised, and to determine who benefits and how those benefits compare with expectations. Evaluation is never simply a matter of “justifying” investment. It provides information which:
Evaluation makes it possible to assess the impacts which the scheme has had on stakeholders (such as travellers and operators) – and on a range of policy objectives such as the environment, safety, sustainability and efficiency. The stakeholder perspective for minority groups who may be disproportionately affected – positively or negatively – will be important, for example people with a disability or vulnerable road users, such as pedestrians and cyclists.
Organisations that fail to undertake full and proper evaluations of their ITS deployments are at a disadvantage when trying to justify proposals for further investments in the future. This applies whether the organisation is a public authority trying to determine the direction of future transport policies, a road operator considering its investment priorities or a commercial organisation that sees new business opportunities. A well-planned and documented evaluation of an existing scheme, helps justify and gain support for the next one.
The type of evaluation to be undertaken is determined by the evaluation objectives. They may include:
Stakeholders involved in commissioning an evaluation of ITS can include policy makers at national or local level, road or transport authorities, transport operators and users, and the organisations directly involved in Road Network Operations. Depending on the scale and complexity of the evaluation - it may be carried out in-house or by specialist consultants or university researchers.
The involvement of Road Network Operators in the evaluation of ITS for Road Network Operations includes:
Road network operators and public road authorities often find it difficult to make a business case for ITS. Evaluation results can support this process. The US DOT’s ePrimer’s Module 12 discusses how to make the business case.
Evaluation of an ITS deployment can help make the business case for investment by:
Public Sources of Evaluation Results
Evaluation results for ITS schemes have been compiled and consolidated in several resources. Some are databases which can be searched for results relating to specific types of scheme or meeting particular objectives. These are invaluable in helping identify expected impacts and potential performance measures. They also demonstrate the advantages of reporting evaluation results using a common framework – to compare studies which may have been carried out in different countries and with different requirements for evaluation and reporting. Sources of ITS evaluation results include: