Reporting the Results

Principles

The following key principles are useful to follow in reporting the results of monitoring and evaluation.

Audience

Consider the audience for the results. Separate reports may be useful for different stakeholders. Policy makers may require an overview of results and a summary of the implications. Practitioners may benefit from a more detailed technical report. It may be appropriate to develop an information leaflet, web page or news article to summarise some of the main impacts of the ITS for the general public.

Transparency

Make sure that the results are reported in a transparent way, easy to understand and easy to compare with other studies. Include details of the statistical confidence in quantitative results – where this is appropriate. Provide background supporting information about the context of the scheme (perhaps include references to other sources for further background information) to help users transfer the results to other contexts.

Balance

Include both positive and negative impacts to provide a balanced report. Ensure that both qualitative and quantitative impacts are reported – and avoid giving the impression that more value is placed on indicators than can be readily be quantified. For example, some impacts can be converted into monetary values and combined with others to provide an overall ratio of costs to benefits. This will not be possible for all indicators – but those that cannot be measured in this way should also be reported. Frameworks such as ‘planning balance sheet’ and ‘multi-criteria analysis’ can be used to summarise results.

Dissemination

Make the results widely available so that decision makers and practitioners elsewhere can learn from them. This will help to build a knowledge base for evaluating ITS and help to reduce the time and effort involved in planning, implementing and evaluating other schemes.

Results can be published through journals, presentations at conferences, webinars, or by submitting reports to web sites that publish case study. Potential platforms include the:

• journal ‘ITS International’ – which includes a regular feature on evaluation results from around the world
• US DOT database of benefits costs and lessons learned, synthesises reports from ITS implementations in the US and elsewhere
• European ELTIS database – provides case studies of results from urban transport schemes
• European Transport Information and Research Portal (TRIP) - synthesises results from research projects across Europe

Content

The following topics are useful to cover in any technical report of evaluation results (which may also include further information as appendices:

• overview or Executive Summary
• description of the problem
• description of the ITS
• description of the evaluation
• results: technical performance, user acceptance, impact assessment, financial assessment
• comparisons with experiences of similar ITS – if possible in similar contexts
• transferability of results: local issues and factors which may affect whether similar results can be achieved in other schemes

Sources of guidance on evaluation planning and reporting.

There are several sources of guidance on writing evaluation plans and reporting results:

• the UK Department for Transport’s ITS Toolkit includes advice on planning evaluation and reporting results in the section on ‘Feeding back results’
• the EasyWay Handbook on Evaluation Best Practice includes guidance on planning evaluation with an evaluation checklist, relevant impacts and indicators for different ITS services, recommended indicators and a structure for reporting results. See EU ITS Portal
• the EasyWay Project Evaluation Guidelines for ITS on European road networks include guidance on planning and conducting monitoring and evaluation, with details of how the recommended process has been applied in a specific project – and a diagram summarising the approach to the evaluation. See EU ITS Portal
• in Finland, the guidelines for the evaluation of ITS projects provide guidance on different types of indicator and evaluation methods. It includes four summary evaluation reports and describes the stages involved in applying the approach
• in the Netherlands, the guidelines for ITS evaluation include an outline of the contents of an evaluation plan
• a collection of papers from the Victoria Transport Policy Institute in Canada provides examples and guidance on evaluation in transport (not specifically ITS)
• the World Bank’s guidance on evaluation includes planning the evaluation, developing performance indicators and reporting results. It includes evaluation questions on institutional issues such as governance, partnerships and financing