Piloting, Feedback and Monitoring

Even apparently simple tasks such as administering a questionnaire should be piloted. This is because the design of the questionnaire (and the management processes around the questionnaire) may be found deficient or ambiguous when exposed to actual users. Piloting is really important and should not be missed out.

Pilot studies should be well designed with clear objectives, clear plans for collecting and analysing results, and explicit criteria for determining success or failure. Pilot studies should be analysed in the same way as full scale deployments.

In general, the benefits of a pilot study can be identified under four broad headings:

• process - this confirms (or not) the feasibility of implementation of the full ITS deployment (for example including user uptake and data processing capability)
• resources - this helps evaluate time and budget issues in full deployment such as the power and communication costs associated with new hardware
• management - this covers potential issues including personnel requirements and data security
• technical - this provides initial evidence of technical system performance but also, and more importantly, that the outcomes (such as safety, congestion reduction, information provision) are as intended and that behavioural changes by road users are as expected

Monitoring the use of ITS is important because users may not react as anticipated and their behaviour may adapt and change over time. Since Road Operators are particularly concerned about changes in behaviour that may decrease safety, any (so called) “risk compensation” needs to be identified. This occurs when changes, such as the introduction of ITS, makes users feel safer and they respond (possibly unconsciously) by adopting riskier behaviours.

Monitoring the use of ITS can take many forms. Some examples are:

• the number of drivers diverting from a route in response to a Variable Message Sign warning of congestion ahead
• the percentage of drivers using a hand-held mobile phone
• the percentage of drivers opting to use free-flow tolling tags (rather than paying cash)
• the time taken for a control room operative to implement a particular road closure measure
• usage by public transport users of a free bus travel planning application

Feedback is information that comes directly from users of ITS about the satisfaction or dissatisfaction they feel with the product or service. Feedback can lead to early identification of problems and to improvements.

When the ITS users are “internal”, such as traffic control room staff or on-site maintenance workers – encouraging feedback has to be addressed as part of the organisational culture. Ideally a “no blame” culture will exist that allows free expression about what works well and does not when ITS is incorporated within wider social and organisational settings. Some industries have an anonymous feedback channel to allow comment on systems and operations. Explicit and overt mechanisms to respond to feedback help encourage further contributions from ITS users.

Feedback from road users about ITS has to be carefully interpreted as it may relate to the wider transport system of which ITS is just the visible part. For example, a complaint about the setting on variable speed limit signs may arise because information about incident clearance is not speedily transmitted to a traffic control centre.

Many organisations publish a service level promise or “customer charter” and this may include feedback mechanisms. Road Operators may choose to implement feedback channels that are passive (such as publishing address/phone/email/web address) or adopt more active mechanisms (such as questionnaires and surveys).