The Usability of Deployed Telephone Dialog Systems: An Evaluation

By Kerstin Klöckner and Anthony Jameson (2006)

Proceedings of the 2006 IASTED International Conference on Computational Intelligence, San Francisco.

Abstract

A large amount of research has been conducted on ways of making spoken dialog systems usable, and many guidelines have been formulated. This paper presents a complementary perspective on spoken dialog systems by focusing not on research prototypes but on a broad sample of 35 deployed dialog systems, of which a comparative evaluation was conducted in the context of the 2005 Voice Award competition for German-language telephone-based dialog systems. Considering in turn each of several evaluation dimensions, we discuss the main recurrent usability problems and some of the design solutions that have been used to combat such problems. One general conclusion is that many of the observed problems could principle have been avoided with the help of standard guidelines and technology.