Assessing the Usability of a Dialogue Management System Designed in the Framework of a Rapid Dialogue Prototyping Methodology

Authors: Rajman, Martin; Bui, Trung H.; Rajman, Andréa; Seydoux, Florian; Trutnev, Alex; Quarteroni, Silvia

Source: Acta Acustica united with Acustica, Volume 90, Number 6, November/December 2004 , pp. 1096-1111(16)

Buy & download fulltext article:

Price: $33.00 plus tax (Refund Policy)

Abstract:

The goal of this contribution is to present how the notion of dialogue management evaluation was integrated in the rapid dialogue prototyping methodology (RDPM), designed and experimented by the authors in the framework of the InfoVox project. We first describe the proposed RDPM. The general idea of this methodology is to produce, for any given application, a quickly deployable dialogue-driven interface and to improve this interface through an iterative process based on Wizard-of-Oz experiments (i.e. dialogue simulations). More precisely, the methodology is decomposed into five consecutive steps, namely: (1) producing a task model for the targeted application; (2) automatically deriving an initial dialogue model (and the associated dialogue-driven interface) from the produced task model; (3) using the generated interface to carry out Wizard-of-Oz experiments to improve the initial dialogue model; (4) carrying out an internal field test to further refine the dialogue model; and (5) carrying out an external field test to evaluate the final dialogue model and dialogue-driven interface. In a second part, we more specifically focus on evaluation. We describe the integration of various evaluation aspects in the design steps corresponding to the internal and external field tests. The internal field test involved 20 "internal" users (typically the designers of the system, other members of the involved laboratories, willing relatives, etc.) and was used to further adapt the prototype. The various modifications made were validated during a second internal field test with a reduced group of seven users. The external field test involved a larger population of 50 "external" users (i.e. users that did not have any a priori knowledge of the system), randomly selected in all French speaking Swiss cantons. Its main goal was the final evaluation of the produced dialogue model, dialogue-driven interface, and evaluation procedure. Both the internal and external field tests were carried out over the phone. Finally, we concentrate on the assessment results obtained during the external field test for the InfoVox prototype. We first present the retrospective predominant modality and correlation analyses that were integrated in the external field test and then show how the obtained evaluation data can also be used in a more prospective way to derive useful hints for a better dialogue design (prospective correlation analysis). All the presented concepts are systematically illustrated by examples taken from the experiments that were carried out in the InfoVox project.

Document Type: Research article

Publication date: 2004-11-01

More about this publication?

Acta Acustica united with Acustica, published together with the European Acoustics Association (EAA), is an international, peer-reviewed journal on acoustics. It publishes original articles on all subjects in the field of acoustics, such as general linear acoustics, nonlinear acoustics, macrosonics, flow acoustics, atmospheric sound, underwater sound, ultrasonics, physical acoustics, structural acoustics, noise control, active control, environmental noise, building acoustics, room acoustics, acoustic materials, acoustic signal processing, computational and numerical acoustics, hearing, audiology and psychoacoustics, speech, musical acoustics, electroacoustics, auditory quality of systems. It reports on original scientific research in acoustics and on engineering applications. The journal considers scientific papers, technical and applied papers, book reviews, short communications, doctoral thesis abstracts, etc. In irregular intervals also special issues and review articles are published.
Editorial Board
Information for Authors
Subscribe to this Title
Information for Advertisers
Terms & Conditions
ingentaconnect is not responsible for the content or availability of external websites

Assessing the Usability of a Dialogue Management System Designed in the Framework of a Rapid Dialogue Prototyping Methodology

Buy & download fulltext article:

Tools

Key

Text size:

Useful pages