Combining Auditory and Visual Menus

Despite growing interest in touch screen and gesture interfaces for auditory menus, there usually is lacking interoperability between visual and auditory menus. The same control logic for both visual and auditory domains could facilitate switching to eyes-free use when needed and improve accessibility for visually impaired users. This paper presents an efficient control interface for both domains and usability tests with three interaction methods. Results show that auditory and visual menus with the same control logic can provide a fast and usable interface to control devices. Furthermore, the same auditory menu can be accessed with a gesture interface. Overall, the touch screen interaction with a visual display was fastest, the touch screen interaction with auditory display was almost as fast, while the gesture interface with an auditory display was slowest. The novel interface paradigm is explained by an example application that allows eyes-free touch screen and gesture access to a music collection on a mobile phone.