Multimodal Interaction with Mobile Devices: Outline of a Semiotic Framework for Theory and Practice
2006 (English)In: Proceedings of Wireless Networks and Systems 2006, 2006Conference paper (Refereed)
This paper explores how interfaces that fully uses our ability to communicate through the visual, auditory, and tactile senses, may enhance mobile interaction. The first step is to look beyond the desktop. We do not need to reinvent computing, but we need to see that mobile interaction does not benefit from desktop metaphors alone. The next step is to look at what we have at hand, and as we will see, mobile devices are already quite apt for multimodal interaction. The question is how we can coordinate information communicated through several senses in a way that enhances interaction. By mapping information over communication circuit, semiotic representation, and sense applied for interaction; a framework for multimodal interaction is outlined that can offer some guidance to integration. By exemplifying how a wide range of research prototypes fit into the framework today, it is shown how interfaces communicating through several modalities may enhance mobile interaction tomorrow.
Place, publisher, year, edition, pages
Mobile devices, Interaction, Multimodality, Framework, Semiotics
Language Technology (Computational Linguistics)
IdentifiersURN: urn:nbn:se:uu:diva-18045OAI: oai:DiVA.org:uu-18045DiVA: diva2:45816