Workshop Papers Downloads
- Keynote presentation
Using Passive Brain-Computer Interfaces for Cognitive Workload Assessment during Learning: A Novel Methodological Approach
Peter Gerjets - Inferring
Cognitive States from Multimodal Measures in Information Science
Jacek Gwizdka and Michael J. Cole [Presentation slides] - Gesture
Dynamics: Features Sensitive to Task Difficulty and Correlated with Physiological Sensors
Lisa Anthony, Patrick Carrington, Peng Chu, Christopher Kidd, Jianwei Lai, Andrew Sears [Presentation slides] - Classification
of Cognitive Load from Task Performance & Multichannel Physiology during Affective Changes
Sazzad Hussain, Siyuan Chen, Rafael A. Calvo and Fang Chen [Presentation slides] - Cognitive Load
Measurement with Pen Orientation and Pressure
Kun Yu, Julien Epps and Fang Chen [Presentation slides] - Facial Response
to Video Content in Depression
Gordon McIntyre, Roland Goecke, Michael Breakspear and Gordon Parker [Presentation slides] - Fusing
Utterance-Level Classifiers for Robust Intoxication Recognition from Speech
Felix Weninger and Björn Schuller
Work in progress:
- An Immersive View
Control Method Using EMG Signals of Users' Eyelid Movements
Masaki Omata, Satoshi Kagoshima, Atsumi Imamiya, Xiaoyang Mao - Handling Noise in
Audio-Visual Emotion Recognition
Ntombikayise Banda and Peter Robinson - Assessment of the
Emotional State by Psycho-physiological and Implicit Measurement
Michael A. Bedek, Ben Cowley, Paul Seitlinger, Martino Fantato, Simone Kopeinik, Dietrich Albert and Niklas Ravaja
Background
Multimodal, as well as unimodal, interactive data from behavioural signals such as speech, facial expression, mouse and pen input has recently been shown to provide indications as to the cognitive and emotional state of the user. Similarly, research into physiological signals such as electroencephalograph (EEG), galvanic skin response (GSR), electromyograph (EMG), heart rate, respiration and eye-movement detected by a camera-based tracker has demonstrated differences for contrasting cognitive states, e.g. the completion of complex or simple tasks. By studying individual physiological, interactive and other behavioural features, and also their fusion/integration, new holistic insights can be provided into the physical manifestations of various cognitive and emotional states - including affect, stress and workload factors.
Scope and Expected Impact
The use of multiple sensors in human-computer interaction (HCI) systems allows the collection of a wide variety of datasets. This, together with an array of machine learning techniques can shed light on implicit behavioural measures, and the kinds of cognitive and emotional states that can give rise to these signal patterns. The amount of data that is being collected now makes this challenge a very topical one. There are no standards by which multimodal data can be collected and labelled (interactive or otherwise), and there are no standards by which multimodal data can be induced and therefore interpreted. This full-day workshop will provide a unique opportunity to define and shape the future research agenda in unimodal/multimodal cognitive and emotional inference, as a community, acting as a focal point and stimulus for significant new research and collaboration. The development of systems based on these research insights can be expected to have a dramatic effect on future interfaces, opening up a new world of interaction and joint human-computer collaboration that is optimised to exploit the unique capabilities of each. Other related avenues of investigation include exploring the relative merits of different sensors, and correlations between features extracted from different sensor signals.
Guest Speakers
- Prof. Sharon Oviatt (Incaa Designs, USA)
- Sharon Oviatt currently is President of Incaa Designs, a nonprofit 501(c)3 organization that researches, designs, and evaluates new educational interfaces. Her research focuses on human-centered design and cognitive modeling, communication technologies, spoken language, pen-based and multimodal interfaces, mobile and educational interfaces. Examples of recent work involve the development of novel design concepts for educational interfaces (science and math), robust interfaces for real-world mobile and field environments, adaptive conversational interfaces with animated software characters, digital paper and pen-based interfaces for individuals and groups, and lifespan interfaces for children through the elderly. She has published over 140 scientific articles in a wide range of venues, including work featured in recent special issues of Communications of the ACM, Human Computer Interaction, Transactions on Human Computer Interaction, IEEE Multimedia, Proceedings of IEEE, and IEEE Transactions on Neural Networks. She was General Chair of the International Conference on Multimodal Interfaces (ICMI) in 2003, and is Founding Chair of ICMI's Advisory Board. In 2000, she received an NSF Special Extension for Creativity Award for pioneering work on mobile multimodal interfaces.
- Prof. Peter Gerjets (KMRC, Germany)
- Prof. Gerjets is currently head of the Hypermedia Lab, at the Knowledge Media Research Center in Tübingen. Since 2002 Peter Gerjets has been working there as principal research scientist, and as full professor for research on learning and instruction at the University of Tuebingen. Before taking over his current position at the University of Tuebingen, Peter Gerjets has been working as Assistant Professor at the Saarland University in Saarbruecken. He was honored with the Young Scientist Award of the German Cognitive Science Society in 1999. From 1997 to 1998 Peter Gerjets was granted a postdoctoral scholarship of the DFG-Graduate College for Cognitive Science in Saarbruecken. From 1991 to 1995 he was a Research Associate at the University of Goettingen. Peter Gerjets received his PD (Habilitation) from the Saarland University in 2002 with a thesis on "Cognitive control in hypertext-based learning and problem solving: Goal management, strategy selection, and adaptive information utilization". In 1994 he received his Ph.D. (Dr. rer. nat.) from the University of Goettingen where he also received his diploma in psychology in 1991. His research interests are related to earner control, cognitive load, adaptive user strategies, learning with dynamic visualizations, designing multimedia learning environments and supporting different learners' competences to use such environments.
Topics of Interest
- Automatic recognition of cognitive load, stress and emotions via unimodal or multimodal signals
- New methods and frameworks for the fusion and integration of multimodal signals
- Methodology for eliciting cognitive and emotional states and design of user experiments
- Dimensional approach to the estimation of cognitive and emotional states
- Novel applications of interfaces utilising cognitive and emotional state inference capabilities
- New sensors and modalities demonstrated to provide cognitive or emotion inference properties
Paper Submission
The workshop solicits original and unpublished papers that address a wide range of issues concerning, but not limiting to the list of topics. Submissions must be sent in PDF to the following email address: MMCogEmS2011@nicta.com.au. Authors should submit papers not exceeding 4 pages in total and in the ACM format (http://www.acm.org/sigs/pubs/proceed/template.html).
Accepted papers will be presented at the workshop in either oral or poster format and will appear in the ICMI conference proceedings. Note that accepted papers will not be automatically published in the ACM Digital Library, but selected authors will be invited to submit an extended version of their papers to a journal special issue. At least one author of each paper must register the ICMI conference and attend the workshop to present the paper. Please refer to the ICMI 2011 website for more information about registration.
Organisers
- Fang Chen (NICTA, Australia)
- Julien Epps (University of New South Wales, Australia)
- Natalie Ruiz (NICTA, Australia)
- Eric Choi (NICTA, Australia)
Program Committee
- Eliathamby Ambikairajah (University of New South Wales, Australia)
- Bert Arnrich (ETH Zurich, Switzerland)
- Carlos Busso (University of Texas (Dallas), USA)
- Rafael Calvo (University of Sydney, Australia)
- Paul Corballis (University of Auckland, New Zealand)
- Henry Gardner (Australian National University, Australia)
- Roland Goecke (University of Canberra, Australia)
- Shamsi Iqbal (Microsoft Research, USA)
- Jongwha Kim (University of Augsburg, Germany)
- Sungbok Lee (University of Southern California, USA)
- Nigel Lovell (University of New South Wales, Australia)
- Saturnino Luz (Trinity College Dublin, Ireland)
- Bjoern Schuller (TUM, Germany)
- Andrew Sears (Rochester Institute of Technology, USA)
- Jianhua Tao (Chinese Academy of Sciences (Beijing), P. R. China)
Acknowledgements
We thank the Air Force Office of Scientific Research for its support of this workshop.