SPRING 2000 COURSE, CSE 597K:
MULTIMODAL
HUMAN-COMPUTER INTERACTION
3 credit, Meeting hours flexible,
Schedule number 706506, Limited Enrollment.
Prerequisite: consent of instructor
MEETING TIMES -> Wed, 4:35 pm in
room 302, POND LAB
(sometimes Mon, 4:35 in same room)
Lectures
planned, progressive notes, references etc.
PROJECTS
Course description:
Recent advances in various
signal processing technologies coupled with an explosion in the available
computing power has given rise to a number of novel human-computer interaction
(HCI) modalities - speech, vision-based gesture recognition, eye tracking,
EEG, etc. Successful embodiment of these modalities into an interface has
the potential of easing the HCI bottleneck that has become noticeable with
the advances in computing and communication. It has also become increasingly
evident that the difficulties encountered in the analysis and interpretation
of individual sensing modalities may be overcome by integrating them into
a multimodal human-computer interface.
In this course we will
examine research issues and recent progress in ``Multimodal Human-Computer
Interaction'', i.e. use of more than one modalities for building a novel
human-computer interface. We will consider some of the emerging novel
input modalities for HCI and the fundamental issues in integrating them
at various levels - from early ``signal'' level, intermediate ``feature''
level, to late ``decision'' level. We will discuss the different computational
approaches that may be applied at the different levels of modality integration.
We will review several demonstrated multimodal HCI systems and applications,
e.g. iMAP and QUICKSET. Despite all the recent developments, it is clear
that further research is needed for interpreting and fusing multiple sensing
modalities in the context of human-computer dialog.
The goal of this course
is prepare the students for undertaking research in this exciting, emerging
area.
Grading and Modus Operandi:
The course grade will
be based upon class participation and project.
Expected class participation
includes:
-
Presenting
one (or two, depending on schedule) paper in class
-
Reading
each paper that is presented in class (before the start of the class)
-
Critiquing
and actively participating in the discussion about the paper in class
A semester-long class project that would include a programming
assignment that is closely matched with the individual's interests in the
general area of multimodal human-computer interaction. The successful completion
of the class project would include the following stages:
-
Project proposal: a short (1-2 pages)
abstract describing the goals and steps in the proposed project
-
Mid-term report: a short progress report
-
Class presentation: about 20 minutes
presentation of the main challenges faced and results of the project
-
Final Report: Describing the implmentation
clearly and documented programs that were developed.
For more information, contact the instructor:
Rajeev Sharma, Email him: rsharma@cse.psu.edu,
or drop by his office: 317 Pond Lab,
or call him:
863-0147.
More information
about Rajeev Sharma's research can be obtained from his home page