ABSTRACT
Indexed by Track Subject
Indexed by Title
Indexed by Author
 Multimodal Information Systems and GIS: The Informedia Digital Video Library
Track:  New Technology and Technology Integration
Author:   Andreas M. Olligschlaeger
Alexander Hauptmann
Modern information systems are becoming increasingly multimodal (i.e., information retrieval and display has to occur across a variety of sources and data formats such as the World Wide Web, audio, video satellite imagery, and relational database systems). One of the challenges in GIS is to create the ability to automatically geocode multimodal data in order to allow information retrieval and display via map queries. This presentation discusses two methodologies used in the Informedia project that accomplish this. The first uses GPS to encode latitude and longitude coordinates directly onto video. The GPS information is then processed and stored in a relational database system and synchronized with the video, allowing for real-time simultaneous display of video and map information. In other words, users can watch the location where the video was taken in real time, as well as query a map for all video that was filmed in a particular area. The second methodology uses natural language processing and entity extraction techniques to automatically geocode transcribed or speech-recognized text from news broadcasts. The geocoded location information is also synchronized with the video. During video playback, maps are animated, in that places mentioned during the video are highlighted as they are mentioned during video playback. In addition it is possible to query the map for all news broadcasts that mention certain places or regions. The two techniques described above will be demonstrated using the Informedia Digital Video Library developed at Carnegie Mellon University's School of Computer Science. The GIS portion of the project was developed using Esri's MapObjects and fully integrated into the Informedia client application.

Andreas M. Olligschlaeger
Carnegie Mellon University
5000 Forbes Ave.
Pittsburgh, PA15213

Telephone: (724) 251-9029
Fax: