Tour Guide

Overview

Many visitors attend various functions at The Unviersity of Wisconsin - La Crosse. High school students and parents visit the campus at various times of year, others visit for conferences or sports activities. This project will develop a smart-phone app that will allow a visitor to take video of any scene on campus and have this video overlayed with a) directions to a certain building or b) listen to a narrated description of the visualized scene. This project will use a hybrid strategy of image-similarity and appearance-based recognition strategies to identify scenes.

Appearance-based localization

In appearance-based localization, scenes are not represented explicitly as geometric models, but are represented implicitly as a database of reference images or image-features that are collected at known positions. These images a collected apriori during a training phase and pre-processed for optimized lookups. To identify the location of a person viewing a smartphone image of a scene in the database, the smart-phone image features are extracted and compared against those in the reference database. Localization in this way is an active area of research that involves cross-cutting expertise in image processing, artificial intelligence, and algorithmic design.

Figure 1 shows an image along with a feature-set extracted with the Fab-Map technique. These features are then compared against a database of features collected at some earlier time; such that the location and of the person taking the photo; along with the direction in which the person is looking can be determined with reasonable accuracy.

The Fab-Map technique is robust in the presence of occlusions and dynamic scene changes. Figure 2 illustrates that the algorithm is able to perform a highly confident localization between two sets of images having significantly different actors within the scenes themselves.

The Project : Write an Android App

This project will create an Android app that performs localizaton along with providing narrations and directions to goal locations. The system is expected to be web-based; requiring server interaction to a) store the reference database, b) compute similarity rankings, and c) stream narration or other localization data to the handheld device.