Objective:

Given the current trend towards miniaturization of electronics, it is feasible that in the near future a person will easily carry any number of powerful electronic devices on his or her body. The availability of powerful computers in increasingly small packages will allow algorithms in computer vision - a field that has traditionally been constrained to high-end desktops - to run on laptops and handhelds.

Consequently, advances in computer vision will soon be able to contribute to a traditionally "low-tech" field : assistive devices for the blind. The goal of this project is to develop a system that extracts information from a moving camera and presents important or especially salient visual features in alternative sensory modalities.

Using a laptop, digital video camera, and a haptic display we aim to create a system that captures images and renders key features via haptic and auditory feedback. We have several specific feedback mechanisms that we hope to implement; if they are successful, we will explore other features that are easily computed in real-time:

• The horizontal imbalance of optic flow can be represented as a sound; either the pitch or left/right pan of the sound will likely be used to represent the degree of optic flow mismatch. This will hopefully be effective as an aid in navigating hallways; it has traditionally been a fairly effective technique for robot navigation.

• The distance between the cameras and any approaching obstacles - obtained via disparity information (if stereo vision is available) or depth-from-motion algorithms (if only a single camera is available) - can also be represented via a sound. Volume seems to be an appropriate representation for depth.

• We hope to allow the user to capture a static picture from the camera, which will be processed by the system to generate a countour map and a depth map. Either of these images will be available to the user via a force-feedback haptic device.

Once the basic architecture is in place, we hope to explore the psychological aspects of the project, with a particular focus on the specific visual features that are most useful to a user and the specific methods of presentation that most effectively convey the relevant visual features.

Group Members:

Dan Morris (agentmorris@gmail.com)

Neel Joshi (neel@stanford.edu)

• Vision algorithms:
• Real-Time Optical Flow (1994) - Ted Camus
• Optical Flow Based Robot Navigation
• Maze Navigation Using Optical Flow (1996) - Andrew P. Duchon
• Google Search with many more similar links
• Real Time Flow sample code

• Image capture architecture:
• Developing Real-Time Computer Vision Applications for Intel Pentium III based Windows NT Workstations
• Microsoft Vision SDK
• IEEE-1394 Digital Camera Windows Driver

• Haptic device architecture:
• Sensable Technologies, developers of the "Phantom" line of haptic devices, for which we will be developing our project
• eTouch, the API we will be using to interface with the Phantom

• Gratuitous picture of the Phantom: