Real-time Vision-Based Camera Tracking for Augmented Reality Applications

Dieter Koller , Gudrun Klinker, Eric Rose, David Breen, Ross Whitaker, and Mihran Tuceryan

In Proceedings of the Symposium on Virtual Reality Software and Technology (VRST-97), Lausanne, Switzerland, September 15-17, 1997, pp. 87-94.


Augmented reality deals with the problem of dynamically augmenting or enhancing (images or live video of) the real world with computer generated data (e.g., graphics of virtual objects). This poses two major problems: (a) determining the precise alignment of real and virtual coordinate frames for overlay, and (b) capturing the 3D environment including camera and object motions. The latter is important for interactive augmented reality applications where users can interact with both real and virtual objects. Here we address the problem of accurately tracking the 3D motion of a monocular camera in a known 3D environment and dynamically estimating the 3D camera location. We utilize fully automated landmark-based camera calibration to initialize the motion estimation and employ extended Kalman filter techniques to track landmarks and to estimate the camera location. The implementation of our approach has been proven to be efficient and robust and our system successfully tracks in real-time at approximately 10~Hz.

The document is available online as zipped postscript (1555838 Bytes) or PDF file (3170422 Bytes).

Last modified on Thursday, June 12, 1997 Dieter Koller (