Machine Vision Based Traffic Surveillance

An output of the machine vision based traffic surveillance system, producing tracks of the detected cars.

Caltrans (California Department of Transportation) is funding a project at UC Berkeley to investigate the applicability of using machine vision for traffic surveillance of california highways. This is a joint project of the Vision Group of Prof. Jitendra Malik for investigating the machine vision components, and the AI Group of Prof. Stuart Russell for symbolic reasoning (e.g. detecting abberant driver behaviour or lane changes) using the raw traffic data extracted from the machine vision component. A major constraint is that this system is supposed to run in (near) real-time in the near future, i.e. we can not use any sophisticated, complex model based approaches, but simple and fast algorithms.

Our current approach is based on 2D contour tracker that attains a level of accuracy and reliability which lies beyond currently available systems. High accuracy and reliability are obtained by using an explicit occlusion reasoning step. For that purpose we employ a contour tracker based on intensity and motion boundaries. The motion of the contour of the vehicles in the image is assumed to be well describable by an affine motion model with a translation and a change in scale. A contour associated to a moving region is initialized using a motion segmentation step which is based on image differencing between an acquired image and a continuously updated background image. A vehicle contour is represented by a closed cubic spline the position and motion of which is estimated along the image sequence. In order to employ linear Kalman Filters we decompose the estimation process in two filters: one for estimating the affine motion parameters and one for estimating the shape of the contours of the vehicles. Occlusion detection is performed by intersecting the depth ordered regions associated to the objects. The intersection is then excluded in the motion and shape estimation. This procedure also improves the shape estimation in case of adjacent objects since occlusion detection is performed on slightly enlarged regions. In this way we obtain robust motion estimates and trajectories for vehicles even in the case of occlusions.

The positions of the vehicles are passed to a Bayesian inference net for situation assesment. This includes detection of stalled vehicles and accidents as well as statistical information such as lane usage and speed.

Related Publications:

Related Work:

Related work at the U. of California, Irvine can be found here: Dieter Koller .