Local Analysis for 3D Reconstruction
of Specular Surfaces

Silvio Savarese and Pietro Perona

California Institute of Technology

 

Abstract. Specular reflections carry valuable information on surface shapes. A curved mirror surface produces "distorted" images of the surrounding world. For example a straight line reflected by a curved mirror is in general a curve. It is clear that such distortions are systematically related to the shape of the surface. Our goal is to explore the geometry linking the shape of a curved mirror surface to the distortions produced on a scene it reflects. To this effect, we assume a simple calibrated scene composed of lines passing through a point. We demonstrate that local information about the geometry of the surface may be recovered up to the second order from either the orientation and curvature of the reflected images of three or more intersecting lines.

Figure 1. M.C. Escher (1935). Still Life with Spherical Mirror

Introduction and motivation. One of the main tasks of a visual system is computing the shape of the objects. A number of cues, notably stereoscopic disparity, texture gradient, motion parallax, contours and shading, have been shown to carry valuable information on surface shape, and have been studied extensively both computationally and psychophysically. The study of shading has been mostly restricted to the Lambertian case. Specularities have been mostly ignored with the significant exception of the analysis of specular highlights. We are interested in exploring the possibility of recovering information on the shape of a surface from the specular component of its reflectance function. Since we wish to ignore the contributions of shading, we will study surfaces that are perfect mirrors. A curved mirror surface produces `distorted' images of the surrounding world (see Fig.1). For example, the image of a straight line reflected by a curved mirror is, in general, a curve. It is clear that such distortions are systematically related to the shape of the surface. Is it possible to invert this map, and recover the shape of the mirror from the images it reflects? Our goal is to explore the geometry linking the shape of a curved mirror surface to the distortions produced on a scene it reflects. To this effect, we assume a simple calibrated scene composed of lines passing through a point. We demonstrate that local information about the geometry of the surface may be recovered up to the second order from either the orientation and curvature of the reflected images of three or more intersecting lines. We view our analysis as a promising start in the quest of computing the global shape of specular surfaces under fairly general conditions. The case of an uncalibrated world appears much more challenging and will require most certainly the integration of additional cues and some form of prior knowledge on the likely statistics of the scene geometry.

The Setup. We assume a calibrated world composed of the simplest primary structures: one point and one or more lines through it. The point and the lines are reflected by the mirror surface onto the image plane of a calibrated camera, where the intersection, orientation and curvature of such reflections are measured. The relationship between the local geometry of the surface around the point of reflection and the measurements is analyzed.

Figure 2. The Setup

 

The theory. We start our analysis by considering a scene composed by only one point xp. The scene point is reflected by the mirror surface (at xm) and imaged into the image plane of the camera yielding the observation xi. If Oc is the center of the camera, the first order geometry of the surface (namely, position xm and normal nm) at xm is known up to parameter s, s being the distance between Oc and xm.

Figure 3. One point constraint

With only one scene point, the problem is underconstraint. Thus, let us consider a line through xpo and its corresponding parametric representation xp(t). See Fig. 4. As t varies, we define a mapping between the scene line, the corresponding reflected curve f(t) on the mirror surface and the observed curve fi(t).

Figure 4. The mapping.

We consider now a more suitable reference system with origin at the reflection point xmo and coordinate axis (u,v,w) defined by the surface normal vector nmo and surface tangent plane at xmo (see Fig. 5). In this new reference system, the mirror surface can be locally expressed (around xmo) by its corresponding Special Monge form:

Figure 5. A new reference system.

The Langrange Multiplier Theorem and the Implicit Function Theorem are used to study the differential behavior of f(t) around t=0. The result is that the tangent vector to of f(t) at xmo (see Fig. 4) can be expressed as function of the second order surface parameters a,b,c and the distance parameter s. The orientation of to in the surface tangent plane is given by the equation in Fig. 6.

Figure 6. The tangent line constraint.

Since the orientation of the tangent of fi(t) at xio can be measured, the orientation of to can be also expressed as function of such measurement and the distance parameter s. Thus the equation in Fig.6 represents the constraint we are searching for -- a relationship between a,b,c,s and the measurement. If we consider more than 1 scene line through xpo, for each line we can write a corresponding tangent line constraint. See [1] for details.

 

Experimental Results. (i) If the mirror surface is a plane, then a=b=c=0. Thus, one point and one line are enough to compute the unknown distance s and the plane normal ns.

Figure 7. Reconstruction of a planar mirror.

(ii) If the mirror surface is a sphere, then a=b and c=0. Thus, one point and two lines are enough to compute the unknown distance s, the sphere curvature a and normal nso at xmo.

Figure 8. Reconstruction of spherical mirror.

(iii) For generic surfaces a,b,c,s are unknown. It turns out that at least 3 scene lines are necessary in order to compute the distance parameter s (although a few number of ghost solutions may arise) and the second order local surface parameters a,b,c (up to one unknown parameter) [2]. Such unknown parameter may be estimated by measuring the curvature of the reflected lines in the image plane and by comparing the measured curvatures with those computed from analytical differentiation. The analysis is valid in a neighborhood around the reflection point where the third order terms of the surface may be neglected [3]. Experimental results with generic surfaces are shown in Fig. 9 and 10.

Figure 9. Reconstruction of a cylinder.

Figure 10. Reconstruction of a sauce pan's lid.


Publications.

[1] S. Savarese and P. Perona, "Local Analysis for 3D Reconstruction of Specular Surfaces", in Proc. of IEEE Conference on Computer Vision and Pattern Recognition, Kawa'i, USA, December 2001.

[2] S. Savarese and P. Perona, "Local Analysis for 3D Reconstruction of Specular Surfaces -- part II", in Proc. of 7th European Conference of Compter Vision, Danmark, May 2002.

[3] S. Savarese, M. Chen and P. Perona, "Second Order Local Analysis for 3D Reconstruction of Specular Surfaces", in Proc of 1st International Symposium on 3D Data Processing, Visualization and Transmission, IEEE Press, 2002