The problem of estimating the projection matrix is the subject of a considerable number of publications of which important results are presented in [Fau93]. Usually this problem is treated under a different point of view by making the assumption that a larger number of points can be identified or tracked in the image, as in the application we are aiming at. To obtain a unique solution of the problem, at least 6 points (in a general position) are necessary. For our application it is particularly interesting to investigate, if pose estimation is, at all, possible with three or at least four points. This particular problem has already been focused on long before the advent of computer vision and we can find a review of different solution strategies in the work of Haralick et al. [HLON94]. The main result states that we can, in general, calculate the pose from three points, but we might obtain up two four distinct solutions. This is not disturbing for our purposes, because we are working with image sequences. By imposing smoothness constraints on the motion of the tracked points, we can eliminate irrelevant solutions.
Figure 6: The Pinhole Camera Model