Student: Mario E. Munich 
Faculty: Pietro Perona 
Support: NSF, NYI 
Short Description


A number of biometric techniques have been proposed for personal identification in the past. Among the vision-based ones, we can mention face recognition, fingerprint recognition, iris scanning and retina scanning. Voice recognition or signature verification are the most widely known among the non-vision based ones. Signature verification requires the use of electronic tablets or digitizers for on-line capturing and optical scanners for off-line conversion. These interfaces have the drawback that they are bulky and complicated to use, increasing the complexity of the whole identification system. Cameras, on the other hand, are much smaller and simple to handle, and are becoming ubiquitous in the current computer environment. The goal of this project is to develop a signature verification technique suitable for  signatures captured by our camera-based acquisition system 

Handwriting recognition is still an open problem, even though it has been extensively studied for many years. Signature verification is a reduced problem that still poses a real challenge for researchers. The literature on signature verification is quite extensive and shows two main areas of research, off-line and on-line systems. Off-line systems deal with a static image of the signature, i.e. the result of the action of signing while on-line systems work on the dynamic process of generating the signature, i.e. the action of signing itself. The system proposed in this paper falls within the category of on-line systems since the visual tracker of handwriting captures the timing information in the generation of the signature. 


The camera-based acquisition system uses computer vision techniques and estimation theory to track the position of  the pen tip in the image plane. The verification algorithm compares the 2D shape of the signatures using a translation-invariant metric. 

BLOCK DIAGRAMSystem Block Diagram 


In the following figures we show several signatures and forgeries acquired in real-time with our system. 

Example signature
Example signature


The following figure shows the signature verification system. Given the subject's name and a new acquired signature, the system decides whether the signature corresponds to the subject's name. The decision is based on a measurement of the similarity between the acquired signature and a prototype of the subject's signature. The prototype has been previously extracted from a training set of signatures. 

sv.GIF (41612 bytes) 

The signature verification system is based upon the measurement of similarity between signatures. Therefore, this computation is the key element of the system. There are several methods proposed in the literature in order to compare signatures. We choose to use Dynamic Time Warping (DTW)  in order to perform the comparison between signatures. The present implementation of DTW for signature verification attempts to perform the best alignment of the 2D shape of the signatures, i.e., we find the warping function that has the minimum cost of aligning the planar curves that represent signatures.  We note that the pen up strokes drawn by each subject were as consistent as the pen down strokes. This observation agrees with the belief that signatures are produced as a ballistic or reflex action, without any visual feedback involved. Therefore, we used the full signing trajectory in our experiments.We do not perform any type of normalization on the signatures since we consider that users are very consistent on their style of signing, they write their signatures with a similar slant, in a similar amount of time, with similar dimensions and with a similar motion. 


In most of the literature on signature verification, a time-based parameterization of the functions to be aligned was used. There is no clear reason for using this parameterization of the signatures rather than another one, e.g. arc-length parameterization. The arc-length pameterization of the signature is loosely dependent on time and on the dynamics of signing, even though it keeps the causality of the signature's generation. This weak dependence on the dynamics of signing seems contrary to the traditional idea that pen dynamics is a key element in detecting forgeries. However, the use of the arc-length parameterization is a first step towards achieving invariance with respect to Euclidean transformations of the signatures. Going one step further, we could use a parameterization that provides a certain degree of invariance with respect to affine transformations of the signatures. This parameterization has been described in the literature and has been called affine arc-length. 


The system needs to extract a representation of the training set that will yield minimum generalization error. DTW provides the optimal alignment of two signatures, so in the case in which there are more than two examples in the training set, there is no clear way of aligning all of them at the same time. We perform only pairwise alignment between all elements in the training set. The signature that yields minimum alignment cost with all the remaining ones is chosen to perform the final matching. All signatures are placed in correspondence with this particular one. The prototype that represents the training set is computed as the mean of the aligned signatures. The individual residual distances between each of the signatures in the training set and this reference signature are collected in order to estimate the statistics of the alignment process. This statistics is subsequently used for classification. In the followgin figure we show a few signatures from the training set and the prototype. 

Example signature
Example signature
Prototype signatures


There are two different errors that characterize the performance of the algorithm. The Type I error (or False Rejection Rate (FRR)), measures the number of true signatures classified as forgeries as a function of the classification threshold. The Type II error (or False Acceptance Rate (FAR)), evaluates the number of false signatures classified as real ones as a function of the classification threshold. The performance of the system is provided by these two errors as a function of the classification threshold. Clearly, we can trade-off one type of error for the other type of error. As an extreme example, if we accept every signature, we will have a 0% of FRR and a 100% of FAR, and if we reject every signature,we will have a 100% of FRR and a 0% of FAR. The curve of FAR as a function of FRR, using the classification threshold as a parameter, is called the error trade-off curve. It provides the behavior of the algorithm for any operation regime and it is the best descriptor of the performance of the algorithm. In practice, this curve is often characterized by the equal error rate, i.e., the error rate at which the percentage of false accepts equal the percentage of false rejects. This equal error rate provides an estimate of the statistical performance of the algorithm, i.e., it provides an estimate of its generalization error.  


We collected signatures from 56 subjects, 18 of them women and 4 left handed. Each of them provided 25 signatures, 10 of them to be used as the training set and the other 15 to be used as the test set. We also collected 10 forgeries for each of the subjects in the database. The test set allows us to compute the FRR. We computed the FAR in two different ways. First, we used all the signatures from the other subjects as random forgeries, and second, we used the acquired forgeries. 

One common problem of many on-line systems for signature verification is the lack of examples needed to build a reliable model for a signature and to asses the performance of the algorithm. This problem is inherent to the application since it is not feasible to ask a subject for all the examples of his/her signature required to perform these two tasks reliably. Thus, we have to build a model of the signature that will perform well in practice and we have to infer the generalization error of the algorithm, all with very few examples. We could increase the number of examples in both the training and test set by using Duplicate Examples, if we know that the model that we are building should be invariant with respect to some transformation of the examples.  

The following figures show the error trade-off curves for our system. 
Error trade-off curves for random forgeries
Error trade-off curves for intentional forgeries


We have developed a vision-based system for personal identification based on signature tracking. Our system achieves and equal error rate of 0.2% for random forgeries and 3.3% for intentional forgeries. We have shown the importance of choosing the proper parameterization of the signatures in order to achieve good performance. We have also described a method that allows to overcome the lack of examples in order to estimate the generalization error of the algorithm. 


[1] M. E. Munich and P. Perona, "Camera-Based ID Verification by Signature Tracking", in Proceedings of the  5th European Conference on Computer Vision EECV'98 (pages: 782-796), Freiburg, Germany, June 1998. (16 pages postscript)

[2] M. E. Munich and P. Perona , "Visual-Based ID Verification by Signature Tracking" , in Proceedings of the 2nd International Conference on Audio- and Video-Based Person Authentication, Washington, March 1999. 

[3] M. E. Munich and P. Perona, "Visual signature verification using affine arc-length", in Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, Fort Collins, Colorado, June 1999. (7 pages postscript) .

Mario Enrique Munich - mariomu AT vision DOT caltech DOT edu