Introduction

Human faces captured in real-world conditions present 
large variations in shape and occlusions due to differences 
in pose, expression, use of accessories such as sunglasses and hats and interactions with objects (e.g. food). Current face landmark estimation approaches struggle under such conditions since they fail to provide a principled way of handling 
outliers. We propose a novel method, called Robust 
Cascaded Pose Regression (RCPR) which reduces exposure 
to outliers by detecting occlusions explicitly and using robust shape-indexed features. We show that RCPR improves 
on previous landmark estimation methods on three popular 
face datasets (LFPW, LFW and HELEN). We further 
explore RCPR’s performance by introducing a novel face 
dataset focused on occlusion, composed of 1,007 faces presenting a wide range of occlusion patterns. RCPR reduces 
failure cases by half on all four datasets, at the same time as 
it detects face occlusions with a 80/40% precision/recall.

Contributions

Citation

Results

LFPW, HELEN, LFW

COFW

COFW Dataset

    Our face dataset is designed to present faces in real-world conditions. Faces show large variations in shape and occlusions due to differences in pose, expression, use of accessories such as sunglasses and hats and interactions with objects (e.g. food, hands, microphones,
etc.). All images were hand annotated in our
lab using the same 29 landmarks as in LFPW. We annotated both the landmark positions as well as their occluded/unoccluded state. The faces are occluded to different degrees, with large variations in the type of occlusions encountered. COFW has an average occlusion of over 23%.
To increase the number of training images, and since 
COFW has the exact same landmarks as LFPW, for training 
we use the original non-augmented 845 LFPW faces + 500 COFW faces (1345 total), and for testing the remaining 507 COFW faces. To make sure all images had occlusion labels, we annotated occlusion on the available 845 LFPW 
training images, finding an average of only 2% occlusion.

Code


 The following Matlab code shows how to run our landmark estimation algorithm (RCPR) on the COFW dataset to reproduce the paper's results. We also include an example to perform full face landmark tracking in video, using a pre-trained RCPR model in combination with our tracking algorithm published in BMVC13.

Back to top

© 2013, Xavier
P. Burgos-Artizzu, Pietro
Perona and Piotr
Dollár

Flag Counter