Object appearance is controlled by a multitude
of factors. Some of these factors (such as
illumination) are irrelevant to object
identity and hamper recognition. In 
this paper, we propose an approach to object
recognition that is based on learning to 
eliminate these irrelevant, distracting
factors. This approach is complementary to
most previous schemes, which attempt to learn
the relevant factors. The proposed approach
has two main advantages. First, we show
that major distracting factors can be learned
with very limited training data. As a result,
the proposed scheme requires much less
training data than the previous
schemes. Second,
we show that the problem of estimating
distracting factors can be formulated in a
general learning framework. As a result,
arbitrary distracting factors can be learned 
systematically and automatically 
from training data. This single framework can
successfully learn factors as diverse as
illumination (for faces) and font style (for
characters).