Face Recognition
Face recognition is the process of automatically determining whether two faces are the same person. A number of factors make this a challenging problem for computers. Faces in images and video can be captured at various resolutions, quality, and lighting conditions. Different cameras have different imaging properties. Moreover, people’s facial expressions as well as their pose with respect to the camera can vary widely, and facial characteristics can change dramatically as people age over time. As such, our face recognizers, like our detectors, have been trained using novel statistical learning methods, to deal with these diverse factors and provide accurate results on real-world data.
Unparalleled Accuracy on Real-World Data
By design, our face recognition technology performs accurately on real-world, uncontrolled data. For example, our recognizers correctly identify the 24 images of our VP of Research and Development (Michael Nechyba) pictured below, despite some challenging factors:
- Substantial time lapse between photos (2003 – 2009)
- Appearance variations due to weight loss, facial hair, hair style, hats and glasses
- Uncontrolled pose and facial expressions
- Uncontrolled lighting, both indoor and outdoor
- Low-resolution / low-quality face imagery
Unparalled Low-Resolution Recognition
Our recognizers are trained to operate on low-resolution faces. In our staged approach, the recognizers analyze facial features at three separate resolution levels – namely, 12, 20 and 25 pixels between the eye centers. This approach has led to unparalleled accuracy on challenging low-resolution media.
Unparalleled Face Acquisition
The first step in face recognition is the acquisition of faces in visual media. Face acquisition for the purposes of recognition requires not only face detection, but precise alignment prior to matching faces. We perform this alignment automatically through pose estimation and landmark localization.
Large-Scale Reasoning
We have developed technology to perform large-scale reasoning over visual media. Large-scale reasoning allows us to automatically cluster faces into same-subject groups. This in turn allows us to organize and index image collections or video archives based on who appears when and where.
Video Recognition Examples
When we combine face recognition with face tracking over videos, we can recognize not just individual face instances, but “face tracks” as well. Below is a short video clip taken from NBC‘s “The Office.” For this clip, tracks are compared against the cast of characters. Unidentified face tracks are annotated in white and occur either because the person is not a member of the cast, or the track contains no good frontal / near-frontal face instances.
Recognition Across Pose
Currently, our commercially available face recognition technology is limited to frontal and near-frontal faces. However, we are actively working to extend our technology to perform recognition across the full pose spectrum. This work will allow face matching for non-frontal poses (semi-profile, profile) as well as across poses (e.g. profile to frontal comparisons).