caricature 2452345234

How does a computer recognize a face?

Since 9/11, there’s been a renewed interest in facial recognition algorithm to catch terrorists trying to slip into the country but ten years later, the system ain’t anywhere close to perfect (yes, even including Facebook’s creepy facial recognition system)

Perhaps they’re going about it the wrong way, according to Ben Austen of Wired. Rather than taking biometric measurements of the size of a person’s nose or eyes, computers would do well to learn from caricaturists instead…

Did you hear the one about the vision scientist who used only caricaturists as his test subjects? He exaggerated his findings! Pawan Sinha, director of MIT’s Sinha Laboratory for Vision Research, and one of the nation’s most innovative computer-vision researchers, knows that caricatures are meant to be humorous, grotesque, and outlandish—he dabbles as a caricaturist himself, drawing occasionally for university publications. But Sinha also contends that these simple, exaggerated drawings can be objectively and systematically studied and that such work will lead to breakthroughs in our understanding of both human and machine-based vision. His lab at MIT is preparing to computationally analyze hundreds of caricatures this year, from dozens of different artists, with the hope of tapping their intuitive knowledge of what is and isn’t crucial for recognition. He has named this endeavor the Hirschfeld Project, after the famous New York Times caricaturist Al Hirschfeld.

Quite simply, the Hirschfeld Project would reverse-engineer the caricaturist’s art. By analyzing sketches, Sinha hopes to pinpoint the recurring exaggerations in the caricatures that most strongly correlate to observable deviations in the original faces. The results, he believes, will ultimately produce a rank-ordered list of the 20 or so facial attributes that are most important for recognition: “It’s a recipe for how to encode the face,” he says. In preliminary tests, the lab has already isolated what seem to be important ingredients—for example, the ratio of the height of the forehead to the distance between the top of the nose and the mouth.