Isn't it amazing! We are all able to recognize, with a high degree of accuracy, individuals whom we have not seen for months or even years. How do we do it? Or, more importantly, how can we teach a machine to do it??

If this was possible (and to a degree it is) we could simplify many aspects of life: clearance at an airport; foregoing fingerprinting; apprehension of felons; simplifying passwords; and many others. There are, of course, those who say such identification would be a blatant invasion of privacy, and should not be allowed. More about that later.

But first, what is it that we are able to do almost intuitively, and that we would like machines to be able to do?
• We want to "memorize" enough about the face we are seeing to be able to remember it at a later date
• We want, upon seeing a face, to be able to match it with a template -- a face previously seen.

Before we get into the How-Do-We-Do-It business, let's look at who is doing it, and why.

…On a general level, law enforcement officials can match a picture of a person with a known data base.

…The German Federal Police have used a system to allow certain people to pass automated border controls at their Frankfurt Rhein-Mann airport.

…The Australian Customs Service has an automated system that uses facial recognition. The face of an individual is compared with an image encoded in a passport to permit entry.

…The U.S. Department of State operates a very large face recognition system; more than 75 million photographs are used for visa processing.

…At the 2001 Super Bowl in Tampa Bay, Florida, a system was used to search for potential criminals and terrorists.

…In Mexico voter fraud was frustrated by comparing facial images to those already in the voter database.

…Blacklisted customers at casinos are being turned away after being identified by facial recognition systems.

…Work is ongoing to associate a picture of an ATM customer with a bank database, thus obviating the need for a debit card.

…Several potentially attractive but professionally trivial applications have to do with bars and restaurants. For instance, what is the ratio of men to women in the bar; what is the average age of patrons; how busy is the bar?

…On a personal level, and perhaps just as trivial, an application has to do with the home TV. Which of the family members is sitting in front of the TV, and therefore what programs should be (or, more important, should not be) viewed?

As we will see, the algorithms used for facial recognition are extremely complex. But our modern day electronics and our sophisticated programs are quite capable of handling such challenges, and numerous computers are now equipped with very powerful systems. Google's Picasa digital image organizer has a built-in face recognition system. Apple iPhoto includes a system by which people can tag recognized people on photos. Sony's Picture Motion Browser analyses and sorts photos. Windows Live Photo Gallery employs face recognition. And Facebook includes facial recognition technology.

Just how is this all done? A crime victim can be shown a book of mug shots, and come relatively close to identifying a likely perpetrator. But what if the book contains 1 million mug shots? It's out of the question. And here is where the computer comes in.

The simplest and most straightforward method of memorizing a face is by identifying a set of distinguishing features, or landmarks. These would include distance between the eyes, the width of the nose, the depth of the eye sockets, the shape of the cheekbones and the length of the jaw line. This information can all be digitized, and automatically compared with the equivalent data from a large number of possibilities.

A more sophisticated system is the Principal Components Analysis (PCA). It is not a new system, having been developed in 1988 by Kirby and Sirivich. With PCA 2 photos are compared with each other. Each section of the photos (they are called eigenfaces) is digitized, and the results are compared. High-level compression techniques allow the system to disregard unnecessary data, but the result is an extremely good matching technique.

Not perfect, of course, but good.

Recent advances have moved the technology from 2-Dimensional to 3-Dimensional. With 3-D technology the shape of a face can be included -- for instance, the contour of the eye sockets, nose, and chin. Also, a change in lighting, or a viewing angle, is less significant.

This leaves us with the appropriateness of the technology. Is it, or should it be, illegal to take pictures of people? Certainly the advances in camera technology permit a picture of a large crowd to be segmented to get a highly detailed picture of a single individual.

Given enough surveillance cameras, and good software, is it possible for a government or other organization to know where you are, and what you are doing, at all times? How should this be controlled? Can it be controlled?

Facial recognition technology continues to move ahead. As its accuracy improves so also will its applications.

It will be a continuing challenge to use this technology responsibly.

What is your experience with this? Tell your fellow readers now!