Face identification by deformation measure

This paper studies the problem of face identification for the particular application of an automatic cash machine withdrawal: the problem is to decide if a person identifying himself by a secret code is the same person registered in the database. The identification process consists of three main stages. The localization of salient features is obtained by using morphological operators and spatio-temporal information. The location of these features are used to achieve a normalization of the face image with regard to the corresponding face in the database. Facial features, such as eyes, mouth and nose, are extracted by an active contour model which is able to incorporate information about the global shape of each object. Finally, the identification is achieved by face warping including a deformation measure.


Introduction
Research studies related to face recognition from 2-D frontal viewsmay be shared in two mainclasses according towhether they refer toafeature based approach or toapixel approach. The first class consists inextracting the position of several facial featuresinordertor estrictthe data space. Severals t atisticalm ethods, generallyc lassical algorithms of pattern recognition, are then used to identify faces using these measures [13]. The pixel approach is ag lobal one that requires few knowledge about face geometry. Some studies are linked top a ttern matching:t he measures used to identify faces are correlations between templateregions. Other methods useprincipal component analysis applied on pixel's grey level values [12].
The application, described in this paper, is a systemsimilar toanautomaticcash machine withdrawal where the user identifies himselfbyusing a credit card withasecret code. Iti spossible to enhance the securityo fsuch a transaction by verifying thatt he observed face is identicalt oastored face of the sameperson or tocompare the face to information (points, curves, quantitative values, )o b tained by learning and stored in the database. We proposeanidentity check system using warping between faces as a measure of likeness. Knowing the assumed identity of a face (supposed by the secret code), the system comparesthis face with that one in the database corresponding to the samep e r son. This is achieved by warping one face inorderto fitthe other one.
Sincewehavechosen touse the geometry and the locations of facial features (such asthe mouth, noseandeyes) to achieve warping, each person of the database is represented by this information.
The face identification schemec o n sists of three main stages.I n the first step, in section 2, facial features are located and the image is normalized so that featureslocation match thoseofthe face corresponding to this person in the database. In section 3, the features boundaries are extracted using a specificactive contour model.T h is modelisbased on Fourier descriptors and is able to incorporate information aboutt he globals hape of each object.T h e last stage is described in section 4 and consists in measuring the warping between the normalized face and the corresponding face in the database.

Face normalization
In the context of our application, the image acquisition is achieved withafixed lightning and is acquired from a near frontal view throughoutt he temporals equence. The normalization process is based on this constraint.
The first step of our facial featuresl ocalization process concernsthe eyes. The noseandthe mouthwill be further detected using somefacial geometry hypothesis.
The presence of specular spots (usually in the neighborhood of the irises) due to the directlightning allowsto locate the eyes using a "peak" morphological operator which has the effect of bringing outthe localmaximaofthe grey level values function. A rough estimation of the eyes position is consequentlyo b tained by looking for the two regions corresponding to the highest values of the "peak" image. Since the lightning used for the acquisition is frontal, these estimations correspond approximately to the irises positions.
In order to locate the nosea n dthe mouth, we then use a spatio-temporalinformation by computing the sum of the spatio-temporal gradient norm over the sequence: where represents the grey level value, and the spatial components, the temporal component and the number of images of the sequence. is a normalization term depending on the temporalsampling of the sequence and on the motion of the different features during the sequence. Asitisshown on figure 1, the image permits topick up regions corresponding toaw e a kc o n trast but having a high deformation, like the mouth, and regions having a high contrast like the shade due to the noserelief. The localization of the nosea n dthe mouth in the image consists in searching, along the mediatrixo fthe segmentj oining the two irises, the regions having the two higher values of the image .I no r d e rto minimize the computing time, the image is computed in small windows on both sides of the mediatrixb e tween the two irises. Figure 2 showst he results obtained by this algorithm and it can be seen thatthe localization of the lower part of the nose is obtained witha good accuracy. The point corresponding to the position of the mouth depends on its motion during the sequence and is not alwayst he same physical point.W h e n the motion is important, this pointi sl ocated within the moutha n d otherwise it is generally located on the boundary of the lips.
In order toc o mpute the image ,w eu se the first 30 images of the sequence (withasampling rateof25hz). Using a shorter sequence leadst o less accurater e sults for the temporalinformation will be less significant.
In nextsection we make useforthe system of an active contour modeltoextract facial features boundaries: inorder touse the same initialization and parameters for the active contour model withdifferentimages acquired from one face, we normalize the image. This is achieve by applying a similarityusing the irises location as references. Since the active contour parameters and the curves used for initialization are nott he samef o r different faces, thesefeatures will be stored in the database with the facial characteristics.

Facial feature extraction
An active contour modelisan energy minimizing curve guided by internal constraints and influenced by image forcest hat pull it towardsi m age edges. Since the global regularityconstraints used by genericactive contour models (such as "snakes" [8] [3]) are included within the internal energy function, these models do not allow tor e fine the space of admissible curvestoaspecific shape class.I no rder to take into accountthe shape of facial features, several methods based on deformable templates have been developed.
Yu ille, Cohen and Hallinan [13] proposed a set of deformable templates for this purpose. Each one of these templatesisdesigned toe x tract a specificfe a ture. For example the templateused toextractthe eye boundary consists of a circle bounded by two parabolas and possesses 11 parameters.
Craw, Tock and Bennett [6] described a model which consists of a set of 40 landmark points, each one representing the position of a particular part of the structure tobelocated. Several algorithms are designed todeal withaparticular feature using a statistical knowledge of the landmark positions. Ag lobal algorithm controls the interaction between them. A similar method has been proposed by Cootes,T ailor and Lanitis [4] using statisticallybased models which iteratively move towardsstructuresin images after a learning process on similar images.
In this section we proposeaparametrical active contour model using Fourier descriptors.T h e a im is tod e sign a models ufficiently generalt oe x tractt he boundary of several facial features while taking into accountt he specific geometry of each feature. This model needst o tune few parameters.

Active contourmodelusingFourier descriptors
The useofFourier descriptors for active contour models has been introduced by Staib and Duncan [11] ino r d e rto extract an object boundary. Their method is based on the use of probabilityd istributions on the parameters.I no r d e rto be less sensitive to the initial parameter value, we proposea variational approach similar to the method used in the snake model [8].
An elliptic Fourier representation of a closed curve is a parametrical curve defined by: 0 cos sin (1) where is a2 2 matrix, the number of harmonics used tod e scribe the curve and the angular parameterization index.
Such a class of curves does not permit torepresent effectively the boundaries of objects having an irregularitypoint withinaregular section, such asthe corner of the mouthor eyes.
where and are the parametersm odel, is the number of harmonics used todescribe the curve and is the rotation matrixofangle .T h u s, the phase shift between the two bases used for and permits todescribe a half-ellipsewith four parameters.
The curve modeling the boundary of the object, is obtained by minimizing a functionalsimilar to the snake energy: The first term inside each one of the integrals is an image potential equalt o the oppositeo fthe square of the image gradient ( 2 )a n dthe second one is an elasticity term associated to the curve tension.
In order toextract boundaries of the mouthandeyes we use one harmonic tod e scribe the lower boundary, and harmonics for the upper boundary, .T h u s hasthe form defined in equation (2) and is defined by:

Results
The algorithm used toextract featuresisa fine tocoarse scheme( see figure 3). Defining two half-ellipses asi nitialization, the convergence is first achieved using a single harmonicforthe curve , then the first result is used as an initialization toget a curve with two harmonics until we obtainacurve described by harmonics fitting the object boundary.
Ast he number of harmonicsi ncreases, the regularity constraints are smaller and the boundary can be described more preciselywithabetter stability.

Face Warpingforidentification
Severalm ethods have been proposed to achieve face recognition using featureslocation. Some studies use this information ino r d e rtoc a lculate the correlation between the regions corresponding toasamefeature in two different images [1]. The measure is then equalt o the mean of the correlation values.O ther methods utilize a more geometrical approach by defining a discriminating measure using physiological knowledge [7], results of principal component analysis [5] or topological graphs [9].
In this paper, two faces are compared by a deformation measure. Given a set of landmark points on the face, the measure is a quantification of the bending that hast ob e applied to the face, so that its landmark points match the ones of the other face. To thisissue, we utilize a class of deformations similar to the one used for normalization by Reisfeld[ 1 0 ] . W h ileh is aim wast o normalize facesi n order toapplyaclassification on the image data, we propose to achieve the identification using the deformation measure value asthe discriminating information.

Face characterization and database
The landmark points used tor e p r e sentt he face are located on the the features boundaries which have been extracted as described in section 3. Using all the points of the boundaries for the matching process will be useless since the information is redundant (like neighbor points on the same boundary) and somepoints cannot be easilyextracted from the curves describing the boundaries (due to the parameterization which is noti ntrinsic). We made the choice of 15 points displayed on figure 4. Thesepoints characterize the position of the features as well astheirglobal geometries.

Figure 4. pointscharacterizing a face
However someo fthe chosen landmark points are mobile since they are located on a feature which is subjectt o temporal deformation. Therefore, inor dertoconstructthe databasew eu seasequence of images of peopleu ttering aparticular sentence whileblinking. Then, on each image of the sequence, the features boundaries are extracted on a normalized image (asi ti ss hown in sections 2 and 3) and for each landmark point we calculate the distribution of its position. Normality tests showed thatt hesed istributions may be represented by Gaussian laws.T h u s,ap e r son is represented in the databasebyaset of 4 values: where is the mean position of the landmark point and its standard deviation on the horizontal and vertical axes. Bookstein[2]showed the interest of the thinplate splines for image registration purpose since they can be used to solve the problem of scattered data interpolation while minimizing the warping.

Deformation measure
The total amount of bending of a surface ( being a twicelyd ifferentiable function : 2 ), can be quantified by the value of the functional: Given a set of landmark points on one face and on a second one, the function : 2 2 , thatm atches each point to withrespectto its standard deviation and that is least bent according to the functional (3), minimizesthe functional: Thus,aweightisassociated to each landmark point which decreasest he influence of a pointi f its standard deviation is large. The function is constructed so that is always defined and the points having small standard deviation are nottoo influent.
The transformation , thatm i nimizes ,h a st he following form: Severaltypes of function can be used inor derto obtainaminimum for .I n p a r ticular the useo f 2 log (withU(0)=0) leadstoanunique solution verifying the propertyo fthe thinp late spline. The coefficients 1 2 3 1 2 3 1 and 1 can be calculated by solving a linear system as described in [2] [10].
The quantity may be used to measure the likeness between two faces.T h ea ssumed identitywill then be confirmed if this value is smaller than a given threshold. The thresholdvalue is linked to the accuracy of the systemthat is wished for. If it is small, it can hapen thatthe identityof aperson will be wronglyrejected;if it is large, the identification test will then be less restrictive. Lastely, the choice of the tresholdvalue must take several parametersinto accountsuch asthe image acquisition and the accuracy of the localization and extraction processes.

Conclusions
This paper presents a set of methodsthat are used to construct an identity check system.T h elocalization of facial features (such asthe eyes,noseandmouth) is achieved using a morphological operator as well asthe spatio-temporal information. This method is robust provided thatlightning is frontal.I ti sused to normalize the face image withr egard to the corresponding face in the database. The active contour model proposed for feature boundary extraction is a parametricallydeformable model using Fourier descriptors. This modeli sdesigned so that few parameters are needed toe x tracti rregular objects like the eyes or the moutha n d the stability is increased by applying a coarse to fine scheme to the convergence process. A face is represented in the databasebyaset of landmark points belonging to the features boundaries as well ast he standard deviations of the points calculated on a sequence of images. Finally the likeness between two facesi squantified using a deformation measure. This deformation correspondsto the bending to be applied on a face, so that its landmark points match the ones of the other face. To this issue, an enhancement of a standard method used to interpolate scattered datahas been proposed which allowstoa ssociateaw e ightt o each landmark pointso that points having a small standard deviation have more effect on the bending.