The need to identify humans based primarily on imagery of facial features is of great importance. Conventional computer-aided methods often rely on a comparison of previously collected visible images recorded under precise illumination conditions. However, such methods are prone to great variability and uncertainty with slight variation with lighting and/or skin pigmentation and are rendered completely ineffective for conditions that lack proper illumination, e.g., nighttime surveillance. As a result, many researchers have proposed using thermal infrared (IR) imagery as a basis set for human identification, since it depends on thermal emission rather than reflected light and is relatively unaffected by changes in visible illumination.

Unfortunately, thermal IR imagery of human subjects often lacks the contrast and spatial resolution required to display fine facial details when compared with conventional visible imaging systems. As a result, conventional thermal images of human faces are described as exhibiting a “ghosting” effect in which the facial profiles lack the definition and texture required for effective human identification [Fig. 1(a)].

 

Fig. 1. (a) Conventional long-wave-infrared (LWIR) thermal image (gray scale) in which the facial features lack the detail necessary for robust human identification. (b) The same image except the polarimetric information has been extracted and used to generate a “degree-of-linear-polarization” (DoLP) image as described by Eq. (6).

Download Full Size | PDF

There has been growing interest among researchers investigating polarization-based thermal imaging techniques in which information inherent in the image-forming radiance is retained and displayed as a 2D image, i.e., thermal polarimetric imaging [1–5]. It is known that both man-made and naturally occurring materials/objects emit radiation in the thermal IR that exhibits a preferential linear polarization state that is orthogonal to reflection-induced polarization [6]. This preferential polarization is thought to originate by an anisotropy generated by the superposition of elemental thermal radiators that are located near the surface–air interface of a material [7]. This in turn gives rise to the directional nature of the spectral emissivity, i.e., $\epsilon$ ($\lambda$, $\theta$, $\phi$, $T$) [8]. By generating a 2D image based solely on the polarization state, one can construct a more detailed “polarimetric” image that is extremely sensitive to changes in surface texture and geometry [Fig. 1(b)]. It should be noted that the images shown in Fig. 1 were recoded outside under clear sky conditions in which the ambient radiant levels were considered low.

The most common approach for describing the polarization state for emitted or reflected light is to measure the Stokes parameters, $S0$, $S1$, $S2$, and $S3$ [9]. For imaging applications, these Stokes parameters are determined on a pixel-by-pixel basis in order to reconstruct a 2D image. The Stokes parameters are determined by measuring the intensity of radiance, which is transmitted through a polarizer/wave-plate pair that is oriented at various angles in order to measure the magnitude of a particular state.

The Stokes images $S0$, $S1$, $S2$, and $S3$ are defined by Eqs. (1)–(4):

For radiance that is total or partially polarized, we define a degree-of-total-polarization (DoP) image as

However, for most applications that involve remote passive polarimetric imaging in the thermal IR, $S3$ is very small and rarely measurable and is taken to be zero [10]. As a result, Eq. (5) is reduced to the more common degree-of-linear-polarization (DoLP) image and is defined as

 

Fig. 2. Typical set of Stokes imagery recorded by a spinning achromatic retarder (SAR) LWIR polarimetric sensor. (Top left, rotating clockwise) $S0$, $S1$ (normalize), DoLP, and the $S2$ (normalized) polarimetric images (shown with a false color pallet) of a human subject recorded outside under clear sky conditions in which the ambient radiant levels were considered low, i.e., less than $2.0\mathrm{W}/({\mathrm{m}}^2\mathrm{sr}\mathrm{\mu m})$.

Download Full Size | PDF

 

Fig. 3. Typical ghosting effect often seen in conventional LWIR thermal imagery (left). Also shown is the corresponding LWIR DoLP polarimetric image (right) in which detailed facial features are more apparent.

Download Full Size | PDF

There are a variety of optical techniques implemented in various polarimetric camera designs, which are appropriate for polarization state filtering and analysis in the thermal IR [10]. Examples include a division-of-time approach, which uses a spinning achromatic retarder (SAR) arrangement; a division-of-amplitude (DoA) in which polarized beam-splitting plates are used to project different polarization states of the scene onto one, two, or more focal plane arrays (FPA); and a division of focal plane (DoFP) design in which micropolarizers oriented at different angles are attached to individual pixels that make up the FPA. Each approach has advantages and disadvantages associated with them, and a detailed description of the various designs can be found in Gurton et al. [11].

For the imagery and video presented in this study, a division-of-time SAR, LWIR polarimetric sensor was used [11]. The system consisted of a Stirling-cooled mercury cadmium telluride FPA, with pixel array dimensions $640\times 480$, and a spectral response range 7.5–11.1 μm. The SAR approach uses a rapidly spinning phase-retarder, which is mounted in series with a linear wire-grid polarizer. A sequence of 32-bit images are recorded at 120 Hz frame rate, which is streamed to disk for computer processing. A Fourier modulation technique is applied to the pixel readout, and a series expansion is calculated and inverted, which yields the coefficients used to generate and display the Stokes and DoLP images as described in Eqs. (1)–(6) in real time [10].

An obvious disadvantage to this approach is that the object of interest must be relatively stationary so that the image sets used in the Stokes calculations are spatially well registered. However, for conducting basic research in which the main objective is to record well-calibrated polarimetric imagery, the SAR design offers many advantages, e.g., (1) the optical configuration used in the SAR approach offers optimum radiometric throughput, and (2) the full spatial resolution of a given FPA is fully utilized.

Figure 3 shows a side-by-side video comparison of a conventional LWIR thermal video (left) and the corresponding LWIR DoLP polarimetric video (right) in which a test subject initiates various facial movements. As seen in the video, certain portions of the face are easier to distinguish/recognize in the polarimetric resolved scene (right) when compared with the conventional thermal only (left) video, i.e., eyebrows, eyelashes, teeth, wrinkles in the forehead, etc.

We should note that, in practice, the enhancement seen in LWIR thermal polarimetric imagery is greatest when the intensity of ambient background radiance is low compared with the emission levels of the human subject. This is because the polarization state induced by Fresnel reflection of the ambient background radiance is orthogonally polarized to a state that is generated by pure thermal emission [8]. At the FPA the two orthogonal components partially cancel, resulting in a net reduction of the polarization state that is received by the sensor.

However, even under less than ideal conditions as demonstrated in Fig. 3 (i.e., the Media 1 file was recorded within the confines of the laboratory where the background levels produced by the walls, ceiling, fixtures, etc., were comparable to the radiance level emitted by the human), there was still a sufficient “net” degree of linear polarization present at the FPA, which was capable of producing the enhanced details and textures shown on the right side of Fig. 3.