Open Access
Add to BibSonomy
Abstract
Passive millimeter and terahertz wave imaging has become a significant potential technique for human security check and scene monitoring. Due to the small difference of the brightness temperatures between human body and concealed objects, the temperature sensitivity and spatial resolution of radiometers are always the key performance indexes which are difficult to improve. Therefore, when the hardware performance is given, improving detectivity becomes a pressing need. In this paper, a physically-based concealed object enhancement method using multi-polarization information is presented. The polarization model and polarization property of human body and concealed objects have been analyzed. By fusing multiple polarization images, we can obtain a complete polarization image in which the contrast between human body and concealed objects is enhanced and stable. The experimental results of simulation and measurement demonstrate the enhancement performance, and Differential Signal Noise Ratio (DSNR) is obviously improved by using the proposed method.
© 2020 Optical Society of America under the terms of the OSA Open Access Publishing Agreement
1. Introduction
Millimeter and terahertz wave technology has attracted increasing attention from academia and industry in recent years [1–3]. Passive millimeter wave (PMMW) and passive terahertz wave (PTW) sensors can passively generate thermal radiation data at all day and night condition by detecting the naturally occurring millimeter and terahertz radiations and reflections from scenes. There are several windows (low atmospheric losses) of frequencies between the resonances of oxygen and water, which are centered at 35, 94, 140 and 220 GHz [4]. Therefore, PMMW/PTW sensing has been used for radio astronomy, earth surface and atmosphere remote sensing, moon observation [5–7]. Due to the non-radiation damage and the penetrability of most types of clothing and camouflage, PMMW/PTW imaging has a significant potential in human security screening [3,8–10].
Since the 1990s, researchers have designed many kinds of PMMW/PTW security systems and tentatively conducted many human imaging experiments in public places [11–14]. The experimental results demonstrate the ability of detecting concealed objects (e.g., metal gun, metal knife, explosive, phone). Although a variety of human security systems have been developed, it is still difficult to reach the level of large-scale commercial application. The reason is not only the lack of radiometer performance (e.g., temperature sensitivity and spatial resolution), but also the insufficiency of obtained information (e.g., single or dual polarization and single frequency band data). When the radiometer hardware performance is given, how to improve detectivity is an important problem to be resolved. In general, the gray-scale image of brightness temperature is used to detect and recognize concealed objects in previous works, such as traditional object detection [15,16], object segmentation [17], resolution improving [18], machine learning detection [19,20] and so on. The brightness temperature differences may be caused by material dielectric property, physical temperature, 3D structure, surface roughness, etc. Therefore, the obtained information from the brightness temperature image of single or dual polarization is not enough.
Polarimetric measurement is an effective approach to acquire more information of scenes. Since the 1990s, it has been used to retrieve the wind direction and speed of sea surface, the thickness of sea ice and so on [21,22]. In recent years, several mmW/THz polarimetric radiometers have been developed to investigate the polarization phenomenon of close-range scenes. Based on the polarization properties, some interesting applications have been reported in our previous works and other existing researches, such as material classification [23–26], surface orientation estimation [27,28], object enhancement [29,30], image segmentation [31,32], etc. From our previous experiment work [33], multi-polarization imaging and processing is an combination of physical and mathematical method. The additional information received by multi-polarization radiometers might contribute to enhance the brightness temperature image of human security screening.
In this paper, a physically-based enhancement method is proposed to relieve the problem of sensor performance lack. We analyze the multi-polarization brightness temperature model of human security check scene. Then the physical principle of polarization property difference between human body and concealed objects is investigated. The analysis results indicate that the brightness temperatures of human body and concealed objects differ greatly and stably for the complete vertical polarization. Additionally, we present a conversion method to obtain the complete vertical polarization image in which the concealed object contrast is enhanced. The simulation and measurement experiments have been conducted to verify the enhancement performance in the end.
2. Physical principle and method
2.1 Polarization model
Passive mmW/THz sensors acquire data and generate images by receiving the naturally emitting mmW/THz radiations and reflections from observed scenes. For a facet of non-transparent object, the observation geometrical sketch of brightness temperature is shown in Fig. 1. In the object surface coordinate base $(x, y, z)$, $z$ is the normal vector of position O, and plane $xoy$ is in the tangent plane of position O. $\theta$ and $\phi$ are the observation incident angle and azimuth angle of radiometer. $\theta _i$ and $\phi _i$ are the incident angle and azimuth angle of incident ambient. In this paper, we only consider linear polarization case. $\alpha$ represents the linear polarization angle of object surface brightness temperature ($-180^\circ \sim 180^\circ$). $T_{Binc,\alpha }$ is the ambient brightness temperature incident on the object, which contributes to brightness temperature with $\alpha$ polarization. $e_\alpha$ and $r_\alpha$ denote the mmW/THz emissivity and reflectivity of object surface. $T_{obj}$ is the physical temperature of object.
Fig. 1. Observation geometrical configuration of non-transparent object. $z$ is the normal vector of position O. $\alpha$ represents the linear polarization angle of object facet brightness temperature. (a) General case in which $T_{Binc,\alpha }$ is from all directions theoretically. (b) Special case in which $T_{Binc,\alpha }$ is mainly from the specular reflection direction.
The radiation from the object surface includes the self-emission and the reflection portion of ambient radiation. In the general case of Fig. 1(a), $T_{Binc,\alpha }$ is from all directions theoretically. Hence, the arbitrary linear polarization brightness temperature of object facet at the position O can be written as
In the special case of Fig. 1(b), the object surface is approximately smooth and the reflection portion only contains the specular reflection, so $\Omega _i=\Omega$ and $r_\alpha (\Omega ) = 1-e_\alpha (\Omega )$. In this case, formula (1) can be written as
For the human skin or concealed objects under clothing, their mmW/THz radiations propagate in clothing and atmosphere and then are filtered by radiometer antenna. To differ from $T_{B\alpha }$, the brightness temperature before antenna is expressed as $T_{AR\alpha ^o}$. $\alpha ^o$ is the observation linear polarization angle of radiometer antenna, which is defined by regarding the horizontal plane as the reference plane. As shown in Fig. 2(a), in the antenna coordinate system ($i, h, v$), $i$ is parallel to observation direction, $h$ is parallel to horizontal level, and $v$ is both perpendicular to $i$ and $h$. Considering the single linear polarization antenna, the electric field direction ($E$ plane) of antenna determines the polarization mode of received signals. $\alpha ^o$ is the included angle between $E$ direction and $h$ axis, so the radiometer antenna receives the horizontal polarization signal when $\alpha ^o = 0^\circ$. Namely, $\alpha ^o = 0^\circ$ and $\alpha = 90^\circ$ denote horizontal and vertical polarization observation of radiometer antenna, respectively. Thus $\alpha ^o$ can changes from $-180^\circ$ to $180^\circ$.
Fig. 2. Polarization angle definition in antenna coordinate system. (a) Antenna polarization angle, $\alpha ^o$. $E$ is the electric field direction (E plane) of linear polarization antenna. (b) Azimuth angle of object surface normal vector, $\varphi$, in the view along the observation line.
From Fig. 2(b), there are two coordinate systems: object surface coordinate base $(x, y, z)$ and radiometer antenna coordinate base ($i, h, v$). $z'$ is the projected vector of $z$ on the plane $hv$. Human body and concealed objects have a complex 3D structure. Due to the relative difference between object surface orientation and horizontal plane, there are an included angle between axis $v$ and normal vector $z'$, namely $\varphi$. Therefore, the polarization mode of radiometer antenna is not inconsistent with the polarization mode of object brightness temperature. There is a specific relationship between $\alpha ^o$ and $\alpha$, that is
where $\varphi$ is positive value when $z'$ deflects to negative axis of $h$, while $\varphi$ is negative value when $z'$ deflects to positive axis of $h$. Therefore, considering the effect of clothing, atmosphere and the antenna filtering, the converted brightness temperature $T_{AR\alpha ^o}$ is written asNote that, $T_{AR\alpha ^o}$ in this paper does not represent the antenna radiometric temperature $T_{A\alpha ^o}$ of radiometer which is the distribution integrated over $4\pi$ solid angle according to the antenna weighting function, that is
In summary, due to the polarization filtering of radiometer antennas, the polarized brightness temperature received by radiometers is different from the original one. Therefore, multi-polarization observation can gain additional brightness temperature information.
2.2 Polarization property
Human skin and three types of typical objects (i.e., dielectric solid, metal solid and liquid) have been selected to analyze the polarization properties of brightness temperatures. For liquid objects (gasoline), we have considered the outer packing and the double-layer medium which includes the object and the package. PE package has been taken as example and its thickness is set to 0.2mm. The relative permittivity and conductivity parameters over the $90-100$GHz band are obtained from the published papers [34–36]. For the nontransparent objects, the emissivity is obtained based on Kirchhoff’s law of thermal radiation ($e = 1- r$). We use Fresnel’s equations for the single-layer medium [4]. For the double-layer medium, we use the equivalent emissivity model described in Ref. [4]. Then $T_{ARh}$ and $T_{ARv}$ can be calculated by Eq. (2) and (7).
As shown in Fig. 3(a), H and V polarization emissivities of non-metal materials are significantly different at the most of incident angles, while that of aluminum both approximate to zero. Concealed objects may have complex shape and orientation, so the reflection portion on the object surface is not always from the imaging environment. As shown in Fig. 3(b), we simplify three principal types of observation traces. Different color of right side denotes different trace region. For Trace 1, the first reflection portion on the object surface is from the human body, so $T_{Binc}$ is high and can be set to 304.8K (the approximately average brightness temperature of human skin) for convenient description. For Trace 2, the reflection portion on the object surface is from the imaging environment, so $T_{Binc}$ is low and can be set to 298K (i.e., the indoor brightness temperature at $25^\circ$C). For Trace 3, a majority of skin surfaces meet this case and $T_{Binc}$ is from the imaging environment. From Fig. 3(b), the regions of Trace 3 is much smaller than other regions.
Fig. 3. Calculated emissivities, brightness temperatures ($T_{ARh}, T_{ARv}$) and their differences as a function of incidence angle at 94GHz for four materials. H and V represent horizontal and vertical polarization, respectively. (a) Emissivity. (b) Different observation trace regions. (c) $T_{ARh}$ and $T_{ARv}$ with the low-radiation reflection of Trace 2 and Trace 3. (d) $T_{ARh}$ and $T_{ARv}$ with the high-radiation reflection of Trace 2 and Trace 3. (e) Brightness temperature difference between Trace 2 and Trace 3. (e) Brightness temperature difference between Trace 1 and Trace 3.
The physical temperature of concealed objects $T_{obj}$ is set to 302K ($29^\circ$C), and the physical temperature $T_{skin}$ of human skin is set to 307K ($34^\circ$C) [37]. Figure 3(c) shows $T_{ARh}$ and $T_{ARv}$ of Trace 2 and Trace 3. Figure 3(d) shows $T_{ARh}$ and $T_{ARv}$ of Trace 1 and Trace 3. The brightness temperature of human body is theoretically larger than that of concealed objects at most angles of incidence. In particular, $T_{ARv}$ is larger than $T_{ARh}$ for Trace 2 and Trace 3, while $T_{ARv}$ is smaller than $T_{ARh}$ for Trace 1.
Further, we calculate the brightness temperature differences between human skin and concealed objects. We assume that object surfaces have nearly the same polarization angle and incident angle with their nearby skins. From Figs. 3(e) an 3(f), the difference of vertical polarization is larger and more stable than that of horizontal polarization at most angles of incidence. Note that concealed objects may have various shapes and orientations and then the object surface and its nearby skin may have different linear polarization angle and incident angle. So the above properties are not perfectly strict. In practice, the common concealed objects generally have many surfaces which are nearly parallel to the body skin. Therefore, most surfaces have the similar polarization angle and incident angle with their nearby skins.
2.3 Enhancement method
In the practical application, system noise must be added into the measured brightness temperature. Then the noise fluctuation may cover the difference of horizontal polarization at a large portion of incident angles. Therefore, if all vertical polarization brightness temperatures can be acquired, the concealed objects will be enhanced theoretically.
As analyzed in section 2.1, because there is a deflection angle $\varphi$ between $\alpha ^o$ and $\alpha$ for 3D structure objects, $T_{A\alpha ^o}$ of each pixel in image is generally not equal to the object surface brightness temperature at the polarization angle of $\alpha ^o$. Therefore, we should consider that how to obtain the vertical polarization brightness temperature of each position in observation scene. From formula (10), we can resolve $T_{Av}$ by measuring three $T_{A\alpha ^o}$ of at least three polarization angles. Note that, the precondition of formula (6) is that the incident ambient radiation is unpolarized. In the human security check scene, the reflection environment incident on human body and concealed objects is mainly the indoor cavity in which the radiation can be controlled to close to unpolarized.
In this paper, we call $T_{Av}$ as the complete vertical polarization brightness temperature. In the complete vertical polarization image, all pixels represent the vertical polarization brightness temperatures of observation scenes. Theoretically, at least three different linear polarization brightness temperatures $T_{A\alpha ^o}$, $\alpha ^o_i\in [0, 180), i=1,2,{\ldots }k, k{\geq }3$, can be used to calculate $T_{Av}$ based on the least square fitting algorithm. As a special example, we can capture three polarization data ($T_{A\alpha ^o_i}, i=1,2,3$) with $\alpha ^o_i$ separated by $45^\circ$. According to Eq. (10), The resolved parameters are written as
3. Verification experiments
3.1 Simulation experiment
In recent years, many researchers including us have carried out PMMW image simulation for given typical scenes [38–45]. In the simulation of multi-polarization imaging of human screening, we have considered that: 1) Multiple objects can be inputted and their dielectric property, thickness, physical temperature can be arbitrarily defined; 2) Different layered structures of objects can be calculated with different emissivity models; 3) Add a module for calculating the attenuation and re-radiation of clothing.
Figure 4 shows the 3D model of human body and concealed objects. The 3D model of human body is shown in Fig. 4(a). The grid point graph of 3D models of human and object is shown in Fig. 4(b). We have set six concealed objects under the clothing of man. The concealed objects have been highlighted in red. The material information have been labeled near the objects in Fig. 4(b). Because the influence of clothing is slight at W-waveband (e.g., 94GHz), the attenuation value is set to 0.4dB uniformly [46]. For liquid objects, we have considered their outer packing (taking PE as an example) and the PE thickness is set to 0.2mm. The specific simulation parameters have been listed in Table 1. It is worth noting that the object physical temperature is between the human physical temperature and ambient physical temperature due to the heating effect of human body and clothing.
Table 1. Specific simulation parameters
Figure 5 shows the simulated PMMW images and processing results at the antenna polarization angles of $0^\circ , 45^\circ , 90^\circ , -45^\circ$. From Figs. 5(a)–5(d), the original observed brightness temperature images $T_{A\alpha ^o}$ have the obvious differences in local areas. The set Gaussian noises is $\sigma (\Delta T_B)$ = 1.5K (the standard deviation of brightness temperature fluctuation). From Figs. 5(e)–5(h), the noise fluctuation has been smoothed partly by using Wiener filter [47]. Most obvious fluctuations have been reduced. The metal object (aluminum cylinder) has the lowest brightness temperature which is about equal to the ambient physical temperature. So the contrast between human body and metal object is obvious, and it is easy to detect metal objects. For each $T_{A\alpha ^o}$ image, there are several suspected concealed objects in different positions. It is difficult to detect concealed objects based on arbitrary $T_{A\alpha ^o}$ image.
Fig. 5. Simulated images and processing results. (a)–(d) Original observed brightness temperature images with Gaussian noises of $\sigma (\Delta T_B)$ = 1.5K. Four polarization angles include $\alpha ^o_1=0^\circ$, $\alpha ^o_2=45^\circ$, $\alpha ^o_3=90^\circ$, $\alpha ^o_4=-45^\circ$. (e)–(h) Denoised images with Wiener filter. (i) Enhanced brightness temperature image $T_{Av}$. (j)–(l) Stokes parameter images.
By using three polarization images ($T_{A0^\circ }, T_{A45^\circ }, T_{A90^\circ }$), the complete vertical polarization image $T_{Av}$ can be calculated and shown in Fig. 5(i). From the enhanced image, the contrast between human body and concealed objects has been obviously improved and all suspected positions have been eliminated. For example, the wood knife is obvious in Fig. 5(e) but is blurry in Figs. 5(f) and 5(g). The gasoline cuboid is is blurry in Fig. 5(e) but is obvious in Figs. 5(f) and 5(g).
In addition, as shown in Figs. 5(j)–5(l), we have calculated the common Stokes parameters to compare with our enhanced image. In this paper, we fuse multiple linear polarization images, so we only discuss the Stokes parameters of linear polarization, i.e., $S_0$, $S_1$, $S_2$. From Fig. 5(j), due to the average operation, $S_0$ of the human body has a more uniform distribution than any single-polarization image. Qualitatively, the enhanced image has a higher contrast than $S_0$. In particular, the concealed objects are nearly invisible in $S_1$ and $S_2$ images. Most positions of the human body and concealed objects have a small angle of incidence, so the orthogonal polarization difference is small. Conversely, the contour edges of human body are visible due to the big brightness temperature difference between orthogonal polarization states. The positive and negative property of $S_1$ and $S_2$ represents the surface orientation feature. For $S_1$, the positive number means that the surface orientation tends to the horizontal direction, while the negative number implies that the surface orientation tends to the vertical direction. For $S_2$, the positive number means that the surface orientation tends to the $-45^\circ$ direction, while the negative number implies that the surface orientation tends to the $45^\circ$ direction.
3.2 Measurement experiment
To further verify the presented enhancement method, the real indoor measurement experiments have been carried out. Figure 6(a) shows the human body and concealed objects. Several objects (e.g., knife, vice, water bottle) have been hidden under the clothing on the human body. The background of human is constructed by wedge absorbing materials and foamed materials, so the mmW radiation of surrounding is approximately unpolarized and uniform. The air physical temperature of imaging scene is 294.2K. The average physical temperature of human body is about 307.1K. The center frequency is 94GHz and the bandwidth is 4GHz. The radiometric sensitivity is better than 1K/5ms. The system spatial resolution of focal spot is about 2cm. The imaging distance is 3m. Four images of linear polarization brightness temperature were captured by a 94GHz multi-polarization scanning imaging radiometer at four antenna polarization angles of $0^\circ$, $45^\circ$, $90^\circ$, $-45^\circ$. In our measurements, the radiometer can simultaneously obtain two orthogonal polarization images. By rotating the radiometer around the observation axis, we can acquire an image at any linear polarization angle. Due to the system limitation, we obtained four polarization images by time-sharing sampling. To ensure the consistency of the imaging scene, the male volunteer tried to keep the same posture during measurements. The theoretical precondition of the proposed method is that $T_{Binc}$ is approximately unpolarized. To ensure the unpolarized condition, a man-made surrounding with absorbing blackbody materials was set up, which is similar to the structure of microwave anechoic chamber. The male volunteer stood inside the man-made surrounding.
Figures 7(a)–7(d) show the original observed brightness temperature images. The distributing characteristics of human brightness temperatures is similar to that of simulations. Taking the head area as an example, we compare different polarization images. For $T_{A0^\circ }$ image, the left and right parts have higher brightness temperatures than the upper and lower parts. Because the polarization state of left and right surfaces is vertical for the horizontal polarization observation ($\alpha ^o=0^\circ$). Consequently, the areas of high brightness temperatures rotates $45^\circ$ for $T_{A45^\circ }$ image, and rotates $90^\circ$ for $T_{A90^\circ }$ image. The similar characteristics are also obviously embodied in shoulders and upper chest. It’s worth noting that the handle of vice is almost invisible in the measurement images. This phenomenon is primarily caused by its increasing physical temperature, non-metal material and small size. The vice has been covered by clothing for a long time before finishing experiments, so its physical temperature is close to that of body surface. Therefore, the non-metal vice handle has a high brightness temperature level comparable to body surface.
Fig. 7. Measured images and processing results. (a)–(d) Original observed brightness temperature images. Four polarization angles include $\alpha ^o_1=0^\circ$, $\alpha ^o_2=45^\circ$, $\alpha ^o_3=90^\circ$, $\alpha ^o_4=-45^\circ$. (e)–(h) Denoised images with Wiener filter. (i) Enhanced brightness temperature image $T_{Av}$. (j)–(l) Stokes parameter images.
Figures 7(e)–7(h) show the denoised brightness temperature images. The noise fluctuations have been reduced and the brightness temperature images have been smoothed obviously. So the concealed objects have enhanced partly. For each $T_{A\alpha ^o}$ image, there are more than four suspected concealed objects. The water bottle is not very obvious in the $T_{A90^\circ }$ image.
Figure 7(i) shows the enhanced image by using our method. The overall brightness temperatures of human body have been obviously improved. Therefore, the fluctuations of human background are reduced, and the contrast between human body and concealed objects becomes large and stable. Additionally, as shown in Fig. 7(j)–7(l), the common Stokes parameters have also been calculated to compare. The distribution characteristics of measured results are similar to that of simulated results. The enhanced image has a highest contrast qualitatively.
4. Discussion
4.1 Performance analyses
From the above experimental results, the method presented in this paper can fuse the advantages of multiple polarization images and improve the consistency of brightness temperature contrast. To quantitatively analyze the enhancement performance, we have defined a Differential Signal Noise Ratio (DSNR) to evaluate the contrast and its stability. The evaluation index DSNR is defined by
The calculated DSNR results of simulation and measurement experiments have been listed in Table 2 and Table 3. Because the concealed objects are nearly invisible in $S_1$ and $S_2$ images, we have not listed their DSNR to compare. The data with blue color denote the obviously low DSNR. The boldface data denote the highest DSNR. The original images have the lowest DSNR. For most cases, $S_0$ image has the higher DSNR than any single-polarization image. Obviously, the enhanced images have the highest DSNR. DSNR index is closely related to object structure, object dielectric property and hiding position. From the detailed DSNR data, several characteristics can be found:
Table 2. DSNR of simulation experiment (unit: dB)
Table 3. DSNR of measurement experiment (unit: dB)
(1) Different polarization has different DSNR. The data with blue color may occur in each polarization. There is no single polarization that can always obtain a high DSNR. Therefore, arbitrary single polarization has certain limitations. By fusing multiple polarization information, DSNR can be obviously improved.
(2) Different object has different enhancement performance. The dielectric property and outline structure of the object directly affect the brightness temperature level and the polarization difference. For the object with large polarization difference (e.g., most non-metal materials and inclined surfaces with a large incident angle), the resolved $T_{Av}$ of the object can lead to a high $\langle T_{A,object}\rangle$, which reduces DSNR. But the increased human $T_{Av}$ ($\langle T_{A,human}\rangle$) and the reduced background noise ($\sigma (T_{A,human})$) can compensate this loss and obtain a high DSNR. For the object with no or very small polarization difference (e.g., metal materials), $\langle T_{A,object}\rangle$ remains almost unchanged. So its DSNR is improved more obvious than that of the former.
(3) Different concealed position has different enhancement performance. Human body has a complex 3D structure. Some positions have a large incident angle (e.g., $40^\circ \sim 85^\circ$), such as chest edge, shoulders, and legs. From Fig. 3(b), the human body position with a large incident angle has a large polarization difference, and the resolved $T_{Av}$ of human body is more stable than $T_{Ah}$ at the incident angles of $40^\circ \sim 85^\circ$. Consequently, for the concealed position with a large polarization difference, we can get an obviously large $\langle T_{A,human}\rangle$ and an obviously small $\sigma (T_{A,human})$ from the resolved $T_{Av}$. Therefore, a better enhancement performance can be obtained for the concealed positions with a large incident angle.
4.2 Applicability analyses
It’s worth noting that there is a nonideal problem for the human body positions with multiple reflections (e.g., armpits, the inner sides of hands in the measurement experiments as shown in Fig. 6(b)). These areas have multiple reflected brightness temperatures which are not meet the condition of Eq. (10). The enhancement method almost has no effect on these areas. Therefore, for the metal knife in the measurement experiments, the evaluated DSNR of the enhanced image is not much more outstanding than that of the original and denoised images. In fact, the position of the metal knife can obtain a good enhancement performance theoretically. In order to ensure the applicability of collaborative inspections, it is necessary to require the checked human to open their arms and legs. For example, the standing posture in the simulation experiments is a good demonstration.
Concealed objects may have complex shape and orientation. We should note that Fig. 3(b) has only considered three principal types. In practice, some object surface may have sunken regions, as shown in Fig. 8(a), so the first reflection will be itself and then the indoor environment (even have multiple reflections). This case usually have the nearly equal horizontal and vertical polarization brightness temperature. So our method nearly has no negative impact on image quality at least. Besides, as shown in Fig. 8(b), the regions of Trace 2 may have the surfaces whose incident angle is very different from that of the body skin. The largest difference of incident angle is $45^\circ$. So Fig. 3(e) cannot describe this case. In fact, this case usually occurs in the small regions (close to the contour edges) of common dangerous objects. Therefore, this case is similar to Trace 1 and is a limitation for our method.
Our method enhances the image contrast by fusing multi-polarization images, so we should simultaneously acquire multi-polarization images to ensure the consistency of imaging scenarios. In our measurements, we just developed an experimental system by time-sharing sampling, which cannot be directly used to the practical applications. For the exiting single-polarization focal plane imaging radiometers, we can add a mechanical device to rotate all array elements. This approach will reduce the frame rate, so it may be not applicable to the real-time scenarios. In fact, a fully polarimetric imaging radiometer can simultaneously acquire four linear polarization images and two circular polarization images [48]. The key components include an ortho mode transducer (OMT) and a phase network for correlation. Therefore, if we develop a focal plane imaging system by taking this similar radiometer as the array element, our method can be applicable to imaging scenarios in which the people are walking.
5. Conclusion
In order to relieve the problem of radiometer performance lack and obtain more information of observation scenes, a physically-based enhancement method by using multi-polarization information is proposed. The multi-polarization brightness temperature model of human body and concealed objects is analyzed. The clothing and atmosphere are treated as the attenuation media. By investigate the physical principle of polarization property difference between human body and concealed objects, we have found that, for the complete vertical polarization, there is a great and stable contrast between human body and concealed objects. The conversion method to obtain the complete vertical polarization image has been presented. From simulation and measurement experiment results, the enhancement method presented in this paper is simple and effective, which fuses the advantages of multiple polarization images. By extracting the complete vertical polarization brightness temperatures of all pixels, the contrast consistency of human and concealed objects can be improved overall. Additionally, the suspected positions can be largely eliminated in the enhanced image.
The results suggest that the presented method might be also applied to other applications, not just security screening. For example, in the remote target detection, the interesting targets always move or fly in the complex terrain background. The presented method can also enhance the contrast between target and background. It is worth mentioning that the main price of the presented method is the possible increasing of system hardware cost. The integration technique of mmW/THz devices becomes more and more mature, so the multi-polarization imaging technique is worth applying to practical systems. The multi-polarization system with low cost and fast imaging ability will been developed in future work. $S_3$ (circular polarization) of Stokes parameters can add more information about shape and orientation, which is valuable for enhancement and detection. This issue will be also further investigated in future.
Funding
China Postdoctoral Science Foundation (2019M660640); National Natural Science Foundation of China (61901242).
Acknowledgments
The authors would like to thank Dr. Dong Zhu from Tsinghua University, Dr. Yan Hu from Huazhong University of Science and Technology for discussing data processing, and Prof. Harvey Rutt from University of Southampton for helpful comments.
Disclosures
The authors declare no conflicts of interest.
References
1. L. Yujiri, M. Shoucri, and P. Moffa, “Passive millimeter-wave imaging,” IEEE Microw. Mag. 4(3), 39–50 (2003). [CrossRef]
2. M. R. Fetterman, J. Grata, G. Jubic, W. L. Kiser, and A. Visnansky, “Simulation, acquisition and analysis of passive millimeter-wave images in remote sensing applications,” Opt. Express 16(25), 20503–20515 (2008). [CrossRef]
3. R. Appleby and R. N. Anderton, “Millimeter-wave and submillimeter-wave imaging for security and surveillance,” Proc. IEEE 95(8), 1683–1690 (2007). [CrossRef]
4. F. T. Ulaby, D. G. Long, W. J. Blackwell, C. Elachi, A. K. Fung, C. Ruf, K. Sarabandi, H. A. Zebker, and J. Van Zyl, “Microwave radar and radiometric remote sensing (University of Michigan, 2014).
5. Y. Su, S. Zhang, X. Shao, and J. Yang, “The dense filamentary giant molecular cloud g23. 0–0.4: birthplace of ongoing massive star formation,” ApJ 811(2), 134 (2015). [CrossRef]
6. M. Martín-Neira, D. LeVine, Y. Kerr, N. Skou, M. Peichl, A. Camps, I. Corbella, M. Hallikainen, J. Font, J. Wu, S. Mecklenburg, and M. Drusch, “Microwave interferometric radiometry in remote sensing: An invited historical review,” Radio Sci. 49(6), 415–449 (2014). [CrossRef]
7. G.-P. Hu, K. L. Chan, Y.-C. Zheng, and A.-A. Xu, “A rock model for the cold and hot spots in the chang e microwave brightness temperature map,” IEEE Trans. Geosci. Remote Sensing 56(9), 5471–5480 (2018). [CrossRef]
8. S. Yeom, D. Lee, H. Lee, J. Son, and V. P. Gushin, “Distance estimation of concealed objects with stereoscopic passive millimeter-wave imaging,” Prog. Electromagn. Res. 115, 399–407 (2011). [CrossRef]
9. N. A. Salmon, “3-d radiometric aperture synthesis imaging,” IEEE Trans. Microwave Theory Tech. 63(11), 3579–3587 (2015). [CrossRef]
10. M. Kowalski, M. Kastek, M. Walczakowski, N. Palka, and M. Szustakowski, “Passive imaging of concealed objects in terahertz and long-wavelength infrared,” Appl. Opt. 54(13), 3826–3833 (2015). [CrossRef]
11. V. G. Kolinko, S.-H. Lin, A. Shek, W. Manning, C. Martin, M. Hall, O. Kirsten, J. Moore, and D. A. Wikner, “A passive millimeter-wave imaging system for concealed weapons and explosives detection,” in “Proceedings of SPIE, Optics and Photonics in Global Homeland Security,” vol. 5781 (International Society for Optics and Photonics, 2005), vol. 5781, pp. 85–92.
12. N. M. Vaidya and T. Williams, “A novel approach to automatic threat detection in mmw imagery of people scanned in portals,” in “Proceedings of SPIE, Passive Millimeter-Wave Imaging Technology XI,” vol. 6948 (International Society for Optics and Photonics, 2008), vol. 6948, p. 69480F.
13. H. Sato, K. Sawaya, K. Mizuno, J. Uemura, M. Takeda, J. Takahashi, K. Yamada, K. Morichika, T. Hasegawa, H. Hirai, H. Niikura, T. Matsuzaki, and J. Nakada, “Development of 77 ghz millimeter wave passive imaging camera,” in “IEEE Sensors 2009 Conference,” (IEEE, 2009), pp. 1632–1635.
14. C. Zheng, X. Yao, A. Hu, and J. Miao, “Initial results of a passive millimeter-wave imager used for concealed weapon detection bhu-2d-u,” Prog. Electromagn. Res. 43, 151–163 (2013). [CrossRef]
15. S. Yeom, D.-S. Lee, Y. Jang, M.-K. Lee, and S.-W. Jung, “Real-time concealed-object detection and recognition with passive millimeter wave imaging,” Opt. Express 20(9), 9371–9381 (2012). [CrossRef]
16. V. A. Trofimov, V. V. Trofimov, I. E. Kuchik, and I. L. Shestakov, “New algorithm for detection of dangerous objects hidden on a human body using passive thz camera,” in “Proceedings of SPIE, Millimetre Wave and Terahertz Sensors and Technology IX,” vol. 9993 (International Society for Optics and Photonics, 2016), vol. 9993, p. 999305.
17. W. Yu, X. Chen, and L. Wu, “Segmentation of concealed objects in passive millimeter-wave images based on the gaussian mixture model,” J. Infrared, Millimeter, Terahertz Waves 36(4), 400–421 (2015). [CrossRef]
18. H. Fang and L. Yan, “Parametric blind deconvolution for passive millimeter wave images with framelet regularization,” Optik 125(3), 1454–1460 (2014). [CrossRef]
19. S. López-Tapia, R. Molina, and N. P. de la Blanca, “Using machine learning to detect and localize concealed objects in passive millimeter-wave images,” Eng. Appl. Artif. Intel. 67, 81–90 (2018). [CrossRef]
20. M. Kowalski, “Real-time concealed object detection and recognition in passive imaging at 250 ghz,” Appl. Opt. 58(12), 3134–3140 (2019). [CrossRef]
21. S. H. Yueh, W. J. Wilson, F. K. Li, S. V. Nghiem, and W. B. Ricketts, “Polarimetric brightness temperatures of sea surfaces measured with aircraft k-and ka-band radiometers,” IEEE Trans. Geosci. Remote Sensing 35(5), 1177–1187 (1997). [CrossRef]
22. P. S. Narvekar, G. Heygster, T. J. Jackson, R. Bindlish, G. Macelloni, and J. Notholt, “Passive polarimetric microwave signatures observed over antarctica,” IEEE Trans. Geosci. Remote Sensing 48(3), 1059–1075 (2010). [CrossRef]
23. J. P. Wilson, C. A. Schuetz, T. E. Dillon, D. L. Eng, S. Kozacik, and D. W. Prather, “Display of polarization information for passive millimeter-wave imagery,” Opt. Eng. 51(9), 091607 (2012). [CrossRef]
24. F. Hu, Y. Cheng, L. Gui, L. Wu, X. Zhang, X. Peng, and J. Su, “Polarization-based material classification technique using passive millimeter-wave polarimetric imagery,” Appl. Opt. 55(31), 8690–8697 (2016). [CrossRef]
25. J. Su, Y. Tian, F. Hu, Y. Cheng, and Y. Hu, “Material clustering using passive millimeter-wave polarimetric imagery,” IEEE Photonics J. 11(1), 1–9 (2019). [CrossRef]
26. Y. Cheng, F. Hu, Y. Hu, Z. Zhao, and Y. Wang, “C-curve feature of complex permittivity estimation based on multi-polarization measurement in passive millimeter-wave sensing,” Opt. Lett. 44(15), 3765–3768 (2019). [CrossRef]
27. H. Suess and M. Soellner, “Fully polarimetric measurements of brightness temperature distributions with a quasi-optical radiometer system at 90 ghz,” IEEE Trans. Geosci. Remote Sensing 43(5), 1170–1179 (2005). [CrossRef]
28. Y. Cheng, F. Hu, L. Gui, L. Wu, and L. Lang, “Polarization-based method for object surface orientation information in passive millimeter-wave imaging,” IEEE Photonics J. 8(1), 1–12 (2016). [CrossRef]
29. W.-G. Kim, N.-W. Moon, H.-K. Kim, and Y.-H. Kim, “Linear polarization sum imaging in passive millimeter-wave imaging system for target recognition,” Prog. Electromagn. Res. 136, 175–193 (2013). [CrossRef]
30. Y. Cheng, F. Hu, L. Gui, J. Su, B. Qi, S. Liu, and M. Huang, “Linear polarisation property and fusion method for target recognition in passive millimetre-wave polarimetric imaging,” Electron. Lett. 52(14), 1221–1223 (2016). [CrossRef]
31. J. Tedeschi, B. Bernacki, D. Sheen, J. Kelly, and D. McMakin, “Fully polarimetric passive w-band millimeter wave imager for wide area search,” in “Proceedings of SPIE, Polarization Science and Remote Sensing VI,” vol. 8873 (International Society for Optics and Photonics, 2013), vol. 8873, pp. 88730V–1–88730V–14.
32. L. Wu, J. Zhu, S. Peng, Z. Xiao, and Y. Wang, “Post-processing techniques for polarimetric passive millimeter wave imagery,” Appl. Comput. Electrom. 33(5), 512–518 (2018).
33. Y. Cheng, F. Hu, H. Wu, P. Fu, and Y. Hu, “Multi-polarization passive millimeter-wave imager and outdoor scene imaging analysis for remote sensing applications,” Opt. Express 26(16), 20145–20159 (2018). [CrossRef]
34. J. W. Lamb, “Miscellaneous data on materials for millimetre and submillimetre optics,” Int. J. Infrared Millimeter Waves 17(12), 1997–2034 (1996). [CrossRef]
35. S. Alekseev and M. Ziskin, “Human skin permittivity determined by millimeter wave reflection measurements,” Bioelectromagnetics 28, 331–339 (2007). [CrossRef]
36. E. Arik, H. Altan, and O. Esenturk, “Dielectric properties of ethanol and gasoline mixtures by terahertz spectroscopy and an effective method for determination of ethanol content of gasoline,” J. Phys. Chem. A 118(17), 3081–3089 (2014). [CrossRef]
37. R. Appleby and S. Ferguson, “Submillimetre wave imaging and security: imaging performance and prediction,” in “Proceedings of SPIE, Millimetre Wave and Terahertz Sensors and Technology IX,” vol. 9993 (International Society for Optics and Photonics, 2016), vol. 9993, pp. 999302–1–999302–12.
38. N. A. Salmon, “Polarimetric passive millimeter-wave imaging scene simulation including multiple reflections of subjects and their backgrounds,” in “Proceedings of SPIE, Passive Millimetre-Wave and Terahertz Imaging II,” vol. 5989 (International Society for Optics and Photonics, 2005), vol. 5989, p. 598917.
39. N. A. Salmon, “Outdoor passive millimeter-wave imaging: Phenomenology and scene simulation,” IEEE Trans. Antennas Propag. 66(2), 897–908 (2018). [CrossRef]
40. C. Zhang and J. Wu, “Image simulation for ground objects microwave radiation,” Journal of Electronics and Information Technology 29, 2425–2728 (2007).
41. M. Murakowski, J. Wilson, J. Murakowski, G. Schneider, C. Schuetz, and D. Prather, “3d rendering of passive millimeter-wave scenes using modified open source software,” in “Proceedings of SPIE, Passive Millimeter-Wave Imaging Technology XIV,” vol. 8022 (International Society for Optics and Photonics, 2011), vol. 8022, p. 80220B.
42. C. Yin, M. Zhang, and Y. Bo, “Multilayer brightness temperature tracing method for rough surface scene simulation in passive millimeter-wave imaging,” Int. J. Antenn. Propag. 2016, 1–13 (2016). [CrossRef]
43. B. Qi, L. Lang, Y. Cheng, S. Liu, F. Hu, X. He, P. Deng, and L. Gui, “Passive millimeter-wave scene imaging simulation based on fast ray-tracing,” in “2016 IEEE International Geoscience and Remote Sensing Symposium (IGARSS),” (IEEE, 2016), pp. 2642–2645.
44. Y. Cheng, F. Hu, L. Gui, Y. Hu, and Z. Han, “Passive millimeter-wave polarization characteristics of several common structures,” in “Proceedings of SPIE, Polarization Science and Remote Sensing VIII,” vol. 10407 (International Society for Optics and Photonics, 2017), vol. 10407, p. 1040711.
45. Y. Hu, F. Hu, Y. Cheng, J. Su, and X. Zhang, “Passive mmw radiation imaging simulation in human security,” in “Proceedings of National Microwave Millimeter wave Conference, China,” (Chinese Institute of Electronics, 2018), pp. 1141–1144.
46. A. J. Gatesman, A. Danylov, T. M. Goyette, J. C. Dickinson, and W. Hoen, “Terahertz behavior of optical components and common materials,” in “Proceedings of SPIE, Terahertz for Military and Security Applications IV,” vol. 6212 (International Society for Optics and Photonics, 2006), vol. 6212, pp. 62120E–1–62120E–12.
47. M. D. Ramírez, C. R. Dietlein, E. Grossman, and Z. Popović, “Unsupervised image segmentation for passive thz broadband images for concealed weapon detection,” in “Proceedings of SPIE, Terahertz for Military and Security Applications V,” vol. 6549 (International Society for Optics and Photonics, 2007), vol. 6549, pp. 65490J–1–65490J–7.
48. A. Duric, A. Magun, A. Murk, C. Matzler, and N. Kampfer, “The fully polarimetric imaging radiometer spira at 91 GHz,” IEEE Trans. Geosci. Remote Sensing 46(8), 2323–2336 (2008). [CrossRef]
