Millimeter wave imaging is finding rapid adoption in security applications such as the detection of objects concealed under clothing. A passive imaging system can be realized as a stand-off type sensor that can operate in open spaces, both indoors and outdoors. In this paper, we address real-time outdoor concealed-object detection and segmentation with a radiometric imaging system operating in the W-band. The imaging system is equipped with a dielectric lens and a receiver array operating at around 94 GHz. Images are analyzed by multilevel segmentation to identify a concealed object. Each level of segmentation comprises vector quantization, expectation-maximization, and Bayesian decision making to cluster pixels on the basis of a Gaussian mixture model. In addition, we describe a faster process that adopts only vector quantization for the first level segmentation. Experiments confirm that the proposed methods provide fast and reliable detection and segmentation for a moving human subject carrying a concealed gun.
©2011 Optical Society of America
Millimeter wave (MMW) imaging readily penetrates fabrics and is useful to detect objects concealed under clothing [1–4]. It has thus been developed for various commercial and military applications [1,5–7]. Passive MMW imaging generates interpretable images in low-visibility conditions such as fog, rain, dust, and smoke. One application of the passive MMW imaging system is a stand-off type sensor that can scan people moving in an unconstrained flow . Unfortunately, in these applications, image quality is often degraded by low-level signals and system noise [8–10]. Low spatial resolution due to a limited aperture size is another hindrance to obtaining high-quality images. There have been efforts to develop automatic detection and recognition of concealed objects with passive MMW imaging [11–14]. Multilevel thresholding was addressed to detect concealed objects in terahertz (THz) images . The metallic object detection with a Gaussian mixture model (GMM) has been researched in . Multilevel expectation-maximization (EM) has been proposed to cluster pixels according to GMM . However, this method requires manual initialization of the GMM parameters. The longitudinal distance of concealed objects is estimated by the corresponding segmentation-based features in .
This paper addresses real-time detection and segmentation of concealed objects under clothing in an outdoor open space. Our passive imaging system operates around 94 GHz to generate images with a 1 Hz frame rate. Automatic recognition of detected objects is achieved by multilevel segmentation. The multilevel segmentation comprises the global (first level) and local (second level) segmentation. The global segmentation separates the subject’s body area from the background area. During the local segmentation, only the inside of the body area is processed and the concealed object is segmented from the body area. This multilevel segmentation is improved from , adopting a series of clustering (unsupervised learning) methods such as vector quantization (VQ), EM algorithm for the GMM parameter estimation, and Bayesian decision rule. The VQ algorithm initializes the parameters of the GMM at each segmentation level. The EM process estimates the GMM parameters iteratively and the Bayesian decision rule decides which cluster each pixel belongs to. An alternative method is also developed for faster processing, which utilizes only the VQ during the global segmentation. The experimental and simulation results show that two proposed methods provide fast and reliable detection and segmentation of the object concealed in a moving human subject. The alternative method (shortened process) is able to reduce the average computational time by 87.9% from the fully-processed method (full process). However, the full process provides better segmentation performance than the shortened process.
The paper is organized as follows: In Section 2, we briefly describe a real-time outdoor stand-off, passive MMW imaging system. The multilevel segmentation method is discussed in Section 3. In Section 4, we present the experimental and simulation results. Conclusion follows in Section 5.
2. A passive millimeter wave imaging system
The passive MMW imaging system consists of a high-density polyethylene lens with a diameter of 50 cm, reflective mirror, one-dimensional (1D) receiver array composed of 30 receiver channels, and a mechanical scanner, which rotates the receiver array . The focal length and the field of view (FOV) of the lens are 500 mm and 17 × 17°, respectively, thus the 30-channel 1D receiver array corresponds to 30 pixels in the range of 17°. The incoming W-band electromagnetic wave is focused on the feed antennas of a 1D receiver array located at the focal plane of the lens. Figures 1(a) and 1(b) show the passive MMW imaging system and a block diagram of the quasi-optical configuration, respectively. Each receiver channel is composed of a dielectric rod antenna, four low noise amplifiers, and a detector. The measured receiver gain is more than 45 dB in the desired frequency band. There are three factors in determining the signal sensitivity: the receiver bandwidth, the noise figure of the amplifier, and the integration time . This passive MMW imaging system has 10 mrad spatial resolution and 1.5 K temperature resolution. The integration time of one channel is 10 msec.
3. Image segmentation for concealed object recognition
The overall segmentation process is composed of preprocessing, global and local segmentation, and body area erosion as illustrated in Fig. 2 . Each segmentation level is composed of the VQ process, EM algorithm, and Bayesian decision rule. The morphological erosion of the body area is performed in order to estimate the accurate contour of the body area. The background area is eliminated during the windowing process after the global segmentation. Alternatively, the EM algorithm and the Bayesian decision process are removed during the global segmentation for faster processing. The removed stages for the shortened process are shown in dotted lines in Fig. 2.
3.1 VQ process
The VQ process generates two clusters from a passive MMW image. The VQ process also initializes the parameters of the GMM in the full process while it only clusters pixels in the shortened process. The VQ in this research is the k-means algorithm equipped with a splitting initialization . Figure 3 shows the VQ procedure to generate two clusters and from a vector set . The initial vector is the mean of an input vector set S, Δ is a splitting vector, and is the number of pixels in the image. In this paper, each vector in the set S corresponds to a pixel’s intensity, thus scalar quantization is actually performed. The final and are clusters corresponding to each of the segmented areas. If it is desired to obtain more than two clusters this process can repeat. After the VQ process, the GMM parameters are initialized by the sample means, sample variances, and sample proportions of the clusters, and .
3.2 EM algorithm and Bayesian decision process
The histogram is modeled with a mixture of Gaussian distributions as follows:
where and are the mean and the variance of the component (cluster) k, respectively, Gk represents the event of component k, is the number of components which is set 2 in this paper, and Ndenotes the Gaussian probability density function. The EM algorithm iteratively estimates the parameters; , , and each per cluster. Figure 4 shows the block diagram of the EM algorithm; ε is the termination criteria, and is the log-likelihood; . More detailed expectation and maximization equations with GMM are found in [13,16].
The Bayesian decision rule uses the posterior probability density functions which are obtained by the EM algorithm. Each pixel is assigned to one of two clusters generating a binary image as
3.3 Performance evaluation
The average probability of error  is adopted to evaluate the segmentation performance as follows:
where and indicate the segmented area and the ground truth area of the concealed object, respectively, I is the entire image, and denotes the number of pixels in the area.
4. Experimental and simulation results
The passive MMW images of a human subject hiding a gun (Colt 45 – M1911A1) are captured for 5 seconds with a 1 Hz frame rate. Figure 5(a) is the picture of the gun, and Fig. 5(b) is a visual movie of the human subject. Figure 6(a) is the movie of the passive MMW images capturing the same human subject. These passive MMW images are resized to be 3 times larger than the original size using the bicubic interpolation during preprocessing, thus the image size becomes pixels. It is noted that the integration time of one channel is 10 msec, thus the total integration time is 240 msec for an image of 24 × 30 pixels. Figures 6(b) and 6(c) are the segmentation results by the full and the shortened process, respectively.
Figure 7 illustrates the segmentation results of the 3rd frame by the full process. Figures 7(a) and 7(b) are the GMM fitting, overlapped on the normalized histogram after the global and local segmentation, respectively. Figures 7(c) and 7(d) show the body and the concealed object area after the global and the local segmentation stage, respectively. It is noted that the histogram in Fig. 7(b) represents only the inside of the body area demonstrated in Fig. 7(d) since the background is excluded by the windowing process after the global segmentation. Figures 8(a) and 8(b) show the average probability of error and the computational time of the full and the shortened process, respectively. The average computational time of the shortened process is 0.134 sec while that of the full process is 1.11 sec, thus the computational time is reduced by 87.9%. The number of iteration is 2 for each level of segmentation. The number of iteration depends on the initial parameters of the GMM including the number of clusters. It is noted that all the programs are implemented with MATLAB 7.8 on a standard computer with a dual core 2.66 GHz CPU. Thus, the program is expected to run faster when specialized hardware is used. The average probability of error is increased from 0.0041 to 0.007 by about 1.7 times. The computational time can be significantly reduced by eliminating the EM algorithm during the global segmentation. However, the segmentation performance is degraded because only the VQ is used to segment the body and the background area. Although the same clustering methods are used during the local segmentation it can be seen that the performance is highly dependent of the global segmentation results.
In this paper, we have presented a real-time, concealed object detection and segmentation process using a passive imaging system and multilevel segmentation. The imaging system generates frames every second by scanning a human subject in an outdoor area. The multilevel segmentation is improved with the VQ initialization to realize the automatic recognition of a concealed object. We have also developed a faster process to significantly reduce computational time. This system can be expanded to scan a flow of people even more than one person at a time. Additional investigation of the proposed system which includes identifying multiple objects from a flow of people remains for future studies. We also leave the classification of concealed objects in the future research topic.
This work was supported by the SamsungThales and Mid-career Researcher Program through NRF grant funded by the MEST (No. 2009-0083786).
References and links
1. L. Yujiri, M. Shoucri, and P. Moffa, “Passive millimeter-wave imaging,” IEEE Microw. Mag. 4(3), 39–50 (2003). [CrossRef]
2. H.-M. Chen, S. Lee, R. M. Rao, M.-A. Slamani, and P. K. Varshney, “Imaging for concealed weapon detection: a tutorial overview of development in imaging sensors and processing,” IEEE Signal Process. Mag. 22(2), 52–61 (2005). [CrossRef]
3. National Research Council, Assessment of millimeter-wave and terahertz technology for detection and identification of concealed explosives and weapons (National Academies Press, Washington, D.C., 2007).
4. R. Appleby and R. N. Anderton, “Millimeter-wave and submillimeter-wave imaging for security and surveillance,” Proc. IEEE 95(8), 1683–1690 (2007). [CrossRef]
5. M. R. Fetterman, J. Grata, G. Jubic, W. L. Kiser Jr, and A. Visnansky, “Simulation, acquisition and analysis of passive millimeter-wave images in remote sensing applications,” Opt. Express 16(25), 20503–20515 (2008). [CrossRef] [PubMed]
6. F. Buersgens, G. Acuna, and R. Kersting, “Millimeter wave probing of the acoustic phase for concealed object detection,” Opt. Express 15(14), 8838–8843 (2007), http://www.opticsinfobase.org/oe/abstract.cfm?URI=oe-15-14-8838. [CrossRef] [PubMed]
7. A. Denisov, A. Gorishnyak, S. Kuzmin, V. Miklashevich, V. Obolonskv, V. Radzikhovsky, B. Shevchuk, B. Yshenko, V. Uliz’ko, and J. Son, “Some experiments concerning resolution of 32 sensors passive 8 mm wave imaging system,” in Proceeding of the 20th International Symposium on Space Terahertz Technology, ISSTT 2009 (National Radio Astronomy Observatory, 2009), pp. 227–229.
8. K. B. Cooper, R. J. Dengler, N. Llombart, T. Bryllert, G. Chattopadhyay, I. Mehdi, and P. H. Siegel, “An approach for sub-second imaging of concealed objects using terahertz (THz) radar,”J. Infrared, Millimeter Terahertz Waves 30, 1297–1307 (2009).
9. H. Sato, K. Sawaya, K. Mizuno, J. Uemura, M. Takeda, J. Takahashi, K. Yamada, K. Morichika, T. Hasegawa, H. Hirai, H. Niikura, T. Matsuzaki, S. Kato, and J. Nakada, “Passive millimeter-wave imaging for security and safety applications,” Proc. SPIE 7672, 76720V (2010).
10. M.-K. Jung, Y.-S. Chang, S.-H. Kim, W.-G. Kim, and Y.-H. Kim, “Development of passive millimeter wave imaging system at W-band,” in Proceedings of the 34th International Conference on Infrared, Millimeter, and Terahertz Waves, IRMMW-THz 2009 (IEEE Microwave Theory and Techniques Society, 2009), pp. 1–2.
11. X. Shen, C. R. Dietlein, E. Grossman, Z. Popovic, and F. G. Meyer, “Detection and segmentation of concealed objects in terahertz images,” IEEE Trans. Image Process. 17(12), 2465–2475 (2008). [CrossRef] [PubMed]
12. C. D. Haworth, Y. de Saint-Pern, D. Clark, E. Trucco, and Y. R. Petillot, “Detection and tracking of multiple metallic objects in millimeter-wave images,” Int. J. Comput. Vis. 71(2), 183–196 (2007). [CrossRef]
13. D.-S. Lee, S. Yeom, J. Y. Son, and S.-H. Kim, “Automatic image segmentation for concealed object detection using the expectation-maximization algorithm,” Opt. Express 18(10), 10659–10667 (2010). [CrossRef] [PubMed]
14. S. Yeom, D.-S. Lee, H. Lee, J. Son, and V. P. Guschin, “Three-dimensional passive millimeter-wave imaging and depth estimation,” Proc. SPIE 7690, 76900W, 76900W-6 (2010). [CrossRef]
15. A. Gersho and R. M. Gray, Vector Quantization and Signal Compression (Kluwer Academic Publishers, Boston, MA., 1992).
16. C. M. Bishop, Neural Networks for Pattern Recognition (Oxford University Press, 1995), Chap. 2.