1. Introduction

Millimeter wave (MMW) imaging readily penetrates fabrics and is useful to detect objects concealed under clothing [1–4]. It has thus been developed for various commercial and military applications [1,5–7]. Passive MMW imaging generates interpretable images in low-visibility conditions such as fog, rain, dust, and smoke. One application of the passive MMW imaging system is a stand-off type sensor that can scan people moving in an unconstrained flow [2]. Unfortunately, in these applications, image quality is often degraded by low-level signals and system noise [8–10]. Low spatial resolution due to a limited aperture size is another hindrance to obtaining high-quality images. There have been efforts to develop automatic detection and recognition of concealed objects with passive MMW imaging [11–14]. Multilevel thresholding was addressed to detect concealed objects in terahertz (THz) images [11]. The metallic object detection with a Gaussian mixture model (GMM) has been researched in [12]. Multilevel expectation-maximization (EM) has been proposed to cluster pixels according to GMM [13]. However, this method requires manual initialization of the GMM parameters. The longitudinal distance of concealed objects is estimated by the corresponding segmentation-based features in [14].

This paper addresses real-time detection and segmentation of concealed objects under clothing in an outdoor open space. Our passive imaging system operates around 94 GHz to generate images with a 1 Hz frame rate. Automatic recognition of detected objects is achieved by multilevel segmentation. The multilevel segmentation comprises the global (first level) and local (second level) segmentation. The global segmentation separates the subject’s body area from the background area. During the local segmentation, only the inside of the body area is processed and the concealed object is segmented from the body area. This multilevel segmentation is improved from [13], adopting a series of clustering (unsupervised learning) methods such as vector quantization (VQ), EM algorithm for the GMM parameter estimation, and Bayesian decision rule. The VQ algorithm initializes the parameters of the GMM at each segmentation level. The EM process estimates the GMM parameters iteratively and the Bayesian decision rule decides which cluster each pixel belongs to. An alternative method is also developed for faster processing, which utilizes only the VQ during the global segmentation. The experimental and simulation results show that two proposed methods provide fast and reliable detection and segmentation of the object concealed in a moving human subject. The alternative method (shortened process) is able to reduce the average computational time by 87.9% from the fully-processed method (full process). However, the full process provides better segmentation performance than the shortened process.

The paper is organized as follows: In Section 2, we briefly describe a real-time outdoor stand-off, passive MMW imaging system. The multilevel segmentation method is discussed in Section 3. In Section 4, we present the experimental and simulation results. Conclusion follows in Section 5.

2. A passive millimeter wave imaging system

The passive MMW imaging system consists of a high-density polyethylene lens with a diameter of 50 cm, reflective mirror, one-dimensional (1D) receiver array composed of 30 receiver channels, and a mechanical scanner, which rotates the receiver array [10]. The focal length and the field of view (FOV) of the lens are 500 mm and 17 × 17°, respectively, thus the 30-channel 1D receiver array corresponds to 30 pixels in the range of 17°. The incoming W-band electromagnetic wave is focused on the feed antennas of a 1D receiver array located at the focal plane of the lens. Figures 1(a) and 1(b) show the passive MMW imaging system and a block diagram of the quasi-optical configuration, respectively. Each receiver channel is composed of a dielectric rod antenna, four low noise amplifiers, and a detector. The measured receiver gain is more than 45 dB in the desired frequency band. There are three factors in determining the signal sensitivity: the receiver bandwidth, the noise figure of the amplifier, and the integration time [10]. This passive MMW imaging system has 10 mrad spatial resolution and 1.5 K temperature resolution. The integration time of one channel is 10 msec.

 

Fig. 1 (a) Passive MMW imaging system. (b) A block diagram of the quasi-optical configuration.

Download Full Size | PDF

3. Image segmentation for concealed object recognition

The overall segmentation process is composed of preprocessing, global and local segmentation, and body area erosion as illustrated in Fig. 2 . Each segmentation level is composed of the VQ process, EM algorithm, and Bayesian decision rule. The morphological erosion of the body area is performed in order to estimate the accurate contour of the body area. The background area is eliminated during the windowing process after the global segmentation. Alternatively, the EM algorithm and the Bayesian decision process are removed during the global segmentation for faster processing. The removed stages for the shortened process are shown in dotted lines in Fig. 2.

 

Fig. 2 Overall procedure of concealed object segmentation.

Download Full Size | PDF

3.1 VQ process

The VQ process generates two clusters from a passive MMW image. The VQ process also initializes the parameters of the GMM in the full process while it only clusters pixels in the shortened process. The VQ in this research is the k-means algorithm equipped with a splitting initialization [15]. Figure 3 shows the VQ procedure to generate two clusters $S_{+}$ and $S_{-}$ from a vector set $S=\left\{x_1,\mathrm{...},x_{n_p}\right\}$. The initial vector $y_0$ is the mean of an input vector set S, Δ is a splitting vector, and $n_p$ is the number of pixels in the image. In this paper, each vector in the set S corresponds to a pixel’s intensity, thus scalar quantization is actually performed. The final $S_{+}$ and $S_{-}$ are clusters corresponding to each of the segmented areas. If it is desired to obtain more than two clusters this process can repeat. After the VQ process, the GMM parameters are initialized by the sample means, sample variances, and sample proportions of the clusters, $S_{+}$ and $S_{-}$.

 

Fig. 3 VQ process.

Download Full Size | PDF

3.2 EM algorithm and Bayesian decision process

The histogram is modeled with a mixture of Gaussian distributions as follows:

where ${\mu }_k$ and ${\Sigma }_k$ are the mean and the variance of the component (cluster) k, respectively, G_k represents the event of component k, $n_k$ is the number of components which is set 2 in this paper, and Ndenotes the Gaussian probability density function. The EM algorithm iteratively estimates the parameters; ${\mu }_k$, ${\Sigma }_k$, and $P\left(G_k\right)$ each per cluster. Figure 4 shows the block diagram of the EM algorithm; ε is the termination criteria, and $L_i$ is the log-likelihood; $L_i={\displaystyle \sum _{j=1}^{n_p}\log p_i\left(x_j\right)}$. More detailed expectation and maximization equations with GMM are found in [13,16].

 

Fig. 4 EM algorithm.

Download Full Size | PDF

The Bayesian decision rule uses the posterior probability density functions which are obtained by the EM algorithm. Each pixel is assigned to one of two clusters generating a binary image as

3.3 Performance evaluation

The average probability of error [13] is adopted to evaluate the segmentation performance as follows:

where $F_S$ and $F_O$ indicate the segmented area and the ground truth area of the concealed object, respectively, I is the entire image, and $|\cdot |$ denotes the number of pixels in the area.

4. Experimental and simulation results

The passive MMW images of a human subject hiding a gun (Colt 45 – M1911A1) are captured for 5 seconds with a 1 Hz frame rate. Figure 5(a) is the picture of the gun, and Fig. 5(b) is a visual movie of the human subject. Figure 6(a) is the movie of the passive MMW images capturing the same human subject. These passive MMW images are resized to be 3 times larger than the original size using the bicubic interpolation during preprocessing, thus the image size becomes $72\times 90$pixels. It is noted that the integration time of one channel is 10 msec, thus the total integration time is 240 msec for an image of 24 × 30 pixels. Figures 6(b) and 6(c) are the segmentation results by the full and the shortened process, respectively.

 

Fig. 5 (a) picture of the gun; (b) visual movie (Media 1).

Download Full Size | PDF

 

Fig. 6 Movies of (a) passive MMW images (Media 2), (b) full process segmentation (Media 3), (c) shortened process segmentation (Media 4).

Download Full Size | PDF

Figure 7 illustrates the segmentation results of the 3rd frame by the full process. Figures 7(a) and 7(b) are the GMM fitting, overlapped on the normalized histogram after the global and local segmentation, respectively. Figures 7(c) and 7(d) show the body and the concealed object area after the global and the local segmentation stage, respectively. It is noted that the histogram in Fig. 7(b) represents only the inside of the body area demonstrated in Fig. 7(d) since the background is excluded by the windowing process after the global segmentation. Figures 8(a) and 8(b) show the average probability of error and the computational time of the full and the shortened process, respectively. The average computational time of the shortened process is 0.134 sec while that of the full process is 1.11 sec, thus the computational time is reduced by 87.9%. The number of iteration is 2 for each level of segmentation. The number of iteration depends on the initial parameters of the GMM including the number of clusters. It is noted that all the programs are implemented with MATLAB 7.8 on a standard computer with a dual core 2.66 GHz CPU. Thus, the program is expected to run faster when specialized hardware is used. The average probability of error is increased from 0.0041 to 0.007 by about 1.7 times. The computational time can be significantly reduced by eliminating the EM algorithm during the global segmentation. However, the segmentation performance is degraded because only the VQ is used to segment the body and the background area. Although the same clustering methods are used during the local segmentation it can be seen that the performance is highly dependent of the global segmentation results.

 

Fig. 7 GMM fitting results after (a) global segmentation, (b) local segmentation. Segmented images after (c) global segmentation stage, (d) local segmentation stage.

Download Full Size | PDF

 

Fig. 8 (a) Average probability of error. (b) Computational time.

Download Full Size | PDF

5. Conclusion

In this paper, we have presented a real-time, concealed object detection and segmentation process using a passive imaging system and multilevel segmentation. The imaging system generates frames every second by scanning a human subject in an outdoor area. The multilevel segmentation is improved with the VQ initialization to realize the automatic recognition of a concealed object. We have also developed a faster process to significantly reduce computational time. This system can be expanded to scan a flow of people even more than one person at a time. Additional investigation of the proposed system which includes identifying multiple objects from a flow of people remains for future studies. We also leave the classification of concealed objects in the future research topic.