Open Access
Add to BibSonomy
Abstract
We use a Wiener deconvolution filter to deblur the streak image of streak tube imaging lidar to improve the spatial resolution of the system and reduce the edge blurring effect in point clouds. Experiments were performed to investigate the performance of the deconvolution method. Results show that the spatial resolution improved from 9 to 4.5 mm, and the root-mean-square errors of the edge regions are effectively reduced. Additionally, the transition section decreases from 14 to 5.6 mm when the target is 5 m away from the receiver.
© 2019 Optical Society of America under the terms of the OSA Open Access Publishing Agreement
1. Introduction
Recently, streak tube imaging lidar (STIL) has attracted significant attention owing to its high accuracy, wide field of view, and fast data rate [1–9]. A typical schematic of an STIL is shown in Fig. 1 [10]. Laser is shaped into a fan beam by cylindrical lenses and forms a thin strip footprint on the target. Echo signals are collected by the camera lens and imaged onto the photocathode of the streak tube. Subsequently, the echo signal is converted into electrons, and these electrons are accelerated to a phosphor screen by a high voltage. A pair of deflection plates with a time-dependent voltage is applied on the path of the electrons to provide the electrons an offset that is proportional to the deflection voltage. Subsequently, the electrons hit the screen of the streak tube and form a streak image. Therefore, the different vertical positions in the time-resolved channel on the screen are relative to the arrival time of the signal, and we can calculate the target distance according to the streak image. Finally, the streak image is amplified by a microchannel plate (MCP) and recorded by a charge-coupled device (CCD). The horizontal position of the streak image represents the spatial information of the target, whereas the vertical position the distance information. For the latter, we used the centroid weight method to calculate the vertical position of each time-resolution channel. More information regarding the STIL is available in [11].
Fig. 1. Schematic of the STIL system. (a) - schematic of the laser after the emission. (b) - schematic of the data collection process. (c) - streak image on the CCD.
Spatial resolution is an important feature for many lidar applications, such as urban exploration and survey [12], as well as monitoring and management of vegetation landscapes and wetlands [13,14]. However, components in the STIL system, such as the camera lens, streak tube, MCP, fiber optic taper, and CCD, will blur the streak image. The target detected by STIL using the blurred image will be larger than its real size, and the edge blurring effect will become more obvious. Hence, the spatial resolution will be reduced, and the accuracy of the edge regions will be poor. Therefore, the streak image of STIL must be deblurred to increase the spatial resolution and reduce the adverse impact of the edge blurring effect.
In this study, the Wiener deconvolution filter is used to deblur the streak image of STIL, and the deconvwnr function in the Matlab software is used to implement it. To the best of our knowledge, this is the first work that deblurs a streak image to improve the spatial resolution and decrease the impact of the edge blurring effect on accuracy. The point-spread function (PSF) of our STIL system is measured to realize the Wiener deconvolution filter. Experiments were performed to investigate the spatial resolution, depth resolution, and root-mean-square error (RMSE) of the original streak image and the image after Wiener deconvolution filtering. Two and three plane targets were used to investigate the edge blurring effect of the original streak image and the image after Wiener deconvolution filtering.
2. Methodology
Many methods can be used to deblur images, and they can be classified into two types: blind deconvolution and nonblind deconvolution methods [15]. As the PSF of our system can be measured in advance, we used nonblind deconvolution methods to deblur the streak images. The Richardson–Lucy method [16,17] does not consider the effects of noise, and the deblurred image will converge locally. The streak images at different times are different; therefore, it is impossible to use multiple images [18] to improve the resolution. The Fourier-wavelet regularized deconvolution method [19] can deblur the streak image effectively; however, the number of pixels in the streak image is strictly limited. Therefore, the Wiener deconvolution filter, which is easier to realize, is selected to deblur the streak image.
The echo signal obtained by the STIL system can be expressed as follows:
where * denotes convolution; x(t) is the unknown original signal at time t; h(t) is the impulse response of our system; n(t) is unknown additive noise, independent of x(t); y(t) is our observed signal. Our goal is to obtain g(t) such that we can estimate x(t) as follows: where $\hat{x}\textrm{(}t\textrm{)}$ is the echo signal estimated using the deconvolution method. The Wiener deconvolution filter provides g(t), which can be most easily described in the frequency domain:We detected the point target to calculate the PSF of our system. In reality, the ideal point-source object does not exist. Meanwhile, because we used a CCD to detect the streak image, we could assume that targets with a size smaller than one pixel after focusing by the camera lens were point-source objects. The PSF measured by this method is the PSF of our detection system except for the camera lens. Because the impact of the PSF of the camera lens was negligible, we assumed that the PSF obtained as such was the PSF of the entire receive system.
A laser beam was focused on a standard reflectivity plate with a reflectivity of 99%, and a light spot was formed on the reflectivity plate. We detected the light spot 5.3 m away from the camera lens. The focal length of our camera lens was 50 mm, and the size of one pixel of the CCD was 10 µm ×10 μm. Therefore, if the diameter of the light spot was less than 1 mm, we could treat it as a point-source object. We controlled the diameter of the light spot and obtained the streak image of the light spot. Subsequently, we fitted it with a two-dimensional Gaussian function [20]. The fit results of the streak image are shown in Fig. 2. The expression of the fit function is shown in Eq. (4):
Fig. 2. Fit results of the intensity image of the point-source object streak image. (a) - the main plot; (b) - the contour plot.
3. Results and discussions
Herein, the spatial resolutions of the original streak image and after Wiener deconvolution filtering are compared. Three plates, as shown in Figs. 3(a) and 3(b), were used to investigate the spatial resolution. Each metal plate was 100 mm long, 80 mm wide, and 2 mm thick and was made of aluminum. Its surface was a diffusely reflective surface with a reflectivity of approximately 75%. As shown in Fig. 3(b), the two front plates are equidistant from the camera lens, and the distance between the rear and front plates is 500 mm. The distance between the two front plates changes from 1 to 60 mm. Figures 3(c) and 3(d) show the intensity image of the original streak image and the image after the Wiener deconvolution filtering of the target when the gap width is 10 mm, with red lines passing through the center of the streak image. The intensity values of pixels on the red line are shown in Figs. 3(e) and 3(f). Two maximum intensity values in each side of the gap and one minimum intensity value inside the gap are labeled in Figs. 3(e) and 3(f); for example, the number “338,28” in Fig. 3(e) means that the minimum intensity value is 28 and it is located at the 338th pixel. These extreme values were used to calculate the spatial contrast, as shown in Eq. (5):
Fig. 3. (a) Top view of the target. (b) Three-dimensional view of the target. (c) Intensity image of the original streak image when the gap width is 10 mm. (d) Intensity image of the image after Wiener deconvolution filtering when the gap width is 10 mm. (e) Intensity of the pixels on the red line, and the calculated Cs of the original streak image. (f) Intensity of the pixels on the red line, and the calculated Cs of the image after Wiener deconvolution filtering.
It is well known that contrast is susceptible to the image intensity; therefore, we controlled the maximum intensity of the pixels on the red line to between 100 and 110 to calculate Cs. The calculated results are shown in Fig. 4(a). It is clear that the Cs of the image after Wiener deconvolution filtering is less than 0.5 when the gap is smaller than 4.5 mm and larger than 0.5 when the gap is larger than 4.5 mm. Therefore, the spatial resolution limit of the image after Wiener deconvolution filtering is 4.5 mm. The spatial resolution limit of the original streak image is 9 mm. Figures 4(b)–4(i) show the intensity values of the pixel on the red line, and the calculated Cs of the original image and the image after Wiener deconvolution filtering when the gaps are 2, 4.5, 9, and 18 mm. The two maximum intensity values and one minimum intensity value in each subfigure are labeled.
Fig. 4. Spatial resolution results of the original streak image and the image after Wiener deconvolution filtering. (a) Cs results of the original streak image and the image after Wiener deconvolution filtering when the gap width changes from 1 to 60 mm. (b), (d), (f), (h) Intensity and calculated Cs of the pixels on the red line of the original streak image when the gap widths are 2, 4.5, 9, and 18 mm. (c), (e), (g), (i) Intensity and calculated Cs of the pixels on the red line on the image after Wiener deconvolution filtering when the gap widths are 2, 4.5, 9, and 18 mm.
As shown in Fig. 3(d), some distortions appear in the image after Wiener deconvolution filtering, which will affect the depth resolution and RMSE.
We used two nonadjacent metal plates, as shown in Fig. 5(a), to study the effect of Wiener deconvolution filtering on the depth resolution. As shown in Fig. 5(b), the two plates are 40 mm away from each other, and we moved the right plate from 0 to 200 mm. Figures 5(c) and 5(d) are the original streak image of the target and the image after Wiener deconvolution filtering. The vertical pixel values of 100 streak images of the same target were used to draw a histogram, as shown in Figs. 5(e) and 5(f). Furthermore, the histogram was used to calculate depth contrast, as shown in Eq. (6):
Here, Cd is the depth contrast of the streak image. We assume that the two plates can only be distinguished when Cd is less than or equal to 0.5. Nmax1 and Nmax2 are two maximum count numbers corresponding to the position of the two plates, and Nmin is the minimum count number between Nmax1 and Nmax2. When the depth distance between the two plates is small, only one maximum count number will be obtained. In this case, Nmin = Nmax1 = Nmax2 can be considered; therefore, Cd equals 1.Fig. 5. (a) Front view of the target. (b) Top view of the target. (c) Intensity image of the original streak image when the depth distance is 12 mm. (d) Intensity image of the image after Wiener deconvolution filtering when the depth distance is 12 mm. (e) Histogram of the vertical pixel values of 100 original streak images and the calculated Cd, when the depth distance is 12 mm. (f) Histogram of the vertical pixel values of 100 images after Wiener deconvolution filtering and the calculated Cd, when the depth distance is 12 mm.
The calculated results of Cd are shown in Fig. 6(a). It is clear that the limits of depth resolution of the original streak image and the image after Wiener deconvolution filtering are 9 and 10 mm, respectively. Figures 6(b)–6(i) show the histogram and the calculated Cd when the depth distances are 8, 9, 10, and 15 mm.
Fig. 6. Depth resolution results of the original streak image and the image after Wiener deconvolution filtering. (a) Cd results of the original streak image and the image after Wiener deconvolution filtering when the depth distance changes from 1 to 200 mm. (b), (d), (f), (h) Intensity and calculated Cd of the pixels on the red line of the original streak image when the gap widths are 8, 9, 10, and 15 mm. (c), (e), (g), (i) Intensity and calculated Cd of the pixels on the red line of the image after Wiener deconvolution filtering when the gap widths are 8, 9, 10, and 15 mm.
We investigate the effect of Wiener deconvolution filtering on the RMSE under three conditions: (a) single plane; (b) two planes; (c) three planes. In our system, each vertical pixel corresponds to 10 mm in the depth direction, and each horizontal pixel 0.7 mm in the horizontal spatial direction.
The calculated vertical position and RMSE results of a single plane, as shown in Fig. 7(a), are shown in Fig. 7(b). The RMSEs of the original image and the image after Wiener deconvolution filtering are 0.20s and 0.22 pixels, respectively, i.e., 2.0 and 2.2 mm, respectively. Two plates were used to investigate the two planes’ condition. As shown in Figs. 8(a) and 8(b), the depth distance between the two plates is set as 500 and 1000 mm, respectively. The calculated vertical position and RMSE results, when the depth distances are 500 and 1000 mm, are shown in Figs. 8(c) and 8(d), respectively. When the depth distance is 500 mm, the RMSEs of the original streak image and the image after Wiener deconvolution filtering are 7.86 and 5.19 pixels, respectively, i.e., 78.6 and 51.9 mm, respectively. When the depth distance is 1000 mm, the RMSEs of the original streak image and the image after Wiener deconvolution filtering are 16.25 and 10.75 pixels, respectively, i.e., 162.5 and 107.5 mm, respectively. Additionally, the transition section of the vertical position result of the original streak image is 20 pixels, i.e., 14 mm, in both Figs. 8(c) and 8(d). Meanwhile, the transition section of the vertical position result of the image after Wiener deconvolution filtering is 8 pixels, i.e., 5.6 mm, in both Figs. 8(c) and 8(d).
Fig. 7. (a) Front view and top view of one plane target. (b) Calculated vertical position of one plane target.
Fig. 8. (a), (b) Top view and left view of the two plane targets, the depth distances are set as 500 and 1000 mm. (c), (d) The calculated vertical position and RMSE when the depth distances are 500 and 1000 mm.
For the three planes’ condition, we used the same target as that in Fig. 3(a) to study the effect of Wiener deconvolution filtering. We assumed that the front two planes were on the same plane to simplify this condition. We moved the right plate to control the gap width. The RMSEs of the original streak image and the image after Wiener deconvolution filtering for different gap widths are shown in Fig. 9(a). Figures 9(b)–9(d) show the distance when the gap width is 8, 16, and 25 mm. As shown in Fig. 9(a), the RMSEs of the image after Wiener deconvolution filtering are smaller than those of the original image, especially when the gap width is between 7 and 15 mm. As shown in Figs. 9(b) and 9(c), the calculated vertical position of the image after Wiener deconvolution filtering is closer to the theoretical curve in the rear plane area. It is clear from Fig. 9(d) that the calculated vertical position of both conditions will reach the bottom in the rear plane area when the gap width is sufficiently large. Additionally, the transition section of the image after Wiener deconvolution filtering is smaller than the original streak image.
Fig. 9. (a) RMSE of the original streak image and the image after Wiener deconvolution filtering of the three plane targets when the gap width changes from 2 to 50 mm. (b), (c), (d) Calculated vertical position and RMSE of the three plane targets when the gap widths are 8, 16, 25 mm.
4. Conclusion
In this study, we used Wiener deconvolution filtering to deblur a streak image to improve its spatial resolution and decrease the impact of the edge blurring effect on the accuracy. The results, shown in Table 1, indicated that the spatial resolution improved from 9 to 4.5 mm, i.e., a doubled spatial resolution. We discovered that Wiener deconvolution filtering introduced distortions. Owing to the distortion, we investigated the depth resolution and RMSE. Results indicated that the depth resolution decreased slightly from 10 to 9 mm, and the RMSEs of the original image and the image after Wiener deconvolution filtering of one plane target were 2.0 and 2.2 mm, respectively. For surveying, this level of reduction in depth resolution and RMSE was negligible. Additionally, the RMSEs of the two and three plane targets of the image after Wiener deconvolution filtering were smaller than those of the original streak image. Furthermore, the transition section for the two plane targets decreased from 14 to 5.6 mm, which implied that Wiener deconvolution filtering could effectively decrease the edge blurring effect. The experiments demonstrated that Wiener deconvolution filtering could be used to produce sharper streak images.
Table 1. Experimental results of the original streak image and the image after Wiener deconvolution filtering (units: mm)
Funding
National Key Scientific Instrument and Equipment Development Projects of China (2012YQ040164); Technological Innovation Talents Foundation of Harbin City in China (2017RAQXJ056).
Acknowledgments
We appreciate Fang Lin's help in understanding the improved Wiener deconvolution filtering.
Disclosures
The authors declare no conflicts of interest.
References
1. J. Gao, J. Sun, and Q. Wang, “Experiments of ocean surface waves and underwater target detection imaging using a slit Streak Tube Imaging Lidar,” Optik 125(18), 5199–5201 (2014). [CrossRef]
2. Z. Tian, Z. Cui, L. Zhang, T. Xu, Y. Zhang, and S. Fu, “Control and image processing for streak tube imaging lidar based on VB and MATLAB,” Chin. Opt. Lett. 12(6), 67–70 (2014). [CrossRef]
3. J. Gao, J. Sun, and Q. Wang, “Experiments on the range resolution measurement of a slit streak tube imaging lidar,” Optik 126(21), 3084–3087 (2015). [CrossRef]
4. G. Ye, R. Fan, Z. Chen, W. Yuan, D. Chen, and P. He, “Range accuracy analysis of streak tube imaging lidar systems,” Opt. Commun. 360, 7–14 (2016). [CrossRef]
5. G. Ye, R. Fan, Z. Chen, X. Xu, P. He, and D. Chen, “Effect of time bin size on accuracy of streak tube imaging lidar,” Chin. Opt. Lett. 14(2), 21101–21104 (2016). [CrossRef]
6. W. Xia, S. Han, N. Ullah, J. Cao, L. Wang, J. Cao, Y. Cheng, and H. Yu, “Design and modeling of three-dimensional laser imaging system based on streak tube,” Appl. Opt. 56(3), 487–497 (2017). [CrossRef]
7. K. Xu, “Monolithically integrated Si gate-controlled light-emitting device: science and properties,” J. Opt. 20(2), 024014 (2018). [CrossRef]
8. J. Yu, S. K. Jha, L. Xiao, Q. Liu, P. Wang, C. Surya, and M. Yang, “AlGaN/GaN heterostructures for non-invasive cell electrophysiological measurements,” Biosens. Bioelectron. 23(4), 513–519 (2007). [CrossRef]
9. K. Xu, Y. Chen, T. A. Okhai, and L. W. Snyman, “Micro optical sensors based on avalanching silicon light-emitting devices monolithically integrated on chips,” Opt. Mater. Express 9(10), 3985–3997 (2019). [CrossRef]
10. B. C. Redman, A. J. Griffis, and E. B. Schibley, “Streak Tube Imaging Lidar (STIL) for 3-D Imaging of Terrestrial Targets,” in Proceeding of 2000 Meeting of the MSS Specialty Group on Active E-O Systems, (Academic, 2000).
11. Z. Chen, R. Fan, G. Ye, T. Luo, J. Guan, Z. Zhou, and D. Chen, “Depth resolution improvement of streak tube imaging lidar system using three laser beams,” Chin. Opt. Lett. 16(4), 041101 (2018). [CrossRef]
12. B. Bonczak and C. E. Kontokosta, “Large-scale parameterization of 3D building morphology in complex urban landscapes using aerial LiDAR and city administrative data,” Comput. Environ. Urban 73, 126–142 (2019). [CrossRef]
13. S. Rapinel, E. Fabre, S. Dufour, D. Arvor, C. Mony, and L. and Hubert-Moy, “Mapping potential, existing and efficient wetlands using free remote sensing data,” J. Environ. Manage. 247, 829–839 (2019). [CrossRef]
14. A. Räsänen and T. Virtanen, “Data and resolution requirements in mapping vegetation in spatially heterogeneous landscapes,” Remote Sens. Environ. 230, 111207 (2019). [CrossRef]
15. R. Fergus, B. Singh, A. Hertzmann, S. T. Roweis, and W. T. Freeman, “Removing camera shake from a single photograph,” ACM Trans. Graph. 25(3), 787–794 (2006). [CrossRef]
16. W. H. Richardson, “Bayesian-based iterative method of image restoration,” J. Opt. Soc. Am. 62(1), 55 (1972). [CrossRef]
17. L. B. Lucy, “An iterative technique for the rectification of observed distributions,” Astron. J. 79, 745 (1974). [CrossRef]
18. A. Rav-Acha and S. Peleg, “Two motion-blurred images are better than one,” Pattern Recognit. Lett. 26(3), 311–317 (2005). [CrossRef]
19. R. Neelamani, H. Choi, and R. Baraniuk, “ForWaRD: Fourier-wavelet regularized deconvolution for ill-conditioned systems,” IEEE Trans. Signal Process. 52(2), 418–433 (2004). [CrossRef]
20. K. Xu, “Silicon MOS Optoelectronic Micro-Nano Structure Based on Reverse-Biased PN Junction,” Phys. Status Solidi A 216(7), 1800868 (2019). [CrossRef]
