Open Access
Add to BibSonomy
Abstract
Feeble object detection is a long-standing problem in vision based underwater exploration work. However, because of the complicated light propagation situation and high background noise, underwater images are highly degraded. Noise is not always detrimental. Logical stochastic resonance (LSR) can be a useful tool for amplifying feeble signals by utilizing the constructive interplay of noise and a nonlinear system. In the present study, an appropriate LSR structure with a delay loop is proposed to process a low-quality underwater image for enhancing the vision detection accuracy of underwater feeble objects. Ocean experiments are conducted to demonstrate the effectiveness of the proposed structure. We also give explicit numerical results to illustrate the relationship between the structure of LSR and the correct detection probability. Methods presented in this paper are quite general and can thus be potentially extended to other applications for obtaining better performance.
© 2017 Optical Society of America
1. Introduction
Due to the absorption and scattering effect caused by water molecular and suspended particles, images acquired underwater suffer from low contrast and significant noise. Processing such low-quality images by conventional image processing methods is difficult [1,2]. Classical image pre-processing methods, such as denoising and contrast enhancement, seldom obtain good results over the range of operating conditions often encountered. For the last decades, researchers have worked hard to enhance the underwater image quality. Many interesting results are obtained, such as reverse-convolution restoration [3], polarization based enhancement [4], wavelength depended enhancement [5], and deep learning based enhancement [6]. However, few image enhancement algorithms work effectively in turbid water. Hence, the high-level image processing, including object detection and recognition, can be difficult or ineffective. Detecting feeble objects in turbid water environment is also too difficult to be realized using vision based methods under very turbid water conditions. Thus, it is desirable to investigate particular methods for improving feeble objects detection in low-quality underwater images.
Noise is not always detrimental and can be a useful component, especially when it is in nonlinear systems. One of the most famous examples of such phenomena is stochastic resonance (SR), in which the feeble input information can be amplified and optimized by the help of noise in the nonlinear system [7]. Murali [8,9] applied SR to help design reconfigurable and reliable logic gates and introduced the concept of logical stochastic resonance (LSR). As a new idea in nonlinear dynamics, LSR has attracted immense attention from various areas. The two elements of LSR, namely, system structure and noise character, have been widely studied by experts [10–12]. LSR has been applied mainly in tunnelling diodes [13], nanomechanical devices [14] and synthetic gene networks [15,16], etc.
By its very nature, detecting feeble objects from low quality underwater images is a question of feeble signal detection. Linear detection methods can achieve satisfying performance when the background noise can be modelled as a Gaussian noise. However, nonlinear methods can be more suitable than linear ones when the underwater situation is significantly complicated [17]. Further, the aforementioned problem is a logical one. Thus, solving this problem using LSR is reasonable. In our previous work, we tested the usefulness of LSR in underwater image processing to detect feeble objects [18]. The results showed that LSR exhibits good performance. However, the nonlinear system used is not specially designed for the image processing task, and the inherent noise character of underwater ambient situation is not fully discussed. Thus, we propose an improved LSR system with an extra delay loop in this paper to increase the detection accuracy.
The paper is organized as follows: In Section 2 we present the theoretical foundations of LSR based underwater image processing. In Section 3, we show why a delay loop should be considered and also consider the function of the whole LSR based system. In Section 4, a comparison between LSR based results of system with delay loop and the one without is performed. Both subjective and objective analyses are given. Section 5 summarizes the main results of the paper.
2. LSR based underwater imaging processing
The full underwater light propagation model, described by Duntley in [19] is as follows :
Where tNd(z, θ, ϕ) is the observed radiance. It is a function of the depth of the observer z. θ and ϕ are the zenith and azimuth angles between the observer and the target, respectively. The observed radiance is a mixture of the radiance at the object, tN0(zt, θ, ϕ), and the radiance in the water column (the background-light), N(zt, θ, ϕ). In these terms zt represents the depth of the target. The attenuation coefficient α(z), is the sum of two terms, namely, the scattering coefficient β and the absorption coefficient γ, both of which are functions of z. The distance between the observer and the target is denoted by d. The radiance attenuation function, K(z, θ, ϕ), indicates how the atmospheric solar light changes with depth.Theoretically, the knowledge of the inherent properties of water, α, β, and γ will permit us to predict the propagation of light in the water and then to restore the high quality image. However, all these parameters depend on the location d (in a three dimensional space) and are also time-varying. Therefore, the corresponding measurements are a complex task. A few simplifications must be conducted to allow the ease of computation. First of all, the water is assumed to be homogeneous, so K may be considered a unified varieble. The absorption coefficient γ can be considered as constant over a limited distance, for instance over d. The scattering effect is more complicated versus absorption, and can be further decomposed as backward scattering and forward scattering. Specifically, forward scattering can be modelled as a Gaussian filter [1]. Precisely obtaining backward scattering is difficult. However, researchers have proved that scattering affects the resulting images in a correlated way [3]. In our model, we simulate this effect as an extra delay loop that is correlated with the neighboring radiance.
In this paper, we use two states (i.e. yes or no) to label the detection result. Thus, we choose the widely used double-well potential as the base system. Furthermore, the backward scattering effect on the captured image is added as an extra delay loop to the base system. The main schematic of the whole feeble object processing method is shown in Fig. 1. We first sketch a region of interest (ROI) to narrow down the computing cost, shown as Fig. 1(b). The ROI area should be dimensionality reduced to a 1D form. There are ways to realize the dimensionality reduction. For example, expanding the image in the row or column sequence or in the illumination direction. The 1D signal stream is imported to the nonlinear potential well from the start and followed by a delay with a modulated weight. Additional noise is also added to stimulate the pixel to jump into the desired stable state as in the classical stochastic resonance algorithm. With the interplay of nonlinear system and the additional noise, the detection result will be obtained as the output of the whole system, such as in Fig. 1(g).
Fig. 1 The main schematic flow of proposed method. (a) The raw image with sketched ROI window (red rectangle). (b) Selected ROI image(containing number 4). (c) Expanded selected image to 1D form, and processing each pixel of the 1D rows from the beginning parallelly. The local pixel (blue) and corresponding τ − step delayed pixel (pink) are the inputs of the nonlinear bistable system. (d) Added gaussian noise. (e) Nonlinear bistable system, multilayer structure is used to realize parallel computing. (f) The output of nonlinear bistable system. (g) Final detection output.
3. LSR with delay loop
The bistable nonlinear system with delay loop can be described by the following Langevin equation [20]
here, V′(x(t), x(t − τ)) is the derivative of the symmetric double-well potential with a delay component, which is given by where a and b are the coefficients of the linear and non-linear terms, respectively. c is the weight of linear delay. The bias r has the effect of asymmetrizing the two potential wells [18]. Changing r can morph the logic operation of the system easily. I(t) represents the input noisy signal. t(t ≥ 0) is the time or sample index. When I(t) is time-varying signal, t represents the sampling time. When I represents a serially sampled image signal, t can be considered as the pixel location. τ is the delay length of the system, and it has the same meaning as t, which means the delay effect is spatially related. η(t) denotes a modulated additional zero-mean Gaussian white noise with autocorrelation function <η(t)η(0)>= 2Dδ(t), and D is the noise power.The effective potential of Eq. (2) is given by [21]:
which has three fixed points: two stable states at and an unstable state xu = 0. The presence of small delay loop changes the shape of the effective potential. Increasing the length of delay τ raises the depth of the potential well, while strong delay loop strength results in the deep potential well and narrows the distance between the two stable states. The effective potential defined by Eq. (3) is shown in Figs. 2(a) and 2(b). At the same time, in order to illustrate the effect of delay loop on the result of the whole system, an example is plotted as Fig. 2(c). The first column of Fig. 2(c) is the 1D input signal, which consists of feeble information (red) and heavy background noise. The later three columns show the response results of the system when then delay loop τ is different. The system is tuned specifically to obtain OR logic. It is clear that the logic output of the system without delay loop (τ = 0) is incorrect in the 11th time period, as highlighted in red, and the outcome is wrong in the 2nd, 5th, 11th time periods when the delay length is τ = 40, in Fig. 2(c), while all the output is correct when τ = 10. In this example, the presence of a moderate delay loop (τ = 10) enhances the accuracy robustness of the system.Fig. 2 (a) The effective potential of Eq. (3) for different c with τ = 40. (b)The effective potential of Eq. (3) for different time delay with c = 0.2. Other parameters of the system are set as a = 0.8, b = 1, r = 0, and the noise power D = 0 for both plots. (c) The input and output of the LSR system with delay loop. From top to bottom, panel 1 shows the input I(t), panel 2–4 show the responses of the system with τ set to be 0, 10, and 40, respectively. Other parameters of the system are set as a = 1.25, b = 1, c = 0.2, r = 0.3, and the noise power D = 0.5.
In practice, there are three parameters, a, b and r to modulate the nonlinear bistable system. Moreover, the weight of delay loop c and delay length τ should be tuned according to the actual imaging and water condition. At the same time, whether the system can obtain LSR and realize the optimum state or not, the power of the additional noise D strongly affects the resultant output. In this paper, we first set a = 1.25, b = 1 and r = 0.3 empirically to get an asymmetric bistable potential well [Fig. 1(f)]. As formulated in Eq. (4) the range of delay loop weight c should be from 0 to a. The delay length is a coefficient in accordance with the water quality, and will increase with the increasing distance from object to camera. As another preparatory work, to make the algorithm adaptive, we normalize the dimensionally reduced 1D signal to range [01] and compute the variance. The additional noise power is originally added as the computed variance, and may be modulated later to get better overall performance. LSR theory points out that the system may perform the best over a narrow range of noise power. However, what is the optimum noise power is not given by the theory. So we need to tune the noise intensity manually, or with the help of optimization methods.
The differential equation of Eq. (2) is numerically solved by a four step Rugger-Kutta algorithm. Our computing platform is MacBook Pro with 2.5Ghz Intel Core i7 processor. The simulation software is MATLAB 2013B.
4. Experiment results
4.1. Subjective analyses
Figure 3 shows three groups of badly degraded images (the first column) captured in offshore seawater and the corresponding detection results obtained by our method. For clear illustration, we list the results of the numbers respectively next to each raw image. The images on the upper row of each group are the results of LSR without delay loop and the ones on the bottom row are the results with delay loop. Without loss of generality, the size of the images is modulated to be same.
Fig. 3 Badly degraded images captured in offshore seawater (left column) and the corresponding detection results obtained by our method. The images (rightmost) on the upper row of each group are the results of LSR without delay loop and the ones on the bottom row are the results with delay loop. (a) Image at distance d = 1m from camera to test object with artificial illumination. (b) Image at distance d = 1m from camera to test object with natural illumination. (c) Image at distance d = 1.3m from camera to test object with artificial illumination. The parameters of the system are set as a = 0.85, b = 1, c = 0.45, r = −0.05, and the noise power D = 0.3. τ is set to be 3, 5, and 7 from up to bottom.
The raw images clearly show a blue-green tone, which is characteristic of the scattering character of seawater. The images are low quality because of the high degree of scattering noise producing low contrast and poor underwater visibility, which is a common scenario in offshore area or river water. Such highly degraded images are the object of our study. Our study mainly aims to detect the shape of the characters on the object calibration board correctly without any pre-processing procedure.
Among the three raw images shown in Fig. 3, the image in Fig. 3(a) is relatively clear. This indicates that the signal to noise ratio (SNR) of this image is higher than that of the other two. Thus, we first use it in examining the effect of the extra delay loop. Obviously, the use of LSR can help to separate the feeble object from the background without any pre-processing procedure. The detection result is good regardless of whether an extra delay loop is applied. Notably, the flaws on the both groups of results in Fig. 3(a) are different. In the system without delay loop, the resulting images are clear. However, some parts of the numbers are not detected correctly. For example, the left-top parts of number 2 and 3 are lost. Moreover, the calibration lines, which constitute the shape of T following the number 3, are completely undetected. Compared with the upper images, the bottom show much better detection result. Specifically, the edge of the numbers is more accurate in the bottom images. The left-top parts of number 2 and 3 are also detected correctly. Furthermore, the calibration lines behind number 3 are detected too.
The flaws on the results of LSR can be removed easily by simple image processing method, such as medium filter. However, we do not conduct such processing in order to highlight the capability of LSR.
The raw images in Figs. 3(b) and 3(c) are very faint because of the backward scattering and light absorption. In Fig. 3(b), several numbers can be observed in the raw image. However, the edge of the numbers cannot be distinguished. Little useful information can be obtained from the original image in Fig. 3(c) visually. Popular image enhancement and restoration algorithms, such as dark channel [22] and Fattel’s [23] method, can induce slight enhancement effect in such images. The useful information has already been overwhelmed by the heavy noise.
The detection result in Fig. 3(b) indicates that, in such low-quality image, LSR still can be applied to obtain a relatively satisfactory result regardless of whether delay loop is applied. However, the resulting images with delay loop exhibit improved detection correction than those without delay loop. The noise in the upper processed images is greater than that in the bottom images. The raw image in Fig. 3(c) is completely blurred. The detection result is poor even with the help of LSR. Although the separation result is obtained without delay loop, the information from the resulting image cannot be identified. Extra flaws in the form of spatial noise exist in the images. However, the number can be recognized easily, which is an important enhancement. A much better result is obtained when delay loop is added to the LSR system.
On the basis of analyzing the above three groups of images, we conclude that LSR can help detect feeble objects from low-quality images. The extra delay loop provides the model with high precision and better detection result, especially when the background noise is large.
4.2. Objective analyses
To analyze the performance of LSR with delay loop in underwater image processing throughly, we also calculate the probability P of getting a correct detection result. In this paper, we give the explicit performance about the third raw picture in Fig. 3(c) as the example. Since there are many impurities floating in water which also enter the scope of image, it is hard to judge whether it is right or wrong to detect such things. So we first manually crop the part only contain the number character from the original image. To be specific, we crop number 1, 2, 3, and 4 area separately. And then carefully label the truth-value in every cropped picture (To guarantee the validity, this step is carried out with the help of a ground-truth image of the calibration board).
The correct probability P for each cropped picture is obtained by comparing the detection result of LSR with delay loop with the truth-value image. Two auxiliary ratios are calculated. One is the object detection probability ratio, as
where no is the number of pixels, which are correctly detected as the object pixels, in the result image. not is the total number of pixels labeled as object pixel in the truth-value image. The other one is the background detection probability ratio, as where nb is the number of pixels, which are correctly detected as the background pixels, in the result image. nbt is the total number of pixels labeled as background pixel in the truth-value image.Then, P is obtained as
Here, ∊ is the normalization coefficient to modulate the degree of importance of Po and Pb. In this experiment, as we aim to detect the correct number character, we set a nominal value ∊ = 0.8.Based on the above method, we can study the interplay of noise power D and delay loop τ with the correct probability P. Figure 4 shows four groups of the cropped original image, the labeled truth-value image, probability of correct detection versus added noise power P − D curve and the colormap of probability of correct detection versus added noise power and delay length P − D&τ.
Fig. 4 Four groups of the cropped original image, the labeled truth-value image, probability of correct detection versus added noise power P − D curve and the colormap of probability of correct detection versus added noise power and delay length P − D&τ. Here, P − D curve is obtained by setting τ = 10. Other parameters of the system are set as: a = 0.85, b = 1, c = 0.4, r = −0.05. The four original images are cropped from the same original picture [Fig. 3(c)].
Figure 4 clearly illustrates several phenomena. Firstly, when the delay loop τ is fixed, the correct probability P will change non-monotonically, first increase and then decrease, as the intensity of additional noise becomes larger. This is the typical performance of LSR. Secondly, shown as the P − D&τ colormap, for each particular noise intensity, a moderate additional delay loop will increase the correct probability of detection, which validates the effectiveness of the LSR with delay loop method in underwater image processing. Thirdly, the system parameters to obtain an optimal detection result for each number are not same, and the maximum correct probability is also not consistent. So, to obtain optimal detection result, a local-correlated parameter setting is recommended, which means that a set of different system parameters may be needed in different local patches.
5. Conclusion
In summary, we propose a LSR with delay loop system to detect feeble objects from low-quality underwater images. The addition of an extra delay loop provides the system with high precision in processing underwater images. Ocean experiments illustrate the enhancement effect of delay loop on the feeble object detection. Explicit numerical results about the performance of LSR based feeble object detection are shown to illustrate the relationship between the correct detection probability with the coefficients of the system. Typical LSR phenomenon is obtained.
However, although the proposed method may detect feeble objects in very low SNR, it does have some drawbacks. Foremost, the method in this version can only be used in simple background, including the illumination condition. When the background is complicated, correct detection of feeble objects may be very hard or impossible. Often in highly turbid water, the background is highly uniform at source-object separation of multiple attenuation lengths 1/α(z) where lighting non-uniformity due to surface wave refraction phenomena, or artificial lighting defects are less prominent. So this method may prove to be useful for exploration activities in highly turbid conditions.
Funding
National Natural Science Foundation of China (No. 61703381); Natural Science Foundation of Shandong Province (No. ZR2017BF006); China Postdoctoral Science Foundation (No. 2016M590658); Fundamental Research Funds for the Central Universities (No.201713017).
References and links
1. S. Raimondo and C. Silvia, “Underwater image processing: State of the art of restoration and image enhancement methods,” EURASIP J. Adv. Signal Processing pp. 1–15 (2010).
2. J. S. Jaffe, “Underwater optical imaging: the past, the present, and the prospects,” IEEE J. Oceanic Eng. 40, 683–700 (2015). [CrossRef]
3. G. Wang, B. Zheng, and F. F. Sun, “Estimation-based approach for underwater image restoration,” Opt. Lett. 36, 2384–2386 (2011). [CrossRef] [PubMed]
4. B. Huang, T. Liu, H. Hu, J. Han, and M. Yu, “Underwater image recovery considering polarization effects of objects,” Opt. Express 24, 9826–9838 (2016). [CrossRef] [PubMed]
5. J. Y. Chiang and Y.-C. Chen, “Underwater image enhancement by wavelength compensation and dehazing,” IEEE Trans. Image Process. 21, 1756–1769 (2012). [CrossRef]
6. Y. Li, H. Lu, J. Li, X. Li, Y. Li, and S. Serikawa, “Underwater image de-scattering and classification by deep neural network,” Comput. Electrical Eng. 54, 68–77 (2016). [CrossRef]
7. R. Benzi, A. Sutera, and A. Vulpiani, “The mechanisim of stochastic resonance,” J. Phys. A: Mathmatical Genenral 14, 453–457 (1981). [CrossRef]
8. K. Murali, I. Rajamohamed, S. Sinha, W. L. Ditto, and A. R. Bulsara, “Realization of reliable and flexible logic gates using noisy nonlinear circuits,” Appl. Phys. Lett. 95, 194102 (2009). [CrossRef]
9. K. Murali, S. Sinha, W. L. Ditto, and A. R. Bulsara, “Reliable logic circuit elements that exploit nonlinearity in the presence of a noise floor,” Phys. Rev. Lett. 102, 194102 (2009). [CrossRef]
10. N. Wang and A. Song, “Set-reset latch logical operation induced by colored noise,” Phys. Lett. A 378, 1588–1592 (2014). [CrossRef]
11. N. Wang and A. Song, “Parameter-induced logical stochastic resonance,” Neurocomputing 155, 80–83 (2015). [CrossRef]
12. H. Zhang, T. Yang, W. Xu, and Y. Xu, “Effects of non-gaussian noise on logical stochastic resonance in a triple-well potential system,” Nonlinear Dynamics 76, 649–656 (2014). [CrossRef]
13. L. Worschech, F. Hartmann, T. Kim, S. Höfling, M. Kamp, A. Forchel, J. Ahopelto, I. Neri, A. Dari, and L. Gammaitoni, “Universal and reconfigurable logic gates in a compact three-terminal resonant tunneling diode,” Appl. Phys. Lett. 96, 042112 (2010). [CrossRef]
14. D. N. Guerra, A. R. Bulsara, W. L. Ditto, S. Sinha, K. Murali, and P. Mohanty, “A noise-assisted reprogrammable nanomechanical logic gate,” Nano Lett. 10, 1168–1171 (2010). [CrossRef] [PubMed]
15. N. Wang and A. Song, “Enhanced logical stochastic resonance in synthetic genetic networks,” IEEE Trans. Neural Networks Learning Systems 12, 2735–2739 (2015).
16. A. Sharma, V. Kohar, M. D. Shrimali, and S. Sinha, “Realizing logic gates with time-delayed synthetic genetic networks,” Nonlinear Dynamics 76, 431–439 (2014). [CrossRef]
17. H. Chen, L. R. Varshney, and P. K. Varshney, “Noise-enhanced information systems,” Proc. IEEE 102, 1607–1621 (2014). [CrossRef]
18. B. Zheng, N. Wang, H. Zheng, Z. Yu, and J. Wang, “Object extraction from underwater images through logical stochastic resonance,” Opt. Lett. 41, 4967–4970 (2016). [CrossRef] [PubMed]
19. S. Q. Duntley, “Light in the sea,” J. Opt. Soc. Am. 53, 214–233 (1963). [CrossRef]
20. L. Gammaitoni, P. Hänggi, P. Jung, and F. Marchesoni, “Stochastic resonance,” Rev. Mod. Phys. 70, 45–105 (1998). [CrossRef]
21. T. Frank, “Delay fokker-planck equations, novikovs theorem, and boltzmann distributions as small delay approximations,” Phys. Rev. E 72, 011112 (2005). [CrossRef]
22. K. He, J. Sun, and X. Tang, “Single image haze removal using dark channel prior,” IEEE Trans. Pattern Analysis Machine Intelligence 33, 2341 (2011).
23. R. Fattal, “Single image dehazing,” ACM Trans. Graphics 27, 72 (2008). [CrossRef]
