Open Access
Add to BibSonomy
Abstract
In underwater images, the significant sources of distortion are light attenuation and scattering. Existing underwater image restoration technologies cannot deal with the poor contrast and color distortion bias of underwater images. This work provides a new underwater image restoration approach relying on depth map optimization and background light (BL) estimation. First, we build a robust BL estimation model that relies on the prior features of blurriness, smoothness, and the difference between the intensity of the red and blue-green channels. Second, the red-light intensity, difference between light and dark channels, and disparity of red and green-blue channels by considering the hue are used to calculate the depth map. Then, the effect of artificial light sources on the underwater image is removed using the adjusted reversed saturation map. Both the subjective and objective experimental results reveal that the images produced by the proposed technology provide more remarkable visibility and superior color fidelity.
© 2022 Optica Publishing Group under the terms of the Optica Open Access Publishing Agreement
1. Introduction
The ocean contains a lot of energy and is essential to the survival and continuation of the earth. Photographs captured underwater by waterproof cameras can provide valuable information. However, light deviates from its route before entering the camera because the light is scattered by the particles suspended in the water, resulting in diminished quality, blur, and haze in underwater images [1–6]. In addition, blue light with the shortest wavelength travels the longest in water, which is why underwater images often exhibit varying degrees of color distortion [7,8]. Deteriorated underwater photographs have a decreased practical application value, so it is critical to devise a strategy for recovering the color and improving the contrast of these images [9–14].
Diverse approaches for restoring underwater images with various scenarios and distortion levels have been created. However, because of the inaccuracies in BL estimation and transformation maps, the effectiveness under different scenarios of these existing techniques is often poor, which means there is still much to be done in dealing with color deviation and enhancing contrast. The current work provides an underwater image restoration approach that relies on BL estimation and depth map fusing to rebuild the natural look of underwater images. First, we present an approach for estimating background light that relies on blurriness, smoothness, and the difference between the intensity of the red and blue-green channels. Then, we separately compute the red-light intensity, the difference between light and dark channels, and the red and blue-green channels disparity by considering the hue, which produces a depth map. Then, the transmission map is calculated using the obtained depth map, which is accomplished by applying the adjusting reversed saturation map (ARSM) principle to enhance the transmission map.
The following are the technical contributions of the present paper:
- (1) We present a BL estimation approach that relies on blurriness, smoothness, and the difference between the intensity of the red and blue-green channels. It estimates the background light accurately and effectively solves the problem of background light overestimation in scenes with white objects or bright pixels in the near field.
- (2) A better depth map model is developed. To obtain the final depth map, we independently calculate the depth map of the red-light intensity, the difference between light and dark channels, and the disparity of the red and blue-green channels by considering the hue. This makes the estimation of the transmission map more reliable.
- (3) After the transmission map (TM) is computed using the depth map, the adjusted reversed saturation map is utilized to remove the effect of artificial light sources, resulting in a more precise TM estimation.
- (4) Experiments reveal that when compared with existing underwater image processing techniques, our approach provides color correction, boosts contrast, and brings out detailed texture information, resulting in improved visual effects.
The flowchart of the suggested method is shown in Fig. 1, the rest of the paper is as follows: The existing underwater image processing methods are presented in Section 2. The proposed approach is described in depth in Section 3. Section 4 provides the outcomes of the experiments, and Section 5 presents the conclusions.
2. Related work
The current paper introduces three image processing methods: image restoration methods based on imaging models [15,16] and an enhancement approach based on convolutional neural networks and traditional enhancement methods. The restoration method relying on the imaging model is shown in Fig. 2.
Fig. 2. The underwater image restoration method based on imaging model and comparison of underwater images before and after restoration.
2.1 Model-free method
Traditional enhancement methods usually improve image quality by directly adjusting the pixel values. Zhuang et al. [5] proposed an algorithm that utilizes multistep gradient priors of reflection and illumination to effectively enhance underwater images, preserve naturalness, enhance details and suppress noise. Fu et al. [17] developed a retinex-based color enhancement scheme and variational framework to cope with the color cast problem (RBE). However, this technology ignores the degradation of underwater images, directly changing the pixel values, resulting in excessive enhancement and loss of detail in the processed results. Huang et al. [18] suggested a relatively global histogram stretching shallow image enhancement approach based on comprehensive histogram stretching with adaptive parameter acquisitions (RGHS), which was divided into two parts: contrast adjustment and color correction. However, because of its less robust color correction method, it cannot remove color casts. Another novel method to improve underwater image quality was proposed by Zhou et al. [19]. In this method, multi-feature priors of underwater images were extracted and combined to enhance their visual quality.
Model-free physical approaches can increase visual effects to some degree, but because of the intricacy of the underwater environment and lighting circumstances, these enhancement methods do not take into account degradation factors, such as red-light attenuation and scattering of light, making it challenging to obtain high-quality images from underwater scenes.
2.2 Model-based method
Carlevaris-Bianco et al. [20] estimated underwater scene depth based on the maximum intensity prior (MIP). However, in scenes with dim background light, this method usually estimates the distant view as a close view. Dark channel prior (DCP) [21] is often used to dehaze a single image. Because of the long wavelength of red light, the attenuation is faster underwater. Taking this characteristic into account, Drews-Jr et al. [22] established an underwater dark channel prior that only deals with the green and blue channels (UDCP). Peng et al. [23] proposed a generalization of the dark channel prior to compute the difference between observed intensity and ambient light to determine the transmission map of the scenes (GDCP). Lee et al [24]. propose a superpixel-based color-balanced underwater image restoration method. When there are white objects or bright pixels in the close range, because DCP and its variants usually use a certain global value as an approximation of the background light, the estimated approximation of background light is too large, which means the visual effect of the restored result is not good. The current paper proposes a method for estimating background light that simultaneously considers blurriness, smoothness, and intensity difference between the red and blue-green channels. In our method, the problem of excessively large background light estimations for underwater images containing white objects in the foreground was solved effectively. Depth-map estimated by MIP-based methods usually leads to miscalculation. Our method establishes an improved depth map estimation model, and separately calculates the red-light intensity difference between light and dark channels, the difference between green–blue and red channels by considering the hue, and red-light intensity. Therefore, the transmission map from the depth map is more accurate and can be adjusted by the reverse saturation map to remove the impact of artificial light sources on underwater image scenes.
2.3 Deep learning-based method
In recent years, deep learning algorithms have increasingly been applied to various visual tasks. Water-Net was proposed by Li et al. [25], but the restoration results are poor contrast. Relying on underwater image priors, Li et al. [26] developed a convolutional neural network (CNN) framework to improve underwater image issues, and this CNN was found to have excellent generality for different underwater scenes. Li et al. [11] came up with a network model with multicolor space embedding, here guided by media transfer, to boost the visual effect of underwater scenes. To estimate pixel-level as well as higher-order curves, Li et al. [26] trained a lightweight network model that can improve the color projection of underwater images, which adjusted the dynamic range of input degraded images. Because of the intricacy of the undersea world, it is still tough to obtain high-quality training data, and there are still problems in recovering the details of images. Existing deep learning frameworks still suffer from color casts or blurred details in underwater images because it is incapable of extracting deep features fully.
3. Proposed method
The current paper presents an efficient underwater image restoration solution consisting of four steps. First, we build a robust BL estimation model that relies on prior features of blurriness, smoothness, and the difference between the intensity of the red and blue-green channels. Next, a depth map is estimated from the red-light intensity, the difference between light and dark channels, and the difference between the R and G-B channels by considering the hue. Then, the effect of artificial light sources on the underwater image is removed using the adjusted inverse saturation map. Comparison by using our experimental results with state-of-the-art underwater processing techniques showed that our method achieves color correction and enhances contrast and edge texture information simultaneously, which indicates that the restored images produce better visual effects.
3.1 Background light (BL) estimation
BL estimation is one of the critical links in underwater image restoration when it comes to determining the visual effects of the restored image, which include tone, contrast, and scene brightness. Typically, there are two ways of BL estimation: 1) As the global background light, the largest pixel value in the degraded image can be selected. 2) The brightest pixel in the largest 0.1% in the dark channel can be selected. Suppose the underwater image has many white objects or highlighted pixel regions. In that case, the estimated value of the background light is likely to be too large, resulting in underexposure and a loss of details after restoration. Therefore, such methods do not have strong robustness or adaptability for estimating background light.
In general, the background light is captured in smooth areas that are away from the camera. The surrounding area is characterized by blurring and significant chromatic aberrations. To this end, the estimation of the background light adopts a scoring mechanism that integrates the smooth area, blur degree, and difference between the red channel and blue-green channels intensity of the image. The channel with the highest score is determined as the final BL estimation value. Since we consider both hue and chromatic aberration, this method can eliminate interference, including white objects or bright pixels in the near field. The proposed method considers blur, smooth, hue, and chromatic aberration, so it can also effectively solve the poor restoration effect due to the invalidation of prior knowledge caused by the lighting conditions of underwater images.
We develop a subregion score $S({{I_i}} )$ to determine a suitable picture pixel value to represent the image tone, which is determined by the blurring degree ${S_b}$, the smoothness of the subregion pixels ${S_\omega }$, and the red light attenuation degree ${S_r}$. Therefore, the score for the subregion can be calculated as follows:
where ${I_i}$ represents the subregion, $i \in 1,2,3,4$. The first scoring mechanism of the background light relies on the blur degree of the input image, here using the quadtree segmentation algorithm to loop five times to obtain the most blurred area. The blur estimation of the underwater images is mainly divided into three steps. The initial blur map can be calculated as follows:To obtain smooth subregion pixels, we introduce the difference between the mean and standard deviation of the pixels. This is because of the close link between image brightness and average pixel values. The brighter the image, the larger the average value of the pixels. The lower the standard deviation, the more stable the pixels, making the subregions smoother. As a result, the flatness of the region can be described as follows:
where $I_c^{mean}$ represents average intensities of the color channel $c$ of the subregion ${I_i}$, and ${\gamma _c}$ stands for the standard deviation of the subregion ${I_i}$. Because red light has the longest wavelength. It is attenuated the most in underwater surroundings. While blue and green light has shorter wavelengths, they can travel further underwater, and their pixels have lower red-light intensity and higher blue and green light intensity. As a result, locations with such light intensity feature in the background light scoring mechanism receive higher scores:Fig. 3. Examples of background light estimation for the proposed method. (a) Original degraded input image. (b) Quadtree iterative method to select the background light. (c) The results of the background light estimation.
Figure 4 displays the recovered images, based on blur, flatness, and maximum difference between red and blue-green channels as the background light selection mechanism and our method. The background light estimation value obtained by the method is based only on the blur map as the background light scoring mechanism is too small, resulting in a darker scene after restoration. However, considering only the degree of flatness and the chromatic aberration of hue as the background light scoring mechanism, the estimated value of the background light will be too large, and the restored image will be overexposed. It is not difficult to find that our method as a background light scoring mechanism obtains the most accurate background light and produces the best visual results. Figure 5 presents the color space maps of the underwater scenes before restoration and uses different background light candidates for comparison. As seen in Fig. 5(e), our approach adjusts to the greatest extent possible for the red component. It produces a more uniform color distribution, implying that the suggested method better handles color distortion in underwater scenes.
Fig. 4. An example of background light comparison. (a) Degraded images. (b)-(d) obtained by separately defined background light scoring criterion. (e) is obtained by our method. (f) were obtained using the proposed method to estimate the transmission maps.
Fig. 5. Background light assessment results. The first row shows the original image, and the restoration results based on the individually defined BL scoring criteria and our method. The second row displays the corresponding RGB color space maps.
3.2 Depth estimation
We propose combining three depth estimation methods to estimate the final scene depth maps. First, we use the difference between bright and dark channels to estimate the depth. The between bright ${I_{bright}}$ is written as follows:
The difference between the bright channel ${I_{bright}}$ and dark channel ${I_{dark}}$ is used in the estimation of the depth ${I_{{depth}}}$, ${I_{{depth}}}$ is expressed as:
We first get a first estimation of the depth map ${I_{{depth}}}$, The larger the value of ${I_{{depth}}}$, the greater the depth of the Scenes:
where ${H_\textrm{s}}(T )$ represents the stretch function:Next, underwater images tend to be blue or green because of the extreme absorption of red light in the undersea environment. The second candidate for depth is based on the chromatic aberration that considers the hue. First, we determine the dominant hue of the input image:
Our second depth map is estimated to be as follows:
This depth map considers that the bigger disparity between the blue channel and red channel for a blue-hued scene point, the farther the point is away from the camera. The bigger disparity between the value of the green channels and red channels for a green-hued scene point, the farther away the point is from the camera.
Our third depth map uses the red-light intensity of the image to estimate depth. The following is how the red channel is defined:
where $R({x} )$ used for the estimation of the depth map:For a pixel location, this depth map considers that the point is closer to the camera in a scene that preserves more red light. Hence, an estimation method for depth is proposed:
Figure 6 shows three candidate depth maps ${{d}_{{D_1}}},{{d}_{{D_2}}},{{d}_{{D_3}}}$ and depth maps constructed using our method. The depth maps obtained by ${{d}_{{D_1}}}$ and ${{d}_{{D_2}}}$ incorrectly regard the fish and corals in the foreground as the background. The depth maps estimated by ${{d}_{{D_3}}}$ is not as accurate as the depth map estimated by the proposed method, and the restoration results effectively remove color distortion while presenting more natural colors and better contrast.
Fig. 6. Depth map estimation contrast tests. (a) Original image. (b-d) The estimated depth maps by ${{d}_{{D_1}}}$, ${{d}_{{D_2}}}$ and ${{d}_{{D_3}}}$. (e) The estimated depth maps using the proposed method. (g) The images were restored by our method.
3.3 Remove artificial light
To calculate the transmission of the red channel, the distance between the position of the scene and the camera is calculated. Here, we first find the distance between the camera and the closest scene position ${{d}_0}$. The estimate is determined using the largest disparity between the background light intensity and observed degraded image:
Combine with Eq. (20) and Eq. (22), the final scene depth is given as Eq. (23):
where D is the proportionality constant that transforms the relative distance into the actual distance. Based on the final scene depth, the transmission of the red channel is estimated as follows: where ${\beta ^{r}}$ is the spectral volume attenuation coefficient for the red channel for the vast majority of the world’s marine environments ${\beta ^{r}} \in \left[ {\frac{1}{8},\frac{1}{5}} \right]$[28,29]. We have performed an estimate of red channel transmission, but we have not considered the effects of artificial light sources. Next, we further address exposure problems involving the influence of artificial light and the effects of uneven lighting by using an adjusted reversed saturation map [30]. In Eq. (25), we define a type of saturation that can describe the chromatic purity of a pixel:The purest manifestation of color is when it is fully saturated. In other words, white light contains all wavelengths of energy, so adding white light causes the color to lose its saturation. As a result, an image with low saturation can be regarded as having a lot of white light that is caught in it. In an underwater scene, since the red light decays faster, and the blue light and green light attenuate to a lesser degree, in an underwater scene without artificial light sources, the saturation is significantly greater than in scenes with artificial light sources. To improve the predicted fine transmission and eliminate the effects of nonuniform and artificial light, we utilize the reversed saturation map (RMS), which is defined in Eq. (26). Areas with high reversed saturation map values are usually areas that are more affected by artificial light.
To effectively improve the uniformity of light, we introduce the fitting parameter $\lambda \in [{0,1} ]$ as an effective scalar multiplier, and the adjusted reversed saturation map (ARMS) is calculated as follows:
where $\lambda $ is set to 0.7. When estimating the ARSM, the transmission information in the artificially illuminated area can simply be used to correct the error transmission. In the absence of artificial lighting, transmission information from regions of the ARSM low-intensity region is preserved. Therefore, the corrected transmission estimation for the R channel can be calculated as follows:As seen in Fig. 7, the corrected transmission estimation using the reverse saturation map is more accurate, and the lighting intensity of parts in the image with artificial light sources is reduced, while the illumination intensity of the other sections of the image is maintained.
Fig. 7. ARSM. (a) Original image. (b) The coarse R channel transmission map without ARSM. (c) An accurate R channel transmission map using ARSM. (d) Image recovered using our method.
In [12], the link between the attenuation coefficients of different color channels is obtained using the unique optical characteristics of water:
The final recovery formula is as follows:
Proposed color correction introduces the white balance algorithm based on the optimal gain factor [30]. The RGB space map before and after restoration is displayed in Fig. 8(a), which shows an evaluation of the restoration capabilities of the method. In the image processed by this approach, the intensity of each color channel is extended. For the red (as illustrated in Fig. 8(c1)), the pixel intensity values are increased, while for the green and blue channels, the pixel intensity values are reduced (as shown in Fig. 8(c2) and Fig. 8(c3)). Here, the color perception of the restored underwater image is greatly enhanced. We evaluate our detail enhancement capabilities by applying the Prewitt algorithm to degraded and restored images (Fig. 8(d)). After image restoration, the visible edge detail is increased.
Fig. 8. (a) Input Images. (b) The RGB space maps of the original and restored images. (c) The R/G/B channel histograms of the original and restored images. (d) The edge detail detection of the original and restored images.
4. Experiments and results
The following section shows how much the effectiveness and robustness can be improved by adopting the proposed method for many underwater scenes. Comparison between multiple underwater enhancement methods, including RBE [17], RGHS [18], Water-Net [25], DCP [21], GDCP [23], underwater image enhancement convolutional neural network (UWCNN) [13], and underwater light attenuation prior (ULAP) [31] was conducted with qualitative and quantitative evaluation. Water-Net [25] extracts a random set of 800 pairs of images from the UIEB dataset [25] to generate the training set, and the remaining 90 pairs of images from UIEB are regarded as the test set. UWCNN uses the RGB-D NYU-V2 indoor dataset [32] to construct a dataset of ten types of underwater images, which consists of 1449 images. UWCNN selects the first 1000 images as the training set and the remaining 449 images as the test set.
4.1 Qualitative comparison
First, we verify the enhancement effect of the algorithm on the overall details and local details of underwater images through comparative experiments. The overall details and local details are shown in Fig. 9 and Fig. 10, respectively. Figure 9 represents the effectiveness of the proposed method in enriching the detailed texture information of underwater images compared with the input degraded images. In the red-marked part of Fig. 10(b), the proposed method improves the local details of the image, such as the texture information of the statues and corals.
Fig. 9. Edge detection comparison test. (a) Input image. (b) The edge detection image of the degraded image. (c) The restored image is based on the proposed method. (d) Edge detection images of the restored results are based on our method.
Fig. 10. Local details comparison test. (a) Input image. (b) The restoration results are based on the proposed method.
Second, we show the results obtained by contrasting methods of various underwater image processing schemes on the underwater images of two tones from the UIEB dataset [25]. These schemes include RBE [17], RGHS [18], Water-Net [25], DCP [21], GDCP [23], UWCNN [13] and ULAP [31].
The results are shown in Fig. 11(b) indicate that RBE [17] has the problem of excessive enhancement and black artifacts in the output image of some scenes. This is because RBE modifies pixel values directly without considering degradation reasons, leading to over-enhancement when correcting underwater images. RGHS [18] helps remove haze and denoise by designing global histogram stretching and using bilateral filtering to remove the influence of noise on the image. In Fig. 11(c), the RGHS processing results failed to correct color distortion in the underwater scenes because this method has problems of poor robustness in the color correction [18]. The proposed method in the current paper better reflects the actual tone of the underwater images.
Fig. 11. Qualitative comparison of image enhancement between the proposed and state-of-the-art methods. (a) Original image (b) RBE [17] (c) RGHS [18] (d) Water-Net [25] (e) DCP [21] (f) GDCP [23] (g) UWCNN [13] (h) ULAP [31] (i) proposed method.
Low contrast and brightness have been discovered in the results from Water-Net [25] and UWCNN [13] methods, as shown in Fig. 11(d) and Fig. 11(g). The restoration results of UWCNN show excessive greenness and blurred structure. It can be seen that our method effectively improves the contrast of the images and the visibility. The restoration images of DCP [21] lead to errors in global BL estimation in those scenes containing white objects and bright pixels in the close range. A red light appears attenuated in water but is not separately compensated for by the red component, so the recovery results of DCP usually do not remove color distortion. Figure 11(f) presents the underwater image restoration results of the GDCP [23] method. Because dark pixels are selected as the background light in some specific scenes, the restored images will have reduced visibility, be overexposed and lose some edge details. In Fig. 11(g), ULAP [31] does not consider the effects of color distortion, cannot handle different color shifts, and introduces orange in the restored images of some scenes.
Because of the estimating approach of the background light and depth map, the images processed using the suggested approach have a more realistic look, as illustrated in Fig. 11(i). Because the proposed BL estimation method relies on the prior features of blur, smoothness, and chromatic aberration, a more accurate background light can effectively eliminate the color cast of the input image. The color bias is greatly corrected, which can achieve better visual quality. The red-light intensity, the difference between light and dark channels, and the disparity between the red and green-blue channels that consider the hue calculate the depth map. Then, the ARMS removes the influence of the artificial light source of the underwater images, and the most accurate transmission map is obtained through calculation, so the restored images have the highest visibility. In addition, the restored results tend to retain more edge details and have a higher contrast because our restoration model of depth map estimation plays a crucial role. In summary, the processed results of our proposed method have more detailed information, higher contrast, and better-restored images.
In addition, to more appropriately assess the effectiveness and reliability of the approach, we compare the underwater images of different tones in the UCCS dataset with those based on the methods of RBE [17], RGHS [18], Water-Net [25], DCP [21], GDCP [23], UWCNN [13] and ULAP [31]. These methods improved the clarity of underwater images to varying degrees. Still, the restoration results of RGHS and DCP failed to effectively remove the color cast, as shown in Fig. 12(c) and Fig. 12(e). RBE enhances the restoration results and introduces many black artifacts. As shown in Fig. 12(d), Water-Net has better restoration effects and retains more details, but there is still a certain gap between these and the restoration results of the current paper. The color of the algorithm in the present paper is richer, while Water-Net makes underwater images appear darker. As shown in Fig. 12(f), GDCP produces overexposure in the recovery results, and some images also exacerbate the overall blurring of the original image. The overall results of UWCNN [13] show that the problems of low contrast and color bias in underwater images have not been solved. ULAP [31] does not take into account the effects of color distortion, and cannot handle different color shifts. The comparison in different environments shows that the restoration results of the algorithm in the current paper not only have the highest contrast and color saturation but also retain edge details and change the ambiguity of the original images to obtain better visual quality.
Fig. 12. Qualitative comparison of image enhancement between the proposed and state-of-the-art methods. (a) Original image (b) RBE [17] (c) RGHS [18] (d) Water-Net [25] (e) DCP [21] (f) GDCP [23] (g) UWCNN [13] (h) ULAP [31] (i) proposed method.
4.2 Quantitative comparisons
The current paper evaluates image restoration quality based on contrast color and overall effect. A comprehensive evaluation of the hue, saturation, and contrast of the recovered images was done by adopting the underwater color image quality evaluation index UCIQE [33]. The contrast of the restored images is assessed using the patch contrast quality metric (PCQI) [34]. Higher PCQI values indicate higher contrast. Underwater image quality measures (UIQM) [35] represent a linear combination of hue, clarity, and contrast, with higher values indicating higher contrast. Mean Squared Error (MSE) is the mean squared sum of the differences between the processed image pixel values and the original pixel values. The larger the value of MSE, the larger the color contrast gap between the two images. The average gradient (AG) can be used to measure the clarity of the fused image. It can be considered that the larger the average gradient, the better the image clarity.
Table 1 shows the quantitative evaluation results of Fig. 11 in UCIQE [33], PCQI, and UIQM [35]. In addition, quantitative evaluation for the above indicators on the complete UIEB dataset is also provided to eliminate the influence of separated samples on the algorithm indicators in Table 2. In our method, UIQM is ranked first and the rest of the four indicators are in the top three in the whole UIEB dataset. Learning-based methods tend to perform better, but on the entire UIEB dataset, the deep learning-based methods UWCNN and Water-Net are far lower than the proposed method in this paper on all four objective metrics. It shows that our method effectively enhances the details and color information of the images, and has a better visual effect.
Table 1. Comparison of the UCIQE, UIQM, and PCQI of Fig. 11
Table 2. Average quantitative metrics of different approaches in UIEB Dataset
The results reveal that UCIQE [33], UIQM [35], AG, and MSE of the method are greater than those of previous techniques, indicating that it can better restore image visibility as well as clarity and balance the hue, saturation, and contrast. RGHS [18] directly adjusts pixels to achieve the optimal result for underwater image restoration, resulting in over enhancement without considering degradation reasons, so the indicator UCIQE is slightly higher than in our method. The objective indicators of DCP [21] are not much different from the original image, so the value of MSE is much smaller than our method. Water-Net [25] is another method based on a deep learning technique that does not perform well in enhancing contrast. The main drawback of GDCP [23] is the overexposure issue due to the choice of background light with insufficient brightness. All metrics of ULAP [31] and UWCNN [13] on the entire dataset are lower than the method proposed in this paper. The visual quality of images generated by our solutions is often higher than other ones, hence better balancing the chroma, contrast, and color saturation of the images. The evaluation results from UCIQE [33], UIQM [35], AG, and MSE demonstrated that our method has superior robustness and effectiveness in image restoration with different degraded images.
5. Conclusion
Based on the characteristics of underwater imaging, we have presented a novel underwater restoration approach of BL estimation and depth map fusion, including a new background light and depth map estimation method. The estimation model of background light relies on blur, chromatic aberration, and smoothness. Accurate background light can effectively remove the color cast problem of underwater images. At the same time, it can avoid the overestimation of background light that is caused by white close-ups or bright pixel areas. To increase the contrast and chromatic fidelity of the image, a differential depth map estimation model based on the red light intensity, the difference between light and dark channels, and the difference between red and green-blue channels by considering the hue is introduced. In addition, the approach can remove the influence of artificial light on underwater images and conserve the features on the edges of the images, which benefits from using an inverse saturation map. Comprehensive evaluations indicate the effectiveness of the BL estimation method, the rationality of the estimation approach through the proposed depth map, and the superior performance of our model.
Funding
National Natural Science Foundation of China (U20A20161).
Disclosures
The authors declare that there are no conflicts of interest related to this paper.
Data availability
Data underlying the results presented in this paper are not publicly available but may be obtained from the authors upon reasonable request.
References
1. T. Li, S. Rong, W. Zhao, L. Chen, Y. Liu, H. Zhou, and B. He, “Underwater image enhancement using adaptive color restoration and dehazing,” Opt. Express 30(4), 6216–6235 (2022). [CrossRef]
2. T. Li, Q. Yang, S. Rong, L. Chen, and B. He, “Distorted underwater image reconstruction for an autonomous underwater vehicle based on a self-attention generative adversarial network,” Appl. Opt. 59(32), 10049–10060 (2020). [CrossRef]
3. W. Ren, J. Pan, H. Zhang, X. Cao, and M. Yang, “Single image dehazing via multi-scale convolutional neural networks with holistic edges,” Int J Comput Vis 128(1), 240–259 (2020). [CrossRef]
4. J. Zhou, T. Yang, W. Ren, D. Zhang, and W. Zhang, “Underwater image restoration via depth map and illumination estimation based on a single image,” Opt. Express 29(19), 29864–29886 (2021). [CrossRef]
5. P. Zhuang, C. Li, and J. Wu, “Bayesian retinex underwater image enhancement,” Engineering Applications of Artificial Intelligence 101, 104171 (2021). [CrossRef]
6. J. Zhou, D. Zhang, W. Ren, and W. Zhang, “Auto color correction of underwater images utilizing depth information,” IEEE Geoscience and Remote Sensing Letters, 19 (2022).
7. J. Zhou, D. Zhang, and W. Zhang, “Classical and state-of-the-art approaches for underwater image defogging: a comprehensive survey,” Front Inform Technol Electron Eng 21(12), 1745–1769 (2020). [CrossRef]
8. J. Zhou, T. Yang, W. Chu, and W. Zhang, “Underwater image restoration via backscatter pixel prior and color compensation,” Engineering Applications of Artificial Intelligence, 111 (2022).
9. Y. Guo, H. Li, and P. Zhuang, “Underwater image enhancement using a multiscale dense generative adversarial network,” IEEE J. Oceanic Eng. 45(3), 862–870 (2020). [CrossRef]
10. C. Li, J. Guo, and C. Guo, “Emerging from water: underwater image color correction based on weakly supervised color transfer,” IEEE Signal Process. Lett. 25(3), 323–327 (2018). [CrossRef]
11. C. Li, S. Anwar, J. Hou, R. Cong, and W. Ren, “Underwater image enhancement via medium transmission-guided multi-color space embedding,” IEEE Trans. on Image Process. 30(99), 4985–5000 (2021). [CrossRef]
12. S. Anwar and C. Li, “Diving deeper into underwater image enhancement: a survey,” Signal Processing: Image Communication 89 (2019).
13. C. Li and S. Anwar, “Underwater scene prior inspired deep underwater image and video enhancement,” Pattern Recognition 98, 107038 (2020). [CrossRef]
14. J. Zhou, Y. Wang, W. Zhang, and C. Li, “Underwater image restoration via feature priors to estimate background light and optimized transmission map,” Opt. Express 29(18), 28228–28245 (2021). [CrossRef]
15. J. Jaffe, “Computer modeling and the design of optimal underwater imaging systems,” IEEE J. Oceanic Eng. 15(2), 101–111 (1990). [CrossRef]
16. B. Mcglamery, “A computer model for underwater camera systems,” Proc. SPIE 0208, 221–231 (1980). [CrossRef]
17. X. Fu, P. Zhuang, H. Yue, Y. Liao, X. P. Zhang, and X. Ding, “A retinex-based enhancing approach for single underwater image,” 2014 IEEE International Conference on Image Processing (ICIP), 4572–4576 (2015).
18. D. Huang, W. Yan, S. Wei, J. Sequeira, and S. Mavromatis, “Shallow-water image enhancement using relative global histogram stretching based on adaptive parameter acquisition,” International Conference on Multimedia Modeling 10704, 453–465 (2018). [CrossRef]
19. J. Zhou, D. Zhang, and W. Zhang, “Underwater image enhancement method via multi-feature prior fusion,” Applied Intelligence, 1–23 (2022).
20. N. Carlevaris-Bianco, A. Mohan, and R. M. Eustice, “Initial results in underwater single image dehazing,” IEEE Oceans, 1–8 (2010).
21. K. He, J. Sun, and X. Tang, “Single image haze removal using dark channel prior,” IEEE Transactions on Pattern Analysis and Machine Intelligence 33(12), 1397–1409 (2009). [CrossRef]
22. J. Drews, E. Nascimento, F. Moraes, S. Botelho, and M. Campos, “Transmission estimation in underwater single images,” IEEE International Conference on Computer Vision Workshops, 825–830 (2013).
23. Y. Peng, K. Cao, and P. Cosman, “Generalization of the dark channel prior for single image restoration,” IEEE Transactions on Image Processing 27(6), 2856–2868 (2018). [CrossRef]
24. H. Lee, S. Moon, and I. Eom, “Underwater image enhancement using successive color correction and superpixel dark channel prior,” Symmetry 12(8), 1220 (2020). [CrossRef]
25. C. Li, C. Guo, W. Ren, R. Cong, J. Hou, S. Kwong, and D. Tao, “An underwater image enhancement benchmark dataset and beyond,” IEEE Trans. on Image Process. 29, 4376–4389 (2020). [CrossRef]
26. C. Li, C. Guo, and C. L. Chen, “Learning to enhance low-light image via zero-reference deep curve estimation,” IEEE Transactions on Software Engineering (2021).
27. Y. Peng and P. Cosman, “Underwater image restoration based on image blurriness and light,” IEEE Trans. on Image Process. 26(4), 1579–1594 (2017). [CrossRef]
28. J. Chiang, Y. Chen, and Y. Chen, “Underwater image enhancement: using wavelength compensation and image dehazing,” 13th International Conference on Advanced Concepts for Intelligent Vision Systems (ACIVS), 372–383 (2011).
29. W. Jagger and W. Muntz, “Aquatic vision and the modulation transfer properties of unlighted and diffusely lighted natural waters,” Vision Res. 33(13), 1755–1763 (1993). [CrossRef]
30. W. Song, Y. Wang, D. Huang, A. Liotta, and C. Perra, “Enhancement of underwater images with statistical model of background light and optimization of transmission map,” IEEE Trans. Broadcast. 66(1), 153–169 (2020). [CrossRef]
31. W. Song, Y. Wang, D. Huang, and D. Tjondronegoro, “A rapid scene depth estimation model based on underwater light attenuation prior for underwater image restoration,” 19th Pacific-Rim Conference on Multimedia (PCM), 678–688 (2018).
32. N. Silberman, D. Hoiem, P. Kohli, and R. Fergus, “Indoor segmentation and support inference from rgbd images,” 12th European Conference on Computer Vision (ECCV), 746–760 (2012).
33. M. Yang and A. Sowmya, “An underwater color image quality evaluation metric,” IEEE Trans. on Image Process. 24(12), 6062–6071 (2015). [CrossRef]
34. S. Wang, K. Ma, H. Yeganeh, Z. Wang, and W. Lin, “A patch-structure representation method for quality assessment of contrast changed images,” IEEE Signal Process. Lett. 22(12), 2387–2390 (2015). [CrossRef]
35. K. Panetta, C. Gao, and S. Agaian, “Human-visual-system-inspired underwater image quality measures,” IEEE J. Oceanic Eng. 41(3), 541–551 (2016). [CrossRef]
