1. Introduction

Underwater optical imaging technology plays an important role in underwater optics and ocean optics. It is an important tool for humans to understand the ocean, develop and utilize the ocean, and protect the ocean. This technology has been widely used in the underwater target recognition, underwater archaeology, submarine resource exploration, and other fields [1]. However, due to the internal optical characteristics of pure water and the optical attenuation caused by the medium (the scattering and absorption of solute molecules and suspended particles) in the water, the quality of underwater optical images is severely reduced. The poor visibility and color cast caused by the effects of underwater optical imaging reduce the ability to extract valuable features from underwater images for further processing [2–4]. Therefore, to obtain high-quality underwater optical images and improve research in the field of underwater optics, it is necessary to restore and enhance degraded underwater optical images. At present, the restoration and enhancement of underwater optical images have been studied from several perspectives, including the use of dedicated hardware, polarization filtering technology, and scene model estimation. An important approach is the methods using specialized hardware. For example, the divergent underwater lidar imaging (UWLI) system can accurately capture nanosecond-level fast gated images in highly turbid water [5], thereby significantly reducing the noise related to water scattering in underwater images. Unfortunately, these complex acquisition systems are very expensive and consume power.

A second approach consists of polarization-based methods. These approaches use several images of the same scene captured with different degrees of polarization, as obtained by rotating a polarizing filter fixed to the camera. For example, to suppress noise in pixels of distant objects with low medium transmittance, an adaptive filtering method has been proposed [6]. Although polarization filtering technology can be used to restore distant areas, it is not suitable for dynamic acquisition. Schechner and Karpel [7] proposed and successfully demonstrated a computer vision-based method to eliminate degradation factors in the underwater optical imaging process.

The third approach employs multiple images [8] or an approximation of the scene model [9]. Narasimhan and Nayar [8] presented a physics model that describes the appearances of scenes in uniform bad weather. A system for browsing casual outdoor photos combined with the existing georeferenced digital terrain model has been proposed [9]. Although it has achieved sufficient dehazing and view synthesis functions, the enhancement effect of the image is not ideal.

The fourth type of method takes advantage of the similarity of light propagation in haze and underwater areas. In this fusion method, results are obtained from two original blurred image inputs by applying white balance and contrast enhancement programs [10]. Ju et al. [11] proposed an improved atmospheric scattering model (IASM) to overcome the inherent limitations of the traditional atmospheric scattering model and combined it with the acceleration framework based on the Gauss-Laplace pyramid to increase the calculation speed. However, underwater imaging is even more challenging because the extinction resulting from scattering depends on the wavelength of light, i.e., on the color component.

Recently, researchers have proposed other algorithms. Peng and Cosman [12] proposed a depth estimation method for underwater scenes based on image blurriness and light absorption, which is employed to enhance underwater images. An image restoration method based on a dark channel prior (DCP) is proposed [13]. It assumes that the radiance of objects in a natural scene is small in at least one color component. Therefore, the area with low transmittance is defined as the area with the smallest color value. Liu et al. [14] developed wavefront engineering technology based on internal guide stars, which increased the speed of optical focusing inside the scattering medium by two orders of magnitude and focused the scattered light inside the dynamic scattering medium. By imaging the target, we demonstrated the first focusing of scattered light inside a dynamic scattering medium containing living tissue. Li et al. [15] proposed a real-time color correction deep learning method based on an underwater unsupervised generative confrontation network, called WaterGAN. Similarly, Li et al. [16] proposed a method of image scattering and classification based on deep neural networks. Although the scattering is eliminated and the color is enhanced, the edge details are not well maintained. Zeng et al. [17] proposed a quaternion principal component analysis network (QPCANet) that uses quaternion theory to extend PCANet for color image classification. In contrast to PCANet, QPCANet considers the spatial distribution information of the RGB channels in the color image and uses the quaternion field to represent the color image to ensure greater intraclass invariance. [18] and [19] proposed a fusion-based underwater optical image enhancement method. The fusion framework obtains input and weight metrics from the degraded version of the image and implements an effective edge-preserving noise reduction strategy to support the temporal coherence between adjacent frames. Khan and Akmeliawati [20] developed a neural network-based dynamic integral sliding mode control (NNDISMC) with an output differential observer for higher derivative estimation and neural networks to estimate nonlinear functions assumed to be unknown. This method has good robustness. The deep convolutional neural network (CNN) method is used to perform super-resolution analysis of underwater optical images of different bands [21–24]. Li et al. [21] proposed an underwater image enhancement convolutional neural network (UWGAN) model based on the underwater optical scene prior and reconstructed a clear underwater image through the underwater scene of synthetic training data. Fabbri et al. [23] proposed a novel underwater optical image enhancement method based on a generative adversarial network (UGAN), whose purpose is to further improve the input of vision-driven behavior downstream of the custom pipeline. Zhu et al. [24] developed unpaired image-to-image translation using cycle-consistent generative adversarial networks (CycleGAN). Their goal was to use training on a set of image pairs to learn the mapping between input and output images.

For underwater optical images, it is worth mentioning that there are also some special enhancement methods. Song et al. [25] proposed an image enhancement method consisting of underwater optical image restoration and color correction. The method provided a reliable background light (BL) estimation statistical model and estimated the transmission map (TM) based on the new underwater dark channel prior (NUDCP), which dehazed the image and gave it a natural appearance. From the perspective of style level and feature level adaptation, Ye et al. [26] proposed an unsupervised style adaptive network (SAN) to address the problem of common learning. Although the problem of color distortion and low contrast is solved, the problem of the loss of edge details is not yet sufficiently resolved. Inspired by the morphology and function of the bony fish retina, Gao et al. [27] proposed an underwater optical image enhancement mechanism, which greatly improved image quality. From the perspective of deep learning networks, Guo et al. [28] proposed a multiscale densely generated confrontation network (GAN), which effectively preserves the edge details of optical images. Fu et al. [29] proposed a single underwater optical image enhancement method based on retinex, which effectively solved the problems of underexposure. Zhou et al. [30] proposed an Airy spiral phase filter to enhance the edges of isotropic and anisotropic images and verified the effectiveness of the enhancement method by using amplitude contrast and phase contrast objects. Li et al. [31] proposed an enhancement method for optical wavefront shaping, which achieves the final effect by obtaining hologram and edge information inversion decoding after the scattering medium. Xia et al. [32] attributed image degradation to the wideband loss of high spatial frequency information caused by the high refractive index and proposed a high spatial spectrum enhancement reconstruction method, which effectively enhanced the edge details of the image. Chen et al. [33] proposed an accelerated processing method based on a fully convolutional network to improve the dehazing effect and realism. Peng et al. [34] proposed a generalized dark channel prior (GDCP), which incorporated adaptive color correction into an image formation model. A dehazing method based on the principle of minimum information loss and a contrast enhancement algorithm based on a priori histogram distribution, which effectively improves the visual effect, has been proposed [35].

The above methods will improve underwater optical images to a certain extent in one or several of the following aspects, yet underwater image restoration is still challenging due to the unpredictable variation in vision properties in undersea environments. There is no method that can achieve these effects more comprehensively than current standards: dehazing, denoising, color correction, detail enhancement, and uneven lighting brightness processing, in particular, white objects in the image, floating particles in the foreground, artificial light in the background region, or dim background light. Many state-of-the-art methods cannot solve these problems well. To compensate for the problems of existing methods, this paper proposes an underwater optical image enhancement method based on background light estimation and improved adaptive transmission fusion. First, this paper proposes a BL estimation method based on the gray close operation. Then, an improved adaptive transmission fusion (IATF) method is proposed, and TMs based on the dark channel prior (DCP-W) and TMs based on the new underwater optical attenuation prior (NULAP) are calculated separately. Next, fusion obtains the rough TMs. Underwater light attenuation prior (ULAP) and adjusted reverse saturation map (ARSM) principles are further used to compensate for and refine TMs. Finally, this paper proposes a statistical colorless slant correction fusion smoothing filter method, which greatly improves the quality of underwater optical images.

The contributions of this paper are summarized as follows:

 

Fig. 1. Underwater light propagation scene. Entry into the water from the atmosphere results in the gradual attenuation of light. Point x represents the scene point closest to the camera.

Download Full Size | PDF

Reflected light reaching the camera through an object is shown in Fig. 1. Scattering is a phenomenon caused by light interaction with the medium in water and is divided into backscattering and forward scattering. Backscattering describes the scattering of ambient light according to the line of sight before finally propagating to the image plane, resulting in large reductions in scene contrast. Forward scattering occurs when part of the reflected light is transmitted at a small angle and easily causes image blur.

2. Related work

2.1 Background light estimation

A simplified image formation model (IFM), often used to approximate the propagation equation of underwater scattering ([36,37]), is described as

Substituting Eq. (2) into Eq. (1), we get:

The values of ${t^c}(x )$ for an entire image can form a TM, which describes the portion of the scene radiance that is not scattered or absorbed and reaches the camera. When the haze is uniform, ${t^c}(x )$ can be expressed as:

2.2 Transmission map estimation

Many factors impact the image in the underwater environment, including dust-like particles in the medium that cause changes in the wavelength-dependent attenuation coefficient. He et al. [13] proposed the discrete cosine transform. It is believed that in most nonsky patches, at least one pixel RGB channel in the local patch has a very low intensity in the statistical prior of outdoor haze-free images:

Applying the smallest filter to Eq. (1) and dividing it by ${B^c}$, we can get:

Putting (6) into (7), ${t_{DCP}}(x )$ can be calculated:

In underwater scenes, the red attenuation is the fastest, so we only consider defining the $DC{P_{gb}}$ of the G and B channels as underwater DCP (UDCP) [12,34], and then using ${I^c}(y )$ instead of ${I^{c^{\prime}}}(y )$, ${t_{UDCP}}(x )$ can be obtained. The TM is estimated by calculating the difference between the maximum intensities of the R channel and the G-B channel. The formula is as follows:

Unlike the above method, the scene depth is estimated by combining three depth maps, which include the maximum filter, maximum intensity, and image blur of the red channel:

3. Methods in this article

A flowchart of the methods used in this paper is shown in Fig. 2.

 

Fig. 2. Flowchart for the proposed method.

Download Full Size | PDF

3.1 Background light estimation based on the gray close operation

The background light ${B^c}$ is an important factor in determining the color and brightness of the underwater image. When the value of ${B^c}$ is too large, overexposure will occur, and if the value of ${B^c}$ is too small, the light will be too dark. We propose a background light estimation method based on gray close operation and can eliminate the interference of white objects. In Eq. (1), when ${\rho ^c}(x )\to 0$ and ${\rho ^c}(x )\to \infty $, ${B^c}$ can be calculated. The analysis of absorption or scattering characteristics shows that the color of the image is produced by absorbing light of a specific frequency, which comes from vertical light. This shows that for lighter or darker objects, at least one of the three channels has a very small reflection coefficient and is close to zero. Therefore, we take the minimum of the three channels on both sides of Eq. (3) at the same time:

Among all the color channels, the red channel has the lowest residual value and BL. This shows $t_{\min}^c(x )= {t^r}(x )$, $B_{\min}^c(x )= {B^r}$. Replace $t_{\min}^c(x )$ and $B_{\min}^c(x )$ in Eq. (14) with ${t^r}(x )$ and ${B^r}$ :

After the above analysis, ${B^r}$ can be calculated when ${\rho ^c}(x )\to 0$ and ${\rho ^c}(x )\to \infty $. For scenes with white objects, there will be the opposite trend. This is because the minimum reflection coefficient of white objects $\rho _{\min}^{whitec}(x )\to 1$, which makes the assumption invalid. To obtain ${B^c}$ robustly, this paper adopts the gray-scale operation method to avoid the interference of white objects:

Taking into account the attenuation and correction of the three channels, the BL estimates of the other two channels are [12]:

Among them, ${\beta ^c}$ is expressed as [37]:

Here, the wavelength values of the three channels are defined as 620 nm, 540 nm, and 450 nm.

3.2 Transmission map fusion estimation

In this section, we propose an IATF method. First, the dark channel prior-water (DCP-W) method [42] is used to estimate TMs1, a NULAP-based method is proposed to obtain TMs2, and finally, the fusion operation is performed. In this method, to avoid the erroneous estimation of the near-field light and the far-field light and to reduce the interference of artificial light, we use the ULAP and ARSM methods to estimate more accurate TMs.

3.2.1 Transmission estimation based on DCP-W

Here, we use the DCP-W method to standardize Eq. (1) and perform the minimization operation:

The transmission coefficients of the green and red channels [42] are estimated as follows:

3.2.2 Transmission estimation based on NULAP

We found that the ULAP [18] method has obvious inaccuracy when processing and recognizing the close-range distance (the shortest distance from the camera to the object) and the long-range distance (the distance from the camera to the distant background) in the scene. Furthermore, when the color of the object is white, it is easy to regard the white object in the foreground as the background scene. Therefore, we propose a NULAP method. We know that the difference between the maximum value of G-B intensity (MVGB) and the value of R intensity (VR) has a strong relationship with the depth of the scene. We build a scene depth model to estimate the TM.

In order to learn the coefficients ${\omega _0}$, ${\omega _1}$ and ${\omega _2}$ accurately, we need relatively correct training data. We collected a large number of underwater images from the dataset in this paper, calculate the scene depth maps, and then manually select the most accurate 100 data sets as the final training data. A guided filter [38] is further used to refine the scene depth map and give the edge details a smoother effect.

To train the model, we take the ratio of training and testing dataset as 7:3 and use 10- fold cross-validation. The best learning result is ${\omega _0}$=0.53214831, ${\omega _1}$=0.51309828 and ${\omega _2}$=-0.91066193.

The scene depth map estimated above is the relative distance. Next, the absolute distance between the camera and the nearest scene point needs to be estimated. From the estimated ${B^c}$ and ${I^c}(x )$, the absolute distance ${d_0}$ is calculated:

For the green and blue channels, we can also calculate:

3.2.3 Transmission fusion estimation

Image fusion refers to the process of creating a single composite image from multiple input images and further integrating the relevant information of two (or more) images of the same scene into one output image. Research on the existing image fusion methods and their applications show that the image output by the fusion method is of higher quality than any single input image. The benefits of image fusion are reduced uncertainty, improved reliability, expanded space and time coverage, and compacted representation of information. Therefore, the purpose of the IATF method proposed in this paper is to reduce the impact of unpredictable factors in the underwater environment on underwater optical images.

To reduce the influence of factors such as potential artificial light and speckles while maintaining better scene details during the image fusion process, we conducted a saliency process analysis and ARSM calculation.

Next, we further solve the exposure problem through ARSM [24], including the influence of artificial light and uneven light.

Because the area with higher red channel intensity (AL) in the underwater blue-green image has lower saturation in the color space, we defined the chromaticity purity of the pixel [24] as follows:

A color loses its saturation upon the addition of white light, which contains energy at all wavelengths. Therefore, saturation reduction in a certain area of an image can be achieved by adding white light to the area. In an underwater image, the saturation of a scene without AL will be much greater than that of the artificially illuminated area. This phenomenon can be represented by the following RSM, where a high RSM value usually denotes the AL-affected area.

As shown in Fig. 3, transmission estimation is inaccurate when the light is uneven or artificial light is applied. Moreover, the comparison of RSM and ARSM processing of the original image revealed that the overall brightness of the image was improved by ARSM.

According to ARSM and Eq. (36), transmission estimation was further modified to reduce the intensity of the light in certain areas while maintaining the intensity in other areas of the image.

 

Fig. 3. ARSM. (a) Original image. (b) Inaccurate transmission estimation. (c) RSM. (d) ARSM, setting λ = 0.7.

Download Full Size | PDF

3.3 Scene light recovery

We assumed that when ${J^c}({x = \alpha } )= 1$ or 0, there exists a brightest or darkest pixel, α, that satisfies the following formula:

The reliability of the attenuation ratio depends on accurate estimation of the ${B^c}$, whereas the transmission of other channels continues. We defined an adaptive attenuation ratio, $\delta _c^{\prime}$, to adjust the transmission using Eq. (1) and Eq. (4) to express the resulting saturation constraint of the restored image as follows:

Leading to

After obtaining BL and final transmission estimates, the scene brightness, $\,J$, is calculated as follows:

 

Fig. 4. Examples of enhancing underwater images. (a) Original image. (b) The corresponding TM. (c) Restored image and (d) Enhanced image.

Download Full Size | PDF

Figure 4 fully shows the comparison of results between different stages of the method in this paper. First, the BL of the original image is obtained from Section 3.1. The BL of the three images from top to bottom are: (0.03, 0.02, 0.02), (0.04, 0.61, 0.89) and (0.06, 0.87, 1). The TMs are further calculated (in section 3.2) according to the BL obtained in Section 3.1, and the result is shown in Fig. 4(b). Then, the result of scene light restoration is obtained according to Section 3.3, as shown in Fig. 4(c). Finally, the result of underwater image enhancement is obtained through Section 3.4, as shown in Fig. 4(d).

3.4 Image enhancement

The image enhancement method proposed in this paper is a fusion smoothing filter method based on statistical colorless slant correction [28].

After the image color is corrected and enhanced, an edge preservation smoothing filter is applied. Inspired by the WLS filter [38] and global guided image filtering [44,45,46], the edge preservation smoothing filter formula is as follows:

The edge preservation smoothing filter is an image that is smoothed and a vector field:

Similarly, the matrix representation is as follows:

The matrices ${D_x}$ and ${D_y}$ represent discrete differential operators and the matrices ${B_x}$ and ${B_y}$ are described as:

By using the fast separation method in [26,27], O* can be solved quickly. Through the above operations, underwater images can well achieve image dehazing, denoising, color correction, and edge detail enhancement, thereby achieving ideal visual effects.

4. Results and evaluation

To concretely prove the effectiveness of this method, we conduct experimental comparisons with several state-of-the-art methods, including the new underwater dark channel prior (NUDCP) [25], a fusion-based method [19], the histogram prior [35], CycleGAN [24], UGAN [23], and UWCNN [21]. The first three are nondeep learning methods, and the last three are deep learning methods. The experimental dataset contains 3000 underwater images with different scenes, and the sample dataset includes 100 underwater images (Fig. 5).

 

Fig. 5. Sample test image. From left to right and then top to bottom: “Image-1” through “Image-100”.

Download Full Size | PDF

In order to make the method in this paper more representative and efficient, this requires our data set to contain underwater images of various depths, various underwater environments, extreme conditions (uneven light, artificial light, etc.), and various entities. The experimental data set of this paper mainly comes from UIEBD [47], RUIE [48], DIV2K [49], Fish4Knowlege, and Google Images. Then, some of these images are divided into the following categories: dim images, hazy images, images with different scattering and viewing angles (low backscatter, high backscatter, forward-looking, and downward-looking), greenish and bluish images, color distortion images, images with uneven lighting, and images with artificial light. Therefore, the data set in this paper can represent the general distribution of underwater images. The results of the different methods and the corresponding reference images are shown in Figs. 7–12. We conduct qualitative and quantitative comparisons and complexity analyses of different datasets.

 

Fig. 6. Image comparison before and after enhancement using the proposed method. (a) Original image and the image (b) before and (c) after enhancement.

Download Full Size | PDF

 

Fig. 7. Comparison results for dim underwater images. From left to right are (a) original images and the results generated by (b) NUDCP [25], (c) the fusion-based method [19], (d) the histogram prior [35], (e) CycleGAN [24], (f) UGAN [23], (g) UWCNN [21], and (h) the proposed method.

Download Full Size | PDF

 

Fig. 8. Comparison results for a hazy underwater image. From left to right are (a) original images and the results generated by (b) NUDCP [25], (c) the fusion-based method [19], (d) the histogram prior [35], (e) CycleGAN [24], (f) UGAN [23], (g) UWCNN [21], and (h) the proposed method.

Download Full Size | PDF

 

Fig. 9. Comparison results of underwater images with different scattering and viewing angles. From left to right are (a) original images and the results generated by (b) NUDCP [25], (c) the fusion-based method [19], (d) the histogram prior [35], (e) CycleGAN [24], (f) UGAN [23], (g) UWCNN [21], and (h) the proposed method.

Download Full Size | PDF

 

Fig. 10. Comparisons of bluish and greenish underwater images. From left to right are (a) original images and the results generated by (b) NUDCP [25], (c) the fusion-based method [19], (d) the histogram prior [35], (e) CycleGAN [24], (f) UGAN [23], (g) UWCNN [21], and (h) the proposed method.

Download Full Size | PDF

 

Fig. 11. Comparison of the effects of color enhancement and detail preservation. From left to right are (a) original images and the results generated by (b) NUDCP [25], (c) the fusion-based method [19], (d) the histogram prior [35], (e) CycleGAN [24], (f) UGAN [23], (g) UWCNN [21], and (h) the proposed method.

Download Full Size | PDF

 

Fig. 12. Examples of underwater image enhancement in extreme scenes with uneven lighting conditions and artificial light environments. From left to right are (a) the original images and the results generated by (b) NUDCP [25], (c) the fusion-based method [19], (d) the histogram prior [35], (e) CycleGAN [24], (f) UGAN [23], (g) UWCNN [21], and (h) the proposed method.

Download Full Size | PDF

4.1 Qualitative estimation

We show the comparison of the results of the enhancement method and the restoration method in this article in Fig. 6.

To further prove the enhancement effect of the statistical colorless slant correction fusion smoothing filter method proposed in this paper, we performed an ablation experiment to show the contrast effect before and after the proposed enhancement method. Figure 6(b) and Fig. 6(c) depict the experimental results before and after the enhancement method, respectively. Through the comparison of experimental results, it can be seen that before using the enhancement method proposed in this article, the color saturation of the image and the sharpness of edge details need to be improved. The effect of the enhancement method is more natural, and the brightness of the image (including the background light in the distance) has been significantly improved. Therefore, the statistical colorless slant correction fusion smoothing filter method proposed in this paper has good effects on color enhancement, edge detail preservation, and image brightness adjustment.

Next, we compared the experimental results in different scenes with those of other existing methods, as shown in Figs. 7–12. The scenes are dim light, haze, different scattering and observation angles, bluish and greenish, color distortion and edge detail loss, extreme scenes with uneven lighting conditions, and artificial light environments.

It can be seen from Fig. 7 that NUDCP [25] can improve the dim environment and the color saturation to a certain extent, but the effect of maintaining edge details and image clarity needs to be further improved. Fusion-based methods [19] can enhance the overall color of the image, making the picture more textured, but cannot present white objects in the scene. The histogram prior [35] can improve the overall brightness of the picture but does not achieve good results in terms of edge detail and overall clarity of the image. CycleGAN [24] achieves good results in terms of color saturation of the image, but the overall brightness of the image is low. The effect achieved by UGAN [23] is remarkable, achieving good results in image brightness, color enhancement, and clarity. UWCNN [21] improves the problems of dim light and color saturation, but the edge details of the image are not well preserved. The method proposed in this paper has great improvements in image contrast, color saturation, and sharpness.

Figure 8 represents a hazy underwater optical scene. It can be seen from the experimental results that the above methods can achieve the dehazing effect, but considering other factors, the fusion-based method [19] does not achieve good results in terms of color saturation, the histogram prior [35] does not achieve a good processing effect on the foreground and background light, and CycleGAN [24] has a good effect in terms of color saturation, but the visual effect of the image is still not ideal. UGAN [23] and UWCNN [21] achieve poor results in edge detail and sharpness.

The four images in Fig. 9 are low-backscatter, high-backscatter, forward-looking, and downward-looking images. NUDCP [25] achieves a good effect in terms of improving the scattering. Although the fusion-based method [19] and histogram prior [35] can eliminate the effects of scattering, they cannot improve the effect of scene images containing greenish hues. CycleGAN [24] makes the overall brightness darker while improving the degree of scattering. UGAN [23] and UWCNN [21] can enhance the saturation of colors but do not improve the clarity of the image very well. It is not difficult to see that the method proposed in this paper achieves the best visual effects in low-backscatter, high-backscatter, forward-looking, and downward-looking.

Figure 10 represents bluish and greenish underwater optical scenes. NUDCP [25] can solve the bluish and greenish problems very well but does not improve the clarity of the image very well when dealing with the blueness of the image. The fusion-based method [19] can significantly improve the clarity of the image, which is significantly better than the effect achieved by NUDCP [25]. The histogram prior [35] can improve the sharpness of the image, but the brightness is darker. CycleGAN [24] and UGAN [23] are not very effective in dealing with the problem of bluishness and greenishness, but they improve the edge detail and sharpness of the image to a certain extent. UWCNN [21] can well eliminate bluish and greenish problems and improve the clarity of the scene target but is prone to artifacts when dealing with distant background light.

Figure 11 represents an image of a scene with color distortion and loss of edge detail. The above methods can enhance the color and edge texture details to a certain extent. NUDCP [25] and the fusion-based method [19] do not achieve natural visual effects for objects appearing in dark areas. The histogram prior [35] does not achieve better edge detail preservation. CycleGAN [24], UGAN [23], and UWCNN [21] can enhance the color saturation and edge details, but the enhancement effect is not good for the local scene area under dim light.

Figure 12 represents extreme scenes with uneven lighting conditions and artificial light environments. NUDCP [25] can solve the interference of uneven light and artificial lighting but will also cause the loss of edge details and will produce an unclear image due to the interference of white objects in the scene. The fusion-based method [19] can eliminate the vignetting caused by artificial lighting, but objects in dark areas cannot be presented. The histogram prior [35], CycleGAN, and UGAN can solve the problem of unclearness in an uneven lighting environment but cannot improve the effect very well for scenes with artificial lighting. UWCNN [21] can effectively reduce the influences of uneven illumination and artificial light. However, when dealing with uneven light scenes, objects in dimly lit areas in the distance are prone to artifacts. In scenes with artificial light, it is not possible to achieve an enhancement effect in dark areas.

It can be seen from Figs. 7–12 that NUDCP [25] can improve the dim underwater light environment very well but cannot preserve the edge texture details of the image well. The fusion-based method [19], the histogram prior [35], and CycleGAN [24] can enhance the color and eliminate bluish and greenish phenomena but cannot solve the overall brightness problem well. UGAN [23] and UWCNN [21] can effectively maintain edge details and improve image clarity, but they cannot improve the visual effect of unevenly illuminated scenes. The underwater image enhancement method based on background light estimation and improved adaptive transmission fusion proposed in this paper can achieve better results in image dehazing, color enhancement, edge detail preservation, and uneven illumination processing.

4.2 Quantitative analysis

We use entropy (Entropy), a gradient average (GA), an underwater image quality metric (UIQM) [21,36], an underwater color image quality evaluation metric (UCIQE) [25,50], and a natural image quality evaluator (NIQE) [40] for quantitative analysis. The entropy of the image can be used to represent the characteristics of image statistics according to the average signal; the size of the entropy value intuitively defines the quality and clarity of an image. The AVG not only reflects the degree of image clarity but also indicates changes in edge texture details; larger AVG values result in decreased blur. The UCIQE metric is a linear combination of the chroma, saturation, and contrast of underwater images. The NIQE uses space domain natural scene statistics, where a small value represents better quality. The UCIQE and UIQM metrics are used for underwater image assessment. The UCIQE metric is designed specifically to quantify the nonuniform color cast, blurring, and low contrast that characterize underwater images. The UIQM is a linear combination of color, clarity, and contrast and comprises the following processes: underwater image chromaticity measurement (UICM), underwater image sharpness measurement (UISM) [27], and underwater image contrast measurement (UIConM) [12]. A larger UIQM value suggests better image enhancement performance. The UIQM is calculated as follows:

Twenty-four sample images (including all the images in this article, numbered 1, 2… 24) and 3000 underwater images are used to confirm the effectiveness of the proposed method. We randomly divide the 3000 underwater images into three groups (G1, G2, and G3) (Table 1). In Table 2, Avg1 represents the average value obtained from 24 images. Avg2 represents the average value of the corresponding G1, G2, and G3. Most of the results show that the proposed method returns better evaluations for underwater images, indicating its superiority in image dehazing, deblurring, edge detail preservation, and color enhancement.

Table 1. UIQM COMPARISON BETWEEN METHODS

View Table | View all tables in this article

Table 2. Average UIQM, UCIQE, and NIQE values of the original images in Fig. 5 and their enhanced versions from all methods

View Table | View all tables in this article

The experimental analysis results are shown in Table 1. In general, the three deep learning methods achieve better results than the three non-deep learning methods. Among the three non-deep learning methods, the evaluation index of NUDCP [25] is the best, indicating that the achieved effect is the best. Among the three deep learning methods, the evaluation index of the UWCNN [21] is the best, indicating that the achieved effect is the best. The evaluation index of the method proposed in this paper is better than those of all the comparison methods above, which further illustrates the effectiveness of the method. Table 2 shows the average UIQM, UCIQE, and NIQE values of the original images in Fig. 5 and their various enhanced versions. Our method outperforms the other methods.

4.3 Complexity analysis

The proposed method estimates each curve in a pixel-by-pixel manner. For a pixel with m × n dimensions, the complexity of the initial transmission estimation is O (m × n). According to previous descriptions of evaluation and analysis [35], the computational complexity of refined transmission is also O (m × n).

Here, we used an Intel Xeon E5-1630 (v.3.0) 3.7 GHz CPU and MATLAB R2015b for our analyses. The calculation times for NUDCP [25], the fusion-based method [19], the histogram prior [35], CycleGAN [24], UGAN [23], and UWCNN [21] were all significantly slower than that of the proposed method when processing underwater optical images at various resolutions. The comparison values are presented in Table 3.

Table 3. Comparison of the Computation Times Required by the Methods (in Seconds)

View Table | View all tables in this article

Table 3 shows the running time of different comparison methods. As the image pixels increase, the running time will also increase. On the whole, the method proposed in this paper has the fastest overall operating efficiency as the value of pixels increases.

5. Discussion

This paper proposes an underwater optical image enhancement method based on background light estimation and improved adaptive transmission fusion. It has obvious effects on dehazing, color enhancement, edge detail preservation, and light processing in extreme environments (uneven light and artificial light). First, this paper proposes BL estimation based on the gray close operation. The IFM minimization operation combines the DCP prior theory and the size of different wavelengths to obtain the BL values of the three channels, which not only accurately estimates the BL value but also avoids the interference of white objects in the scene. When the minimum reflection coefficient of white objects $\rho _{min}^{whitec}(x )\to 1$, the method will be affected, so after the gray close operation, $\hat{\rho }_{min}^c(x )\to 0$ can result, thereby eliminating the interference of these factors.

Then, we propose the IATF method, which makes TM estimation more accurate than that with other existing methods. Our estimation method is compensated for and improved based on the existing methods [18,19], and [36]. The influence of some other interference factors is eliminated to achieve more accurate TM estimation. Specifically, our method is divided into the following steps: 1) Calculate $\widetilde {{t_1}}$ by adopting the method based on DCP-W. The IFM is normalized, and the blue TM is estimated by combining the sizes of the attenuation coefficients of the three colors. Then, the TM of the green and red channels is calculated through the transmission coefficient map [42]. 2) $\widetilde {{t_2}}$ is calculated by the NULAP method proposed in this paper. The scene depth model is established, and the relative distance is estimated from the scene depth map according to the attenuation ratio of the channel [39,40]. Then, the guided filter [38] is used to further refine the scene depth map to obtain the distance between the camera and the scene point. The actual distance $\; \widetilde {{t_2}}$ can be obtained by formulas (26), (27), and (13). 3) TM fusion is performed by formulas (31) and (32) to obtain rough TMs and is further compensated for by the ARSM method to eliminate the influence of artificial light and speckles to obtain accurate TMs after fusion. Finally, the underwater optical scene is restored based on the obtained ${B^c}$ and $t_n^c(x )$.

Finally, this paper proposes a statistical colorless slant correction fusion smoothing filter method, which aims to enhance the color, details, and brightness of restored underwater optical images. First, based on the statistical colorless slant correction [27], the color-corrected image $J_{CR}^c$ is calculated. Then, inspired by the WLS filter [38] and global guided image filtering [44], we introduced an edge-preserving smoothing filter formula and a vector matrix and generated an enhanced underwater optical scene image. Qualitative and quantitative analyses showed that our proposed method based on background light estimation and improved adaptive transmission fusion has obvious advantages in underwater optical image dehazing, edge detail preservation, color enhancement, and uneven light processing (including artificial light). The result is better than that of other existing methods.

6. Conclusions

Currently, humans are engaged in an increasing number of underwater activities. Underwater optics and imaging technology play an increasingly important role, but the absorption and scattering of light in water have seriously affected the development of this field. This paper proposes a method based on background light estimation and improved adaptive transmission fusion. First, BL estimation based on the gray close operation is proposed. The proposed method avoids the interference of white objects in underwater optical scenes. Then, we propose an image restoration method, IATF. The TM values estimated by the DCP-W and NULAP methods are fused to obtain a rough TM value, and then the ARSM method is further compensated for and improved to obtain an accurate TM value. In this way, the underwater optical image is restored. Finally, we propose a statistical colorless slant correction fusion smoothing filter method to enhance color and edge details. Qualitative and quantitative analyses indicate that the proposed method shows better color enhancement and uneven underwater light processing (including artificial light) than other existing methods.