Enhancement of underwater vision by fully exploiting the polarization information from the Stokes vector

Yi Wei; Yi Wei; Pingli Han; Pingli Han; Pingli Han; Fei Liu; Fei Liu; Xiaopeng Shao; Xiaopeng Shao

doi:10.1364/OE.433072

1. Introduction

Optical underwater imaging is of great significance in both marine resource exploration and underwater target detection due to its ability to provide abundant information on the targets [1–3]. However, the presence of scattering medium such as underwater suspended particles in the optical imaging path can generate scattered light, and some of this light is scattered back along the line of sight from the medium, veiling the target and leading to a reduction in the image quality [4–6].

Several approaches have been proposed to overcome this fundamental, yet practical, problem [7–9]. Some studies have suggested that the backscattered light can be modulated and then compensated for in image postprocessing when using artificial illumination sources in highly turbid media, such as structured light [10] and time-gating techniques [11]. However, modulating the scattered light requires many frames to achieve good results. Further, such systems may be complex, expensive, and ineffective for real-time detection. Imaging techniques based on physical models can be used for real-time scene reconstruction, among which polarization imaging is of particular interest due to its simple system, low cost, and excellent performance in recovering the target information [12,13]. The existing polarimetric imaging models often assume that the target reflected light is unpolarized [14,15] or that the degree of polarization (DOP) of the target light is not zero but do not consider the angle of polarization (AOP) [4]. The purpose of these assumptions is to derive an expression of target reflected light from the degraded polarization sub-images, namely I_max with maximum visible backscattered light and target light and I_min with minimum visible backscattered light and target light [16–18]. However, in practice, the target reflected light is always partially polarized and its DOP and AOP are different from those of the backscattered light. This makes it impossible to obtain the ideal polarization sub-image by rotating the polarizer [19], and thus the existing methods can cause significant estimation errors.

Overall, the inability of the existing models to simultaneously consider DOP and AOP of light wave leads to poor imaging performance. To address this issue, in this paper, we propose an underwater polarimetric imaging model by fully exploiting the polarization information (including DOP and AOP) from the Stokes vector for the enhancement of underwater vision. The model utilizes the form of Stokes vector to describe the transmission process of light wave. The spatial variation of the polarization information of the backscattered light is also considered in this model. Further, an optimization function is established based on the independent characteristics of target and backscattered lights, which is used to remove the backscattered light and achieve clear underwater vision. The real-world experimental results show that the proposed method can remove the backscattered light and retain the intensity information of the target to the maximum extent.

2. Underwater polarimetric imaging model

Consider a point on the surface of a target in the scattering medium whose global coordinates are (X, Y, Z). The (x, y) coordinates of the image plane are set to be parallel to the global axes (X, Y). In the scattering medium (Fig. 1(a)), the total scattered light intensity I_total measured by the detector contains information from the backscattered light B_total and target reflected light T_total, all of which are partially polarized with different vibrational directions [20]. Here, the horizontal direction (X-axis direction) is considered as the reference axis. Then, Φ is the AOP of target reflected light, which is defined as the angle between the vibration directions of the polarized component of the target reflected light and the X-axis. Similarly, β and δ are the AOP of I_total and B_total. To establish the proposed model, we assume that the unpolarized and polarized components of the target reflected light (backscattered light) are T_n and T_p (B_n and B_p), respectively. The measured intensity without polarizer can be expressed as

(1)$${I_{total}} = {B_n} + {B_p} + {T_n} + {T_p} = {B_{total}} + {T_{total}}$$

The DOPs of target reflected light p_obj and the backscattered light p_scat are defined as

(2)$$\begin{aligned} {p_{obj}} &= \frac{{{T_p}}}{{{T_n} + {T_p}}}\\ {p_{scat}} &= \frac{{{B_p}}}{{{B_n} + {B_p}}} \end{aligned}$$

Fig. 1. (a) Polarimetric imaging model; (b) Relationship between the vibration directions of the target reflected light, backscattered light, and total scattered light.

Download Full Size | PDF

By rotating the polarizer installed in front of the detector, the change in the light intensity received by the detector can be expressed as follows:

(3)$$\mathop I\limits^ \wedge{=} \frac{{{B_n}}}{2} + {B_p}{\cos ^2}({\psi - \omega } )+ \frac{{{T_n}}}{2} + {T_p}{\cos ^2}(\omega )$$

where $\mathop I\limits^ \wedge $ and ω represent the intensity received by the detector and the angle of rotation of the polarizer, respectively. ψ = δ − Φ, which indicates the angle between the vibration directions of the fully polarized part of the target reflected light and backscattered light. The initial rotation angle of the polarizer is the same as the direction of vibration of T_p. Equation (2) can be used to obtain the required polarization images for image recovery. In the previous studies, these polarization images are indicated by I_max and I_min, which can be expressed as

(4)$$\begin{aligned}{I_{\max }} &= \frac{{{B_n}}}{2} + {B_p} + \frac{{{T_n}}}{2} + {T_p}\\ {I_{\min }} &= \frac{{{B_n}}}{2} + \frac{{{T_n}}}{2} \end{aligned}$$

However, in general, the ideal I_max and I_min cannot be obtained. Figure 2 shows the relationship between the intensity received by the detector and the angle of rotation of the polarizer under different values of ψ. Obviously, only when the value of ψ is zero, the I_max and I_min are obtained by rotating the polarizer. In the cases with non-zero ψ, although the curves still have peaks and valleys, the peak values are smaller than I_max and the valley values are larger than I_min. In practice, significant estimation errors can be caused by traditional polarization imaging methods as the value of ψ is generally non-zero.

Fig. 2. Relationship between the intensity received by the detector and the angle of rotation of the polarizer under different values of ψ.

Download Full Size | PDF

To avoid the errors caused by using I_max and I_min for image reconstruction, we propose a method based on Stokes vector to estimate the target information. The intensity and polarization (including DOP and AOP) information of light waves can be completely described by four physical observables representing independent features of the electric vector, commonly known as the Stokes vector [I Q U V] [21–23]. In practical application, linear polarization is easier to generate and detect than circular polarization, thus only the linear polarized component in the scattered medium is examined in this paper [4]. The DOP and AOP of linear polarized component are expressed as

(5)$$\begin{aligned} {P_{linear}} &= \frac{{\sqrt {{Q^2} + {U^2}} }}{I}\\ {A_{linear}} &= \frac{1}{2}\arctan \frac{U}{Q} \end{aligned}$$

where P_linear and A_linear are the DOP and AOP of linear polarized component, respectively. Based on Stokes vector theory, the distribution relation of polarization characteristics of I_total, T_total and B_total can be analyzed, as shown in Fig. 1(b). After obtaining the polarization images at four different angles (0°, 45°, 90°, and 135°), the Stokes vectors of the target reflected light [I_target Q_target U_target] and backscattered light [I_back Q_back U_back] in the scene are shown in Eqs. (6a) and (6b), respectively.

(6a)$$\begin{aligned} {I_{t\arg et}} &= \frac{{{T_n}}}{2} + {T_p}{\cos ^2}\phi + \frac{{{T_n}}}{2} + {T_p}{\sin ^2}\phi = {T_n} + {T_p}\\ {Q_{t\arg et}} &= \frac{{{T_n}}}{2} + {T_p}{\cos ^2}\phi - \frac{{{T_n}}}{2} - {T_p}{\sin ^2}\phi = {T_p}{\cos ^2}\phi - {T_p}{\sin ^2}\phi \\ {U_{t\arg et}} &= \frac{{{T_n}}}{2} + {T_p}{\cos ^2}({45 - \phi } )- \frac{{{T_n}}}{2} - {T_p}{\sin ^2}({45 - \phi } )= 2{T_p}\cos \phi \sin \phi \end{aligned}$$

(6b)$$\begin{aligned} {I_{back}} &= \frac{{{B_n}}}{2} + {B_p}{\cos ^2}\delta + \frac{{{B_n}}}{2} + {B_p}{\sin ^2}\delta = {B_n} + {B_p}\\ {Q_{back}} &= \frac{{{B_n}}}{2} + {B_p}{\cos ^2}\delta - \frac{{{B_n}}}{2} - {B_p}{\sin ^2}\delta = {B_p}{\cos ^2}\delta - {B_p}{\sin ^2}\delta \\ {U_{back}} &= \frac{{{B_n}}}{2} + {B_p}{\cos ^2}({45 - \delta } )- \frac{{{B_n}}}{2} - {B_p}{\sin ^2}({45 - \delta } )= 2{B_p}\cos \delta \sin \delta \end{aligned}$$

Then, the Stokes vector of the total light [I_total Q_total U_total] acquired by the detector can be expressed as

(7)$$\begin{aligned} {I_{total}} &= {T_n} + {T_P} + {B_n} + {B_p}\\ {Q_{total}} &= {T_p}\cos 2\phi + {B_p}\cos 2\delta \\ {U_{total}} &= 2{T_p}\cos \phi \sin \phi + 2{B_p}\cos \delta \sin \delta \end{aligned}$$

Equations (6) and (7) denote the proposed polarimetric imaging model. However, only three parameters (I_total, Q_total, and U_total) in this model can be calculated by the obtained polarization sub-images, and the other parameters still need to be solved to obtain the target information. The computation of these parameters is described in sections 3.2 and 3.3.

3. Underwater image reconstruction based on the Stokes vector

3.1 Image reconstruction algorithm

Equations (2) and (7) are combined to obtain a quaternion system of first-order equations with T_n, T_p, B_n, and B_p as the independent variables, which is shown as follows:

(8)$$\begin{array}{l} \left\{ {\begin{array}{c} {{I_{total}}}\\ {{Q_{total}}}\\ 0\\ 0 \end{array}\begin{array}{c} = \\ = \\ = \\ = \end{array}\begin{array}{c} {{T_n}}\\ {}\\ {{p_{obj}}{T_n}}\\ {} \end{array}\begin{array}{c} + \\ {}\\ + \\ {} \end{array}\begin{array}{c} {{T_p}}\\ {{T_p}\cos 2\phi }\\ {({p_{obj}} - 1){T_p}}\\ {} \end{array}\begin{array}{c} + \\ {}\\ {}\\ {} \end{array}\begin{array}{c} {{B_n}}\\ {}\\ {}\\ {{p_{scat}}{B_n}} \end{array}\begin{array}{c} + \\ + \\ {}\\ + \end{array}\begin{array}{c} {{B_p}}\\ {{B_p}\cos 2\delta }\\ {}\\ {({p_{scat}} - 1){B_p}} \end{array}} \right. \Rightarrow \\ \left[ {\begin{array}{c} {{I_{total}}}\\ {{Q_{total}}}\\ 0\\ 0 \end{array}} \right] = \left[ {\begin{array}{cccc} 1&1&1&1\\ 0&{\cos 2\phi }&0&{\cos 2\delta }\\ {{p_{obj}}}&{({p_{obj}} - 1)}&0&0\\ 0&0&{{p_{scat}}}&{({p_{scat}} - 1)} \end{array}} \right]\left[ {\begin{array}{c} {{T_n}}\\ {{T_p}}\\ {{B_n}}\\ {{B_p}} \end{array}} \right] = {W_T}\cdot \left[ {\begin{array}{c} {{T_n}}\\ {{T_p}}\\ {{B_n}}\\ {{B_p}} \end{array}} \right] \end{array}$$

(9)$$\begin{aligned}&{T_{total}} = {f_1}({{I_{total}},{Q_{total}},{U_{total}},\phi ,\delta ,{p_{scat}}} )\\ &= \frac{{\left[{\sin 2\phi ({{I_{total}}{p_{scat}}\cos 2\delta - {Q_{total}}} )- {I_{total}}{p_{scat}}\cos 2\phi \sin 2\delta + {U_{total}}\cos 2\phi } \right]({{I_{total}}{p_{scat}}\cos 2\delta - {Q_{total}}} )}}{{({{I_{total}}{p_{scat}}^2\cos 2\delta - {Q_{total}}{p_{scat}}} )({\cos 2\delta \sin 2\phi - \cos 2\phi \sin 2\delta } )}} \end{aligned}$$

(10)$$\left\{ \begin{array}{l} {T_{total}} = {f_1}({{I_{total}},{Q_{total}},{U_{total}},{p_{obj}},\delta ,{p_{scat}}} )\\ = \frac{{{I_{total}}{p_{scat}}\cos 2\delta - {Q_{total}}}}{{{p_{scat}}\cos 2\delta - {p_{obj}}\cos 2\phi }}\\ \phi = {{{\left( {\arcsin \frac{{{U_{total}}{p_{scat}}\cos 2\delta - {Q_{total}}{p_{scat}}\sin 2\delta }}{{\sqrt {{{({{I_{total}}{p_{scat}}{p_{obj}}\cos 2\delta - {Q_{total}}{p_{obj}}} )}^2} + {{({{U_{total}}{p_{obj}} - {I_{total}}{p_{scat}}{p_{obj}}\sin 2\delta } )}^2}} }} - K} \right)} / 2}}\\ K = \arccos \frac{{{I_{total}}{p_{scat}}{p_{obj}}\cos 2\delta - {Q_{total}}{p_{obj}}}}{{\sqrt {{{({{I_{total}}{p_{scat}}{p_{obj}}\cos 2\delta - {Q_{total}}{p_{obj}}} )}^2} + {{({{U_{total}}{p_{obj}} - {I_{total}}{p_{scat}}{p_{obj}}\sin 2\delta } )}^2}} }} \end{array} \right.$$

where W_T is the transmission matrix. By inverting Eq. (8), it is easy to derive an approximate T as a function of I_total, Q_total, Φ, δ, p_obj, and p_scat. Thus, the target light can be expressed as a function of ${T_{total}} = f({{I_{total}},{Q_{total}},\phi ,\delta ,{p_{obj}},{p_{scat}}} )$. To reduce the numbers of unknown parameters in Eq. (8), we combine the third term of Eq. (7) with Eq. (8), and then T_total is obtained, as shown in Eqs. (9) and (10).

Both Eqs. (9) and (10) are the exact results, enabling acquisition of target information, given the I_total, Q_total, U_total, Φ(or p_obj), δ, and p_scat. It may be noted that the parameters I_total, Q_total, and U_total represent the Stokes vector of total scattered light from the underwater scene, which can be calculated from four polarization images with intervals of 45° [24]. Therefore, the key to clearly reconstruct a scene is to calculate polarization information of target light and backscattered light. These parameters are generally unknown, which highlights the novelty of this paper.

3.2 Estimation of polarization characteristics of backscattered light

The previous imaging methods based on polarization information assume that the DOP and AOP of backscattered light are constant. However, in real environment, especially under active light source illumination, the DOP of backscattered light is not a constant, and its value changes gradually with the scene distribution [25]. Specifically, Fig. 3(a) displays a raw underwater image with poor vision effect and low contrast. The mean value of DOP of the region marked by the red box is chosen as the DOP of backscattered light of the whole scene, and the detection result based on the traditional polarization-based underwater imaging technology is shown in Fig. 3(b). Figure 3(c) shows an enlarged view of the selected area in Fig. 3(b). Although the traditional method provides good result in Fig. 3(c), the badminton on the right side of Fig. 3(b) is still covered by backscattered light, leading to poor visual effect. The is due to the differences in the DOPs of backscattered light in the whole scene. Thus, if the DOP marked by the red box is considered as the DOP of the backscattered light in the whole scene, the backscattered light cannot be removed completely, leading to the poor detection result.

Fig. 3. (a) Original underwater image. (b) Detection result based on the traditional polarization-based underwater imaging technology. (c) Enlarged view of (b).

Download Full Size | PDF

To accurately estimate the DOP and AOP of backscattered light, it is necessary to analyze the variation of backscattered light in the scattering medium. In the underwater imaging process, the receiving surface of the detector and the light source are on the same side relative to the scattering medium and are generally perpendicular to the direction of the incident light [6]. The polarization characteristics of the backscattered light are related to two factors. The first one is the vertical distance between the receiving point and the center of the incident beam, which is called the transverse distance. The other is the thickness of the scattering medium in different regions, which is called the longitudinal distance (also refers to the distance between the observation point and the detector).

The designed underwater experiment is shown in Fig. 4. The direction of the axis of the polarizer is parallel to the y axis. The control variable method is adopted to examine the effect of transverse distance and longitudinal distance on the backscattered light. Firstly, the light absorbing material made up of black velvet is fixed at a certain position to ensure that the thickness of the scattering medium remains stable, and the detector position is gradually moved from the point A to the point C. During the movement, the detector records the polarization information of the backscattered light at each centimeter of movement. Secondly, the detector position is fixed, and the light absorbing material is gradually moved backwards. Here also, the detector records the polarization information of the backscattered light at each centimeter of movement. The experimental results are shown in Fig. 5.

Fig. 4. Schematic of polarization-based underwater imaging to analyze the changes in the polarization state of light under water.

Download Full Size | PDF

Fig. 5. Experimental results. (a) The red line indicates the variation in the DOP with longitudinal distance, the black line indicates the variation in the DOP with transverse distance. (b) The red line indicates the variation in the AOP with longitudinal distance, the black line indicates the variation in the AOP with transverse distance. The volume of skim milk added in the experimental environment of (a) and (b) was 40 ml. (c) The red line indicates the variation in the DOP with longitudinal distance, the black line indicates the variation in the DOP with transverse distance. (d) The red line indicates the variation in the AOP with longitudinal distance, the black line indicates the variation in the AOP with transverse distance. The volume of skim milk added in the experimental environment of (c) and (d) was 60 ml.

Download Full Size | PDF

It is clear that the effect of longitudinal distance on the polarization characteristics of backscattered light vary is much less than that of the transverse distance. The observed phenomenon, i.e., the backscattered light at the same position does not change when the detection range is more than 1 m, has been described in detail in Ref. [4]. In practical applications, the distance from the target to the detector is generally more than 1 m, so the change in the polarization characteristics of backscattered light in the longitudinal direction is ignored in this paper. On the contrary, there is a significant spatial variation in the AOP and DOP of the backscattered light with the transverse distance, and it is reasonable to expect that the measured AOP and DOP of backscattered light are non-uniform. Here, the extrapolation method is employed to deduce the spatial distribution values of AOP and DOP [25]. In this method, it is assumed that DOP and AOP of backscattered light in the whole scene can be represented as a function of the pixel coordinates in the image. Firstly, the pixels corresponding to the background (without target) are used to estimate the local ${\mathop p\limits^ \wedge _{scat}}({x,y} )$ and $\mathop \delta \limits^ \wedge ({x,y} )$. The spatial distribution of ${p_{scat}}({x,y} )$ and $\delta ({x,y} )$ in the whole scene can be deduced by polynomial fitting function of the measured distribution of the backscattered light ${\mathop p\limits^ \wedge _{scat}}({x,y} )$ and $\mathop \delta \limits^ \wedge ({x,y} )$ in the known background region. Then, the polynomial functions are used for the fitting function of ${\mathop p\limits^ \wedge _{scat}}({x,y} )$ and $\mathop \delta \limits^ \wedge ({x,y} )$

(11)$$\begin{aligned} p_{scat}^{{n_1}}({x,y} )&= \sum\limits_{i,j = 0}^{{n_1}} {{p_{ij}}{x^i}{y^j}} \\ \delta _{}^{{n_2}}({x,y} )&= \sum\limits_{i,j = 0}^{{n_2}} {{q_{ij}}{x^i}{y^j}} \end{aligned}$$

The polynomial function in Eq. (11) is composed of multiple cross terms. (x,y) represents the coordinates of the pixels in the image, which corresponds to the possible relation with ${\mathop p\limits^ \wedge _{scat}}({x,y} )$ and $\mathop \delta \limits^ \wedge ({x,y} )$. n₁ and n₂ represent the order of polynomial function of p_scat and δ, respectively. p_ij and q_ij are the coefficients of polynomial function, which are obtained using the least squares method by minimizing the difference ${\left\|{{{\mathop p\limits^ \wedge }_{scat}}({x,y} )- p_{scat}^{{n_1}}({x,y} )} \right\|^2}$ and ${\left\|{\mathop \delta \limits^ \wedge ({x,y} )- \delta_{}^{{n_2}}({x,y} )} \right\|^2}$ in the background region.

3.3 Estimation of polarization characteristics of the target

In this section, a de-correlation method is proposed to estimate the polarization characteristics of target reflected light. Taking Eq. (10) as an example, we aim to estimate the DOP of the target. The de-correlation method is based on the observation that using a wrong value for p_obj increases the crosstalk between the estimated scattered light B_total and the target reflected light T_total [4]. To quantify the crosstalk, mutual information (MI) is used, which is a quantity that denotes the mutual statistical dependency of the two random variables B_total and T_total. A high value indicates some statistical dependency between the variables. Therefore, the problem of determining target DOP can be transformed into the following optimization problem:

(12)$$p_{obj}(x,y) = \mathop {\arg \min }\limits_{p_{obj}\in [0,1]} \left\{ {MI\left[ {B_{total}(p_{obj}),T_{total}(p_{obj})} \right]} \right\}$$

The MI of B_total and T_total is given by

(13)$$\begin{aligned} & MI\left[{{B_{total}}\left({{p_{obj}}} \right),{T_{total}}\left({{p_{obj}}} \right)} \right]\\ &= \sum\limits_{b \in {B_{total}}} {\sum\limits_{t \in {T_{total}}} {prob({b,t} )\log \left[ {\frac{{prob({b,t} )}}{{prob(b )prob(t )}}} \right]} } \end{aligned}$$

where prob(b,t) represents the joint probability distribution function of pixels in B_total and T_total. prob(b) and prob(t) are the marginal distribution functions of B_total and T_total, respectively.

On the other hand, if we need to calculate the information of the target through Eq. (9), Eq. (13) can also be used to calculate the AOP of target reflected light by replacing the p_obj in Eq. (10) with $\phi$.

4. Real-world experiment and results

A schematic of underwater polarization imaging experiment is shown in Fig. 6. The light from a LED (THORLABS M660L4, the model of LED power is LEDD1B T-Cube) is converted into polarized light through a polarizer (THORLABS LPVISE200-A) before entering the medium. The volume of the glass tank is 50 × 70 × 40 cm³. To simulate the scattering environment, 120 l tap water is mixed with 60 ml skimmed milk in the glass tank [6,8]. The detector (CANON EOS 77D) and the light source are placed on the same side of the glass tank. A polarizer is placed in front of the detector, which is rotated to obtain polarization images at angles of 0°, 45°, 90°, and 135°. To simplify the setup, we can also use a polarization camera (LUCID TRI050S-P) to capture these images directly.

Fig. 6. Schematic of the experimental setup.

Download Full Size | PDF

After estimating the polarization information of the target and the backscattered light, the target information can be obtained by Eq. (9) or Eq. (10). The result is shown in Fig. 7. A metal vernier caliper is considered as the raw object, which is covered with a rough waterproof paper in certain regions, and some letters are written on the waterproof paper. The traditional method exhibits a good target reconstruction performance on rough surface because the target has a very small DOP, which is consistent with its assumption. However, it is not effective for the metal target because such a target has a high polarization-maintaining property. In addition, the backscattered light in the whole scene cannot be removed due to the assumption that the backscattered light has uniform polarization characteristics, which is clearly illustrated by the left area in Fig. 7(b). Considering the spatial variation characteristics of polarization information of backscattered light, the proposed method effectively recovers the target information and completely removes the backscattered light.

Fig. 7. (a) Raw intensity image. (b) Detection result based on the traditional polarization-based underwater imaging technology. (c) Detection result based on the proposed model.

Download Full Size | PDF

To verify the universality of the proposed method, Fig. 8 shows the result of applying it to images taken during three different experiments in the underwater environment. The left, center, and right columns shows the raw intensity image I_total, the target information T_total, and the estimated backscattered light B_total, respectively. The experimental results suggest that the quality of the underwater image can be efficiently enhanced by our method, which is particularly effective in the cases where both the DOP and AOP of target light contribute to the polarization, such as experiment 1.

Fig. 8. Results of three different experiments. (a) Raw intensity images I_total. (b) Reconstructed target information T_total. (c) Estimated backscattered light B_total.

Download Full Size | PDF

In principle, the intensity of the backscattered light should be the same in the regions of the target at the same distance from the detector [5], but the intensity of the target regions is significantly different in the backscattered light images in experiment 1. This is because our model still assumes that the target's DOP is a constant, resulting in the loss of a small part of the target's energy. However, compared with other methods, our method greatly reduces the loss of target information. If the material properties of the target in the detector's field of view are the same, this means that the polarization of the target's reflected light is a constant. In this case, the proposed method can accurately estimate the target information without any loss. As shown in experiments 2 and 3, the intensity of the regions with identical target-to-detector distance is the same in the image of backscattered light, which is consistent with the backscattered light image in the ideal case.

5. Fidelity of reconstruction algorithm based on the fully exploited Stokes vector

In practical applications, there exist a variety of materials with different polarization properties in the scattering medium that need to be detected. The value of DOP of the reflected light from the same target may vary from 0 to 1. Similarly, the value of the AOP may vary from 0 to 2π. Although the existing polarization-based methods are effective in removing the backscattered light, since the assumptions in these methods are contrary to the actual situation, they lose some part of the target energy. Consequently, it is necessary to analyze and compare the fidelity of the existing methods and the proposed method. To this end, we simulated the imaging process of the target with all possible DOPs and AOPs in the scattering medium, and the processing results of different methods are shown in Fig. 9.

Fig. 9. Simulation of a scene in a scattering medium. The target is a linearly varying distance map with values ranging between [0.04, 1] m. (a) Sensed scene with backscattered light; (b) Simulated target information; (c) Backscattered light component. The Stokes vector images of the total scattered light represent (a) I, (d) Q, and (c) U.

Download Full Size | PDF

As shown in Fig. 9, the simulated target has 5×5 regions with different polarization characteristics. The intensity of the target reflected light ranges from 0 (blue) to 1 (red). For simplicity, it is assumed that the value of target DOP is distributed between 0 and 1, and the value of target AOP is distributed between 0 and 2π. For the backscattered light, since the distance between the target and the detector may have multiple values, different detection distances are set for each intensity region in the target image, and the intensity of backscattered light generated at such a distance is shown in Fig. 9(c). In this paper, the DOP and AOP of the backscattered light are set to 0.6 and 0, respectively. After defining these initial parameters, the polarization images needed for image reconstruction are determined according to the existing polarization-based descattering methods [4,15,26], and then the solutions of these methods are used to calculate the clear scene. Finally, the mean squared error (MSE) in Eq. (14) is used to measure the fidelity of these methods [18]. Smaller MES implies that the recovered target information is more similar to the real target information, indicating better image quality.

(14)$$MSE = \sum\limits_{i = 1}^m {\sum\limits_{j = 1}^n {{{{{\left({T({i,j} )- T^{\prime}({i,j} )} \right)}^2}} / {\left({m\ast n} \right)}}} }$$

where T’ represents the clear image of the target obtained by the above methods; (i,j) indicates the pixel coordinates in the image; m and n represent the number of rows and columns in the image, respectively. The MSE of images reconstructed by different methods is shown in Table 1.

Table 1. MSE of images reconstructed by different methods

View Table

In the first method (Table 1), the polarization images with the maximum and minimum light intensity expressed in Eq. (15) are obtained by rotating the polarizer mounted in front of the camera, and then the two polarization images are processed to obtain the clear target image.

(15)$$\begin{aligned} {I_{\max }} &= {{T(1 - {P_{obj}})} / 2} + T{P_{obj}}{\cos ^2}(\phi - 0)\\ &\quad + {{B(1 - {P_{scat}})} / 2} + B{P_{scat}}{\cos ^2}(\delta - 0)\\ {I_{\min }} &= {{T(1 - {P_{obj}})} / 2} + T{P_{obj}}{\cos ^2}(\phi - 90)\\ &\quad + {{B(1 - {P_{scat}})} / 2} + B{P_{scat}}{\cos ^2}(\delta - 90) \end{aligned}$$

In this method, it is assumed that the AOP of the target reflected light is the same as that of the backscattered light, and in general, the images obtained by this method only contain the information of maximum and minimum backscattered light. The target reflected light is not in the same state as that assumed by the model. This method can be divided into two types according to the processing technique: assuming that the target has no DOP and calculating the target DOP through MI [4,27].

In the second method, the 0°, 45°, 90°, and 135° polarization images expressed in Eq. (16) are obtained, and the polarization images with the maximum and minimum light intensity expressed in Eq. (17) are calculated. Then, these images are processed according to the first method.

(16)$$\begin{aligned}I^{\prime}(0 )&= {{T(1 - {P_{obj}})} / 2} + T{P_{obj}}{\cos ^2}(\phi - 0)\\ &\quad + {{B(1 - {P_{scat}})} / 2} + B{P_{scat}}{\cos ^2}(\delta - 0)\\ I^{\prime}({45} )&= {{T(1 - {P_{obj}})} / 2} + T{P_{obj}}{\cos ^2}(\phi - 45)\\ &\quad + {{B(1 - {P_{scat}})} / 2} + B{P_{scat}}{\cos ^2}(\delta - 45)\\ I^{\prime}({90} )&= {{T(1 - {P_{obj}})} / 2} + T{P_{obj}}{\cos ^2}(\phi - 90)\\ &\quad + {{B(1 - {P_{scat}})} / 2} + B{P_{scat}}{\cos ^2}(\delta - 90)\\ I^{\prime}({135} )&= {{T(1 - {P_{obj}})} / 2} + T{P_{obj}}{\cos ^2}(\phi - 135)\\ &\quad + {{B(1 - {P_{scat}})} / 2} + B{P_{scat}}{\cos ^2}(\delta - 135) \end{aligned}$$

(17)$$\begin{aligned}{I_{\max }} &= I \times \left( {1 + \frac{{\sqrt {{Q^2} + {U^2}} }}{I}} \right)/2\\ {I_{\min }} &= I \times \left( {1 - \frac{{\sqrt {{Q^2} + {U^2}} }}{I}} \right)/2 \end{aligned}$$

where

(18)$$\begin{aligned}I &= I^{\prime}(0 )+ I^{\prime}({90} )= T + B\\ Q &= I^{\prime}(0 )- I^{\prime}({90} )= T{P_{obj}}\cos 2\phi + B{P_{scat}}\cos 2\delta \\ U &= I^{\prime}({45} )- I^{\prime}({135} )= 2T{P_{obj}}\cos \phi \sin \phi + 2B{P_{scat}}\cos \delta \sin \delta \end{aligned}$$

The images with the maximum and minimum light intensity obtained by this method contain the maximum visible I_total and minimum visible I_total, but the target reflected light and the backscattered light are not in the state of maximum and minimum light intensity because they have different AOPs, as shown in Fig. 1(b) [15,28].

In the third method, two polarized images are obtained, whose polarization states are orthogonal to each other, and they are at an angle of 45° with the vibration direction of the backscattered light, as shown in Eq. (19).

(19)$$\begin{aligned}{I_1} &= {{T(1 - {P_{obj}})} / 2} + T{P_{obj}}{\cos ^2}(\phi - 45)\\ &\quad + {{B(1 - {P_{scat}})} / 2} + B{P_{scat}}{\cos ^2}(\delta - 45)\\ {I_2} &= {{T(1 - {P_{obj}})} / 2} + T{P_{obj}}{\cos ^2}(\phi + 45)\\ &\quad + {{B(1 - {P_{scat}})} / 2} + B{P_{scat}}{\cos ^2}(\delta + 45) \end{aligned}$$

The intensity of backscattered light in the two images is exactly the same, so part of the target energy can be obtained by subtracting the two images [26]. The MSE shows that the proposed method has the best performance in recovering the true information of the target as compared to the other methods.

6. Summary

In this study, an underwater polarimetric imaging model was established by fully exploiting the Stokes vector, i.e., by taking both DOP and AOP of light wave into consideration. The optical transmission process was modeled with the polarization information derived from the Stokes vector, which facilitated an effective reconstruction of the target information. Besides removing the backscattered light, the proposed model specifically uses Stokes vectors to recover target information when both DOP and AOP of the target light contribute to the polarization of the total light. The MSE between the reconstructed image and the original clear image was measured to verify the accuracy and superiority of the proposed method over the existing methods.

Funding

Science and Technology on Electro-Optical Information Security Control Laboratory (61421070203); National Natural Science Foundation of China (62005203, 62075175).

Acknowledgement

The authors thank X. Li, K. Yang, Y. Cai, S. Zhang, S. Sun and F. Chen for stimulating discussions regarding algorithm implementation and fruitful discussions on the imaging model

Disclosures

The authors declare no conflicts of interest.

Data availability

Data underlying the results presented in this paper are not publicly available at this time but may be obtained from the authors upon reasonable request.

References

1. D. M. Kocak, F. R. Dalgleish, F. M. Caimi, and Y. Y. Schechner, “A focus on recent developments and trends in underwater imaging,” Mar. Technol. Soc. J. 42(1), 52–67 (2008). [CrossRef]

2. A. I. Joseph, R. H. Michael, M. O. Gabrielle, A. S. Eric, and G. F. James, “Optical coherence microscopy in scattering media,” Opt. Lett. 19(8), 590–592 (1994). [CrossRef]

3. M. Sheinin and Y. Y. Schechner, “The Next Best Underwater View,” CVPR, Washington, DC, USA (2016).

4. T. Treibitz and Y. Y. Schechner, “Active Polarization Descattering,” IEEE Trans. Pattern Anal. Mach. Intell. 31(3), 385–399 (2009). [CrossRef]

5. Y. Y. Schechner, S. G. Narasimhan, and S. K. Nayar, “Polarization-Based Vision Through Haze,” Appl. Opt. 42(3), 511–525 (2003). [CrossRef]

6. F. Liu, P. Han, Y. Wei, K. Yang, S. Huang, X. Li, G. Zhang, L. Bai, and X. Shao, “Deeply seeing through highly turbid water by active polarization imaging,” Opt. Lett. 43(20), 4903–4906 (2018). [CrossRef]

7. K. Nakai, Y. Hoshi, and A. Taguchi, “Color image contrast enhacement method based on differential intensity/saturation gray-levels histograms,” 2013 International Symposium on Intelligent Signal Processing and Communication Systems, Naha, 2013, 445–449.

8. D. M. Kocak and F. M. Caimi, “The Current Art of Underwater Imaging—With a Glimpse of the Past,” Mar. Technol. Soc. J. 39(3), 5–26 (2005). [CrossRef]

9. K. He, J. Sun, and X. Tang, “Single image haze removal using dark channel prior,” IEEE Trans. Pattern Anal. Mach. Intell. 33(12), 2341–2353 (2010). [CrossRef]

10. S. G. Narasimhan, S. K. Nayar, B. Sun, and S. J. Koppal, “Structured Light in Scattering Media,” Proc. 10th IEEE Int’l Conf. Computer Vision, 1, 420–427 (2005).

11. G. R. Fournier, D. Bonnier, L. J. Forand, and P. W. Pace, “RangeGated Underwater Laser Imaging System,” Opt. Eng. 32(9), 2185–2190 (1993). [CrossRef]

12. N. K. Soni, R. Vinu, and R. K. Singh, “Polarization modulation for imaging behind the scattering medium,” Opt. Lett. 41(5), 906–909 (2016). [CrossRef]

13. P. Han, F. Liu, K. Yang, J. Ma, J. Li, and X. Shao, “Active underwater descattering and image recovery,” Appl. Opt. 56(23), 6631–6638 (2017). [CrossRef]

14. Y. Y. Schechner and N. Karpel, “Clear underwater vision,” CVPR, Washington, DC, USA, I (2004).

15. X. Li, F. Liu, P. Han, Y. Wei, D. Zhao, X. Shao, and X. Zhao, “Research on polarization dehazing through the coaxial and multi-aperture polarimetric camera,” OSA Continuum 2(8), 2369 (2019). [CrossRef]

16. X. Li, H. Hu, L. Zhao, H. Wang, Y. Yu, L. Wu, and T. Liu, “Polarimetric image recovery method combining histogram stretching for underwater imaging,” Sci Rep 8(1), 12430 (2018). [CrossRef]

17. J. Park, J.-H. Park, H. Yu, and Y. Park, “Focusing through turbid media by polarization modulation,” Opt. Lett. 40(8), 1667–1670 (2015). [CrossRef]

18. B. Huang, T. Liu, H. Hu, J. Han, and M. Yu, “Underwater image recovery considering polarization effects of objects,” Opt. Express 24(9), 9826–9838 (2016). [CrossRef]

19. N. Agarwal, J. Yoon, E. Garcia-Caurel, T. Novikova, J. C. Vanel, A. Pierangelo, A. Bykov, A. Popov, I. Meglinski, and R. Ossikovski, “Spatial evolution of depolarization in homogeneous turbid media within the differential Mueller matrix formalism,” Opt. Lett. 40(23), 5634–5637 (2015). [CrossRef]

20. J. Cariou, B. L. Jeune, J. Lotrian, and Y. Guer, “Polarization effects of seawater and underwater targets,” Appl. Opt. 29(11), 1689–1695 (1990). [CrossRef]

21. F. Gori, “Measuring Stokes parameters by means of a polarization grating,” Opt. Lett. 24(9), 584–590 (1999). [CrossRef]

22. J. E. Solomon, “Polarization imaging,” Appl. Opt. 20(9), 1537–1544 (1981). [CrossRef]

23. J. Liang, L. Ren, E. Qu, B. Hu, and Y. Wang, “Method for enhancing visibility of hazy images based on polarimetric imaging,” Photonics Res. 2(1), 38–44 (2014). [CrossRef]

24. S. Tomczyk, R. Casini, A. G. Wijn, and P. G. Nelson, “Wavelength-diverse polarization modulators for Stokes polarimetry,” Appl. Opt. 49(18), 3580–3586 (2010). [CrossRef]

25. H. Hu, L. Zhao, X. Li, H. Wang, and T. G. Liu, “Underwater image recovery under the nonuniform optical field based on polarimetric imaging,” IEEE Photonics J. 10(1), 1–9 (2018). [CrossRef]

26. J. S. Tyo, M. P. Rowe, E. N. J. Pugh, and N. Engheta, “Target detection in optically scattering media by polarization-difference imaging,” Appl. Opt. 35(11), 1855–1870 (1996). [CrossRef]

27. J. Liang, W. Zhang, L. Ren, H. Ju, and E. Qu, “Polarimetric dehazing method for visibility improvement based on visible and infrared image fusion,” Appl. Opt. 55, 8221–8226 (2016). [CrossRef]

28. X Li, F Liu, P Han, S Zhang, and X Shao, “Near-infrared monocular 3D computational polarization imaging of surfaces exhibiting nonuniform reflectance,” Opt. Express 29(10), 15616–15630 (2021). [CrossRef]

	Methods	MSE
First method	Assume P_obj= 0	0.3436
First method	Calculate P_obj through MI	0.4268
Second method	Assume P_obj= 0	0.3720
Second method	Calculate P_obj through MI	0.3991
Third method		0.2526
Our method		0.1518

Enhancement of underwater vision by fully exploiting the polarization information from the Stokes vector

Abstract

1. Introduction

2. Underwater polarimetric imaging model

3. Underwater image reconstruction based on the Stokes vector

3.1 Image reconstruction algorithm

3.2 Estimation of polarization characteristics of backscattered light

3.3 Estimation of polarization characteristics of the target

4. Real-world experiment and results

5. Fidelity of reconstruction algorithm based on the fully exploited Stokes vector

6. Summary

Funding

Acknowledgement

Disclosures

Data availability

References

Data availability

Cited By

Figures (9)

Tables (1)

Equations (20)

Optics Express