Expand this Topic clickable element to expand a topic
Skip to content
Optica Publishing Group

Single image dehazing algorithm based on optical diffraction deep neural networks

Open Access Open Access

Abstract

Single image dehazing is a challenging task because of the hue and brightness distortion problems due to the atmospheric scattering. These problems limit the perceptual fidelity, as well as information integrity, of a given image. In this paper, we propose an image dehazing method based on the optical neural networks dehazing by simulating optical diffraction. The algorithm is trained from a large number of hazy images and their corresponding clean images. The experimental results demonstrate that the proposed method has reached an advanced level in both PSNR and SSIM dehazing performance indicators, and the amount of calculation is less than most artificial neural networks.

© 2022 Optica Publishing Group under the terms of the Optica Open Access Publishing Agreement

1. Introduction

Haze is usually an atmospheric phenomenon that is caused by coagulation nucleus condensed of water vapor and tiny particles in the air when water vapor is sufficient. When the haze is too thick and the coagulation nucleus is too dense, airglow is seriously affected by scattering and refraction which often leads to the phenomenon of reduced contrast, blurred details, and low brightness in the collected images, making the image quality decline. In a natural environment, outdoor images are more or less affected by haze. Generally speaking, the impact mainly includes two aspects that one is affecting the transmission of image information, and the other is affecting the accuracy and efficiency of image processing and subsequent tasks. This greatly threatens the application of image processing technology in intelligent transportation, remote sensing, and autonomous driving, etc. of advanced computer vision tasks. Because of the important application value of image dehazing in related fields, image restoration in hazy weather has always been a key issue of concern to scholars.

Since most traditional dehazing methods do not pay enough attention to the physical nature of image degradation [13], images are severely distorted when the haze is unevenly distributed or the lighting environment is unsatisfactory. With the rapid development of deep learning technology, image dehazing methods based on deep learning have gradually become a research hot spot [47]. This type of method can be divided into two main types: one is to use the neural networks to estimate the parameters in the atmospheric degradation model and output the intermediate transmission map to achieve image dehazing; the other is to directly process the input hazy image and then get the image which is dehazed, namely End2End Networks of deep learning. Most of the dehazing algorithms currently prefer to use the End2End Networks. Cai et al. first proposed the End2End image dehazing system, Dehaze-Net, which used the convolutional neural networks to learn the characteristics of haze, and estimated the mapping relationship between the hazy image and the intermediate transmission image to achieve the image dehazing effect [4]. He et al. noticed that the previous dehazing algorithms did not pay enough attention to the atmospheric light intensity in the atmospheric scattering model, so they proposed DCPDN and used it to estimate the atmospheric light intensity [5]. Ren et al. transformed the input image multiple times and merged the weight matrix generated by each transformation to effectively reduce the halo effect during the image restoration processing [6]. Chen et al. used the GAN networks to solve the problem of mesh artifacts without relying on a priori knowledge, which greatly improved the SSIM index [7].

However, when the task is more complex, the training time of the neural networks is longer, the power consumption is higher, and the hardware requirements are higher, which greatly increases the application cost. As the size of electronic transistors gradually approaches their physical limits, Moore's Law is difficult to sustain [8,9]. In terms of physical properties, optical computing naturally has the advantages of parallelism, high-speed interconnection, low or even no energy consumption, which makes it possible for photons to replace electrons for computing and information transmission in the future [1017]. Lin et al. verified the feasibility of optical diffraction neural networks [14], and Hughes et al. proved the mapping relationship between the optical diffraction neural networks and the standard neural networks [18]. The above researches provide the possibility to combine the optical diffraction neural networks with the application of typical vision processing problems. Based on the above ideas, this paper combines the optical diffraction neural networks with image dehazing, and carries out the research on the image dehazing method based on optical diffraction neural networks. The research is all realized by optical method, making full use of the parallel characteristics of the beam, which greatly improves the calculation speed of the deep neural network. And it is verified by simulation that the algorithm only needs less calculation than the neural networks of the electronic platform to achieve a higher dehazing effect, and it has broad prospects in the field of image dehazing.

2. Proposed method

This part introduces the principle of image degradation based on the atmospheric scattering model, the principle and basic structure of the optical diffraction neural networks.

2.1 The atmospheric scattering model

Due to the influence of dust droplets, liquid droplets, and colloids scattered, the air will become an uneven medium. As shown in Fig. 1, when the beam passes through, part of the light will deviate from the original propagation direction and diverge to the surroundings to reduce the image quality. With enhancing the non-uniformity of the medium, the degree of the light scattering is increasing. According to the relationship between the size of suspended particles in the air and the wavelength of the incident light, scattering can be divided into Rayleigh scattering (particle diameter is much smaller than the wavelength) and Mie scattering (particle diameter is equivalent to the wavelength), where Mie scattering is the main reason of the reduction in imaging quality in hazy weather [19,20].

 figure: Fig. 1.

Fig. 1. Physical model of atmospheric scattering.

Download Full Size | PDF

According to the atmospheric scattering model, on a hazy day, the image received by the image receiving device is a combination of the attenuation of the reflected light from the target scene after passing through the suspended particles and the superposition of the suspended particles scattering, which can be mathematically expressed by the following equation:

$$I(x,y)\textrm{ = }J(x,y)t(x,y) + A[1 - t(x,y)]$$

In Eq. (1), $I(x,y)$ is the artificially generated hazy image, $J(x,y)$ is the original input image, A is the global atmospheric light intensity, $t(x,y)$ is the medium transmission map, $(x,y)$ are the pixel coordinates. When the atmosphere is homogeneous, $t(x,y)$ can be expressed as:

$$t(x,y)\textrm{ = }\exp [ - \beta d(x,y)]$$
where $\beta$ is the attenuation coefficient of the atmosphere, $d(x,y)$ is the distance between the imaging system and the object. Currently, most researches about image dehazing algorithms are based on the atmospheric scattering model. Therefore, as shown in Fig. 2, suspended particles during the scattering process will cause the original image color transmission distortion and visual effect attenuation.

 figure: Fig. 2.

Fig. 2. Effects of Atmospheric Scattering on Images. (a)Hazy-free image (b)Fogged image in A = 0.7, $\beta $=0.04 (c) Fogged image in A = 0.7, $\beta $=0.08 (d) Fogged image in A = 0.5, $\beta $=0.08.

Download Full Size | PDF

2.2 Optical diffraction neural networks

The optical diffraction neural networks is a new optical neural networks architecture proposed in recent years, and it has been experimentally proved that any function of statistical learning can be completed by optical neural networks [14]. The networks architecture combines several diffraction gratings as neurons into a diffraction surface as a layer of neural networks, and the multi-layer neural networks is connected to each other through coherent light for information transmission to complete specific tasks. In Fig. 3 the optical diffraction neural networks is similar to the fully-connected neural networks. Their neurons are directly connected to each other for forwarding propagation and then the networks parameters are trained through the loss function. The biggest difference between the two types of neural networks is that the weights of the connections emitted by the neurons of the optical diffraction neural networks are shared while the connections between the neurons of the fully-connected networks are independent of each other.

 figure: Fig. 3.

Fig. 3. Networks structure model. (a) Optical diffraction neural networks (b) Fully-connected networks.

Download Full Size | PDF

Each neuron on the diffraction layer of the optical diffraction neural networks is connected with the neuron of the next layer according to the Rayleigh-Sommerfeld diffraction equation [21]:

$$W_i^l(x,y,z)\textrm{ = }\frac{{z - {z_i}}}{{{r^2}}}(\frac{1}{{2\pi r}} + \frac{1}{{j\lambda }})\exp (\frac{{j2\pi r}}{\lambda })$$
where $l$ refers to the $l\textrm{ - }th$ layer network, $i$ is the $i\textrm{ - }th$ neuron in the $l\textrm{ - }th$ layer at $({x_i},{y_i},{z_i})$, $j\textrm{ = }\sqrt { - 1} $, $\lambda $ is the wavelength of the input light, and $r$ is the Euclidean distance, $r\textrm{ = }\sqrt {{{(x - {x_i})}^2} + {{(y - {y_i})}^2} + {{(z - {z_i})}^2}}$. According to Huygens principle and angular spectrum propagation theory, the diffraction wave between layers can be regarded as the superposition of the secondary waves emitted by each grating, so the weight of neurons can be adjusted by adjusting the grating parameters [22]. The propagation equation of each neuron is as follows:
$${t_i}({x_i},{y_i},{z_i})\textrm{ = }\alpha _i^l({x_i},{y_i},{z_i})\exp [j\phi _i^l({x_i},{y_i},{z_i})]$$

The transfer coefficient is composed of amplitude and phase, which is a complex number. $\alpha $ and $\phi $ respectively represent the modulation of the amplitude and the phase of the neuron. One of them or both can be modulated at the same time. Therefore, the output of the $i\textrm{ - }th$ neuron on the $l\textrm{ - }th$ layer can be expressed as:

$$n_i^l(x,y,z)\textrm{ = }W_i^l(x,y,z) \cdot t_i^l({x_i},{y_i},{z_i}) \cdot \sum\nolimits_k {n_k^{l - 1}} ({x_i},{y_i},{z_i})\textrm{ = }W_i^l(x,y,z) \cdot |A|\cdot {e^{j\triangle \vartheta }}$$

3. Networks proposed design

In this section, the details of optical diffraction neural networks are introduced. Then the forward and backpropagation are analyzed. In addition, the networks parameters are given in detail.

3.1 Forward propagation design

To find a more concise and effective networks structure, this article assumes that the amplitude of the neuron $\alpha _i^l({x_i},{y_i},{z_i})$ is a constant, ideally 1, only modulating the phase and using the Sigmoid function to limit the phase value $\phi _i^l({x_i},{y_i},{z_i})$ in $0\textrm{ - }2\pi$. In order to improve the generalization and reasoning ability of the diffraction neural networks [23], Leaky-ReLU [24] is selected as the activation function of the diffraction neural networks. As shown in Fig. 4, it is a piecewise linear function that cuts the negative part and the positive part. When the input signal $x \ge 0$ the output is equal to the input signal, and when the input signal $x < 0$ Leaky-ReLU assigns a nonzero slope to all negative values.

 figure: Fig. 4.

Fig. 4. Mathematical models of Leaky-ReLU function.

Download Full Size | PDF

Leaky-ReLU can not only avoid the problem of the disappearance of the neural networks gradient but also prevent the death of neurons (‘dying ReLU’ problem) [25]. In this study, $\alpha $ in the Leaky-ReLU function is selected as 0.2. The equation is:

$$f(x)\textrm{ = }\left\{ \begin{array}{ll} x&{x \ge 0}\\{0.2 \cdot x}&{x < 0}\end{array} \right.$$

Therefore, the forward propagation equation of the optical diffraction neural networks can be expressed as:

$$\left\{ \begin{array}{l} n_{i,p}^l\textrm{ = }w_{i,p}^l \cdot t_i^l \cdot f(m_i^l)\\ m_i^l\textrm{ = }\sum\nolimits_k {n_{k,i}^{l - 1}} \\ t_i^l\textrm{ = }\exp (j \cdot S(\phi_i^l({x_i},{y_i},{z_i})))\\ S(x)\textrm{ = }\frac{{2\pi }}{{1 + {e^{ - x}}}} \end{array} \right.$$
where $n_{i,p}^l$ represents the connection between the $i\textrm{ - }th$ neuron on the $l\textrm{ - }th$ layer and the $p\textrm{ - }th$ neuron in the next layer. As shown in Fig. 5, the data is transmitted in the optical diffraction neural networks in the form of complex numbers. Until it is transmitted to the output layer, the data is freely transmitted to the photodetector according to the Rayleigh-Sommerfeld diffraction equation. Then the photodetector obtains the output result by detecting the intensity of the light field, and the expression is:
$$s_i^{M + 1}\textrm{ = }|m_i^{M + 1}{|^{^2}}$$

 figure: Fig. 5.

Fig. 5. Algorithmic steps for optical diffraction neural networks.

Download Full Size | PDF

The output result is compared with the training target, and the error generated by the comparison is back-propagated to iteratively update the weight value of the neuron.

3.2 Back propagation design

In order to train the designed diffraction neural networks, this article uses error backpropagation algorithm and the Adam optimization algorithm [26]. A loss function is defined to evaluate the performance of the output of the diffraction neural networks relative to the required target, and the algorithm iteratively optimizes the parameters of the diffraction neural networks $\phi _i^l$ to minimize the loss function. Aiming at the proposed networks structure, this paper selects mean absolute error (MAE) as the loss function to calculate the output light field intensity $s_i^{M + 1}$ and the target is the haze-free images corresponding to the input hazy images $g_k^{M + 1}$, the equation is:

$$MAE(\phi _i^l)\textrm{ = }\frac{1}{k}\sum\nolimits_k {|s_k^{M + 1} - g_k^{M + 1}|} $$
where $k$ represents the number of measurement points of the output light field.

3.3 Networks parameters

In this article, the input light wavelength $\lambda $ is selected as 750 µm, and the side length of each layer of networks is 9 cm. It is composed of 300*300 neurons of the same size. Excluding the input layer and output layer, the neural networks contains a total of 5 diffraction layers for training. The interval between layers is 4 mm. The number of measurement points of the output light field of the output layer is set to be the same as the number of neurons in each layer $k\textrm{ = }90000$. The learning rate of the Adam optimization algorithm is set to 0.00001. The networks is trained for 100 rounds, and the training batch size is 2.

4. Experiment and result analysis

In this section, the training dataset is described and optical diffraction neural networks variations are investigated. The proposed method is compared with the state-of-the-art methods for the synthetic and natural images. In addition, the experiment results and the qualitative and quantitative analyses are presented.

4.1 Training dataset

The training of the image dehazing networks needs to take the hazy images as the samples and use the images that are the same as the samples (such as position, brightness, etc.) except for haze as the label. Such datasets are difficult to obtain through real scenes, so many studies [47] synthesize data by artificially adding haze to real images. This article uses the IMAGENET dataset [27] as the original image. The IMAGENET dataset covers most of the images in life, which is a total of 1000 categories and more than 10 million pieces of data, all from the real world. According to Eq. (1), the original images are input into the atmospheric scattering model, as shown in Fig. 6. This article uses the atmospheric scattering model with the atmospheric light intensity of 0.7 and the attenuation coefficient of the atmosphere of 0.08 to artificially perform on 55,000 images randomly obtained from the IMAGENET dataset, adding haze and randomly dividing it into the training set and test set according to the ratio of 10:1. Since the images of the IMAGENET dataset are in color and the optical diffraction neural networks is based on coherent light propagation, the image dataset needs to be preprocessed into grayscale images. Color images can also be applied to the optical diffraction deep neural networks, for example, the red, green, and blue channels of the color images are input into the networks independently in parallel, but this method is not used in this article. In addition, in order to assess the networks performance more objectively and fairly, this article also uses the RESIDE dataset [28] for testing. For testing, the RESIDE dataset contains 500 synthetic hazy images in Synthetic Objective Testing Set (SOTS) and 10 synthetic hazy images in Hybrid Subjective Testing Set (HSTS) with their respective ground truths. The RESIDE dataset is a large-scale benchmark consisting of both synthetic and real-world hazy images.

 figure: Fig. 6.

Fig. 6. Dataset images. (a)The label dataset (b)The input dataset.

Download Full Size | PDF

4.2 Model simulation results

The experimental part of this article uses Python3.6.0 and TensorFlow1.4.0 as the experimental platform, using Windows 10 system, with Intel i7-8700 CPU@3.2GHz, 64.0G RAM, NVIDIA GeForce RTX 2080Ti GPU. In order to objectively evaluate the effectiveness of the algorithm in this article, we will use the mainstream image restoration evaluation index structure similarity (SSIM) [28] and peak signal-to-noise ratio (PSNR) [29] to analyze the experimental results. SSIM is an evaluation index that measures the similarity of two images from the three perspectives of structure, brightness, and contrast. SSIM ranges from 0 to 1 and when the value of SSIM is equal to 1, the two images are identical. The equation is:

$$SSIM\textrm{ = }[l{(x,y)^\alpha } \cdot c{(x,y)^\alpha } \cdot s{(x,y)^\alpha }]$$
$$l(x,y) = \frac{{2{\mu _x}{\mu _y} + {c_1}}}{{\mu _x^2 + \mu _y^2 + {c_1}}}\textrm{ }c(x,y) = \frac{{2{\sigma _x}{\sigma _y} + {c_2}}}{{\sigma _x^2 + \sigma _y^2 + {c_2}}}\textrm{ }z(x,y) = \frac{{2{\sigma _{xy}} + {c_3}}}{{{\sigma _x}{\sigma _y} + {c_3}}}$$
where $\alpha \textrm{ = }\beta \textrm{ = }\gamma \textrm{ = }1$, ${\mu _x}$ and ${\mu _y}$ are the mean values of $x$ and $y$ respectively. $\sigma _x^2$ and $\sigma _y^2$ are the variances of x and $y$, respectively. ${\sigma _{xy}}$ is the covariance of $x$ and $y$, c1, c2, c3 are three non-zero constants used to avoid dividing zero.

PSNR is an objective criterion for the amount of noise in the image. With the increase of PSNR value, the lower the degree of image distortion. The equation can be expressed as:

$$PSNR\textrm{ = }10 \cdot {\log _{10}}(\frac{{MAX_I^2}}{{MSE}})$$
where $MA{X_I}$ is the pixel value of the largest pixel in the image, and $MSE$ is the mean square error of the image. For two grayscale images $I$ and $K$ with a size of $m\ast n$, the equation is:
$$MSE\textrm{ = }\frac{1}{{mn}}\sum\limits_{i = 0}^{m - 1} {\sum\limits_{j = 0}^{n - 1} {||I(i,j) - K(i,j)|{|^2}} } $$

As shown in Fig. 7, when the model parameters are not optimized, the quality of the output result is bad after the input image is spread freely through the networks. According to the evaluation of the SSIM index, the SSIM value of the image under free propagation and the haze-free image is only 0.5709, which is mainly caused by the physical nature of the optical diffraction neural networks to further freely diffract the input image. After the model is trained, the hazy image can be restored better by modulating the phase of the diffracted light. Subjectively, the image quality is significantly improved. At this time, the SSIM value of the image and the hazy-free image is 0.7749. The trained model’s SSIM value is higher 0.204 than free propagation. Judging from the effect of dehazing in Fig. 8, the detail fidelity of the restored image is obviously improved, and the interference of haze is obviously eliminated.

 figure: Fig. 7.

Fig. 7. Networks dehazing effect. Free Space and Output Dehazed Image are the output images of the networks before and after training, respectively.

Download Full Size | PDF

 figure: Fig. 8.

Fig. 8. Subjective effects of IMAGENET dataset and RESIDE dataset. From top to bottom are the fogged image, the restored image and the original image. The first three columns belong to the IMAGENET dataset, the rest belong to RESIDE dataset.

Download Full Size | PDF

In order to achieve the optimal dehazing effect, this paper supervises the training effect of the diffraction neural networks. As shown in Fig. 9 and Fig. 10, the networks performance increases with the number of iterations and finally stabilizes. The optimal value of SSIM is 0.8137 and the PSNR value is 22.1901. This article is also compared with other mainstream image dehazing algorithms [4,3034]. From Table 1, it can be seen that the algorithm designed in this article has achieved better results which the output image has stronger structural similarity and better details of the repair effect.

 figure: Fig. 9.

Fig. 9. Recovery result under different epochs.

Download Full Size | PDF

 figure: Fig. 10.

Fig. 10. Plot for the mean square error with iterations.

Download Full Size | PDF

Tables Icon

Table 1. Comparison of experimental results with different networks under the IMAGENET dataset and the RESIDE dataset

In order to obtain a more compact and less computationally intensive networks structure, the networks has made some sacrifices in accuracy, but despite this, the dehazing effect of the networks has reached an advanced level according to SSIM and PSNR from Table 1. And since this algorithm uses optical signals as input, it does not need to prepare the image in advance, therefore, it has greater potential in real-time image processing compared with other dehazing methods.

4.3 Parameter analysis

In order to test the performance of the algorithm proposed in this paper and the ability to dehaze different hazy environment, this paper analyzes the networks layer size, the networks layer spacing, and the influence of atmospheric light intensity and attenuation coefficient of the atmosphere on algorithm performance in the case of 100 rounds of training and a training batch size of 2. After changing the networks structure parameters, the networks is trained for the same number of times and tested using the IMAGENET and RESIDE test set respectively. According to Fig. 11, it can be seen that when the number of neurons in each layer of the networks is less than 160,000 (400*400), the image dehazing effect is enhanced with the increase of the number of neurons. But when the number of neurons reaches a certain number, the dehazing effect tends to be stable. Because the light can only travel within the range of the diffraction angle under the condition of a certain networks spacing, the neurons in the next layer are unable to effectively use all the information passed from the previous layer. In order to realize the miniaturization and compactness of the diffraction networks, the networks should be as compact as possible under the premise of satisfying the performance. And unlike tasks such as recognition and classification that require the network to obtain global information about objects, imaging tasks such as dehazing only require the networks to obtain information around each pixel. Even in the experimental analysis, it can be found that the performance of the networks decreases when the networks spacing is too large and the connectivity is too high. According to Fig. 12, for IMAGENET dataset, when the number of neurons is constant, the maximum SSIM value is 0.8127 as the networks layer distance is 0.004 m, and the PSNR value reaches the maximum of 21.2088 as the networks layer distance is 0.006 m. And for RESIDE dataset, the maximum SSIM value is 0.8411 as the networks layer distance is 0.005 m, and the PSNR value reaches the maximum of 22.4239 as the networks layer distance is 0.006 m.

 figure: Fig. 11.

Fig. 11. SSIM and PSNR rate with different number of neurons under the IMAGENET and RESIDE.

Download Full Size | PDF

 figure: Fig. 12.

Fig. 12. SSIM and PSNR rate with different layer distance under the IMAGENET and RESIDE.

Download Full Size | PDF

Therefore, the diffraction neural networks with a networks spacing is 0.005 m and the number of neurons in each layer is 160,000 (400*400) is selected to analyze the dehazing ability of different hazy environments. Respectively input the hazy image generated by the different atmospheric light intensity $A$ and the different attenuation coefficient of the atmosphere $\beta \; $ to the diffraction neural networks. As shown in Fig. 13, subjectively, the effect of the diffraction neural networks on different environments is similar. As shown in Fig. 14 and Fig. 15, according to the analysis of objective indicators of SSIM and PSNR, they only fluctuate within the range of 0.01 and 0.4 respectively. The reason the PSNR fluctuates in a larger range is the PSNR is based on the error-sensitive evaluation index and the human eyes are more sensitive to brightness and contrast, the PSNR evaluation index is often inconsistent with the subjective perception of the human eyes. The data shows that different hazy scenes have little effect on the performance of the diffraction neural networks, and the dehazing algorithm is robust to scene brightness and haze density.

 figure: Fig. 13.

Fig. 13. (a)The Input images at different atmospheric light intensities. (b)The output images of (a). (c)The Input images with different transmission coefficients. (d)The output images of (c).

Download Full Size | PDF

 figure: Fig. 14.

Fig. 14. SSIM and PSNR rate with different atmospheric light intensity $A$.

Download Full Size | PDF

 figure: Fig. 15.

Fig. 15. SSIM and PSNR rate with different attenuation coefficient of the atmosphere $\beta $.

Download Full Size | PDF

5. Conclusion

The increase of the particle content in the atmosphere causes the light to not travel in a straight line, resulting in a decrease in image quality. In this paper, an image dehazing algorithm based on diffraction neural networks provides a new idea for image dehazing technology. According to the evaluation indexes of SSIM and PSNR, the hazy image quality of the algorithm is higher and the dehazing effect of different hazy scenes is stable and robust. Secondly, since the algorithm uses light signals as input, there is no need to preprocess the hazy image after the model training is completed, so it has greater potential in real-time image processing compared with other dehazing methods. However, the algorithm has high requirements on the input optical frequency, and it is concluded from most researches that the input frequency is mainly in the terahertz band. And how to dehaze images with broadband continuous light will be a good research direction in the future.

Funding

National Natural Science Foundation of China (52171343); Dalian high level Talents Innovation Support Plan (2019RJ08); Dalian key field innovation team (2021RT05); Liaoning Province "Unveiling the list and Taking the lead" project (85210038); Fundamental Research Funds for the Central Universities (3132021230).

Disclosures

The authors declare no conflicts of interest.

Data availability

Data underlying the results presented in this paper are not publicly available at this time but maybe obtained from the authors upon reasonable request.

References

1. J. Kim, L. Kim, and S. Hwang, “An advanced contrast enhancement using partially overlapped sub-block histogram equalization,” IEEE Trans. CSVT. 11(4), 475–484 (2001).

2. J. Jaeho, Y. Bae, and J. B. Ra, “Contrast-Enhanced Fusion of Multisensor Images Using Subband-Decomposed Multiscale Retinex,” IEEE Trans. on Image Process. 21(8), 3479–3490 (2012). [CrossRef]  

3. C. H. Yeh, L. W. Kang, M. S. Lee, and C. Y. Lin, “Haze effect removal from image via haze density estimation in optical model,” Opt. Express 21(22), 27127–27141 (2013). [CrossRef]  

4. B. Cai, X. Xu, K. Jia, C. Qing, and D. Tao, “Dehazenet: an end-to-end system for single image haze removal,” IEEE Trans. on Image Process. 25(11), 5187–5198 (2016). [CrossRef]  

5. Z. He and V. M. Patel, “Densely Connected Pyramid Dehazing Network,” in Proceedings of IEEE Conference on Computer Vision and Pattern Recognition (IEEE, 2018), pp. 3194–3203.

6. W. Ren, L. Ma, J. Zhang, J. Pan, X. Cao, W. Liu, and Y. Mh, “Gated fusion network for single image dehazing,” in Proceedings of IEEE Conference on Computer Vision and Pattern Recognition (IEEE, 2018), pp. 3253–3261.

7. D. Chen, M. He, Q. Fan, J Liao, L. Zhang, D. Hou, L. Yuan, and G. Hua, “Gated Context Aggregation Network for Image Dehazing and Deraining,” in Proceedings of IEEE Winter Conference on Applications of Computer Vision (IEEE, 2019), pp. 1375–1383.

8. M. Robert A, Unconventional Computational Problems: A Volume in the Encyclopedia of Complexity and Systems Science (Springer, New York, 2009).

9. S. Jm, B. Sm, M. Rp, and N. Sw, “Superconducting optoelectronic circuits for neuromorphic computing,” Phys. Rev. Appl. 7(3), 034013 (2017). [CrossRef]  

10. R. Hide, “Optics of the Atmosphere: Scattering by Molecules and Particles,” Phys. Bull. 28(11), 521 (1977). [CrossRef]  

11. D. Psaltis, D. Brady, G. X. g, and S. Lin, “Holography in artificial neural networks,” Nature 343(6256), 325–330 (1990). [CrossRef]  

12. A. V. Krishnamoorthy, G. Yayla, and S. Esener, “Design of a scalable opto-electronic neural system using free-space optical interconnects,” in Proceedings of IJCNN-91-Seattle International Joint Conference on Neural Networks (IEEE, 1992), pp. 527–534.

13. Y. Shen, N. C. Harris, S. Skirlo, D. Englund, and M. Soljacic, “Deep learning with coherent nanophotonic circuits,” Nat. Photonics 11(7), 441–446 (2017). [CrossRef]  

14. M. Robert A, Principles of Neuromorphic Photonics: A Volume in the Encyclopedia of Complexity and Systems Science (Springer, New York, 2009).

15. X. Lin, Y. Rivenson, N. T. Yardimci, M. Veli, M. Jarrahi, and A. Ozcan, “All-optical machine learning using diffractive deep neural networks,” Science 361(6406), 1004–1008 (2018). [CrossRef]  

16. J. Chang, S. Vincent, D. Xiong, H. Wolfgang, and W. Gordon, “Hybrid optical-electronic convolutional neural networks with optimized diffractive optics for image classification,” Sci. Rep. 8(1), 12324–10 (2018). [CrossRef]  

17. N. M. Estakhri, B. Edwards, and N. Engheta, “Inverse-designed metastructures that solve equations,” Science 363(6433), 1333–1338 (2019). [CrossRef]  

18. T. W. Hughes, I. Williamson, M. Minkov, and S. Fan, “Wave physics as an analog recurrent neural network,” Sci. Adv. 5(12), 034013 (2019). [CrossRef]  

19. E. Hansen J and D. Travis L, “Light scattering in planetary atmospheres,” Space Sci. Rev. 16(4), 527–610 (1974). [CrossRef]  

20. H. Hulst, “Light scattering by small particles,” Phys. Today 10(12), 28–30 (1957). [CrossRef]  

21. J. W. Goodman, Introduction to Fourier optics (Roberts and Company Publishers, 2005), Chap. 4.

22. E. J. Mccartney and F. F. Hall, “Scattering Phenomena. (Book Reviews: Optics of the Atmosphere. Scattering by Molecules and Particles),” Science 196(4294), 1084–1085 (1977).

23. Z. Ying, L. Bohan, Z. Yujun, J. Yue, C. Y. Chiuan, C. Peng, J. G. Boong, L. Junwei, and D. Shengwang, “All Optical Neural Network with Nonlinear Activation Functions,” OPTICA 6(9), 1132–1137 (2019). [CrossRef]  

24. B. Xu, N. Wang, T. Chen, and M. Li, “Empirical Evaluation of Rectified Activations in Convolutional Network,” Computer ence (2019).

25. L. Lu, Y. Shin, Y. Su, and K. Ge, “Dying ReLU and Initialization: Theory and Numerical Examples,” CiCP 28(5), 1671–1706 (2020). [CrossRef]  

26. D. Kingma and J. Ba, “Adam: A Method for Stochastic Optimization.” ArXiv14126980 Cs (2014).

27. D. Jia, D. Wei, R. Socher, L. Lj, L. Kai, and L. Ff, “ImageNet: A Large-Scale Hierarchical Image Database,” in Proceedings of IEEE Conference on Computer Vision and Pattern Recognition (IEEE, 2019), pp. 248–255.

28. B. Li, W. Ren, D. Fu, D. Tao, D. Feng, and W. Zeng, “Benchmarking Single Image Dehazing and Beyond,” IEEE Trans. on Image Process. 28(1), 492–505 (2019). [CrossRef]  

29. W. Zhou, B. Ac, S Hr, and S. Ep, “Image quality assessment: from error visibility to structural similarity,” IEEE Trans. Image Process. 13(4), 600–612 (2014). [CrossRef]  

30. H. Kwon, Y. Kaist, and S. Lin, “Data-driven depth map refinement via multi-scale sparse representation,” in Proceedings of IEEE Conference on Computer Vision and Pattern Recognition (IEEE, 2015), pp. 159–167.

31. D. Makkar and M. Malhotra, “Single Image Haze Removal Using Dark Channel Prior,” in Proceedings of IEEE Conference on Computer Vision and Pattern Recognition (IEEE, 2011), pp. 2341–2353.

32. B. Li, X. Peng, Z. Wang, J. Xu, and F. Dan, “AOD-Net: All-in-One Dehazing Network,” in Proceedings of IEEE International Conference on Computer Vision (IEEE, 2017), pp. 4780–4788.

33. K. Mei, J. a, and J. Li, “Progressive Feature Fusion Network for Realistic Image Dehazing,” LNCS 11361, 203–215 (2019). [CrossRef]  

34. K. Swami and S. K. Das, “CANDY: Conditional Adversarial Networks based Fully End-to-End System for Single Image Haze Removal,” inProceedings of International Conference on Pattern Recognition, pp. 3061–3067 (IEEE, 2018).

Data availability

Data underlying the results presented in this paper are not publicly available at this time but maybe obtained from the authors upon reasonable request.

Cited By

Optica participates in Crossref's Cited-By Linking service. Citing articles from Optica Publishing Group journals and other participating publishers are listed here.

Alert me when this article is cited.


Figures (15)

Fig. 1.
Fig. 1. Physical model of atmospheric scattering.
Fig. 2.
Fig. 2. Effects of Atmospheric Scattering on Images. (a)Hazy-free image (b)Fogged image in A = 0.7, $\beta $=0.04 (c) Fogged image in A = 0.7, $\beta $=0.08 (d) Fogged image in A = 0.5, $\beta $=0.08.
Fig. 3.
Fig. 3. Networks structure model. (a) Optical diffraction neural networks (b) Fully-connected networks.
Fig. 4.
Fig. 4. Mathematical models of Leaky-ReLU function.
Fig. 5.
Fig. 5. Algorithmic steps for optical diffraction neural networks.
Fig. 6.
Fig. 6. Dataset images. (a)The label dataset (b)The input dataset.
Fig. 7.
Fig. 7. Networks dehazing effect. Free Space and Output Dehazed Image are the output images of the networks before and after training, respectively.
Fig. 8.
Fig. 8. Subjective effects of IMAGENET dataset and RESIDE dataset. From top to bottom are the fogged image, the restored image and the original image. The first three columns belong to the IMAGENET dataset, the rest belong to RESIDE dataset.
Fig. 9.
Fig. 9. Recovery result under different epochs.
Fig. 10.
Fig. 10. Plot for the mean square error with iterations.
Fig. 11.
Fig. 11. SSIM and PSNR rate with different number of neurons under the IMAGENET and RESIDE.
Fig. 12.
Fig. 12. SSIM and PSNR rate with different layer distance under the IMAGENET and RESIDE.
Fig. 13.
Fig. 13. (a)The Input images at different atmospheric light intensities. (b)The output images of (a). (c)The Input images with different transmission coefficients. (d)The output images of (c).
Fig. 14.
Fig. 14. SSIM and PSNR rate with different atmospheric light intensity $A$.
Fig. 15.
Fig. 15. SSIM and PSNR rate with different attenuation coefficient of the atmosphere $\beta $.

Tables (1)

Tables Icon

Table 1. Comparison of experimental results with different networks under the IMAGENET dataset and the RESIDE dataset

Equations (13)

Equations on this page are rendered with MathJax. Learn more.

I(x,y) = J(x,y)t(x,y)+A[1t(x,y)]
t(x,y) = exp[βd(x,y)]
Wil(x,y,z) = zzir2(12πr+1jλ)exp(j2πrλ)
ti(xi,yi,zi) = αil(xi,yi,zi)exp[jϕil(xi,yi,zi)]
nil(x,y,z) = Wil(x,y,z)til(xi,yi,zi)knkl1(xi,yi,zi) = Wil(x,y,z)|A|ejϑ
f(x) = {xx00.2xx<0
{ni,pl = wi,pltilf(mil)mil = knk,il1til = exp(jS(ϕil(xi,yi,zi)))S(x) = 2π1+ex
siM+1 = |miM+1|2
MAE(ϕil) = 1kk|skM+1gkM+1|
SSIM = [l(x,y)αc(x,y)αs(x,y)α]
l(x,y)=2μxμy+c1μx2+μy2+c1 c(x,y)=2σxσy+c2σx2+σy2+c2 z(x,y)=2σxy+c3σxσy+c3
PSNR = 10log10(MAXI2MSE)
MSE = 1mni=0m1j=0n1||I(i,j)K(i,j)||2
Select as filters


Select Topics Cancel
© Copyright 2024 | Optica Publishing Group. All rights reserved, including rights for text and data mining and training of artificial technologies or similar technologies.