Open Access
Add to BibSonomy
Abstract
Due to the small core diameter, a single-core multimode fiber (MMF) has been extensively investigated for endoscopic imaging. However, an extra light path is always utilized for illumination in MMF imaging system, which takes more space and is inapplicable in practical endoscopy imaging. In order to make the imaging system more practical and compact, we proposed a dual-function MMF imaging system, which can simultaneously transmit the illumination light and the images through the same imaging fiber. Meanwhile, a new deep learning-based encoder-decoder network with full-connected (FC) layers was designed for image reconstruction. We conducted an experiment of transmitting images via a 1.6 m long MMF to verify the effectiveness of the dual-function MMF imaging system. The experimental results show that the proposed network achieves the best reconstruction performance compared with the other four networks on different datasets. Besides, it is worth mentioning that the cropped speckle patterns can still be used to reconstruct the original images, which helps to reduce the computing complexity significantly. We also demonstrated the ability of cross-domain generalization of the proposed network. The proposed system shows the potential for more compact endoscopic imaging without external illumination.
© 2022 Optica Publishing Group under the terms of the Optica Open Access Publishing Agreement
1. Introduction
In recent years, multimode fibers (MMFs) have gradually become a popular tool in the field of image transmission, especially in medical endoscopy applications [1–4]. Compared with fiber bundles consisting of a large number of single-mode fiber cores, a single-core MMF can obtain higher resolution and less invasion for image transmission in endoscopic imaging applications [5]. However, the coherent light propagating through MMFs forms a random speckle at the output end because of mode coupling and mode interference. The speckle pattern carries the information of the input spatial distribution, which can be reconstructed by computation.
In previous medical applications, fiberoptic endoscopes usually consist of two optical fibers: one illuminating fiber as an external illumination for transmitting light and illuminating the surface of the observed object; one imaging optical fiber for transmitting the reflected image to the observation end. The size of the endoscopy is determined by both the illuminating fiber and the imaging fiber. Similar to the dual-fiber systems, almost all previous MMF imaging systems adopted the external illumination scheme [6–17], which limits the compactness, flexibility and practicability of imaging systems.
Moreover, image reconstruction from speckle patterns is another challenge in MMF imaging systems. MMF was first utilized for image transmission in 1967 by Spitz and Wertz [18], who used phase conjugation method to eliminate the distortion. Then several methods developed from scattering media imaging, such as digital scanning and holography technique, have been applied to solve the problem of image reconstruction from speckles [5,19]. In recent years, with the growth of computing power, deep learning techniques have started to be applied in MMF imaging [7–17]. Compared with traditional methods, deep learning techniques are faster in image reconstruction and show better robustness to the environment [8,9]. Some neural networks such as U-net [10] and single hidden layer dense neural network (SHL-DNN) [11] have been proven effective. By collecting a large number of input images and their matching speckles, the networks are trained to learn the mapping relationship between the two, ultimately allowing the network to reconstruct original images from the received speckles.
However, these neural networks borrowed from other imaging tasks are not fully suitable for MMF imaging. In MMF imaging system, localized information is encoded in the global distribution of speckles, which makes the speckle nearly a random pattern that contains almost no relevant local features of natural images [11,12]. Therefore, it is more appropriate to use full-connected (FC) layers than convolutional layers for feature extraction of speckle patterns. On the other hand, the reconstruction target is natural image, so using a FC layer at the output end of networks would lose prior knowledge of natural images, such as local smoothness. Therefore, it is more reasonable to use a convolutional layer rather than an FC layer at the output end of network, which can improve the quality of reconstructed images [20–22]. As a result, an encoder-decoder network with FC layers is more suitable for the speckle-to-image task.
In this paper, a new dual-function MMF imaging system is proposed, which modulates the image information on a speckle pattern to implement illumination and image transmission based on a single MMF simultaneously. Specifically, the light source is injected into one port of the MMF coupler and then forms a speckle pattern to illuminate the object being observed, while the reflected light is output at another port on the same side of the coupler and captured by the camera. In this manner, we can illuminate the object and collect the speckles on the same side of MMF, which makes the MMF imaging system more compact and flexible. In addition, we also design a new network for image reconstruction. An FC layer is used as the input layer before an encoder-decoder network structure to extract features from random speckles effectively. In the experiment, we compare the proposed network with the original encoder-decoder network and SHL-DNN on several different datasets. Furthermore, two encoder-decoder networks which contain FC layers at the middle or output end are also designed for comparison. We transmit various kinds of images with the proposed system and obtain a significant improvement of reconstruction quality using our network, which demonstrates the potential of our system in endoscopic imaging. We also study the influence of different crop sizes and crop locations on the reconstruction fidelity. Because each pixel of the received speckle pattern is contributed by large areas of the input images, cropped speckle patterns still have the ability of reconstruction, which can significantly reduce the computing complexity. Finally, we demonstrate the ability of cross-domain generalization of the proposed network, which shows the application potential of the system in different scenarios.
2. Methods
2.1 Experimental setup
The experiment setup of dual-function MMF imaging system is shown in Fig. 1(a). A laser beam of 632.8 nm wavelength is generated by a He-Ne laser (Thorlabs, HNL210LB), and the coherence length is approximately 30 cm. The beam is injected into the port 1 of a 1.6 m long MMF coupler (50:50 Split, 105-µm-core, 0.22 NA, Thorlabs) after an attenuator (NDC-100C-4M, Thorlabs) and an objective (OBJ1, PLN 40×, Olympus). The length of the fiber on the left of the coupler is 0.8 m, and the length of the coupler is 0.075m. The attenuator is used to adjust the intensity of the light source and OBJ1 is used to enlarge the size of the beam. After being transmitted through the MMF coupler, the beam is output at port 2 and magnified by OBJ2 (same as OBJ1). Next, the light is modulated by a 1280×768 pixels digital micromirror device (DMD, V-7001, VIALUX), which loads input images by a computer. The distance between port 2 and the left end of OBJ2 is 0.6 mm, and the distance between DMD and the right end of OBJ2 is 180 mm. Because orientation of the micromirror axis of rotation is 45° and input illumination optical axis is oriented at 24° relative to the window normal, when light hits the plane of DMD vertically, it does not come out vertically. In order to make light illuminate the DMD and reflect in the same path, the DMD has to be placed at an oblique diagonal angle, which tilts modulated images diagonally. Note that the output of the MMF is not a uniform spot, but a random speckle pattern due to the transmission through the MMF, i.e., the image generated by DMD is illuminated by a speckle pattern. The beam splitter and camera 1 (Basler ace acA1300-200um) are used to observe the reflected images. As shown in Fig. 1(b), when an all-white image is modulated on DMD, camera 1 captures a speckle pattern. The input condition of the light source is fixed to avoid the change of the illumination pattern. The modulated light beam is reflected and coupled into port 2 after shrunk by OBJ2. The light beam is further transmitted by the MMF coupler and output at the port 3. Finally, the output speckle pattern is magnified by OBJ3 (same as OBJ1) and captured by a 1280×1024-pixel monochrome camera 2 (Basler ace acA2000-165um). Due to the reflection of the fiber end of port 2, camera 2 captures a relatively weak speckle pattern when there is no image on DMD, as shown in Fig. 1(c). The inherent noise generated by this experimental system makes it more challenging to reconstruct the original images from the speckle patterns.
In data collection stage, the modulated images on the DMD are from three different datasets, namely Fashion-mnist [23], Quickdraw [24] and SIPaKMeD [25], to verify the effectiveness of the system on different datasets. The Fashion-mnist and Quickdraw dataset are datasets consisting of 28×28 grayscale fashion products or simple drawings. The SIPaKMeD dataset consists of colorful images of isolated cells. These three kinds of images are converted and resized to 192×192 pixels grayscale and modulated on the center of DMD. The size of the speckle patterns captured by the camera are 832×832 pixels. 30000 speckle patterns of each dataset are collected by the proposed system for further data processing.
Fig. 1. (a) Experiment setup for dual-function MMF imaging. DMD: digital micromirror device; OBJ: microscope objective lens; BS: beam splitter. (b) An image captured by camera 1 when an all-white image is modulated on DMD. (c) An image captured by camera 2 when DMD does not modulate any image
2.2 Network structure
Convolutional neural networks (CNNs), such as U-net [10], have been widely used for MMF imaging tasks. U-net usually consists of an encoder with down-sampling units, a decoder with up-sampling units, and skip connections. The input and output images of such end-to-end CNNs are usually natural images with relevant local features. CNNs can extract features from input images, while skip connections between encoder and decoder transmit edge and texture information of the image from the shallow layers to the deeper layers. However, the information at the shallow layer is chaotic in the MMF imaging task, and the retention of skip connections will degrade the reconstruction performance of the network [12,13]. Moreover, since the input speckle patterns do not contain relevant local features of the natural images, it is not appropriate to extract the information directly by CNNs. Several studies have demonstrated that SHL-DNN is able to produce the same results as U-net [11], since localized information is encoded in the global distribution of speckles, and FC layers can extract information from both local and global features. Nonetheless, the SHL-DNN processes the images as one-dimensional information, which will lose the structural information of the output images. Facing the above concerns, we propose an FC-encoder-decoder network to improve the reconstruction performance of MMF imaging.
The proposed network structure is shown in Fig. 2(a). The speckle patterns are cropped out in the center to 624×624-pixel size and resized to 192×192 pixels. The original images are resized to 48×48 pixels. Input speckle patterns are flattened into a size of 36864×1, and then transformed into a size of 2304×1 after the FC layer. The output of the FC layer is reshaped into a size of 48×48, which is followed by an encoder-decoder structure. The encoder-decoder structure is made of two down-sampling units and two up-sampling units, and each unit consists of two 3×3 convolutional layers followed by the rectified linear unit (ReLU). The down sampling and up sampling are implemented by max-pooling layers and transposed convolutional layers, respectively. The output layer contains a 1×1 convolutional layer and a tanh function.
Fig. 2. Image reconstruction model structure of (a) FC-encoder-decoder network, (b) encoder-decoder network, (c) SHL-DNN, (d) Encoder-FC-decoder network and (e) Encoder-decoder-FC network.
We additionally train four different networks for comparison to verify the performance of FC-encoder-decoder network. Figure 2(b) shows an encoder-decoder structure network without FC layers [13]. An SHL-DNN [11] is shown in Fig. 2(c). Furthermore, in order to investigate the effect of the FC layer in different places of the network, we design two networks for comparison. A network with FC layers between encoder and decoder structure is shown in Fig. 2(d) [12]. Figure 2(e) shows a network with FC layer output. In Fig. 2, all arrows of the same color represent the same calculation structure and parameters.
27000 data pairs of each dataset are selected as the training data and 3000 for testing. Adam optimizer is used to minimize the objective function mean square error (MSE) between the network output and truth images. The learning rate is set as 0.0001. The network is implemented by Pytorch, and trained on one NVIDIA 3090 GPU with Intel Xeon Gold 5218 CPU for 150 epochs.
3. Experiment results
3.1 Reconstruction results with different networks
Figure 3 shows the results of image reconstruction for two images of each dataset. Images in row 2 illustrates the received images in camera 1 with modulating the original images onto a speckle pattern, which still have some outlines of the original images. Because the DMD is placed diagonally, the modulated images are also diagonal. The reconstruction results in row 4 to 8 are selected from the test set. Peak Signal to Noise Ratio (PSNR) and structural similarity index metrics (SSIM) [26] of 3000 test sets are calculated for evaluating the reconstruction results, which is shown in Table 1. Although all the networks are able to reconstruct the images, our network achieves the best results on all datasets. Our proposed network performs 2.99 dB higher in average PSNR over the encoder-decoder network without the FC layer and 2.12 dB higher over the SHL-DNN. In the Fashion-mnist dataset, only our proposed network can accurately reconstruct the dark spot of the shoe and the handle of the handbag. In the Quickdraw dataset, the reconstructed images of our method are structurally accurate without artifacts. In the SIPaKMeD dataset, the difference between the reconstructed images and the original images is still visible. However, our network can reconstruct the cell nuclei and cell edges more clearly and accurately, which provides convenience for subsequent cell recognition and classification.
Table 1. PSNR and SSIM of reconstruction images with different networks and datasets
It is obvious that both SHL-DNN and Encoder-decoder-FC network, which contain FC layers at the output end, are significantly lower in SSIM than other networks. It can be seen that the reconstructed images of these two networks are more different from the original images in intensity and structure compared with other networks. Since the output of networks is natural images with local smoothness, FC layers loses the structural information and make the images uneven. Among the other three networks, our FC-encoder-decoder network achieves the optimal reconstruction results due to the addition of an FC layer at the input end. It shows that the FC layer can improve the ability of feature extraction from random speckle patterns.
Figure 4 shows the curves that the PSNR and SSIM of test sets versus training epochs. All five networks converge after 150 epochs of training, while our network achieves better image reconstruction fidelity than other networks. However, the networks cost very different times per epoch, since the numbers of network parameters are different. The time required for training 150 epochs of each network separately is recorded in Table 2. Our proposed network requires far less training time than other networks. The training time of SHL-DNN and Encoder-FC-decoder network is quite long, since they contain multiple FC layers, which needs a large number of parameters. Nonetheless, although our FC-encoder-decoder network also contains an FC layer, it contains fewer down-sampling units than encoder-decoder network, which still results in less training time. As shown in Fig. 4(c) and Fig. 4(f), the training process becomes unstable because the cell images in SIPaKMeD dataset are more complex. However, the average training results in our network are still better than those in other networks.
Fig. 4. (a)-(c) PSNR of test sets versus epochs in (a) Fashion-mnist, (b) Quickdraw and (c) SIPaKMeD; (d)-(f) SSIM of test sets versus epochs in (d) Fashion-mnist, (e) Quickdraw and (f) SIPaKMeD.
Table 2. 150 epochs training time for each network
Although the proposed network is superior to the other networks mentioned above in terms of reconstruction results and computational speed, it still has room for improvement. In recent years, with the deepening study of neural network structure, networks with more complex structure have been used for super-resolution tasks to amplify edge and texture information in reconstructions [27,28]. In the field of scattering media imaging, some studies also proposed a deep CNN based on multi-channel residual blocks for super-resolution and denoising [29]. The application of these networks in MMF imaging system deserves further research, and it has great significance for improving imaging resolution.
3.2 Reconstruction with different crop size and crop location of speckle patterns
When the images projected on the proximal side of the MMF travel with different propagation constants along the fiber length, localized image information will be spread across the fiber cross-section. Moreover, the spreading of localized information is further enhanced when the light exits the fiber and interferes to produce the speckle pattern [14]. As a result, each pixel of the recorded distal speckle pattern is contributed by large areas of the input images. In other words, it is possible that a small part of the speckle pattern contains almost all necessary information for reconstruction. It is valuable to evaluate the influence of different cropped areas of images on reconstruction fidelity. We crop the center of the speckle patterns to 832×832, 624×624, 500×500, 374×374 and 250×250 pixels for image reconstruction. Meanwhile, we change the crop location of the 250×250 pixels to evaluate the influence of crop location on the reconstruction fidelity. All the cropped speckle patterns are resized to 192×192 pixels and trained by our proposed FC-encoder-decoder network for 150 epochs.
Figure 5 shows that reconstruction fidelity decreases with decreasing crop sizes in general. For a crop size of 250×250 pixels, equivalent to around 9.03% area of the full speckle pattern, the PSNR decreases by 1.89dB and the SSIM decreases by 0.047. Although larger crop size has a larger down sampling ratio, it still achieves better reconstruction performance than smaller crop size, which indicates that a larger crop size contains more information. Particularly, for a crop size of 624×624 pixels, equivalent to 56.25% area of the full speckle pattern, the reconstruction fidelity is almost the same as that of full speckle pattern. It can be determined that the speckle pattern contains a lot of invalid information in the four corners of the image as it is circular. Therefore, in the previous experiments with different networks, we adopt the crop size of 624×624 pixels for less calculation.
Fig. 5. Reconstruction results with different crop size and crop location. Row 1: Crop boxes of speckle patterns. In fact, the speckle patterns of the three images are different, the crop boxes just indicate the crop size and location. Row 2-4: Reconstruction results of three datasets. PSNR and SSIM are average of test set, not the value of a single image.
Images in the three columns on the right of Fig. 5 show the influence of crop location on the reconstruction fidelity. Reconstruction fidelity of center cropped speckle is similar to that of right cropped speckle, which indicates that the information of input image has been evenly spread over the whole speckle pattern. However, reconstruction fidelity of top right cropped speckle is significantly lower than that of the other two, because the crop image contains some invalid information.
3.3 Cross-domain generalization
Transferability of networks is quite valuable in deep learning tasks [30]. In practice, it is usually difficult to collect a large number of examples similar to target objects in advance. Meanwhile, it is valuable to demonstrate that the proposed network learns the underlying physical transmission model instead of memorizing specific patterns. Therefore, we conduct an experiment that images from one kind of dataset are reconstructed with a network trained by different images from another dataset.
Additional 10000 data pairs of MNIST [31] dataset are collected when we collect 30000 data pairs of Fashion-MNIST dataset. We train the FC-encoder-decoder network by 30000 datasets from Fashion-MNIST and test it with 10000 datasets from MNIST. Several reconstruction results from test set are shown in Fig. 6. The mean PSNR of test sets is 14.97dB, and the mean SSIM of test sets is 0.672.
Fig. 6. Reconstruction results of Cross-domain generalization. Training sets are images from Fashion-mnist and test sets are from MNIST.
Although there is a little decline of reconstruction evaluation index in the test domain compared with the training domain, it still indicates that the proposed network possesses the ability of cross-domain generalization. Since the datasets we used contain obvious characteristics, it is difficult to reconstruct more complex images with networks they trained. However, training networks on a higher-entropy dataset helps to achieve better cross-domain generalization performance, which is also a method of optimization in practical applications [30].
4. Conclusion
We propose and experimental demonstrate a dual-function MMF imaging system with FC-encoder-decoder network. The speckle patterns are utilized to modulate images for MMF imaging at the first time. By modulating the images on speckle patterns, illumination and image transmission are simultaneously implemented, which avoids the use of external illumination and thus allows a more compact endoscopic imaging. We also design a new network including an FC layer followed by an encoder-decoder structure, which is able to extract features from speckles more sufficiently. We experimentally verify the effectiveness of the proposed MMF imaging system by transmitting images of three datasets. Compared with the other four methods, the proposed algorithm shows a significant improvement. In addition, we demonstrate that the cropped speckle patterns can still be used for reconstruction, which can significantly reduce the computing complexity. Finally, we show the cross-domain generalization ability of the proposed network. The results show that our method has the potential for compact and low-cost biological endoscopy.
Funding
National Key Research and Development Program of China (2018YFB2201803); National Natural Science Foundation of China (61821001, 61901045); Fund of State Key Laboratory of Information Photonics and Optical Communication BUPT (IPOC2021ZT18).
Disclosures
The authors declare no conflicts of interest.
Data availability
Data underlying the results presented in this paper are not publicly available at this time but may be obtained from the authors upon reasonable request.
References
1. D. Psaltis and C. Moser, “Imaging with multimode fibers,” Optics and Photonics News 27(1), 24–31 (2016). [CrossRef]
2. T. Čižmár and K. Dholakia, “Exploiting multimode waveguides for pure fibre-based imaging,” Nat. Commun. 3(1), 1–9 (2012). [CrossRef]
3. S. Turtaev, I. T. Leite, T. Altwegg-Boussac, J. M. Pakan, N. L. Rochefort, and T. Čižmár, “High-fidelity multimode fibre-based endoscopy for deep brain in vivo imaging,” Light: Sci. Appl. 7(1), 922018). [CrossRef]
4. M. Plöschner, T. Tyc, and T. Čižmár, “Seeing through chaos in multimode fibres,” Nat. Photonics 9(8), 529–535 (2015). [CrossRef]
5. Y. Choi, C. Yoon, M. Kim, T. D. Yang, C. Fang-Yen, R. R. Dasari, K. J. Lee, and W. Choi, “Scanner-free and wide-field endoscopic imaging by using a single multimode optical fiber,” Phys. Rev. Lett. 109(20), 203901 (2012). [CrossRef]
6. P. Caramazza, O. Moran, R. Murray-Smith, and D. Faccio, “Transmission of natural scene images through a multimode fibre,” Nat. Commun. 10(1), 2029 (2019). [CrossRef]
7. N. Shabairou, E. Cohen, O. Wagner, D. Malka, and Z. Zalevsky, “Color image identification and reconstruction using artificial neural networks on multimode fiber images: Towards an all-optical design,” Opt. Lett. 43(22), 5603–5606 (2018). [CrossRef]
8. P. Fan, T. Zhao, and L. Su, “Deep learning the high variability and randomness inside multimode fibers,” Opt. Express 27(15), 20241–20258 (2019). [CrossRef]
9. S. Resisi, S. M. Popoff, and Y. Bromberg, “Image transmission through a flexible multimode fiber by deep learning,” arXiv preprint arXiv:2011.05144 (2020).
10. N. Borhani, E. Kakkava, C. Moser, and D. Psaltis, “Learning to see through multimode fibers,” Optica 5(8), 960–966 (2018). [CrossRef]
11. C. Zhu, E. A. Chan, Y. Wang, W. Peng, R. Guo, B. Zhang, C. Soci, and Y. Chong, “Image reconstruction through a multimode fiber with a simple neural network architecture,” Sci. Rep. 11(1), 1–10 (2021). [CrossRef]
12. Y. Li, Z. Yu, Y. Chen, T. He, J. Zhang, R. Zhao, and K. Xu, “Image Reconstruction Using Pre-Trained Autoencoder on Multimode Fiber Imaging System,” IEEE Photonics Technol. Lett. 32(13), 779–782 (2020). [CrossRef]
13. S. Kurekci, A. T. Temur, M. E. Odabas, G. Afshari, and E. Yüce, “Deep learning-based image transmission through a multi-mode fiber,” in Unconventional Optical Imaging II, 11351 (International Society for Optics and Photonics), 1135126 (2020).
14. E. Kakkava, B. Rahmani, N. Borhani, U. Teğin, D. Loterie, G. Konstantinou, C. Moser, and D. Psaltis, “Imaging through multimode fibers using deep learning: The effects of intensity versus holographic recording of the speckle pattern,” Opt. Fiber Technol. 52, 101985 (2019). [CrossRef]
15. X. Zhang, Z. Yu, Z. Meng, K. Ding, Z. Ju, and K. Xu, “Experimental demonstration of a multimode fiber imaging system based on generative adversarial networks,” in Asia Communications and Photonics Conference, (Optical Society of America, 2019), T4A. 4.
16. Z. Yu, Z. Ju, X. Zhang, Z. Meng, F. Yin, and K. Xu, “High-speed multimode fiber imaging system based on conditional generative adversarial network,” Chin. Opt. Lett. 19(8), 081101 (2021). [CrossRef]
17. P. Fan, M. Ruddlesden, Y. Wang, L. Zhao, C. Lu, and L. Su, “Learning enabled continuous transmission of spatially distributed information through multimode fibers,” Laser Photonics Rev. 15(4), 2000348 (2021). [CrossRef]
18. E. Spitz and A. Werts, “Transmission des images à travers une fibre optique,” Comptes Rendus Hebdomadaires Des Seances De L Academie Des Sciences Serie B 264(14), 1015 (1967).
19. R. Di Leonardo and S. Bianchi, “Hologram transmission through multi-mode optical fibers,” Opt. Express 19(1), 247–254 (2011). [CrossRef]
20. I. Goodfellow, Y. Bengio, and A. Courville, Deep learning (MIT press, 2016).
21. W. Rawat and Z. Wang, “Deep convolutional neural networks for image classification: A comprehensive review,” Neural computation 29(9), 2352–2449 (2017). [CrossRef]
22. D. Ulyanov, A. Vedaldi, and V. Lempitsky, “Deep image prior,” in Proceedings of the IEEE conference on computer vision and pattern recognition, 2018), 9446–9454.
23. H. Xiao, K. Rasul, and R. Vollgraf, “Fashion-mnist: a novel image dataset for benchmarking machine learning algorithms,” arXiv preprint arXiv:1708.07747 (2017).
24. J. Jongejan, H. Rowley, T. Kawashima, J. Kim, and N. Fox-Gieg, “The Quick, Draw! - A.I. Experiment,” (2016), https://quickdraw.withgoogle.com/.
25. M. E. Plissiti, P. Dimitrakopoulos, G. Sfikas, C. Nikou, O. Krikoni, and A. Charchanti, “SIPAKMED: A new dataset for feature and image based classification of normal and pathological cervical cells in Pap smear images,” in 2018 25th IEEE International Conference on Image Processing (ICIP), (IEEE, 2018), 3144–3148.
26. A. Hore and D. Ziou, “Image quality metrics: PSNR vs. SSIM,” in 2010 20th international conference on pattern recognition, (IEEE, 2010), 2366–2369.
27. V. Sitzmann, J. N. Martel, A. W. Bergman, D. B. Lindell, and G. Wetzstein, “Implicit neural representations with periodic activation functions,” arXiv preprint arXiv:2006.09661 (2020).
28. T. Xu, P. Zhang, Q. Huang, H. Zhang, Z. Gan, X. Huang, and X. He, “Attngan: Fine-grained text to image generation with attentional generative adversarial networks,” in Proceedings of the IEEE conference on computer vision and pattern recognition, 2018), 1316–1324.
29. W. Zhang, S. Zhou, X. Sui, G. Gu, and Q. Chen, “Single image detecting enhancement through scattering media based on transmission matrix with a deep learning network,” Opt. Commun. 490, 126896 (2021). [CrossRef]
30. M. Deng, S. Li, Z. Zhang, I. Kang, N. X. Fang, and G. Barbastathis, “On the interplay between physical and content priors in deep learning for computational imaging,” Opt. Express 28(16), 24152–24170 (2020). [CrossRef]
31. L. Deng, “The mnist database of handwritten digit images for machine learning research [best of the web],” IEEE Signal Processing Magazine 29(6), 141–142 (2012). [CrossRef]
