Open Access
Add to BibSonomy
Abstract
Low-dose imaging techniques have many important applications in diverse fields, from biological engineering to materials science. Samples can be protected from phototoxicity or radiation-induced damage using low-dose illumination. However, imaging under a low-dose condition is dominated by Poisson noise and additive Gaussian noise, which seriously affects the imaging quality, such as signal-to-noise ratio, contrast, and resolution. In this work, we demonstrate a low-dose imaging denoising method that incorporates the noise statistical model into a deep neural network. One pair of noisy images is used instead of clear target labels and the parameters of the network are optimized by the noise statistical model. The proposed method is evaluated using simulation data of the optical microscope, and scanning transmission electron microscope under different low-dose illumination conditions. In order to capture two noisy measurements of the same information in a dynamic process, we built an optical microscope that is capable of capturing a pair of images with independent and identically distributed noises in one shot. A biological dynamic process under low-dose condition imaging is performed and reconstructed with the proposed method. We experimentally demonstrate that the proposed method is effective on an optical microscope, fluorescence microscope, and scanning transmission electron microscope, and show that the reconstructed images are improved in terms of signal-to-noise ratio and spatial resolution. We believe that the proposed method could be applied to a wide range of low-dose imaging systems from biological to material science.
© 2023 Optica Publishing Group under the terms of the Optica Open Access Publishing Agreement
1. Introduction
Low-dose imaging technique is extensively applied in biological, clinical medicine, crystallography, and many other research fields [1–8]. Decreasing incident dose is a direct and feasible way to reduce or perhaps eliminate phototoxicity and radiation damage [9,10]. However, in general, image quality is proportional to the total incident dose. Under low-dose illumination conditions, Poisson and Gaussian mixed noise become the main factors influencing the signal-to-noise ratio and resolution of the image. Recently, a large number of studies is devoted to denoising the mixed Poisson–Gaussian noise [11–14]. Most denoising methods have to estimate the variance of the noise or require variance stabilization, but in practical applications, the variance of the noise is unknown or changeable. The signal-dependent Poisson noise that is neither additive nor multiplicative is harder to be removed [15], which is a challenge in the image processing community.
Deep learning, as a data-driven optimization method, has gained significant popularity in addressing the problem of image reconstruction in coherent diffraction, holograms, ghost imaging, image super-resolution and denoising [16–25]. One of the key limitations of the data-driven learning method is the need for a large data set with the corresponding high-quality ground truth images [26]. Obtaining sufficient numbers of ground truth images is costly, and in many practical applications, it is difficult to gain high-quality images (or ground truth images). For example, in computed tomography, the ground truth images can be captured by scanning with a high dose and over a long time [27]. In living cell imaging, it is infeasible to obtain high-quality images at a high dose illumination condition from samples that are sensitive to radiation or photo-toxicity [28]. To address the problem of insufficient ground truth images, deep learning methods for image reconstruction are developed through unsupervised learning with prior information. Currently, physics-driven learning methods have been successfully used for inverse problems to eliminate the need for ground truth images [29–31]. Noise2Noise method uses independent pairs of noisy images to successfully reconstruct high-quality images [32]. Acquisition of such pairs with independent and identically distributed noise is feasible for (quasi) static scenes [33], while it is difficult to acquire two noisy measurements of the same information in a dynamic process.
In this work, we developed a learning-based denoising method that incorporates the noise statistical model into a deep convolution network. The proposed method requires only pairs of noisy images without high-quality ground truth images, and the network parameters are optimized through the interplay between the neural network and the mathematical statistics model of noise. The idea of combining a mathematical statistics model to train a neural network is inspired by Noise2Noise [32], which enables training through noisy targets with zero-mean noise. Here, a Poisson and Gaussian mixed noise model by the quantum nature of light and thermal noise is used in our study, which has a non-zero mean. Numerical simulations with optical microscopy data under different illumination conditions and scanning transmission electron microscopy images with a low dose are performed to verify our proposed method. Based on the proposed method, we set up an optical microscope to capture two images with the same information but containing independent and identically distributed noise in a single shot. Specifically, two images that are split by a beam splitter can be simultaneously captured in different zones of a camera with a global shutter. Thus, based on the single shot measurement, a dynamic process can be reconstructed by the proposed method. A biological dynamic process of living cells under a low-dose illumination condition recorded and reconstructed using the proposed method. In addition, we also applied this method to image static samples on a fluorescence microscope and a scanning transmission electron microscope under low-dose conditions. Experimental results demonstrate that the proposed method can effectively improve the low-dose imaging quality in terms of signal-to-noise ratio, contrast, and spatial resolution, holding promise in biological and material science research.
The paper is organized as follows: Section 2 describes the formation of the denoising method. In section 3, we verify the proposed method using numerical simulations for optical microscopy and scanning transmission electron microscopy. In section 4, the optical experimental setup is described (Section 4.1). A proof-of-principle experiment is firstly carried out on a USAF target by an optical microscope (Section 4.2). Then, we apply the method to a biological dynamical process captured using an optical microscope (Section 4.3), static samples imaged by a fluorescence microscope (Section 4.4), and a scanning transmission electron microscope (Section 4.5). Section 5 concludes.
2. Methodology
The measurement is practically subject to Gaussian thermal and Poisson noise because of the photoelectric conversion process and the quantum nature of light [14,34], which can be expressed as:
where $S(p)$ denotes the noiseless image that is a function of $p$ (the photon flux in photons per pixel); $\mathrm {P}\{\sigma \}$ denotes a Poisson random variable with mean $\sigma$; the average operator $\left \langle {\cdot }\right \rangle$ is used as a normalization factor in the denominator. $\mathcal {N}$ denotes a Gaussian random variable with zero mean and variance $\mu ^{2}$, which is a signal-independent noise.A typical learning method to restore the high-quality image is to attempt to learn a mapping function $R$ (e.g., convolutional neural networks) from a large number of pairs, a noisy image $\hat {I}$ and the corresponding noiseless image $S(p)$ as the ground-truth, by solving
Instead, our learning method to restore the high-quality image by using one pair of noisy images is formulated as:
Fig. 1. (a). Flowchart of the training process with one pair of noisy images. (b). Architecture of the artificial neural network.
According to the statistical property of the noise model, the specific analysis can be described as follow:
Thus, after optimization, the mapping function $R_{\theta ^{*}}$ can be used to restore the noiseless image with the additional $\sigma$ that is the mean of a Poisson random variable:
The $\sigma$ does not affect the resolution of the restored image but increases the overall brightness and we can normalize the reconstructed result to reduce the effect of $\sigma$. Practically, the Poisson noise is neither additive nor multiplicative, but proportionally to light dose. The mean $\sigma$ of the Poisson random variable is not a constant under different illumination conditions and hardly estimates accurately. In our study, we need only ensure that each pair of images is under the same illumination condition without the need for a noise variance.
A number of options can be used as a mapping function $R$. In this study, $R$ is based on the U-Net architecture, as shown in Fig. 1(b). Details of the double convolution operation are shown in the dash line box, which contains 2*[2D convolution + Batch normalization + ReLU]. All 2D convolutions in double convolution block are carried with kernel size=3, stride=1 except the last one which has kernel size=1, stride=1. The 2D transposed convolution is performed with kernel size=2 and stride=2. And the 2D max pooling is performed with kernel size=2. The neural network is implemented using the Pytorch framework version 1.10 based on Python 3.7.8. on a desktop workstation (E5-2630 @ 2.20GHz CPU and NVIDIA GTX 3090 GPU).
3. Simulation
3.1 Numerical simulation on optical microscope
The optical microscope is the most commonly used imaging equipment in many areas of research including biological, life, and health sciences. However, phototoxicity can arise with visible light, infrared, and ultraviolet in biological imaging [37,38]. Imaging under low-dose conditions is helpful to reduce or perhaps eliminate phototoxicity. To verify the ability of the proposed method, numerical simulations are performed on Nuclei Segmentation In Microscope Cell Images dataset [35]. 220 different images are used, 200 images for training and 20 images for testing. Pairs of noisy images ($\hat {I}$) are generated in accordance with Eq. (1). The raw images from the dataset are cut into $512\times 512$ pixels and normalized to $0-255$ with a uint8 format. We take three illumination conditions, i.e., $p=100$ photons/pixel, $p=500$ photons/pixel, and $p=5000$ photons/pixel, as examples to examine the performance. The noiseless images ($S(p)$) under different illumination conditions are presented in Fig. 2. When the camera is fixed bit depth or dynamic range, one can clearly see that the noiseless images have obvious gray-level steps under low-dose conditions, as shown in Fig. 2(a1) and (b1). Specifically, when $p=100$ photons/pix, the noiseless image has only 4 grayscale levels from 0 to 3 plotted in Fig. 2(a2). Even without noise, the details, and contrast of the image are reduced with photon flux decreasing. Due to the Poisson noise and Gaussian noise, the quality of the image gets even worse under low-dose illumination conditions. The noisy images under different illumination conditions with the Poisson noise and Gaussian noise are shown in Fig. 3(a1), (b1), and (c1). The Poisson noise is generated by the "imnoise" Matlab function based on the magnitude $\lambda$ of noiseless images with uint8 format and the Gaussian noise is added with zero-mean and variance $\mu ^2 = 0.1\times max(\hat {I})$. From the results, one can see that as the photon flux decreases, the image noise will increase dramatically resulting in a resolution decline. When the photon flux $p=100$ photons/pix, the two nuclei are hardly distinguished, as shown in the enlarged red box in Fig. 3(a1). To reconstruct noisy images with different illumination conditions, we trained different networks for each illumination condition. Figure 3(a2), (b2), and (c2) show the reconstruction results, which demonstrate that the noise is suppressed effectively, even under a low-dose illumination condition. From the enlarged red box in Fig. 3(a2), (b2), and (c2), the two nuclei are easy to distinguish under different illumination conditions. The cross-sections are highlighted by lines of noisy images and reconstructed images, as shown in Fig. 3(a3), (b3), and (c3), indicating that the reconstructed results are relatively smooth and close to the ground truth. Specifically, the proposed method has better performance in large value positions as shown in Figs. 3(a3), (b3), and (c3) at pixels 0-5, 20-28, and 50-55. However, at some very low signal positions, such as at pixels 10-20, the reconstruction values are lower than the ground truth. Note that the proposed method is based on a statistical model and the noise is increased with the reduction of the signal. Consequently, with only one pair of images, a very low signal with high noise results in reconstruction performance degradation.
Fig. 3. Reconstruction results of numerical simulation optical microscope data under different illumination conditions. (a1), (b1), and (c1) are the noisy images as the network input. (a2), (b2), and (c2) are the reconstructed images by the network. (a3), (b3), and (c3) are the intensity line profile analysis of the noisy image, reconstructed image, and ground truth.
Figure 4. (a1), (b1), and (c1) show the loss curves of the training and testing datasets. The peak signal-to-noise ratio (PSNR) [39] values of training and testing datasets in the optimization process for each illumination condition. And the detailed information of the testing datasets and network output results are summarized in Table 1. One can see that the PSNR values of the reconstructed images are improved effectively compared to the input noisy images. Our numerical simulation results verify that the proposed method can reconstruct a high-quality image from the noisy images based on the optimization process in Eq. (4), and reflect the robustness of the proposed method with different illumination conditions.
Fig. 4. Loss curves and PSNR values of training and testing dataset under different illumination conditions during the training process.
Table 1. Detailed information of the testing dataset and reconstructed results.a
3.2 Numerical simulation on scanning transmission electron microscopy imaging
Low-dose imaging is widely used in electron microscopy, which can prevent radiation-induced damage to samples [9,40,41]. Low signal-to-noise ratio is the major problem for low-dose imaging. Compared with optical microscopes, it is much more time-consuming to obtain large numbers of electron microscopy images for training [42]. Therefore, in this section, we only use one pair of simulation images to train the network.
To validate the denoising capability of the proposed method under a low-dose condition in scanning transmission electron microscopy (STEM) imaging, we performed STEM image simulation on a monolayer tungsten disulfide ($WS_2$) with defects and dopants. One pair of high-angle annular dark field (HAADF) images is generated using the software package abTEM [43]. The corresponding ground truth images are generated under a high-dose condition, i.e., 1e8 $e^{-}/$Å2, without noise, as shown in Fig. 5(a). And the pair of images, as the network input, is generated under a low-dose condition, i.e., 1e3 $e^{-}/$Å2, with Poisson and Gaussian noise ($\mu ^2=5$). One of the noisy images is shown in Fig. 5(b). Compared to the ground-truth images, the S columns in the low-dose images can hardly be visible, buried in the background noise owing to the low-dose condition. After training, the network output result is shown in Fig. 5(c). From the intensity line profile analysis in Fig. 5(d), the signal of the sample is reconstructed with high quality, and the noise is effectively suppressed. We use the PSNR value to measure the quality of the reconstructed image. The PSNR between the reconstructed images and the ground truth is 34.08 dB, while the corresponding value associated with the input image is 15.94 dB. The real parts of Fourier transformations of the ground truth, network input and network output are shown in Fig. 5(a1), (b1), and (c1), respectively. It can also be seen that the Fourier transform of the network output shows stronger spots, similar to that of the ground truth image. The corresponding spatial resolution for the ground truth and network output is 0.092 nm, and for the network input is 0.103 nm. In general, we can intuitively see that the results of PSNR evaluation and Fourier transformations of the network output image are superior to the noisy input image.
Fig. 5. Denoising results of high-angle annular dark field imaging on the scanning transmission electron microscope. (a) is the ground truth imaging under dose = 1e8 $e^{-}/$Å2 without noise. (b) is the network input image under dose = 1e3 $e^{-}/$Å2 with Poisson and Gaussian mixed noise. (c) is the network reconstruction result. (a1), (b1), and (c1) are the Fourier transformations of (a), (b), and (c), respectively. (d) is the intensity line profile analysis highlighted in (a), (b), and (c).
4. Experiments and analysis of results
4.1 Optical experiment setup
The proposed method requires two independent and identically distributed noisy images with the same information, which are hardly captured in a dynamic sense. We designed a feasible light path in an optical microscope, which is able to capture two noisy images in one single shot. The two images have independent and identically distributed noise because they are distributed in different areas on one sensor and Poisson statistics is the quantum nature of light [34]. The optical experiment setup is described in Fig. 6. An illumination beam emitted from a LED white-light source with a continuously variable power of 0W-3W is first condensed by a microscope condenser and then illuminated on a sample. We use a 10$\times$ objective lens with a numerical aperture (NA) of 0.25 to magnify the sample image. These processes described above are set up based on an optical microscope, as shown in the dashed box in Fig. 6(b). Then, two beamsplitters (splitting ratio of 50:50) and two mirrors are used to split the image into two. Finally, using different areas of a CMOS camera (Basler, a2A4504-18umPRO, $2.74\;\mu m$ pixel size) record the two images in one single shot. It means that we can get two images with the independent and identically distributed noise of the same scene at the same time by a CMOS with a global shutter, even if the observed object is moving. An example of the captured raw image is shown in Fig. 6(c). Finally, using the cross-correlation algorithm [44] that is able to ensure the alignment of images at the sub-pixel level, the raw image is cut into two $512\times 512$ images, ${\hat {I}^{\prime }}$ and ${\hat {I}^{\prime \prime }}$, for the network training.
Fig. 6. (a) is the schematic layout of the experimental geometry. (b) is the optical experiment setup based on an optical microscope. (c) is an example of one frame captured by the camera. LED: light emitting diode, MC: microscope condenser, S: sample, OL: objective lens, M: mirror, BS: beamsplitter, CMOS: complementary metal oxide semiconductor.
The advantages of the experimental setup for dynamic process data collection are obvious. Firstly, the beamsplitters can assure that the two images are generated from one signal. Secondly, only one global shutter camera is used for capturing the two images in one frame, which ensures the complete synchronization of the two images. Thus, each frame of the dynamic process contains two images with independent identically distributed noise. According to Eq. (4), the proposed method can be used to reconstruct a dynamic process based on the collected data.
4.2 USAF resolution target on optical microscope
As a proof-of-principle experiment to verify the performance of the proposed method and model on optical microscope experiment data, we use a USAF resolution target as a sample and collect 3 groups of data under different illumination conditions, $p= 100,\; 500,\; 5000$ photons/pixel. For each illumination condition, one pair of images are collected as the training data set. The reconstructed results after 200 epochs of training are shown in Fig. 7. With the decrease of light dose, the noise increases obviously, as presented in Fig. 7(a1), (b1), and (c1). When $p=100$ photons/pix, the line pair 1, group 9 is recognizable in the reconstruction result, as shown in Fig. 7(a2), and the corresponding frequency and line width are 512 $mm^{-1}$ and 0.98 $\mu m$ respectively. In order to investigate the improvement of the PSNR, the relative standard deviation (RSD) of the yellow box in Fig. 7 is calculated as follows:
where $x_i$ is the pixel value; n is the total pixel number in the yellow box; $\bar {x}$ is the corresponding average value. The RSD results are illustrated in Table 2. The RSD values of the network input images decrease as the photon flux increases. Compared with the network input images, all the cross-section lines of reconstructed images are relatively smooth with different illumination conditions, as shown in Fig. 7(a3), (b3), and (c3). Those results demonstrate that our method is robust for different illumination conditions and can suppress the noise in experimental data.Fig. 7. Experimental reconstructions of the USAF resolution target under different illumination conditions.(a1), (b2), and (c1) are the network input images ${\hat {I}^{\prime }}$ and the three-dimensional view of the section marked by the pink box after normalization. (a2), (b2), and (c2) are the reconstructed images $R_{\theta ^{*}}(\hat {I}^{\prime })$ and the three-dimensional view of the section marked by the red box after normalization. (a3), (b3), and (c3) are the intensity line profile analysis of the raw images and reconstructed images. The white scale bar in a1 is 5 $\mu m$. The yellow box of $100\times 100$ pix is used to calculate the RSD value.
Table 2. The RSD value of the yellow box in Fig. 7
4.3 Dynamic process of living cells on optical microscope
In the natural environment, living cells are rarely exposed to a high dose of illumination. To image the living cells in situ and avoid photo-toxicity, decreasing the incident dose is a feasible way to explore the dynamic processes of living cells. Here, we take a flow of chloroplast of black algae as a living biological dynamic process example under two different illuminations to confirm the performance of the proposed method. A time series of data is collected in 60 seconds with 24 FPS, a total of 1440 pairs of raw data. The dynamic process data are continuously changed and similar in each frame. Thus, only part of the data is used to train the network, and using the trained network reconstructs the whole dynamic process due to the generalization ability of the learning method. In this study, 120 pairs of raw images with 0.5 seconds interval are selected to train the network, and then using the trained network to reconstruct the dynamic process. After 200 epochs of training about 12 mins with batch size 20, part of the results is shown in Fig. 8. The reconstruction results of the whole dynamic process in Visualization 1 and Visualization 2.
Fig. 8. Part reconstruction results of the living cell dynamic process under different illuminations. (a) The photon flux in photons per pixel $p=500$ photons/pixel. (b) The photon flux in photons per pixel $p=5000$ photons/pixel.
4.4 Fluorescence microscope
Fluorescence microscope is a basic tool for biological research. In order to obtain a high-quality fluorescence image, a considerable dose of light is needed to excite fluorescence light because the fluorescence process itself is inefficient [28]. However, strong light is often harmful to biological cells. We next train a network with pairs of fluorescence microscopy data under a low-light condition to test whether the proposed method is effective on fluorescent acquisitions.
Figure 9 shows two examples of the reconstruction results of the fluorescence microscope. Each example is trained with one pair of noisy images. From the enlarged region of the yellow box in Fig. 9, the network output images show more details of the Spinach Leaf slice than the input images. It can be seen that the quality of the image is improved and the noise is suppressed. In order to further quantify the effect of denoising, we selected the blue box region of the black background in Fig. 9(a) to calculate their standard deviation values. The standard deviation value of the network reconstruction image is 0.4, whereas the corresponding value of the noisy image is 14.4. The reconstruction results show that our method considerably reduces the level of noise.
Fig. 9. Two reconstruction results of fluorescence microscope data under a low-light dose condition. The second and fourth columns ( from left to right ) are enlarged images of the yellow box in the first and third columns. The blue box in the first column is used to calculate the standard deviation value.
4.5 Scanning transmission electron microscope
To verify the performance of our method for experimental STEM imaging, one pair of noisy images of $WS_2$ is collected under a low-dose condition. In STEM experiment, it is difficult to capture two noisy images in one single shot. Fortunately, most samples in material science can be considered static or quasi-static during imaging. Thus, the two noisy images are obtained by sequential imaging at the same sample region. The images were acquired using a Nion HERMES-100 SETM with a focused electron-probe of $\sim 1 $ Å in diameter after C3/C5 correction. The accelerating voltage was 60 kV and the beam current was $\sim 1\;pA$. An annular dark-field (ADF) detector was used with an inner cutoff angle of $\sim 44$ mrad, corresponding to medium-angle annular dark-field (MAADF) imaging. The pixel size is 0.039 Å × 0.039 Å with a pixel dwell time of $1 \;\mu s/pixel$ corresponding to a radiation dose of $\sim 4.1e3\;e^{-}/$Å2.
Figure 10(a1) shows the STEM-MAADF image of $WS_2$ with a dosage of $\sim 4.1e3\;e^{-}/$Å2. It is hard to visualize the S sites from the raw image, which is consistent with the numerical simulation in section 3.2. After 300 epochs of training for about $21\;s$ using one pair of noisy images, the reconstructed result is shown in Fig. 10(b1). The network reconstructed image appears much sharper than the raw image, and the S atomic-sites can now be clearly visible. This difference is illustrated by the corresponding Fourier transforms shown in Fig. 10(a2) and (b2) where the spots visible in the reconstructed result transform are stronger and expand to higher frequency. The corresponding spatial resolution of the network input image and the network output image is $0.137\;$Å and $0.092\;$Å, respectively. These results demonstrate that the quality of the network output image is significantly improved in both spatial resolution and contrast as compared with the as-captured low-dose image.
Fig. 10. Reconstruction results by one pair of experimental STEM-MAADF images.(a1) is the network input image. (a2) is the real part of the Fourier transform of (a1). (b1) is the reconstruction result and (b2) is the corresponding Fourier transform of (b1).
5. Conclusion
We have developed an unsupervised denoising method for low-dose imaging on light microscopes and electron microscopes. By introducing the noise statistical model to the loss function of a deep learning method, the proposed method can reconstruct a high-quality image by using one pair of noisy images. Numerical simulation results show that the proposed method has robust performance under different illumination conditions. For experimental demonstration, we set up a feasible light path in an optical microscope to capture pairs of noisy images for a dynamic process in single-shots, which makes it possible to reconstruct a high-quality biological dynamic process under a low-dose condition using our proposed method. We have demonstrated the proposed methods with experimental data in a dynamic process of an optical microscope and a static sample in a fluorescence microscope. Furthermore, we demonstrate that the proposed method can be applicable to experimental STEM low-dose imaging, where the reconstructed STEM-MAADF image from one pair of noisy images shows a clear improvement in spatial resolution and image contrast. The proposed denoising method should be applicable to diverse imaging systems under a low-dose illumination condition, opening exciting new possibilities for imaging dynamical processes of sensitive samples.
Funding
National Key Research and Development Program of China (2021YFB3602604); National Natural Science Foundation of China (61975205, 62075221, 62131011); Fusion Foundation of Research and Education of CAS; University of Chinese Academy of Sciences; Fundamental Research Funds for the Central Universities; Funded Project of Hebei Province Innovation Capability Improvement Plan, China (20540302D).
Disclosures
The authors declare no conflicts of interest.
Data availability
Data underlying the results presented in this paper are not publicly available at this time but may be obtained from the authors upon reasonable request.
References
1. D. Ardila, A. P. Kiraly, S. Bharadwaj, B. Choi, J. J. Reicher, L. Peng, D. Tse, M. Etemadi, W. Ye, G. Corrado, D. P. Naidich, and S. Shravya, “End-to-end lung cancer screening with three-dimensional deep learning on low-dose chest computed tomography,” Nat. Med. 25(6), 954–961 (2019). [CrossRef]
2. I. Shiri, A. Akhavanallaf, A. Sanaat, Y. Salimi, D. Askari, Z. Mansouri, S. P. Shayesteh, M. Hasanian, K. Rezaei-Kalantari, A. Salahshour, S. Sandoughdaran, H. Abdollahi, H. Arabi, and H. Zaidi, “Ultra-low-dose chest ct imaging of covid-19 patients using a deep residual neural network,” Eur. Radiol. 31(3), 1420–1431 (2021). [CrossRef]
3. H. Chen, Y. Zhang, M. K. Kalra, F. Lin, Y. Chen, P. Liao, J. Zhou, and G. Wang, “Low-dose ct with a residual encoder-decoder convolutional neural network,” IEEE Trans. Med. Imaging 36(12), 2524–2535 (2017). [CrossRef]
4. J. Bland-Hawthorn, J. Bryant, G. Robertson, P. Gillingham, J. O’Byrne, G. Cecil, R. Haynes, S. Croom, S. Ellis, M. Maack, P. Skovgaard, and D. Noordegraaf, “Hexabundles: imaging fiber arrays for low-light astronomical applications,” Opt. Express 19(3), 2649 (2011). [CrossRef]
5. M. M. Marim, E. Angelini, J. C. Olivo-Marin, and M. Atlan, “Off-axis compressed holographic microscopy in low-light conditions,” Opt. Lett. 36(1), 79 (2011). [CrossRef]
6. J. M. Wolterink, T. Leiner, M. A. Viergever, and I. Išgum, “Generative adversarial networks for noise reduction in low-dose ct,” IEEE Trans. Med. Imaging 36(12), 2536–2545 (2017). [CrossRef]
7. G. R. Small, B. J. Chow, and T. D. Ruddy, “Low-dose cardiac imaging: reducing exposure but not accuracy,” Expert Review of Cardiovascular Therapy 10(1), 89–104 (2012). [CrossRef]
8. F. Arfelli, M. Assante, V. Bonvicini, A. Bravin, G. Cantatore, E. Castelli, L. D. Palma, M. D. Michiel, R. Longo, A. Olivo, S. Pani, D. Pontoni, P. Poropat, M. Prest, A. Rashevsky, G. Tromba, A. Vacchi, E. Vallazza, and F. Zanconati, “Low-dose phase contrast x-ray medical imaging,” Phys. Med. Biol. 43(10), 2845–2852 (1998). [CrossRef]
9. J. P. Buban, Q. Ramasse, B. Gipson, N. D. Browning, and H. Stahlberg, “High-resolution low-dose scanning transmission electron microscopy,” J. Electron Microsc. 59(2), 103–112 (2010). [CrossRef]
10. F. Mubaid and C. M. Brown, “Less is more: longer exposure times with low light intensity is less photo-toxic,” Microsc. Today 25(6), 26–35 (2017). [CrossRef]
11. D. L. Donoho, “Nonlinear wavelet methods for recovery of signals, densities, and spectra from indirect and noisy data,” in In Proceedings of Symposia in Applied Mathematics, (Citeseer, 1993).
12. F. Murtagh, J.-L. Starck, and A. Bijaoui, “Image restoration with noise suppression using a multiresolution support,” Astron. Astrophys. 112, 179 (1995).
13. K. Dabov, A. Foi, V. Katkovnik, and K. Egiazarian, “Image denoising by sparse 3-d transform-domain collaborative filtering,” IEEE Trans. on Image Process. 16(8), 2080–2095 (2007). [CrossRef]
14. H. Talbot, H. Phelippeau, M. Akil, and S. Bara, “Efficient poisson denoising for photography,” in 2009 16th IEEE International Conference on Image Processing (ICIP), (IEEE, 2009), pp. 3881–3884.
15. F. Luisier, T. Blu, and M. Unser, “Image denoising in mixed poisson–gaussian noise,” IEEE Trans. on Image Process. 20(3), 696–708 (2011). [CrossRef]
16. A. Sinha, J. Lee, S. Li, and G. Barbastathis, “Lensless computational imaging through deep learning,” Optica 4(9), 1117–1125 (2017). [CrossRef]
17. D. Yang, J. Zhang, Y. Tao, W. Lv, S. Lu, H. Chen, W. Xu, and Y. Shi, “Dynamic coherent diffractive imaging with a physics-driven untrained learning method,” Opt. Express 29(20), 31426–31442 (2021). [CrossRef]
18. L. Shi, B. Li, C. Kim, P. Kellnhofer, and W. Matusik, “Towards real-time photorealistic 3d holography with deep neural networks,” Nature 591(7849), 234–239 (2021). [CrossRef]
19. S. Guo, Z. Yan, K. Zhang, W. Zuo, and L. Zhang, “Toward convolutional blind denoising of real photographs,” in Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, (2019), pp. 1712–1722.
20. F. Wang, C. Wang, M. Chen, W. Gong, Y. Zhang, S. Han, and G. Situ, “Far-field super-resolution ghost imaging with a deep neural network constraint,” Light: Sci. Appl. 11(1), 1–11 (2022). [CrossRef]
21. C. Dong, C. C. Loy, K. He, and X. Tang, “Image super-resolution using deep convolutional networks,” IEEE Trans. Pattern Anal. Mach. Intell. 38(2), 295–307 (2016). [CrossRef]
22. H. Wang, Y. Rivenson, Y. Jin, Z. Wei, R. Gao, H. Günaydın, L. A. Bentolila, C. Kural, and A. Ozcan, “Deep learning enables cross-modality super-resolution in fluorescence microscopy,” Nat. Methods 16(1), 103–110 (2019). [CrossRef]
23. I. Kang, F. Zhang, and G. Barbastathis, “Phase extraction neural network (phenn) with coherent modulation imaging (cmi) for phase retrieval at low photon counts,” Opt. Express 28(15), 21578–21600 (2020). [CrossRef]
24. K. Wang, Q. Kemao, J. Di, and J. Zhao, “Deep learning spatial phase unwrapping: a comparative review,” Adv. Photonics Nexus 1(1), 014001 (2022). [CrossRef]
25. C. Zuo, J. Qian, S. Feng, W. Yin, Y. Li, P. Fan, J. Han, K. Qian, and Q. Chen, “Deep learning in optical metrology: a review,” Light: Sci. Appl. 11, 1–54 (2022). [CrossRef]
26. G. Wang, J. C. Ye, and B. De Man, “Deep learning for tomographic image reconstruction,” Nat. Mach. Intell. 2(12), 737–748 (2020). [CrossRef]
27. J. Ma, J. Huang, Q. Feng, H. Zhang, H. Lu, Z. Liang, and W. Chen, “Low-dose computed tomography image restoration using previous normal-dose scan,” Med. Phys. 38(10), 5713–5731 (2011). [CrossRef]
28. V. Magidson and A. Khodjakov, Chapter 23 - Circumventing Photodamage in Live-Cell Microscopy, vol. 114 of Methods in Cell Biology (Academic Press, 2013).
29. F. Wang, Y. Bian, H. Wang, M. Lyu, G. Pedrini, W. Osten, G. Barbastathis, and G. Situ, “Phase imaging with an untrained neural network,” Light: Sci. Appl. 9(1), 77 (2020). [CrossRef]
30. F. Wang, C. Wang, C. Deng, S. Han, and G. Situ, “Single-pixel imaging using physics enhanced deep learning,” Photonics Res. 10(1), 104–110 (2022). [CrossRef]
31. R. Li, G. Pedrini, Z. Huang, S. Reichelt, and L. Cao, “Physics-enhanced neural network for phase retrieval from two diffraction patterns,” Opt. Express 30(18), 32680–32692 (2022). [CrossRef]
32. J. Lehtinen, J. Munkberg, J. Hasselgren, S. Laine, T. Karras, M. Aittala, and T. Aila, “Noise2noise: Learning image restoration without clean data,” arXiv, arXiv:1803.04189 (2018). [CrossRef]
33. A. Krull, T.-O. Buchholz, and F. Jug, “Noise2void-learning denoising from single noisy images,” in Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, (2019), pp. 2129–2137.
34. M. Deng, S. Li, A. Goy, I. Kang, and G. Barbastathis, “Learning to synthesize: robust phase retrieval at low photon counts,” Light: Sci. Appl. 9(1), 36 (2020). [CrossRef]
35. J. Benesty, J. Chen, Y. Huang, and I. Cohen, “Pearson correlation coefficient,” in Noise reduction in speech processing, (Springer, 2009), pp. 1–4.
36. O. Ronneberger, P. Fischer, and T. Brox, “U-net: Convolutional networks for biomedical image segmentation,” in International Conference on Medical image computing and computer-assisted intervention, (Springer, 2015), pp. 234–241.
37. D. I. Pattison and M. J. Davies, “Actions of ultraviolet light on cellular structures,” EXS 6(96), 131–157 (2006). [CrossRef]
38. R. Hoebe, C. Van Oven, T. W. Gadella, P. Dhonukshe, C. Van Noorden, and E. Manders, “Controlled light-exposure microscopy reduces photobleaching and phototoxicity in fluorescence live-cell imaging,” Nat. Biotechnol. 25(2), 249–253 (2007). [CrossRef]
39. P. Gupta, P. Srivastava, S. Bhardwaj, and V. Bhateja, “A modified psnr metric based on hvs for quality assessment of color images,” in 2011 International Conference on Communication and Industrial Application, (IEEE, 2011), pp. 1–4.
40. G. McMullan, A. Clark, R. Turchetta, and A. Faruqi, “Enhanced imaging in low dose electron microscopy using electron counting,” Ultramicroscopy 109(12), 1411–1416 (2009). [CrossRef]
41. D. B. Carlson, J. E. Evans, and K. Maaz, “Low-dose imaging techniques for transmission electron microscopy,” The Trans. Elect. Microscope 95, 85–98 (2012). [CrossRef]
42. S. J. Pennycook and P. D. Nellist, Scanning transmission electron microscopy: imaging and analysis (Springer Science & Business Media, 2011).
43. J. Madsen and T. Susi, “The abtem code: transmission electron microscopy from first principles,” Open Research Europe 1, 24 (2021). [CrossRef]
44. M. Guizar-Sicairos, S. T. Thurman, and J. R. Fienup, “Efficient subpixel image registration algorithms,” Opt. Lett. 33(2), 156–158 (2008). [CrossRef]
