In the rapidly evolving field of computational imaging, ptychography has emerged as a powerful technique capable of producing high-resolution phase and amplitude images from diffraction patterns. It involves translating a thin object through overlapping illuminations and measuring the resulting diffraction patterns behind the object with a camera sensor [1,2]. Subsequently, the complex-valued image is constructed through an iterative optimization algorithm, necessitating the formulation and minimization of a loss function, alternatively referred to as the objective, cost, or error function. Ptychography has found applications in a wide range of topics, including label-free biological imaging [3–5], optical metrology [6–10], and atomic-resolution imaging using electron beams [11–13].

Fundamentally, the basis of ptychographic reconstructions is the detection of photon counts on a camera sensor and, therefore, subject to Poissonian noise even under ideal measurement conditions. Given the assumption of an underlying noise model, a powerful and robust optimization strategy is the maximum-likelihood estimation (MLE) principle [14]. By leveraging MLE in ptychography, one seeks to estimate the studied object parameters that render the observed diffraction patterns most probable [15–19]. However, an additive camera readout noise is often neglected, thus leaving a gap in the fidelity of reconstructions. This is a salient concern as the presence of readout noise, typically Gaussian, is an important element of practical ptychographic measurements within the visible spectrum. Ignoring this noise source oversimplifies the underlying statistical model and introduces errors to the reconstructed image, especially when the detected photon counts and the signal-to-noise ratio (SNR) are low. The distinction between a Poissonian and a mixed Poisson–Gaussian noise model is depicted in the simulated images presented in Fig. 1.

 

Fig. 1. Comparative visualization of a grayscale image distorted by different noise types. Panel (A) depicts an image with simulated Poissonian noise, while panel (B) illustrates the effects of simulated mixed Poisson–Gaussian noise resulting from additive readout noise. Inlaid values indicate peak signal-to-noise ratio (PSNR) with respect to the ground truth.

Download Full Size | PDF

In this Letter, we propose a loss function for automatic differentiation ptychography that explicitly incorporates both Poissonian and Gaussian noise sources. This approach brings us closer to the real-world conditions of ptychographic measurements, thereby paving the way for superior performance in image reconstruction under challenging noise conditions. We outline a practical method to incorporate camera readout noise in computational imaging. Furthermore, we provide a comprehensive comparison between the image reconstruction quality using a mixed-statistics loss function and that of a conventional loss function which presumes solely Poissonian noise statistics. For this, we present reconstruction results obtained from both experimental and numerical data.

In ptychography, the typical reconstruction approach involves minimizing a loss function representing the difference between the intensity values of the observed diffraction pattern $X_k$ and the anticipated diffraction patterns $I_k(\boldsymbol{\theta })$ as determined by a parameter set $\boldsymbol{\theta }$, which embodies the object under investigation, at all pixel locations indexed by $k$. In the presence of measurement noise, it is insightful to tackle the problem of ptychographic reconstruction by maximizing the likelihood of the observed given the object parameters. From this probabilistic perspective, one seeks the object parameters that make the observed data most likely, which renders object retrieval more robust in scenarios of low SNR.

In the Supplement 1 (section 1), we elaborate on deriving the maximum-likelihood estimation (MLE) loss functions for two different types of noise statistics. When operating under the assumption of Poissonian counting noise, the loss function $L_{\mathrm {Poisson}}$ that yields the maximum-likelihood estimate is expressed as

To validate the beneficial effect of the loss function $L_{\mathrm {Mixed}}(\boldsymbol{\theta })$ experimentally, we are considering a standard ptychography setup in a transmission geometry (Fig. 2). A circular 500-µm pinhole is illuminated with coherent light with a wavelength of 561 nm and relayed to the sample plane using two lenses with a magnification of $M=3$. Therefore, a binary target sample is illuminated at 80 scanning positions with an overlap of approximately 60 % between adjacent positions, which lies within the ideal regime according to [20]. The scattered light is captured by a CMOS camera positioned 38 mm away from the object. To ensure a comprehensive comparison as a function of SNR, we employ four different exposure settings per scanning position, spanning a range from 30 µs to 300 ms, with each subsequent exposure time differing by a factor of 10 (see Fig. 3). For each exposure time, we determine the variances $\sigma _k^2$ by capturing a stack of 300 dark images. This quantifies the readout noise level associated with each pixel $k$ of our camera. A comprehensive overview of further details about the experimental implementation and methodology can be found in Section 2 of Supplement 1.

 

Fig. 2. Schematic drawing of our ptychography setup used in the experiment and for numerical simulations. A 500-µm pinhole is illuminated and relayed onto the object using a 2-lens system. The object is shifted laterally through the beam using an XY stage. A CMOS camera records the diffraction intensities at a distance of 38 mm downstream of the object.

Download Full Size | PDF

 

Fig. 3. Comparison of image reconstruction qualities from ptychographic datasets with decreasing signal-to-noise ratio (SNR) from left to right. Top row: Ptychographic datasets at different camera exposures with total illumination intensity, respectively. Second row: Reconstructions using a loss function assuming solely Poissonian noise statistics. Third row: Reconstructions using the loss function defined by Eq. (2), which incorporates mixed Poisson–Gaussian noise statistics. The edge length of every image equals 3.5 mm.

Download Full Size | PDF

By maintaining a constant illumination power, we acquire four distinct ptychographic datasets, each corresponding to a different signal-to-noise ratio (SNR), as demonstrated in the top row of Fig. 3. With these datasets in hand, we proceed to perform image reconstructions utilizing two different loss functions (Eqs. (1) and (2)) within an automatic differentiation-based ptychography framework, as detailed in [21] and similar to [22]. For this analysis, we precalibrate the illumination field using an additional high SNR measurement and restrict our optimizations to the complex-valued object transmission functions, particularly under decreasing SNR conditions. This approach enables us to isolate the impact of the loss function choice from the convergence behavior associated with an unknown illumination field. The reconstruction procedure is explained in more detail in Section 3 of Supplement 1, and the source code and raw data are available in [23] under open licenses. In scenarios characterized by a high SNR, no noticeable disparity in the image quality is observed between the two approaches. However, when confronted with low-SNR conditions, where the signal becomes immersed within the readout noise, the advantages of the mixed-statistics MLE loss function become clear. The reconstructed images exhibit superior quality and reveal finer details that would otherwise remain obscured without accounting for the readout noise statistics.

For a quantitative analysis and validation of our experimental findings, we generate a simulated object with phase and amplitude contrasts which we treat as the ground truth. Using numerical simulation and the ground truth object, we compute the noisy diffraction patterns assuming Poissonian photon count statistics and an additive Gaussian readout noise with a standard deviation of $\sigma = {1.5}\,\textrm{counts}$. The simulation allows for varying the illumination intensity given as the total number of photons in an otherwise fixed illumination field that approximates the experimental conditions shown in Fig. 3, thereby controlling the measurement SNR.

The achieved reconstruction quality of the object $O$ with respect to the ground truth $O_\mathrm {gt}$ can now be quantified using the correlation coefficient $C = \frac {{\left | \left \langle \overline {O}_\mathrm {gt}, O \right \rangle \right |}}{{\|O_\mathrm {gt}\| \cdot \|O\|}}$ as defined and motivated in [21] as a function of illumination intensity (Fig. 4). Here, $\overline {O}_\mathrm {gt}$ denotes the complex conjugate of $O_\mathrm {gt}$, $\left \langle \cdot, \cdot \right \rangle$ denotes the dot product, and $\|\cdot \|$ denotes the norm. The simulation confirms the same trend that we observe from the experiments: for low SNR, optimization using a mixed-statistics loss function yields significantly better reconstruction results. In the regime of high illumination intensities, all loss functions converge excellently (up to machine precision) as the readout noise becomes irrelevant, and all assumed underlying probability density functions become valid approximations. Image reconstruction using a Gaussian loss function $L_{\mathrm {Gaussian}}(\boldsymbol{\theta })=\sum _{k=1}^N\left ( X_k- I_k(\boldsymbol{\theta })\right )^2$ performs worst at low illumination intensities. However, introducing a weighting term as motivated in [24] leads to a noteworthy improvement (shown in red) using $L_{\mathrm {normMSE}}(\boldsymbol{\theta })=\sum _{k=1}^N\left ( \frac {X_k- I_k(\boldsymbol{\theta })}{\mathrm {sg}[I_k(\boldsymbol{\theta })] + \epsilon }\right )^2$, where $\epsilon =10^{-3}$ and $\mathrm {sg}[\cdot ]$ indicates a stop-gradient function. This normalized MSE loss function can be interesting in cases where $\sigma _k^2$ is impractical to obtain.

 

Fig. 4. Correlation between ground truth and reconstructions as a function of the total photon count in the illumination field based on numerical simulations. The dashed blue line represents reconstructions derived from intensity data where negative values have been zero-cropped, while the solid line represents reconstructions that incorporate negative values, which can arise due to the additive Gaussian noise component. (A) Amplitude and phase contrast reconstruction of the simulated object based on high-intensity diffraction patterns. For comparison, with an illumination intensity of 3.4×10⁵ photons the reconstructions achieved with $L_\mathrm {Mixed}$ (B) and $L_\mathrm {Poisson}$ (C) are shown. The edge length of every image equals 3.5 mm.

Download Full Size | PDF

The intensity readout at a given pixel may be negative due to the additive Gaussian component in the noise statistics. This can occur in practice in an experiment via background subtraction, when areas on the camera sensor detect only low intensities. As a practical measure to keep the loss function real-valued when calculating the square root of intensity for the Poissonian loss function, negative-intensity values are customarily forced to zero [25,26]. Hence, in both simulation and experimental scenarios, we assign zero to negative-intensity values when optimizing $L_{\mathrm {Poisson}}$. However, by zero-cropping the intensity data, we may inadvertently eliminate valuable information, thereby causing a potential bias in our reconstruction results. To examine this bias and quantify the potential information contained within negative values, we also test $L_{\mathrm {Mixed}}$ with zero-cropped data, as shown in Fig. 4. The quality of reconstruction in these conditions falls between the results from optimizing $L_{\mathrm {Poisson}}$ with zero-cropped data and $L_{\mathrm {Mixed}}$ with unaltered data. This observation suggests that the enhanced reconstruction quality derived from using a mixed-statistics loss function can partially be credited to the statistical information encapsulated in the negative pixel values resulting from background subtraction.

The results presented in this study underscore the importance of considering mixed Poisson–Gaussian noise statistics in ptychographic image reconstruction. We have demonstrated, through both experimental and simulated data, that using an MLE loss function that considers this mix of noise statistics improves the reconstruction quality, particularly in low signal-to-noise ratio conditions. This enhanced performance indicates that the mixed-statistics loss function can extract more information from the measured data by accurately accounting for the underlying noise statistics. An interesting outcome of this study concerns the practice of zero-cropping negative-intensity values. We find that this practice introduces a bias into the reconstructions, highlighting the importance of preserving all statistical information in the data.

It is worth noting that some types of detectors bypass the issue of a significant Gaussian readout noise, such as high-performance photon-counting hybrid pixel detectors notably used in x ray ptychography [27,28]. In such cases, optimizing $L_{\mathrm {Poisson}}(\boldsymbol{\theta })$ can yield excellent reconstruction results. Future research could focus on studying the convergence behavior of a mixed-statistics loss function when the illumination field is jointly optimized, as we have observed that optimizing $L_{\mathrm {Mixed}}(\boldsymbol{\theta })$ occasionally leads to a less reliable convergence when dealing with a poor initial estimate for the illumination field. To ensure valid comparison and to attain image retrieval under extremely ill-posed conditions, we included the additional step of precalibrating the illumination field in this study. Such a step is typically unnecessary in well-posed ptychographic reconstructions [29] or other approaches to noise-robust phase retrieval methods [30,31].

In summary, the findings presented here could potentially propel significant advancements in the field of computational imaging, leading to improved image retrieval under challenging noise conditions. By offering a more accurate reflection of real-world ptychographic measurements, a loss function that considers mixed Poisson–Gaussian noise statistics could greatly contribute to various fields, including material science, biology, and nanotechnology, where high-quality image reconstruction under low-SNR conditions is critical. Moreover, the utility of a mixed-statistics loss function is not just limited to ptychography but extends to many computational and gradient-based imaging methods, broadening its applicability [32–36].