1. Introduction

The electromagnetic field in visible light oscillates in the vicinity of 430–770 THz, so imaging detectors typically record the time averaged power at each pixel, i.e. intensity. Recovering the phase relies on the phase difference of the field between pixels being rendered as measurable intensity variations. Interferometry achieves this through the precise alignment of interfering beams and typically requires phase unwrapping of the collected data. However, propagation of light involves self-interference of a beam where its initial phase and intensity determine the spatial evolution of intensity. Propagation-based phase retrieval techniques solve an inverse problem to recover phase from measured intensities subject to a physical model of propagation [1–6]. In this work, the transport of intensity equation (TIE) is used for phase retrieval. The TIE assumes a short propagation distance to linearize the propagation model [7, 8]. The phase recovered using the TIE does not require unwrapping [9]. It is also simple to implement experimentally as it does not require the precsise alignment typical of interferometry. The intensity simply needs to be measured after propagation between two closely spaced planes in free space [10, 11]. This makes it an especially useful technique in regimes where high quality optical elements are not readily available, such as x–ray [3, 7], neutron beam [12] and electron beam [13,14] imaging.

Consider a paraxial, monochromatic field propagating in free space along the z axis. It obeys a paraxial wave equation of the form

The physics of Eq. (1) can also be represented in terms of propagation of the real–valued intensity and phase through a pair of coupled, nonlinear differential equations

A simple imaging system to characterize sample phase via the TIE consists of 4-f relay optics and a detector that can be moved through focus to collect under-focused, in-focus and over-focused intensities. When the sample is illuminated by a normally incident plane wave, the intensity at each defocused plane can be approximated, for small distance Δz from focus, by

Equation (5) offers a means of retrieving phase by solving this second-order, elliptic partial differential equation for ϕ(x, z) provided intensities at the two defocused planes are measured. Various strategies have been proposed in the literature to use additional defocused planes to better approximate the axial derivatives to improve the reconstruction of phase [15–19]. In this work, we examine the effect of the illumination source shape on the TIE and for simplicity use only two defocused planes for Eq. (5), although our results could be adapted in a straightforward manner to include additional defocused images.

A common technique for finding the phase from the TIE is to define an auxiliary function Θ [8] such that

Fourier transform methods allow these Poissons’s equations to be solved efficiently [20, 21] and will enable us to easily incorporate the effects of extended sources in what follows. Let ℱ and ℱ⁻¹ represent the Fourier and inverse Fourier transform integrals

The auxiliary function method involves the inversion of two Laplacians (division by H_L), which significantly amplifies low–frequency noise. Equation (5) implies that the intensity variations measured at the detector depend on derivatives of the phase, so that slowly varying features in phase (those corresponding to low spatial frequencies) produce weak intensity variations at the detector which can be easily corrupted by noise. This is problematic for low spatial frequencies where noise overcomes signal, leading to reconstructions with significant low–frequency artifacts.

Low–frequency noise artifacts can be addressed by using more than two defocused images since further defocus distances make the low–frequency phase contributions to intensity more apparent [15–19, 25]. However, these methods may not always be practical to implement as they lengthen acquisition times and may place strict requirements on mechanical stability and alignment of the system. Regularization methods in the inversion process may be adopted to reduce low frequency artifacts, [26, 27], although Tikhonov regularization is most commonly employed by replacing H_L in Eq. (9) with a regularized transfer function H_LT given by

A more straightforward method to reduce low frequency noise artifacts is to simply increase the measured signal by collecting more photons which can be achieved by acquiring more images (or increasing exposure time) as well as using a brighter source. While this may be practical in some cases (e.g. synchrotron sources, laser illumination or static samples being scanned without throughput requirements), this is not always possible. For example, in a microscope, halogen or LED illumination is typically used in which the maximum power of the source would limit low–frequency performance, particularly if acquisition time is limited due to imaging live samples or throughput requirements. Tabletop x-ray sources also have intensity limitations which can restrict the quality of phase reconstructions if exposure time is limited.

In what follows, we propose a method to reduce low–frequency artifacts and improve reconstruction quality in cases under two constraints: i) the phase imaging system employs Köhler illumination where the illumination source’s maximum intensity is limited, and ii) the total acquisition time is limited.

This article is organized as follows: In section 2.1, we review the effects of using a finite source in a Köhler illumination system on the TIE and demonstrate that there is a fundamental tradeoff under the above constraints between reducing low–frequency artifacts and introducing blurring or high–frequency deconvolution artifacts. In section 2.2, we propose the source diversity method which combines data using multiple source sizes to improve the reconstruction. In section 3, the experimental apparatus to modulate the source size using a spatial light modulator is described. Simulated and experimental results for this system are presented and analyzed in Section 4.

2. Theory and method

2.1. The TIE with a finite source

An imaging system, such as a microscope, with Köhler illumination will not exactly satisfy Eq. (5), which assumes a single, normally incident coherent plane wave as illumination. Köhler illumination will be modeled as an incoherent primary source [29] consisting of intensity profile I_s(x) located at the focal plane of a condenser lens of focal length f. If the source is a single point emitter located at position x, it will produce collimated beam whose propagation direction is along the vector ẑ + x/f as shown in Fig. 1(a). If this tilted beam is used as illumination with the assumption that the diameter of the beam is significantly larger than the sample, the intensity produced in the detector plane at a defocused distance ±Δz will have a profile given by Eq. (4) but shifted transversely by an amount ±Δz(x/f) and scaled globally by source intensity. For an incoherent primary source, each source point produces intensity that adds incoherently at the detector, resulting in a total intensity given by the convolution of the (scaled) source intensity profile with the TIE signal [30–32],

 

Fig. 1 (a) An off-axis point source one focal length away from a lens produces plane waves that travel with transverse vector direction −x/f. (b) Blur due to the source scales with defocus. A circular source of radius r illuminates a sample placed at z (shown as vertical dotted line). Rays passing through a single point on the sample will produce a circular spot in the defocused planes located at z ± Δz of diameter 2Δzr/f.

Download Full Size | PDF

Equation (11) can be expressed as

The system’s transfer function will provide a good measure of the a signal–to–noise ratio (SNR) in the frequency domain. Since the covariance of noise at the detector pixels is expected to be approximately zero for both thermal and shot noise dominated cases, we will assume the noise power spectrum is uniform over frequency space. The strength of a given frequency component of Θ relative to this noise is therefore proportional to H_Tot. As in the case of H_L alone, artifacts will be produced at any frequencies where H_Tot takes on small values (so that noise dominates the measurement).

In order to consider the case of imaging with different source sizes, we will restrict ourselves for now to the case of thermal noise dominated imaging with a fixed integration time and a fixed maximum intensity that can be produced by any point on the source. Since thermal noise does not vary with the source used, the noise power spectrum is similar and H_Tot can be directly compared for different sources to assess the system’s spectral performance. This is illustrated for circular sources of different sizes in Fig. 2(a). At frequencies where H_Tot is small for a given source, the measurement will produce a weak signal which is easily overcome by noise. For circular sources, H_s produces a Jinc function which when multiplied by H_L produces the lobed structure seen in Fig. 2(a). The use of larger sources shifts the lobes to lower spatial frequencies which strengthens the signal measured at low frequencies (a result of gathering more photons in the fixed integration time). However, the higher frequency nulls of H_Tot also shift to lower frequencies (a result of blurring with the source).

 

Fig. 2 (a) For thermal noise dominated imaging, H_Tot is a metric for system performance with respect to noise in the spatial frequency domain, which is illustrated for circular sources of differing radii r. (b) For shot–noise dominated imaging, normalized total transfer function H̄_Tot is a metric for system performance with respect to noise, since the noise variance increases with source size. This is illustrated for circular sources of differing radii r. Sizes in mm refer to source sizes used in the experimental setup (see Section 3). For all instances, Δz/f = 0.0067.

Download Full Size | PDF

It is also worthwhile to consider shot noise dominated imaging. Since shot noise variance increases with source size due to increased intensity captured at the detector, the advantage of using larger sources is reduced and the normalized transfer function ${\overline{H}}_{\text{Tot}}=H_{\text{Tot}}/\sqrt{H_{\text{s}}(\mathbf{0})}$ allows direct comparison of the system’s spectral performance for different sources. This is plotted for circular sources in Fig. 2(b). Results are similar to the thermal–noise–dominated case, but with reduced gain at low frequencies as source size is increased.

This frequency domain analysis leads us to a simple rule that governs the performance of TIE phase reconstruction in systems with Köhler illumination under fixed exposure time and maximal source intensity. When using a single source, there is a fundamental tradeoff between low–frequency and high–frequency performance depending on the source size used. Large sources will imporove low–frequency performance at the expense of high–frequency performance while small sources will improve high–frequency performance at the expense of low–frequency performance. Poor low frequency performance will manifest as large, blob–like structures across the reconstructed phase. High frequency degradation will manifest in inversion in one of two ways when large sources are used: i) If deconvolution is attempted to recover high spatial frequency phase information, grainy, high–frequency artifacts are expected in the reconstruction. ii) If deconvolution is not attempted, the reconstructed phase will be blurred, often resulting in under-estimation of the phase.

2.2. Source Diversity TIE

The fundamental tradeoff mentioned above holds only for single–source TIE. With multiple sources, one could gather low–frequency data from the large sources and high–frequency data from small sources to adequately measure both frequency domains. This can be readily illustrated by comparing system performance in the frequency domain, which is illustrated in Fig. 3. First, consider a single, small (r = 1.5 mm) circular source used as illumination in a system with maximum resolution of 0.23 μm⁻¹ in comparison to using three sources of r = 1.5, 2.7, 3.3 each for 1/3 of the single–source exposure time. The transfer function for the single–source and the sum of the three–source cases are shown in Fig. 3(a). The combined–source case outperforms the single–source case at most spatial frequencies, in particular at the lowest spatial frequencies, which will reduce low frequency artifacts. Additionally, although the combined transfer function does dip below that of the single source in several places, it avoids the several nulls which appear. For a large source (r = 3.5 mm), the transfer functions are compared in Fig. 3(b). Notice that when only considering the low spatial frequencies, the single source case performs better than the combined sources (its transfer function is larger at low frequencies), which is expected since fewer photons are captured using multiple sources than a single, large source. However, it presents many nulls at higher frequencies. Although the combined system’s transfer function dips below the single large source’s in many places, it avoids nulls and therefore produces generally superior high frequency reconstructions. This is consistent with our claim that a combination of sources avoids the loss of high frequency information associated with large sources while performing better at low frequencies than small sources. Normalized shot noise transfer functions H̄_Tot are presented for the small (r = 1.5 mm) and large sources (r = 3.5 mm) in Figs. 3(c) and 3(d), respectively. Notice that although the benefit of using multiple sources is reduced since noise also increases with source size, a similar trend holds: combined sources perform better at low frequencies than small sources and avoid the significant loss of high frequency information associated with large sources.

 

Fig. 3 For thermal noise dominated imaging: (a) the total transfer function H_Tot for a single, small circular source (r = 1.5 mm) (dashed, red) versus the combined transfer function which is the sum of the transfer functions of the r = 1.5, 2.7 and 3.5 mm sources each with 1/3 of the single–source exposure (solid, orange); (b) H_Tot for a single, large circular source (r = 3.5 mm) (dashed, red) versus the combined transfer function (solid, orange). For shot noise dominated imaging: (c) the normalized total transfer function H̄_Tot for an (r = 1.5 mm) (dashed, red) circular source versus the combined transfer function (solid, orange); (d) H̄_Tot for an (r = 3.5 mm) (dashed, red) circular source versus the combined transfer function (solid, orange). Poorly sampled frequency regions (aside from the one at |u| = 0) are labeled with * for the single–source and + for the multi–source cases. Notice that the * regions are nulls of the transfer function while the + regions are small, but non–null values.

Download Full Size | PDF

In order to optimally combine measurements using multiple sources, the measurement process must first be modeled and then inverted to solve for the phase. A set of N TIE measurements taken with N sources can be written as a set of N linear equations in the Fourier domain as

Equations (16)–(18) represent the key result of this work which we refer to as source diversity TIE. They describe a method to combine data acquired using multiple sources under Köhler illumination in an optimal way to reconstruct the phase. The product $H_{\text{s}}^i{H}_{\text{L}}$ describes the spatial frequency content of the phase measured using the i’th source. While individual measurements suffer from the fundamental tradeoff between low– and high–frequency performance, the combination does not.

Note that Eq. (16) produces will suffer from artifacts wherever the denominator becomes small. (These are poorly measured spatial frequencies where noise will be amplified.) Since each measurement relies on the physics of propagation, the Laplacian transfer function H_L is present as a global factor in the demoninator and, as already discussed, presents problems at low frequencies. Therefore, in practice this is always replaced with the Tikhonov regularized version H_LT of Eq. (10) (as it must be in Eq. (17) as well). If the sum ${\sum }_{i=1}^N{\left|H_{\text{s}}^i(-\mathrm{\Delta }z\mathbf{u}/f)\right|}^2$ presents small values as well, regularization or denoising will likely be required. Because it can provide measurements above noise across a broader range of spatial frequencies than single–source measurements, source diversity can provide a less noisy starting point even if further denoising is desirable.

3. Experimental details

Our experimental setup is illustrated in Fig. 4. The illumination system consisted of a LED (λ ≈ 620 nm, Thorlabs model# M625L3) followed by a ground glass diffuser which was relayed onto a programmable secondary source consisting of a liquid crystal on silicon (LCOS) based spatial light modulator (SLM). The LCOS chip consisted of 1400 × 1050 pixels of approximately 10 μm in size, was extracted from a Canon Realis SX-50 projector and was capable of 8-bit intensity control per pixel. Because LCOS based SLMs operate by rotating the polarization of the incoming light, a pre-polarizer was used to polarize the incoming light which, after being modulated by the SLM, was filtered by a post-polarizer. The LCOS was controlled as a secondary computer display to project source patterns. This source was placed one focal length (75 mm) from a collimating lens to produce the required Köhler illumination on the sample. The SLM allowed rapid modification of the shape and size of the source function without altering any physical components within the setup.

 

Fig. 4 Schematic diagram of the experimental setup used.

Download Full Size | PDF

The illuminated sample was imaged onto a detector using 4-f relay optics. The camera, mounted on a motion-stage, was displaced by increments of Δz = 500 μm to capture under-focused, in-focus and over-focused images using an Aptina camera with a pixel size of 2.2 μm (model# MT9P031I12STMH ES). In order to fairly compare TIE with a single source to source diversity, data was acquired with fixed total integration time. In the case of source diversity, this integration time was divided equally among all source sizes used. In both simulation and experiment, three circular sources of radii r = 1.5, 2.7 and 3.5 mm were used whose transfer functions are illustrated in Fig. 2(a). The combined transfer function which represents the source diversity system’s spectral performance is illustrated in Fig. 3(a).

4. Results

4.1. Simulated results

In order to demonstrate the tradeoffs between low and high frequency performance of the source diversity method, we first simulated results that would be produced by the experiment described in Sec. 3. The phase object was a 50 nm deep star target etched in glass of refractive index 1.5, illustrated in Fig. 5(a). Defocused images were simulated using 3 circular source sizes: small (1.5 mm radius), medium (2.7 mm radius) and large (3.5 mm radius). Additive, Gaussian, thermal noise was added to the simulated intensity measurements with SNR≈ 9.6, 14.8 and 17.0 dB for the r = 1.5, 2.7 and 3.5 mm spots, respectively. For all following results, H_L was replaced with H_LT with ρ ≈ 0.85 mm⁻¹ in order to preserve features of the object while eliminating the worst low–frequency noise. Because circular sources produce H_s’s with nulls, some degree of regularization is required to prevent division by zero in Eq. (14) when deconvolving using single–source measurements. Although many approaches could be used, we simply apply Tikhonov regularization to the deconvolution step, resulting in a solution of the form

 

Fig. 5 (a) Star target for simulated results, etched 50 nm deep in a piece of glass (refractive index n = 1.5). (b) Cartoon face for experimental results, etched 50 nm deep in a piece of glass (n ≈ 1.5) used in experiment. (c) Depth profile along the dashed line in (b).

Download Full Size | PDF

Results of this phase reconstruction are illustrated in Fig. 6. We consider three cases of regularization of the deconvolution step: Figures 6(a1)–6(a3) illustrate weak regularization ρ_s = 0.1 in which the phase, though deconvolved, is obscured by high–frequency artifacts. This produces a result that would be unacceptable for most applications without additional denoising. Figures 6(b1)–6(b3) illustrate moderate regularization ρ_s = 1000 which reduces the high frequency artifacts without leaving significant blurring, although residual high frequency artifacts still make quantitative phase retrieval difficult at individual pixels. Low frequency blob–like artifacts in the small source image are quite evident. Figures 6(c1)–6(c3) illustrate strong regularization ρ_s = 50000 where only minor high frequency artifacts are visible. However, at this level of regularization the amount of deblurring is reduced as is evident by the ring–like blurring artifacts towards the center of the target in the larger source images. Figures 6(d1)–6(d3) illustrate phase retrieval in which deconvolution is not attempted.

 

Fig. 6 Etch depth calculation of a 50 nm deep star target (600×600 pixels) using simulated propagation and circular sources. Sample thickness (in nm) reconstructed using single sources of the indicated size along with (a1)–(a3) deconvolution with minimal regularization ρ_s = 0.1, (b1)–(b3) deconvolution with moderate regularization ρ_s = 1000, (c1)–(c3) deconvolution with strong regularization ρ_s = 50000 and (d1)–(d3) no deconvolution. (e) Reconstruction using source diversity and the three indicated sources.

Download Full Size | PDF

High frequency artifacts are not introduced here, but the resulting phase profiles produced with larger sources show significant blurring. The source diversity approach is illustrated in 6(e) and produces phase with significantly reduced high– and low–frequency artifacts and no significant blurring.

These results highlight the fundamental tradeoff under our assumptions of limited exposure time and source intensity. Because of the poorly sampled high spatial frequencies in single–source cases, significant regularization/denoising is required to avoid blurred images if large sources are used. If small sources are used to avoid significant blurring, low frequency artifacts increase. By using a source diversity approach, both low and high frequency components of the phase can be adequately sampled to produce an image with reduced low– and high–frequency artifacts. Note that while there is some residual high–frequency noise in the source diversity reconstruction, it is minimal, especially considering that the only regularization applied was to the inverse Laplacian. These artifacts are likely due to somewhat weakly sampled spatial frequencies denoted with + symbols in Fig. 3. The corresponding images for single sources with minimal regularization, Figs. 6(a1)–6(a3), present significantly more high frequency artifacts. Source diversity presents a much improved starting point prior to denoising/regularization.

4.2. Experimental results

Experimental results were acquired using the system described in Sec. 3. Transfer functions for the sources were generated from the known patterns projected on the SLM. The sample was a cartoon face etched to 50 nm depth in glass of refractive index 1.5 as illustrated in Figs. 5(b) and 5(c). Results for inversion to recover sample thickness are shown in Fig. 7. Deconvolved phase results using single sources present similar results to simulation and so we show only moderate regularization (ρ_s = 1000), which presents sharp images with reduced high–frequency artifacts in Fig. 7(a1)–7(a3). Notice that there are still significant artifacts in the images acquired with larger sources, and significant low frequency noise in the background of the small–source image. For comparison, we show what the results would look like if the source blur was left in (undeconvolved reconstructions) in Fig. 7(b2)–7(b3). The results are significantly blurred for larger sources, but do not have high frequency deconvolution artifacts. The reconstruction with a small source (r = 1.5 mm) in Fig. 7(b1) is sharp since blurring is minimal, but low frequency background noise is still evident. Source diversity results for these three sources are illustrated in Fig. 7(c), which shows little blurring compared to Fig. 7(b2) and 7(b3), reduced high–frequency artifacts compared to single–source deconvolution in Fig. 7(a2) and 7(a3) and reduced low–frequency artifacts in comparison to the small source images in Fig. 7(a1) and 7(b1). The residual ringing artifacts present in the source diversity image were found to be due to a slight mismatch between our blur model and experiment and could perhaps be further reduced with more accurate system characterization. In order to more quantitatively demonstrate the impact of blurring and low/high–frequency artifacts on etch depth reconstruction, profiles were taken along the dashed line shown in Fig. 5(c) for each of the reconstructions in Fig. 7. These results are presented in Fig. 8. As expected, deconvolution produces more high frequency artifacts for the larger sources Fig. 8(b2)–8(b3) than the smaller one (b1). However, the low frequency artifacts in the small source image are clearly obvious from the fact that the background (the region outside of the etched wells) is not flat, as can be seen by its deviation from the dashed reference line which indicates the expected backgound value of phase. The use of larger sources removes much of this low frequency noise. For phase reconstruction without deconvolution, the background shows similar low–frequency performance: the small source Fig. 8(b1) produces a background phase with strong variation while larger sources 8(b2) and 8(b3) produce significantly flatter backgrounds. However, this comes at the expense of blurring which reduces the estimated depth and broadens the width of the etched regions in the reconstruction. Source diversity, Fig. 8(c), produces etch depth and width similar to that of the small source 8(a1) and 8(b1), but with considerably less low frequency noise. Some residual high frequency noise remains, but it is of a similar level to the deconvolved results for the small source. In source diversity, regularization was only applied to the inverse Laplacian. Therefore, source diversity results can serve as a relatively low–noise starting point for further denoising/regularization whereas the deconvolved results in Fig. 8(a1)–8(a3) have already been significantly regularized.

 

Fig. 7 Experimental etch depth calculation of a 50 nm depth cartoon face (600×600 pixels) data using circular sources of the indicated sizes using (a1)–(a3) deconvolution with moderate regularization ρ_s = 1000 and (b1)–(b3) no deconvolution. (c) Illustrates reconstruction using source diversity.

Download Full Size | PDF

 

Fig. 8 Line plot of etch depth calculation of a 50 nm depth cartoon face (600×600 pixels) data using circular sources of the indicated sizes using (a1)–(a3) deconvolution with moderate regularization ρ_s = 1000 and (b1)–(b3) no deconvolution. (c) Illustrates reconstruction using source diversity. Dashed line indicates the expected, constant background value of the phase target.

Download Full Size | PDF

In summary, the benefits of source diversity are: (i) It requires no regularization aside from the inverse Laplacian to present relatively low–noise phase reconstructions. (ii) It presents fewer low frequency artifacts than the small–source results. (iii) It performs better at high frequencies (less blurring/high–frequency artifacts) than large–source results.

5. Concluding remarks

We have demonstrated the use of multiple source sizes in a TIE phase imaging system utilizing Köhler illumination, a technique which we call source diversity TIE. Finite sources cause the light to diverge after passing through the sample, which results in blurring when defocusing the camera to acquire the TIE measurements. However, if the total integration time and the maximum intensity of the source are limited, as might be the case when imaging a dynamic sample under a microscope, gathering more light during an acquisition relies upon using larger sources. In turn, larger sources improve low–frequency performance which is limited by the physics of the TIE while reducing high–frequency performance due to blurring. We have illustrated the use of multiple sources (source diversity TIE) through transfer function analysis and shown that it can improve sampling at both low and high frequencies while single–source measurements must trade off low and high frequency performance. We have attempted to use minimal regularization in our reconstructions, although if further denoising of the resulting images is required, source diversity still provides a cleaner phase reconstruction with fewer prior assumptions on the phase than do single–source deconvolved reconstructions.

The use of arbitrary sources was enabled through an SLM–based secondary source. Here, we have chosen a range of source sizes without optimizing the size or shapes considered and captured equal–time exposures for each source size used. Future work will consider optimal combinations of source sizes and exposure times, which are expected to vary for different samples or noise models. Rather than simple least–squares inversion and Tikhonov regularization as used here, the inversion algorithms could also be optimized. We have also not considered here the fact that the TIE is an approximation and is valid, at a given finite defocus Δz, only for sufficiently small spatial frequencies satisfying u² ≪ (λΔz)⁻¹. One could choose, for a given defocus distance, a Gaussian source that effectively blurs out frequencies failing this requirement, potentially improving the reconstructions. Moreover, for objects whose phase profile has known support in the frequency domain, sources could be employed to optimize measurements for those frequencies rather than targetting broad coverage as we have done here.

For objects with slowly varying phase features, the CTF method could also be employed as it is accurate for longer propagation distances, and source diversity is expected to readily extend to these algorithms as well. Lastly, source diversity can be extended to uncollimated, incoherent, primary sources which will allow it to be readily extended to x–ray sources, where the convolution is over a collection of spherical waves emitted by the source [7].