Early in the development of tomographic imaging, Crowther, De Rosier, and Klug put forward (Section 6.2, [1]) a simple relationship of

Multislice ptychographic tomography, based on multislice ptychography [2], has recently been introduced by Li and Maiden [3] as a way to obtain three-dimensional reconstructions of objects that extend beyond the depth of focus of an imaging system or its equivalent. Li and Maiden observed that one can obtain a high-quality 3D reconstruction with fewer illumination angles than would be expected for conventional tomography based on the Crowther criterion, but without providing a simple expression for how much the Crowther criterion could be relaxed. In a somewhat different approach, Hovden et al. used through-focal depth sectioning in a scanning transmission electron microscope, and they obtained high-resoIution 3D reconstructions with fewer illumination angles than required by the Crowther criterion [4]; yet, again, no specific expression was given for how much one can relax the Crowther criterion. We put forward here an explanation based on the fact that resolving objects into separate planes along one viewing direction fundamentally alters the transverse-axial Fourier space distribution of the information obtained, so that these observations are in fact expected for fundamental reasons. This reduction in the number of illumination angles required should also apply to other 3D reconstruction methods that allow for wavefield propagation effects within the specimen, such as diffraction microscopy [5–7].

Crowther Criterion: tomography is based on the acquisition of pure projection images of an object as it is rotated, providing the data needed to obtain a 3D reconstruction. With a 2D imaging detector aligned to a specimen on a vertical rotation axis $\widehat{y}$, each $y$th row of the detector separately obtains a series of 1D images of one vertical slice of the object (Fig. 1A) with $N_t$ transverse pixels each of the size $\mathrm{\Delta }t$. Therefore, the transverse-axial Fourier or reciprocal space represenation of an individual projection image will have $N_t$ complex pixels extending to a maximum transverse spatial frequency of $u_t=1/(2\mathrm{\Delta }t)$ based on Nyquist–Shannon sampling [8,9]. Because a pure projection image provides no information on the relative placement of different object features along the $\theta =0$ illumination axis direction, each projection image is only one pixel deep (the zero-spatial-frequency pixel) along the axial direction $\widehat{z}$ in Fourier space. Therefore, a 1D object slice image acquired at a rotation angle $\theta =0$ maps into $N_x=N_t$ pixels in the $u_x$ direction and $N_z=1$ pixels in the $u_z$ direction in transverse-axial Fourier space, as shown in Fig. 1B. When the illumination is rotated by an angle $\theta$, so is the transverse-axial Fourier space representation of the information obtained. We note that this transverse-axial or $(x,z)$ representation in Fourier space $(u_x,u_z)$ is different from the bowtie-shaped support [10–12] of information obtained in the Fourier transform of the Radon transform $p(\theta ,u)$, which is more analagous to a sinogram representation of tomographic data in $(\theta ,x)$.

 

Fig. 1. Schematic representation of the Crowther criterion in conventional tomography. In conventional tomography, each slice $y$ of the object along the direction $\widehat{y}$ of the rotation axis is mapped onto one row of a detector. One then obtains one-dimensional pure projection images of the object with $N_t$ transverse pixels of width $\mathrm{\Delta }t$ in the transverse direction A, and a depth of precisely one pixel at zero spatial frequency in the axial direction (because there is no way to distinguish between different axial positions in a pure projection). For an angle $\theta =0°$, the Fourier transform of this image yields an array with $N_x=N_t$ pixels in the transverse or $u_x$ direction and $N_z=1$ pixels in the axial or $u_z$ direction in transverse-axial Fourier space B. As the object is rotated, so is the information obtained in Fourier space, so the $(u_x,u_z)$ Fourier space is filled, as shown in C. The Crowther criterion of Eq. (3) is effectively a statement that one must provide complete, gap-free coverage of all pixels around the periphery in transverse-axial Fourier space.

Download Full Size | PDF

As the illumination direction is rotated by $\theta$ about the object axis and object slice projection images are obtained, each of these $N_t\times 1$ arrays rotated by $\theta$, in principle, can be assembled in transverse-axial Fourier space, as shown in Fig. 1C. (Note that care must be taken in how data are interpolated from the angular-transverse or $(\theta ,t)$ array of each projection onto the transverse-axial or $(x,z)$ grid [13–15], and one must apply the $1/|u|$ ramp filter of filtered backprojection to account for the overweighting of information at low transverse-axial spatial frequencies where there is much information overlap, as shown in Fig. 1C.) Once this is done, one can simply take an inverse Fourier transform of the assembled complex information and obtain a transverse-axial real space image of the object slice. In fact, this conceptually simple procedure is not the usual approach in practical tomography experiments due to the subtleties noted above of interpolating complex information from tilted projections into pixels in a regular 2D array in Fourier space. In addition, one can use algebraic reconstruction algorithms [16] and their variants which can incorporate various constraints on the object to reduce the presence of artifacts in the reconstructed object slice that would otherwise appear due to incomplete information in Fourier space. However, regardless of the reconstruction method used, the information available in transverse-axial Fourier space is as shown in Fig. 1C.

An examination of the assembled information in Fourier space, as shown in Fig. 1C, reveals the required number of illumination angles $N_{\theta }$ that should be used to obtain full information in transverse-axial Fourier space. The size of one pixel in Fourier space is

Because one projection image fills in pixels at both positive and negative maximum spatial frequencies, one needs to cover the circumference of a half-circle, or $\pi u_t$, with $N_{\theta }$ rotational samples at the spacing $\mathrm{\Delta }{u}_t$. Therefore, one requires $N_{\theta }(\mathrm{\Delta }{u}_t)=\pi u_t$, giving

Multislice Ptychographic Tomography: one way to obtain pure projection images of a sufficiently thin 3D object is through the use of ptychography. In 2D imaging via ptychography, one illuminates a larger object with a finite-extent coherent illumination spot, and then moves the spot to a series of overlapping positions on the object while collecting the far-field diffraction intensities [17,18]. The fact that one object feature appears in many diffraction patterns due to the probe (illumination spot) overlap means that iterative phase retrieval algorithms [19,20] can serve as the basis for robust reconstructions of the complex exit wave leaving the object [21,22]. This has led to the development of x-ray ptychographic tomography [23] for nanoscale 3D imaging.

Because objects, in fact, can extend over a larger axial distance, the pure projection approximation does not always apply. In an imaging system, Fresnel propagation of the point spread function gives rise to a depth of field about the axial focus position beyond which the point spread or probe function undergoes significant change; this gives rise to an axial resolution [24] of ${\delta }_a=1.22\lambda /{\text{N.A.}}^2$ for a circular lens. If one writes the numerical aperture as being given by the maximum scattering angle detected of $\text{N.A.}=\lambda /(2\mathrm{\Delta }t)$, one arrives at an axial resolution of

In order to employ the advantages of ptychography (where the image resolution is limited not by the resolution of a lens, but by the maximum angle at which one detects and phases coherent diffraction magnitudes) for objects that extend beyond the depth of focus limit, multislice ptychography was introduced [2], as shown in Fig. 2. This approach treats the object as being composed of several distinct axial planes, each of which contain a “slice” of optical modulation that can be applied to the incident wavefield, followed by free-space propagation to the plane of the next “slice” in a way reminiscent of multislice forward-modeling calculations developed for electron microscopy [30,31]. This approach has been shown to be successful in visible light [2,26], x-rays [27,28], and electron microscopy [29]. (It accounts for multiple optical interaction planes so from a wavefield propagation point of view it accounts for multiple scattering, though it does not allow for the complete loss of optical path length information due to plural elastic scattering of individual photons that occurs, for example, in fog or clouds.) In x-ray experiments [28], an empircal observation of the maximum usable separation between planes gave $5.2{(\mathrm{\Delta }t)}^2/\lambda$, which is in fair agreement with Eq. (4).

 

Fig. 2. Multislice ptychography [2] uses Fresnel propagation of the illumination probe to determine which of many axial slices an object is localized to. While propagating a single probe function between many axial planes (as well carrying out the standard ptychographic action of translating the probe to multiple transverse positions), at each axial plane the net wavefield is decomposed into contributions from the probe and from the optical modulation provided by any specimen features at that plane. This is accomplished through the fact that the probe is modified as it propagates plane-to-plane, so that prior experimental demonstrations [2,26–29] have clearly shown the ability to separate even broad, low-spatial-frequency objects into their respective planes along the illumination direction. Multislice ptychography (as well as through-focus imaging methods that succeed to various degrees in separating objects into their respective planes along the illumination direction) therefore delivers axial slice images which can have objects of all transverse sizes (and therefore content at all transverse spatial frequencies) reconstructed at separate axial planes.

Download Full Size | PDF

Multislice ptychographic tomography has recently been used to image objects to which the pure projection approximation does not apply [3]. In this approach, for each illumination angle, a number $N_a$ of axial planes were reconstructed through an extended object, as shown in Figs. 2 and 3A. These planes were separated by a distance $\mathrm{\Delta }a$ approximately equal to the depth resolution of Eq. (4). (The use of a much smaller plane separation distance $\mathrm{\Delta }a\ll {\delta }_a$ would lead to difficulties in the factorization of features to one plane versus another, and the Fourier plane power would not extend to high spatial frequencies in $u_a$ with too fine a plane separation distance.) In these experiments [3], the net optical effect from each plane was then added together (and phase unwrapping was applied) to synthesize a pure projection image from each illumination direction before using standard tomographic reconstruction algorithms to obtain a 3D image.

 

Fig. 3. Schematic representation of the Crowther criterion in multislice ptychographic tomography. Because multislice ptychography obtains separate reconstructed images for each axial slice (A; see also Fig. 2) and because these objects at each plane can be larger or small and, thus, contain information at all transverse spatial frequencies, the information that a multislice ptychographic image delivers in $(u_x,u_z)$ or transverse-axial Fourier space is as shown in B. If one then carries out multislice ptychography from a variety of illumination angles, one fills in Fourier space, as shown in C. The equivalent of the Crowther criterion (Fig. 1C) for multislice ptychographic tomography is then determined by setting the angular separation $\mathrm{\Delta }\theta$ between illumination directions to a value so that one provies complete, gap-free coverage of all pixels around the periphery in transverse-axial Fourier space, leading to Eq. (6).

Download Full Size | PDF

While multislice ptychography provides an especially good way to separate complex objects into their different axial plane positions so that they can then be combined to synthesize a pure projection image, there are also a variety of approaches for taking through-focus image sequences in conventional imaging and assigning features to their plane of sharpest focus [32–36]. These approaches must contend with the nature of the 3D point spread function and its associated optical transfer function for the imaging mode and contrast mechanism chosen, so the per-axial-slice images they deliver may not provide full information at all transverse spatial frequencies. However, they do deliver some information on objects located at depth planes separated by about a depth of field [Eq. (4)], again leading to image data, as shown in Fig. 2. Such an approach has been used in high angle dark field electron microscopy [4], where it was shown empirically that one then needed fewer illumination angles for a high-quality 3D reconstruction than one might have assumed from the Crowther criterion of Eq. (3).

Let us then consider the information provided by multislice imaging along one illumination direction, with a transverse-axial Foruier plane representation, as shown in Fig. 3B. The multislice reconstruction will produce $N_a$ axial pixels with a maximum axial spatial frequency of $u_a=1/(2\mathrm{\Delta }a)$. How do these pixels map onto the same grid in Fourier space as the transverse information, given that the real space pixel sizes of $\mathrm{\Delta }t$ and $\mathrm{\Delta }a$ are quite different? These $N_a$ reconstruction planes (which extend axially by ${\delta }_a/2$ on either side of the plane) must cover the object diameter $D$, so we have $N_a\mathrm{\Delta }a=D$, but also $N_t\mathrm{\Delta }t=D$ for the object to fit within the transverse field of view, giving $\mathrm{\Delta }a=N_t\mathrm{\Delta }t/N_a$. Including the negative spatial frequency limit $-u_a$, the axial planes are mapped onto Fourier space over a span of $N_a^{\prime }=2u_a/\mathrm{\Delta }{u}_t$ pixels which, when combined with $\mathrm{\Delta }a=N_t\mathrm{\Delta }t/N_a$, give

Now we return to the approach used earlier to calculate the Crowther criterion, but with multislice imaging instead of with pure projections. As before, the size of one pixel in Fourier space is $\mathrm{\Delta }{u}_t$ as given by Eq. (2), and we need to cover the circumference of a half-circle, or $\pi u_t$, with $N_{\theta ,a}$ rotational samples. However, each rotational sample now extends over not 1 pixel in the axial direction, as shown in Fig. 1B, but $N_A$ pixels, as shown in Fig. 3B. Therefore, we have

In the first demonstration of multislice ptychographic tomography [3], it was shown in Li and Maiden’s Fig. 7 that one can synthesize projections from other viewing directions by calculating the shift of separate axial planes along the new synthesized viewing direction before plane addition. These synthesized viewing directions were added with actual viewing direction data to provide high-quality 3D reconstructions using fewer actual illumination directions than one would have expected from the Crowther criterion. However, these synthesized projections involve the acquisition of no new image information, so the conclusion of Eq. (6) stands unchanged. Beyond-Crowther reconstruction quality was also observed when using learning algorithms to reconstruct diffraction microscopy data obtained using a limited number of actual illumination directions [7], as well as in through-focal depth sectioning electron microscopy example described above [4]. Because all of these approaches intrinsically account for multislice propagation effects, the information mapping considerations outlined here and the result of Eq. (6) provide a firm conceptual reason for these observed improvements.

The approach of adding together the optical contributions of the multislice planes (along with phase unwrapping) has provided a successful demonstration [3] of tomographic imaging of objects that extend beyond the depth resolution limit of Eq. (4). However, it shifts the information mapping from what is shown in Fig. 3B back to a synthesized pure projection, as shown in Fig. 1B, with additional synthesized illumination angle information obtained as described above. These additional, but synthesized, pure projection images obscure the reason why one can collect information from fewer illumination angles than one would expect from the conventional Crowther criterion. That is why we have proposed here modifying the conventional Crowther criterion of Eq. (3) with the multislice imaging version of Eq. (6).