Open Access
Add to BibSonomy
Abstract
Fluorescence microscopy is a powerful method for producing high fidelity images with high spatial resolution, particularly in the biological sciences. We recently introduced coherent holographic image reconstruction by phase transfer (CHIRPT), a single-pixel imaging method that significantly improves the depth of field in fluorescence microscopy and enables holographic refocusing of fluorescent light. Here we demonstrate that by installing a confocal slit conjugate to the illuminating light sheets used in CHIRPT, out-of-focus light is rejected, thus improving lateral spatial resolution and rejecting noise from out-of-focus fluorescent light. Confocal CHIRPT is demonstrated and fully modeled. Finally, we explore the use of beam shaping and point-spread-function engineering to enable holographic single-lens light-sheet microscopy with single-pixel detection.
© 2019 Optical Society of America under the terms of the OSA Open Access Publishing Agreement
1. Introduction
Fluorescence microscopy (FM) is routinely used in biological studies to collect high contrast images of nearly transparent specimens. The ability to specifically label structures of interest with fluorescent probes and collect images with high signal-to-noise ratio (SNR) has transformed biological studies in the past three decades. While there are a myriad of FM techniques that provide high contrast images of a vast array of specimens, there are still challenges that limit the use of FM for specific applications.
A major limitation of FM for biological imaging is the long acquisition time required to form three-dimensional (3D) images. Wide-field microscopy rapidly collects two-dimensional (2D) images with camera sensors that directly record an image of the object. Unfortunately, wide-field imaging does not provide axial sectioning, and thus formation of a 3D image is impossible with wide-field imaging in the traditional configuration – although a variety of deconvolution methods have appeared that enable the recovery of 3D representations of the specimen [1]. In addition, scattering of the light emitted from the specimen leads to significant blurring in the resulting 2D images.
In response, a resurgence of interest in wide-field microscopy with oblique illumination [2] has occurred over the past decade [3,4]. This type of imaging is often referred to as selective plane illumination microscopy (SPIM) and has tremendous advantages, particularly in eliminating photobleaching of out-of-focus regions of the sample. Although the illumination light in SPIM provides axial sectioning by exciting only a thin region in the axial dimension, 3D image collection requires the specimen and illumination to be scanned relative to one another. Light-sheet imaging also relies on a camera and is therefore susceptible to image degradation due to scattering and distortion from aberrations in the specimen. Finally, most versions of SPIM rely on two or more objective lenses that share a focal plane and are therefore in close proximity to one another. This presents a limitation in the size of specimens that can be imaged, although scattering and aberrations limit many SPIM systems to small specimens anyway.
Laser-scanning FM methods encode 3D images by raster scanning one or more tightly focused point-spread-functions (PSFs) and the specimen relative to one another [5–7]. While a significant benefit of laser-scanning FM methods is the use of a large-area, single pixel detector, and thus improved performance in scattering media, the need to move the PSF through the object in three dimensions places bounds on the acquisition rate. A variety of multifocal laser-scanning approaches have been reported that partially overcome the limitation of serial voxel acquisition with varying degrees of success [8–11].
Conversely, there exist a variety of imaging methods that employ spatially coherent light to acquire multi-dimensional images of a specimen in a single measurement, including digital holography (DH) [12–14] and ankylography [15,16]. While the details of phase recovery can differ from method to method, coherent diffractive imaging (CDI) techniques exploit the spatial phase of an illuminating light source to back-propagate an image collected on a camera and form a 3D representation of the object. Whatever the details of the method, CDI has a tremendous advantage in image acquisition speeds since the number of measurements necessary to generate a 3D representation of the object is reduced to only a few 2D images. However, CDI necessitates spatially coherent signal light since an image is recovered using the spatial phase recorded in the camera plane. This eliminates the possibility of performing traditional CDI measurements with spatially incoherent light, including fluorescence. Further, the interference pattern is collected with a camera sensor, which restricts the applicability of CDI methods to objects with little or no scattering [17].
Due to the speed of CDI methods and the power of fluorescent labeling, a technique that enables CDI with spatially incoherent light is very desirable – particularly if the detection scheme utilizes single-element detection and is robust to linear scattering of the emitted fluorescent light. Recently, we developed coherent holographic image reconstruction by phase transfer (CHIRPT) microscopy – a single-pixel FM technique that enables fluorescent CDI [18–20]. To date, we have demonstrated fluorescent holographic imaging, aberration correction in post-processing, and fluorescent diffraction tomography. CHIRPT enables fluorescent CDI by illuminating the specimen with two interfering light sheets with a spatiotemporal modulation pattern that transfers the spatial phase difference between the two illumination beams into temporal modulations of the emitted light.
While there are many advantages of using CHIRPT to image fluorescent samples, a number of technical challenges remain that must be overcome to take full advantage of the spatial phase recovered with this technique. For example, CHIRPT enables a dramatic increase in the depth-of-field (DOF) over traditional imaging with tightly focused illumination. Whereas the DOF scales as NA−2 for typical FM methods, the DOF scales as NA−1 in CHIRPT [19]. While this allows for 2D holographic refocusing in the (x̂, ẑ)-plane over an exceptionally large DOF, it is necessary to illuminate the object with tightly-focused light sheets to achieve diffraction-limited spatial resolution in the orthogonal transverse dimension (ŷ). As a result, the effective DOF over which CHIRPT can refocus 3D images is restricted to the traditional limit to maintain spatial resolution in the vertical dimension. In addition, the SNR of images collected from regions within the traditional DOF is corrupted by out-of-focus background light.
One method to restrict the region of holographic propagation and improve SNR in CHIRPT microscopy is to utilize a confocal detection slit to control the effective DOF [21]. In this Article, we put forth a theoretical description of CHIRPT imaging with a confocal detection slit and provide both experimental images and numeric simulations that corroborate our results. Our experimental and simulated images show that the confocal slit eliminates out of focus fluorescent light, improving the SNR and ensuring diffraction limited spatial resolution over the full DOF. We then simulate engineering the illumination light sheets and the detection PSF to be Airy beams that accelerate in opposing directions, allowing the recovery of large DOF CDI of fluorescence with diffraction-limited spatial resolution. These simulated 3D images demonstrate how confocal CHIRPT can be used in conjunction with PSF engineering to create a single-lens, single-pixel SPIM-like imaging system.
2. Effective DOF in CHIRPT microscopy
CHIRPT forms images of an object emitting spatially incoherent light, such as fluorescence, by analyzing the light emitted due to the spatiotemporal modulation of illumination intensity created by a pair of interfering light sheets with a time-varying crossing angle [18,19,22]. At each time point in the measurement of a fluorescent image, the signal generated by the single pixel detector is simply the overlap integral of the object fluorophore concentration, c(r), and the illumination intensity, Iill(r, t): St = 〈Iill(r, t) c(r)〉r, where we have omitted constants such as the efficiency of the detector and assume that the PSF of detection is approximately unity [23]. Here we assume that the fluorescent light emitted by the object is collected by a single-element detector from a region significantly larger than the object. As a result, the bounds of the overlap integral are well approximated by infinity. In addition, we assume the object concentration does not vary with time to simplify the following analysis.
A temporal variation in the intensity of the emitted fluorescent light arises from the interference of two light sheets with electric fields of the form: , where Γ(r) is the spatial profile of the light sheet, and kj,t and φj,t represent the time-dependent spatial frequency and spatial phase accumulated by the j-th diffracted light sheet. The diffracted light sheets are generated by illuminating a rotating modulation mask with a single spatially-coherent light sheet and re-imaging the modulation mask onto the specimen [18, 20]. A spatial filter placed conjugate to the mask plane is used in CHIRPT to isolate only one diffracted beam and the undiffracted beam [18]. The varying angle of the scanning beam, u1(r, t), is determined by the time-dependent k-vector k1 = (k sin θt, 0, k cos θt), where θt is the angle with respect to the time-stationary reference field, u0(r), propagating along the optic axis with k0 = (0, 0, k) (Fig. 1). The illumination intensity is then: Iill(r, t) = |u0(r) + μt u1(r, t)|2. Here, the relative amplitude, μt, sets the fringe visibility in the illumination intensity due to interference of the two light sheets, and ultimately determines the modulation transfer function (MTF) for the microscope [22].
Fig. 1 Schematic of the epi-fluorescent collection scheme in confocal CHIRPT. PMT: photomultiplier tube; CS: confocal slit; TL: tube lens; DM: dichroic mirror; WD: working distance.
The phase signal φt varies rapidly compared to the phase difference between the two light sheets, ΔΦ(r, t) = Δkt · r = (k1,t − k0) · r, allowing complex field interference products to be isolated due to a “carrier” frequency, i.e., a phase shift that sweeps as the crossing angle and transverse spatial frequency are varied [18, 19]. In CHIRPT, one generally takes a single sideband (SSB) term demodulated by φt and given by [22]
The signal is thus formed by projections of the spatial frequencies of the object at Δkt, and gated spatially by the envelope of the illumination intensity. In CHIRPT and other spatial-frequency projection imaging (SFPI) methods, one can scan the transverse spatial frequency linearly in time, so that kx,t = k sin θt = 2π κ t [10, 23–25]. This produces spatial frequency projections along the x̂-direction. Formation of a 3D image requires scanning of the object (or illumination) along ŷ and ẑ directions to form a full 3D image. The object is then easily reconstructed from these data, leading to an object estimate of [22] where ⊗ denotes a convolution between the object and the SFPI imaging impulse response.For CHIRPT, the impulse response is
We call this a coherent spread function, even though we collect spatially incoherent light, because the isolation of the SSB that corresponds to a pair of field interference terms leads to a complex impulse response that contains a phase, which represents the phase difference between the two illumination fields accumulated by propagating from the modulation mask to a point in the object [19]. Note that we may write the imaging model in the spatial frequency domain as Ĉ(k) = CTF(k) C(k), where Ĉ(k) is the Fourier transform of Eq. (2), and the coherent transfer function (CTF) is defined by CTF(k) = 〈μt[Γ̃(k) ⊗ δ(k − Δkt)]k〉t, and C(k) and Γ̃(k) are the Fourier transforms of the spatial distribution of fluorophores and the intensity profile of the illuminating light sheets respectively. The collection of all of the emitted light includes light that is blurred along the ŷ-direction due to spreading of the light sheet intensity, Γ(r), which degrades the resolution along the ŷ direction and adds additional shot noise to the recovered image.Placing a confocal slit in the fluorescence detection arm of the microscope such that it lies in an image plane conjugate to both the line focus on the modulator disk and the focal plane of the light sheets in the specimen restricts the volume from which fluorescent light is collected on the detector. This confocal slit thereby places bounds on the integration volume of the overlap integral, and leads to a modified SSB of the form
where the volume gating function (VGF) confines the collection volume and is defined by Here, the vector ρ = xx̂ + yŷ describes the lateral coordinates in the object, z describes the defocus in the object, PSF(ρ, z) is the point spread function for detection of the spatially incoherent fluorescent light, which depends on the wavelength of the emitted fluorescence an the NA of the collection objective lens, M is the magnification of the object plane in the image plane, and D(ρi) describes the transmissivity of the confocal slit in the image plane. Note that the image plane corresponds to the plane in which the confocal slit is placed. Assuming that the focal plane of the microscope and the confocal slit reside in conjugate image planes of a 4-f collection system consisting of an objective lens and a tube lens, the magnification from the object plane to the image plane is: M = ftube/ fobj, where ftube and fobj are the focal lengths of the tube lens and objective lens, respectfully.An example of a computed PSF and the resulting VGF is shown in Fig. 2, where the green distribution is the scalar PSF computed for collection of fluorescent light with of wavelength of 555 nm and NA of 0.8 [26], and the purple distribution is the VGF resulting from the convolution of the PSF with a demagnified confocal slit. One notes that the VGF is “extruded” in the lateral dimension – a result of the high aspect ratio of the slit.
Fig. 2 Computed VGF created with a confocal slit. (a) Visualizations of the detection PSF (green) and the VGF (purple) resulting from a confocal slit. (b) The detection PSF at zo = 0. (c) The VGF at zo = 0. (d) The detection PSF at zo = −25 μm. (e) The VGF at zo = −25 μm. The multiplicative factors in (d) and (e) indicate that the distribution has been multiplied by that factor to bring the peak intensity to the same as the distributions in the focal plane for visualization. The PSF was computed for λem = 555 nm and detection NA of 0.8. Scale bars: 10 μm.
The VGF restricts the bounds of the integration of the spatial frequency projections formed in Eq. (1). With the introduction of the VGF, the CSF now reads
where Γeff(r) ≡ ψ(r) Γ(r) is the effective light sheet due to both the illumination profile and the confocal slit. The CTF becomes where Γ̃eff(k) = ψ̃(k) ⊗ Γ̃(k), and where ψ̃(k) and Γ̃(k) are the Fourier transforms of ψ(r) and Γ(k) respectively.3. Results
Equation (6) makes it clear that imaging the fluorescent light emitted from the specimen onto a confocal slit leads to a restriction of the imaging volume beyond that provided by the spatial profile of the illuminating light sheet alone – the effective spatial gating is defined by the product of the VGF with the illuminating light sheet amplitude. Figure 3 shows this effect for three differing slit sizes and a constant illuminating light sheet profile. Here, Γ(r) was a Gaussian-intensity cylindrical beam with a focal spot size in the vertical dimension that was computed with a fill-factor of unity: f0 = 1 [27]. Figures 3(a)–3(c) show slices through the VGF, ψ(0, y, z), for slit sizes of 500 μm, 200 μm, and 5 μm respectively. Meanwhile, Figs. 3(d)–3(f) show slices through the corresponding effective sectioning function, Γeff(0, y, z). While there is little difference in the effective light sheet sectioning with a 500 μm slit and a 200 μm slit (Fig. 3(d) and Fig. 3(e)), the effective sectioning is dramatically improved with a 5 μm slit (Fig. 3(f)). This is due to the size of the VGF being larger than the light sheet profile Γ(r) for the larger slit sizes, while the VGF with a 5 μm slit is similar in size to the spatial profile.
Fig. 3 The VGF, ψ(r), and effective light sheet sectioning, Γeff(r), for a variety of slit sizes. A 2D slice through the VGF in the (y, z) plane, taken at x = 0, is shown for slit sizes of (a) 500 μm, (b) 200 μm, and (c) 5 μm. The corresponding effective light sheet sectioning for each slit size, Γeff(r), are shown in panels (d)–(f). The illuminating light sheet, Γ(r), was computed in each case with λex = 488 nm, fill factor of f0 = 1, and a numerical aperture of 1.1. Scale bars: 25 μm.
To test the ability to control the imaging volume in CHIRPT using a confocal slit, we collected images with an adjustable slit in the epi-directed fluorescence detection line of a CHIRPT microscope. Details of the experimental setup have been described at length elsewhere [19]. Briefly, we utilized a 532 nm continuous wave laser to excite fluorescence in a CHIRPT microscope using a modulation mask with a maximum spatial frequency of 70 lines/mm. The modulation mask was re-imaged to the specimen plane of the microscope with a magnification of 95× using a dual stage image relay system that incorporated a 0.8 NA objective lens (Zeiss N-Achroplan 50×/0.8NA). Epi-fluorescent light emitted from the specimen was image relayed to an adjustable confocal slit with the objective lens and a 100-mm focal length tube lens providing 30.4× magnification. A photomultiplier tube (PMT) was placed behind the objective lens to measure the confocally filtered fluorescent light. A schematic of the microscope is shown in Fig. 1. Images of a 15-μm diameter shell stained fluorescent bead (LifeTechnologies, FocalCheck Slide 1, Well A1) were collected in the (x̂, ẑ) plane by translating the bead axially and collecting 1D CHIRPT images at each ẑ position. The complex-valued CHIRPT image was isolated from the real-valued voltage measurements as described in previous reports [18–20].
Figure 4 displays 2D images of the shell-stained bead collected with varying confocal slit sizes. In Fig. 4(a), the confocal slit was removed from the microscope and, as a consequence, the limiting aperture of the collection optics was the PMT sensor, which was 5 mm in diameter (Hamamatsu, H7422P-40). Confocal slit sizes of 200 μm and 75 μm were used to collect the images displayed in Fig. 4(b) and Fig. 4(c) respectively. The 75 μm slit size was selected by observing a live reconstructed line image and adjusting the size of the slit to achieve an SNR of approximately 2 for a single scan.
Fig. 4 Experimental and simulated 2D images in the (x, z) plane at y =0 for various sizes of the confocal detection slit. Images were collected with slit sizes of (a) 5 mm (the size of the detection PMT aperture), (b) 200 μm, and (c) 75 μm. Corresponding simulated images for the same slit sizes are shown in panels (d)–(f).
Figures 4(a)–4(c) clearly demonstrate that as the size of the confocal slit is decreased, the axial sectioning capability of the CHIRPT microscope is improved. This effect was anticipated due to the similar behavior of point-scanning confocal microscopes, as well as reports of other fluorescent imaging schemes utilizing SFPI illumination patterns in conjunction with spatial filtering in the fluorescence detection system [28]. Moreover, the analysis in the preceding section indicates that use of a confocal aperture will result in a gating function, ψ(r), that restricts the volume of a measured CHIRPT image.
We also note that fluorescent imaging with line-scanning confocal microscopes has been described elsewhere [29]. Since such systems are confocal only in one dimension instead of two, their axial sectioning capability is slightly degraded as compared to a traditional point-scanning confocal microscope. The confocal CHIRPT microscope we describe here has the same limitations in axial sectioning capability since the system is only confocal in one dimension (ŷ).
To test the theoretical analysis presented in the previous section, we simulated 3D images of a 15-μm diameter shell-stained bead using the same parameters as our experimental implementation. The CSF and CTF for illumination were computed as described in previous work, where the CSF under plane-wave illumination was first computed and then augmented by the spatial profile of the light sheets, Γ(r) [22]. The VGF was then created by first computing a scalar PSF for the objective lens at the emitted fluorescent wavelength [26], and then convolving the collection PSF with a demagnified image of the slit as indicated in Eq. (5). An image of the object was then computed by convolving the total CSF, including the VGF, with the object.
Figures 4(d)–4(e) display 2D images extracted from the resulting 3D image data for confocal slit widths corresponding to the images in Fig. 4(a)–4(c). The high fidelity of the experimentally collected images and the simulated images suggests that the model presented for the VGF in the previous section is appropriate for most imaging conditions.
Both the experimental and simulated images in Fig. 4 were Fourier transformed to examine the form of the images in k-space. From the form of a measured CHIRPT image shown in Eq. (2), i.e., a convolution of the CSF with the fluorophore distribution, an image in k-space should be the product of the object’s spatial frequency distribution with the CTF of the imaging system. The CTF for CHIRPT has been investigated at length both theoretically and experimentally, and appears as a slice through the Ewald shell in k-space under plane wave illumination [18–20]. Under light sheet illumination, the spatial frequency support of this arc in k-space is broadened along the longitudinal spatial frequency axis [22].
Given the inverse relationship between DOF and axial spatial frequency support, we expected that the spatial frequency support of the arc in k-space would be increased as the size of the slit is decreased. Indeed, Fig. 5 demonstrates this relationship and, once again, shows high fidelity between the experimentally collected data and the numeric simulations.
Fig. 5 Reciprocal space representations of the images shown in Fig. 4. Experimental data is shown in panels (a)–(c), while simulated data is displayed in panels (d)–(f). Note that the experimental data displays large DC-peaks that result from edge effects in computing the Fourier transform numerically with the FFT. In addition, the simulated data does not display the noise observed in the experimental data as we did not include noise in the simulation.
While the extent of the axial spatial frequency support is accurately predicted by our numeric model, there is a discrepancy in the curvature of the arc in k-space between the experimental and simulated images. This discrepancy in curvature is constant with the size of the confocal slit, and therefore must arise from a physical mechanism that is independent of fluorescence detection. Indeed, we find that the difference in curvature arises from a discrepancy in the index of refraction of the mounting medium surrounding the fluorescent sphere. While the numerical model assumed a bead immersed in air and imaged with an air-immersion objective lens, the bead used in the experimental measurements was surrounded by an optical mounting cement with an index of ∼1.52 [30]. Thus the longitudinal spatial frequency in the experimental data is reduced due to refraction of the scanning beam in the optical cement. This difference in curvature results from a spatial phase that varies with the fourth power of the lateral spatial frequency incident on the specimen at each scan time, and therefore is formally equivalent to spherical aberration.
3.1. PSF engineering in CHIRPT
Introducing a confocal slit in CHIRPT offers a number of advantages for the images recovered by scanning the specimen relative to the illumination. Since the slit acts as a spatial filter that reduces or completely removes contributions to the measured signal from of out-of-focus fluorescent light, the resulting images are optically sectioned. This optical sectioning both preserves the spatial resolution in the vertical dimension and decreases the shot noise from background light. Unfortunately, the drawback of using a confocal slit in CHIRPT is a loss of DOF, which necessitates scanning in the both the ŷ and ẑ dimensions to form a 3D image. Thus the inherent phase-sensitivity of CHIRPT that permits CDI with fluorescence is negated and imaging speed suffers.
The power of CHIRPT is recovery of a 3D image from a measurement in the lateral plane, where the specimen (or illumination) need only be scanned in the vertical dimension. This ability enables high-speed imaging of fluorescent objects, but rests on having a large DOF in the ẑ dimension. Unfortunately, the confocal slit erodes this primary advantage of CHIRPT by decreasing the DOF.
When the size of the slit is adjusted such that the VGF gates fluorescent light from only those fluorophores residing within the DOF of the illuminating light sheet, images can be propagated in the axial dimension with isotropic spatial resolution in the lateral plane. Therefore, within this 3D volume defined by the lateral (x̂) extent of the illuminating light sheets, the range of the scan in the vertical (ẑ) dimension, and the DOF in the axial dimension, a single 2D image can be propagated to produce a 3D representation of the object with high fidelity. While this volume is limited axially by the conventional DOF of the focused light sheets, it can be greatly expanded by exploiting beam shaping methods to increase the DOF of both the focused illumination and the VGF.
To fully utilize holographic refocusing of fluorescent images in CHIRPT, non-diffracting illumination beams are a natural choice. Illumination with a non-diffracting beam ensures that the spatial resolution in the vertical dimension is preserved over a large DOF since the shape of the illuminating light sheet does not vary over a relatively large propagation distance. As a result, the volume over which holographic images can be propagated with high fidelity is expanded. One choice of non-diffracting illuminating light sheets that could prove useful is a Bessel light sheet [31]. However, large sidelobes on the illuminating beam could corrupt the resulting images, thus reducing the fidelity of a propagated image compared to a full 3D scan with optical sectioning.
Instead, we propose the use of 1D Airy light sheets, also called accelerating light sheets [32,33], to illuminate the specimen. Airy beams, like Bessel beams, are propagation invariant over relatively long distances and can therefore provide enhanced DOF. Unlike Bessel beams, the sidelobes that appear on an Airy beam are asymmetric due to the cubic-order pupil phase required to produce them, and appear on only one side of the main intensity lobe. This offers a potential solution for increasing the DOF in CHIRPT while maintaining the spatial resolution in the vertical dimension.
As we noted in Eq. (6), the CSF for the CHIRPT microscope depends on the effective light sheet formed by the product of the illuminating light sheet and the VGF. Our proposed solution to the DOF limitation in CHIRPT is to construct an illuminating light sheet and VGF that are both 1D Airy beams with opposite accelerations. This can be accomplished by applying a cubic pupil phase to the illuminating light sheet, while applying a cubic pupil phase of the opposite sign to the detection PSF in Eq. (5). Consequently, the VGF and the illuminating light sheet exhibit sidelobes appearing on opposite sides of their main intensity lobes. The imaging properties of CHIRPT depend on the product of the illumination envelope and the VGF, which suppresses the sidelobes – producing a clean and flat light sheet over a remarkably long DOF. An example of this concept is shown in Fig. 6, which displays cross sections of a 1D Airy illumination beam, an “Airy VGF”, and the resulting effective light sheet that determines the vertical extent of the CSF. As Fig. 6c demonstrates, the DOF of the effective light sheet is increased by an order of magnitude.
Fig. 6 Simulated spatial envelopes for CHIRPT, computed with cubic pupil phase to create accelerating light sheets (i.e., 1D Airy beams). Slices in the (y, z)-plane are shown through (a) Γ(r), (b) ψ(r), and (c) the effective light sheet envelope, Γeff(r). Dashed lines indicate the approximate DOF for illumination and detection under the same conditions with a Gaussian-intensity light sheet and an unmodified detection PSF. Scale bar: 10 μm.
The utility of this beam-shaping scheme for CHIRPT microscopy with numeric propagation becomes clear when compared to standard CHIRPT imaging, which utilizes a large-area detector without confocal gating, and to CHIRPT with a confocal slit. We simulated 3D CHIRPT images of an array of beads with random positions to compare all three of these cases. In each case, a 3D image that would be acquired by physically scanning the object relative to the illumination was computed using the methods described in our previous work [22]. A 2D lateral image at the geometric focal plane was extracted from the resulting 3D image and used as the “ground truth” image with no defocus. Next, an image located approximately 4 confocal parameters away from the geometric focus was extracted from the 3D image, where a confocal parameter was defined based on the DOF in the vertical dimension. Finally, the defocused image was numerically propagated back to the geometric focal plane of the microscope using the known phase variation introduced by the CHIRPT illumination pattern [18,19,22], and compared to the ground truth image.
Figure 7 displays the results of this simulation for all three cases: no confocal slit with 1D Gaussian beam illumination, a confocal slit with 1D Gaussian beam illumination and an unaberrated detection PSF, and our Airy light sheet illumination and detection scheme. The purpose of this simulation is to compare an image measured in the focal plane (ground truth image) with a numerically propagated image that was measured with the object defocused by more than the DOF of the illuminating Gaussian light sheet. In order for holographic propagation to be a useful method to rapidly acquire images of an object, one must ensure that the propagated image closely matches what could be directly measured by scanning the object. From Fig. 7 it is clear that standard CHIRPT imaging that does not make use of a confocal slit suffers from highly anisotropic lateral resolution in the refocused image. This is a result of the relatively small DOF of the illuminating light sheets. Specifically, CHIRPT does not record a defocus phase in this dimension, and as a result image of an object away from the geometric focal plane can only be refocused in the lateral (x̂) dimension.
Fig. 7 Simulated images comparing the use of a confocal slit and Airy light sheets in CHIRPT. (a)–(c) Images in the focal plane (z =0). (d)–(f) Images from ∼4 confocal parameters away from the focal plane (z =4 DOFy). (g)–(i) The defocused images propagated back to the focal plane with the angular spectrum propagator. Scale bar: 10 μm.
The situation is somewhat improved with the use of a confocal slit to isolate the measured fluorescence to only the region around the DOF of the illuminating light sheet. Objects that are outside this DOF are effectively removed from the image, and thus the refocused image displays less of the anisotropic spatial resolution issues as the case when no slit is used. However, this also means that the effective holographic refocusing region for confocal CHIRPT is relatively small, being limited to a region that could be axially scanned rapidly.
By shaping both the illuminating light sheets and the VGF with the Airy light sheet scheme described above, the fidelity of the refocused image is dramatically improved. Since the effective DOF of Γeff(r) is larger than the simulated defocus, the refocused image and the ground truth images are nearly indistinguishable. Thus the axial range over which holographically propagated images closely resemble images that would be directly measured by scanning the object is indeed enhanced in CHIRPT with beam shaping and PSF engineering methods.
To further demonstrate this high fidelity between scanned 3D images and holographically propagated images, we numerically propagated the ground truth image over the same axial range as the images that simulated scanning the object in 3D. Figure 8 displays volume plots of scanned images in the left column, while the right column displays volume plots of the image recovered by propagating a single 2D image. In each case, the propagated image was computed from the image measured at the geometric focus, which is represented by the shaded plane in the left column. Propagated images were then deconvolved by the known CHIRPT CTF using a complex deconvolution algorithm [34], it is once again clear that CHIRPT with Gaussian beam illumination has serious shortcomings for propagations over distances greater than the DOF of the illuminating light sheet – both with and without a confocal slit in place. Conversely, the scanned and propagated images recovered using beam shaping and PSF engineering display high fidelity. We note that although the scanned image with a confocal slit shows the highest fidelity with the specimen of spherical beads, the time required to physically translate the object to recover such an image negates the advantages of coherent diffractive imaging of fluorescence provided by CHIRPT.
Fig. 8 Scanned (left column) vs. propagated (right column) images with various CHIRPT configurations. In all cases, the 2D image corresponding to zero defocus (gray plane in the scanned images) was propagated to compute the 3D images in the right column. (a) Scanned image with Gaussian beam illumination and no confocal slit. (b) Propagated image computed from the 2D image at zero defocus in panel (a). (c) Scanned image with Gaussian illumination and a 3 μm slit. (d) Propagated image computed from the 2D image at zero defocus in panel (c). (e) Scanned image with Airy beam illumination using a 3 μm slit and an 1D Airy beam PSF in the detection optics. (f) Propagated image computed from the 2D image at zero defocus in panel (e).
Finally, we quantified the fidelity of scanned versus propagated images for each imaging mode by computing the mean squared error (MSE) between the lateral images at each axial position (Fig. 9). While the MSE for each method is very low near the focal plane and over a range corresponding to approximately the DOF of the illuminating light sheet, the MSE when Airy beams are used for illumination and detection is lower over a notably increased range.
Fig. 9 Mean squared error (MSE) computed between scanned and propagated lateral images as the axial position is varied. Gaussian beam illumination without a confocal slit: blue dashed line. Gaussian beam illumination with a confocal slit: red dotted line. Airy beam illumination and detection: solid gold line.
4. Discussion
CHIRPT microscopy has great potential for increasing image acquisition rates in fluorescence microscopy by enabling CDI methods such as holographic refocusing to take to the place of physically scanning the specimen. To date, we have demonstrated acquisition of line images with CHIRPT up to 600 lines/s [19]. Coupled with the numeric refocusing of fluorescent images that CHIRPT enables, this provides a path toward collecting 3D information at high rates.
As an illustrative example, let us assume 4400 lines/s are scanned with a CHIRPT microscope – a feat demonstrated with a technique similar to CHIRPT called FIRE microscopy [28]. Since CHIRPT enables digital propagation of each of these line images in the axial dimension, this microscope has effectively encoded 4400 2D images in the (x̂, ẑ) plane in one second. By scanning the light sheets in the ŷ dimension (perpendicular to the line focus), one can collect volume images with CHIRPT. If one assumes that the use of resonant galvonometric scanners or polygonal mirrors renders the time required to scan the lights sheets negligible, then a 3D image composed of 128 (x, z) planes could be measured in approximately 29 ms – enabling video-rate 3D fluorescent imaging at ∼34 volumes/s.
Conversely, if one were required to scan the sample in both the ŷ and ẑ dimensions, 3D image acquisition rates drop considerably. Assuming 128 axial scan points, the imaging rate to acquire a 3D volume in this example drops to approximately 0.27 volumes/s, requiring 3.7 s to acquire a 3D image. However, since the axial scan requires moving the specimen and illumination relative to one another, the inertia associated with this movement and settling time required will increase the acquisition times beyond this idealized example.
In addition, the single-element detector inherent in the CHIRPT microscope renders the system somewhat impervious to optical scattering. In total, these advances could pave the way for a new class of high-speed fluorescent microscopy. Unfortunately, the illumination pattern in CHIRPT does not encode spatial phase differences accumulated between the light sheets in the vertical dimension. As a result, images can only be numerically refocused in the lateral dimension.
From a computational perspective, numerical refocusing of CHIRPT data is relatively cheap and fast to implement. This is because propagating a single line image acquired with CHRIPT to create a 2D image in the (x̂, ẑ) plane can be implemented with the fast Fourier transform (FFT), as described in our previous work (cf. Supplementary Information of Ref. [19]). Therefore, if one were to simply acquire data as quickly as possible to be processed “offline” at a later time, images could be reconstructed plane-by-plane with a simple FFT-based reconstruction that is fast and requires little computer memory. To create a live 3D image rendering while collecting CHIRPT images, reconstruction with a dedicated graphical processing unit (GPU) would be desirable but not necessarily required.
In this work, we have proposed a method to circumvent the limitations of fluorescent CDI in CHIRPT by incorporating a confocal slit and PSF engineering. Our simulated images based on a rigorous mathematical analysis of confocal CHIRPT show striking similarity to measured images. Moreover, our simulated images that explore beam shaping methods in conjunction with confocal detection suggest that one-dimensional refocusing can be overcome. While we have only simulated one possible configuration for improving the DOF in CHIRPT holography, there are a myriad of possibilities that could be explored with programmable pupil phase modulators such as liquid crystal spatial light modulators (LCSLM).
We envision using this ability to extend the DOF in CHIRPT to develop single-lens SPIM systems that are robust to scattering and rely on single-pixel detectors with high gain and low noise, such as PMTs, to incorporate photon-counting measurements. In addition, the CHIRPT illumination scheme may enable localization microscopy SPIM (LM-SPIM), since the signal recovered from a single fluorescent molecule can be inverted to uncover the centroid of the molecule.
Funding
W.M. Keck Foundation; National Institute of Health (NIH) (R21EB025389, R21MH117786); National Science Foundation (NSF) (1707287); NeuroNex Technology Hub:Nemonic: Next generation multiphoton imaging consortium.
Acknowledgments
The content is solely the responsibility of the authors and does not necessarily represent the official views of the National Institutes of Health.
Disclosures
The authors declare that there are no conflicts of interest related to this article.
References
1. P. Sarder and A. Nehorai, “Deconvolution methods for 3-d fluorescence microscopy images,” IEEE Signal Process. Mag. 23, 32–45 (2006). [CrossRef]
2. H. Siedentopf and R. Zsigmondy, “Uber sichtbarmachung und größenbestimmung ultramikoskopischer teilchen, mit besonderer anwendung auf goldrubingläser,” Annalen der Physik 315, 1–39 (1902). [CrossRef]
3. J. Huisken and D. Y. R. Stainier, “Selective plane illumination microscopy techniques in developmental biology,” Development 136, 1963–1975 (2009). [CrossRef] [PubMed]
4. B.-C. Chen, W. R. Legant, K. Wang, L. Shao, D. E. Milkie, M. W. Davidson, C. Janetopoulos, X. S. Wu, J. A. Hammer, Z. Liu, B. P. English, Y. Mimori-Kiyosue, D. P. Romero, A. T. Ritter, J. Lippincott-Schwartz, L. Fritz-Laylin, R. D. Mullins, D. M. Mitchell, J. N. Bembenek, A.-C. Reymann, R. Böhme, S. W. Grill, J. T. Wang, G. Seydoux, U. S. Tulu, D. P. Kiehart, and E. Betzig, “Lattice light-sheet microscopy: Imaging molecules to embryos at high spatiotemporal resolution,” Science 346, 1257998 (2014). [CrossRef] [PubMed]
5. P. Davidovits and M. D. Egger, “Scanning laser microscope,” Nature 223, 831 (1969). [CrossRef] [PubMed]
6. P. Davidovits and M. D. Egger, “Scanning laser microscope for biological investigations,” Appl. Opt. 10, 1615–1619 (1971). [CrossRef] [PubMed]
7. W. Denk, J. Strickler, and W. Webb, “Two-photon laser scanning fluorescence microscopy,” Science 248, 73–76 (1990). [CrossRef] [PubMed]
8. Buist, Müller, Squier, and Brakenhoff, “Real time two-photon absorption microscopy using multi point excitation,” J. Microsc. 192, 217–226 (1998). [CrossRef]
9. J. Bewersdorf, R. Pick, and S. W. Hell, “Multifocal multiphoton microscopy,” Opt. Lett. 23, 655–657 (1998). [CrossRef]
10. E. E. Hoover, J. J. Field, D. G. Winters, M. D. Young, E. V. Chandler, J. C. Speirs, J. T. Lapenna, S. M. Kim, S.-y. Ding, R. A. Bartels, J. W. Wang, and J. A. Squier, “Eliminating the scattering ambiguity in multifocal, multimodal, multiphoton imaging systems,” J. Biophoton. 5, 425–436 (2012). [CrossRef]
11. J. J. Field, K. E. Sheetz, E. V. Chandler, E. E. Hoover, M. D. Young, S.-Y. Ding, A. W. Sylvester, D. Kleinfeld, and J. A. Squier, “Differential Multiphoton Laser Scanning Microscopy,” Sel. Top. Quantum Electron. IEEE J. 18, 14–28 (2012). [CrossRef]
12. D. Gabor, “A new microscopic principle,” Nature 161, 777 (1948). [CrossRef] [PubMed]
13. E. Leith and J. Upatnieks, “Wavefront Reconstruction Photography,” Phys. Today 18, 26 (1965). [CrossRef]
14. E. N. Leith and G. J. Swanson, “Recording of phase-amplitude images,” Appl. Opt. 20, 3081–3084 (1981). [CrossRef] [PubMed]
15. K. S. Raines, S. Salha, R. L. Sandberg, H. Jiang, J. A. Rodríguez, B. P. Fahimian, H. C. Kapteyn, J. Du, and J. Miao, “Three-dimensional structure determination from a single view,” Nature 463, 214–217 (2010). [CrossRef]
16. J. Miao, R. L. Sandberg, and C. Song, “Coherent X-Ray Diffraction Imaging,” IEEE J. Sel. Top. Quant. Elec. 18, 399–410 (2012). [CrossRef]
17. D. R. Smith, D. G. Winters, and R. A. Bartels, “Submillisecond second harmonic holographic imaging of biological specimens in three dimensions,” Proc. Nat. Acad. Sci. 110, 18391–18396 (2013). [CrossRef] [PubMed]
18. J. J. Field, D. G. Winters, and R. A. Bartels, “Plane wave analysis of coherent holographic image reconstruction by phase transfer (CHIRPT),” J. Opt. Soc. Am. A 32, 2156–2168 (2015). [CrossRef]
19. J. J. Field, D. G. Winters, and R. A. Bartels, “Single-pixel fluorescent imaging with temporally labeled illumination patterns,” Optica 3, 971–974 (2016). [CrossRef]
20. J. J. Field and R. A. Bartels, “Digital aberration correction of fluorescent images with coherent holographic image reconstruction by phase transfer (CHIRPT),” Proc. SPIE 9713, 97130B (2016). [CrossRef]
21. G. J. Brakenhoff, J. Squier, T. Norris, A. C. Bliton, M. H. Wade, and B. Athey, “Real-time two-photon confocal microscopy using a femtosecond, amplified ti:sapphire system,” J. Microsc. 181, 253–259 (1996). [CrossRef] [PubMed]
22. J. J. Field, K. A. Wernsing, J. A. Squier, and R. A. Bartels, “Three-dimensional single-pixel imaging of incoherent light with spatiotemporally modulated illumination,” J. Opt. Soc. Am. A 35, 1438–1449 (2018). [CrossRef]
23. G. Futia, P. Schlup, D. G. Winters, and R. A. Bartels, “Spatially-chirped modulation imaging of absorption and fluorescent objects onsingle-element optical detector,” Opt. Exp. 19, 1626–1640 (2011). [CrossRef]
24. D. J. Higley, D. G. Winters, G. L. Futia, and R. A. Bartels, “Theory of diffraction effects in spatial frequency-modulated imaging,” J. Opt. Soc. Am. A 29, 2579–2590 (2012). [CrossRef]
25. J. J. Field, K. A. Wernsing, S. R. Domingue, A. M. Allende Motz, K. F. DeLuca, D. H. Levi, J. G. DeLuca, M. D. Young, J. A. Squier, and R. A. Bartels, “Superresolved multiphoton microscopy with spatial frequency-modulated imaging,” Proc. Natl. Acad. Sci. 113, 6605–6610 (2016). [CrossRef] [PubMed]
26. M. Born and E. Wolf, Principles of Optics: Electromagnetic Theory of Propagation, Interference and Diffraction of Light (Cambridge University Press, 1999), 7th ed. [CrossRef]
27. W. R. Zipfel, R. M. Williams, and W. W. Webb, “Nonlinear magic: multiphoton microscopy in the biosciences,” Nat. Biotechnol. 21, 1369–1377 (2003). [CrossRef] [PubMed]
28. E. D. Diebold, B. W. Buckley, D. R. Gossett, and B. Jalali, “Digitally synthesized beat frequency multiplexing for sub-millisecond fluorescence microscopy,” Nat. Photon. 7, 806–810 (2013). [CrossRef]
29. C. J. R. Sheppard, Scanning confocal microscopy (Marcel Dekker, 2003), vol. Encyclopedia of Optical Engineering: Las-Pho of Dekker Encyclopedias Series, pp. 2525–2544.
30. “Datasheet for focalcheck fluorescence microscope test slides,” https://www.thermofisher.com/order/catalog/product/F36909 (2018).
31. F. O. Fahrbach, P. Simon, and A. Rohrbach, “Microscopy with self-reconstructing beams,” Nat. Photonics 4, 1–6 (2010).
32. M. V. Berry and N. L. Balazs, “Nonspreading wave packets,” Am. J. Phys. 47, 264–267 (1979). [CrossRef]
33. G. A. Siviloglou, J. Broky, A. Dogariu, and D. N. Christodoulides, “Observation of Accelerating Airy Beams,” Phys. Rev. Lett. 99, 1–4 (2007). [CrossRef]
34. Y. Cotte, F. Toy, P. Jourdain, N. Pavillon, D. Boss, P. Magistretti, P. Marquet, and C. Depeursinge, “Marker-free phase nanoscopy,” Nat. Photon. 7, 1–5 (2013).
