Open Access
Add to BibSonomy
Abstract
Refocusing after Scanning using Helical phase engineering (RESCH) microscopy has previously been demonstrated to provide volumetric information from a single 2D scan. However, the practical application of this method is challenging due to its limited image acquisition speed and spatial resolution. Here, we report on a combination of RESCH and multifocal structured illumination microscopy (MSIM) to improve the image acquisition speed and spatial resolution. A phase mask is introduced to modulate the conventional point spread function (PSF) to the double-helix PSF (DH-PSF), which provides volumetric information, and meanwhile, sparse multifocal illumination patterns are generated by a digital micromirror device (DMD) for parallel 3D subdiffractive imaging information acquisition. We also present a strategy for processing the collected raw data with a Richardson-Lucy deconvolution and pixel reassignment algorithm to improve the spatial resolution of the depth estimation and imaging performance. The proposed 3D image scanning microscopy can record 3D specimen information and the corresponding depth information from a single multi-spot 2D planar scan, which ensures faster data acquisition, larger field of view, and higher spatial resolution than RESCH. Finally, we demonstrate the capability of our system with actual experiments.
© 2018 Optical Society of America under the terms of the OSA Open Access Publishing Agreement
1. Introduction
Confocal laser scanning microscopy (CLSM) [1] is a powerful imaging tool providing high resolution and optical sectioning, and it typically finds application in life sciences, semiconductor inspection, and materials science. In its standard optical configuration, an excitation focal point is raster-scanned across the field of view (FOV) to generate a 2D or 3D image. Subsequently, a pair of confocal pinholes is used to reject out-of-focus light, which leads to improved image contrast relative to wide-field microscopy. Traditionally the pinhole is set to the standard size of 1 Airy unit, which already improves the spatial resolution compared to wide-field imaging by a typical factor of 1.06. The resolution can be further improved by closing the pinhole even further. But this comes at the price of a drastic reduction of detection efficiency and signal-to-noise ratio (SNR). To solve the above problems, Sheppard [2] used a pixelated image sensor such as a CCD or CMOS to replace the point detector in conventional confocal microscopy, and he demonstrated that in principle, super-resolution is achievable with no loss of signal intensity. The concept was implemented experimentally more than 20 years later by Müller and Enderlein [3], who termed the method image scanning microscopy (ISM). Coupled with pixel reassignment, ISM can provide a maximal lateral-resolution improvement ofover wide-field fluorescence microscopy. For conventional ISM, the 3D information of the specimen is acquired by axial scanning, which decreases the time resolution. In recent years, in order to improve the temporal resolution of ISM, certain methods have combined ISM with point spread function (PSF) engineering to reconstruct 3D images from single 2D scanning [4–6]. Jesacher et al. [4] introduced a double-helix phase mask in the emission path of ISM, and this approach was termed RESCH (Refocusing after Scanning using Helical phase engineering) microscopy, which allows for post-acquisition refocusing within an axial range of roughly 400 nm. However, this image reconstruction method leads to a lower image resolution than the conventional ISM. For improving the spatial resolution of RESCH microscopy, Roider et al. [5] presented a multi-view deconvolved RESCH (MD-RESCH) to achieve a 20% resolution advantage along all three axes compared to RESCH. Furthermore, Roider et al. [6] also presented a generalized pupil-phase engineering for ISM, denoted as eISM (engineered ISM), by sculpturing the excitation and detection PSFs into helical shapes of opposing handedness to collect 3D information from a single 2D scan without the requirement of physical refocus within an axial range of 3 µm. However, the reconstruction algorithms in these approaches are complicated, and hundreds of iterative calculations are needed. In addition, the major drawback of the above mentioned methods is still the slowness of the 3D imaging. At each scan position of the single laser focus, an image of the excited region has to be recorded, limiting the scan speed by the frame rate of the imaging camera used. So, the long acquisition time of recording 3D information restricts the application of these methods in life sciences.
Recently, a single-shot 3D imaging system with a DH-PSF was proposed [7,8] to realize fast 3D imaging. However, in this approach, the lateral resolution of the depth estimation is restricted by light diffraction and the cepstrum-based reconstruction algorithm. Meanwhile, multifocal structured illumination microscopy (MSIM) [9] uses sparse 2D excitation patterns and post-processing to obtain optically sectioned images with double resolution. With this system, ISM images can be obtained with excellent speed for 2D imaging. However, this approach requires z-stack scanning for 3D imaging, which limits the system’s 3D imaging speed.
In order to solve the above mentioned problem, here, we present a hybrid microscopy technique, MSIM using helical phase engineering (MSIMH), which combines the resolution-doubling capabilities of the MSIM with the post-acquisition refocusing of the RESCH. The technique is inspired by the concepts underlying MSIM and RESCH. For RESCH, since the specimen is moved over a scanning step distance, recording a single 2D image at each illumination location is far too slow for live cellular beings, and the image reconstruction method does not yield a suitable lateral resolution. The speed of image acquisition can be dramatically increased when multiple diffraction-limited excitation spots are used simultaneously and the whole FOV is recorded; consequently, MSIMH uses sparse 2D excitation patterns generated with a digital micromirror device (DMD) integrated into conventional wide-field microscopy in the excitation path. In the emission path, a DH diffraction mask is introduced to modulate the conventional PSF to the double-helix PSF (DH-PSF) [10–12]. After post-processing, we reconstruct the all-in-focus image and its depth information from a single multi-spot 2D planar scan. Finally, we verify the imaging performance by observing fluorescent beads, mitochondria of HeLa cells, and bovine pulmonary artery endothelial (BPAE) cells.
2. Setup and image formation
2.1 Experimental setup
The optical configuration of the MSIMH is shown in Fig. 1. The excitation beam from a solid-state laser source (Coherent, Sapphire 488-200 CW CDRH) is expanded by a factor of with a 4f system constructed from two achromatic lenses (Thorlabs, f = 10 mm and f = 60 mm). The collimated beam is subsequently directed onto a DMD (Digital Light Innovations, D4100 DLP) that is 24° off-normal, such that in its “on” position, the micromirror tilts the output beam normal to the DMD face. Subsequently, a 4f configuration (Thorlabs, f = 75 mm and f = 75 mm) images the DMD face at the back focal plane of a lens (Thorlabs, f = 300 mm), and an optical iris is placed at the Fourier plane of the 4f system to block the undesired diffraction orders of the excitation light generated by the DMD. Each DMD pixel of size is demagnified by a factor of by the telescope system consisting of a lens (f = 300 mm) and objective (Nikon,, NA = 1.27) to dimensions of in the specimen plane.
Fig. 1 Optical configuration of multifocal structured illumination microscopy with helical phase engineering (MSIMH).
In the emission path, the emitted fluorescence from the specimen is collected by the same objective (Nikon, NA = 1.27). Next, the DH phase mask is placed in the Fourier plane of a 4f system, and it is optically conjugated to the objective back focal plane to modulate the pupil phase of the emission light. Here, we note that high-efficiency phase-only masks can provide mode conversion efficiencies of nearly 90%. Finally, the specimen is imaged by a scientific CMOS (sCMOS) camera (ORCA Flash 4.0 V2).
2.2 Generation of double-helix PSF and depth estimation
The DH-PSF is an engineered PSF that has two transverse lobes that rotate about the optical axis with image defocus. It can provide accurate depth estimation over a large axial working range, and it is widely used for particle tracking, superlocalization, and fluorescence imaging in 3D. In this context, researchers have proposed several methods to obtain the DH-PSF [13–15]. In this study, the DH phase mask was designed by use of the method described in [13]. When the mask is introduced in the emission path, it transforms the conventional PSF into a DH-PSF. Figure 2(a) shows the phase distribution of the mask. In our study, before biological experiments, fluorescent beads (T8872, Thermo Fisher Scientific Inc., USA) were attached to a glass coverslip and immersed in water. Subsequently, this sample was placed on a 3D linear translation sample stage and moved along the Z-axis with a step size of 100 nm. The relationship between rotation angle θ and object distance Z was calibrated by successive imaging, as shown in Figs. 2(b, c). We note here that this relationship remains linear over a large range, and the corresponding depth range that can be utilized is about 5.5 μm. Beyond this range, the shape of the DH-PSF undergoes distortion, which can drastically decrease the precision of depth estimation.
Fig. 2 (a) Phase distribution of double helix (DH) phase mask. (b) Intensity distribution of the DH point spread function (DH-PSF) at different positions along Z-axis. (c) Relationship between the two lobe rotation angles of the DH-PSF and position of Z-axis.
2.3 Image formation
Figure 3(a) shows the sparse 2D periodic lattice of the excitation foci as generated by the DMD. A black pixel indicates the “on” state of the micromirror and a white one represents the “off” status. A single illumination spot corresponds to DMD mirror pixels in the “on” position. As the pattern of the DMD is switched “on”, the pixels of the “on” position follow the red line to move a pixel distance of 120 nm along the sample plane. The distance between the scan points in the DMD is set to 30 px (horizontal) and 26 px (vertical) to avoid overlap of two adjacent DH-PSFs. The resulting 780 raw exposures taken at 125 Hz (for ourFOV) correspond to a 6 s 3D volumetric information acquisition rate. The fluorescence image (at the sample plane) of the excitation foci in a uniform solution of Rhodamine 6G is shown in Fig. 3(b). Here, we note that the distance between the illumination points will affect the acquisition speed and image quality. Widely spaced foci afford less cross-talk, but correspond to more scan steps, and therefore, a suitable distribution of illumination lattice yields a good imaging result. As the DH phase mask is introduced in the detection path, the standard PSF of the system is converted into the DH-PSF, which comprises a double-Gaussian point distribution along the horizontal direction, as shown in Fig. 3(c). If the sample moves away from the focal plane, the DH-PSF rotates around the midpoint between the two double-Gaussian points. When all preloaded modes are switched once, the entire specimen in the FOV is fully illuminated.
Fig. 3 (a) Scanning pattern of digital micromirror device (DMD). (b) The fluorescence image of the excitation foci in a uniform solution of Rhodamine 6G at the sample plane. (c) The fluorescence image of the excitation foci in a uniform solution of Rhodamine 6G at the sample plane with double helix (DH) phase mask.
In our study, next, we processed the acquired raw images to reconstruct the specimen information. For a single set I of the raw images, as shown in Fig. 4(a), the DH spots are located close to each other, and it is difficult to determine the location of the excitation spots in the FOV. To simplify the data processing and improve the computation speed, we removed the DH phase mask and used a uniform fluorescent dye as a reference specimen. The sparse Gaussian spots were detected by the sCMOS. Here, we remark that it is useful to locate the position of each illumination spot in the sample plane and record the coordinates as a reference. After introduction of the DH mask, the raw image I is divided into a series of subareas according to the reference coordinates, and it is ensured that there is only one DH-PSF per subarea. The (1D) expression for the measured intensity using the synthetic pinhole SP in a subarea is:
Here, denote the convolution operator, ρ denotes the specimen fluorophore emission, x is the 1D coordinate in the camera, and h1 and h2 the excitation- and emission-intensity PSFs, respectively, with h2 being a DH. The synthetic pinhole SP, which is generated by double-Gaussian functions with the same standard deviation (σ = 183 nm), to match the lobe positions and reject light on the surrounding pixels; Fig. 4(b) shows the corresponding result, wherein the positions of the two Gaussian points have been determined by Gaussian fitting to the two lobes of the DH-PSF. This step can obtain the rotation angle of each DH spots and calculate the corresponding z-value in each x-y-scanning position. Here, we remark that a well-aligned pinhole can improve the resolution slightly without greatly reducing the signal strength. A smaller pinhole size rejects more out-of-focus light and affords increased contrast, although the very smallest pinhole size will generate a slight artifact. Next, the masked DH-PSFs were recovered by deconvolution with the corresponding estimated PSF, as shown in Fig. 4(c). The shape and distance of the main lobes of the estimated PSF are experimentally measured, and the PSF angle is as same as that of the corresponding DH-PSF in I. In our method, we chose the widely used Richardson–Lucy algorithm [16] as the deconvolution algorithm to recover the engineered PSF. After the DH-PSFs are recovered to the Gaussian point, pixel reassignment is used to improve the lateral resolution as follows: Firstly, the center positions of the spots are located for each frame, a double-size image matrix is constructed, and the raw data from a square area around the spots are copied to the new image matrix at a position with the coordinates multiplied by a factor of two. This procedure “shifts” the information from each pixel by half its distance to the optical axis. Lastly, the final image is “shrunk” by a factor of two. The pixel reassignment result corresponding to Fig. 4(c) is shown in Fig. 4(d).Fig. 4 (a) Single set of raw images. (b) Application of digital pinholes around each double helix point spread function (DH-PSF). (c) Deconvolution of DH-PSF in (b) to conventional PSF. (d) Resultant image after pixel reassignment.
3. Experimental results
To demonstrate the extended depth-of-field afforded by our microscopy, we first used 100-nm-diameter fluorescent beads (T8872, Thermo Fisher Scientific Inc., USA) attached to a glass coverslip and immersed in agar solution as the sample. Figure 5(a) shows the conventional wide-field image of the fluorescent beads without the DH phase mask, while the MSIMH image result for the same FOV is shown in Fig. 5(b). A given square area of interest selected from the image result (indicated by the yellow box) and its magnified view of this square area are shown in Fig. 5(c). The light spots in the wide-field image are spread out as the fluorescent beads are far from the focal plane, while the MSIMH approach provides a brighter and more distinct image of the defocused beads, thereby demonstrating the extended depth-of-field feature of our microscopy approach.
Fig. 5 (a) Fluorescent bead image obtained via wide-field microscopy (b) Fluorescent bead image obtained by multifocal structured illumination microscopy with helical phase engineering (MSIMH). (c) Magnified view of areas of interest corresponding to the box region in (a) and (b).
Next, we examined the imaging capabilities of our system by acquiring images of mitochondria in live HeLa cells. The specimen was labeled with the use of Rhodamine 123 and excited by a 488-nm laser to emit green fluorescence (λ = 530 nm). Figure 6 compares the wide-field image z-stack with data from a single 2D scan as acquired by MSIMH. Figures 6(a) and 6(b) show the wide-field image at z = 0 μm and −1 μm, respectively. In Fig. 6(a), most of the mitochondria in the center of the sample are defocused, and the image appears blurry with very poor resolution of the structure information. In order to obtain information of the defocused portion, the objective lens should be refocused or the sample stage moved (Fig. 6(b). For MSIMH, a single 2D scan was acquired at z = 0 μm; Fig. 6(c) shows the corresponding recovered object. The image exhibits a sharp and bright view of the specimen, and the structure information in defocused layers is also imaged at the focal plane. The different depth areas of the sample are indicated by colors as per the reference color bar provided alongside. In contrast to the conventional wide-field approach, we find that MSIMH can expand the imaging depth in the range from −1 μm to 1 μm. More importantly, MSIMH can afford further improvement of the imaging resolution.
Fig. 6 (a) Wide-field image of mitochondria at z = 0 μm. (b) Wide-field image of mitochondrion at z = −1 μm. (c) Volumetric image obtained via multifocal structured illumination microscopy with helical phase engineering (MSIMH) at z = 0 μm. The MSIMH volumetric image was generated from 780 raw images, each acquired in 8 ms, for a volumetric acquisition time of 6 s.
Next, we sampled a section of bovine pulmonary artery endothelial (BPAE) cells (F36924, Thermo Fisher Scientific Inc., USA) to characterize the lateral resolution of the MSIMH. The F-actin of the cell was labeled with Alexa Fluor 488 phalloidin, which could be efficiently excited by the laser to emit green fluorescence (λ = 520 nm). As regards the conventional wide-field image shown in Fig. 7(a), the fluorescence signal is surrounded by a large amount of background noise from the sample, and its resolution is smaller than the theoretical resolution. Then we used the Richardson-Lucy algorithm to deconvolute the wide-field image with estimated PSF is shown in Fig. 7(b), the spatial resolution of the image is slightly improved and details become more visible. On the other hand, the multifocal-excited DH-PSF data were pin-holed and deconvoluted to the standard PSF, and subsequently, the data were summed as a wide-field image, as shown in Fig. 7(c). This process improves the image resolution and contrast and also recovers the features obscured by diffraction. Next, pixel reassignment was implemented in each of the pin-holed and deconvoluted frames, which step can further improve the resolution. The resulting summed image is shown in Fig. 7(d). Certain areas of interest were selected from the boxed regions in Figs. 7(a)-7(d), whose magnified views are shown in Figs. 7(e) and 7(f). From Fig. 7(e), we note that our MSIMH is able to image two adjacent microfilaments that are difficult to distinguish under wide-field illumination. Figure 7 (g) shows the intensity profiles through colored dashed line in Fig. 7(e). From Fig. 7(e), we can see that the MSIMH has a better resolution capability compared to the conventional wide-field illumination microscopy with deconvolution processing. Finally, the full-width at half maximum (FWHM) intensity of microfilaments corresponding to Fig. 7(f) is shown in Fig. 7(h). These results indicate that the MSIMH achieves nearly twice the resolution of conventional wide-field microscopy and a 1.48 times resolution of deconvolved wide-field microscopy.
Fig. 7 Resolution and contrast enhancement in multifocal structured illumination microscopy with helical phase engineering (MSIMH). (a) Wide-field image of F-actin, (b) deconvoluted wide-field image(c) pin-holed and deconvoluted image, (d) MSIMH image. (e, 1 to 4) Magnification of white box region in (a), (b), (c) and (d). (f, 5 to 8) Magnification of yellow box region in (a), (b), (c) and (d). (g) Plots of intensity along the colored dashed lines in (e). (h) Plots of intensity along the colored lines in (f). The FWHM values are: wide-field, 356 nm; deconvoluted wide-field, 288 nm; pin-holed and deconvoluted image 280 nm; and MSIMH, 194 nm.
As demonstrated above, this imaging method is capable of obtaining the volumetric image of the specimen from a single multi-spot 2D planar scan. With a special deconvolution strategy, this microscope is more suitable for studying the sparsely distributed subcellular structures, such as microtubules, mitochondria, neurons etc, in extended depth-of-field.
4. Conclusions
In our study, we developed a multifocal scanning microscopy approach capable of obtaining 3D information of a specimen from a single 2D scan. This method combines MSIM with a DH phase mask in the emission path, which affords extended depth-of-field images with depth information and higher resolution via the application of adapted synthetic pinholes and data processing. This 3D microscopy method requires lesser 2D scan time, which makes it faster than other image-based scanning method. So, the method offers the benefit of potentially lower phototoxic side effects. In addition, the system can be switched to operate in the MSIM-only mode by removing the phase mask from the path. We believe that our approach can significantly contribute to the further development of multifocal structured illumination microscopy for biological imaging.
Funding
National Basic Research Program of China (2015CB352005, 2017YFA0700500); National Natural Science Foundation of China (NSFC) (61775144, 61525503, 61620106016, 81727804, 61605127); The Guangdong Natural Science Foundation (2018A030313362, 2014A030312008, 2017A030310132); Shenzhen Science and Technology Planning Project (JCYJ20170818141701667, JCYJ20170412105003520, JCYJ20170818144012025, JCYJ20150930104948169, JCYJ20160328144746940, GJHZ20160226202139185, JCYJ20170412105003520, SWCYL20150331010038); China Scholarship Council (201708440486); and China Postdoctoral Science Foundation (010865).
References
1. T. Wilson, Confocal Microscopy (Academic, 1990).
2. C. J. R. Sheppard, “Super-resolution in confocal imaging,” Optik (Stuttg.) 80(2), 53–54 (1988).
3. C. B. Müller and J. Enderlein, “Image scanning microscopy,” Phys. Rev. Lett. 104(19), 198101 (2010). [CrossRef] [PubMed]
4. A. Jesacher, M. Ritschmarte, and R. Piestun, “Three-dimensional information from two-dimensional scans: a scanning microscope with postacquisition refocusing capability,” Optica 2(3), 210–213 (2015). [CrossRef]
5. C. Roider, R. Heintzmann, R. Piestun, and A. Jesacher, “Deconvolution approach for 3D scanning microscopy with helical phase engineering,” Opt. Express 24(14), 15456–15467 (2016). [CrossRef] [PubMed]
6. C. Roider, R. Piestun, and A. Jesacher, “3D image scanning microscopy with engineered excitation and detection,” Optica 4(11), 1373–1381 (2017). [CrossRef]
7. Z. Wang, Y. Cai, Y. Liang, X. Zhou, S. Yan, D. Dan, P. R. Bianco, M. Lei, and B. Yao, “Single shot, three-dimensional fluorescence microscopy with a spatially rotating point spread function,” Biomed. Opt. Express 8(12), 5493–5506 (2017). [CrossRef] [PubMed]
8. R. Berlich, A. Bräuer, and S. Stallinga, “Single shot three-dimensional imaging using an engineered point spread function,” Opt. Express 24(6), 5946–5960 (2016). [CrossRef] [PubMed]
9. A. G. York, S. H. Parekh, D. Dalle Nogare, R. S. Fischer, K. Temprine, M. Mione, A. B. Chitnis, C. A. Combs, and H. Shroff, “Resolution doubling in live, multicellular organisms via multifocal structured illumination microscopy,” Nat. Methods 9(7), 749–754 (2012). [CrossRef] [PubMed]
10. S. R. P. Pavani, A. Greengard, and R. Piestun, “Three-dimensional localization with nanometer accuracy using a detector-limited double-helix point spread function system,” Appl. Phys. Lett. 95(2), 021103 (2009). [CrossRef]
11. G. Grover, S. R. P. Pavani, and R. Piestun, “Performance limits on three-dimensional particle localization in photon-limited microscopy,” Opt. Lett. 35(19), 3306–3308 (2010). [CrossRef] [PubMed]
12. G. Grover, S. Quirin, C. Fiedler, and R. Piestun, “Photon efficient double-helix PSF microscopy with application to 3D photo-activation localization imaging,” Biomed. Opt. Express 2(11), 3010–3020 (2011). [CrossRef] [PubMed]
13. S. R. P. Pavani and R. Piestun, “High-efficiency rotating point spread functions,” Opt. Express 16(5), 3484–3489 (2008). [CrossRef] [PubMed]
14. G. Grover, K. DeLuca, S. Quirin, J. DeLuca, and R. Piestun, “Super-resolution photon-efficient imaging by nanometric double-helix point spread function localization of emitters (SPINDLE),” Opt. Express 20(24), 26681–26695 (2012). [CrossRef] [PubMed]
15. C. Roider, A. Jesacher, S. Bernet, and M. Ritsch-Marte, “Axial super-localisation using rotating point spread functions shaped by polarisation-dependent phase modulation,” Opt. Express 22(4), 4029–4037 (2014). [CrossRef] [PubMed]
16. W. H. Richardson, “Bayesian-based iterative method of image restoration,” J. Opt. Soc. Am. 62(1), 55–59 (1972). [CrossRef]
