Open Access
Add to BibSonomy
Abstract
Super-resolution microscopy has revolutionized the field of biophotonics by revealing detailed 3D biological structures. Nonetheless, the technique is still largely limited by the low throughput and hampered by increased background signals for dense or thick biological specimens. In this paper, we present a pixel-reassigned continuous line-scanning microscope for large-scale high-speed 3D super-resolution imaging, which achieves an imaging resolution of 0.41 µm in the lateral direction, i.e., a $\sqrt 2 $× resolution enhancement from the raw images. Specifically, the recorded line images are first reassigned to the line-excitation center at each scanning position to enhance the resolution. Next, a modified HiLo algorithm is applied to reduce the background signals. Parametric models have been developed to simulate the imaging results of randomly distributed fluorescent beads. Imaging experiments were designed and performed to verify the predicted performance on various biological samples, which demonstrated an imaging speed of 3400 pixels/ms on millimeter-scale specimens. These results confirm the pixel-reassigned line-scanning microscopy is a facile and powerful method to realize large-area super-resolution imaging on thick or dense biological samples.
© 2024 Optica Publishing Group under the terms of the Optica Open Access Publishing Agreement
1. Introduction
Super-resolution microscopy is a crucial technology in life science to visualize and quantify structures and processes at sub-diffraction limit resolution. For example, single molecular localization imaging techniques, such as stochastic optical reconstruction microscopy [1] and photoactivated localization microscopy [2], as well as stimulated emission depletion microscopy [3], have improved the imaging resolution to tens of nanometers. However, the ultrahigh resolution is respectively achieved at the expense of low imaging speed and phototoxicity. These limitations restrict their application in long-term large-scale high-speed super-resolution imaging. In comparison, structured illumination microscopy (SIM) achieves modest resolution enhancement with higher photon efficiency, imaging rate, and lower phototoxicity [4].
Traditional SIM generates multiple phase-shifted structured images, which carry high-frequency information, obtained in different orientations via interference to reconstruct a super-resolved image [4,5]. To address the issue of high background signals or noises in thick or dense biological samples, techniques, such as total internal reflection fluorescence SIM (TIRF-SIM) [6] and grazing incidence SIM (GI-SIM) [7], employ a confined and laminated illumination to mitigate background signals. However, the surface-bounded excitation in TIRF-SIM limits imaging depth, and the tradeoff between depth of focus and thickness of the excitation beam in GI-SIM limits the effective field of view. Other background subtraction methods, such as LiMo [8] and HiLo [9–11] algorithms, have shown promising results for fast volumetric imaging but the resolution is diffraction-limited.
Pixel reassignment-based SIM, i.e., imaging scanning microscopy (ISM), has emerged as a promising super-resolution technique in recent years [12]. ISM can simultaneously enhance the lateral resolution by a factor of $\sqrt 2 $ and reject out-of-focus background signals without in-focus photon loss through the use of a pinhole or light sheet illumination [13]. The principle of ISM is to reassign the detected photons to the center of each scanning position. Compared to traditional SIM, the pixel reassignment strategy is more robust and reliable, with a similar level of resolution enhancement.
Typical ISM systems have a serial scanning configuration (e.g., using a pair of galvanometric mirrors) and suffer from low imaging speed. To address the issue, parallel scanning ISM have been developed by utilizing multiple foci generated by a digital micromirror device [14] or spinning disk [15], which achieved higher imaging speeds at the expense of increased computational complexity and the need to identify multiple center positions for reassigning pixels. [16]. As such, real-time processing becomes challenging. While instant SIM addressed this issue with parallelized optical pixel reassignment [17], it requires a delicate balance between crosstalk and the degree of parallelization and further complexes the system design. Recently, Wu et al. combined line-illuminations and pixel reassignment in a multi-view confocal microscopy system to achieve isotropic super-resolution [18].
In this paper, we present pixel-reassigned line-scanning microscopy (PRLM) to achieve continuous large-scale super-resolution 3D imaging, where the recorded line images are reassigned to the line-excitation center at each scanning position to enhance the resolution, followed by the application of the HiLo algorithm to remove background signals. The simple design of PRLM makes it a facile, compact, low-cost solution for imaging dense and thick biological samples with super-resolution.
2. Principle
2.1 Pixel reassignment for a continuous line-scanning system
Figure 1(a) presents the optical configuration of the PRLM, where a 491-nm continuous-wave laser is the light source (CalypsoTM, Cobolt). The laser is first expanded by two lenses (L1, f = 7.5 mm, AC050-008-A, Thorlabs and L2, f = 250 mm, AC254-250-A, Thorlabs), and subsequently shaped into a line by a cylindrical lens (CL, f = 150 mm, ACY254-150-A, Thorlabs). Next, the line illumination is relayed to the focal plane via a 4-f system, i.e., L3 (f = 200 mm, AC254-200-A, Thorlabs) and the objective lens (OL, S Plan Fluor 40×/NA 0.6, Olympus), as a tightly focused line. The specimen is placed on a precision XYZθZ stage (XY: PLANARDL-300XY, Z: WAFERMAXZ, θZ: ACS-100LP, Aerotech) for continuous scanning. The emission is collected by the OL and separated from the excitation laser via a long-pass dichroic mirror (DM, ZT488rdc, Chroma), a band-pass emission filter (EF, ET525/50m, Chroma), and lastly focused by a doublet lens (L4, f = 200 mm, AC254-200-A, Thorlabs) to an sCMOS camera (ORCA-Flash 4.0, Hamamatsu). The camera is used as a line-scan detector in the sub-array readout mode with a maximum frame rate of 12.8 kHz.
Fig. 1. Principle of pixel-reassigned line-scanning microscopy (PRLM). (a) System configuration. M1-M2: high reflectivity mirrors; DM: dichroic mirror; EF: emission filter; L1-L4: lenses; CL: cylindrical lens; OL: objective lens. (b) Scanning sequence for large specimens. (c) Pixel reassignment of detected emissions by a factor of m. (d) Image fusion and HiLo algorithm: pixel-reassigned x-line-scanning images (xPR) and y-line-scanning images (yPR) are registered and subsequently combined using the wavelet-based image fusion method to generate an isotropic and super-resolved image (uPR). Next, modulated in-focus signals in xPR and yPR are extracted to obtain demodulated images (dPR). By applying a high-pass filter to uPR and a low-pass filter to dPR and combining the results, the final image (PRHiLo) with enhanced contrast can be obtained.
As illustrated in Fig. 1(b), when imaging a large specimen, the specimen is to be scanned continuously in the vertical direction (i.e., x-axis); after the first stripe is imaged; the specimen is moved to the left (i.e., y-axis) to perform continuous scanning for the second stripe; this process repeats until the entire sample is imaged. Next, the specimen is rotated 90° to repeat the continuous imaging again for the entire specimen. The vertical and horizontal scanning and imaging of the specimen will allow the pixel-reassignment algorithm to enhance the imaging resolution in both lateral directions. During line-scanning, the stage moves at a constant speed; and the exposure time for each pixel is determined by the ratio between the pixel size (i.e., 162.5 nm) and stage velocity. As such, the maximum scanning speed of the system is calculated to be 162.5 nm × 12.8 kHz = 2.08 mm/s.
To perform pixel reassignment, the detected emission is reassigned to the center of the excitation line by a factor of m (Fig. 1(c)). The optimal value of m is determined by the point spread function (PSF) of both the excitation and emission signals. Conventionally, in an optical system of a circular pupil, the PSF of the excitation and emission can be described by a Gaussian function [12], which is expressed as
2.2 Wavelet-based image fusion
As mentioned in Section 2.1, pixel-reassigned line-scning is performed in both vertical and horizontal directions to achieve isotropic resolution enhancement; and the resulting pixel-reassigned images, namely xPR (vertical direction or x-axis) and yPR (horizontal direction or y-axis), are combined via a wavelet-based fusion approach. As shown in Fig. 1(d), firstly, xPR and yPR are registered to eliminate the misalignment [19]. Next, the wavelet-based fusion algorithm is applied to extract the information in the frequency domain to reconstruct a super-resolved image. Specifically, the fusion process includes the following two steps:
- (1) Perform discrete wavelet transform (DWT) to apply low- and high-pass filters in the horizontal and vertical directions [20], which yields four sub-frequency bands, including low-frequency components in horizontal and vertical directions (i.e., LL), low-frequency components in the horizontal direction and high-frequency components in the vertical direction (i.e., LH), high-frequency components in the horizontal direction and low-frequency components in the vertical direction (i.e., HL), and high-frequency components in horizontal and vertical directions (i.e., HH). This process is mathematically described in Eq. (2) and (3), where xLL, xLH, xHL, xHH, and yLL, yLH, yHL, yHH represent the four coefficients of xPR and yPR, respectively.
- (2) Fuse the information from both axes to generate a super-resolved image via Eq. (4). Specifically, the minimum value of xLL and yLL coefficients, xLH, yHL, and the maximum value of xHH and yHH coefficients are combined into a matrix, followed by an inverse DWT to generate a super-resolved image with isotropic lateral resolution (i.e., uPR).
2.3 Suppression of background signals via HiLo algorithm
To suppress the background signals and enhance image contrast, a modified HiLo algorithm [11] is developed and applied to the fused pixel-reassigned images (uPR). As only in-focus signals are modulated by the line illumination in our system, we can calculate the demodulated images (dPR) by Eq. (5),
3. Simulation
In this section, we first performed numerical simulations in MATLAB based on a parametric scalar model [21] to predict the performance of the PRLM. In the simulations, the numerical aperture (NA), refraction index, excitation and emission wavelengths were set to 0.6, 1.000 (air), 491 nm, and 520 nm, respectively, which are consistent with our experimental parameters; and the PRLM was to image randomly distributed 80-nm fluorescent beads. To verify the effectiveness of pixel reassignment and HiLo algorithm for PRLM, we first simulated the imaging of sparsely distributed beads, where the slit size was set to 1.6 µm, which correspond to 10 pixels on the sCMOS camera. The results are presented in Fig. 2(a) - 2(e), where one may observe that with the application of pixel reassignment to the x-line-scanning images (xLS), xPR are generated, showing a noticeable resolution improvement by a factor of ∼1.4 in the x-axis and 1.2 in the z-axis (Fig. 2(d) and 2(e)). Notably, two beads that are originally non-separable in the xLS become distinguishable in the xPR along the x-axis and in the uPR in the two lateral directions. Next, we simulated the imaging of densely distributed beads. As shown in Fig. 2(f)–2(k), the resolution is dramatically reduced due to the increased background signals from the out-of-focus beads. By subtracting the background signals based on the modified HiLo algorithm, PRHiLo (η = 4) (Fig. 2(i)) were generated with improved resolving capabilities of 0.32 µm and 1.6 µm in the lateral and axial directions, respectively, restoring the predicted resolution level, demonstrated in the sparse case.
Fig. 2. Simulated imaging of sparse and dense fluorescent beads. (a-c) Simulated cross-section views of the sparsely distributed beads, including x-line-scanning images (xLS, slit width = 1.6 µm), xPR, and uPR in the x-y and x-z plane. (d, e) Comparison of the normalized intensity distributions along the x-axis and z-axis for results in (a-c); the corresponding full width at half maximum (FWHM) of the imaged beads in the upper right corners are 0.45 µm, 0.30 µm and 0.31 µm along the x-axis and 2.3 µm, 1.9 µm and 1.8 µm along the z-axis for xLS, xPR, and uPR respectively. (f-i). Simulated cross-section views of the densely distributed beads, including xLS, xPR and PRHiLo in the x-y and x-z plane, respectively. (j, k) Normalized intensity distributions of the dashed lines in (f-i) along the x-axis and z-axis, respectively. The FWHMs of the beads in PRHiLo is 0.32 µm along the x-axis and 1.6 µm along the z-axis. Scale bar: 1 µm.
4. Experimental characterization
In the experiments, PRLM was used to image 200-nm fluorescent beads (F8848, Thermo Fisher) over an imaging volume of 338 × 338 × 10 µm3 with an axial scanning step of 200 nm. The results are presented in Fig. 3, where one may find them to be consistent with the simulations. Figure 3(a)–3(c) present intensity profiles, i.e., PSF, of a randomly selected fluorescent bead without (Fig. 3(a)) and with pixel reassignments (Fig. 3(b) and 3(c)), where both the lateral and axial PSFs have been effectively improved for the case of uPR. Figure 3(d) and 3(e) plot the lateral and axial PSF profiles of Fig. 3(b) and 3(c), respectively. From Fig. 3(d), it is found that the FWHMs along the x-axis and y-axis are 0.41 µm and 0.59 µm respectively, which correspond to with and without pixel reassignments. As the ratio between 0.59 and 0.41 is 1.4, the resolution enhancement is consistent with the theory prediction. (Note that after horizonal scanning (y-axis) and image fusion, the final image, i.e. uPR, exhibits isotropic lateral resolution as can be seen in Fig. 3(c).) In Fig. 3(e), one may find the axial resolution is also effectively improved after pixel reassignment, where the FWHMs of xLS and uPR along the z-axis are 5.0 µm and 3.3 µm, respectively.
Fig. 3. Characterizing the PSF of PRLM. (a-c) Cross-section views of the imaged fluorescent bead in the x-y and x-z plane for xLS, xPR and uPR, respectively. (d) Normalized intensity distribution and least-squares fitted Gaussian profiles of the fluorescent bead along the x-axis and y-axis as indicated by the dashed lines in (b); the FWHMs along the x-axis and y-axis represent images without and with pixel reassignment, which are found to be 0.59 µm and 0.41 µm, respectively. (e) Normalized intensity distributions and their Gaussian fitted profiles along the z-axis shown in (a) and (c); the corresponding FWHMs are 5.0 µm and 3.3 µm, respectively. Scale bar: 1 µm.
5. Biological applications
Lastly, we applied PRLM to image different biological samples, including U-2 OS cells and pollen grains. Figure 4 presents the imaging results of U-2 OS cells with microtubes (labeled by Tubulin-Atto 488) over a large imaging field of 1.5 × 1.5 mm2. The total imaging time is 25 sec, including the scanning of five strips in both the horizontal and vertical directions, as well as the moving and stitching of the sample. As shown in Fig. 4(a) and 4(b), the resolution of the images processed by the pixel reassignment and HiLo algorithm is significantly improved. (Note above and below the dashed line in Fig. 4(b) shows processed and raw line-scanning images.) Fig. 4(c)–4(e) present zoom-in xLS, xPR, and PRHiLo (η = 7.0) images from the dashed box in Fig. 4(b), where one may observe that xPR show improved lateral resolution; and PRHiLo show improved optical-sectioning results in comparison to xLS. The intensity profiles along the dashed lines in Fig. 4(c)–4(e) are plotted in Fig. 4(f) for better comparison, which are consistent with the observations in Fig. 4(c)–4(e).
Fig. 4. Imaging microtubes of U-2 OS cells via PRLM. (a) Cross-section view in the x-y plane for xLS. (b) Enlarged view of the white box in (a), where xLS and PRHiLo imaging results are shown below and above the dashed line for easy comparison. (c-e) Enlarged cross-sectional views of the dashed box in (b) for xLS, xPR, and PRHiLo, respectively. (f) Normalized intensity profiles along the dashed lines indicated in (c-e) for xLS, xPR, and PRHiLo, respectively.
Figure 5(a) presents 3D imaging results of a pollen grain sample (304264, Carolina Biological Supply) with an imaging volume of 105 × 105 × 20 µm3. Note that for ease of comparison, the raw line-scanning images (xLS) and the processed images (PRHiLo) are shown below and above the dashed line in Fig. 5(a). Figure 5(b) and 5(c) present zoom-in optical cross-sectional views for the selected pollen in the dashed box of Fig. 5(a) at different depths, where one may find that after processing with the pixel reassignment algorithm, the spikes on the pollen show an improved resolution. After further applying the HiLo algorithm for background subtraction, PRHiLo (η = 8.5), the results show significantly improved optical-sectioning capability and image contrast. The intensity profiles across the yellow, blue, and orange dashed lines in Fig. 5(a)–5(c) are plotted in Fig. 5(d) to quantify the resolution enhancement. These results are consistent with Fig. 4 and demonstrate the practicality of using PRLM for high-speed, high-resolution 3D imaging.
Fig. 5. Imaging pollen grains via PRLM. (a) 3D imaging results of a pollen grain sample in a color-coded 2D map, where the raw images (xLS) and the processed images (PRHiLo) are respectively shown below and above the dashed line; (b, c) Enlarged cross-sectional views of a pollen grain at two different depths (z = 4 µm and z = 8 µm) in the dashed box of (a) for xLS, xPR, and PRHiLo. (d) Normalized intensity profiles of the yellow, blue, and orange dashed lines in (a). Scale bar: 10 µm.
6. Conclusion
In conclusion, we have presented the design and characterization of a line-scanning microscope for continuous high-speed 3D imaging. By applying the pixel reassignment algorithm, wavelet-based image fusion algorithm, and modified HiLo algorithm to the line-scanning platform, we demonstrated an imaging resolution of 0.41 µm and imaging speed of 3400 pixels/ms. First, simulated and imaging experiments on fluorescent beads were performed to characterize the PSF of the PRLM. Next, biological imaging experiments on microtubes of U-2 OS cells and pollen grains were performed to demonstrate the 3D imaging capability with improved resolution and optical-sectioning capability. These results verify that PRLM is a facile and powerful tool for visualizing large-area biological samples with super-resolution, which may find important applications in biophotonics.
Funding
Research Grants Council, Collaborative Research Fund (C5031-22GF); Innovation and Technology Commission, Innovation Technology Fund (ITS/222/21FP); Hong Kong Center for Cerebro-Cardiovascular Health Engineering (COCHE-1.5).
Disclosures
The authors declare no conflicts of interest.
Data availability
Data underlying the results presented in this paper are available upon reasonable request.
References
1. M. J. Rust, M. Bates, and X. Zhuang, “Sub-diffraction-limit imaging by stochastic optical reconstruction microscopy (STORM),” Nat. Methods 3(10), 793–796 (2006). [CrossRef]
2. E. Betzig, G. H. Patterson, R. Sougrat, et al., “Imaging intracellular fluorescent proteins at nanometer resolution,” Science 313(5793), 1642–1645 (2006). [CrossRef]
3. G. Vicidomini, P. Bianchini, and A. Diaspro, “STED super-resolved microscopy,” Nat. Methods 15(3), 173–182 (2018). [CrossRef]
4. Y. Wu and H. Shroff, “Faster, sharper, and deeper: structured illumination microscopy for biological imaging,” Nat. Methods 15(12), 1011–1019 (2018). [CrossRef]
5. X. Chen, S. Zhong, Y. Hou, et al., “Superresolution structured illumination microscopy reconstruction algorithms: a review,” Light: Sci. Appl. 12(1), 172 (2023). [CrossRef]
6. D. Li, L. Shao, B. C. Chen, et al., “Extended-resolution structured illumination imaging of endocytic and cytoskeletal dynamics,” Science 349(6251), aab3500 (2015). [CrossRef]
7. Y. Guo, D. Li, S. Zhang, et al., “Visualizing Intracellular Organelle and Cytoskeletal Interactions at Nanoscale Resolution on Millisecond Timescales,” Cell 175(5), 1430–1442.e17 (2018). [CrossRef]
8. Q. Zhong, A. Li, R. Jin, et al., “High-definition imaging using line-illumination modulation microscopy,” Nat. Methods 18(3), 309–315 (2021). [CrossRef]
9. Z. Fu, J. Chen, G. Liu, et al., “Single-shot optical sectioning microscopy based on structured illumination,” Opt. Lett. 47(4), 814–817 (2022). [CrossRef]
10. W. Lin, D. Wang, Y. Meng, et al., “Multi-focus microscope with HiLo algorithm for fast 3-D fluorescent imaging,” PLoS One 14(9), e0222729 (2019). [CrossRef]
11. D. Lim, T. N. Ford, K. K. Chu, et al., “Optically sectioned in vivo imaging with speckle illumination HiLo microscopy,” J. Biomed. Opt. 16(1), 016014 (2011). [CrossRef]
12. I. Gregor and J. Enderlein, “Image scanning microscopy,” Curr. Opin. Chem. Biol. 51, 74–83 (2019). [CrossRef]
13. L. Qiao, H. Li, S. Zhong, et al., “Laterally swept light-sheet microscopy enhanced by pixel reassignment for photon-efficient volumetric imaging,” Adv. Photonics Nexus 2(1), 016001 (2022). [CrossRef]
14. A. G. York, S. H. Parekh, D. Dalle Nogare, et al., “Resolution doubling in live, multicellular organisms via multifocal structured illumination microscopy,” Nat. Methods 9(7), 749–754 (2012). [CrossRef]
15. O. Schulz, C. Pieper, M. Clever, et al., “Resolution doubling in fluorescence microscopy with confocal spinning-disk image scanning microscopy,” Proc. Natl. Acad. Sci. U. S. A. 110(52), 21000–21005 (2013). [CrossRef]
16. J. E. McGregor, C. A. Mitchell, and N. A. Hartell, “Post-processing strategies in image scanning microscopy,” Methods 88, 28–36 (2015). [CrossRef]
17. A. G. York, P. Chandris, D. D. Nogare, et al., “Instant super-resolution imaging in live cells and embryos via analog image processing,” Nat. Methods 10(11), 1122–1126 (2013). [CrossRef]
18. Y. Wu, X. Han, Y. Su, et al., “Multiview confocal super-resolution microscopy,” Nature 600(7888), 279–284 (2021). [CrossRef]
19. M. Guizar-Sicairos, S. T. Thurman, and J. R. Fienup, “Efficient subpixel image registration algorithms,” Opt. Lett. 33(2), 156–158 (2008). [CrossRef]
20. G. Pajares and J. Manuel de la Cruz, “A wavelet-based image fusion tutorial,” Pattern Recognit. 37(9), 1855–1872 (2004). [CrossRef]
21. E. Remacha, L. Friedrich, J. Vermot, et al., “How to define and optimize axial resolution in light-sheet microscopy: a simulation-based approach,” Biomed. Opt. Express 11(1), 8–26 (2020). [CrossRef]
