Open Access
Add to BibSonomy
Abstract
Multifocal structured illumination microscopy (MSIM) is the parallelized version of image scanning microscopy (ISM) that is created by using many excitation spots, which provides a two-fold resolution enhancement beyond the diffraction limit with a frequency of 1 Hz per 3D picture, but scattered and out-of-focus light in thick samples degrades MSIM optical sectioning performance. Herein, we introduce a new optical sectioning method in MSIM via illumination fluctuation. The proposed method suppresses the out-of-focus light by taking full advantage of the statistic property of MSIM raw data and has no requirement of changing the system setup or projecting more illumination patterns. Experimental results demonstrate that the method can reduce the out-of-focus light by 7.25 times in optical sectioning image.
© 2020 Optical Society of America under the terms of the OSA Open Access Publishing Agreement
1. Introduction
For the excellent ability in optical sectioning, confocal laser scanning microscopy (CLSM) [1,2] has been widely applied in the bio-imaging community to obtain clear images of three-dimensional (3D) fluorescent organelles or tissues. Usually, it involves raster-scanning the samples with a single laser focal spot, and using pinholes to attenuate the defocus signal. By using a smaller pinhole, CLSM can obtain improved lateral resolution beyond the diffraction limitation, but at the expense of reducing the signal-to-noise ratio (SNR). Living specimens are often damaged by the high-intensity excitation light, and temporal changes in a living specimen are often too rapid to be imaged by CLSM. In the last decades, much effort has been devoted in this field to acquire higher spatial resolution. By combining conventional confocal-laser scanning microscopy with fast wide-field CCD detection, Muller et al. proposed image scanning microscopy (ISM) [3], which doubles the lateral optical resolution without the loss of SNR. With a similar microscope setup, the virtual k-space modulation optical microscopy (Vikmom) [4] also improve the lateral resolution by a factor of two through frequency modulation. However, their imaging speed are still slow to observe many dynamic process of live cellular organelles.
York et al. proposed multi-spot array scanning imaging method, called multifocal structured illumination microscopy (MSIM) [5], in which digital micromirror device (DMD) is applied to rapidly project sparse multi-spot patterns, the multifocal-excited images are then captured. These images will be pinholed, re-scaled, summed and deconvoluted to improved resolution and contrast. With all these efforts, MSIM can obtain optically sectioned images with ∼145 nm lateral and ∼400 nm axial resolution at ∼1Hz frame rates. However, the parallelized excitation and detection also cause problems in MSIM, including the loss of signal as well as increased background associated with the inability to properly assign scattered light and deep defocus signals when scanning through a thick fluorescent sample [6]. Although the deconvolution technique can help to reduce the background [5,7], it however has structure-dependence performance, mainly increasing the image contrast for the sparse samples but showing little effect on the dense samples. Dussaux et al. introduced a differential multipoint-scanning confocal imaging [8], which weakens the residual out-of-focus background by subtracting conventional wide-field image from the multipoint-scanning confocal image. Nevertheless, this method may degrades the focus signal and SNR.
In this work, an illumination fluctuation based optical sectioning (OS) imaging method is presented to suppress the out-of-focus light in MSIM images. Because the fluorophores locates at the focus and out-of-focus planes respectively will make differential responses to the changed exciting light intensities, the OS image can be obtained by computing the variation of emission light with a sequence of shifted illumination patterns, as applied in structured illumination microscopy (SIM) [9,10] and dynamic speckle illumination (DSI) [11] microscopy. MSIM also stimulates fluorophores with a number of sparse multi-spot patterns, and the recorded MSIM raw images thus can be used to calculate OS image as well. Compared with the differential multipoint-scanning confocal imaging, our proposed method has no requirement to simultaneously capture additional wide-field images, and does not decrease the focus signal and SNR. The much better background suppressing ability of our proposed method than that of digital pinhole array has been demonstrated with experimental results.
2. Methods
2.1 System setup
The typical optical configuration of the MSIM is shown in Fig. 1. The excitation beam is expanded with two achromatic lenses (Lens 1 and Lens 2). The collimated beam is then directed onto DMD. Subsequently, DMD splits the excitation beam into multiple beamlets by controlling the state of each micromirror with the prestored digital patterns. An optical iris is used to block the undesired diffraction light generated by the DMD. After passing through the tube lens, dichroic mirror and objective, the multi-spot array excitation beam illuminates the sample finally. In the emission path, the emitted fluorescence from the specimen is collected by the same objective. The emission light is imaged by a scientific CMOS (sCMOS) camera.
2.2 Background generation
In order to image 3D structure of samples with fluorescence microscopy, the out-of-focus light must be suppressed. Although parallelized illumination and detection may significantly improves the temporal resolution, it however also increases the image background notably while a thick sample being scanned [6]. Even the problem is universal in MSIM, its reason does not be explained carefully before. In this paper, the difficulty to reject the out-of-focus light with physical (or digital) pinhole array is illustrated in detail, as shown in Fig. 2.
Fig. 2. Analysis on the optical sectioning ability of pinhole array with fluorophores locating at different out-of-focus distances.
The image of the point source located in the object plane is a 3D diffraction pattern and can be described with 3D model of point spread function (PSF) [12]. It can be deduced from 3D PSF that, compared with the focused fluorophore, the fluorescence of a defocused fluorophore will spread to wider region on the image plane but at a lower intensity level. And this situation will become more severe with the longer off-focus distance, which is illustrated in Fig. 2 (comparing the fluorescence distributions of Fluorophore 1, 2 and 3).
In MSIM, we will stimulate the fluorophores with sparse multi-spot illumination patterns and reject the out-of-focus light through digital pinhole array. As being revealed in Fig. 2, sparse stimulation and digital pinhole array only can guarantee to reject the out-of-focus light of less defocused fluorophores (Fluorophore 2). Comparatively, the deep defocused fluorophore (Fluorophore 3) may generate a much bigger diffraction spot, which radius is larger than the distance between the multiple illumination spots. This out-of-focus light is inevitably collected by the adjacent digital pinholes (as indicated with purple arrows), even the pinhole is nearly closed, and leads to strong background signal in MSIM image. It will decreases the image contrast and greatly reduce the 3D image ability of MSIM. Although the optical sectioning ability can be enhanced by increasing the distance between the illumination spots and digital pinholes, but at the sacrifice of imaging speed.
2.3 Optical sectioning method
Apart from using pinhole (or pinhole array), the OS image also can be obtained in SIM and DSI by using shifted illumination patterns and computing the variation of emission light. In MSIM, the sample is illuminated with shifted multi-spot arrays and each fluorohpore is excited with different light intensities. The multi-spot illumination patterns pi in MSIM can be described as:
After exciting the samples s with the multi-spot illumination patterns pi, we could capture the multi-spot image as:
As seen in Eqs. (1) and (2), with the shifted multi-spot illumination patterns pi, each fluorophore is excited with varied light intensity. Therefore, it is possible to reduce the out-of-focus light by computing the variance of emission fluorescence with MSIM raw images. By referring the SIM technique [9,10], we can also describe the multi-spot illuminated images Ii as follows:
As mentioned in Section 2.1, the out-of-focus signal Iout will occupies a wider area on the focus plane and has a weaker intensity with the increased off-focus distance. It means that, with the same set of shifted illumination patterns, the defocused fluorophore will leads less fluorescence fluctuation on the focal plane than the focused fluorophore. Thus, by computing the variance of MSIM raw images, the out-of-focus signal Iout can be then greatly suppressed and the image contrast is dramatically improved.
Although the variance distribution is also calculated in SIM and DSI to obtain the OS image, their imaging depth however is severely limited (less than 10 µm) for the contrast of illumination patterns being degraded by scatting [13]. In comparison, much higher imaging depth (up to 50 µm) can be attained by using MSIM technique [14]. Thus, we may acquire the OS image with large imaging depth by calculating the variance image in MSIM. To avoid the amplification of the uneven in the sample brightness, we obtain the OS image IOS by computing the standard deviation (SD) $\sqrt {V({{I_i}({\boldsymbol x} )} )}$ in practice. It should be noted that the OS image is formed merely by processing the conventional MSIM raw images, no system changes or more illumination patterns are required in this paper.
3. Experimental results
MSIM raw images are captured and processed to demonstrate the ability of suppressing background in our proposed method. In our experimental system, a solid-state laser source (Coherent, Sapphire 488-200 CW CDRH) is used to generate excitation beam. Each DMD pixel size(10.8 µm×10.8µm) is demagnified by a factor of 90× by the telescope system consisting of a lens (f = 300 mm) and a water objective (Nikon, 60×, NA = 1.27) to dimensions of 120 nm×120 nm in the specimen plane. The fluorescence light was then imaged by a SCMOS (ORCA Flash 4.0 V2) with pixel size of 108 nm (after magnification). All the images were processed in MATLAB (R2015b) on a computer equipped with 8 CPU (Intel Xeon E5-1620v4 @3.5GHz). In this experiment, a total of 224 shifted multi-spot illumination patterns are used to scan samples.
Firstly, a commercial sample (FluoCells prepared slide #3, 16 µm cryostat section of mouse kidney stained with Alexa Fluor 488 WGA) is tested in this paper. A cross-section of renal capsule is imaged with multi-spot illumination patterns [Fig. 3(a)] and the corresponding wide-field is then obtained by summing these multi-spot images [Fig. 3(b)]. Due to the dense structures of the sample, strong out-of-focus light can be observed both in the multi-spot image and the wide-field image. The huge difference of intensity fluctuation between the focus and defocused fluorophores is clearly exhibited in Fig. 3(c). These two signal sequences are collected from two positions of multi-spot images (position 1 and position 2), as shown in Fig. 3(a). With the shifted multi-spot illumination patterns, the recorded emission fluorescence of the focus fluorophores dramatically changes while that of the defocused fluorophores keeps nearly stable.
Fig. 3. Wide-field image, pinholed image and SD image of renal capsule.(a) Multi-spot image. (b) Wide-field image.(c) Intensity fluctuation of focus and defocused signal sequences. (d) Digital pinholed image (with pinhole array). (e) SD image. (f) Normalized intensity profiles along white lines in (b, d and e).
We have quantitatively demonstrated the background suppressing ability of our proposed method by calculating the ratio of mean value (M1/M2 = 1.6) and that of standard deviation (S1/S2 = 11.6) of the two signal sequence. In this experiment, the mean values of the two signal sequences indicate the intensities of the two positions in wide-field image, while the standard deviations mean the intensities in the SD image. Thus, the intensity ratio of focus signal and defocused signal in wide-field image can be obtained by calculating the ratio of mean values (M1/M2). Similarly, that in SD image can be achieved with the ratio of standard deviations (S1/S2). The two ratio values indicate that, by computing SD image, the intensity ratio of focus signal and defocused signal can be improved by 7.25 times. It also means the intensity of out-of-focus light may be weakened by 7.25 times.
The obvious background in pinholed image [seen in Fig. 3(d)] demonstrates its weak ability of background suppressing. Comparatively, by dramatically suppressing background, the image contrast is greatly enhanced in SD image [Fig. 3(e)], especially in the region of interest (ROI). The contrast enhancement (from 0.36 to 0.66) is indicated with the normalized intensity profiles [Fig. 3(f)]. In the experiment, the digital pinhole with Gaussian distribution is applied. Its full width at half maximum (FWHM) is set as the 1.5 times wider than that of the system PSF. Besides, the digital pinhole method spends 7.75 s to obtain the OS image, and the SD image can be calculated within 2.26 s, which demonstrates the higher computational efficiency of our proposed method.
To demonstrate the large imaging depth of our proposed method, we imaged the section sample with different depths. A positioning system with high precision (PI nano XYZ Piezo Stage, P-545.3R7) is used to achieve the step distance of 200 nm. The wide-field image, pinholed image and SD image of different imaging depths (Δz = 3µm, 8 µm and 16 µm) are shown in Figs. 4(a), 4(b), and 4(c). As expected, SD image still has excellent ability of suppressing out-of-light even at the imaging depth of 16 µm. Comparison of the whole 3D imaging results can be seen in Visualization 1.
Fig. 4. Background suppressing with different imaging depths (Δz = 3µm, 8 µm and 16 µm). (a-c) Wide-field image, pinholed image and SD image, respectively.
Although our proposed method can greatly reduce the background signal, it however has the less ability in suppressing the noise than the digital pinhole method, which is illustrated in Fig. 5. In MSIM, multi-spot images are usually captured with short exposure time, and in this case the readout noise (Gaussian noise) is the main noise source. There, the captured multi-spot images which contain small noise in it and corresponding wide-field image, pinholed image and SD image are taken as the reference. The Gaussian noise with different amplitudes (M = K*Im, Im is the maximum intensity of the multi-spot images) are added to the multi-spot images, as shown in Fig. 5(a). Figures 5(b)–5(d) illustrate the obtained wide-field images, pinholed images and SD image with different noise level. The PSNR (peak signal to noise ratio) is calculated to compare the ability of suppressing noise quantitatively, as seen in Fig. 5(e). The wide-filed images and the pinholed images are obtained by superposing the noisy raw images and pinholed noisy multi-spot images respectively, and the noise may be counteracted. Thus, the wide-filed images and the pinholed images contain less noise than the SD images, especially for the high noise level. Nevertheless, the PSNR of SD images is higher than that of raw images [shown in Fig. 5(e)], which means our proposed method still has certain ability of suppressing noise.
Fig. 5. Comparison of the ability of suppressing noise. (a-d) Raw images, wide-field images, pinholed image and SD image with different noise levels, respectively. (e) Noise-PSNR curves.
In digital pinhole method, narrowing the lattice spacing can increase the image efficiency but at the cost of rising background intensity. Comparatively, our proposed method is insensitive to the variance of lattice spacing as demonstrated in Fig. 6. For this character, our proposed method can be applied in MSIM to reduce the number of illumination patterns and increase the image efficiency.
Fig. 6. Reconstructing OS images with different lattice spacing (FluoCells prepared slide #1, F-actin within fixed BPAE cells). (a) Raw images with different lattice spacing (120 images, 224 images and 418 images). (b-c) Obtained wide-field images and SD images.
4. Conclusions
In this paper, we firstly explained the weak ability of pinhole array to suppress background in MSIM image. Then, an illumination fluctuation based OS image generation method is introduced in MSIM to greatly reduce the intensity of out-of-focus light. Experimental results revealed that the intensity of out-of-focus light can be dramatically decreased (by 7.25 times) in SD image at the wide range of imaging depth. With the enhanced ability of rejecting the background signal, our proposed method can be applied to image the 3D structures of thick and dense biological samples. Besides, the proposed method can improve the image efficiency for the insensitivity to the narrowed lattice spacing. It has higher computational efficiency than the digital pinhole method, but less ability in suppressing noise.
Funding
National Natural Science Foundation of China (61835009, 61975131, 61775144, 61525503, 61620106016); Department of Education of Guangdong Province (2016KCXTD007); Natural Science Foundation of Guangdong Province (2014A030312008, 2018A030313362); Shenzhen Technical Project (JCYJ20170412105003520, JCYJ20170818141701667, JCYJ20170818144012025).
Disclosures
The authors declare no conflicts of interest.
References
1. T. Wilson, Confocal Microscopy (Academic, 1990).
2. C. J. R. Sheppard, “Super-resolution in confocal imaging,” Optik (Stuttg.) 80(2), 53–54 (1988).
3. C. B. Müller and J. Enderlein, “Image scanning microscopy,” Phys. Rev. Lett. 104(19), 198101 (2010). [CrossRef]
4. C. F. Kuang, Y. Ma, R. J. Zhou, G. A. Zheng, Y. Fang, Y. K. Xu, X. Liu, and P. T. C. So, “Virtual k-space modulation optical microscopy,” Phys. Rev. Lett. 117(2), 028102 (2016). [CrossRef]
5. A. G. York, S. H. Parekh, D. DalleNogare, R. S. Fischer, K. Temprine, M. Mione, A. B. Chitnis, C. A. Combs, and H. Shroff, “Resolution doubling in live, multicellular organisms via multifocal structured illumination microscopy,” Nat. Methods 9(7), 749–754 (2012). [CrossRef]
6. M. Ingaramo, A. G. York, P. Wawrzusin, O. Milberg, A. Hong, R. Weigert, H. Shroff, and G. H. Patterson, “Two-photon excitation improves multifocal structured illumination microscopy in thick scattering tissue,” Proc. Natl. Acad. Sci. U. S. A. 111(14), 5254–5259 (2014). [CrossRef]
7. F. Ströhl and C. F. Kaminski, “A joint Richardson—Lucy deconvolution algorithm for the reconstruction of multifocal structured illumination microscopy data,” Methods Appl. Fluoresc. 3(1), 014002 (2015). [CrossRef]
8. C. Dussaux, V. Szabo, Y. Chastagnier, J. Fodor, J. F. Léger, L. Bourdieu, J. Perroy, and C. Ventalon, “Fast confocal fluorescence imaging in freely behaving mice,” Sci. Rep. 8(1), 16262 (2018). [CrossRef]
9. K. O. Holleran and M. Shaw, “Optimized approaches for optical sectioning and resolution enhancement in 2D structured illumination microscopy,” Biomed. Opt. Express 5(8), 2580–2590 (2014). [CrossRef]
10. H. Shabani, A. Doblas, G. Saavedra, E. S. Ortiga, and C. Preza, “Improvement of two-dimensional structured illumination microscopy with an incoherent illumination pattern of tunable frequency,” Appl. Opt. 57(7), B92–B101 (2018). [CrossRef]
11. D. Lim, K. K. Chu, and J. Mertz, “Wide-field fluorescence sectioning with hybrid speckle and uniform-illumination microscopy,” Opt. Lett. 33(16), 1819–1821 (2008). [CrossRef]
12. J. B. Sibarita, “Deconvolution microscopy,” Adv. Biochem. Engin. /Biotechnol. 95, 201–243 (2005). [CrossRef]
13. C. Ventalon and J. Mertz, “Quasi-confocal fluorescence sectioning with dynamic speckle illumination,” Opt. Lett. 30(24), 3350–3352 (2005). [CrossRef]
14. Y. Wu and H. Shroff, “Faster, sharper, and deeper: structured illumination microscopy for biological imaging,” Nat. Methods 15(12), 1011–1019 (2018). [CrossRef]
