1. Introduction

The ISM had been experimentally demonstrated in 2010 [1] for fluorescence confocal microscopy, based on theoretical considerations by Sheppard in 1988 [2]. The method allows to double the frequency support of a confocal microscope to perform super-resolution imaging. It consists of sampling the image produced by each point of a raster-scanning confocal microscope rather than just counting the total number of photons, as in classic confocal microscopy. The reconstruction process leads to resolution enhancement where each detected spot is reassigned at a double distance as compared to the actual scan in the sample. Image scanning microscopy is also named photon reassignment [3] or rescan confocal microscopy [4] based on different implementations. Even though it has been used mainly for fluorescence imaging, resolution enhancement can also be achieved for label-free imaging [5]. Particularly, for reflectance confocal microscopy, the reduction by $\sqrt 2 $ of the point spread function (PSF) conducts to a doubled frequency support as compared to regular confocal microscope [6]. It usually leads to resolution doubling even for quasi-periodic structures’ imaging [7].

The original experimental approach of the ISM technique [1] requires numerical reassignment of all PSF-like images obtained on each point in the sample to get a super-resolution imaging effect. The whole process is slow and limits the acquisition rate. Therefore, to speed up the acquisition and simplify the data management, a fully optical photon reassignment can be implemented. The straightforward approach is to optically reassign each collected point on camera by a scanning mirror such as a galvanometer [4]. Then, to improve the scanning speed, a combination of galvanometer and resonant mirror has been proposed [6]. The fastest current versions, developed for fluorescence imaging, rely on parallelized point scanning using micro-lens arrays on spinning disk [8] or in a 2D periodic disposition [9]. 2D imaging at 100 Hz is thus possible but with reduced background rejection compared to single-point scanning. The incoherent nature of fluorescence imaging reduces the choice for optical photon reassignment strategies; switching to label-free imaging opens different strategies.

In this paper, we describe a novel image-scanning microscopy scheme dedicated to label-free imaging with some applications. We demonstrate 2D super-resolution imaging at up to 24kHz without requiring any synchronization between the camera and the microscope optomechanical parts. Our setup is designed to work under the optical photon reassignment regime and is based on the combination of a double-sided resonant mirror [10] and a diffraction grating [11] to form a 2D image. After introducing our fast label-free ISM, we present the characterization of the system in terms of resolution and aberrations. The system is fast and sensitive enough to perform Single Particle Tracking (SPT) in water with nanoparticle size down to 20 nm for gold and 50 nm for silica. Finally, we illustrate the interest of super-resolution label-free imaging capability at high speed on small biological micro-organisms by capturing freely moving Lactobacillus in water.

2. Concept, setup and resolution

A chromatic line is generated in the (y) direction with a spatially coherent white source (super-continuum laser) and a diffraction grating. This approach, known as spectrally encoded confocal microscopy [11–13], allows us to perform instant imaging in one direction. In the label-free configuration, the chromatic line is coherent and elastically backscattered by the sample and can thus be recombined (using the same diffraction grating) into a single point; then it can be filtered by a single pinhole (Fig. 1(a)) for a background rejection equivalent to single-point scanning confocal microscope. And, for fast scanning/de-scanning along the orthogonal (x) direction, a resonant mirror is used. Together, this strategy allows us to perform a 2D scanning at the resonant speed with equivalent de-scan and optimal background rejection capabilities.

 

Fig. 1. (a): Setup schematic. (SCL): Super Continuum Laser. (BS): Beam Splitter. (DG₁, DG₂): Diffraction Grating. (L₁-L₇): Lens. (RM): Resonant Mirror. (SL): Scan Lens. (TL): Tube Lens. (OBJ): Objective. (Ph): Pinhole. (M₁-M₃): Mirrors. (Camera) sCMOS Camera. (b): Example of the chromatic image formed in the image plane (see Visualization 1).

Download Full Size | PDF

Let us discuss how to perform asynchronous optical photon reassignment and achieve analogic super-resolution imaging. After background rejection by the pinhole, we use a second diffraction grating (DG2) with grooves density twice larger than the one in the illumination/descan path (DG1). Each chromatic (PSF) is separated twice as far along the (y) axis compared to the object plane. For the other (x) direction, the light beam is rescanned by the other face of the resonant mirror [10], but with a doubled beam size as compared to the opposite face. This leads to a doubled PSF separation along (x) direction in the final image plane, equivalent to having a doubled rescan amplitude. So the total magnification is doubled in the detector plane while keeping the PSF size unchanged. With this design, the image forms at the speed of the resonant mirror, and photons are reassigned optically for super-resolution. A complete super-resolved image is created analogically at 24kHz, as our resonant mirror oscillates at 12kHz. Nevertheless, the digital image is acquired at 1kHz which is limited by our camera.

The detailed setup is shown in Fig. 1(a). A supercontinuum laser beam (LEUKOS Electro-VIS 430) first goes to a 50:50 beam splitter plate (BS) and then to a blazed diffraction grating (DG1, 150 gr/mm, Edmund optics, 67% efficiency@500 nm) to create a chromatic line using the first order of diffraction. Light is then adapted to the resonant mirror dimension by a 2:1 telescope (L1 = 300 mm, L2 = 150 mm). A double-sided resonant scan (RM, 12kHz, Novanta) is placed before the scan lens (SL, Thorlabs CLS-SL) and tube lens (TL, Thorlabs TL-400A). A water immersion objective (OBJ, Nikon 60X, NA = 1.27 WI) focalized the light onto the sample. The back-scattered light from the sample follows the same path until it reaches DG1. There, the chromatic line is recombined into a white beam and is filtered by a 400 µm pinhole inserted in a 4f system with L3 = L4 = 200 mm. Light is then diffracted by a second blazed diffraction grating (DG2, 300 gr/mm, Edmund optics, 67% efficiency@500 nm); only the first order of diffraction is selected. Another 4f system (L5 = L6 = 400 mm) is used along with guiding mirrors to conjugate the pupil plane at the other side of the resonant mirror with a doubled diameter as compared to the opposite face. Finally, a camera (Orca Flash4 C11440, Hamamatsu) is placed at the focal plane of L7 to acquire the image. The 2D super-resolved image (displayed at 24 kHz) can be visualized directly without electronic synchronization, using a white screen instead of a camera (see Visualization 1). The overall efficiency from the light backscattered by the sample up to the sensor plane is ${\approx} 20\%$ (considering the efficiency of both descan and rescan gratings (DG1 and DG2) as well as the 50:50 beam splitter). Figure 1(b) shows what a chromatic image looks like.

The sample is placed on a motorized XY scanning stage (Thorlabs, MLS203), and the objective is mounted on a linear Z stage (Newport, M-426). The microscope is controlled via homemade LabVIEW software (NI). Manufacturer’s software (HCImage) is used for image acquisition independently. Image processing and measurements have been performed with Labview, Matlab, and ImageJ.

To characterize the resolution of the system, we image 60 nm gold nanoparticles (Sigma Aldrich, 742015) immobilized on a #1.5 coverslip, covered by microscope immersion oil to minimize the reflection from the coverslip in the nanoparticle plane. A conventional reflectance confocal microscope without ISM has been built, and it shares the same objective for resolution comparison. For this microscope, a monochromatic 520 nm laser is scanned via a 2D galvanometer mirror system (GVS002, Thorlabs) and injected between L1 et DG1 by a flip mirror (not represented in Fig. 1(a)). After de-scanning with the galvanometer mirrors, a pinhole of 1 Airy unit (AU) is used to reject the background of the backscattered light and a photodiode synchronized with the 2D scanner is used to reconstruct the confocal image. Figure 2(a) shows the PSF image from this conventional reflectance confocal microscope, and Fig. 2(b) is the PSF image obtained from our chromatic asynchronous ISM. The intensity profiles are plotted in Fig. 1(c) and fitted by a Gaussian function to find each full width at half maximum (FWHM). These FWHM values are 240 ± 20 nm for conventional confocal reflectance microscope and 170 ± 10 nm for chromatic ISM @λ=500 nm, NA = 1.27. The PSF narrowing factor is measured at 1.4 ± 0.1 close to the expected $\sqrt 2 $ gain factor for ISM methods [14]. The coherent modulation transfer function is calculated by taking the 2D Fourier Transform of each PSF and then average radially. The result is presented in Fig. 2(d) where the gain factor of frequency support could reach 1.8 (for a theoretical maximum value of 2). Due to the different wavelengths used for imaging, we also study the PSF sharpening dependency across the whole field of view (FOV). Figure 2(e) shows the PSF narrowing factor compared to the theoretical PSF size across a FOV of 20 µm, which corresponds to wavelength spread from 480 nm to 600 nm (so 540 nm ±11%). The FWHM of 10 different PSFs has been measured for the statistical study (average and standard deviation are calculated). The chromatic ISM PSF consistently outperforms the theoretical diffraction-limited value by at least 1.3 times over the whole field of view.

 

Fig. 2. (a): Image of a 60 nm Gold bead considered as the PSF for conventional confocal reflectance confocal microscope without ISM using a monodetector. (b): same as (a) with our super-resolved ISM. (c): Line profile of (a), (b) and its Gaussian fit for FWHM measurement. (d): coherent Modulation Transfer Function (cMTF) of the system without ISM and with ISM, frequency is normalized to theoretical diffraction limit frequency. (e): FWHM of PSFs at different wavelengths in Field of View (FOV) and PSF narrowing factor (expected to be $\sqrt 2 $ in dash-dotted line).

Download Full Size | PDF

3. Single particle tracking

SPT is crucial for characterizing diluted nano-objects [15] because the Brownian motion of particles is directly related to the particle size. For non-actively moving particles, smaller particles exhibit faster Brownian motion. The mean squared displacement (MSD) of particles between successive frames can be used to determine the size of each tracked particle, given the viscosity of the medium. During the camera’s integration time, if the MSD of a particle is larger than the PSF size, the acquired image becomes distorted, thus increasing localization error. Moreover, since particle size determination from its trajectory is converging in $\sqrt M $, with M the tracking length, it is important to track the particle during a large number of frames [16]. A fast imaging system is therefore interesting for single particle tracking to reduce the image distortion and increase the tracking length of a particle.

If the particle is moving relatively slow (MSD << PSF size), SPT is similar to single molecule localization in fluorescence microscopy (SMLM) in which the localization error of PSF centroid is determined by the total number of collected photons as predicted by Fisher information theory and Cramer-Rao lower bound [17,18]. The SPT fails when multiple particles are present in the same PSF volume, when there is for instance aggregation between particles. Various approaches have been described to improve the localization precision for fluorescent particles. Wide-field structured illumination has been proposed to enhance the localization precision in fluorescent single-particle tracking [19,20]. The signal-to-noise ratio (SNR) can be improved via background reduction using confocal laser scanning fluorescence microscopy [21,22]. More recently, it has been demonstrated that fluorescence ISM for SPT leads to a twofold improvement in localization precision [23].

In our label-free scheme, the localization precision is improved thanks to resolution enhancement, background rejection, and fast acquisition speed of our system. Indeed, the PSF size reduction by $\sqrt 2 $ results in an increase of amplitude by 2, consequently increase SNR and unlocks tracking at higher density for the same illumination power. To quantify the localization precision of our system, we perform the tracking of multiple immobilized 60 nm gold particles within the field of view. The position of each particle is subtracted to the average position of all tracked particle to avoid thermomechanical drifts [24]. The lateral localization precision is then calculated as standard deviation of the resulted position. As expected, the static lateral localization precision ${\sigma _x}\; $ along the x direction (equivalent to ${\sigma _y}\; $ along the y direction) follows a linear dependence concerning to $\frac{1}{{\sqrt {{N_p}} }}$, with ${N_p}$ the number of detected photons (Fig. 3(a)). The evolution of ${\sigma _x}\; $ with the particle defocus within the depth of field of the confocal microscope (${\approx}{\pm} 500\; nm$, Fig. 3(b)) follows as expected a quadratic dependency. For ${N_p} = 3500$, and a particle at focus, the localization precision reaches 4.3 nm.

 

Fig. 3. (a): Static localization error ${\sigma _x}\; $ as a function of the number of photons ${N_p}$. (b): Static localization error ${\sigma _x}\; $ at a different focus for N_p= 3500 photons. (c): Crop of single nanoparticles acquired @1000 Hz for SiO₂ 100 nm, SiO₂ 50 nm, and Au 20 nm moving in the water (see Visualization 2) and their intensity profile (with background subtracted), the scale bar is 1 µm and color-coded according to the intensity profile. (d): Distribution of nanoparticle hydrodynamic size for each type of particle obtained from diffusion equation and Stoke-Einstein relation.

Download Full Size | PDF

We also demonstrate the ability of our setup in detecting and tracking nanoparticles with Brownian movement in water. To do so, we image 20 nm gold nanoparticles (sigma Aldrich, 741965), 100 nm and 50 nm silica nanoparticles (Sigma Aldrich, 797936) at a few tens of microns in the solution, using 8 kW/cm² of average illumination power (25 MW/cm² peak power). The acquisition rate of our camera has been fixed at 1000 Hz (maximum possible), and the sampling has been optimized for SPT [17] at 96 nm/pixel in the object space. The PSF is sampled with 1.85 pixels, and a crop around a single particle in each acquisition along with its intensity profile is presented in the inset of Fig. 3(c) for signal-to-noise ratio comparison. Although they are reflectance images, the super-localization and tracking algorithms developed for fluorescence imaging can be applied since the signal looks very similar (see Visualization 2). The tracking process for each particle was done by the plugin Trackmate in ImageJ [25]. The MSD was calculated for each particle and linear fitted to find the diffusion coefficient. the hydrodynamic size is then deduced from the Stoke-Einstein equation [26]. Figure 3(d) shows good agreement between the measured distribution of the hydrodynamic size and manufacturer size which illustrate that the system can detect and track gold nanoparticle of size down to 20 nm and silica particle of size down to 50 nm at depth in the sample.

4. Imaging of freely moving bacteria

We then moved to biological sample label-free where fast and super-resolved imaging is important. We have considered Lactobacillus moving freely in water. These non-motile bacteria are commonly found in yogurt and cheeses and have a rod shape of dimension $\{{1 - 1.5} \}\times \{{0.7 - 1} \}\; $µm [27]. Capturing freely-moving bacteria would require a high-speed microscope with good resolution. Typically, agarose immobilizes the bacteria on the coverslip surface to avoid bacteria fast movements. In our experiment, there was no immobilization and the sample consists in a small portion of the rehydrated probiotic, putted directly on the coverslip for observation. Images are acquired at 100 Hz @ 200W/cm² average illumination power. These illuminations generate high SNR images and can be reduced more in power for long-duration imaging without compromising the measurements. Figure 4(a) demonstrates an image of the sample where a dividing bacterium is followed easily during a few seconds at 20 µm above the coverslip. Different instants when the bacteria are close to focus are also shown in Fig. 4(b)-(e) (see Visualization 3 for full acquisition). The backscattered intensity can be interpreted as the local dry mass density, which is higher at the tips of bacteria where the protein complex responsible for division during cytokinesis is present [28]. This signal is proportional to the refractive index mismatch with the medium, far from absorption and optical resonance. In Fig. 4(f) and Fig. 4(g), the measured diameter (600 nm) and length of the imaged bacteria during its mitosis ($2 \times 1.5$ µm) reach the tabulated values. These dimensions are close to conventional optical method resolution and our resolution improvement along with speed enhancement pave the way to the study of small micro-organisms in their native and unmodified environment.

 

Fig. 4. (a): Raw Image of Lactobacillus. (b-e): Image of Lactobacillus at different instants (see Visualization 3 for full acquisition). (f), (g): Line profile along its length and cross section respectively.

Download Full Size | PDF

5. Conclusion

We have introduced a new approach to label-free microscopy using ISM, enabling 2D super-resolution imaging at a speed of 24kHz, which is limited by the resonant mirror. The demonstration with nanoparticle localization and fast tracking at 1000 frames per second alongside imaging of moving Lactobacillus, showcases our system's capabilities in terms of speed, localization precision, and resolution. The interest in super-localization with ISM lies in both increased signal-to-noise ratio (SNR) and detection PSF size reduction, consequently, improved localization precision [23,29], as well as compatibility with higher-density solutions of particles without the need for increased illumination power. Lowering this power is always beneficial to avoid alteration or damaging fragile samples, such as biological objects.

The label-free high-speed super-resolution imaging capability is key for small live sample imaging (such as bacteria or viruses) as well as for transient phenomenon detection in tissues. Currently, the camera can capture 20% of scattered light from the objective, allowing for fast imaging with illumination compatible with live sample imaging. This could be further improved using prisms rather than diffraction gratings but with an increased risk of aberration and chromatic focal shifts.

For 3D imaging, we did not try to perform high-speed focusing. However, since our setup remains fully compatible with classical focusing methods, it is possible to scan faster in the Z direction by introducing an adaptive lens whose focal length can be varied electrically [30]. Moreover, the scanning rate could be further enhanced by implementing the technique outlined in a recent article, which involved using a lens array to multiply the scan speed [31]. Rotating scattering microscopy [32] has been proposed to increase resolution in an evanescent-wave scheme, so limited to the sample/coverslip interface. Here, we can achieve the resolution gain in volume, with efficient background rejection and without requiring Total internal reflection objectives. Furthermore, our asynchronous super-resolution scheme avoids any synchronization issues and post-processing artifacts that could occur in structured illumination approaches or numerical ISM.

We see potential for applications beyond biology and nanoparticle analysis, including fast control of surface nano-structuring [7].