Expand this Topic clickable element to expand a topic
Skip to content
Optica Publishing Group

Full field-of-view hexagonal lattice structured illumination microscopy based on the phase shift of electro–optic modulators

Open Access Open Access

Abstract

High throughput has become an important research direction in the field of super-resolution (SR) microscopy, especially in improving the capability of dynamic observations. In this study, we present a hexagonal lattice structured illumination microscopy (hexSIM) system characterized by a large field of view (FOV), rapid imaging speed, and high power efficiency. Our approach employs spatial light interference to generate a two-dimensional hexagonal SIM pattern, and utilizes electro-optical modulators for high-speed phase shifting. This design enables the achievement of a 210-µm diameter SIM illumination FOV when using a 100×/1.49 objective lens, capturing 2048 × 2048 pixel images at an impressive 98 frames per second (fps) single frame rate. Notably, this method attains a near 100% full field-of-view and power efficiency, with the speed limited only by the camera's capabilities. Our hexSIM demonstrates a substantial 1.73-fold improvement in spatial resolution and necessitates only seven phase-shift images, thus enhancing the imaging speed compared to conventional 2D-SIM.

© 2024 Optica Publishing Group under the terms of the Optica Open Access Publishing Agreement

1. Introduction

Super-resolution (SR) optical microscopy plays a pivotal role in enabling the observation of microstructures at spatial resolutions that surpass the diffraction limit [1]. Typically, high numerical aperture objective lenses are employed to achieve optimal spatial resolution, although this often involves compromises in the field of view. In the current landscape of super-resolution microscopy technology, there is a burgeoning interest in advancing high-throughput methodologies, emphasizing large FOVs, fast speeds, and high power efficiency. Particularly beneficial when examining samples with expansive microstructures such as tissues and actin, high-throughput techniques facilitate a more comprehensive revelation of sample information. Moreover, in dynamic observations, high-throughput methodologies can offer increased convenience for researchers. Recent advancements in super-resolution microscopy have successfully integrated high-throughput capabilities with high resolution. Examples include high-throughput stimulated emission depletion microscopy [2], single-molecule localization microscopy (SMLM) [3] and structured illumination microscopy (SIM). Among these, SIM stands out as a non-scanning imaging technique that does not necessitate high-intensity illumination lasers or specialized fluorescent dyes. Thus, it boasts large FOV, high imaging speed and minimal sample damage, particularly in live-cell imaging [4,5].

Various methods are employed to enhance the throughput of SIM. In conventional SIM systems, the +/- 1st diffraction order beams, diffracted by a phase grating, are commonly used [6]. However, the mechanical movement of the grating imposes limitations on the imaging speed, often restricted to several seconds. To address this challenge, spatial light modulators (SLMs) [7] and digital micromirror devices (DMDs) [8,9] have been introduced to replace the phase grating. The significant advantage of SLMs and DMDs lies in their ability to achieve rapid translation and rotation of the pattern, thereby improving speed to dozens of frames per second (fps). Later, a spatial light modulator with ferroelectric liquid crystals (FLCSLM) has been developed [1012], enabling even faster pattern switching. Presently, FLCSLM-based SIM systems can achieve a remarkable 564-fps single frame rate [11]. However, constrained by the size and pixel number of SLMs or DMDs, the number of fringes typically does not exceed 600, limiting the maximum FOV to ∼108 × 108 µm2. In addition, the effective diffraction area of a SLM limits the power efficiency to only ∼30%. A SIM method that combines a two-dimensional grating for pattern generation and an SLM for phase shifting has been proposed [13]. Although it sacrifices speed and resolution, it improves the FOV to 690 × 517 µm2, yet still falls short of achieving the full FOV. The use of a scanning galvanometer group in conjunction with piezoelectric platforms enables free adjustment of the spatial period of the pattern [14]. Electro-optical modulators (EOMs) have been suggested to replace piezoelectric platforms, resolving the system's speed issue [15]. Consequently, a single frame rate of 364 fps can be achieved. However, the size of the scanning galvanometer mirror limits further improvements in the FOV. Recently, a fiber-based SIM system employing single-mode fibers to form hexagonal arrays for pattern generation has been proposed [16,17], which demonstrates ∼150 × 150 µm2 size single FOV image with a single frame rate of up to 396 fps. Similar to scanning galvanometer-based SIM systems, the fiber-based SIM system utilizes non-diffractive optics, resulting in a larger FOV and higher power efficiency. Unfortunately, the fiber coupling efficiency restricts efficiency to 100%, and improvements in the phase-shifting accuracy of the fiber phase shifter are still needed. While these methods have increased the throughput of SIM to varying degrees, none have fully utilized the FOV of the objective lens.

In addition to using cosine-shaped interference fringes as structured illumination patterns for super-resolution imaging, three or more beams are interfered to create structured illumination patterns with intensity variations along multiple lateral directions. Schropp demonstrated the efficacy of this pattern type in achieving enhanced spatial resolution without the necessity to alter the pattern direction. Specifically, he proposed the application of a seven-step phase-shifted hexagonal lattice-structured illumination for SIM imaging [18]. A FLCSLM is employed to generate hexagonal lattice illumination patterns for super-resolution imaging via three first-order diffracted beams. However, this approach introduces constraints on the power efficiency and FOV of the SIM system [19]. These limitations are effectively addressed by the fiber-based hexagonal lattice SIM [20]. This system integrates the optical waveguide, beam splitter, and thermal phase shifter into a compact, unified optical device. Despite these advancements, challenges persist in terms of phase-shifting stability. Additionally, the spatial frequency of the hexagonal lattice pattern is ∼0.57 times the cutoff frequency, resulting in a spatial resolution of ∼254 nm with 532 nm excitation light and a 60×/NA1.4 objective lens.

In this study, a large FOV and fast speed hexagonal lattice-structured illumination microscopy (hexSIM) is presented. The method can achieve a full field of view, which means that the FOV can cover the full sensor size of the camera and nearly the full flat field number of the objective lens (Apo TIRF 100×/1.49 Oil, Nikon), with the speed limited only by the camera. A hexagonal lattice pattern is generated through the interference of three s-polarized spatial beams. Instead of using physical gratings or SLM/DMD, the system utilizes combined polarizing beam splitters (PBSs) and electro-optical modulator (EOMs) to generate coherent beams and perform phase shifting. This combined system effectively overcomes the limitations of the imaging FOV imposed by the grating structure or scanning galvanometer area, while maintaining nearly 100% optical power efficiency. This high efficiency allows the utilization of low-power lasers, specifically 50 mW, as opposed to FLCSLM-based SIM systems, which typically require lasers of 200 mW. After being expanded by a 4-f system, the structured illumination pattern achieves a 210-µm diameter SIM illumination FOV with the 100×/NA1.49 objective lens. After expanded by the objective lens, the FOV sufficiently covers the entire sensor size (13.3 × 13.3 mm2) of the CMOS camera. The use of EOMs facilitates faster phase shifting compared to FLCSLM, consequently further enhancing the imaging speed. We also propose a cross-correlation method to precisely calibrate the half-wave voltage of the EOM.

2. Methods

2.1 Hexagonal lattice pattern generation

The experimental setup for generating lattice SIM patterns using spatial light interference is illustrated in Fig. 1(a). The coherent Gaussian beams had optical axes oriented at 120° angles to each other, and their polarization states were adjusted to s-polarization using half-wave plates to optimal modulation contrast of the hexagonal lattice pattern, as shown in Fig. 1(b). The angular disparity of 120° between the s-polarization of the three beams in pairs introduces non-interference components, leading to a reduction in modulation depth compared to the sinusoidal stripes commonly employed in conventional 2D-SIM. Subsequently, the three beams were horizontally incident on the sloped surfaces of each of the three right-angled prism mirrors before tilting upward upon reflection. A base structure was formulated for placing the right-angled prismatic mirrors and lifting them to an angle of 3.8°, thus resulting in a tilt angle of 7.6° for the beams after reflection, as shown in Fig. 1(c). Subsequently, the three beams interfered, thereby forming a two-dimensional hexagonal lattice pattern. The interference surface had a conjugate association with the interference surface on the plane of the sample. A mounted achromatic lens pair (Stock#46-010, Edmund Optics) with 1:2.5 magnification imaging was employed to image the lattice pattern using a CMOS camera (MER2-1070-10GM, Daheng Imaging). Figure 1(d) shows the hexagonal lattice pattern captured by a CMOS camera. Figure 1(e) demonstrates the intensity profile in Fig. 1(d). The modulation depth in our implementation is ∼0.72. Compared to the conventional 2D-SIM approach, hexSIM can reduce the number of original images required for reconstruction. Using seven images with different phases, the hexSIM achieved this result without altering the illumination direction. Here, the hexagonal lattice pattern was phase shifted using electro–optical modulators to control the phases of the two beams. EOM1 was used to shift the phase by 2π/7 steps, whereas EOM2 was used to shift the phase by 6π/7 steps; essentially, the phase difference between the three beams was changed in steps of 2π/7, 4π/7, and 6π/7, respectively.

 figure: Fig. 1.

Fig. 1. (a) Scheme for forming hexagonal lattice patterns using spatial light interference, with a mounted achromatic lens for imaging the hexagonal lattice patterns onto a CMOS camera. (b) Positional distribution of three beams (green spots) and polarization directions. (c) Base structure for placing right-angle prismatic reflector. (d) Hexagonal lattice patterns captured by the CMOS camera. (e) Intensity profile along the yellow line in (a), highlighting the values used to calculate the modulation depth. Scalebar, 0.1 mm in (d).

Download Full Size | PDF

2.2 Experimental setup of Hexagonal lattice SIM

Figure 2(a) shows an overall schematic of the hexagonal lattice-structured illumination microscopy system, which was installed on an inverted fluorescence microscope (Nikon, Ellipse Ti-E). The optical path can be divided into two layers in terms of the spatial distribution in the vertical direction. A photograph of the physical system is presented in Fig. 2(b). The lower layer encompasses a portion of the optical pathway preceding the interference of the three spatial light beams, as shown in Fig. 2(c). The upper layer covers a portion of the optical pathway following the interference of the three spatial light beams, as shown in Fig. 2(d).

 figure: Fig. 2.

Fig. 2. Experimental setup of the hexagonal lattice SIM system with full field of view based on spatial light interference. (a) Three-dimensional rendering of the SIM system construction (side view). (b) Photograph of the physical system. (c) Schematic of optical setup in the lower layer of the system. BC, beam collimator; HWP1-HWP7, half-wave plate; PBS1-PBS2, polarized beam splitter; M1-M7, reflecting mirror; EOM1-EOM2, electro–optical modulator; OPCD, optical path compensation device; RAPMs, right angle prism reflectors. (d) Schematic of optical setup in the upper layer of the system. M8-M11, reflecting mirror; L1, achromatic doublet lens; TL1-TL2, tube lens. (e) Three focused spots on the back focal plane of the objective lens are distributed on the circumference of a circle with a radius of 2.5 mm (red circle).

Download Full Size | PDF

A four-color laser (RGB-405-C/488/561/637-Cnm-50mW-DF60082-A, Changchun New Industries Optoelectronics Tech) with a coherent length of 500 mm served as the light source, emitting a Gaussian beam with a beam waist diameter of 2 mm. A beam collimator (BC,60FC-4-M10-01, Schäfter + Kirchhoff) was used to collimate and expand the incident beam to a diameter of 2 mm. A half-wave plate (HWP1, WPA2210-450-650, Union Optic) was used to change the polarization direction of the beam, and a polarizing beam splitter (PBS1, CCM5-PBS201/M, Thorlabs) divided the beam into two beams with an intensity ratio of 1:2. The p-light beam passed through a half-wave plate (HWP2, WPA2210-450-650, Union Optics) to adjust its polarization direction before entering a second polarizing beam splitter (PBS1, CCM5-PBS201/M, Thorlabs). Subsequently, the light was divided into two beams with equal intensity, where light ‘p’ passed through another half-wave plate (HWP1, WPA2210-450-650, Union Optics) and rotated into vertically polarized light. Subsequently, these two light beams were reflected by a pair of mirrors (M1, M2, M3, M4, PF05-03-P01, Thorlabs) and directed into an electro–optical modulator (EOM1, EOM2, EO-PM-NR-C4, Thorlabs) to produce different phase shifts. Before entering an optical path compensating device (OPCD), the ‘s’ light beam reflected from the first polarizing beam splitter was reflected by a pair of mirrors (M5, M6, PF05-03-P01, Thorlabs). The optical path-compensating device comprised a roof prism (OPHR25-S1, Jcoptix), right-angle reflecting prism (RAP120-RA-A, Lbtek), and X-axis stage (PHS-251SR, OptoSigma), which was accurately adjusted for the optical path difference with the other two beams generated by the electro–optical modulator. Subsequently, the beam was reflected by a mirror (M7, PF05-03-P01, Thorlabs) to deflect the beam direction.

All three beams entered a half-wave plate (HWP4, HWP5, HWP6, and WPA2210-450-650, Union Optics) separately, and the direction of the half-wave plate was adjusted such that the three beams could interfere in an s-polarized manner. Subsequently, the three beams were reflected by right-angle prismatic mirrors (RAPMs, OPRR05-S1S, Jcoptix) onto mirror (M8) in Fig. 2(d) and interfered. An achromatic doublet lens (L1, OLD1418-T2 M, Jcoptix) with a focal length of 19 mm was used to focus the beams onto the conjugate plane of the back focal plane of the objective lens and form a 4-f system with a tube lens (TL1, TTL200, Thorlabs) that expanded the beam diameter by a factor of ∼10. After being reflected by a mirror (M11, PF05-03-P01, Thorlabs), they were focused by another tube lens (TL2, TTL200, Thorlabs) onto the back focal plane of an objective lens (Apo TIRF 100X/1.49 Oil, Nikon) to create three focused spots. After the objective lens collimated the beams, hexagonal lattice interference patterns formed on the sample surface. The fluorescence emitted by the sample was obtained using a CMOS camera (C13440-20CU, Hamamatsu). Additionally, a dichroic mirror (DM, ZT 405/488/561/640rpc, Chroma) was used to reflect the laser light onto the objective and transmitted the excited fluorescence. The above hardware was synchronized and controlled by a self-programmed LabView application and multichannel DAQ data acquisition card (PCIe-6738, National Instruments).

The three focused spots defined an internally connected positive triangle determined by a circle of radius 2.5 mm on the back focal plane of the microscope objective (100×/1.49, Nikon), as shown in Fig. 2(e), corresponding to a pattern period of ∼300 nm with the excitation wavelength λ = 561 nm. We estimated the system could achieve a theoretical resolution improvement of 1.73 times. Based on the definition of the Rayleigh limit, the theoretical diffraction limit resolution of the microscope is estimated from 1.22λ/(2 NA) ≈ 237 nm for the emission wavelength λ = 580 nm and NA = 1.49. Given the 2 mm diameter of the incident beam, the three interference beams could reach a diameter of ∼21 mm after expansion by lenses L1 and TL1. Upon traversing the objective lens, this results in an illuminated area with a 210 µm diameter on the sample surface. With sufficient illumination intensity, the system is capable of acquiring 2048 × 2048 pixel images at a capture rate of 98 fps. In electro–optical modulators, the output phase is proportional to the magnitude of the loaded voltage. Considering that the output voltage range of the data acquisition card is limited to −10 V to 10 V, herein, the periodicity of the phase was exploited to maintain the absolute value of the output phase value within π.

2.3 EOM half-wave voltage calibration based on cross-correlation

The accuracy of the phase shift is a key factor affecting the imaging performance of SIM microscopy. The EOM can realize a high-speed and accurate phase shift by utilizing the linear electro–optic effect. Typically, a Michelson interference optical path is established to precisely calibrate the half-wave voltage of the EOM [21]. However, herein, a cross-correlation method [22] was employed to achieve swift and precise calibration of the half-wave voltage of the EOM without requiring the rebuilding of the optical paths. This method calibrates the half-wave voltage by calculating the phase difference and multiplying the complex conjugate of the high-order spectrum of the interference fringe with the high-order spectrum of the latter interference fringe in the frequency domain.

The illumination intensity distribution of the interference fringe in the nth image can be expressed as

$${I_n} = 1 + m\cos ({{\bf k}_0} \cdot {\boldsymbol r} + {\varphi _n}). $$
where $m$ is the modulation intensity; ${{\bf k}_0}$ is the spatial frequency of the fringes; ${\boldsymbol r}$ is the coordinates of the illumination plane; and ${\varphi _n}$ is the initial phase of the fringes. The image acquired by the camera was converted into the frequency domain using a Fourier transform to obtain the spectrum of the fringe illumination image, as follows.
$${\widetilde D_n} = \widetilde O(\textbf{k} )\left[ {\widetilde F(\textbf{k} )+ \frac{m}{2}{e^{i{\varphi_n}}}\widetilde F({\textbf{k} - {\textbf{k}_0}} )+ \frac{m}{2}{e^{ - i{\varphi_n}}}\widetilde F({\textbf{k} + {\textbf{k}_0}} )} \right]. $$
where $\widetilde{\textrm{O}}({\bf k})$ is the optical transfer function (OTF) of the optical system and $\widetilde F({\bf k})$ is the spectrum of the illuminated sample. For the fringe spectrum, $\widetilde F({\bf k}) \approx \mathrm{\delta }({\bf k})$. Therefore, the spectrum had three peaks at ${\bf k} = 0$ and ${\bf k} ={\pm} {{\bf k}_0}$. The spectrum at +1 or –1 order was extracted using a mask. Note that the higher order is symmetrical in the direction; therefore, the difference in the selected level yields a difference only in the positive and negative signs, and the direction of the fringe movement due to the phase change can be determined. Whether the +1 or –1 order is extracted affects only the direction of the phase difference and not its value; i.e., the selection of the +1 or –1 order does not change the result. In this study, the method was illustrated with the +1 order, and the local spectrum at +1 order was approximated as
$$\widetilde D_n^{( + 1)} = \widetilde O({\bf k})\frac{m}{2}{e^{i{\varphi _n}}}\widetilde F({\bf k} - {{\bf k}_0}). $$

The cross-correlation function of two adjacent images can be calculated by multiplying the conjugate of the +1 order spectrum of the nth image by the +1 order spectrum of the n + 1 th image, as follows.

$${[{\widetilde D_n^{( + 1)}} ]^\ast } \cdot \widetilde D_{n + 1}^{( + 1)} = \frac{{{m^2}}}{4}{e^{i({{\varphi_{n + 1}} - {\varphi_n}} )}}{[{\widetilde O({\bf k})\widetilde F({\bf k} - {{\bf k}_0})} ]^2}. $$

In Eq. (4), only ${e^{i({{\varphi_{n + 1}} - {\varphi_n}} )}}$ contains information regarding the phase. Therefore, the phase difference between two adjacent images can be obtained by determining the angle corresponding to the cross-correlation function. The final phase difference was determined using the following Eq. (5).

$$\varDelta \varphi \textrm{ = }\arctan \frac{{{\mathop{\rm{Im}}\nolimits} \left\{ {\sum\limits_{\bf k} {\frac{{{m^2}}}{4}{e^{i({{\varphi_{n + 1}} - {\varphi_n}} )}}{{[{\widetilde O({\bf k})\widetilde F({\bf k} - {{\bf k}_0})} ]}^2}} } \right\}}}{{{\textrm{Re}} \left\{ {\sum\limits_{\bf k} {\frac{{{m^2}}}{4}{e^{i({{\varphi_{n + 1}} - {\varphi_n}} )}}{{[{\widetilde O({\bf k})\widetilde F({\bf k} - {{\bf k}_0})} ]}^2}} } \right\}}}. $$

Herein, simulation experiments were performed to verify the accuracy of the phase difference calculation method. MATLAB was used to generate seven images of simulated fringes with an image size of 96 × 96 pixels, and the phase difference was set to π/7.

The peaks, except for the 0 order, were extracted from the spectra of the seven images. The cross correlation between two adjacent images were calculated using Eq. (4), and ${e^{i({{\varphi_{n + 1}} - {\varphi_n}} )}}$ was performed using the angle function in MATLAB. Finally, six phase-difference results were obtained using Eq. (5). Additionally, the mean phase differences were calculated to minimize the errors. Evidently, the result was 0.4488, and the difference with π/7 ≈ 0.4487 was 0.0001 rad, thus proving the high accuracy of this method.

The experimental setup shown in Fig. 1(a) was employed to calibrate the half-wave voltage of the electro–optical modulators. One of the optical paths with EOM was masked at a time to measure the half-wave voltage of the other unmasked EOM. The other two unmasked light beams interfered to form sinusoidal fringes on the CMOS camera. The EOM was applied with a voltage ranging from –10 V to 9 V in steps of 1 V by the multichannel DAQ data acquisition card, and the camera was controlled to acquire one image for each voltage change, for a total of 20 equal-phase-difference fringe images with a size of 512 × 512 pixels. The flowchart for phase difference calculation is shown in Fig. 3. Further, Fourier transform was performed on the images, and the 0 order point in the frequency domain was removed. Subsequently, the +1 order was extracted from the spectra using a mask with a size of 7 × 7 pixels. The cross-correlation and phase difference of each two adjacent images among these 20 mask images was calculated using Eqs. (4) and (5), respectively. The phase differences of all images were averaged to obtain the final phase difference. As the voltage step was 1 V, the half-wave voltage was calculated as ${V_\mathrm{\pi }}\mathrm{\ =\ \pi /\Delta }\varphi $, where $\mathrm{\Delta }\varphi $ is the phase difference to the first image.

 figure: Fig. 3.

Fig. 3. Flowchart for phase difference calculation. (a) Fringe images captured by CMOS. (b) Process of frequency domain. (c) Relative phase of each image and fitting line of phase–voltage.

Download Full Size | PDF

The half-wave voltages were calibrated at 405, 488, 561, and 640 nm for the two electro–optical modulators. To evaluate the accuracy of the obtained results, the calibration results of a Michelson interferometer were used as standard values. To further minimize the error, four sets of 20 fringe images at each wavelength were considered. Subsequently, the results for each of the four sets were averaged and compared with the standard values, as presented in Table 1. Evidently, the results of the cross-correlation method for the half-wave voltages of the two EOMs were within 1% of the standard values. Considering the problems of camera shaking and environmental noise during the experiment, the results exhibited a decrease in accuracy compared with the simulation experiments; however, the obtained accuracy was still sufficient to meet the accuracy requirements of our system to capture images at the desired phase.

Tables Icon

Table 1. Half-wave voltage of EOMs calibrated with cross-correlation method (measured value) and Michelson interferometer (standard value)

2.4 Reconstruction algorithm

The open-source reconstruction algorithm hexSIMProcessor was used in MATLAB [19]. The SIM reconstruction results obtained in this study were generated using this algorithm. Unless otherwise stated, the relevant parameters used for all reconstructions were: zero order attenuation width, α, of 0.3; zero order attenuation, β, of 0.9; wiener regularization parameter, ω, of 0.1.

3. Experiments and results

3.1 Hexagonal lattice imaging of fluorescent nanoparticles

The first experiment focused on assessing the fundamental performance of the proposed SIM system, particularly in terms of field of view and imaging resolution enhancement. Figure 4 shows the image of 100 nm fluorescent particles excited by a 561 nm light (Nanoparticles 4C flour 100 nm slide, Abberior) under the imaging FOV of 133 × 133 µm2 (2048 × 2048 pixels). The exposure time for this image acquisition was 20 ms. Figures 4(a) and 4(b) show the wide-field image and SIM-reconstructed image of the fluorescent nanoparticles, respectively. Evidently, the imaging resolution of fluorescent nanoparticles was significantly improved in SIM images compared with that in wide-field images. The FWHM of the same 20 dispersed fluorescent particles was measured in the wide-field and SIM-reconstructed images. The averaged FWHM curves are presented in Figs. 4(c) and 4(d). Clearly, the average FWHM in the wide-field image was ∼264 ± 15 nm, while that in the SIM image was significantly reduced to ∼130 ± 7 nm. In addition, the region of interest (ROI) was intercepted in Figs. 4(a) and 4(b), which are highlighted in yellow. It is evident that the two fluorescent nanoparticles 163 nm apart in the SIM image of this region were indistinguishable in the wide-field-of-view image, and the intensity distribution profile on the blue line is shown in Fig. 4(e). These results demonstrate the spatial improvement with hexagonal lattice patterns.

 figure: Fig. 4.

Fig. 4. Wide-field and hexagonal lattice SIM imaging on 100 nm fluorescent nanoparticles. (a) Wide-field image of fluorescent nanoparticles. (b) Hexagonal lattice SIM reconstruction image of fluorescent nanoparticles. (c) Intensity profiles of measurement of 20 nanoparticles in wide-field image(red) and SIM image(blue). (d) Statistics of FWHM of 20 individual nanoparticles in wide-field image(red) and SIM image(blue). (e) Intensity profiles along two yellow lines in the regions of interest in (a) (red) and (b) (blue). Scalebars, 5 µm in (a) and (b), and 0.5 µm in insets.

Download Full Size | PDF

3.2 Hexagonal lattice imaging of fixed huFIB cells

To showcase the imaging capabilities of the proposed HexSIM system for biological samples, super-resolution imaging of microtubules and actin in huFIB cells (GATTA-Cells 4C; GATTAQUANT) was conducted. The imaging of microtubules labeled with Alexa Fluor 555 was performed, utilizing an excitation light wavelength of 561 nm and an imaging FOV measuring 133 × 133 µm2 (2048 × 2048 pixels). The exposure time was set at 10 ms for capturing the images. Figures 5(a) and 5(b) show the wide-field and SIM-reconstructed images of the microtubules, respectively. A comparison between the two images highlights the superior structural resolution achieved by the hexSIM image, with effective suppression of background noise. In Fig. 5(c), the intensity distribution profile of the blue lines within the ROI (Region of Interest) depicted in Figs. 5(a) and 5(b) is illustrated. Clearly, the tightly packed microtubule filaments were almost blurred together in the wide-field image; in contrast, nearly every microtubule was distinguished in the SIM-reconstructed image. We estimated the spatial resolution using a deconvolution-based method [23], which only requires a non-saturated, bandwidth-limited signal with adequate spatial sampling to enable the fast and precise image resolution analysis. This method was performed on the wide-field and SIM images shown in Figs. 5(a) and 5(b), and the results are shown in Fig. 5(d) and 5(e), respectively. The results of the deconvolution analysis [23] indicate that the cut-off frequencies of the wide-field image and the hexSIM image were 3.69 and 7.85 µm−1, with spatial resolutions of 271 and 127 nm, respectively.

 figure: Fig. 5.

Fig. 5. Wide-field and hexagonal lattice SIM imaging on huFIB cell microtubules. (a) Wide-field image of huFIB cell microtubules. (b) Hexagonal lattice SIM reconstruction image of huFIB cell microtubules. (c) Intensity profiles along two yellow lines in the regions of interest in (a) (red) and (b) (blue). (d) Decorrelation-based resolution analysis of the wide-field image; (e) Decorrelation-based resolution analysis of the hexagonal lattice SIM image. Green line, decorrelation functions before high-pass filtering; magenta line, radial average of log of absolute value of Fourier transform of original images; gray lines, all high-pass filtered decorrelation functions; blue to black lines, decorrelation functions with refined mask radius and high-pass filtering range. Blue crosses, all local maximum value. Scalebars, 5 µm in (a) and (b), and 1 µm in insets.

Download Full Size | PDF

Additionally, the actin labelled with Alexa Fluor 647 was captured using an excitation light wavelength of 640 nm, with an imaging FOV measuring 133 × 133 µm2 (2048 × 2048 pixels). The exposure time was set at 10 ms for capturing the images. Figures 6(a) and 6(b) present the wide-field and SIM-reconstructed images of actin. The intensity distribution profile of the blue lines in Fig. 6(c) indicates that the hexagonal SIM image could distinguish between two neighboring filaments that could not be distinguished in the wide-field image. Moreover, a spatial resolution analysis was performed based on the deconvolution of the wide-field and SIM images shown in Figs. 6(a) and 6(b), and the results are shown in Figs. 6(d) and 6(e), respectively. The results of the deconvolution analysis indicate that the cut-off frequencies of the wide-field image and the hexSIM image were 3.16 and 7.54 µm−1, with spatial resolutions of 316 and 133 nm, respectively.

 figure: Fig. 6.

Fig. 6. Wide-field and hexagonal lattice SIM imaging on huFIB cell Actin. (a)Wide-field image of huFIB cell Actin. (b) Hexagonal lattice SIM reconstruction image of huFIB cell Actin. (c) Intensity profiles along two yellow lines in the regions of interest in (a) (red) and (b) (blue). (d) Decorrelation-based resolution analysis of the wide-field image. (e) Decorrelation-based resolution analysis of the hexagonal lattice SIM image. Scalebars, 5 µm in (a) and (b), and 1 µm in insets.

Download Full Size | PDF

4. Discussion

In this paper, we outline the advantages of the EOM-based hexSIM system. The achieved results of the system are presented in Table 2. Subsequent sections will delve into various key performance indicators and provide a comparative analysis with other existing systems.

Tables Icon

Table 2. Achieved performance indicators of the system setup

4.1 Spatial resolution improvement

We employ two-dimensional hexagonal illumination patterns generated by three mutually coherent spatial beams to improve lateral resolution. When the three beams converge at the edge of the pupil, the spatial frequency is only at a factor of $\sqrt 3 $ larger than the coherent cut-off, resulting in a limit of $1 + {\raise0.7ex\hbox{${\sqrt 3 }$} \!\mathord{\left/ {\vphantom {{\sqrt 3 } 2}}\right.}\!\lower0.7ex\hbox{$2$}} \approx 1.87$ spatial resolution improvement, which is lower compared to the conventional 2D-SIM. In our system, the fill factor (FF), defined as the ratio of the radius of the circle containing the three focused spots to the pupil radius, is approximately 0.84 [20]. Note that for hexSIM, the resolution can be estimated using $\Delta {\rho _{SIM}} = \frac{{\Delta {\rho _{WF}}}}{{1 + FF \times \sqrt 3 /2}}$, where $\Delta {\rho _{WF}}$ is the wide-field resolution. Thus, in our system, a resolution improvement of approximately 1.73 times can be achieved, resulting in a spatial resolution of ∼130 nm with an excitation wavelength of 561 nm and a 100×/NA1.49 objective lens. To further enhance resolution, it is necessary to increase the spatial frequency of the structured illumination by elevating the tilt angle of the base structure, as shown in Fig. 1(c). However, this is currently challenging as it requires the creation of another base structure. In addition, an increase of the tilt angle will cause the tube lens to receive only partial beams, resulting in a loss of the FOV and optical energy.

4.2 Limiting factors of FOV

The EOM-based hexSIM demonstrates a capability for a large FOV, achieving a 210-µm diameter SIM illumination FOV and 133 × 133 µm2 imaging FOV with a 100×/NA1.49 objective lens. SIM commonly employs Gaussian beams, where the edge power diminishes to below 14% of the center power, resulting in a darker edge compared to the center in the image. Therefore, an extended FOV promotes a more uniform brightness distribution in the imaging. Several factors contribute to determining the FOV. Firstly, according to relevant international standards, the flat field number of the commercial objective lenses is typically 22 mm, such as those from Nikon, Olympus, Leica, etc. The field number divided by the magnification limits the SIM illumination FOV. Secondly, for SIM using diffractive optics like SLM or DMD-based SIM, the FOV of the pattern is limited by the size and pixel number of the patterning element generating the SIM pattern. Currently, FLCSLM-based SIM can achieve a maximum FOV of 108 × 108 µm2 (100×/NA 1.7), In our system, the use of non-diffractive optics breaks this limitation. Furthermore, laser intensity influences the imaging Signal-to-Noise Ratio (SNR), making high power efficiency crucial in SIM system design. Our system experiences minimal power losses, mainly due to reflection and refraction losses of optical components. As a result, our system only requires a 50 mW laser in such a large field of view, whereas FLCSLM-based SIM typically requires a 200 mW laser. Moreover, the rectangular shape of the camera sensor restricts it to capturing only a partial image of the full field number of the SIM system. In our system, a camera with a larger sensor size will further enhance the FOV.

The primary limiting factor for FOV varies depending on the SIM system architecture. Specifically, in phase grating-based SIM [6], SLM/DMD-based SIM [11] and scanning galvanometer-based SIM [15], the FOV is constrained by the active area of the grating, the size of the SLM/DMD, and the mirror size, respectively. Conversely, in the fiber-based SIM [16] and the EOM-based hexSIM, the FOV is determined only by the aperture of the tube lens used to focus the incident beams onto the back focal plane of the objective lens. For a more explicit comparison, we estimate the field number of existing SIM systems by multiplying the imaging FOV with the magnification of the objective lens. In our system, the field number is calculated as 0.21 × 100 = 21 mm in diameter, nearly filling the full flat field number of the objective lens (22 mm). In contrast, the field number is currently 13.2-mm diameter in the phase grating-based SIM, 15.3-mm diameter in the FLCSLM-based SIM, 9.3-mm diameter in the scanning galvanometer-based SIM, and 8.5-mm diameter in the fiber-based SIM. In the future, the availability of a tube lens with a greater aperture may further enhance of the FOV of our SIM system.

4.3 Achievable imaging speed

This paper highlights the imaging speed advantages of the EOM-based hexSIM, facilitated by high-speed phase shifting. HexSIM requires only seven structured illumination images to generate an enhanced-resolution image without the need for illumination pattern rotation, leading to improved imaging speed compared to conventional 2D-SIM. In existing SIM systems, the acquisition speed of structured illumination images can be limited by the response time of some key devices, including phase grating (∼3 Hz), FLCSLM (∼2 kHz), scanning galvanometer (∼10 kHz), piezoelectric stage (∼7 kHz), fiber phase shifter (∼2.6 kHz) and CMOS camera. In our system, the major limiting factors are the EOM and the CMOS camera. Driven by a 200 V high-voltage amplifier, the EOM achieves a remarkable response rate of 1 MHz and accomplishes phase shifting within 0.1 ms. The imaging speed is then constrained solely by the readout time of the camera. The CMOS camera has a readout time of 9.74 µs per pixel line, with a delay of ∼38.96 µs between receiving a rising edge trigger signal and the shutter opening. To achieve fast speed, the camera reads out from the center pixel line to the top and bottom pixel line simultaneously, doubling the readout speed. Consequently, when acquiring 512 × 512 pixel images with an exposure time of 0.2 ms, a single frame rate of ∼357 fps and a SIM frame rate of ∼51 fps can be achieved. When acquiring 2048 × 2048 pixel images with an exposure time of 0.2 ms, a single frame rate of ∼98 fps and a SIM frame rate of ∼14 fps can be achieved.

5. Conclusion

In this study, we present an innovative SIM system design based on a hexagonal lattice illumination pattern. Our proposed system achieves a remarkable 210-µm diameter SIM illumination FOV with a 1.73 spatial resolution improvement, utilizing a 100×/1.49 objective lens. Unlike conventional SIM systems limited by SLM/DMD pixel count, mirror area of scanning galvanometers, and other factors, our design overcomes these constraints, extending the field of view to cover the full sensor area of the CMOS camera. The EOMs control the phase of the SIM pattern, thus ensuring high-speed and precise phase shifting, making the CMOS camera's readout time the sole limiting factor for imaging speed. We also present a novel cross-correlation-based method for EOM half-wave voltage calibration, eliminating the need for a Michelson interferometer and ensuring high accuracy in phase shifts. Our system adopts a compact design by vertically dividing the optical path into two layers, resulting in a modest footprint of 380 × 340 mm2. Notably, this setup minimizes optical energy loss, enhances power efficiency, and allows for the use of low-power lasers. Moreover, in contrast to the conventional sinusoidal illumination pattern of 2D-SIM, hexagonal lattice illumination eliminates the necessity to change the illumination direction. HexSIM reconstruction only requires seven original images with different phases. The performance of the proposed system was validated through experiments involving fluorescent nanoparticles and multicolor biological samples.

Funding

National Key Research and Development Program of China (2021YFF0700302); National Natural Science Foundation of China (61975188); Natural Science Foundation of Zhejiang Province (LQ23F050010, LY23F050010); National Science Fund for Distinguished Young Scholars (62125504); Ningbo Key Scientific and Technological Project (2022Z123); Research Initiation Project of Zhejiang Lab (2022NKOPI01).

Acknowledgments

We would like to thank Engineer Sijia Zhou from Ningbo Novel Optics Co. Ltd., for her support with the software application and EOM calibration.

Disclosures

The authors declare no conflicts of interest related to this study.

Data availability

The raw data from our experiments and the code for generating the result can be found at [24].

References

1. E. Abbe, “Beiträge ZUR Theorie DES Mikroskops UND DER mikroskopischen Wahrnehmung,” Archiv f. mikrosk. Anatomie 9(1), 413–468 (1873). [CrossRef]  

2. Y. Xue and P. T. C. So, “Three-dimensional super-resolution high-throughput imaging by structured illumination STED microscopy,” Opt. Express 26(16), 20920–20928 (2018). [CrossRef]  

3. J. S. H. Danial, J. Y. L. Lam, Y. Wu, et al., “Constructing a cost-efficient, high-throughput and high-quality single-molecule localization microscope for super-resolution imaging,” Nat. Protoc 17(11), 2570–2619 (2022). [CrossRef]  

4. P. J. Keller, A. D. Schmidt, A. Santella, et al., “Fast, high-contrast imaging of animal development with scanned light sheet-based structured-illumination microscopy,” Nat. Methods 7(8), 637–642 (2010). [CrossRef]  

5. Y. Wu and H. Shroff, “Faster, sharper, and deeper: structured illumination microscopy for biological imaging,” Nat. Methods 15(12), 1011–1019 (2018). [CrossRef]  

6. K. Samanta, A. Ahmad, J. C. Tinguely, et al., “Transmission structured illumination microscopy with tunable frequency illumination using tilt mirror assembly,” Sci. Rep. 13(1), 1453 (2023). [CrossRef]  

7. R. Forster, H. W. Lu-Walther, A. Jost, et al., “Simple structured illumination microscope setup with high acquisition speed by using a spatial light modulator,” Opt. Express 22(17), 20663–20677 (2014). [CrossRef]  

8. M. Li, Y. Li, W. Liu, et al., “Structured illumination microscopy using digital micro-mirror device and coherent light source,” Appl. Phys. Lett. 116(23), 233702 (2020). [CrossRef]  

9. X. Li, S. Xie, W. Liu, et al., “Speckle-free laser projection structured illumination microscopy based on a digital micromirror device,” Opt. Express 29(26), 43917–43928 (2021). [CrossRef]  

10. H.-W. Lu-Walther, M. Kielhorn, R. Förster, et al., “fastSIM: a practical implementation of fast structured illumination microscopy,” Methods Appl. Fluoresc. 3(1), 014001 (2015). [CrossRef]  

11. X. Huang, J. Fan, L. Li, et al., “Fast, long-term, super-resolution imaging with Hessian structured illumination microscopy,” Nat. Biotechnol. 36(5), 451–459 (2018). [CrossRef]  

12. W. Zhao, S. Zhao, L. Li, et al., “Sparse deconvolution improves the resolution of live-cell super-resolution fluorescence microscopy,” Nat. Biotechnol. 40(4), 606–617 (2022). [CrossRef]  

13. J. Zheng, X. Fang, K. Wen, et al., “Large-field lattice structured illumination microscopy,” Opt. Express 30(15), 27951–27966 (2022). [CrossRef]  

14. Y. Chen, W. Liu, Z. Zhang, et al., “Multi-color live-cell super-resolution volume imaging with multi-angle interference microscopy,” Nat. Commun. 9(1), 4818 (2018). [CrossRef]  

15. F. Xu, J. Zhang, D. Ding, et al., “Real-time reconstruction using electro-optics modulator-based structured illumination microscopy,” Opt. Express 30(8), 13238–13251 (2022). [CrossRef]  

16. H. Ortkrass, J. Schurstedt, G. Wiebusch, et al., “High-speed TIRF and 2D super-resolution structured illumination microscopy with a large field of view based on fiber optic components,” Opt. Express 31(18), 29156–29165 (2023). [CrossRef]  

17. T. A. Hinsdale, S. Stallinga, and B. Rieger, “High-speed multicolor structured illumination microscopy using a hexagonal single mode fiber array,” Biomed. Opt. Express 12(2), 1181–1194 (2021). [CrossRef]  

18. M. Schropp and R. Uhl, “Two-dimensional structured illumination microscopy,” J. Microsc. 256(1), 23–36 (2014). [CrossRef]  

19. H. Gong, W. Guo, and M. A. A. Neil, “GPU-accelerated real-time reconstruction in Python of three-dimensional datasets from structured illumination microscopy with hexagonal patterns,” Phil. Trans. R. Soc. A 379(2199), 20200162 (2021). [CrossRef]  

20. M. Calvarese, P. Paie, A. Candeo, et al., “Integrated optical device for Structured Illumination Microscopy,” Opt. Express 30(17), 30246–30259 (2022). [CrossRef]  

21. R. Hongliang, W. Jiuyang, and L. Liren, “Measuring phase retardation and fast axis azimuth of a wave plate using michelson interferometer,” Chin. J. Laser 35(2), 249–253 (2008). [CrossRef]  

22. Y.-Z. Liu and B. Zhao, “Phase-shift correlation method for accurate phase difference estimation in range finder,” Appl. Opt. 54(11), 3470–3477 (2015). [CrossRef]  

23. A. Descloux, K. S. Grussmayer, and A. Radenovic, “Parameter-free image resolution estimation based on decorrelation analysis,” Nat. Methods 16(9), 918–924 (2019). [CrossRef]  

24. J. Cheng, “ hexSIM data,” Zenodo (2023). https://doi.org/10.5281/zenodo.10259165

Data availability

The raw data from our experiments and the code for generating the result can be found at [24].

24. J. Cheng, “ hexSIM data,” Zenodo (2023). https://doi.org/10.5281/zenodo.10259165

Cited By

Optica participates in Crossref's Cited-By Linking service. Citing articles from Optica Publishing Group journals and other participating publishers are listed here.

Alert me when this article is cited.


Figures (6)

Fig. 1.
Fig. 1. (a) Scheme for forming hexagonal lattice patterns using spatial light interference, with a mounted achromatic lens for imaging the hexagonal lattice patterns onto a CMOS camera. (b) Positional distribution of three beams (green spots) and polarization directions. (c) Base structure for placing right-angle prismatic reflector. (d) Hexagonal lattice patterns captured by the CMOS camera. (e) Intensity profile along the yellow line in (a), highlighting the values used to calculate the modulation depth. Scalebar, 0.1 mm in (d).
Fig. 2.
Fig. 2. Experimental setup of the hexagonal lattice SIM system with full field of view based on spatial light interference. (a) Three-dimensional rendering of the SIM system construction (side view). (b) Photograph of the physical system. (c) Schematic of optical setup in the lower layer of the system. BC, beam collimator; HWP1-HWP7, half-wave plate; PBS1-PBS2, polarized beam splitter; M1-M7, reflecting mirror; EOM1-EOM2, electro–optical modulator; OPCD, optical path compensation device; RAPMs, right angle prism reflectors. (d) Schematic of optical setup in the upper layer of the system. M8-M11, reflecting mirror; L1, achromatic doublet lens; TL1-TL2, tube lens. (e) Three focused spots on the back focal plane of the objective lens are distributed on the circumference of a circle with a radius of 2.5 mm (red circle).
Fig. 3.
Fig. 3. Flowchart for phase difference calculation. (a) Fringe images captured by CMOS. (b) Process of frequency domain. (c) Relative phase of each image and fitting line of phase–voltage.
Fig. 4.
Fig. 4. Wide-field and hexagonal lattice SIM imaging on 100 nm fluorescent nanoparticles. (a) Wide-field image of fluorescent nanoparticles. (b) Hexagonal lattice SIM reconstruction image of fluorescent nanoparticles. (c) Intensity profiles of measurement of 20 nanoparticles in wide-field image(red) and SIM image(blue). (d) Statistics of FWHM of 20 individual nanoparticles in wide-field image(red) and SIM image(blue). (e) Intensity profiles along two yellow lines in the regions of interest in (a) (red) and (b) (blue). Scalebars, 5 µm in (a) and (b), and 0.5 µm in insets.
Fig. 5.
Fig. 5. Wide-field and hexagonal lattice SIM imaging on huFIB cell microtubules. (a) Wide-field image of huFIB cell microtubules. (b) Hexagonal lattice SIM reconstruction image of huFIB cell microtubules. (c) Intensity profiles along two yellow lines in the regions of interest in (a) (red) and (b) (blue). (d) Decorrelation-based resolution analysis of the wide-field image; (e) Decorrelation-based resolution analysis of the hexagonal lattice SIM image. Green line, decorrelation functions before high-pass filtering; magenta line, radial average of log of absolute value of Fourier transform of original images; gray lines, all high-pass filtered decorrelation functions; blue to black lines, decorrelation functions with refined mask radius and high-pass filtering range. Blue crosses, all local maximum value. Scalebars, 5 µm in (a) and (b), and 1 µm in insets.
Fig. 6.
Fig. 6. Wide-field and hexagonal lattice SIM imaging on huFIB cell Actin. (a)Wide-field image of huFIB cell Actin. (b) Hexagonal lattice SIM reconstruction image of huFIB cell Actin. (c) Intensity profiles along two yellow lines in the regions of interest in (a) (red) and (b) (blue). (d) Decorrelation-based resolution analysis of the wide-field image. (e) Decorrelation-based resolution analysis of the hexagonal lattice SIM image. Scalebars, 5 µm in (a) and (b), and 1 µm in insets.

Tables (2)

Tables Icon

Table 1. Half-wave voltage of EOMs calibrated with cross-correlation method (measured value) and Michelson interferometer (standard value)

Tables Icon

Table 2. Achieved performance indicators of the system setup

Equations (5)

Equations on this page are rendered with MathJax. Learn more.

I n = 1 + m cos ( k 0 r + φ n ) .
D ~ n = O ~ ( k ) [ F ~ ( k ) + m 2 e i φ n F ~ ( k k 0 ) + m 2 e i φ n F ~ ( k + k 0 ) ] .
D ~ n ( + 1 ) = O ~ ( k ) m 2 e i φ n F ~ ( k k 0 ) .
[ D ~ n ( + 1 ) ] D ~ n + 1 ( + 1 ) = m 2 4 e i ( φ n + 1 φ n ) [ O ~ ( k ) F ~ ( k k 0 ) ] 2 .
Δ φ  =  arctan I m { k m 2 4 e i ( φ n + 1 φ n ) [ O ~ ( k ) F ~ ( k k 0 ) ] 2 } Re { k m 2 4 e i ( φ n + 1 φ n ) [ O ~ ( k ) F ~ ( k k 0 ) ] 2 } .
Select as filters


Select Topics Cancel
© Copyright 2024 | Optica Publishing Group. All rights reserved, including rights for text and data mining and training of artificial technologies or similar technologies.