## Abstract

In structured illumination microscopy (SIM), the imaging speed is expected to be increased to observe living cells. The conventional 2D SIM reconstruction algorithm (RA) requires nine raw images to reconstruct a super-resolution image. Here, we develop a partial-frequency-spectrum (PFS) reconstruction algorithm, based on the subtraction of frequency spectrum, which can reconstruct a super-resolution image by using six raw SIM images (two SIM images for each orientation). Our experiments of actin filament in bovine pulmonary artery endothelial (BPAE) cell imaging indicate that by the PFS algorithm, the frame rate increases. The PFS algorithm can resolve 120 nm in our experiment, which is equivalent to the reconstruction result of conventional 9-frame SIM. The PFS algorithm only requires the phase estimation of the three images. The reconstruction speed is about 5 times faster that of the conventional nine-images SIM method.

© 2019 Optical Society of America under the terms of the OSA Open Access Publishing Agreement

## 1. Introduction

In the past few decades, a variety of super-resolution fluorescence microscopy has been developed [1–5]. Structured illumination microscopy is one of the notable techniques because of the low excitation intensity, the high speed and a twofold resolution gain in linear form [2,6,7]. SIM is achieved by employing the modulated illumination of a sinusoidal pattern. The high-frequency information is shifted into the optical transfer function (OTF), separated from superimposed information components, then shifted back to the correct position to construct a super-resolution image [7].

Dynamic cells observation is one of the advantages of SIM, but the up-to-date imaging speed misses the details of cellular activities and produce undesired results [8–10]. In order to increase the frame rate, the optical system have been greatly improved, such as high-speed imaging devices [11], and the system synchronization optimization [11–13]. In the aspect of reconstruction algorithms (RA), conventional 2D SIM techniques need nine raw images [13–18], the nine exposures for one super-resolution image limit the time resolution of SIM to follow the activities of living cells. So, reducing the number of raw images required is a promising way to increase the SIM imaging speed. Recently, many 4-frame algorithms are developed to improve the frame rate [19–21]. These algorithms require longer calculation time (parallel computing) than the conventional 9-frame method. Lal et al. [22] demonstrate a reconstruction theory in the frequency domain and operate it on numerical simulations successfully. However, it shows little resolution enhancement in the reconstructed images from their experimental SIM images. It is a challenge to have a better trade-off between the spatial resolution and the reconstruction speed.

In this paper, we develop an imaging method, termed partial-frequency-spectrum (PFS) RA, which employs the similarity between wide-field images and raw SIM images to separate the high-frequency components from raw SIM images. PFS algorithm can reconstruct a super-resolution image using six raw SIM images in three orientations, two images with the phase shift of π in each orientation. We simulate the performance of our PFS RA and validate it with experimental raw data. The super-resolution images of bovine pulmonary artery endothelial (BPAE) cells indicate that the PFS algorithm can improve the frame rate. The reconstruction speed is five times faster than that of the conventional 9-frame method. The PFS image can resolve about 120 nm in the experiment, much better than Wiener-filtered images.

## 2. Reconstruction in the PFS algorithm

In a linear system without aberration, the recorded image *D*(*r*) can be written as

*H*(

*r*) is the point spread function (PSF) of the optical system, * is the convolution,

*S*(

*r*) is the sample fluorophore density, and

*I*(

*r*) is the illumination pattern. The sinusoidal structured light

*I*(

*r*)is expressed by Thus, the Fourier transform of Eq. (1) can be written as

**∼**denotes the Fourier transform of a function, $\tilde{H}(k)$ is the system optical transfer function (OTF),

*I*

_{0}is the peak illumination intensity,

*m*is the modulation factor,

*φ*is the phase of illumination pattern, and

*p*is the illumination pattern spatial frequency. Equation (3) shows that the Fourier domain of each raw image is consist of three superimposed components, one centered and two shifted. By illuminating an object with sinusoidal pattern, the high frequency of objects is down modulated, thereby these frequencies are shifted into the OTF. In order to explain the principle briefly, we can simplify Eq. (3) as

*C*,

_{L}*C*

_{H}_{1}and

*C*

_{H}_{2}are three components of the superimposed information. The low-frequency components in

*C*,

_{L}*C*

_{H}_{1}and

*C*

_{H}_{2}have similarities. The estimation of low-frequency components in

*C*

_{H}_{1}and

*C*

_{H}_{2}can be obtained by

*C*, that corresponds to a wide-field image. The PFS algorithm uses these features to remove the low-frequency components from the frequency spectrum of SIM images and separate the high-frequency components.

_{L}A schematic diagram of the reconstruction algorithm is shown in Fig. 1. A sample is illuminated by a sinusoidal pattern in three orientations. Two images (*D*_{1}, *D*_{2}) with the phase shift of π are caught in each orientation. *D*_{1} and *D*_{2} are able to synthesize a wide-field image *D _{w}*. The process in the Fourier domain can be expressed as

*D*

_{1},

*D*

_{2}and

*D*, respectively. ${\tilde{D}_w}(k)$ just the frequency spectrum of wide-field images, also

_{w}*C*in Eq. (4). The illumination spatial frequency

_{L}*p*is calculated by iteratively optimizing the auto-correlation [15]: where $\tilde{V}(k) = {\tilde{D}_1}(k){\tilde{H}^\ast }(k)$. The maximum of |

*V|*is corresponding to the value of

*p*. The illumination phase

*φ*can be obtained by using auto-correlation [17] With the parameters of the illumination pattern, we can separate the high-frequency components by subtraction and move high-frequency components back to the original position, which is the process II in Fig. 1. Combining the frequency spectrum of the three orientations, we get the extended frequency spectrum as the process III.

The specific method of the process II, the subtraction and shifting in frequency domain, is shown in Fig. 2 and Fig. 3. The ways of obtaining the high-frequency component in *C _{H}*

_{1}and

*C*

_{H}_{2}are the same, so we take

*C*

_{H}_{2}for example.

First, remove *C _{L}* from ${\tilde{D}_1}(k)$, as shown in Fig. 3(a). The wide-field image ${\tilde{D}_w}(k)$ is used for

*C*, as the following equation

_{L}*β=*1/2,

*C*can be removed from ${\tilde{D}_1}(k)$.

_{L}Next, we calculate the low frequency in *C _{H}*

_{1}, which can be obtained by shifting ${\tilde{D}_{Wiener}}(k)$ (Wiener-filter of wide-field image) in frequency domain. Following the approach in [14], ${\tilde{D}_{Wiener}}(k)$ can be expressed as follows

*N*is the noise model, which is the average noise power outside the OTF.

_{a}*A*

^{2}|

*k*|

^{−2α}is the estimated signal model, we iterative

*A*,

*α*to find the minimum sum of squared error (SSE) between signal model (wide-field image) and the estimated signal model

*A*

^{2}|

*k*|

^{−2α}. According to the similarity between $\tilde{S}(k)$ and $\tilde{S}(k - p)$, the low frequency spectrum in

*C*

_{H}_{1}can be expressed as follows.

*C*, as shown in Fig. 3(b).

_{H1}Then, subtract the low-frequency spectrum ${\tilde{S}_{shift}}(k)$ from ${\tilde{D}_{1,2}}(k)$, as shown in Fig. 3(b). The process can be expressed as follows:

when the low frequency spectrum in*C*

_{H}_{1}is totally removed (2

*ξ=m*/2), $|{\tilde{D}_{1.3}}(k)|$ corresponds to the minimum value.

*ξ*can be obtained as follows:

*k*<0) can be express as follows:

*m*is 4

*ξ*(

*ξ=m*/4), ${\tilde{S}_H}(k)$ is the frequency spectrum (

*k*<0) of

*C*,

_{H2}*N*is the residual frequency. High-frequency component is completely separated as shown in Fig. 3(c).

_{R}At last, the separated high-frequency components in *R _{A}* need to deconvolute and move back to the original position. $R_{A} =\{ |k| < {K_{OTF}} \cap |k - p| < {K_{OTF}} \cap k \cdot p > 0\} $, consists of the high-frequency spectrum in C

_{H}_{2}and the overlapped part of Wiener-filtered image. This guarantees the continuity between the high and the low frequency spectrum. The spectrum of ${\tilde{S}_{shift}}(k)$ is distorted after deconvolution, as shown in Fig. 3(b). So, after the step above, residual spectrum appears at the position of Fig. 3(c) (the black dotted line), and a part of the residual spectrum is in

*R*. In order to suppress the influence of the residual spectrum, we use an estimated noise model

_{A}*N*=

_{b}*N*+

_{D3}*N*in Wiener filter.

_{r}*N*is the average noise of ${\tilde{D}_{1.3}}(k)$.

_{D3}*N*can be written as

_{r}*N*can be considered as an approximate estimation. Use Wiener filter to obtain $\tilde{S}(k - p) \cdot {R_A}$, which can be written as

_{r}*C*

_{H}_{1}can be obtained as well. Then, employing the Fourier shift theorem, shift the spectrum in

*R*to the original position. All above is the specific methods of process II shown in Fig. 1.

_{A}## 3. Simulation

We simulate the performance of the PFS algorithm. First, we analyze the performance of the PFS algorithm in different Gaussian noise level and use a circle pattern as a test sample, as shown in Fig. 4(a). Here are the simulation parameters, the excitation wavelength λ* _{ex}*=488 nm, the emission wavelength λ

*=515 nm, the objective (100×,*

_{em}*NA*= 1.4), and the pixel size is 40 nm. According to the Abbe’s diffraction limit theory, the maximum spatial frequency of the simulation is 184nm

^{−1}. We set the illumination pattern frequency at 260nm

^{−1}, corresponding to 70% of the maximum frequency supported by OTF. Figure 4(b2) shows the simulation performance of PFS with different Gaussian noise level added. For comparison, Wiener-filtered images and conventional SIM are also shown in Fig. 4(b1) and Fig. 4(b3). The red line box in the low left corner of Fig. 4(a1) shows the original enlarged part without noise level.

The reconstructed images are shown in Fig. 4(b1)–4(b3). They indicate that for the noise level less than 10%, the resolution of the images reconstructed by the PFS algorithm is better than that of images by Wiener filtering. The PFS algorithm can achieve similar reconstruction results to the conventional 9-frames SIM. In the simulations, we use peak signal-to-noise ratio (PSNR) and structural similarity index (SSIM) to evaluate the reconstructed images, as shown in Fig. 4(c1)–(c2). As the noise ratio rises, the quality of PFS images deteriorate more obviously than that of conventional 9 images SIM method, but the PFS algorithm still performs better than Wiener filter.

We also calculated the reconstruction time in our computer (CPU Intel i5) and reconstruct the raw images (image size: 512*512) in Matlab. The PFS algorithm reconstruction time is about 15-17s, which is much faster than the 75-78s required by the conventional 9-frame SIM. Reducing the raw image size to 200*200, the reconstruction time of the PFS algorithm and the conventional 9-frame SIM are 3-3.5s and 13-14s, respectively. Under different datasets, the reconstruction speed is about 5 times faster than conventional SIM methods. Other algorithms that reduce the number of SIM images, require parallel computing and iterative recovery. Those take more time to reconstruct a super-resolution image. For example, the ordinary least squares algorithm, it takes nearly 12 minutes to reconstruct a super-resolution image (150*150) in our computer.

## 4. Experiments and results

In order to prove the feasibility of PFS algorithm in experimental raw SIM images, we prepared BPAE cells as the samples. The experimental conditions are as follows. The excitation wavelength *λ _{ex}*=488 nm, the emission wavelength

*λ*=515 nm, the numerical aperture

_{em}*NA*= 1.49, and the pixel size is 60 nm. For comparison, we also reconstructed the experimental images using Wiener filter method. In order to ensure the validity of the data, we controlled the fringe frequency less than 86% of maximum frequency supported by the OTF.

Figure 5(a) is the reconstructed image by the PFS algorithm. The yellow line box in the low left corner shows the enlarged part. In order to show the details of the images, we enlarged the blue dotted box as shown in Fig. 5(b)–5(d). The wide-field image of Wiener filter method is shown in Fig. 5(b). Figure 5(c) shows the results of PFS algorithm. Figure 5(d) shows the results of conventional SIM. The details of the enlarged parts are mainly microtubules. Obviously, the reconstructed resolution of PFS algorithm is better than that of Wiener-filtered image, as shown in Fig. 5(b) and 5(c). The resolution of the image reconstructed by the PFS algorithm is closer to that of the conventional SIM. For a more intuitive view of the resolution enhancement, we plotted a section intensity of the red line in Fig. 5(a). Wiener-filtered only shows one peak, as depicted in the green line in Fig. 5(e). However, at the same position, the PFS algorithm and conventional SIM have two peaks. the PFS image can resolve 120 nm.

## 5. Conclusion

In this paper, we develop PFS algorithm for SIM to improve the frame rate. This algorithm can reconstruct a super-resolution image by using six raw low-resolution images instead of conventional nine images. In the PFS algorithm, the reconstruction is mainly completed by the subtraction in the frequency domain. The reconstruction process does not require multiple iterative recovery that mainly used in the determination of parameters. The simulations show that the reconstruction speed of PFS algorithm is about 5 times faster than that of conventional SIM methods. In the experiment, we employ PFS algorithm to reconstruct the super-resolution images of BPAE cells. In the experiments, we employ PFS algorithm to reconstruct the super-resolution images of BPAE cells. The reconstruction quality of PFS algorithm is close to the 9-frame algorithm, but the speed is much faster. Because there is a step in the PFS algorithm to estimate the spectrum by using Wiener filtering, the larger the noise ratio of the SIM image is, the more severe the distortion of the spectrum is. This is the characteristic of Wiener filtering. So, better signal-to-noise ratio of SIM images can bring better reconstruction effect. It is a promising algorithm for fast SIM imaging.

## Funding

National Natural Science Foundation of China (61505143); National Key Scientific Instrument and Equipment Development Projects of China (2014YQ510403); National Basic Research Program of China (973 Program) (2013CB933804).

## Disclosures

The authors declare that there are no conflicts of interest related to this article.

## References

**1. **S. W. Hell and J. Wichmann, “Breaking the diffraction resolution limit by stimulated emission: stimulated-emission-depletion fluorescence microscopy,” Opt. Lett. **19**(11), 780–782 (1994). [CrossRef]

**2. **R. Heintzmann and C. G. Cremer, “Laterally modulated excitation microscopy: improvement of resolution by using a diffraction grating,” Proc. SPIE **3568**, 185–196 (1999). [CrossRef]

**3. **M. Hofmann, C. Eggeling, S. Jakobs, and S. W. Hell, “Breaking the diffraction barrier in fluorescence microscopy at low light intensities by using reversibly photoswitchable proteins,” Proc. Natl. Acad. Sci. U. S. A. **102**(49), 17565–17569 (2005). [CrossRef]

**4. **S. T. Hess, T. P. Girirajan, and M. D. Mason, “Ultra-high resolution imaging by fluorescence photoactivation localization microscopy,” Biophys. J. **91**(11), 4258–4272 (2006). [CrossRef]

**5. **M. J. Rust, M. Bates, and X. Zhuang, “Sub-diffraction-limit imaging by stochastic optical reconstruction microscopy (STORM),” Nat. Methods **3**(10), 793–796 (2006). [CrossRef]

**6. **M. G. L. Gustafsson, D. A. Agard, and J. W. Sedat, “Doubling the lateral resolution of wide-field fluorescence microscopy using structured illumination,” Proc. SPIE **3919**, 141–150 (2000). [CrossRef]

**7. **M. G. Gustafsson, “Surpassing the lateral resolution limit by a factor of two using structured illumination microscopy,” J. Microsc. **198**(2), 82–87 (2000). [CrossRef]

**8. **R. Förster, K. Wicker, W. Müller, A. Jost, and R. Heintzmann, “Motion artefact detection in structured illumination microscopy for live cell imaging,” Opt. Express **24**(19), 22121–22134 (2016). [CrossRef]

**9. **X. Huang, J. Fan, L. Li, H. Liu, R. Wu, Y. Wu, L. Wei, H. Mao, A. Lal, and P. Xi, “Fast, long-term, super-resolution imaging with Hessian structured illumination microscopy,” Nat. Biotechnol. **36**(5), 451–459 (2018). [CrossRef]

**10. **J. Demmerle, C. Innocent, A. J. North, G. Ball, M. Muller, E. Miron, A. Matsuda, I. M. Dobbie, Y. Markaki, and L. Schermelleh, “Strategic and practical guidelines for successful structured illumination microscopy,” Nat. Protoc. **12**(5), 988–1010 (2017). [CrossRef]

**11. **P. J. Keller, A. D. Schmidt, A. Santella, K. Khairy, Z. Bao, J. Wittbrodt, and E. H. Stelzer, “Fast, high-contrast imaging of animal development with scanned light sheet–based structured-illumination microscopy,” Nat. Methods **7**(8), 637–642 (2010). [CrossRef]

**12. **P. Kner, B. B. Chhun, E. R. Griffis, L. Winoto, and M. G. Gustafsson, “Super-resolution video microscopy of live cells by structured illumination,” Nat. Methods **6**(5), 339–342 (2009). [CrossRef]

**13. **X. Zhou, M. Lei, D. Dan, B. Yao, Y. Yang, J. Qian, G. Chen, and P. R. Bianco, “Image recombination transform algorithm for superresolution structured illumination microscopy,” J. Biomed. Opt. **21**(9), 096009 (2016). [CrossRef]

**14. **S. A. Shroff, J. R. Fienup, and D. R. Williams, “Lateral superresolution using a posteriori phase shift estimation for a moving object: experimental results,” J. Opt. Soc. Am. A **27**(8), 1770 (2010). [CrossRef]

**15. **A. Lal, C. Shan, and X. Peng, “Structured illumination microscopy image reconstruction algorithm,” IEEE J. Sel. Top. Quantum Electron. **22**(4), 50–63 (2016). [CrossRef]

**16. **W. Kai, M. Ondrej, B. Gerrit, F. Reto, and H. Rainer, “Phase optimisation for structured illumination microscopy,” Opt. Express **21**(2), 2032–2049 (2013). [CrossRef]

**17. **K. Wicker, “Non-iterative determination of pattern phase in structured illumination microscopy using auto-correlations in Fourier space,” Opt. Express **21**(21), 24692–24701 (2013). [CrossRef]

**18. **D. Li, L. Shao, B. C. Chen, X. Zhang, M. Zhang, B. Moses, D. E. Milkie, J. R. Beach, J. A. Hammer 3rd, M. Pasham, T. Kirchhausen, M. A. Baird, M. W. Davidson, P. Xu, and E. Betzig, “ADVANCED IMAGING. Extended-resolution structured illumination imaging of endocytic and cytoskeletal dynamics,” Science **349**(6251), aab3500 (2015). [CrossRef]

**19. **A. Lal, C. Shan, K. Zhao, W. Liu, X. Huang, W. Zong, L. Chen, and P. Xi, “A frequency domain SIM reconstruction algorithm using reduced number of images,” IEEE Trans. Image Process. **27**(9), 4555–4570 (2018). [CrossRef]

**20. **F. Orieux, E. Sepulveda, V. Loriette, B. Dubertret, and J. C. Olivo-Marin, “Bayesian estimation for optimized structured illumination microscopy,” IEEE Trans. Image Process. **21**(2), 601–614 (2012). [CrossRef]

**21. **S. Dong, P. Nanda, R. Shiradkar, K. Guo, and G. Zheng, “High-resolution fluorescence imaging via pattern-illuminated Fourier ptychography,” Opt. Express **22**(17), 20856–20870 (2014). [CrossRef]

**22. **A. Lal, X. S. Huang, and P. Xi, “A frequency domain reconstruction of SIM image using four raw algorithm,” Proc. SPIE **27**(9), 4555–4570 (2018). [CrossRef]