1. Introduction

Fluorescence microscopy is an indispensable apparatus in biological research and clinical diagnosis, as it offers highly specific and highly sensitive information from biological samples. In many applications such as digital pathology and neuron science research, a large and uniform field fluorescence microscopy with high resolution is often desired to promise high throughput, especially for the samples on the slides [1,2]. Confocal imaging, such as Nipkow spinning disk confocal [3,4] microscopy and some multifocal confocal microscopy [5,6], are typically employed for higher resolution. However, the imaging field with high resolution is small due to the off-axis aberrations associated with optical elements in conventional confocal imaging systems. As for a large sample, these systems typically use raster scanning methods for imaging and construct full-view images from many small tiles. In this sequential imaging, stage movement and image refocusing are time-consuming and the image sensor remains idle during the movements, thereby limiting imaging throughput.

Confocal microscopy with a microlens array mitigates those limitations. For instance, Tiziani et al. proposed a conventional confocal system based a microlens array for surface topography [7]. Orth et al. first applied the system in fluorescence at a pixel throughput of up to 4 megapixels/s [8]. Furthermore, Shin et al. imaged with a micro-objective lens module composed of two microlens arrays to increase the working distance and achieved a centimeter-scale field of view (FOV) with a resolution of 1.55 µm [9]. Kim et al. demonstrated a high refractive index optical glass with 0.5 NA via rapid thermal imprinting [10]. In the conventional design, the microlens array operates as a parallel point scanning confocal objective lens and an iris functions as a common pinhole for each microlens to realize confocal imaging. Finally, the microlens is imaged to the camera through the relay lens. The whole imaging field is excited by a microlens array and can be scaled up by expanding the dimensions of the array, which means that it is very easy to obtain centimeter-scale image areas [9]. All of the microlenses can scan and image simultaneously as the stage moves, this parallel imaging strategy can significantly improve the overall imaging speed mainly by reducing non-imaging time. The resolving power of the system depends on the numerical aperture (NA) of the microlens. Each microlens detects only the focal plane information at the center of the optical axis and it is only affected by spherical aberration, therefore, the microlens array can provide a good resolution within a large area [8]. However, the realization of microlens array with very high NA is challenging because of the limitations of the optical geometry and microfabrication technologies of microlens arrays [11]. It is hard to satisfy many applications with finer structures. Reducing the focal length of microlens can increase NA, but the working distance will be very short. In this case, the microlens array easily contacts the sample when adjusting the system, thus being scratched and contaminated by the sample. Besides, in theory, one can enhance the resolution by shrinking the diameter of the iris. However, too much light is rejected, resulting in images with an unusable signal-to-noise ratio (SNR). this is particularly serious in fluorescence imaging.

There are also many super resolution methods that improve resolution for traditional point scanning imaging. For instance, ISM is a useful tool that attains the $\sqrt 2 $-fold resolution improvement for confocal microscopy in theory without sacrificing the detected signal intensity [12–14]. In ISM, the classical single-point detector is replaced with a detector array and each pixel acts as an individual confocal detector with a very small pinhole. Images recorded by each pixel are reconstructed using pixel reassignment (PR) to enhance resolution. However, it is difficult to apply this method in a conventional confocal system [8] with a microlens array because the camera is in a conjugate plane to the microlens array rather than the sample (the left of Fig. 1(1)). This configuration improves the resolution of the relay lens rather than the microlens array.

 

Fig. 1. Scheme of experimental setup. L: lens; M: mirror; HWP: half-wave plate; PBS: polarizing beam splitter; P: polarizer; SLM: spatial light modulator; TL: tube lens; DM: dichroic mirror; Obj: objective; MLA: microlens array; IMA: intermediate multi-spot array; TTS: tip/tilt stage; PZT: piezoelectric scanning stage; LP: long-pass filter. Inset (1): Comparison of the conventional confocal system based a microlens array and proposed system. Inset (2): Detailed diagram of the micro-imaging system through the microlens. ML: microlens; WD: working distance; BFL: back focal length; f: focal length; h: sag height; r: radius; θ: convergence angle.

Download Full Size | PDF

In addition, the microlens array is usually illuminated by a collimating Gaussian beam and this uneven illumination profile results in an intensity difference of excitation point through the microlens array. Especially at the edge of the field of view, the low brightness results in a poor signal-to-noise ratio while restricting many applications, such as quantitative intensity-based analysis [15].

Here, we present a large uniform imaging field confocal fluorescence microscopy with high resolution based on microlens arrays. We employed a microlens array to achieve large area imaging. An intermediate multi-spot array (IMA) generated by a spatial light modulator (SLM) aligned with the microlens array to form a uniform multi-optical probe array that illuminates the sample. Because the camera plane is conjugate to the sample plane with this optical configuration, we can implement ISM with a multi-spot adaptive pixel-reassignment (MAPR-ISM) method to improve the resolving power of system. Nonuniform illumination is compensated with SLM by an optimized doubly weighted Gerchberg–Saxton algorithm (ODWGS). Our design improves the resolution and uniformity of illumination compared to the conventional confocal microscope with a microlens array and SLM by introducing ISM. Fluorescent beads with a diameter of 200 nm were used to quantify the spatial resolution. And the uniformity of the imaging field was also evaluated by a uniform fluorescent plate. Experimental results of both mouse brain slices and melanoma on the slide are presented to verify the applicability and effectiveness of our method.

2. Methods

2.1 Experimental setup

The scheme of the system is shown in Fig. 1. A 488 nm (OBIS 488LS, Coherent, U.S.) laser beam is expanded by a pair of lenses (L1, f = 10 mm and L2, f = 250 mm). The collimated beam is then sent through a half-wave plate (HWP) to adjust the intensity. A polarizing beam splitter PBS (CCM1-PBS251/M, Thorlabs, U.S.) and a polarizer (P) are located on the beam path to generate horizontally polarized light to ensure the maximum modulation efficiency of SLM (PLUTO-2-VIS-016, HOLOEYE, Germany). The light projected onto a SLM will form a uniform multi-spot (50 × 50) pattern by superposing a Fresnel lens pattern (f = 129 mm). Subsequently, a relay 4f optical system (L3, f = 300 mm and L4 = 300 mm) images the multi-spot pattern at the back focal plane of a tube lens TL1 (f = 180 mm, TTL180-A, Thorlabs, U.S.) and an iris is placed at the Fourier plane of the 4f system to block the unwanted diffraction orders generated by the SLM. The excitation multi-spot array is relayed by the tube lens TL1 and an objective Obj at low magnification (1.25×, NA 0.04, UPLSAPO, Olympus, Japan) and reflected by a dichroic mirror DM (Di03-R488-t1-25 × 36, Semrock, U.S.) to form an intermediate multi-spot array aligned with a customized microlens array (70 µm pitch, 68 µm diameter, 115.5 µm effective focal length, fused silica, 1.46 refractive index, Vithin, China). The intermediate multi-spot array is then through the microlens array and illuminates the sample finally. Each microlens would demagnify the corresponding spot in the intermediate multi-spot array plane about 17 times, generating illumination focus limited by diffraction and aberrations (The effective NA of the microlens is about 0.23). Actually, the intermediate multi-spot array combines with a microlens array to form parallel optical probes. The emitted fluorescence from the specimen is collected by its respective microlens to the intermediate multi-spot array and passes through a long pass filter (FF02-520/28-25, Semrock, U.S.) to a scientific complementary metal-oxide semiconductor camera sCMOS (ORCA-Flash 4.0, Hamamatsu Photonics K.K., Japan) via the same relay system composed of Obj and TL2 (f = 180 mm, TTL180-A, Thorlabs, USA). In this optical configuration, the camera plane is conjugated to the sample plane so we can use some super resolution methods to improve system resolution. NA increases as the working distance (WD) of the microlens decreases [16]. However, as the working distance decreases, the magnification of spot in intermediate multi-spot array also increases, resulting in crosstalk between spots. In this study, the designed working distance of the microlens array is 130.8 µm to avoid crosstalk. The microlens array is mounted on a tip/tilt stage (#66-551, Edmund Optics, USA) to be adjusted to be parallel to the sample. Samples are imaged by raster scanning with a piezoelectric scanning stage (P-517.2CD, Physik Instrumente, USA). We apply digital pinhole around each point-spread function (PSF) point on the camera for confocal imaging [17].

2.2 MAPR-ISM method to improve resolving power

An intermediate multi-spot array was designed in a microlens array imaging system so that the camera plane is conjugate to the sample plane. In this optical configuration, we improved system resolution by implementing multi-spot adaptive pixel-reassignment ISM (MAPR-ISM). According to the concept of ISM, a detector array collects the distribution of emission excited by a laser focus, each pixel in the detector array acts like a very small pinhole confocal which records a higher resolution image. The images are spatially reassigned to compensate for the shift of images due to image detection at off-axis positions, which is the so-called pixel-reassignment (PR) method [12,13]. The effective PSF (PSF_eff) associated with the pixel (i, j) in the detector array could be calculated by the following equation:

Consequently, the effective PSF of the pixel would shift approximately half the distance compared to the displacement of the pixel from the optical axis displacement, as shown in Fig. 2(a). In this case, the images recorded by individual pixels in the detector array would be phase-shifted to each other. One could reassign the effective PSFs to the correct position by an appropriate shift (Fig. 2(a)) and sum up all to obtain the PSF of ISM PSF_ISM [13].

 

Fig. 2. Principles of MAPR-ISM. (a) The effective PSF (PSF_eff; black) as a product of excitation PSF (PSF_ex; blue) with detection PSF (PSF_det; red) at different pixel displacements. The position of the detection pixel in the detector array (3 × 3 pixels as a sample) is shown on the left. The yellow circle represents 1 AU Airy disk including 69 detection pixels. (b) A section of a raw camera frame. The white dashed circle represents 1 AU Airy disk. Scale bar, 100 µm. (c) A group of typical shift vectors (indicated by the red arrows) for the multi-spot adaptive pixel reassignment (MAPR-ISM). Scale bar, 500 nm.

Download Full Size | PDF

An appropriate PR factor is very important for resolution improvement in ISM. In general, because of the Stokes shift which results in excitation and detection PSFs of different widths and aberrations which change the shape of excitation and detection PSFs, PR factor a generally deviates from 1/2. In addition, the magnification of spots on the detector array is difficult to determine because the specific parameters of the micro-imaging system through microlens are difficult to measure. In this work, we apply an adaptive PR method to solve this problem (APR) [19]. Shift vectors ${{\boldsymbol s}_{\boldsymbol{ij}}}$ are determined from cross-correlations between the scanned images of the central pixel and the other pixels. The cross-power spectrum of two scanned images is given by:

In our system, the camera captures a fluorescence multi-spot array. We applied multi-spot APR-ISM (MAPR-ISM) to improve resolution. First, optical sectioning is performed by digital pinhole method, where the multi-spot array is multiplied by an aligned Gaussian mask array with a standard deviation of 1.8 pixels. Then each spot in the array is implemented APR-ISM. Each fluorescence spot will be captured by 69 detection pixels, which form a circular detector array with a diameter of 1 Airy unit (AU), as shown in Fig. 2(b). As the diameter is composed of 9 pixels, each pixel represents a sub-Airy pinhole of about $1.0\textrm{ AU /}9 = 0.11\textrm{ AU }$. We imaged fluorescent spheres to obtain the prior adaptive shift vectors of each spot and a group of typical adaptive shift vectors is shown in Fig. 2(c). Furthermore, we employ an iterative Richardson Lucy (RL) algorithm to maximize the effective resolution as used in other ISM work [12,20,21].

2.3 Uniform multi-spot array generated by a SLM with an optimized DWGS algorithm

In practice, the uniformity of the multi-spot array detected by the camera is deteriorated by many factors, such as a nonuniform input laser beam, aberrations due to the imperfect optical setups and the misalignments between the intermediate multi-spot array and the microlens array. Here, we employ an optimized doubly weighted Gerchberg-Saxton algorithm (ODWGS) using signal feedback from a camera to correct the nonuniformity.

The basic principle of the DWGS algorithm [22] for generating M spots array is shown in Fig. 3. The whole algorithm consists of a two-stage iterative process to generate corrected computer generated holograms (CGHs) on SLM: one is an inner iteration for designing a CGH with the conventional weighted-Gerchberg-Saxton (WGS) algorithm [23] and the other is an outer iteration in order to eliminate system-induced nonuniformity by adjusting the intensity of the multi-spot array. In inner iteration (WGS), the initial complex incident field of the SLM plane consists of the amplitude of the laser illumination and a random phase, the image plane amplitude A and phase ϕ are calculated from the initial complex incident field by a Fourier transform (FT). The calculated focus array pattern is compared with the desired focus array pattern to generate a new weighted amplitude. This combine with the previous phase in the image plane and an inverse FT (IFT) is performed to complete the first iteration. The generated phase is used for the next iteration combined with the initial amplitude. In the outer iteration, we use measured fluorescent spot intensity distribution on a camera to adaptively adjust target intensity if the uniformity is not satisfied. To accelerate the DWGS algorithm, here we introduce an optimized coefficient α. The kth weighted function ${v_k}\textrm{(}m\textrm{)}$ for mth spot is given as:

 

Fig. 3. Principle of the ODWGS algorithm to generate a uniform multi-spot array.

Download Full Size | PDF

2.4 Scanning and reconstruction methods

A large imaging area (3.5 mm × 3.5 mm as an example in this work) is divided into many sub-regions by the microlens array. Each microlens acts as a point scanning confocal microscope. Each fluorescence focus is recorded by a group of pixels in a small circular region on the sCMOS camera. The camera acquires a video at 250 frames per second (fps) with an ROI of 700 × 700 pixels. The pixel exposure time is 4 ms. The sample is raster scanned by a closed loop piezo stage in two dimensions to collect information of sample. The piezo stage is driven by a sawtooth wave and scans along the x direction at a speed of 100 µm/s. The scan speed must match the camera frame to avoid image shearing. The raster scan step size is set to be 0.4 µm. Each microlens scans a region of 78 µm × 78 µm which is larger than microlens pitch (70 µm × 70 µm) for image stitching. Here, the image area in microlens array is 3.5 mm × 3.5 mm which contains 2500 microlenses. The acquired data is about 102.4 megapixels while resulting data is about 76.6 megapixels, considering the overlap between tiles. The ultimate ratio between the acquired data and the resulting data is about 1.33. Data recorded by the sCMOS camera is streamed to the computer memory via CameraLink at high speed. Then, the raw data will be processed offline.

Reconstruction method for the full-field image is shown in Fig. 4. As mentioned above, we find the locations of all illumination focus with maximum value projection of raw images at first. Each raw image is multiplied by an aligned Gaussian mask array for digital pinhole processing. Then we reconstruct a single sub-FOV with APR method. Each fluorescent focus will be detected by a small circular region pixel array corresponding 1 AU Airy disk. The diameter of the circle region is 9 pixels and there are 69 shifted sub-Airy pinholes. The image of a sub-FOV generated by one pixel is shifted to each other. Then the images will be shifted back to optical axis (the central pixel as a reference) with priori shifting vector and summed up to implement MAPR-ISM. We can get CLSM images by summing all scanned images generated by each pixel without displacement correction. The full-field image is constructed by stitching together all of the sub-FOVs with nonlinear blending [24]. Finally, the full-field image is deconvolved using the Lucy–Richardson deconvolution algorithm implemented in MATLAB to get better resolution.

 

Fig. 4. Scan and reconstruction method for the full-field image with MAPR-ISM^RL. In adaptive pixel-reassignment (APR) of a sub-FOV, all the 69 shifted images produced by the individual pixel are aligned with the image from the central pixel. Shifted images sum up directly to reconstruct the conventional laser scanning microscope (CLSM) image shown in the dashed box.

Download Full Size | PDF

3. Experiment results

3.1 Imaging resolution

In order to evaluate the resolution of our system, we imaged 200 nm yellow/green fluorescent spheres (FluoSpheres Carboxylate-Modified Microspheres, ThermoFisher, USA). the sub-micron beads are smaller than the optical spot size that can be approximated as PSF of the system. The raster scan step pitch is 0.4 µm and exposure time is 4 ms per pixel. We compared three reconstructed images of the fluorescent spheres: the conventional laser scanning microscope image (CLSM) with 1 AU, the ISM image after the multi-spot adaptive pixel reassignment procedure (MAPR-ISM) and the MAPR-ISM image with RL deconvolution (MAPR-ISM^RL). The typical bead images are shown in Fig. 5(a). For evaluating the resolution improvement, the two-dimensional Gaussians function was fitted to the intensity distributions of the spheres (Fig. 5(b)). The mean full-width-at-half-maximum (FWHM) of the spheres images is 1.35 ± 0.05 µm for the CLSM image, 1.01 ± 0.03 µm for MAPR-ISM image and 0.82 ± 0.05 µm for MAPR-ISM^RL respectively. The resolution of MAPR-ISM method is improved by 1.34 times compared with CLSM. After RL deconvolution (iteration = 10), the image is further improved by 1.64 times. The theoretical value is 1.08 µm for CLSM (1 AU) and 0.76 µm for ISM according to the simulation results based on Eq. (2) and Eq. (3). The measured PSFs are slightly worse due to spherical aberration introduced by reflow microlens profile [25] and fabrication errors of the microlens. We also measured resolution with Fourier ring correlation (FRC) method [26] and FRC measurements for the different types of images are shown in Fig. 5(c). The FRC resolution is 1.44 µm for the CLSM image, 1.06 µm for MAPR-ISM image and 0.87 µm MAPR-ISM^RL, respectively. The FRC analysis demonstrated similar results to FWHM measurements: the APR-ISM method improves the resolution by a factor of ∼ 1.36, which is further improved to a factor of ∼1.65 with MAPR-ISM^RL method. We also evaluated the resolution of MAPR-ISM^RL image in different areas in the imaging field. The results along the diagonal of the FOV are shown in Fig. 5(d). The diagonal length is about 4.9 mm and we measured PSFs of 10 spheres at 1.2 mm intervals. The mean FWHMs of spheres are 0.86 ± 0.06 µm, 0.83 ± 0.04 µm, 0.82 ± 0.04 µm, 0.84 ± 0.05 µm, and 0.88 ± 0.06 µm, about 2.4 mm, 1.2 mm, 0 mm, -1.2 mm, -2.4 mm from the center of imaging field, respectively. The results indicate that the imaging quality is homogeneous across the whole imaging field.

 

Fig. 5. (a) Comparison among CLSM(1 AU), MAPR-ISM, and MAPR-ISM with RL deconvolution (iteration = 10) of a 200 nm fluorescent sphere. Scale bar is 1 µm. (b) Plot of a section through the center of the fluorescent sphere for the different images in a. The FWHMs (Gauss fit) of the sections are 1.35 ± 0.05 µm, 1.01 ± 0.03 µm, and 0.82 ± 0.05 µm, respectively. (c) Resolution (based on the FRC analysis and the 1/7 threshold value) for the three different imaging modalities. The FRC resolution is 1.44 µm for the CLSM image, 1.06 µm for MAPR-ISM image and 0.87 µm MAPR-ISMRL, respectively. (d) Resolutions with MAPR-ISM^RL in different areas along the diagonal of FOV. The mean FWHMs of spheres are 0.86 ± 0.06 µm, 0.83 ± 0.04 µm, 0.82 ± 0.04 µm, 0.84 ± 0.05 µm, and 0.88 ± 0.06 µm, about 2.4 mm, 1.2 mm, 0 mm, -1.2 mm, -2.4 mm from the center of FOV, respectively. The corresponding positions are shown as insets at the upper left corner.

Download Full Size | PDF

Furthermore, we optimized the resolution by selecting an appropriate back focal length (BFL) of the microlens. NA increases as the working distance of the microlens decreases, as indicated in Eq. (7). The working distance is difficult to determine in imaging, in practice, we adjusted the back focal length BFL of the microlens to change the working distance according to the Eq. (8). On the other hand, the ability of resolution improvement with MAPR-ISM method is also affected by BFL of the microlens. Resolution improvement in ISM depends on the size of single elements in the detector array [13]. If the Airy disk projected onto the camera is larger, the individual pixel acts as a smaller pinhole compared with the Airy disk and we could obtain a better resolution. In our system, the magnification of Airy disk is M_relay × M_MLA, here M_relay is the magnification of the relay lens (1.25 ×) and M_MLA is the magnification of the micro-imaging system through a microlens (Fig. 1(2)). M_MLA is adjusted by the back focal length of the microlens. The relevant equations are as follows:

 

Fig. 6. The measured and simulated resolution results of CLSM, MAPR-ISM are shown as a function of BFL. The measured result of MAPR-ISM^RL is also shown.

Download Full Size | PDF

3.2 Imaging uniformity

We designed a CGH to produce a multi-spot array consisting of 50 by 50 optical spots. The CGH was composed of 1000 × 1000 pixels and was superimposed by a phase compensation pattern to correct wavefront distortion [27]. The region-of-interest (ROI) of intensity calculation for each spot was chosen the same as the ROI of 1 AU pinhole in MAPR-ISM (69 pixels on the camera in total). We used a uniform fluorescent plate as a sample to obtain the weighted coefficient of target multi-spot array. The initial multi-spot generated by the conventional WGS algorithm (24 inner iterations) is shown in Fig. 7(a). we can see the nonuniformity in the array which was caused by the uneven illumination, shading effect and so on. The fluorescence multi-spot array was deteriorated by the uneven distribution of the light field. The multi-spot array optimized by the ODWGS algorithm with 3 outer iterations is also shown in Fig. 7(b). The uniformity of the fluorescence spot array has been improved a lot. Intensity nonuniformity σ is reduced to 3% while the initial value before optimization is 26%. Comparison of intensity profiles along the columns of fluorescence spots before and after optimization is shown in Fig. 7(c). The corrected result with the ODWGS algorithm efficiently redistributes the fluorescence intensity over a large area. The intensity histogram of fluorescence spots also shows that the corrected spot array has a narrower distribution (Fig. 7(f)).

 

Fig. 7. Uniformity comparison between an ODWGS method and the conventional WGS method. (a), (b) Camera raw image of 50-by-50 fluorescent spots array generated by the conventional WGS method and an ODWGS method respectively. (c) Intensity profiles measured along lines in (a), (b). (d), (e) Image of uniform fluorescent plate generated by the conventional WGS method and an ODWGS method respectively. (f) Intensity histogram of fluorescence spots in (a), (b). Initial intensity nonuniformity (standard deviation) is 26% (blue), which is reduced to 3% (red) with the ODWGS method.

Download Full Size | PDF

In order to illustrate the uniformity improvements in image acquisition, we imaged the uniform fluorescent plate. Each optical focus was raster scanned to generate a sub-image. The final scanned image results are shown in Fig. 7(d, e), the intensity distribution of both the edges and the center of FOV is greatly improved with the ODWGS algorithm. Our method can effectively increase the imaging field and reduce the brightness difference between sub-images. That will improve accuracy in many biological quantitative analyses, such as cell counting.

3.3 Tissue imaging

To demonstrate the applicability and effectiveness in large fluorescence imaging with our method, we imaged a transgenic mouse brain slice and a melanoma slice. All animal experiments were approved by the Institutional Animal Ethics Committee of Huazhong University of Science and Technology. We imaged a 50-µm-thick Thy1-YFP mouse transgenic brain slice in 164 s. The imaging area is 3.5 mm × 3.5 mm with an output power of 50 mW. The whole FOV is stitched by 50 × 50 = 2500 mosaic images, each with a size of 70 µm × 70 µm. The stitched image has no distinct dark borders between imaging areas. Imaging results were processed by CLSM, MAPR-ISM, MAPR-ISM^RL (10 iterations) respectively and shown in Fig. 8. Here the full field images are shown in Fig. 8 (a-c) and Fig. 8 (a1, b1, c1) are enlarged views of the area 1 with red box corresponding with Fig. 8 (a, b, c). From the magnified views in Fig. 8 (a1, b1, c1), we can observe two close fiber structures could be distinguished clearly in MAPR-ISM image while the corresponding region is blurred in CLSM image. MAPR-ISM^RL method further improves image contrast and shows more details. The normalized intensity profile plot across the structure highlighted by the red arrowheads is shown in Fig. 8 (d). Similar results are shown in Fig. 8 (a2, b2, c2). This demonstrates a significant resolution improvement with MAPR-ISM^RL method.

 

Fig. 8. A 3.5 mm × 3.5 mm image of a 50-µm-thick brain slice. (a) CLSM image. (b) ISM image. (c) ISM with deconvolution. (a1-c1) Enlarged views of the area 1 corresponding to (a-c). (a2-c2) Enlarged views of the area 2 corresponding to (a-c). (d-e) Normalized intensity profiles of the same position indicated by the red arrowheads in (a1-c1) and (a2-c2) respectively. Scale bars in (a-c): 0.5 mm. Scale bars in (a1-c1) and (a2-c2): 50 µm. Scale bars in (i-vi): 10 µm.

Download Full Size | PDF

We also imaged a 50-µm-thick Thy1-GFP melanoma slice to demonstrate uniform imaging and applicability in different biological tissues. The sample has an extent of 3.5 mm × 3.5 mm. The main imaging parameters are the same as those of the mouse brain slice imaging, except laser power was set to 100 mW. The results are shown in Fig. 9. The imaging results with our system showed an obvious improvement in the uniformity for full-field imaging. In contrast, without intensity calibration, the spatially varying intensity produced obvious borders between sub-FOV (Fig. 9(b) and (b1)). Melanoma cells near the edge of the imaging area have poor SNR and resolution because of weak illumination intensity. Imaging results were processed by CLSM, MAPR-ISM, MAPR-ISM^RL (10 iterations) respectively and shown in Fig. 9(i-iii). The melanoma cell granules can be clearly distinguished after MAPR-ISM processing. Image resolution and contrast would be further improved with MAPR-ISM^RL. The normalized intensity profile plot across the structure highlighted by the red arrowheads is shown in Fig. 9(c). This improvement is particularly pronounced in the case of weak fluorescence signals. No photobleaching effect is observed in our system at the current illumination intensity.

 

Fig. 9. A 3.5 mm × 3.5 mm image of a 50-µm-thick Thy1-GFP melanoma slice. (a-b) The full-field MAPR-ISM^RL image with and without ODWGS method, respectively. (a1-b1) Enlarged views of the areas indicated by yellow boxes in (a-b), respectively. (a2) Enlarged view of the area indicated by the blue box in (a). (i-iii) Enlarged views of the area indicated by the red box in (a2) with MAPR-ISM^RL, MAPR-ISM, and CLSM, respectively. (c) Normalized intensity profiles of the same position indicated by the red arrowheads in (i-iii). Scale bars in (a-b): 1 mm; Scale bars in (a1-b1): 100 µm; Scale bars in (a2): 50 µm; Scale bars in (i-iii): 20 µm.

Download Full Size | PDF

4. Discussions

Confocal microscopy with microlens arrays is easily scalable to obtain a large FOV. We use an intermediate multi-spot array illumination generated by SLM. The camera is conjugated to the sample via the intermediate plane that provides two functions. First, an MAPR-ISM super resolution method can be employed to improve resolution. It should also be noted that our system can easily be combined with other computational imaging to get better imaging quality, such as virtual fluorescence emission difference [28] and subtractive imaging [29]. Second, nonuniform illumination introduced by the imperfections in optics is easy to compensate using an SLM with adaptive feedback.

In our multifocal confocal system, the resolution is decided by NA of the microlens. Because the microlens array used for verification is relatively low (∼0.23), the system resolution is only 0.82 µm with MAPR-ISM^RL finally. Low N.A microlens has a low spatial resolution and the system has a higher scanning speed, and we have demonstrated that the details of neurons and cancer cells could be revealed with our system. The spatial resolution can be greatly improved by employing high refractive index glass microlens arrays or diffractive microlens arrays [30] which achieve very high numerical apertures. Parallel point scanning with the microlens array reduces non-imaging time, and it could be powerful tool in applications highly desirable for high throughput with a large FOV [31]. However, the throughput of the demonstrated system is only about 0.62 megapixels/s while commercial spot scanning confocal system, for example Olympus FV3000RS, is about 1-2 megapixels/s under similar conditions [32]. The data acquisition efficiency is mainly limited by the speed of a camera, the size of a microlens array, the pitch of the microlens and the size of Airy disk projected onto the camera. But as the increasing of parallelized channel, higher laser power is required to promise the signal-to-noise ratio. In future work, we could employ a CMOS camera with higher speed. The diameter of the Airy disk projected onto the camera can also be optimized to be 5 pixels. Because each pixel represents a sub-Airy pinhole of about 0.2 AU and the system could still have good ISM effect [33]. We will employ a high-power laser to satisfy our imaging conditions. In this case, data acquisition efficiency may increase by 10 times.

The multi-focus imaging system is easily affected by cross-talk. In our system, we used three methods to eliminate the influence. First, the size of Airy disks was adjusted carefully by changing the magnification of the micro-imaging system through the microlens (Fig. 1(2)) for less cross-talk. Second, the distance between adjacent Airy disks was optimized by selecting an appropriate magnification of the relay lens. Third, each spot in raw images was multiplied by a 2D Gaussian mask to implement digital pinhole preprocessing. The influence of cross-talk caused from a thick sample can be effectively suppressed. We do not observe obvious cross-talk in our images.

5. Conclusions

In this manuscript, a large uniform imaging field confocal fluorescence microscopy method with high resolution based on a microlens array is proposed. Large scale applications become possible by expanding the dimensions of the array. Nonuniform illumination introduced by the imperfections in optics is compensated with SLM by the ODWGS algorithm. The resolution of our prototype is about 1.6 times better with MAPR-ISM after RL deconvolution compared with traditional confocal processing. The results of fluorescent beads, mouse brain slices and melanoma slices confirmed the applicability and effectiveness of our proposed confocal imaging system. A microlens array and SLM are introduced for the first time in a confocal imaging system, which overcomes the limitation of resolution and nonuniformity of illumination of this system. The proposed method will be particularly attractive in microfluidic imaging for high-throughput cytometry due to the large imaging field with high resolution [34].