1. Introduction

The performances of large aperture optical imaging systems are severely influenced by phase aberrations [1], especially when imaging is performed through turbid and/or turbulent media. In order to compensate for such aberrations, adaptive optical elements, such as deformable mirrors, are often used in the design of imaging equipment [2].

The techniques employed to estimate the wavefront aberration and correct it through an adaptive optical element are known as adaptive optics, and are considered a gold standard in optical astronomy and high power laser applications.

Due to the high numerical aperture of the optical systems used, and the inherent optical imperfection in biological samples, optical sectioning fluorescence microscopy techniques would greatly benefit from the application of adaptive optics [3]. In particular, in traditional confocal microscopy techniques, based on point-like excitation and detection through a confocal pinhole, the presence of any aberration causes a loss in signal, as the confocal image of the excitation point becomes wider than the confocal pinhole, resulting in a drastic reduction in the detected fluorescence signal.

Several approaches to the implementation of adaptive optics in optical sectioning microscopy have been proposed and experimentally proven. These approaches can generally be subdivided into direct sensing of the wavefront [4], or iterative metric optimization [5].

Direct sensing is usually performed through a Shack-Hartmann wavefront sensor, and, where applicable, is considered the most effective approach to adaptive optics, as it can provide high frequency, closed loop correction of optical aberrations, guaranteeing the best possible image quality at any given time, even in the presence of dynamic aberrations. However, the use of a wavefront sensor requires the presence of a bright point-like source in the image plane. While such a condition is easily satisfied in astronomy and laser optics applications, fluorescence microscopy is by its own nature mostly employed for imaging weakly fluorescent extended objects, so that the use of wavefront sensors is challenging, often requiring specifically prepared samples with the inclusion of bright fluorescent point like objects to be used as artificial references [6,7].

An alternative approach to adaptive optics in microscopy is provided by optimization techniques. In such case, a metric of the image quality is computed based on the acquired images (e.g. average fluorescence intensity, image sharpness, frequency content), or on a quality metric based on an artificial illumination pattern [8], and an iterative or model based optimization algorithms are applied to estimate the optimal wavefront correction. This approach has the advantage of being easily applicable on most samples, but has the downside of requiring the acquisition of multiple images in order to perform a correction. Given the relatively low frame rate of existing optical sectioning microscopes, the optimization can require several seconds or minutes, precluding dynamic correction of time dependent aberrations, such as those introduced during the acquisition of a focal stack of images, or by dynamic changes of the sample itself.

Moreover, this makes extensive use impractical, since in particular it hampers observation due to the build-up of photobleaching during the optimization procedure.

This paper presents an alternative approach to adaptive optics in optical sectioning fluorescence microscopy. Instead of implementing adaptive optics as an add-on to traditional optical sectioning setups (e.g. laser scanning confocal microscopy, spinning disk microscopy), a dedicated system was realized in order to achieve comparable image quality, with substantially faster aberration correction. The system uses rejected out of focus light from confocal apertures, to compute a performance metric at high frequency during a scanning procedure, allowing for optimization sampling at a much faster rate than the imaging speed of the system.

2. Technique description

High speed dynamic correction of aberrations in optical sectioning microscopy can be obtained, without a wavefront sensor, only if a performance metric can be estimated at a frequency higher than the microscope frame rate.

Since most optical sectioning techniques rely on sample scanning, such performance metric should be independent from the dynamics of the imaging scanning procedure. This is practically impossible in a traditional, single beam, laser scanning confocal microscope, as the only signal acquired, namely fluorescence intensity, varies continuously throughout the field of view. As a consequence, a performance metric calculated over the full image scan is generally used, limiting the optimization frequency to the imaging frame rate. However, if the fluorescence signal is acquired simultaneously in multiple positions uniformly distributed throughout the field of view, the inherent inhomogeneity of the sample is averaged out, and a metric can be computed in a time interval shorter than the full frame time. Moreover, the simple signal intensity has a relatively poor information content regarding the aberration, and a more meaningful measurement, such as the shape of the excitation spots, would make for a more effective metric.

On the base of such considerations, this paper presents an adaptive, multi-aperture confocal microscope, based on a Digital Micromirror Device (DMD) [9]. Such system is capable of measuring the distribution of signal intensity in the proximity of a confocal excitation spot, in a parallelized scanning procedure with multiple programmable apertures.

While never used for adaptive optics applications, the possibility of achieving optical sectioning in fluorescence imaging through a DMD device was first proven by Hanley et Al. in 1998 [10,11], under the name of Programmable Array Microscope (PAM).

The device hereby presented is a Smart Programmable Array Microscope (S-PAM), in which a standard PAM is modified with a high speed camera detecting out of focus fluorescence, and with a deformable mirror in a pupil plane, used to correct aberrations in the system.

A DMD is a digitally controlled optical device, consisting of a bi-dimensional array of reflective elements, which can be individually tilted in a binary fashion at an angle of + α (ON mirror) or −α (OFF mirror). In a S-PAM microscope, the DMD is located in an image plane in the optical path shared by excitation and fluorescence light, at an angle +2α between the normal of the mirror and the excitation light optical axis. In such position, the DMD acts as a binary intensity modulator, deflecting light in the direction of its normal where its elements are ON, and at an angle −4α, far from the useful aperture of the system, where its elements are OFF.

In such configuration, optical sectioning is achieved when only a subset of micromirrors is ON at any given time, as the micromirrors act as point sources for excitation light, and as conjugated pinholes for the fluorescence light. When operated this way, the DMD is practically equivalent to the pinhole disk of a spinning disk microscope [12], and an optically sectioned image can be acquired with a pixelated detector, if all micromirrors turn ON for an equally long period during the detector exposure time.

An important advantage in the use of a DMD instead of a spinning disk is given by the possibility of detecting the out of focus fluorescence light rejected by the pinholes, by realizing a secondary imaging arm (Optimization arm), identical to the one between the DMD and the imaging camera, at an angle −2α from the DMD surface. In such configuration the optimization camera, operated at rates equal or faster than the DMD refresh rate, would detect no light from the locations of the ON pixels, and reveal the out of focus light rejected by the OFF pixels.

It is important to notice that, even in a diffraction limited situation, if the ON pixels are organized in clusters of size equal to the Airy diameter of the system, as usually done in confocal setups, part of the confocal light will in any case be imaged on the OFF mirrors adjacent to the ON cluster, due to diffraction and to the longer wavelength of fluorescence compared to the excitation light. This results in a bright, high contrast area at the edges of the dark image of the confocal aperture. In the presence of optical aberrations, this area becomes wider, and less contrasted compared to the out of focus background light. The width of this bright area, analytically calculated as the second moment of the signal distribution, can be proved to depend quadratically from the amplitude of the aberration [13], and can therefore be used as a metric of the correction performance of an adaptive optical element incorporated in the system.

Since the use of a DMD instead of a spinning pinhole disk does not allow the use of microlenses to maximize the excitation light power at the sample, the S-PAM system has very low excitation efficiency. To compensate for this inconvenience, a high power LED can be employed as excitation source, as the system, by its own nature, does not require coherent excitation light. A further measure adopted for maximization of the signal is scanning patterns consisting of slits instead of arrays of pinholes. The slit pattern was chosen over alternatives(high density grids, random distributions of active pixels) as its geometric regularity ensures minimization of cross talk between apertures, and easy computation of the metric on the acquired images. This solution is often employed in confocal systems [14,15], guaranteeing higher frame rates and signal intensities, at the cost of a slight loss in axial resolution.

3. Optimization procedure

The optimization procedure in the S-PAM microscope is aimed at the minimization of the second moment of the fluorescence intensity distribution in proximity of the confocal slit image on the optimization camera.

Due to the one dimensional symmetry of the slit, some anisotropic aberrations (e.g. cylinder) can not induce detectable variations in the second moment of the distribution. In order to obtain two dimensional symmetric information, the scanning of the sample is performed alternating patterns with horizontal and vertical slits on the DMD, and the exposure of the optimization camera is kept equal to twice the refresh time of the DMD. In such a configuration, the images acquired by the optimization camera show a dark orthogonal grid of confocal slits. The metric is computed as the average second moment of the fluorescence distribution along all visible confocal slits in the image.

Optimization of the metric was achieved through online application of the Data-based Online Nonlinear Extremum-seeker (DONE) algorithm, recently developed and applied in optical coherence tomography by the authors [16].

The DONE algorithm is an optimization algorithm based on the recursive creation of a nonlinear model of the cost function during the optimization procedure itself. At the initialization of the algorithm, a random set of M cosine functions C₁(x),…, C_M (x) is generated. The cost function model, at every step k is defined as:

At the k-th step, the algorithm performs a nonlinear fit on the previous k measurements of the metric, determining the values of the parameters A_1,k, …, A_m,k which best approximate the cost function. The coordinates of the (k + 1) − th measurements are determined as those of the minimum of function f_k plus a small random perturbation. The minimum of f_k is estimated through nonlinear optimization.

As discussed in [16], the use of the DONE algorithm over more classical approaches (e.g. Stochastic gradient descent, hill climbing) provides multiple advantages, such as a high convergence rate and reliability in the presence of low signal to noise ratio.

As an additional advantage for this specific application, the DONE algorithm can run in an indefinite iteration loop, providing online correction for slow temporal dynamics in the aberration, without the need of artificially introducing severe aberrations during the imaging process. In order to adapt the DONE algorithm model to dynamic variations in the aberration, the fitting step of the algorithm is only performed on a subset of recent metric measurements.

Since the exposure of the optimization camera is directly triggered by the DMD refresh, even a minimal computation time causes the loss of frames, effectively halving the optimization iteration frequency. In order to avoid this effect, the optimization procedure runs on three separate, asymmetric threads: one for image acquisition, one for metric estimation, and one for DONE iterations. The refresh rate of the DMD and the active region of the optimization camera are set to a value that ensures that both the computation of the metric and one iteration of the DONE algorithm take a shorter time than the exposure time of the optimization camera. At any given time while the camera is exposing image n, the metric computation thread is computing the metric on image n-1, while the DONE algorithm is performing iteration n-2. A timing scheme is reported in Fig. 1.

 

Fig. 1 Illustration of the timing and synchronization of the DMD, the two cameras, and the optimization algorithms.

Download Full Size | PDF

This leads the algorithm to always miss the two most recent data points in fitting the model, but the effect of this error on the convergence rate is negligible compared to the reduction in the sampling frequency which would be introduced by a synchronous application of the algorithm.

4. Experimental setup

The technique was tested on a custom optical setup, entirely built on a 60×90 cm² vibration isolated optical breadboard. The system is based on a DMD (Lightcrafter 6500 Evaluation Module, Texas Instruments, U.S.) with 1920×1080 square elements of 7.5 μm side length, tilting ±12 degrees with respect to the DMD surface normal, and a maximum refresh rate of 8 kHz. Excitation light is provided by a high power LED (PT-121-B, driven by DK-136M-1 development kit, Luminous, U.S), emitting a total radiant power of ≈ 20W, with a 470 nm dominant wavelength, from a 4×3 mm² surface. Cooling of the LED is provided by a standard computer CPU liquid cooler (H90, Corsair, U.S.) with a custom 3D printed mount. An image of the LED surface is formed at the DMD plane by a 26 mm focal length condenser lens with a numerical aperture of 0.55 (ACL3026U, Thorlabs, U.S.). Wavelength filtering is provided by a standard GFP epifluorescence imaging filter set(MDF-GFP, Thorlabs, U.S.). The optical path of the LED light is tilted at 24 degrees with respect to the normal to the DMD surface.

A back aperture plane is created by conjugating the DMD plane to infinity through a 150 mm achromatic doublet (AC-508-150-A, Thorlabs, U.S.). A 1cm diameter, 69 actuators deformable mirror (DM-69, Alpao, France) is positioned in this back aperture plane at 150 mm from the lens, with a pitch angle of 12 degrees. A telescope formed by two 150 mm focal achromatic doublets is used in 4f configuration to conjugate the DM plane with the back aperture of a microscope objective (40X, 1.0 N.A. water dipping, Carl Zeiss, Germany), in upright configuration. Brightfield imaging is obtained by standard Kohler illumination, while uniformly keeping all elements of the DMD in ON position. A schematic representation of the prototype is reported in Fig. 2.

 

Fig. 2 Optical setup. Left image is a simplified scheme missing a 4f system between the adaptive element and the objective. Right image is the full three dimensional structure of the setup. Blue lines are the excitation light optical path, green lines are the fluorescence light optical path, orange lines are the optical path shared by excitation and fluorescence light. LED:light source, DC:Dichroic cube, DMD:Micromirror device, DM:Deformable mirror, O:Objective, IC:Imaging camera, OC:Optimization camera.

Download Full Size | PDF

The surface of the DMD is imaged on two identical 1920×1080 pixels sCMOS cameras (Optimos, QImaging, Canada), with magnification achieving 1 to 1 pixel mapping. One camera acts as the imaging camera, positioned after the dichroic mirror on the excitation light path, the other one acts as an optimization camera on a symmetrical optical path, at −24 degrees from the normal to the DMD surface, with a standard GFP emission filter to reject scattered excitation light. In order to guarantee distortion-free imaging, the optical conjugation between the DMD and the cameras is performed through telecentric variable magnification lenses (TEC-55 with 2× adapter, Computar, Japan) in Scheimpflug [17] configuration. In order to ensure correction of system induced aberrations, correction was performed on a sample of fluorescent microbeads dried on a thin coverslip, which is assumed to be aberration free, in order to determine an ideal mirror configuration, that was stored as a starting reference point for all other measurements.

5. Single image optimization results

Performance tests were performed on a 16 μm thick tissue slice (Fluocells prepared slide 3, mouse kidney with Alexa Fluor 488, Invitrogen, U.S.A.), at an approximate depth of 10 μm. While the tissue slice is very thin, experiments showed that some sample induced aberrations are present, and the use of an adaptive microscope can increase the image quality.

Images were obtained scanning 15 horizontal slits patterns and 15 vertical slits patterns, for a total of 30 patterns, at a refresh rate of 270 Hz, resulting in an optimization frequency of 135 Hz. Each slit is three DMD pixels wide (approximately 420 nm at the sample plane), with a spacing between slits at the image plane of 6.3 μm.

The imaging camera acquired full images of the sample at a frequency of 3Hz, integrating a single image over three full scans of the field of view. In order to achieve an adequate framerate in the image acquisition, optimization was performed on a 300×300 pixels subregion of the optimization camera. This is equivalent to selecting an area of interest in the image, which would result, in the presence of strongly anisoplanatic aberrations, in a local correction. In such scenario, the user of the microscope would need to chose to either reduce the field of view, or reduce the speed of the correction, which would however still remain faster than the microscope’s imaging speed. The optimization procedure was performed using as a Zernike coefficients base, excluding tip, tilt and defocus, up to the 5th order, for a total of 18 degrees of freedom. The DONE algorithm was run with a total of 600 cosine functions, and fitting on a set of the last 50 metric measurements.

The convergence speed for sample induced aberrations was measured by selecting a focal plane of interest, setting the mirror to its pre determined ideal configuration, and running the optimization for 5 seconds. Optimal convergence was consistently obtained within 1 second from the first step of the optimization procedure. A representative measurement is reported in Fig. 3. The metric value was decreased from ≈ 2.0 to ≈1.7 Airy radii, resulting in an increase in signal of ≈20%, as well as in the appearance of sharp details in the image, which are not distinguishable in the uncorrected image (see panel 3 in Fig. 3).

 

Fig. 3 Example of aberration correction: 1: Sample image. a) without aberration correction. b) after optimization with DONE algorithm. 2: Metric value as a function of time. Vertical dashed lines mark exposure times of the imaging camera. The empirical value for optimal correction is reported as dashed horizontal line. Optimal correction is achieved within 1 second. 3: Detail of the image a) without aberration correction. b) after 5s optimization. 4: Sample image from the optimization camera a) without aberration correction. b) after 5s optimization.

Download Full Size | PDF

It must be noted that, while the second moment metric is reported in Airy radii, the diffraction limited correction for the system would always be over 1.0 Airy radii, due to the presence of a dark stripe at the center of the PSF image on the optimization camera, and for the presence of out of focus background. Empirically, from the microscope testing in a variety of situations, for a thin sample with relatively low background, diffraction limited performances lead to second moment values of ≈ 1.6 – 1.7 Airy radii.

6. Comparison with traditional optimization procedures

In order to compare the performances of the system with those of a standard sensorless adaptive microscope, a known aberration was artificially introduced by setting the deformable mirror actuators to random voltages, and correction of the same aberration, on the same field of view, was performed both through the S-PAM optimization, and through hill climbing optimization of an image metric computed on the imaging camera output. The Hill climbing optimization was performed acquiring 5 images for each of the 18 Zernike polynomials, leading to a total optimization time of 30 seconds. The image metric employed was image sharpness [18], defined as:

 

Fig. 4 a) Sharpness metric convergence speed for an artificially induced wide amplitude aberration. Dashed line is the result of a traditional hill climbing procedure, optimizing for sharpness, performed on sharpness data of the imaging camera, at 3Hz acquisition frequency. Solid line is the recorded values of the sharpness metric during S-PAM optimization at 135 Hz. b) Convergence speed of the S-PAM second moment metric for the same optimization procedure reported in a. c) Image detail for uncorrected image. d) Image detail for hill climbing image optimization. e) Image detail for S-PAM optimization. Scale bar is 10 μm.

Download Full Size | PDF

As expected, the S-PAM optimization procedure greatly outperforms in the traditional image optimization, converging within 10 s even for severe aberrations, and achieving a significant sharpness correction within 2 full images acquisitions. It has to be noticed that, for smaller aberrations, the S-PAM convergence time would be shorter, while the hill-climbing procedure time is fixed and does not depend on the aberration magnitude.

As an additional advantage, the sharpness of the resulting image is higher for the S-PAM optimization, as the hill-climbing algorithm has no guarantee of converging to the minimum of the metric function, unless an optimal base is determined through a previous calibration procedure [18]. It is to be noticed that the hill climb algorithm final metric result could be improved, at the cost of a slower optimization speed, through repeated optimization cycles or performing more measurements per degree of freedom. Moreover, while the sample photobleaching is practically negligible for a 10 s S-PAM optimization, its effect on a hill climb optimization procedure, especially if performed with more than five measurements per degree of freedom, can severely affect the results.

7. Dynamic correction results

As a proof of the stability of correction for slow temporal dynamics in the aberrations, three dimensional image stacks were acquired, by modulating the defocus component of the aberration corrected by the deformable mirror. The microscope was focused at the topmost surface of the sample, and optimization was left running for 10s. After that, a three dimensional image was acquired shifting the focus in steps of 0.2 μm through the full thickness of the sample (16 μm), for a total acquisition time of 27 seconds, without pausing the optimization procedure. As a comparison, the same three dimensional image was acquired without any correction, and by keeping a single correction, obtained for the central plane, throughout the stack acquisition.

Results of an indicative experiment are shown in Fig. 5. In order to quantify the performance enhancement, the same contrast image metric used for the hill climb algorithm in section 6 is reported as a function of imaging depth for the three use cases. It can be observed that while the dynamic and static correction have similar performances at the static optimization location, the performance of the dynamic correction is superior at the extremes of the focal stack.

 

Fig. 5 Example of dynamic aberration correction: a) Last image in a 16 μm thick three dimensional focal stack of images, with active correction. b) Detail of the area indicated by a red square in (a) for uncorrected image. c) same detail for static aberration correction, optimized at the center of the axial range for 10 s. d) same detail for dynamic correction of the aberration during the stack acquisition. e) Value of the sharpness metric as a function of depth for no correction, static correction, and dynamic correction.

Download Full Size | PDF

8. Discussion

In this paper, correction of dynamical optical aberrations during the acquisition in a wavefront sensorless optical sectioning fluorescence microscope is proven possible.

Sensorless dynamic correction is in many respects more useful than existing techniques in a practical application, as it would enable life sciences researchers to operate an adaptive microscope without special sample preparation to include artificial references, and preventing the time loss and possible photobleaching problems introduced by conventional optimization procedures.

Moreover, data presented shows how a dynamic correction can in fact increase the quality of the images obtained in a focal stack acquisition, compared to a standard static correction for a single focal plane.

The results presented were achieved with a newly designed optical scheme for a PAM microscope, in which the spatial distribution of out of focus light rejected by a confocal aperture is used as a metric for wavefront sensorless optimization. Optical aberrations are corrected by a deformable mirror in the pupil plane of the system.

A slight downside of the DMD use for optical sectioning is the low efficiency in the use of excitation light, which forces the use of slits instead of pinholes for out of focus rejection, affecting the axial resolution of the system. However, since the DMD elements act as point sources, a conspicuous advantage of the S-PAM is the possibility of using high power incoherent LED sources instead of coherent sources.

The use of a high power LED light source, as opposed to the conventional coherent sources used, for example, in spinning disk microscopy, makes the microscope eye safe in any situation, and dramatically reduces the cost and complexity of the system.

Moreover, due to the wide availability and low cost of LED sources, a multi wavelength setup could more easily be created on this kind of setup.

As an addition to aberrations correction, the deformable mirror can be used for fast and precise shifting of the focal plane, therefore substituting piezoelectric actuators for objective movements. Moreover, due to the programmable nature of the confocal illumination of the system, random access signal detection techniques, which are increasingly popular in the fields of neuroscience [19–21] and flow velocimetry [22,23], could easily be performed with the system, at least with relatively optically simple samples which do not require two photon excitation (e.g. Zebrafish, C-Elegans).

As a final remark, the whole optical setup of a S-PAM microscope is compact, simple, relatively cost effective, and due to incoherent excitation it does not require a particularly refined alignment procedure. Such characteristics, combined with the ease of use of dynamic aberration corrections, could make this technique a potential candidate for widespread adoption of adaptive optics in life sciences application.