Open Access
Add to BibSonomyAbstract
Increasing the depth-of-field (DOF) while maintaining high resolution imaging has been a classical challenge for conventional microscopes. Extended DOF (EDOF) is especially essential for imaging thick specimens. We present a microscope capable of capturing EDOF images in a single shot. A volumetric optical sampling method is applied by rapidly scanning the focus of a vari-focal microscope objective throughout the extended depths of a thick specimen within the duration of a single detector exposure. An EDOF image is reconstructed by deconvolving the captured image with the response function of the system. Design of a vari-focal objective and algorithms for restoring EDOF images are presented. Proof-of-concept experimental results demonstrate significantly extended DOF compared to the conventional microscope counterparts.
©2010 Optical Society of America
1. Introduction
The majority of conventional microscopes has a limited depth-of-field (DOF). For instance, the theoretical DOF of a 0.25 NA microscope objective is 8 μm, given by nλ/NA 2 [1] where λ is the imaging wavelength (λ = 500 nm), n is the refractive index in the object space (n = 1), and NA is the numerical aperture. As the spatial resolution―1.22λ/NA [1] is improved by either increasing NA or switching to shorter wavelength sources, the DOF further decreases. Although a shallow DOF may improve the axial resolution, it sometimes limits the capability of a conventional microscope for imaging three-dimensional (3-D) samples.
An extended-DOF (EDOF) microscope is capable of retaining the transversal resolution across an extended depth range of several times greater than the theoretical DOF. Such unique feature enables EDOF microscopes for a broad range of applications such as biomedical imaging, materials processing, and medicine, in which the structure of interest may spread throughout a region much thicker than the theoretical DOF of a conventional microscope. For instance, an EDOF microscope had been utilized to localize single fluorescence molecules in thick polymers [2]; in another example, an EDOF microscope was deployed for high speed cell analysis, which required for both large throughput for flow cytometry and detailed imaging of selected individual cells [3].
Over the past decades, numerous approaches, such as wavefront coding and optical sectioning techniques, have been proposed for EDOF microscopes. For the wavefront coding approach, a phase mask, such as one encoded with a cubic wavefront [4,5], is placed at the aperture stop of a microscope objective. The phase mask modifies the incident wavefront and generates a serial of blurry point spread functions (PSFs) which are insensitive to misfocus. A sharp EDOF image can be restored by filtering the captured image with the encoded wavefront. For the optical sectioning technique, a thick specimen is imaged slice by slice at different focal depths. An EDOF image is then reconstructed by superimposing the image stack into a single 2-D image. In this approach the ability to eliminate out-of-focus and scattered light is essential for reconstructing high-quality EDOF image. Some optical sectioning techniques adopt mechanisms to suppress the out-of-focus signals. Examples include confocal microscopy [6], multi-photon fluorescence microscopy [7], selective plane illumination microscopy [8], and structured illumination microscopy [9]. Alternatively computational methods, such as a deconvolution technique [10,11], can be applied in optical sectioning technique without the necessity of eliminating out-of-focus signals at the stage of image acquisition at increased computational costs. A number of image slices, which contain both in-focus and out-of-focus information, are fused and then deconvolved to remove the image blurs originated from the out-of-focus backgrounds.
In this paper, we present a wide field microscope capable of capturing EDOF images at a single shot. Different from the conventional optical sectioning method in which the EDOF image is fused from multiple image slices captured at discrete focal depths [11,12], our approach employs a volumetric optical sampling method by rapidly scanning the focus of a vari-focal microscope objective throughout the extended depths of a thick specimen within the duration of a single detector exposure. The captured image is a natural fuse of infinite slices of images within the focus range of the objective and an EDOF image will then be reconstructed by applying a deconvolution technique. Such method enables single-shot EDOF imaging which in turn may improve the throughput of a microscope for potential real-time applications. Due to the ability to preserve image magnifications across the depth range, our approach eliminates the needs for a careful registration of multiple image slices which is often essential in other optical sectioning techniques in order to seamlessly fuse an EDOF image [10]. A miniature liquid lens [13] is adopted to enable the rapid and continuous change of the focal distance of the microscope objective at high-speed, thus the EDOF microscope has a compact profile and requires no mechanical moving or scanning mechanisms as compared to conventional deconvolution techniques [14]. Computer models further predict that the system’s response functions of the EDOF microscope are nearly invariant across a depth range much greater than the theoretical DOF, similar to an EDOF microscope with encoded wavefront [4,5]. In addition, since the liquid lens is addressable with either discrete or continuous-oscillating focal lengths, the proposed system can be easily switched between a conventional microscope and an EDOF microscope without significant hardware modifications. The multimodality capability enables the flexible control of the microscope, balancing the EDOF range and the system throughput depending upon application requirements.
The rest of the paper is arranged as follows. In Section 2, we describe the volumetric optical sampling method and discuss the image formation theory of the EDOF microscope. In Section 3, we present the design of a vari-focal microscope objective with depth-invariant spatial resolutions. Based on the vari-focal objective and the image formation theory, we further simulate the system’s response functions, i.e. PSFs or modulation transfer functions (MTFs), and compare them to those of a conventional microscope. Finally in Section 4, we present proof-of-concept experimental results based on an off-the-shelf microscope objective integrated with a high-speed liquid lens.
2. Volumetric optical sampling method
A schematic illustration of the volumetric optical sampling method for an EDOF microscope is shown in Fig. 1 . The system consists of three major components: a variable focus microscope objective, an image sensor, and a thick object illuminated by incoherent light sources. For simplicity, the vari-focal objective is assumed to be a thin lens coinciding with the aperture stop. To capture EDOF images, the focus of the microscope objective is rapidly scanned through the extended thickness of the object from far (zf) to near (zn) distances. Assuming the vari-focal objective is synchronized with the frame rate of the image sensor such that the focus of the objective just scans across the depth range during the sensor acquisition period as shown in Fig. 2 , the captured image is therefore a single-shot integration of both in-focus and out-of-focus projections of a 3-D object on the 2-D image plane as the objective rapidly scans its focus across the thickness of the object. Post-processing is then applied to recover the in-focus EDOF image by eliminating the image blurs originated from out-of-focus light.
Fig. 1 Schematic illustration of the volumetric optical sampling method for a thick 1-D specimen with an extended depth range from zn to zf.
Fig. 2 Driving mechanisms of the (a) image sensor and (b) focus of the microscope objective, where 1/T equals to the frame rate of the image sensor and 0.5/T equals to the driving frequency of the liquid lens.
The post-processing step requires the detailed knowledge of the system’s response function, e.g. PSF, across the focusing range. Applying the volumetric sampling approach described above, the accumulated point spread function, PSFx’ , y’, of a point source o(z 0), as the vari-focal objective scans through zn to zf, can be quantitatively determined by
where PSFx’ , y’(z 0,z) is the PSF of the point source o(z 0) when the objective is focusing at z (zn≤z≤zf). The discrete formalization of Eq. (1) giveswhere N is the number of sampled distances. Equation (2) can be numerically implemented to model PSFx’ , y’(z 0) from the accumulation of N + 1 samples of PSFx’ , y’(z 0,z), as z successively increases from zn to zf. The captured image ix’ , y’(z 0) can be derived as the convolution of the point source o(z 0) with the accumulated PSF, PSFx’ , y’(z 0), while the inverse relation indicates that the object o(z 0) can be restored from ix’ , y’(z 0) by applying a deconvolution filter of PSFx’ , y’(z 0):Furthermore, by assuming the EDOF microscope is a linear shift invariant (LSI) system, i.e. PSFx’ , y’ is independent upon z 0, Eq. (3) can be further modified to
through which the entire 3-D object can be retrieved from a single deconvolution filter of PSFx’ , y’(z 0). As will be discussed in Section 4, the LSI condition is satisfied for a given EDOF range by applying the volumetric sampling method.Compared to a similar sampling approach in [14], there exist two major differences between our method and the prior art. Firstly, the focal length of the microscope objective is addressable. Therefore, the EDOF microscope requires no mechanical mechanism, e.g. axial translation of either the sample or the objective, which may potentially limit the sampling speed and system throughput. Secondly, when the vari-focal element coincides with the aperture stop of the microscope as illustrated in Fig. 1, varying the focal distance of the microscope does not change the chief ray angles and thus constant magnification is preserved at different focal distances, which minimizes the variations of image distortions. On the contrary, an EDOF microscope which involves mechanical depth scans [10] requires a careful registration of multiple image slices in order to seamlessly fuse EDOF image due to variations of image magnifications across the depth range.
3. Vari-focal microscope objective
The key to the volumetric sampling approach described in Section 2 lies in the ability to (a) rapidly scan the focal distance of the microscope objective across the extended thickness of a 3-D specimen during a single exposure of a detector, and (b) to obtain the accumulated PSF defined in Eq. (1) and (2).
To achieve rapid focus scanning, miniature high-speed liquid lenses [12,13,15,16], such as ARCTIC320 and ARCTIC314 (Varioptic Inc.), are considered for the design of a wide field EDOF microscope. Figure 3 shows the optical layouts of an optimized vari-focal microscope objective (10x, 0.25 NA) integrated with a liquid lens module―ARCTIC320. A 1/3” CCD detector is selected as the image sensor. The field-of-view (FOV) of the EDOF microscope is 0.64 mm (H) x 0.48 mm (V) in the object space. In the lens model, a half field of 0.4mm is selected for the object to be imaged. Additionally, polychormatic wavelengths of 513.9 nm, 559.0 nm, and 608.9 nm are set in the lens model. In order to achieve depth-invariant spatial resolutions, the vari-focal objective is optimized at multiple zooms with different focal settings of z equals to (a) zn + 0.00 mm, (b) zn + 0.08 mm, and (c) zn + 0.16 mm respectively. The corresponding MTF curves in (d), (e), (f) for each optical zoom suggest consistent and near diffraction-limited imaging throughout the entire focal range of zf-zn = 0.16 mm. Such focal range is ~20 times greater than the theoretical DOF of a 0.25 NA objective of ~8 µm.
Fig. 3 Optical layouts of the vari-focal microscope objective integrated with a liquid lens. The focal distance z is set at (a) zn + 0.00 mm, (b) zn + 0.08 mm, and (c) zn + 0.16 mm respectively. The corresponding diffraction MTF at each focal distance is shown in (d), (e), and (f) respectively.
The vari-focal objective design presented in Fig. 3 is primarily for demonstrating depth-invariant resolutions and for simulating the response functions of the EDOF microscope. This vari-focal objective design may be potentially improved by adding more optical components and adopting aspheric surfaces to achieve higher spatial resolution, wider spectral bandwidth, and the more extended focal range, which however is beyond the major scope of this paper. In a different application, Murali et al. adopted a liquid lens in the design of an optical coherence microscopy with an imaging volume of 8 mm3 [15].
The requirement for depth-invariant imaging is crucial for the LSI condition which is required for reconstructing the EDOF images from the deconvolution of a single PSF indicated by Eq. (4). Figures 4(a) through 4(e) demonstrate a serial of PSFs of a conventional microscope, which are simulated in CODE V by fixing the focal distance of the vari-focal objective at z = zn while varying the object distance z 0 at (a) zn + 0.00 mm, (b) zn + 0.04 mm, (c) zn + 0.08 mm, (d) zn + 0.12 mm, and (e) zn + 0.16 mm, respectively. Apparently the PSF spot size monotonically increases as the defocus distance, z 0-z, increases from 0mm to 0.16mm, indicating a shallow DOF around the focal distance z = zn of the objective. In contrast, Figs. 4(f) through 4(j) demonstrate a serial of accumulated PSFs, PSFx’ , y’(z 0), of the EDOF microscope with z 0 equals to (f) zn + 0.00 mm, (g) zn + 0.04 mm, (h) zn + 0.08 mm, (i) zn + 0.12 mm, and (j) zn + 0.16 mm, respectively. The accumulated PSFs are simulated through the integration of N + 1 samples of PSFx’ , y’[z 0,zn + n × (zf-zn)/N] as indicated by Eq. (2) while each individual PSFx’ , y’ is simulated in CODE V. Additionally N was chosen to be 200 such that the sampling step on the discrete focal distance z is (zf-zn)/N = 0.8 µm, which is sufficiently small compared to the theoretical DOF of ~8µm of the microscope objective. Unlike the counterparts of a conventional microscope shown in (a)-(e), PSFs in (f)-(j) do not show appreciable differences as the object distance z 0 varies from zn + 0.00mm to zn + 0.16mm. Compared to a wavefront-encoded microscope [5], PSFs of the vari-focal objective are similarly insensitive to the object depths, indicating that the EDOF microscope approximates a LSI system and the EDOF image can be restored from a single deconvolution filter of PSFx’ , y’(z 0) in Eq. (4).
Fig. 4 Point spread functions of a conventional microscope as the defocus distance, z 0-z, increases at (a) 0.00 mm, (b) 0.04 mm, (c) 0.08 mm, (d) 0.12 mm, and (e) 0.16 mm, respectively; Point spread functions―PSFx’ , y’(z 0) of the EDOF microscope for z 0 equals to (a) zn + 0.00 mm, (b) zn + 0.04 mm, (c) zn + 0.08 mm, (d) zn + 0.12 mm, and (e) zn + 0.16 mm, respectively.
In order to further quantitatively evaluate the LSI condition for different object depths, MTFs of the EDOF microscope are computed from the Fourier Transform (FT) of the PSFs of (f)-(j). Figure 5 plots the results as z 0 is set at zn + 0.00 mm (red circle marker), zn + 0.04 mm (green square marker), zn + 0.08 mm (blue inverse triangle marker), zn + 0.12 mm (cyan triangle marker), and zn + 0.16 mm (magenta star marker), respectively. As expected, the MTFs in Fig. 5 are considerably worse throughout the frequency domain than the diffraction-limited MTFs in Fig. 3 which correspond to discrete PSF samples at discrete focal depths. In consequence at the stage of image acquisition, the EDOF microscope captures the projection of a 3-D object on a 2-D image plane with everything likely to be blurred similar to that of a wavefront-encoded microscope [4,5]. Since the MTFs are nearly invariant versus the object distance (zn≤z 0≤zf), an in-focus EDOF image can be retrieved by applying the decovolution filter of PSFx’ , y’(z 0) as indicated in Eq. (4).
Fig. 5 Modulation transfer functions of the vari-focal microscope objective with different object distance of z 0 equals to zn + 0.00 mm (red circle marker), zn + 0.04 mm (green square marker), zn + 0.08 mm (blue inverse triangle marker), zn + 0.12 mm (cyan triangle marker), and zn + 0.16 mm (magenta star marker), respectively.
4. Experiments and results
In the previous section, the volumetric sampling method is validated through modeling of the system’s response functions. In this section, proof-of-concept experimental results of the EDOF imaging are demonstrated based on an off-the-shelf microscope objective integrated with a high-speed liquid lens.
Figure 6 illustrates the experimental setup of the EDOF microscope. An off-the-shelf microscope objective with M = 10 and NA = 0.25 is integrated with a liquid lens ARCTIC314 with ~10ms response speed (90% rise time to a step stimulus) [13,16]. As shown in Fig. 6, the liquid lens is directly attached to the back surface of the objective similar to the optimized layout in Fig. 3. The specimen to be imaged was prepared from a few thorns of a living cactus with a cross sectional diameter of about 5µm to 30µm from the tip to the other end. The four thorns to be imaged extend across a depth of ~600μm which was measured by manually translating the sample mounted on a micrometer stage. An incandescent lamp illuminates the sample from an oblique angle. The detector is a black-white CCD sensor (Dragonfly2, Point Grey Research) with 1024x768 pixel resolutions and a frame rate of 15 fps.
Experimental results are demonstrated in Fig. 7 and an accompanying video clip (Media 1). First of all, the focal distance of the microscope objective is dynamically changed step by step between near and far distances, as shown in the first part (I) of the video clip. In this case, the microscope is analogous to a conventional microscope with a shallow DOF. For instance in Fig. 7 (a) where the microscope objective focuses at near distance, only the tip of the front thorn is in sharp focus while all other thorns are strongly blurred due to large defocus; vice versa in Fig. 7 (b), only the thorn at farther distance is shapely imaged after the objective shifts its focus to the corresponding distance. The second part (II) of the video clip shows the captured images of the sample while the focal distance of the objective rapidly scans across the near and far distances at a frequency of 7.5 Hz synchronized with the frame update of the image sensor. This case corresponds to the volumetric sampling condition discussed in Section 2 and all thorns appear to be equally blurred as predicted by the simulated system response functions of the EDOF microscope. A single-shot image is extracted from the video (II) and is shown in Fig. 7 (c). Finally, the captured image (c) is deconvolved with the accumulated PSF which is simulated similarly to those in Fig. 4 (f)-(j), and a clear in-focus EDOF image is restored as shown in Fig. 7 (d) and in part (III) of the video clip. Compared to the images in Figs. 7 (a) and (b), Fig. 7(d) shows that the thorns located at both near and far distances are resolved simultanously in the EDOF image, while either one is strongly blurred in the conventional microscope images, indicating an extended DOF for the proof-of-concept experiments.
Fig. 7 Conventional microscopic images of the sample, as the objective focuses at (a) near and (b) far distances. (c) Single-shot image of the same sample captured by applying the volumetric sampling approach. (d) Restored single-shot EDOF image after applying the deconvolution filter. (Media 1)
To further compare the imaging quality of the EDOF microscope with that of a conventional microscope, two profile lines are marked in Fig. 7—one solid line (100 pixels in length) near the tip portion of the thorn located at the near distance, and the other dashed line (140 pixels in length) near the end portion of the thorn located at the far distance. In Fig. 8 , the normalized pixel intensities along these lines are plotted and compared with each other. For instance in Fig. 8 (a), the pixel values (plotted in solid magenta curve with ‘x’ markers) along the magenta solid line in Fig. 7 (d) are well correlated to those (plotted in solid red curve with ‘o’ markers) along the red solid line in Fig. 7 (a), and they show higher spatial resolution and image contrast than the corresponding part in Fig. 7 (b) (in solid green curve with ‘□’ markers in Fig. 8a). Similarly in Fig. 8 (b), the pixel values along the dashed line in Fig. 7 (d) (dashed magenta curve with ‘x’ markers in Fig. 8b) are well correlated to those (dashed green curve with ‘□’ markers) along the dashed line in Fig. 7 (b), and they show improved image contrast and resolution against those (dashed red curve with ‘o’ markers) in Fig. 7 (a). The results indicate the EDOF microscope prototype is capable of simultaneously imaging the thorns across a significantly extended depth range while maintaining a spatial resolution comparable to its conventional counterpart.
In this proof-of-concept experiment, the results demonstrated an extended DOF range greater than 160μm. Since off-the-shelf objectives were used, we are unable to predict and quantify the depth range within which the LSI condition holds, while similar analysis approaches in [17] may be considered to further quantify the extension ratio of the DOF as opposed to a conventional microscope. Also noticed in Fig. 7 (d), imaging artifacts such as stripes smudge are visible near the thorn/background boundaries. We tentatively attribute these artifacts to (a) uncorrected aberrations in an EDOF prototype using off-the-shelf components, and/or (b) slight mismatch between the simulated and the actual system’s response functions of the microscope objective. In the future, custom designed vari-focal objective will be developed, by which all lens data would be readily available to accurately simulate the system’s response functions and accurately predict the EDOF range.
5. Conclusion
We present a wide field microscope capable of capturing EDOF images in a single shot. A volumetric optical sampling approach is applied by rapidly scanning the focal distance of a vari-focal objective synchronized with the frame rate of the image detector. A single-shot in-focus EDOF image can be restored through the deconvolution filtering with the system’s accumulated response functions. Fast liquid lenses are integrated in the design of vari-focal objectives without mechanical moving components. Proof-of-concept experimental results demonstrate an extended DOF compared to its conventional microscope counterparts.
For future works, custom designed vari-focal objectives will be developed from which the extended DOF range and the system’s response function can be accurately simulated and predicted. The deconvolution process to obtain extended DOF images in the current system is carried off-line, while in the future, deconvolution filtering will be implemented in real time enabling real-time, single-shot, and high-resolution EDOF microscopic imaging.
Acknowledgements
This research was funded in part by NSF grant award # 09-15035.
References and links
1. T. S. Tkaczyk, Field Guide to Microscopy (SPIE Press, 2009).
2. S. R. P. Pavani, M. A. Thompson, J. S. Biteen, S. J. Lord, N. Liu, R. J. Twieg, R. Piestun, and W. E. Moerner, “Three-dimensional, single-molecule fluorescence imaging beyond the diffraction limit by using a double-helix point spread function,” Proc. Natl. Acad. Sci. U.S.A. 106(9), 2995–2999 (2009). [CrossRef] [PubMed]
3. W. E. Ortyn, D. J. Perry, V. Venkatachalam, L. Liang, B. E. Hall, K. Frost, and D. A. Basiji, “Extended depth of field imaging for high speed cell analysis,” Cytometry A 71(4), 215–231 (2007). [PubMed]
4. E. R. Dowski Jr and W. T. Cathey, “Extended depth of field through wave-front coding,” Appl. Opt. 34(11), 1859–1866 (1995). [CrossRef] [PubMed]
5. S. C. Tucker, W. T. Cathey, and E. R. Dowski Jr., “Extended depth of field and aberration control for inexpensive digital microscope systems,” Opt. Express 4(11), 467–474 (1999), http://www.opticsinfobase.org/abstract.cfm?URI=oe-4-11-467. [CrossRef] [PubMed]
6. J. A. Conchello and J. W. Lichtman, “Optical sectioning microscopy,” Nat. Methods 2(12), 920–931 (2005). [CrossRef] [PubMed]
7. F. Helmchen and W. Denk, “Deep tissue two-photon microscopy,” Nat. Methods 2(12), 932–940 (2005). [CrossRef] [PubMed]
8. J. Huisken, J. Swoger, F. Del Bene, J. Wittbrodt, and E. H. K. Stelzer, “Optical sectioning deep inside live embryos by selective plane illumination microscopy,” Science 305(5686), 1007–1009 (2004). [CrossRef] [PubMed]
9. M. A. A. Neil, R. Juskaitis, and T. Wilson, “Method of obtaining optical sectioning by using structured light in a conventional microscope,” Opt. Lett. 22(24), 1905–1907 (1997). [CrossRef]
10. J. B. Sibarita, “Deconvolution microscopy,” Adv. Biochem. Eng. Biotechnol. 95, 201–243 (2005). [PubMed]
11. F. Aguet, D. Van De Ville, and M. Unser, “Model-based 2.5-d deconvolution for extended depth of field in brightfield microscopy,” IEEE Trans. Image Process. 17(7), 1144–1153 (2008). [CrossRef] [PubMed]
12. J. P. Rolland, P. Meemon, S. Murali, K. P. Thompson, and K. S. Lee, “Gabor-based fusion technique for optical coherence microscopy,” Opt. Express 18(4), 3632–3642 (2010), http://www.opticsinfobase.org/abstract.cfm?URI=oe-18-4-3632. [CrossRef] [PubMed]
14. J. A. Conchello and M. E. Dresser, “Extended depth-of-focus microscopy via constrained deconvolution,” J. Biomed. Opt. 12(6), 064026 (2007). [CrossRef]
15. S. Murali, K. P. Thompson, and J. P. Rolland, “Three-dimensional adaptive microscopy using embedded liquid lens,” Opt. Lett. 34(2), 145–147 (2009). [CrossRef] [PubMed]
16. S. Liu and H. Hua, “Time-multiplexed dual-focal plane head-mounted display with a liquid lens,” Opt. Lett. 34(11), 1642–1644 (2009). [CrossRef] [PubMed]
17. C. Pan, J. B. Chen, R. Zgang, and S. L. Zhuang, “Extension ratio of depth of field by wavefront coding method,” Opt. Express 16(17), 13364–13371 (2008), http://www.opticsinfobase.org/oe/abstract.cfm?URI=oe-16-17-13364. [CrossRef] [PubMed]
