Open Access
Add to BibSonomy
Abstract
We propose a resolution enhancement method for lensless in-line holographic microscope (LIHM) with spatially-extended light source, where the resolution is normally deteriorated by the insufficient spatial coherence of the illumination. In our LIHM setup, a light-emitting diode (LED), which was a spatially-extended light source, directly illuminated the sample, and the in-line hologram were recorded by a CMOS imaging sensor located behind the sample. In our holographic reconstruction process, the in-line hologram was first deconvoled with a properly resized image of the LED illumination area, and then back-propagated with scalar diffraction formula to reconstruct the sample image. We studied the hologram forming process and showed that the additional deconvolution process besides normal scalar diffraction reconstruction in LIHM can effectively enhance the imaging resolution. The resolution enhancements capability was calibrated by numerical simulations and imaging experiments with the U.S. air force target as the sample. We also used our LIHM to image the wing of a green lacewing to further demonstrate the capability of our methods for practical imaging applications. Our methods provide a way for LIHM to achieve satisfactory resolution with less stringent requirement for spatial coherence of the source and could reduce the cost for compact imaging system.
© 2017 Optical Society of America
1. Introduction
Digital lensless in-line holographic microscope (LIHM) is a compact and low-cost solution for microscopic imaging [1–4]. Compared with traditional microscope, LIHM has the advantages of wide field-of-view, flexible sample-to-sensor distance and multi-layer imaging capability. Initially the illumination for LIHM is usually laser source [5–7], in this case, the imaging resolution can be computed as 0.5λ/NA according to the spatial frequency components that can be observed, where λ is the wavelength and NA is the numerical aperture [8]. The use of laser source, however, will introduce speckle noise or unwanted interferences (for example, interference with reflections or scatterings from other components in the optical system) [9, 10], and also relatively high cost. Later on, to overcome these issues, light-emitting diode (LED) was used in compact and low-cost LIHM system. The use of LED, however, will results in worse imaging resolution because of the degraded hologram resulted from insufficient spatial and temporal coherence. Thus a small pinhole (10-100 μm) associated with the LED [11–13] or fiber-coupled LED source [14–16] were usually introduced to increase the spatial coherence. However, the pinhole or fiber coupling will significantly reduce the effective illumination power on the sample, which necessitates longer exposure time or results in worse signal-to-noise ratio (SNR) for the hologram acquisition. Furthermore, the use of pinhole or coupled fiber will also introduce additional system cost for assembling and alignment.
In this paper, we propose to remove the pinhole while using LED or other spatially-extended source directly as illumination in the LIHM system and numerically compensate for the spatial coherence issue by a deconvolution process instead of using an additional pinhole or fiber coupled source. According to numerical simulations and experimental results shown later, we found that the imaging resolution with our method was similar as the LIHM system with pinhole-associated LED illumination or fiber-coupled LED source.
In our proposed method, we suppose that the spatially-extended source consists of many infinitely small point sources on the illumination area which are totally spatially incoherent between one another. This is a reasonable assumption for LED because of its luminous mechanism, where the light is generated by spontaneous emission of random electron-hole pair recombination in a P-N junction [17]. With this assumption, the acquired hologram in LIHM with spatially-extended light source illumination can be considered as the incoherent superposition of sub-holograms acquired by using each point source as illumination, that is, the acquired hologram intensity is the sum of the intensity of the sub-holograms. We will explain later that this incoherent superposition process can be modeled as a convolution process, and one of the sub-holograms can be obtained by a deconvolution process. Since the sub-hologram is equivalent to the acquired hologram while using an LED with a pinhole or fiber-coupled LED source, and the sample image can then be reconstructed using traditional holographic reconstruction methods based on scalar diffraction formula.
Our proposed method provides a way to use spatially-extended light source, e.g., LED in LIHM without using an additional pinhole or fiber-coupled LED source. Compared with existing LIHM systems that used laser, pinhole-associated LED illumination, or fiber coupled sources, our method can achieve a speckle-free, power efficient, and low-cost solution for compact microscopic imaging scenario.
In the rest of the paper, we will describe our system setup and the principle of image reconstruction in the LIHM with spatially-extended light source in Section 2, then show the numerical simulation and experimental results in Section 3, and finally discuss and summarize our work in Section 4.
2. System setup and principle of image reconstruction
The system setup of our LIHM system is very simple and shown in Fig. 1, where the schematic is shown on the left and a photograph with enlarged LED image (acquired by microscope) is shown on the right of the figure. An LED chip with center wavelength of 470 nm and illumination area of 0.3 × 0.2 mm (INH-S3216DAB65TP-Y1, INHERE Co.), which is a spatially-extended light source, directly illuminates the sample, and a CMOS imaging sensor with pixel size of 2.2 µm (DMK 72AUC02, the Imaging Sources Europe GmbH) located behind the sample records the in-line hologram caused by the interference of scattered light from the sample and the unscattered light. The coordinate systems are also shown in Fig. 1, where (x1, y1) are the coordinates in the LED plane, (x2, y2) are the coordinates in the sample plane, and (x3, y3) are the coordinates in the imaging sensor plane. In addition, Z1 and Z2 shown in Fig. 1 indicate the source-to-sample and the sample-to-sensor distances. In the experiment, Z1 is set as 3~9 cm and Z2 is set as ~1.5 mm, thus Z1 >> Z2 and in this case, the magnification from sample to sensor plane is almost 1:1 and the illumination light on the sample can be approximately considered as plane wave for point source illumination [18].
Fig. 1 The system setup of the LIHM system with spatially-extended light source illumination and the photograph of the setup used in our experiment. The enlarged LED image acquired by microscope is also shown.
As previously mentioned, any two point sources in the LED illumination area can be considered as spatially incoherent, and the acquired hologram can be considered as the incoherent superposition of sub-holograms acquired by using each point source on the LED area as illumination. Since the size of LED illumination area (0.3 × 0.2 mm) is much smaller than Z2, the sub-holograms can be considered as shifted and scaled versions of the hologram formed by the point source illumination according to the position and luminance of the illumination point source. Thus the acquired hologram intensity I3(x3, y3) in the sensor plane can be written as
where I(x3, y3) is the sub-hologram formed by using a point source at (x1, y1) = (0, 0) with unit luminance, and S(x1, y1) is the luminance distribution of the LED illumination area. The above equation can be written in convolution form aswhere * denotes convolution andThus the acquired hologram in LIHM with spatially-extended light source can be considered as the convolution of the sub-hologram acquired with a point source illumination and a point spread function (PSF), which is a resized (with a shrink ratio of Z1/Z2) and intensity scaled (with a ratio of (Z1/Z2)2) image of the luminous intensity distribution of the source according to Eq. (3). Notice that we are only interested in the relative intensity distribution and the scaled factor (Z1/Z2)2 can be neglected in computation. According to Eq. (2), I(x3, y3) can be computed by a simple deconvolution process from the measured I3(x3, y3) once the PSF is known and we can now obtain the sub-hologram in LIHM as the cases where pinhole-associated LED illumination or fiber-coupled LED source are used.On the other hand, according to angular propagation method in the scalar diffraction theory [19], the hologram acquired with a point source illumination can be written as
where U(x3, y3) is the light field at the imaging sensor plane and can be written aswhere FT{} and FT−1{} represent the Fourier and inverse Fourier transform, U1 is the light field right after the sample plane, λ is the wavelength of light, fx and fy are the corresponding spatial frequencies. Thus once we get I(x3, y3) from the deconvolution process, we can use normal holographic reconstruction method to reconstruct the sample image |U1|2 by scalar propagation. Notice that the well-known twin-image issue will still exist in this case.In summary, our proposed reconstruction method in LIHM with spatially-extended light source consists of two steps, as shown in Fig. 2: (1) deconvolve the measured in-line hologram with the measured PSF; (2) reconstruct the sample image by normal scalar diffraction as in conventional holographic reconstruction. It will be demonstrated in the next section that our reconstruction method can effectively enhance the imaging resolution compared with applying step (2) only as in conventional holographic reconstruction.
3. Numerical simulation and experimental results
To study the effect of resolution enhancement by the additional deconvolution process in LIHM with spatially-extended light source, as shown in Fig. 1, we first did numerical simulations with a U. S. Air Force (USAF) target as the sample. We chose three different source-to-sample distances, i.e., let Z1 = 3, 6, 9 cm, to study the effect of PSF size in the reconstruction. Z2 was set as 1.5 mm to ensure the compactness of the imaging system. The spatially-extended source was set to have a rectangular illumination area of 0.3 × 0.2 mm, which is similar in size as our LED source, and have a luminous intensity distribution of Gaussian shape. Notice that here we didn’t use the measured luminous intensity distribution of our LED for the simulation PSF because we want to show the validity of our method for a more general and common case of light sources with Gaussian luminous intensity distribution. In our simulation, we first computed one sub-hologram with plane-wave illumination by the propagated the USAF target using the angular spectrum method, i.e., Eq. (5), then obtained the simulated in-line holograms by incoherent superposition of the shifted and scaled sub-holograms. Then the sample image was reconstructed with conventional holographic reconstruction (will be called direct reconstruction below) and our proposed two-step method including a deconvolution process before applying conventional holographic reconstruction. In the deconvolution process, we used the simple Wiener filter provided in the MATLAB platform (Mathworks Inc.) [20] with a simulated PSF according to the shape and luminous intensity distribution of the light source. Notice that the PSF size will vary for different Z1 as described by Eq. (3). The wavelength is set as 470 nm to be consistent with our experiments.
The simulation results are shown in Fig. 3, where Fig. 3(a) shows the in-line hologram, direct reconstruction result, the sub-hologram after deconvolution process, and the reconstruction result with our proposed method for Z1 = 3 cm. The simulation PSF is shown at the top-left of the sub-figure of result with proposed method. Because of insufficient spatial coherence, the in-line hologram looks blurry and the direct reconstruction result is also blurry and the bars in the USAF target cannot be clearly discerned. In contrast, the sub-hologram after the deconvolution process becomes sharper and the reconstruction results with our proposed method show much better imaging resolution. Figures 3(b) and 3(c) show similar results for Z1 = 6 cm and 9 cm, respectively. As the source-to-sample distance Z1 gets larger, the spatial coherence issue becomes more and more insignificant as the angular extension of the source becomes small, which can be explained by the van Cittert - Zernike theorm [21]. Thus we can observe better direct reconstruction results for larger Z1 as the in-line holograms are approaching the case of point source illumination. At the same time, the reconstruction with our proposed methods keep on providing similar enhanced-resolution images.
Fig. 3 Numerical simulation results of image reconstructions in LIHM with spatially-extended light source and, for comparison, laser source. (a)-(c) in-line holograms, direct reconstruction results, sub-holograms after deconvolution, and reconstruction results with our proposed methods for source-to-sample distances Z1 = 3, 6, 9 cm, respectively. The corresponding simulation PSF is shown at the top-left of the sub-figures in the last column. (d) in-line holograms and holographic reconstruction result with laser illumination.
For comparison, Fig. 3(d) shows the results under laser illumination simulated by a point source illumination. In this case, no deconvolution process is necessary and we can directly reconstruct the sample image by conventional holographic reconstruction. We can see the reconstruction results with our proposed method for LIHM with spatially-extended source is comparable with direct reconstruction results for LIHM under laser illumination.
We then did experiment with USAF target as the sample to verify the simulation results. The setup of our LIHM system is shown in Fig. 1. Similar as in the simulation, we set Z2 = 1.5 mm and acquired the in-line holograms at Z1 = 3, 6, 9 cm, then did reconstructions by direct holographic reconstruction and our proposed two-step method.
Before computing the deconvolution, we need to measure the PSF precisely. According to Eq. (3), the PSF is determined by Z1, Z2, and the luminance distribution of the source. We first obtained the luminance distribution of the LED by observing the LED illumination area under a microscope. Figure 4(a) shows the microscope image of the LED under 10X objective. The enlarged image in Fig. 4(b) shows the LED illumination area when the LED was turned on. The PSF was then calculated by shrinking the image of LED illumination area by a ratio of Z1/Z2, and down-sampled so that the pixel size is consistent with that of our CMOS imaging sensor, which is 2.2 µm, as shown in Fig. 4(c).
Fig. 4 (a) The microscope image of the LED acquired by 10X objective; (b) The enlarged image of LED illumination area as indicated in the (a); (c) The calculated PSF.
The experimental results of imaging the USAF target are shown in Fig. 5, where Fig. 5(a) shows the inline hologram, direct reconstruction result, reconstruction result with our proposed method, and section curves of bars in the USAF target extracted from the two reconstructed images as indicated in the figure. We can clearly see from the reconstructed USAF target images and section curves that our proposed method can provide better imaging resolution and contrast. Figures 5(b) and 5(c) show similar results for Z1 = 6 cm and 9 cm, respectively. As expected, the direct reconstruction images became better for larger Z1 because of better spatial coherence of light at the sample plane. Interestingly, in contrast to the simulation, we notice that the reconstruction result for Z1 = 6 cm is obviously better than the result for Z1 = 3 cm. This is because the PSF is large for smaller Z1 and the deconvolution process will affected by the additional system noise in experiments [22]. We also observed that the PSF-related deterioration of image quality will be less significant for small PSF sizes by comparing the result for Z1 = 6 cm and Z1 = 9 cm. Nevertheless, we should also notice that Z1 cannot be too large, because when Z1 gets larger, the illumination light intensity on the sample will be weaker and longer exposure time will be necessary or the system SNR will be reduced.
Fig. 5 Imaging results of USAF target by LIHM with LED illumination and, for comparison, pinhole-associated LED and laser illumination. (a)-(c) in-line holograms, direct reconstruction results, reconstruction results with our proposed method, and section curves of bars in the USAF target extracted from the two reconstructed images as indicated at source-to-sample distances Z1 = 3, 6, 9 cm, respectively; (d)(e) in-line holograms, holographic reconstruction result, and section curve of the bar in USAF target as indicated with 25-μm pinhole associated LED and laser illuminations, respectively.
For comparison, Fig. 5(d) shows the in-line hologram, direct reconstruction result, and a section curve of the bar as indicated when a pinhole with diameter of 25 μm was inserted in front of the LED to enhance the spatial coherence. Because of the small diameter of the pinhole, no deconvolution is necessary in this case. We can see that the imaging resolution is similar as the result with our proposed method.
According to Figs. 5(a)-5(c), the smallest feature we can see in the USAF target under LED illumination was element 3 of group 7 with bar width of 3.1 μm, which is worse than numerical simulation results shown previously. On the other hand, smaller features (element 6 of group 7 with bar width of 2.2 μm) can also be observed with laser illumination, as shown in Fig. 5(d). This is mainly because of the insufficient temporal coherence of the LED with spectral width of 10 nm, which cannot be compensated with the deconvolution process in our proposed reconstruction method. In addition, the deconvolution process was also affected by system noise, especially high-frequency noises tended to be amplified after deconvolution [20].
Next we used our LIHM system to image the wing of a green lacewing for further demonstration of our proposed method, as shown in Fig. 6. Figure 6(a) shows the in-line hologram, direct reconstruction result, the sub-hologram after deconvolution process, and the reconstruction result with our proposed method for Z1 = 3 cm. As expected, our proposed method provided a better reconstruction compared to direct holographic reconstruction without the deconvolution process. Figures 6(b) and 6(c) show similar results for Z1 = 6 cm and 9 cm, respectively. Figures 6(d) and 6(e) show the in-line holograms and direct reconstruction results with 25-μm pinhole associated LED and laser illuminations, respectively. We can see that the imaging resolution with our proposed method for direct LED illumination is similar as conventional holographic reconstruction with pinhole-associated LED illumination. And the results with laser illumination was better because of better temporal coherence. For comparison, Fig. 6(f) shows the microscope image acquired with 10X objective.
Fig. 6 Imaging results of the wing of a green lacewing by LIHM with LED illumination and, for comparison, pinhole-associated LED and laser illumination. (a)-(c) in-line holograms, direct reconstruction results, sub-holograms after deconvolution, and reconstruction results with our proposed methods for source-to-sample distances Z1 = 3, 6, 9 cm, respectively; (d)(e) in-line holograms and holographic reconstruction result with 25-μm pinhole associated LED and laser illuminations, respectively; (f) microscope image observed by a 10X objective; (g) wide field-of-view image of the wing sample by LIHM with our proposed method.
We should also notice that we can obtain wide field-of-view image with LIHM as the field-of-view is only limited by the imaging sensor size, which is 5.7 × 4.2 mm in our experiment. Figure 6(g) shows the reconstructed wide field-of-view image of the wing sample with our proposed two-step reconstruction method.
4. Discussion and conclusion
In this paper we present a novel reconstruction method to compensate for the insufficient spatial coherence in LIHM with spatially-extended light source as illumination. The main idea is to treat the blurry in-line hologram caused by insufficient spatial coherence as the convolution of hologram acquired by point source illumination and a PSF determined by the luminous intensity distribution of the spatially-extended illumination source and the ratio of source-to-sample and sample-to-sensor distances. We can thus acquire the sub-hologram equivalent to the point source illumination case by a simple deconvolution process, then conventional holographic reconstruction can be applied. With our proposed method, we can avoid the use of laser which introduces additional speckle noise in LIHM, and also avoid the use of pinhole-associated or fiber-coupled LED source in order to provide sufficient spatial coherence in LIHM, so the system setup can be more compact, low-cost, and robust.
We should notice that the imaging resolution in LIHM depends on not only spatial coherence, but also temporal coherence of the source, as well as the pixel size of imaging sensor [23–25]. We believe it is possible to combine our proposed methods with other resolution-enhancement techniques to further improve the imaging resolution in LIHM with spatially-extended light source.
To demonstrate the capability of our proposed method, we performed numerical simulations with the USAF target as the sample, and imaging experiments with the USAF target and the wing of a green lacewing. The simulation and experimental results clearly show the resolution enhancement capability of our proposed two-step reconstruction method. With our reconstruction method, the LIHM with spatially-extended light source provides a promising solution for compact and low-cost microscopic imaging scenarios.
Acknowledgments
Shanghai Pujiang Program (12PJ1405100); National Natural Science Foundation of China (NSFC) (61205192).
References and links
1. A. Greenbaum, W. Luo, T. W. Su, Z. Göröcs, L. Xue, S. O. Isikman, A. F. Coskun, O. Mudanyali, and A. Ozcan, “Imaging without lenses: achievements and remaining challenges of wide-field on-chip microscopy,” Nat. Methods 9(9), 889–895 (2012). [PubMed]
2. E. Serabyn, K. Liewer, C. Lindensmith, K. Wallace, and J. Nadeau, “Compact, lensless digital holographic microscope for remote microbiology,” Opt. Express 24(25), 28540–28548 (2016). [PubMed]
3. S. Feng, M. Wang, and J. Wu, “Lensless in-line holographic microscope with Talbot grating illumination,” Opt. Lett. 41(14), 3157–3160 (2016). [PubMed]
4. M. Sanz, J. A. Picazo-Bueno, L. Granero, J. García, and V. Micó, “Compact, cost-effective and field-portable microscope prototype based on MISHELF microscopy,” Sci. Rep. 7, 43291 (2017). [PubMed]
5. H. J. Kreuzer, M. H. Jericho, I. A. Meinertzhagen, and W. Xu, “Digital in-line holography with photons and electrons,” J. Phys. Condens. Matter 13, 10729–10741 (2001).
6. U. Schnars and W. P. O. Jüptner, “Digital recording and numerical reconstruction of holograms,” Meas. Sci. Technol. 13, R85–R101 (2002).
7. W. Xu, M. H. Jericho, H. J. Kreuzer, and I. A. Meinertzhagen, “Tracking particles in four dimensions with in-line holographic microscopy,” Opt. Lett. 28(3), 164–166 (2003). [PubMed]
8. Y. Hao and A. Asundi, “Resolution analysis of a digital holography system,” Appl. Opt. 50(2), 183–193 (2011). [PubMed]
9. H. Meng, W. L. Anderson, F. Hussain, and D. D. Liu, “Intrinsic speckle noise in in-line particle holography,” J. Opt. Soc. Am. A 10, 2046–2058 (1993).
10. P. Picart, “Speckle noise in digital holographic images: genesis and reduction methods,” in Digital Holography and Three-Dimensional Imaging, OSA Technical Digest (online) (Optical Society of America, 2017), paper Th1A.1.
11. L. Repetto, E. Piano, and C. Pontiggia, “Lensless digital holographic microscope with light-emitting diode illumination,” Opt. Lett. 29(10), 1132–1134 (2004). [PubMed]
12. C. Oh, S. O. Isikman, B. Khademhosseinieh, and A. Ozcan, “On-chip differential interference contrast microscopy using lensless digital holography,” Opt. Express 18(5), 4717–4726 (2010). [PubMed]
13. J. Garcia-Sucerquia, “Noise reduction in digital lensless holographic microscopy by engineering the light from a light-emitting diode,” Appl. Opt. 52(1), A232–A239 (2013). [PubMed]
14. W. Bishara, U. Sikora, O. Mudanyali, T. W. Su, O. Yaglidere, S. Luckhart, and A. Ozcan, “Holographic pixel super-resolution in portable lensless on-chip microscopy using a fiber-optic array,” Lab Chip 11(7), 1276–1279 (2011). [PubMed]
15. A. Greenbaum and A. Ozcan, “Maskless imaging of dense samples using pixel super-resolution based multi-height lensfree on-chip microscopy,” Opt. Express 20(3), 3129–3143 (2012). [PubMed]
16. E. Moon, M. Kim, J. Roh, H. Kim, and J. Hahn, “Holographic head-mounted display with RGB light emitting diode light source,” Opt. Express 22(6), 6526–6534 (2014). [PubMed]
17. S. Nakamura, T. Mukai, and M. Senoh, “High-brightness InGaN/AlGaN double-heterostructure blue-green-light-emitting diodes,” Appl. Phys. (Berl.) 76, 8189–8191 (1994).
18. W. Bishara, T. W. Su, A. F. Coskun, and A. Ozcan, “Lensfree on-chip microscopy over a wide field-of-view using pixel super-resolution,” Opt. Express 18(11), 11181–11191 (2010). [PubMed]
19. J. W. Goodman, Introduction to Fourier Optics (Roberts & Company, 2004), Chap. 3.
20. R. Gonzalez, R. Woods, and S. Eddins, Digital Image Processing Using Matlab (Prentice Hall, 2003) Chap. 5.
21. M. Born and E. Wolf, Principles of Optics: Electromagnetic Theory of Propagation, Interference and Diffraction of Light (Cambridge University Press, 1980), Chap. 10.
22. W. Wallace, L. H. Schaefer, and J. R. Swedlow, “A Workingperson’s Guide to Deconvolution in Light Microscopy,” Biotechniques 31(5), 1076–1078 (2001). [PubMed]
23. T. E. Agbana, H. Gong, A. S. Amoah, V. Bezzubik, M. Verhaegen, and G. Vdovin, “aliasing, coherence, and resolution in a lensless holographic microscope,” Opt. Lett. 42(12), 2271–2274 (2017). [PubMed]
24. S. Feng, M. Wang, and J. Wu, “Enhanced resolution in lensless in-line holographic microscope by data interpolation and iterative reconstruction,” Opt. Commun. 402, 104–108 (2017).
25. M. Kanka, R. Riesenberg, P. Petruck, and C. Graulig, “High resolution (NA = 0.8) in lensless in-line holographic microscopy with glass sample carriers,” Opt. Lett. 36(18), 3651–3653 (2011). [PubMed]
