Expand this Topic clickable element to expand a topic
Skip to content
Optica Publishing Group

Noniterative sub-pixel shifting super-resolution lensless digital holography

Open Access Open Access

Abstract

Lensless digital holography (LDH) is gaining considerable attention lately due to a simple experimental setup, wide field-of-view, and three-dimensional (3D) imaging capability. Since the resolution of LDH is limited by the Nyquist frequency of a detector array, the major drawback of LDH is resolution, and a lot of efforts were made to enhance the resolution of LDH. Here we propose and demonstrate a fast noniterative sub-pixel shifting super-resolution technique that can effectively enhance the resolution of LDH by a factor of two. We provide detailed frequency-domain formulae for our noniterative frequency-domain super-resolution method. The validity of our proposed method is experimentally demonstrated both for scattering and phase objects.

© 2021 Optical Society of America under the terms of the OSA Open Access Publishing Agreement

1. Introduction

A conventional imaging system is composed of numerous optical components to illuminate an object and collect waves that are scattered or diffracted from the object. The collected object waves by an imaging system are transformed from the original object waves of a sample. This transformation of information in a measured image by an optical imaging system is described by an optical transfer function and its cut-off frequency in the frequency domain. Since the cut-off frequency of an optical system is proportional to the numerical aperture (NA) of collected light in the object space [1], the major approach to improve image quality is hardware-based optical super-resolution imaging that enhances the cut-off frequency of an imaging system by making the NA of collected light larger. Structured illumination microscopy [2], synthetic aperture microscopy [3,4], ptychography [5,6], and coded aperture correlation holography [7,8] are notable hardware-based optical super-resolution schemes that showed significant resolution improvements.

Another important approach to improve the spatial resolution of an image is software-based super-resolution image reconstruction, where a super-resolution image is constructed from multiple low-resolution images by using signal processing techniques [917]. Many ingenious super-resolution image reconstruction schemes have been proposed by using interpolation or interlacing [1821], frequency-domain analysis [1214], and regularization methods [2225]. Improved resolutions are obtained by using multiple images obtained with conventional optical systems. In general, software-based super-resolution image reconstruction is an ill-posed problem because the exact translational shift distances between super-resolution images, noises, and the amount of blurring are unknown within low-resolution images obtained by conventional imaging systems.

Lensless digital holography is a simple and compact imaging system that can image a target object by measuring the object waves with just two elements: a light source and an arrayed detector [2628]. Since there is no optical element between a sample and a detector, LDH has many advantages in applying software-based super-resolution image reconstruction. Image registration or estimating relative shift distances between low-resolution images is simple and straightforward in LDH because translational shift distances can be directly controlled by moving an arrayed detector. And there is no focus blur problem in LDH since an optimum focused image can be obtained by numerical focusing [29,30]. Therefore, aliasing and noise in measured low-resolution images are two major sources of error in LDH for software-based super-resolution image reconstruction [31].

Many software-based super-resolution image reconstruction methods have been reported for LDH [1517,10,32]. However, most of these schemes used iterative regularization algorithms, and it is not easy to apply them to practical real-time imaging applications. In this paper, we propose a deterministic sub-pixel shifting super-resolution technique by using simple arithmetic equations proposed initially by Tsai and Huang [14]. Only four sub-pixel shifted low-resolution images are required to reconstruct a super-resolution image whose Nyquist frequencies are doubled along the horizontal and vertical axes. And we require three super-resolution images taken at different axial positions to extract the complex optical field. Since the registration of low-resolution images is precisely controlled, and image blurring can be effectively reduced by numerical focusing, iterative regularization or optimization is unnecessary for many super-resolution image reconstruction applications in LDH.

We derived linear relations between aliased sub-pixel shifted holograms and the original unaliased hologram in the frequency domain. Then, we provide a matrix that represents the relationship between the unaliased hologram and four aliased sub-pixel shifted holograms. The arithmetic procedures to obtain the original unaliased frequency-domain data of a hologram from four aliased sub-pixel shifted holograms are analogous to those used in structured illumination microscopy [2]. Four sub-pixel shifted low-resolution images along the horizontal, vertical, and diagonal directions are used to double the Nyquist frequencies of the horizontal and vertical directions. The total area of a calculated super-resolution image is four times larger than that of a measured low-resolution image in the frequency domain. Since a super-resolution image is obtained with simple arithmetic operations in the frequency domain without any estimation or iteration process, the computation time of our proposed method is much faster than those of conventional iterative methods. The validity of our proposed method was verified via simulations and experiments. First, the principles of our sub-pixel shifted super-resolution method were presented with simple arithmetic formula in the frequency domain. Then, the procedures to reconstruct a super-resolution image from four sub-pixel shifted low-resolution images are shown with numerical simulations. Third, the performance of our proposed super-resolution method was demonstrated experimentally with two samples: a USAF 1951 resolution target and 2 μm-diameter polystyrene beads.

2. Principle

Figure 1(a) shows a typical LDH system and the data acquisition process of our sub-pixel shifting super-resolution technique. A transmission-type sample is exposed with a fiber-coupled laser source which is located far from the sample. Holograms are captured by putting a detector array just behind a sample without using any optical component. The distance between a detector array and a sample is normally less than a millimeter, and it is much smaller than the distance between the light source and the detector array, the magnification of our LDH system is close to one. The data acquisition process for the sub-pixel shifting super-resolution technique is illustrated in Fig. 1(b) with a one-dimensional (1D) detector array. Two different images are captured while shifting the 1D detector array by a sub-pixel distance. The number of pixels in the detector array is N, and the pixel pitch is p. To make our analysis simple, we assume the fill factor of the detector array is one; each pixel is a perfect square with a side length of p. The frequency difference between two neighboring data points in the frequency domain becomes ${\xi _0} = 1/({Np} )$, and the Nyquist frequency of a sampled image is ${\xi _{Nyquist}} = 1/({2p} )$. In LDH, the cut-off frequency ${\xi _c}$ is defined as the maximum non-zero frequency component of an object wave in front of a detector array. We consider weakly aliased images with ${\xi _{Nyquist}} < {\xi _c} < 2{\xi _{Nyquist}}$. Even when ${\xi _c} > 2{\xi _{Nyquist}}$, the frequency spectrum of a measured image is multiplied by the Fourier transform of the shape of a single pixel, and the amplitudes of the frequency components above $2{\xi _{Nyquist}}$ frequency becomes negligibly small.

 figure: Fig. 1.

Fig. 1. Schematic diagram of a LDH system and the data acquisition process of sub-pixel shifting super-resolution technique. A low-frequency component ${S_L}$ and a corresponding high-frequency component ${S_H}$ are added (aliased) together to form ${L_1}$ and ${L_2}$ within two sub-pixel shifted holograms. ${S_L}$ and ${S_H}$ can be retrieved with simple arithmetic operations from ${L_1}$ and ${L_2}$.

Download Full Size | PDF

For an LDH system with ${\xi _{Nyquist}} < {\xi _c} < 2{\xi _{Nyquist}}$ the complete unaliased frequency components of a 1D image can be retrieved from two sub-pixel shifted 1D images. Fig. (1-c) shows how frequency components above the Nyquist frequency are aliased in measured images. ${S_L}$ represent an unaliased low-frequency component at $({ - {\xi_{Nyquist}} + k{\xi_0}} )$ frequency, and ${S_H}$ is another unaliased high-frequency component at $({{\xi_{Nyquist}} + k{\xi_0}} )$ frequency, where k is an arbitrary integer within a range of [1, N]. The locations of ${S_L}$ and ${S_H}$ are illustrated in Fig. 1(c). We denote ${L_1}$ and ${L_2}$ as two aliased frequency components at $({ - {\xi_{Nyquist}} + k{\xi_0}} )$ frequency in two measured images; one is for the original image, and the other is for a sub-pixel shifted image by a distance d. ${L_1}$ and ${L_2}$ can be written with linear combinations of ${S_L}$ and ${S_H}$ [14].

$$\left( {\begin{array}{cc} 1&1\\ {{e^{i\pi \frac{d}{p}}}{e^{i\pi \frac{{dk}}{{Np}}}}}&{{e^{ - i\pi \frac{d}{p}}}{e^{i\pi \frac{{dk}}{{Np}}}}} \end{array}} \right)\left( {\begin{array}{c} {{S_L}}\\ {{S_H}} \end{array}} \right) = \left( {\begin{array}{c} {{L_1}}\\ {{L_2}} \end{array}} \right)$$

When the sub-pixel shifting distance is half of the pixel pitch, the differences between the two sub-pixel shifted images becomes the largest, and the subsequent differences between ${L_1}$ and ${L_2}$ in the frequency domain becomes the most prominent. For half-pixel shifted images, we have $d = \,p/2$ and eq.(1) can be simplified to

$$\left( {\begin{array}{c} {{S_L}}\\ {{S_H}} \end{array}} \right) = \frac{1}{2}\left( {\begin{array}{cc} 1&{ - i}\\ 1&i \end{array}} \right)\left( {\begin{array}{c} {{L_1}}\\ {{L_2}{e^{ - i\pi \frac{k}{{2N}}}}} \end{array}} \right)$$

Exact unaliased frequency components ${S_L}$ and ${S_H}$ can be retrieved from two measured aliased frequency components ${L_1}$ and ${L_2}$ by using Eq. (2) when ${\xi _{Nyquist}} < {\xi _c} < 2{\xi _{Nyquist}}$.

We can further extend these analyses to two-dimensional (2D) images. We consider a 2D LDH system with ${\xi _{Nyquist}} < {\xi _c} < 2{\xi _{Nyquist}}$ and ${\eta _{Nyquist}} < {\eta _c} < 2{\eta _{Nyquist}}$, where $\xi $ and $\eta $ are spatial frequencies along the horizontal (x-axis) and the vertical (y-axis) directions of an image. Similar to the 1D analyses, the Nyquist frequencies are expressed as ${\xi _{Nyquist}} = 1/({2{p_x}} )$ and ${\eta _{Nyquist}} = 1/({2{p_y}} )$, where ${p_x},{p_y}$ denote the pixel pitches of a 2D arrayed detector along the x and y directions, respectively. Frequency spacings between two neighboring data points along the horizontal and the vertical directions in the frequency domain are $\,{\xi _0} = 1/({M{p_x}} ),\; {\eta _0} = 1/({N{p_y}} )$, where M and N are the number of pixels in an image sensor along the x and y directions, respectively. Figure 2 illustrates how four sub-pixel shifted LDH images are taken and processed in the frequency domain by using the 2D fast Fourier transform (2D-FFT). ${L_t}({m{\xi_0},n{\eta_0}} )$: $t = [{1,\,4} ]$ represent four aliased frequency components calculated from the four sub-pixel shifted images at a given position of spatial frequency coordinates at $\xi = m{\xi _0}$ and $\eta = n{\eta _0}$. t is an integer representing the four sub-pixel shifting displacement vectors ${({{d_x},{d_y}} )_t}$ illustrated in Fi.g 2. We choose the position of the first displacement vector as the origin, then we have ${({{d_x},{d_y}} )_1} = ({0,\; 0} )$. Coordinates $({m{\xi_0},n{\eta_0}} )$ represent a position in the first quadrant within a green square illustrated in the center figure of Fig. 2 when m and n are integers within $[{0,\; M/2} ]$ and $[{0,\; N/2} ]$, respectively. We define $S({m{\xi_0},n{\eta_0}} )$ as an unaliased frequency component at a given position of $({m{\xi_0},n{\eta_0}} )$. Then, each of ${L_{1\sim 4}}({m{\xi_0},n{\eta_0}} )$ can be written with a linear combination of unaliased frequency components: $S({m{\xi_0},n{\eta_0}} )$, $S({m{\xi_0} - 2{\xi_{Nyquist}},n{\eta_0}} )$, $S({m{\xi_0},n{\eta_0} - 2{\eta_{Nyquist}}} )$, and $S({m{\xi_0} - 2{\xi_{Nyquist}},n{\eta_0} - 2{\eta_{Nyquist}}} )$ [14].

$${L_t}({m{\xi_0},n{\eta_0}} )= \mathop \sum \nolimits_{\alpha = 0}^{\alpha = 1} \mathop \sum \nolimits_{\beta = 0}^{\beta = 1} S({m{\xi_0} - 2\alpha {\xi_{nyquist}},n{\eta_0} - 2\beta {\eta_{Nyquist}}} ){\phi _{\alpha ,\beta ,t\,}}.$$
$$\{{{\phi_{\alpha ,\beta ,t}}} \}= \left\{ {exp\left( {i\pi \left( {\left( {{{({ - 1} )}^\alpha }\frac{{{d_{x,t}}}}{{{p_x}}} + \frac{{{d_{x,t}}m}}{{M{p_x}}}} \right) + \left( {{{({ - 1} )}^\beta }\frac{{{d_{y,t}}}}{{{p_y}}} + \frac{{{d_{y,t}}n}}{{N{p_y}}}} \right)} \right)} \right)} \right\}$$
${d_{x,\,t}}$ and ${d_{y,\,t}}$ are displacements along the x-axis and y-axis for a given sub-pixel shifting vector ${({{d_x},{d_y}} )_t}$. Four frequency components $S({m{\xi_0},n{\eta_0}} )$, $S({m{\xi_0} - 2{\xi_{Nyquist}},n{\eta_0}} )$, $S({m{\xi_0},n{\eta_0} - 2{\eta_{Nyquist}}} )$, $S({m{\xi_0} - 2{\xi_{Nyquist}},n{\eta_0} - 2{\eta_{Nyquist}}} )$ are located at different quadrants in the frequency domain. These unaliased frequency components are within the extended frequency range illustrated with a large green square in the right figure of Fig. 2. Equation (3) and (4) generates a 4 by 4 conversion matrix which decomposes four aliased frequency components with four unaliased frequency components. We can obtain four unaliased frequency components from four sub-pixel shifted aliased frequency components by using the inverse matrix of the conversion matrix. This method can be applied to any four sub-pixel shifted images unless the four displacement vectors ${({{d_x},{d_y}} )_t}$ produces a singular conversion matrix. The first sub-pixel shifting displacement vector is at the origin: ${({{d_x},{d_y}} )_1} = ({0,\,0} )$. The three remaining displacement vectors should be within the area of a single pixel: $0 < {d_{x,\,t}} \le {p_x}$, $0 < {d_{y,\,t}} \le {p_y}$. If more than two displacement vectors are located on the x-axis or the y-axis, the conversion matrix becomes singular. We have also a singular conversion matrix if any of the three remaining displacement vectors coincide with $({{p_x},0} )$, $({0,{p_y}} )$, or $({{p_x},{p_y}} )$.

 figure: Fig. 2.

Fig. 2. Four under-sampled images ${l_{1\sim 4}}$ captured with an LDH system by shifting a detector array by four shifting vectors $\{{{{({{d_x},{d_y}} )}_t}} \}$. ${L_{1\sim 4}}({m{\xi_0},n{\eta_0}} )$ are four aliased frequency-domain data calculated by 2D-FFT. $S({m{\xi_0},n{\eta_0}} )$ represents unaliased frequency-domain data

Download Full Size | PDF

For half-pixel shifted images obtained with a 2D arrayed detector, we choose the four displacement vectors as $({0,\,0} )$, $({{p_x}/2,\; 0} )$, $({0,{p_y}/2} )$, $({{p_x}/2,{p_y}/2} )$. Then, Eq. (3) and (4) can be simplified to

$$\left( {\begin{array}{c} {{L_1}(m{\xi_0},n{\eta_0})}\\ {{L_2}(m{\xi_0},n{\eta_0}){e^{ - i\pi \frac{m}{{2M}}}}}\\ {{L_3}(m{\xi_0},n{\eta_0}){e^{ - i\pi \frac{n}{{2N}}}}}\\ {{L_4}(m{\xi_0},n{\eta_0}){e^{ - \frac{{i\pi }}{2}(\frac{m}{M} + \frac{n}{N})}}} \end{array}} \right) = \left( {\begin{array}{cccc} 1&1&1&1\\ i&{ - i}&i&{ - i}\\ i&i&{ - i}&{ - i}\\ { - 1}&1&1&{ - 1} \end{array}} \right)\left( {\begin{array}{c} {S(m{\xi_0},n{\eta_0})}\\ {S(m{\xi_0},n{\eta_0} - 2{\xi_{Nyquist}},n{\eta_0})}\\ {S(m{\xi_0},n{\eta_0} - 2{\eta_{Nyquist}})}\\ {S(m{\xi_0} - 2{\xi_{Nyquist}},n{\eta_0} - 2{\eta_{Nyquist}})} \end{array}} \right).$$
${L_{1\sim 4}}({m{\xi_0},n{\eta_0}} )$ are frequency components of four half-pixel shifted low-resolution images within the small frequency range of $|\xi |< {\xi _{Nyquist}}$ and $|\eta |< {\eta _{Nyquist}}$. $\textrm{S}({m{\xi_0},n{\eta_0}} )$, $S({m{\xi_0} - 2\alpha {\xi_{Nyquist}},n{\eta_0}} )$, $S({m{\xi_0},n{\eta_0} - 2\beta {\eta_{Nyquist}}} )$, $S({m{\xi_0} - 2\alpha {\xi_{Nyquist}},n{\eta_0} - 2\beta {\eta_{Nyquist}}} )$ are four unaliased frequency components within the extended frequency range of $|\xi |< 2{\xi _{Nyquist}}$ and $|\eta |< 2{\eta _{Nyquist}}$. The four unaliased frequency components in the extended frequency range are related to the four aliased frequency components in the smaller frequency range with a simple 4 × 4 matrix in Eq. (5). Since the 4 × 4 matrix is reversible, we can retrieve the frequency components of a super-resolution image within the full extended frequency range, where the Nyquist frequencies along the x and y axes are doubled. Equation (3)-(5) are for frequency components within the first quadrant of a small green box in the center figure of Fig. 2. We can obtain similar relations between aliased and unaliased frequency components in the other quadrants of the green box. All unaliased frequency components can be obtained in the extended frequency range illustrated with a large green box in the right figure of Fig. 2. Procedures to obtain a super-resolution image are illustrated in Fig. 3 (Step 1). There exist twin images in a calculated super-resolution image obtained by the procedure.

 figure: Fig. 3.

Fig. 3. A flowchart of sub-pixel shifting super-resolution technique and multi-height phase retrieval method to obtain complex optical field. (Step 1): We measure four holograms by shifting a sample by half-pixel distances along the x and y axes. One super-resolution image is calculated by using Eq. (5) with an noniterative method. (Step 2): We shifted the sample along the z-axis by a known distance (300 μm in our case), and repeat the process described in (Step 1) to obtain another super-resolution image. We repeat the process of (Step 2) to obtain the third super-resolution image. We calculate the complex optical field of an image by using the MPR method. Finally, the best-focused super-resolution image is obtained by numerical focusing.

Download Full Size | PDF

The twin-image problem can be solved if we obtain the complex optical field of an image. We used the multi-height phase retrieval (MPR) method to extract the complex optical field of an image [17,33,34]. Figure 3 (Step 2) shows the procedure to obtain the best-focused super-resolution image. Three longitudinally shifted super-resolutions images are obtained for the MPR method by moving a detector along the axial direction with a step size of 300 μm. In order to obtain a best-focused super-resolution image, we used the angular spectrum method (ASM) [26,35] for numerical focusing. When an optical field propagates a specific distance z, its output spectrum can be obtained by multiplying the following kernel G to the input spectrum [30].

$$G({z,\xi ,\eta } )= exp \left( {\frac{{2\pi zi}}{\lambda }\sqrt {1 - {\lambda^2}{\xi^2} - {\lambda^2}{\eta^2}} } \right).$$
λ denotes a given center wavelength of a light source. $\xi $ and $\eta $ are spatial frequencies along the x and y axes, respectively. The output optical field R(z,x,y) is computed by the inverse Fourier transform of the output spectrum with
$$R({z,x,y} )= \; {F^{ - 1}}[{F\{{H({0,x,y} )} \}\times G({z,\xi ,\eta } )} ].$$
$F\{{H({0,x,y} )} \}$ is an unaliased complex optical frequency component in the extended frequency range obtained by Eq. (5) and the MPR. Since $H({0,x,y} )$ is the input optical field with discrete data points, a discretized kernel G should be used, whose number of data points is the same as that of the input data points in the extended frequency range. The reconstructed image of a scattering object has its maximum contrast near its focused axial position. We calculate the best focused axial position of an image by using the ASM, where the contrast of an optical output field is maximized. In this study, we used the Sobel operator to calculate the variance of intensity and utilized it as the focus measure to determine the best-focusing axial position of an image [29,36].

3. Experimental results

For experimental demonstration, we constructed an LDH system, which is illustrated in Fig. 1(a). Our LDH setup comprises only four components: a fiber-coupled light source, a collimating lens, a transmission type sample, and a detector array. The detector array is on a three-axis piezoelectric transducer (PZT) stage, which can move the detector along the x, y, and z axes with a nanometer-scale accuracy. We used a single-mode fiber-coupled laser (S1FC635, Thorlabs Inc.) at 635 nm wavelength. Light from the fiber end was collimated by a convex lens with a 35 mm focal length. Holograms were captured by a monochrome CMOS image sensor (DMK24UJ003, The Imaging Source Inc.). The image sensor has 3856 × 2764 pixels with 1.67-μm pixel pitches along the x and y axes. Measured holograms were captured with 8-bit resolution.

3.1 USAF resolution target as a scattering object

We have verified the resolution enhancement of our proposed super-resolution method by using the USAF 1951 resolution target (R3L3S1P, Thorlabs, Inc.). It is placed between the collimated laser light and the image sensor mounted on a PZT stage. Figure 4(a) shows a low-resolution LDH image of the USAF resolution target. The field-of-view (FOV) of each low-resolution hologram equals the image sensor area, which is about 6.44 × 4.62 mm2. Four half-pixel shifted low-resolution holograms were captured with half-pixel (835 nm) distance shifts along the x and y axes. A super-resolution hologram is calculated using the four half-pixel shifted low-resolution holograms in the frequency domain with the procedures described in section 2. The differences of aliased and unaliased spectra in measured data can be hardly noticed in the frequency domain. The intensity plots of four aliased functions L1∼4 and one reconstructed unaliased function S illustrated in Fig. 3 are measured data for the four measured sub-pixel shifted images and the reconstructed super-resolution image of the USAF 1951 resolution target shown in Fig. 4. Since the distance between the sample and the detector array is about 1 mm in our system, the calculated super-resolution hologram is out of focus and has twin images. We used the MPR to eliminate a twin image and obtain a focused super-resolution image. For MPR, we measured two extra longitudinally shifted super-resolution holograms, each of which is calculated from a separate set of four half-pixel shifted low-resolution holograms. From the complex optical field of the super-resolution hologram, we obtained the best-focused image of a measured super-resolution hologram with numerical focusing by using the ASM.

 figure: Fig. 4.

Fig. 4. Comparison of images obtained with three different imaging methods. (a) A typical out-of-focus low-resolution hologram obtained with our LDH setup. Images in (b1) and (c1) are enlarged views of a low-resolution hologram at its best focused plane, which are calculated by using numerical focusing. Images in (b2) and (c2) are obtained with our proposed sub-pixel super-resolution method. Images shown in (b3) and (c3) are obtained with conventional optical microscopy with an NA = 0. 25 objective.

Download Full Size | PDF

Figure 4(b2) and (c2) are zoomed views of the super-resolution image at its best-focused plane. Three horizontal bars in the (9-2) pattern of the USAF 1951 resolution target can be recognized within Fig. 4(c2). The figure on the right side of Fig. 4(c2) is the enlarged pattern of (9-2) and an intensity profile along the vertical line through its center. The widths of three dark rectangles and two white spacings between them are all 0.87 μm along the vertical direction. Therefore, the center-to-center distance between adjacent black rectangles is 1.74 μm.

To demonstrate the resolution enhancement in our proposed method, we have also calculated the best-focused image obtained with low-resolution LDH without sub-pixel shifted super-resolution imaging. Only three longitudianlly shifted holograms are used to obtain the complex optical field of a low-resolution image. The best-focused image is calculated with numerical focusing by using the ASM. Figure 4(b1) and (c1) are enlarged views of the low-resolution image at its best-focused position. Up to the (8-3) pattern can be recognized with the conventional LDH. The right side of Fig. 4(c1) is the enlarged pattern of (8-3) and its intensity profile along the vertical line through its center. The width of three identical dark rectangles and two identical white spacings between them are 1.55 μm for the (8-3) pattern. The center-to-center distance between adjacent black rectangles is 3.10 μm.

We have also compared our results with those of conventional optical microscopy. Figure 4(b3) is an image captured by an optical microscope with a 10× objective (Olympus PLN10X) whose numerical aperture (NA) is 0.25. There was no condenser lens used for the measurement. Figure 4(c3) is an enlarged image of Fig. 4(b3). Up to the (8-6) pattern can be recognized in this figure. The right side of Fig. 4(c3) is the enlarged pattern of (8-6) and its intensity profile along the vertical line through its center. The width of three dark rectangles and two white spacings between them are all 1.10 μm for the (8-6) pattern. The center-to-center distance between adjacent black rectangles is 2.20 μm. The visibilities of line profiles shown on the right side of Fig. 4(c1), (c2), and (c3) are 0.15, 0.16, and 0.15, respectively. These results show that the resolution of our super-resolution scheme is similar to that of a conventional optical microscope with an objective of 10× magnification and 0.25 NA. Note that the FOV of our proposed method is 100 times larger than that of an optical microscope with a 10× objective if we use the same image sensor in both systems. Note that smooth images shown in Fig. 4(b3) and Fig. 4(c3) are conventional microscopy images obtained with a halogen lamp. Other images in Fig. 4 are LDH images are obtained with a laser and show a considerable amount of speckle noise. The reduction of speckle noise in our system is a subject for future studies.

3.2 Polystyrene beads as a phase object

We have also tested our proposed super-resolution method for a phase object. Polystyrene beads of 2-µm diameter (Thermo Scientific 4202A) are fixed on a microscope coverslip with an optical adhesive (Norland, NOA 68) and used as a phase object. Since our proposed method needs to obtain several shifted holograms for the same target, the sample have to be fixed. We used the same experimental setup and reconstruction procedures used for the USAF resolution target. Figure 5(a) is an out-of-focus low-resolution hologram captured by our LDH system. Three longitudinally shifted low-resolution holograms are used to obtain complex electric fields by using the MPR. The best focused low-resolution image is calculated by using the ASM. Figure 5(b1) shows the amplitude of a numerically focused image within an area highlighted with a white dashed square in Fig. 5(a). Since the amplitude contrast of a phase object is minimum at the best-focused plane, we have defocused the image less than 1 μm distance from the best-focused plane. Two small areas depicted with blue and red squares within Fig. 5(b1) are enlarged and displayed on the right side of Fig. 5(b1). Figure 5(c1) is the phase map corresponding to the amplitude image of Fig. 5(b1). Two enlarged images on the right side of Fig. 5(c1) are the two enlarged phase images corresponding to the two amplitude images shown on the right side of Fig. 5(b1). Since polystyrene beads are phase objects, their structures can be seen much apparent in the phase images.

 figure: Fig. 5.

Fig. 5. Resolution enhancements in sub-pixel shifting super-resolution method for phase objects. (a) An out-of-focus low resolution hologram for 2-µm polystyrene beads obtained with our LDH system. Images in (b1) and (c1) are enlarged views of a low-resolution image at its best focused plane. (b1) amplitude map and (c1) phase map. Images in (b2) and (c2) are obtained with our proposed sub-pixel super-resolution method. (b2) amplitude map (c2) phase map

Download Full Size | PDF

We obtained a super-resolution hologram by using four sub-pixel shifted low-resolution images. To use the MPR, we have obtained two extra super-resolution holograms, each of which is calculated from a separate set of four half-pixel shifted low-resolution holograms. The complex optical field of a super-resolution hologram is calculated from the three longitudinally shifted super-resolution holograms. Then, we obtained the best-focused image by using the ASM. Figure 5(b2) shows the amplitude of the best-focused super-resolution image within an area indicated by the white dashed square in Fig. 5(a). Two images on the right of Fig. 5(b2) are enlarged images within the blue and the red squares of Fig. 5(b2). Both figures clearly show much more detailed structures than the two figures on the right side of Fig. 5(b1). Figure 5(c2) is the phase map corresponding to the amplitude image of Fig. 5(b2). Two enlarged images on the right side of Fig. 5(c2) are the two enlarged phase images corresponding to the two amplitude images shown on the right side of Fig. 5(b2). We can see the resolution improvements in images obtained with our proposed sub-pixel shifting super-resolution method. In Fig. 5, the phase difference of the polystyrene beads is similar, but it can be confirmed that the lateral resolution is improved.

4. Conclusion

The major drawback of LDH is its low spatial resolution that is limit by the pixel size of an arrayed detector. Numerous pixel super-resolution methods have been suggested to enhance the spatial resolution of LDH. However, most of those methods are based on iterative algorithms, and their computation speeds are inevitably slow. To alleviate this problem, we proposed a noniterative pixel super-resolution scheme by using four sub-pixel shifted images. We use the properties of aliased frequency components in sub-pixel shifted images in the frequency domain. We provide four independent linear equations that relate frequency components in four aliased sub-pixel shifted images to unaliased frequency components in a super-resolution image. We have demonstrated the validity of our proposed method by imaging two different objects: the USAF 1951 resolution target as a scattering object and polystyrene beads attached to a cover slide as a phase object.

The first advantage of our method is a fast computational speed. It is due to the noniterative algorithm of our method; we use a predetermined inverse matrix to calculate the frequency components of a super-resolution image from those of four undersampled images. The second advantage of our method is no need for image registration. Since the sensor position of our LDH setup is precisely controlled with a PZT stage, the relative positions of four sub-pixel shifted images are predetermined, and image registration is often not required. The third advantage of our method is a simple experimental setup without using any wavelength-scanning or angles-scanning apparatus for a light source. We anticipate that our proposed sub-pixel shifting super-resolution scheme can be applied to any imaging system where the cut-off frequency of an incident wave to an image sensor is larger than the Nyquist frequency of the image sensor. Since our system is based on in-line LDH, we need to measure three phase-shifted holograms to obtain the complex optical field of each sub-pixel shifted image. Lately, there are some papers on off-axis LDH [37,38]. The total number of required holograms can be reduced from twelve to four if off-axis LDH is adapted in our proposed scheme.

Funding

Korea Institute for Advancement of Technology (KIAT), and the Swiss Innovation Agency (Innosuisse) through S.Korea–Switzerland Joint Innovation Project (P0011925); National Research Foundation of Korea (NRF) through the Basic Science Program (2019R1A4A1025958, 2021R1A2C2009090).

Disclosures

No disclosures

Data availability

Data underlying the results presented in this paper are not publicly available at this time but may be obtained from the authors upon reasonable request.

References

1. G. D. Boreman, Modulation transfer function in optical and electro-optical systems (SPIE press Bellingham, WA, 2001). [CrossRef]  

2. M. G. Gustafsson, “Surpassing the lateral resolution limit by a factor of two using structured illumination microscopy,” J. Microsc. 198(2), 82–87 (2000). [CrossRef]  

3. L. Granero, V. Micó, Z. Zalevsky, and J. García, “Synthetic aperture superresolved microscopy in digital lensless Fourier holography by time and angular multiplexing of the object information,” Appl. Opt. 49(5), 845–857 (2010). [CrossRef]  

4. T. S. Ralston, D. L. Marks, P. S. Carney, and S. A. J. N. p. Boppart, “Interferometric synthetic aperture microscopy,” Opt. Express 3, 129–134 (2007). [CrossRef]  

5. A. M. Maiden, M. J. Humphry, F. Zhang, and J. M. J. J. A. Rodenburg, “Superresolution imaging via ptychography,” J. Opt. Soc. Am. A 28(4), 604–612 (2011). [CrossRef]  

6. P. C. Konda, L. Loetgering, K. C. Zhou, S. Xu, A. R. Harvey, and R. Horstmeyer, “Fourier ptychography: current applications and future promises,” Opt. Express 28(7), 9603–9630 (2020). [CrossRef]  

7. M. R. Rai and J. Rosen, “Resolution-enhanced imaging using interferenceless coded aperture correlation holography with sparse point response,” Sci. Rep. 10(1), 1–12 (2020). [CrossRef]  

8. A. Vijayakumar and J. Rosen, “Interferenceless coded aperture correlation holography–a new technique for recording incoherent digital holograms without two-wave interference,” Opt. Express 25(12), 13883–13896 (2017). [CrossRef]  

9. C. Dong, C. C. Loy, K. He, and X. Tang, “Image super-resolution using deep convolutional networks,” IEEE Trans. Pattern Anal. Mach. Intell. 38(2), 295–307 (2016). [CrossRef]  

10. M. Wang, S. Feng, and J. Wu, “Multilayer pixel super-resolution lensless in-line holographic microscope with random sample movement,” Sci. Rep. 7(1), 1–8 (2017). [CrossRef]  

11. S. Borman and R. L. Stevenson, “Super-resolution from image sequences-a review,” in 1998 Midwest symposium on circuits and systems (Cat. No. 98CB36268) (IEEE 1998), pp. 374–378.

12. J. C. Gillette, T. M. Stadtmiller, and R. C. Hardie, “Aliasing reduction in staring infrared imagers utilizing subpixel techniques,” Opt. Eng. 34(11), 3130–3137 (1995). [CrossRef]  

13. S. P. Kim and W.-Y. Su, “Recursive high-resolution reconstruction of blurred multiframe images,” IEEE Trans. on Image Process 2(4), 534–539 (1993). [CrossRef]  

14. R. Y. Tsai and T. S. Huang, “Multiframe image restoration and registration,” Advances in Computer Vision and Image Processing 1, 317–339 (1984).

15. W. Luo, Y. Zhang, Z. Göröcs, A. Feizi, and A. Ozcan, “Propagation phasor approach for holographic image reconstruction,” Sci. Rep. 6(1), 1–15 (2016). [CrossRef]  

16. W. Bishara, T.-W. Su, A. F. Coskun, and A. Ozcan, “Lensfree on-chip microscopy over a wide field-of-view using pixel super-resolution,” Opt. Express 18(11), 11181–11191 (2010). [CrossRef]  

17. A. Greenbaum and A. Ozcan, “Maskless imaging of dense samples using pixel super-resolution based multi-height lensfree on-chip microscopy,” Opt. Express 20(3), 3129–3143 (2012). [CrossRef]  

18. J. Brown, “Multi-channel sampling of low-pass signals,” IEEE Trans. Circuits Syst. 28(2), 101–106 (1981). [CrossRef]  

19. H. Ur and D. Gross, “Improved resolution from subpixel shifted pictures,” CVGIP: Graphical models and image processing 54(2), 181–186 (1992). [CrossRef]  

20. T. Komatsu, K. Aizawa, T. Igarashi, and T. J. I. P. I. Saito, “Signal-processing based method for acquiring very high resolution images with multiple cameras and its theoretical analysis,” IEE Proceedings I (Communications, Speech and Vision)140, 19–25 (1993).

21. A. Fruchter and R. Hook, “Drizzle: A method for the linear reconstruction of undersampled images,” Publ. Astron. Soc. Pac. 114(792), 144–152 (2002). [CrossRef]  

22. M. R. Banham and A. K. Katsaggelos, “Digital image restoration,” IEEE Signal Process. Mag. 14(2), 24–41 (1997). [CrossRef]  

23. M.-C. Hong, M. G. Kang, and A. K. Katsaggelos, “Regularized multichannel restoration approach for globally optimal high-resolution video sequence,” in Visual Communications and Image Processing'97 (International Society for Optics and Photonics1997), pp. 1306–1316.

24. R. C. Hardie, K. J. Barnard, and E. E. Armstrong, “Joint MAP registration and high-resolution image estimation using a sequence of undersampled images,” IEEE Trans. on Image Process. 6(12), 1621–1633 (1997). [CrossRef]  

25. S. Kim, N. K. Bose, and H. M. Valenzuela, “Recursive reconstruction of high resolution image from noisy undersampled multiframes,” IEEE Trans. Acoust., Speech, Signal Process. 38(6), 1013–1027 (1990). [CrossRef]  

26. T. M. Kreis, M. Adams, and W. P. Jüptner, “Methods of digital holography: a comparison,” in Optical Inspection and Micromeasurements II (International Society for Optics and Photonics, 1997), pp. 224–233.

27. L. Repetto, E. Piano, and C. Pontiggia, “Lensless digital holographic microscope with light-emitting diode illumination,” Opt. Lett. 29(10), 1132–1134 (2004). [CrossRef]  

28. A. C. Sobieranski, F. Inci, H. C. Tekin, M. Yuksekkaya, E. Comunello, D. Cobra, A. Von Wangenheim, and U. Demirci, “Portable lensless wide-field microscopy imaging platform based on digital inline holography and multi-frame pixel super-resolution,” Light: Sci. Appl. 4(10), e346 (2015). [CrossRef]  

29. F. Dubois, C. Schockaert, N. Callens, and C. Yourassowsky, “Focus plane detection criteria in digital holography microscopy by amplitude analysis,” Opt. Express 14(13), 5895–5908 (2006). [CrossRef]  

30. H. J. Lee, P. J. Jeon, J. W. Kim, and D. Y. Kim, “An accurate algorithm to find the axial position of an object in lens-free inline digital holography,” in Practical Holography XXXI: Materials and Applications (International Society for Optics and Photonics, 2017), p. 101270P.

31. T. E. Agbana, H. Gong, A. S. Amoah, V. Bezzubik, M. Verhaegen, and G. Vdovin, “Aliasing, coherence, and resolution in a lensless holographic microscope,” Opt. Lett. 42(12), 2271–2274 (2017). [CrossRef]  

32. Z. Luo, A. Yurt, R. Stahl, A. Lambrechts, V. Reumers, D. Braeken, and L. Lagae, “Pixel super-resolution for lens-free holographic microscopy using deep learning neural networks,” Opt. Express 27(10), 13581–13595 (2019). [CrossRef]  

33. P. Almoro, G. Pedrini, and W. Osten, “Complete wavefront reconstruction using sequential intensity measurements of a volume speckle field,” Appl. Opt. 45(34), 8596–8605 (2006). [CrossRef]  

34. G. Pedrini, W. Osten, and Y. Zhang, “Wave-front reconstruction from a sequence of interferograms recorded at different planes,” Opt. Lett. 30(8), 833–835 (2005). [CrossRef]  

35. J. J. Stamnes, “Focusing of two-dimensional waves,” J. Opt. Soc. Am. 71(1), 15–31 (1981). [CrossRef]  

36. Y. Sun, S. Duthaler, and B. J. Nelson, “Autofocusing in computer microscopy: selecting the optimal focus algorithm,” Microsc. Res. Tech. 65(3), 139–149 (2004). [CrossRef]  

37. M. Rostykus and C. Moser, “Compact lensless off-axis transmission digital holographic microscope,” Opt. Express 25(14), 16652–16659 (2017). [CrossRef]  

38. S. Ebrahimi, M. Dashtdar, A. Anand, and B. Javidi, “Common-path lensless digital holographic microscope employing a Fresnel biprism,” Optics and Lasers in Engineering 128, 106014 (2020). [CrossRef]  

Data availability

Data underlying the results presented in this paper are not publicly available at this time but may be obtained from the authors upon reasonable request.

Cited By

Optica participates in Crossref's Cited-By Linking service. Citing articles from Optica Publishing Group journals and other participating publishers are listed here.

Alert me when this article is cited.


Figures (5)

Fig. 1.
Fig. 1. Schematic diagram of a LDH system and the data acquisition process of sub-pixel shifting super-resolution technique. A low-frequency component ${S_L}$ and a corresponding high-frequency component ${S_H}$ are added (aliased) together to form ${L_1}$ and ${L_2}$ within two sub-pixel shifted holograms. ${S_L}$ and ${S_H}$ can be retrieved with simple arithmetic operations from ${L_1}$ and ${L_2}$.
Fig. 2.
Fig. 2. Four under-sampled images ${l_{1\sim 4}}$ captured with an LDH system by shifting a detector array by four shifting vectors $\{{{{({{d_x},{d_y}} )}_t}} \}$. ${L_{1\sim 4}}({m{\xi_0},n{\eta_0}} )$ are four aliased frequency-domain data calculated by 2D-FFT. $S({m{\xi_0},n{\eta_0}} )$ represents unaliased frequency-domain data
Fig. 3.
Fig. 3. A flowchart of sub-pixel shifting super-resolution technique and multi-height phase retrieval method to obtain complex optical field. (Step 1): We measure four holograms by shifting a sample by half-pixel distances along the x and y axes. One super-resolution image is calculated by using Eq. (5) with an noniterative method. (Step 2): We shifted the sample along the z-axis by a known distance (300 μm in our case), and repeat the process described in (Step 1) to obtain another super-resolution image. We repeat the process of (Step 2) to obtain the third super-resolution image. We calculate the complex optical field of an image by using the MPR method. Finally, the best-focused super-resolution image is obtained by numerical focusing.
Fig. 4.
Fig. 4. Comparison of images obtained with three different imaging methods. (a) A typical out-of-focus low-resolution hologram obtained with our LDH setup. Images in (b1) and (c1) are enlarged views of a low-resolution hologram at its best focused plane, which are calculated by using numerical focusing. Images in (b2) and (c2) are obtained with our proposed sub-pixel super-resolution method. Images shown in (b3) and (c3) are obtained with conventional optical microscopy with an NA = 0. 25 objective.
Fig. 5.
Fig. 5. Resolution enhancements in sub-pixel shifting super-resolution method for phase objects. (a) An out-of-focus low resolution hologram for 2-µm polystyrene beads obtained with our LDH system. Images in (b1) and (c1) are enlarged views of a low-resolution image at its best focused plane. (b1) amplitude map and (c1) phase map. Images in (b2) and (c2) are obtained with our proposed sub-pixel super-resolution method. (b2) amplitude map (c2) phase map

Equations (7)

Equations on this page are rendered with MathJax. Learn more.

( 1 1 e i π d p e i π d k N p e i π d p e i π d k N p ) ( S L S H ) = ( L 1 L 2 )
( S L S H ) = 1 2 ( 1 i 1 i ) ( L 1 L 2 e i π k 2 N )
L t ( m ξ 0 , n η 0 ) = α = 0 α = 1 β = 0 β = 1 S ( m ξ 0 2 α ξ n y q u i s t , n η 0 2 β η N y q u i s t ) ϕ α , β , t .
{ ϕ α , β , t } = { e x p ( i π ( ( ( 1 ) α d x , t p x + d x , t m M p x ) + ( ( 1 ) β d y , t p y + d y , t n N p y ) ) ) }
( L 1 ( m ξ 0 , n η 0 ) L 2 ( m ξ 0 , n η 0 ) e i π m 2 M L 3 ( m ξ 0 , n η 0 ) e i π n 2 N L 4 ( m ξ 0 , n η 0 ) e i π 2 ( m M + n N ) ) = ( 1 1 1 1 i i i i i i i i 1 1 1 1 ) ( S ( m ξ 0 , n η 0 ) S ( m ξ 0 , n η 0 2 ξ N y q u i s t , n η 0 ) S ( m ξ 0 , n η 0 2 η N y q u i s t ) S ( m ξ 0 2 ξ N y q u i s t , n η 0 2 η N y q u i s t ) ) .
G ( z , ξ , η ) = e x p ( 2 π z i λ 1 λ 2 ξ 2 λ 2 η 2 ) .
R ( z , x , y ) = F 1 [ F { H ( 0 , x , y ) } × G ( z , ξ , η ) ] .
Select as filters


Select Topics Cancel
© Copyright 2024 | Optica Publishing Group. All rights reserved, including rights for text and data mining and training of artificial technologies or similar technologies.