Open Access
Add to BibSonomy
Abstract
This paper presents the optical design of a digitally switchable multi-focal microlens array which can be used to extend the depth of field in integral imaging systems. The proposed switchable multi-focal microlens array consists of a customized freeform multi-focal microlens array (MLA) and a programmable spatial light modulator. By switching among the different optical powers of the switchable multi-focal MLA, an integral imaging system can render or capture a 3D scene at a large depth range around several central depth planes. We demonstrate the design considerations for a dual-focal microlens array with a primary and secondary focal lengths of 4mm and 4.06mm, respectively. We further validated the design by providing both interferometric measurements of the surface profiles and image contrast and resolution tests of a manufactured MLA prototype.
© 2021 Optical Society of America under the terms of the OSA Open Access Publishing Agreement
1. Introduction
Due to its ability to render correct depth cues with relatively low hardware requirements, integral imaging (InI) has become one of the most promising techniques for capturing and rendering 3D information. Integral photography was originally proposed by Gabriel. M. Lippmann in 1908, where a microlens array (MLA) was placed in front of an image sensor to capture the 4D light field of a 3D scene [1]. Based on the similar principle, an InI-based display can be made to render a 3D scene by combining an MLA or pinhole array with a display device [2–5]. By rendering different perspective images through an MLA to create directional viewing samples, a 3D scene with full parallax can be reconstructed. Generally, a resolution-priority InI (RP-InI) display system which prioritizes spatial resolution over depth of field (DOF) is preferred rather than a depth-priority InI (DP-InI) display system which obtains a 3D reconstruction with constant but low spatial resolution over a large DOF [6]. However, one of the major limitations to a RP-InI system is that it is only capable of rendering 3D scenes in a narrow DOF around its central depth plane (CDP) due to the diffraction and defocus effects of the MLA. When a reconstructed scene significantly deviates from the depth of the CDP, the resolution and contrast of the 3D scene will degrade.
Different types of multi-focal optical elements have been incorporated before or after an InI unit to overcome the DOF limitation in RP-InI systems. For instance, multiple microlens arrays with different focal lengths were interlaced in plenoptic cameras to capture scenes at two or more different focal depths [7]. This method, however, has a low efficiency since only a portion of the micro images are in focus while others are out of focus due to the interlaced lens structure. Alternatively, a tunable optical element with electrically addressable optical power may be incorporated with a conventional MLA to extend the DOF of InI-based systems [8–10]. For example, Shen et al. presented an InI display system with an extended DOF by inserting a polarized bifocal liquid crystal lens [8] or a focus-tunable lens [9] before an MLA. More recently, Huang and Hua demonstrated a high performance InI-based light field head-mounted display (LF-HMD) in which they utilized a tunable lens after the micro InI display unit to extend the DOF [10]. Incorporating an extra tunable element usually requires an optical relay system which will increases the system volume and weight substantially. Another solution is to utilize a focus-tunable MLA such as liquid crystal-based MLA and electrowetting liquid MLA [11–14] to achieve an electrically tunable functionality without introducing extra relay system. These tunable MLAs can offer focus tunability, but the optical performances are inadequate for achieving high optical resolution and dynamic speed.
Recently, we demonstrated a new hybrid solution to extend the DOF of an InI-based HMD system without sacrificing the spatial resolution or increasing the system form factor by incorporating a digitally switchable multi-focal MLA [15]. Figure 1 shows the schematic layout of the proposed system, which consists of a multi-CDP micro-InI unit and an eyepiece. By switching the optical power of the multi-focal MLA, the micro-InI unit can generate several CDPs at different depths in a time multiplexed fashion. When different sets of 2D elemental images are rendered on a micro-display in synchronization with the focus switching of the multi-focal MLA, intermediate virtual objects located at different depth zones can be rendered around these CDPs. These intermediate virtual objects are then magnified by an eyepiece to render the light field of a virtual 3D scene with an extended depth of field, which can be observed at the viewing window. The ray paths of different CDPs are illustrated by the sets of rays in different colors in Fig. 1, where red rays illustrate the rendering of objects in the far depth range through the center zone of the MLA and the yellow rays illustrates the rendering of objects in the near depth range with respect to the viewing window.
The key element in the proposed depth enhancement method shown in Fig. 1 is the digitally switchable multi-focal MLA. In this paper, we present the optical design considerations, fabrication, and testing of a digitally switchable multi-focal MLA which may find a broad range of applications in InI-based imaging and display systems. As explained in Section 2, a multi-focal MLA consists of a custom-designed freeform MLA and a high-speed programmable shutter array (PSA). Each of the microlens produces multiple distinct foci through a set of concentric zones. The PSA has a customized array pattern corresponding to the zone distribution of the microlens to control the light transmission of the different foci. By attaching the PSA to the multi-focal freeform MLA and operating in a time-multiplexing fashion, a digitally switchable multi-focal MLA can be achieved at high switching speed with a compact form factor. Section 3 presents the optical design considerations and results of a dual-focal freeform MLA with a primary and secondary focal lengths of 4mm and 4.06mm., while Section 4 discusses the considerations of a PSA according to the design of the MLA. Section 5 presents the fabrication and experimental results of a dual-focal MLA prototype.
2. Concept
The method for achieving a digitally switchable multi-focal MLA is based on aperture division concept we proposed in [16]. As illustrated in Fig. 2(a), a digitally switchable MLA can be achieved by combining a multi-focal freeform MLA together with a PSA. The freeform MLA is designed in a way such that one surface of each lenslet is divided into multiple concentric zones along its radial direction, as shown in Fig. 2(b). On every lenslet, each surface segment of the concentric zones has its own optical power, yielding different focal points, such as F1, F2, F3, etc., and is represented by different colors. The segments with the same color on different lenslets share the same focal length. To digitally switch between the foci, a PSA need to be attached next to the segmented surfaces of the MLA to control the light transmission through the different focal zones of each lenslet. Figure 2(c) shows the design of the shutter elements on the PSA, which are arranged in the same array pattern as the lenslets in MLA. The aperture of each shutter element is further divided into multiple concentric zones illustrated by the same color schemes as the corresponding focal zones of the lenslets, and each of the colored concentric zones can be switched on or off to modulate the light transmission. To mitigate potential crosstalk between the adjacent focal zones as well as between adjacent lenslets, a narrow black circular zone is inserted between two adjacent color zones and a narrow black square zone is inserted between the adjacent shutter aperture. By synchronously programming the on and off states of the same colored areas on the shutter array to modulate the light transmission through the different regions of each lenslet, the focus of the MLA can be rapidly switched among the multiple foci.
Fig. 2. Schematics of a digitally switchable MLA. (a)Working Principal. (b) Freeform MLA. (c) Programmable Shutter Array.
Implementing a digitally switchable multi-focal MLA as described above requires several critical considerations. For the design of the MLA, the first consideration is the aperture shape, dimension of each lenslet as well as the aperture shapes, dimensions, and focal lengths of the focal zones within a lenslet. Consider the design of an MLA creating N discrete foci, F1, F2, … and FN, respectively, arranged in order along the optical axis. Without loss of generality, we assume the center zone of the lenslet corresponds to the nearest focal point, F1, and yields a primary focal length of f1 which determines the optical magnification of the InI-unit. The focal lengths of the other segmented zones, f2, f3… and fN, respectively, increase as their corresponding radial distances, ri, increase, where ri is the radial distance of the outer edge of the ith focal zone, i=1, 2,…, N as illustrated in Fig. 2(b). Considering the square shape of typical micro-display pixels, the aperture shape of each lenslet is preferably square, with a uniform side length of a, to maximally utilize the pixels and avoid gaps, while a circular aperture shape is preferred for the other non-edge focal zones. Choosing the aperture dimension, a, of the lenslet as well as the radial dimensions and focal lengths for the focal zones involves much more complex considerations than choosing the aperture shapes.
Firstly, the diffraction-limited performance of each focal zone is determined by the size and shape of the aperture as well as the corresponding focal length. The aperture function of each focal zone is expressed as
According to Fresnel diffraction theory, the point spread function for a given aperture can be obtained by applying the Fourier transform to the aperture function defined in Eq. (1).
To ensure diffraction-limited optical performance for the MLA design, the dimensions, the image distances, and the number of focal zones need to be chosen such that the cut-off frequencies determined by the Eq. (2) for the focal zones are substantially higher than the Nyquist frequency determined by the pixel size of a micro-display or an imaging sensor. For instance, consider a micro-display with a pixel size of 5um and a corresponding Nyquist frequency of 100 lps/mm. To ensure the diffraction-limited modulation transfer function (MTF) is larger than 0.4 at the Nyquist frequency, the image space working f-number should be smaller than F/9 for a circular aperture. At a wavelength of 550 nm, the cut-off frequency of an F/9 system is about 202 lps/mm, which is about twice of the Nyquist frequency of the micro-display.
Secondly, the aperture dimension affects the view density in InI-based light field systems. The view density, σview, which characterizes the number of distinct views per unit area projected on the viewing window of the display, is one important property that determines the viewing experience and visual artifacts of an InI-based display. Generally, a large view density generally leads an increase of DOF, improvement of longitudinal resolution, a reduction of accommodation error, and reduction of defocus-blurring artifacts at the cost of spatial resolution and image contrast. The relationship between the aperture dimension, a, and the view density can be explicitly expressed as
Thirdly, the footprint of each elemental view at the viewing window for each focal zone also depends on the shape and size of the aperture. As illustrated in the Fig. 1, di,I and di,O stands for the inner and outer footprint diameter of the ray bundle from a single pixel on the micro-display projected on the viewing window through the display optics. By setting the viewing window at the back focal length of the eyepiece, they can be expressed as Eq. (4) and (5):
Another critical consideration in the MLA design is to ensure that the effective numerical aperture (NA) of each focal zone remains nearly constant such that the light throughput of different focal zones is nearly constant, and the image brightness resulted from different focal zones remain constant. Therefore, the radial aperture dimensions of the different focal zones in each lenslet should approximately satisfy the constraint expressed as [16]
Besides the light throughput consideration, two types of crosstalks can potentially occur. The first type is the crosstalk between different zones within a lenslet, which is similar to the effect in a multifocal lens and has been thoroughly discussed in [16]. The second type is the crosstalk between adjacent lenslets due to the array structure. This can be addressed by optimizing the aperture dimensions of the MLA and PSA together. Therefore, the exact radial aperture of each focal zone needs to be further optimized along with the dimensions of the aperture zones and black zones of the PSA to minimize the crosstalk while ensuring nearly the same light throughput through the different focal zones [16].
To achieve high optical performance and easy fabrication for the MLA design, only one of the surfaces in each lenset is divided into concentric zones with different optical power, while the other surface is optimized as a single aspherical profile with the same focal power for different focal zones. Each circular segment of the divided surface is optimized as an aspherical surface up to 6th order terms carrying slightly different optical power from the central zone, and the corresponding surface sag can be described as
A programmable shutter array plays a critical role in affecting the overall optical performance of a multi-focal MLA. It may be adapted from existing spatial light modulator (SLM) technologies, such as a transmissive active matrix liquid crystal display (LCD) device, a reflective liquid-crystal-on-silicon (LCoS) device, or a reflective digital mirror device (DMD). The adoption of a transmissive LCD would be the most straightforward and compact solution as it can be directly attached to the MLA and programmed into an arbitrary aperture pattern to modulate the light transmission. As discussed in detail in [16], however, the low fill factor of the pixels in typical transmissive LCDs causes significant diffraction artifacts and compromises optical performance. As demonstrated in [16], customized LC-based shutters formed by large LC cells matching the aperture shape and dimensions of the lenslet focal zones can be made to overcome the diffraction limitation of pixelated LCDs. A reflective LCoS or DMD are subject to much less diffraction problems due to their high pixel fill factors, but their reflective nature prevents them to be placed adjacent to the MLA and an optical relay system may be necessary to create a conjugate image of the shutter device on the MLA. Therefore, a customized non-pixelated shutter array is preferred than an LCD or a LCoS. More discussion will be provided in the Section 3 regarding to the optimization of the aperture and black zone dimensions for crosstalk reduction and the experimental result is demonstrated in Section 4.
3. Design of dual-focal freeform MLA
Based on the tradeoff relationships among the DOF, spatial resolution, and view density of InI-based LF displays discussed in [17,18], we further analyzed and concluded that a dual-focal MLA is expected to be adequate for extending the DOF of a LF-HMD system to a depth range of three diopters, while maintaining high angular resolution and offering a view density of 2 × 2 distinct views over a 3-mm eye pupil. According to the method described in Sect. 2, we designed a dual-focal freeform MLA to demonstrate the concept. Table 1 listed the key specifications of the MLA. The micro-display to be used as an image source is a high-resolution OLED display from Sony with a pixel pitch of 8um and a total of 1920 by 1080 pixels, which establishes the Nyquist frequency of 62.5lps/mm in the display space. The focal lengths for the inner and outer foci are 4mm and 4.06mm, respectively. By placing the micro-display at 5.33mm away from the MLA, the dual focal MLA yields transverse magnifications of 3 and 3.19 through the inner and outer foci, respectively, and an axial separation of 1mm between the two CDPs. Consequently, the Nyquist frequencies in the image space are 20.83 and 19.59 lps/mm for the inner and outer foci, respectively. To ensure high optical performance for the micro-InI unit, we set the threshold value for the diffraction-limited modulation transfer function (MTF) to be 50% or higher at the image space Nyquist frequencies on the CDPs for both foci. Based on this requirement, by applying Eq. (2), the aperture size, a, of each lenslet is set as 1.2mm, and the diameter of the aperture for the inner focal zone is set as 0.7mm. The corresponding apertures for the shutter elements are determined accordingly after balancing the throughput and crosstalk effects for the two foci which will be further discussed in Section 4.
Table 1. Design parameters
Figure 3(a) shows the optical layout with ray tracing for a dual-focal lenslet of the MLA. The lenslet images the object of the same distance onto two different image planes depending on which part of the lenslet aperture is turned on, near focal for the inner zone and far focal for the outer zone. Figure 3(b) shows the aperture layouts for the lenslet and shutter, respectively. Note that the inner focal zone has a circular aperture while the aperture of outer focus is rectangular with a circular occlusion in the center. As shown in Fig. 3(b), the shutter aperture for the outer focus, D2,O, is 0.98mm, smaller than the 1.2mm pitch of microlens to eliminate the crosstalk as well as the unpredictable fabrication surface error between adjacent micro lenses.
Fig. 3. Design of dual-focal MLA. (a) Layout of the two foci of one microlens. (b) The aperture on microlens and shutter.
Figures 4(a) and 4(b) plots the MTF curves for the inner and outer focal zones, respectively, where the corresponding images are set at their best focus with an axial separation of 1mm. Both foci almost reach the diffraction-limited optical performance. At the image space Nyquist frequencies for both foci, which are 20.83 lps/mm and 19.59 lps/mm, respectively, the MTF of the inner focus is above 0.55, while the MTF of the outer focus is above 0.4 except the corner field. We further examined the through-focus performance of the two focal zones and the combined through-focus MTF curves for frequency of 21 lps/mm are plotted in Fig. 4(c). For the inner focal zone, as shown by the red dash curve, the MTF drops down to about 20% from its peak value of 0.6 when the image plane is defocused by ±1.2mm from its best focus. For the outer focal zone plotted by the green dash curve, the MTF drops down to about 20% from its peak value of 0.5 when the image plane is defocused by ±0.7mm from its best focus. Illustrated as the purple line, by combining the two focal zones, the combined MTF of the lenslet can maintain 40% or above over a depth range of 2mm, which corresponds to a depth range of 3 diopters in the visual space when it is combined with an eyepiece with a focal length of 25.8 mm as the viewing optics.
Fig. 4. MTF performance of the dual-focal lenslet: (a) iner focal zone; (b) outer focal zone; and (c) through-focus performance.
To account for unexpected fabrication errors, we performed tolerance analysis for the lenslet by applying tolerance values equivalent to commercial-grade fabrication tolerance in which +/-2.5um surface sag is allowed. Figures 5(a) and 5(b) plot the cumulative probability as a function of MTF values for the Nyquist frequency of 62.5 lps/mm in the object space for the inner and outer focal zones, respectively. The result demonstrated that the dual-focal microlens can maintain high performance and high rate of yield even with large amount of manufacture errors.
Fig. 5. Tolerance analysis of the dual focal lenslet: (a) inner focal zone and (b) outer focal zone.
4. Design of the shutter array
As discussed in Section 2, tradeoffs among factors such as diffraction limit, light throughput and crosstalk mitigation have to be made to determine the aperture size of each focal zone of the lenslet as well as the dimensions of the shutter apertures. The dimensions of the shutter aperture elements are primarily driven by the apertures of the different focal zones, but further optimization is required to mitigate crosstalk and light throughput balance. As shown in Fig. 3(b), the aperture of each shutter element for the dual-focal lenslet is divided into four sub-regions. The two primary controllable regions highlighted in green and yellow which can be turned on or off to control light transmission through the inner and outer focal zones, respectively. The two secondary narrow black regions, one between the two focal regions and one surrounding the edge of outer focal region, are utilized to minimize crosstalks between the two focal zones and between adjacent lenslets, respectively. The dimensions of these aperture regions are optimized by analyzing the light throughput through each focal zone and the amount of potential crosstalk for different aperture values.
The dimension of the square-shape black region is mainly determined by the side length, a, of the lenslet aperture, the lenslet fill factor or gap due to fabrication tooling limits, and the amount of crosstalk between adjacent lenslets. Although the side length, a, of each micro lens is set to be 1.2mm as discussed in Section 3, we expect a much smaller effective lenslet aperture or lenslet fill factor due to inevitable tooling gaps between adjacent lenslets during fabrication process. Based on values obtained from optics manufactures, we assume a typical fill factor of 85%, which yields an effective area of 1.04 mm2 out of 1.44mm2. The effective area is determined by the shutter aperture for the outer focus, D2,O, which should be 0.98mm to achieve a fill factor of 85%.
The dimensions of the two primary shutter regions as well as the circular black region are determined by the outer radius, D1,O, of the inner shutter and the inner radius, D2,I, of the outer shutter. To obtain optimal combinations of these aperture dimensions, we analyzed both the light throughputs of each focal zone and the crosstalk between focal zones. Figure 6(a) illustrate the light throughput ratio of the outer focus to the inner focus. To ensure the diffraction-limited performance of both focus, the outer focus always has a larger throughput than the inner focus, as illustrated in the figure. The throughput difference also needs to be balanced to mitigate crosstalk. Figures 6(b) and (c) plot the relationships between the percentage of crosstalk and aperture dimensions for the largest object field, which corresponds to the edge pixel along the diagonal direction in each elemental image, for the inner and outer focal zones, respectively. To limit the crosstalk, we set a criterion that the maximum amount of crosstalk across the fields should be less than 20%. The red rectangle in Fig. 6(b) illustrates the region where that the crosstalk exceeds the criterion for inner focus, while the purple one in Fig. 6(c) is for the outer focus. Correspondingly, the aperture dimensions should not be chosen from the same region in Fig. 6(a). After balancing the brightness, crosstalk and optical performance, the parameters of the shutter array are settled to the values in Table 1.
Fig. 6. (a) Throughput ratio between the outer focus and inner focus. Crosstalk analysis for (b) inner focus and (c) outer focus.
Besides the dimensions of the shutter apertures, another critical consideration is the construction of the two primary shutter apertures, which may be formed by pixelated elements. When a device with pixelated elements is adopted as a programmable shutter array, diffraction effect needs to be accounted for. According to the theoretical diffraction analysis of a pixelated aperture discussed in detail in [16], the intensity of non-zero diffraction order is directly related to the fill factor of the pixel elements when the size of the shutter aperture is relatively much larger than the pitch of the pixels. In the case of a multifocal MLA, where the aperture size of each shutter region is much smaller than a regular lens, we need to model the diffraction effects induced by both limiting dimensions of the shutter apertures and the pixelated aperture structure. We compared the diffraction effects induced by three different PSA technologies–a customized non-pixelated shutter array in which each aperture region is single, large LC cell matching the aperture shape and dimension of the corresponding shutter region, a commercially available transmissive LCD with a pixel fill factor of 0.49, and a commercially available reflective LCoS device with a pixel fill factor of 0.81. The pixel pitch is 50um and 20um for the LCD and LCoS, respectively. The focal length of the MLA is set as 4mm and the diameter of the center aperture is 0.7 mm. An on-axis object is located at 5.33mm and the aperture is assumed to be co-located with the microlens. The optical cutoff frequency in the image space for such a system is 87.5 lps/mm. The point spread functions of the three conditions are shown as Fig. 7(a) to 7(c), while the corresponding MTFs are plotted in Fig. 7(d). While a single peak is observed in the PSF plot for the non-pixelated shutter design several peaks are observed from the PSF plots for the aperture array with pixelated aperture. The number of peaks is determined by the number of pixels within the aperture region and the peak-to-valley value as well as the contrast between the peaks are affected by the fill factor. A low fill-factor LCD yields multiple diffraction peaks with much higher intensity values than those by a high fill-factor LCoS device. The MTF plots for the pixelated devices also show significant periodical fluctuations and such artifacts are expected to induce significant image contrast degradation due to the diffraction of its low fill-factor pixelated structure. The high fill-factor reflective LCoS is subject to much less diffraction problems, but it reflective nature requires an optical relay system to create an optical conjugate on the MLA plane, which will significantly increase the system volume. A customized non-pixelated shutter array is preferred over an LCD or an LCoS.
Fig. 7. Diffraction effect of different type of shutter array. (a)Non-pixelated customized. (b)LCD with a fill factor of 0.49. (c)LCoS with a fill factor of 0.81. (d)MTF comparison.
5. Prototype fabrication and experimental results
Figure 8(a) shows a photograph of the freeform MLA fabricated via a precision diamond turning process. The surface sag and the surface profile are measured using a Zygo Newview 8300 white light interferometer microscope, shown as Fig. 8(b). To evaluate the fabrication deviation from the design, we compared the average of several microlens measurements to the theoretical design profile. Considering the alignment error in measurement, the sag of any point on the design surface can be expressed as
Fig. 8. (a) Photograph of fabricated dual-focal MLA prototype. (b) Interferometric measurement of the freeform surface on a lenslet. (c) Fitting error of the measured surface in (b); and (d) Surface sag deviation from the theoretical design.
Due to the missing data at the edge and corner shown in Fig. 8(b), only an effective area of 1.08mm*1.08mm is used for the fitting. The average error is about 0.1um and the error at each point is illustrated as Fig. 8(c). After removing the tilt and yaw terms introduced by the measurement, we can get the profile deviation from theoretical one by comparing the estimated and design parameters and the surface profile error is shown as Fig. 8(d). We can find the inner focal zone has a relatively more accurate profile, while the outer focal zone has a larger deviation. The maximum absolute surface sag difference is about 2.5um, which is in the range of our design tolerance.
Figures 9(a) and (b) are the captured images of an USAF resolution target through the inner and outer focal zones of a single lenslet, respectively. In this experiment, the optical layout of the test setup is the same as that shown in Fig. 3(a), where the USAF resolution target was used as an object to replace the micro-display, and a Blackfly S USB3 sensor without imaging lens was placed at the image plane to directly capture the image through the MLA. The pixel size of the sensor is 3.45 µm. From the zoomed-in inset view of both figures, we can see that both focal zones are capable of at least resolving Group 7 Element 4 in the images. The corresponding spatial frequency of the Group 7 Element 4 target is about 181 lp/mm, which covers the resolution range of most of the state-of-the-art imaging sensors and displays. Because of the different magnifications, the outer focus will yield slightly larger image size than that of the inner focus as shown in the figures.
We further experimentally validated the diffraction effects of the three different types of shutter array methods discussed in Section 4 and the results are shown as Fig. 10(a) to (c) for a printed non-pixelated aperture, a transmissive LCD, and a reflective LCoS imaged by a relay optics, respectively. Besides the use of different types of shutter array, the experimental setup is similar to the one for capturing Fig. 9 except that the object is a grating target displayed on a monochrome OLED display with a pixel size of 4.7um. To ensure the brightness levels of the images are comparable, the pictures were captured with different gains, 20dB, 35dB, and 30dB for the printed aperture, the LCD, and the LCoS, respectively. Figure 10(d) plots the average intensity profile for a small area marked by a red rectangle in their corresponding images in Fig. 10(a) through (c). The diffraction effects of the low fill-factor pixels in a commercial LCD significantly degrades the image, while the diffraction effects induced by the pixels of a LCoS is hardly noticeable.
Fig. 10. Captured images through a lenslet with different shutter array (a) Printed aperture on transparency captured at 20 dB gain. (b) LCD captured at 35 dB gain. (c) LCoS captured at 30 dB gain (d) Cross section of the captured images.
In the final experiment, we set up a dual-CDP InI based Light field display shown in Fig. 11 to demonstrate the application of the dual-focal MLA for depth enhancement. While the custom-designed programmable shutter array shown in Fig. 3(b) is under fabrication, we used aperture array patterns printed on a transparency as switchable apertures for temporary experiments. A high-resolution micro-display with a pixel pitch of 4.7um is used as the image source and an eyepiece with a focal length of 30 mm is used as the viewing optics to magnify the miniature light field objects rendered by the dual-focal micro-InI unit. An iris is placed at the viewing window located at the back focal point of the eyepiece to simulate the entrance pupil of a human eye. The same Blackfly S USB3 sensor together with a 25 mm focal length camera lens are used to capture images through the entrance pupil, which overall yields an angular resolution of 0.47 arc min per pixel. The two virtual CDPs rendered by the same are located at 1 and 1.82 diopters to the viewing window, respectively, which were calibrated with physical targets.
To demonstrate the optical performance, a virtual 3D target scene consisting of five depth planes located at 2.5, 1.82, 1.5, 1 and 0.5 diopters away from the viewing window was created. On each depth plane three groups of Snellen letter ‘E’s with different spatial resolutions (1, 2, and 3 pixels for the individual strokes or gaps of the letters) and orientations (horizontal and vertical) were rendered. Figure 12 shows the captured images of the reconstructed virtual 3D target while different focal zones were enabled as the camera is focused at different depths. It can be observed that only the targets, located at the same depth as the focal depth of the camera, are well converged and have the highest resolution. When the camera is focused at the depth of 1 diopter, the elemental images for rendering the target located at 1D converge well through both the inner and outer focal zones, as shown in Fig. 12(a) and 12(b), respectively. However, when compare the figures in 12(a) and 12(b), the rendered target demonstrates much higher contrast and sharper image through the inner focal zone of the MLA than the same target through the outer focal zone. On the other hand, when the camera is focused at the depth of 1.82 diopters, the elemental images for rendering the target located at 1.82 diopters converge well through both the inner and outer focal zones shown in Fig. 12(c) and 12(d), respectively. When comparing these two images, however, the rendered 1.82D target through the inner focus, as shown in Fig. 12(c), demonstrates lower contrast and blurrier image than the same target through the outer focal zone shown in Fig. 12(d). Through the outer focal zone, the smallest Snellen letters on the 1.82D target can be well resolved. This result confirm that the outer focal zone provides a higher resolution than the inner focal zone for rendering objects in the near depth range and the inner focal zone provide a higher resolution for rendering object in the far depth range, offering an extended DOF for an InI-based light field display or imaging system.
Fig. 12. Captured images of virtual 3D targets: (a) Inner focus enabled while the camera is focused at 1 diopter; (b) Outer focus enabled while the camera is focused at 1 diopter. (c) Inner focus enabled while camera is focused at 1.82 diopters. (d) Outer focus enabled while the camera is focused at 1.82 diopters.
6. Conclusion
In this work, we presented the optical design of a novel switchable multi-focal MLA. We demonstrate the design considerations for a dual-focal MLA with a primary and secondary focal lengths of 4 mm and 4.06 mm, respectively. We further validated the design by providing both interferometric measurements of the surface profiles and image contrast and resolution tests of a manufactured MLA prototype. We also demonstrated the application of the proposed device for extending the depth of field in an integral imaging based light field display system. Besides its applications in integral imaging systems, the proposed device may also find a range of applications in other fields.
Funding
Intel Corporation.
Disclosures
Dr. Hong Hua has a disclosed financial interest in Magic Leap Inc. The terms of this arrangement have been properly disclosed to The University of Arizona and reviewed by the Institutional Review Committee in accordance with its conflict of interest policies.
Data availability
Data underlying the results presented in this paper are not publicly available at this time but may be obtained from the authors upon reasonable request.
References
1. G. Lippmann, “Epreuves réversibles photographies integrals,” Comptes-Rendus Academie des Sci. 146, 446–451 (1908).
2. X. Xiao, B. Javidi, M. Martinez-Corral, and A. Stern, “Advances in three-dimensional integral imaging: sensing, display, and applications,” Appl. Opt. 52(4), 546–560 (2013). [CrossRef]
3. H. Hua and B. Javidi, “A 3D integral imaging optical see-through head-mounted display,” Opt. Express 22(11), 13484–13491 (2014). [CrossRef]
4. W. Song, Y. Wang, D. Cheng, and Y. Liu, “Light field head-mounted display with correct focus cue using micro structure array,” Chin. Opt. Lett. 12(6), 060010 (2014). [CrossRef]
5. D. Lanman and D. Luebke, “Near-eye light field displays,” ACM Trans. Graph. 32(6), 1–10 (2013). [CrossRef]
6. F. Jin, J. Jang, and B. Javidi, “Effects of device resolution on three-dimensional integral imaging,” Opt. Lett. 29(12), 1345–1347 (2004). [CrossRef]
7. J. S. Jang and B. Javidi, “Large depth-of-focus time-multiplexed three-dimensional integral imaging by use of lenslets with nonuniform focal lengths and aperturesizes,” Opt. Lett. 28(20), 1924–1926 (2003). [CrossRef]
8. X. Shen, Y. J. Wang, H. S. Chen, X. Xiao, Y. H. Lin, and B. Javidi, “Extended depth-of-focus 3D micro integral imaging display using a bifocal liquid crystal lens,” Opt. Lett. 40(4), 538–541 (2015). [CrossRef]
9. X. Shen and B. Javidi, “Large depth of focus dynamic micro integral imaging for optical see-through augmented reality display using a focus-tunable lens,” Appl. Opt. 57(7), B184–B189 (2018). [CrossRef]
10. H. Huang and H. Hua, “High-performance integral-imaging-based light field augmented reality display using freeform optics,” Opt. Express 26(13), 17578–90 (2018). [CrossRef]
11. Y. Lei, Q. Tong, X. Zhang, H. Sang, A. Ji, and C. Xie, “An electrically tunable plenoptic camera using a liquid crystal microlens array,” Rev. Sci. Instrum. 86(5), 053101 (2015). [CrossRef]
12. J. F. Algorri, N. Bennis, V. Urruchi, P. Morawiak, J. M. Sánchez-Pena, and L. R. Jaroszewicz, “Tunable liquid crystal multifocal microlens array,” Sci. Rep. 7(1), 17318 (2017). [CrossRef]
13. C. J. Kim, M. Chang, M. Lee, J. Kim, and Y. H. Won, “Depth plane adaptive integral imaging using a varifocal liquid lens array,” Appl. Opt. 54(10), 2565–2571 (2015). [CrossRef]
14. C. Kim, D. Shin, G. Koo, and Y. H. Won, “Fabrication of an electrowetting liquid microlens array for a focus tunable integral imaging system,” Opt. Lett. 45(2), 511–514 (2020). [CrossRef]
15. X. Wang and H. Hong, “Depth-enhanced head-mounted light field displays based on integral imaging,” Opt. Lett. 46(5), 985–988 (2021). [CrossRef]
16. X. Wang, Y. Qin, H. Hua, Y. H. Lee, and S. T. Wu, “Digitally switchable multi-focal lens using freeform optics,” Opt. Express 26(8), 11007–11017 (2018). [CrossRef]
17. J. H. Park, S. W. Min, S. Jung, and B. Lee, “Analysis of viewing parameters for two display methods based on integral photography,” Appl. Opt. 40(29), 5217–5232 (2001). [CrossRef]
18. H. Huang and H. Hua, “Systematic characterization and optimization of 3D light field displays,” Opt. Express 25(16), 18508–18525 (2017). [CrossRef]
