Expand this Topic clickable element to expand a topic
Skip to content
Optica Publishing Group

A flipping-free 3D integral imaging display using a twice-imaging lens array

Open Access Open Access

Abstract

Integral imaging is a promising 3D visualization technique for reconstructing 3D medical scenes to enhance medical analysis and diagnosis. However, the use of lens arrays inevitably introduces flipped images beyond the field of view, which cannot reproduce the correct parallax relation. To avoid the flipping effect in optical reconstruction, a twice-imaging lens array based integral display is presented. The proposed lens arrangement, which consists of a light-controlling lens array, a field lens array and an imaging lens array, allows the light rays from each elemental image only pass through its corresponding lens unit. The lens arrangement is optimized with geometrical optics method, and the proposed display system is experimentally demonstrated. A full-parallax 3D medical scene showing continuous viewpoint information without flipping is reconstructed in 45° field of view.

© 2019 Optical Society of America under the terms of the OSA Open Access Publishing Agreement

1. Introduction

Integral imaging is a promising three-dimensional (3D) visualization technique for record and display of full-color 3D scenes with continuous viewpoints [17]. It consists of a pickup process and a display process. In the pickup stage, the 3D information about 3D secenes is captured as an elemental image array (EIA) by a lens array and a pickup device like a charge coupled device or by computer generated (CG) image mapping technique [810]. In the display stage, a lens array reproduces the 3D object by imaging the EIA within a certain field of view (FOV). With integral imaging, both horizontally and vertically directional information can be perceived without use of any supplementary glasses, which makes it attractive in many application fields, especially in medical imaging. With the rapid evolution of acquiring 3D imaging medical data at high resolution, great efforts have been devoted to visualizing 3D medical data with integral imaging technique [1115]. However, flipping effect is one of crucial drawbacks preventing the integral imaging from commercial medical applications. For a conventional integral imaging display, a viewer can observe the correct reconstructed 3D image within the FOV as each EI is imaged by the lens located right in front of it. When the viewer is out of the FOV, the flipping effect is observed. This is because the light rays from an elemental image enter its adjacent lenses and form flipped 3D images beyond the FOV. The edge side of the 3D image is therefore not integrated properly and flipped images reproduce the incorrect parallax relation, and thus the physicians’ judgment is severely affected.

To improve understanding for 3D scenes and visual experience of viewers, especially in medical scenarios, it is necessary to eliminate the flipping effect in integral imaging reconstruction. Many researchers have focused on resolving this problem, and eye tracking technique or modification of the optical structure is used. Eye tracking based integral imaging systems [16,17] obtain the viewer’s position and display different EIA based on the tracking information. In order to avoid the flipping effect, Wang et al. proposed a partially-overlapped viewing zone based integral imaging system [17] to generate a corresponding adaptive EIA based on viewers’ position and reconstruct the 3D scene with the obtained EIA in real time. Unfortunately, this kind of device can only track one viewer’s position so that it cannot satisfy multiple viewers. Some novel optical arrangements for the integral imaging were reported. A curved lens array with tilted physical barrier was implemented so the light rays from an elemental image only entered its corresponding lens [18]. However, this structure was bulky and required complex manufacturing. Cuenca et al. proposed a multiple-axis telecentric relay system [19], where three camera lenses with pinholes were used to accomplish the telecentricity condition in three directions. Thus, three sets of perpendicular and oblique barriers were generated that prevented the light rays from passing through the adjacent lenses. Unfortunately, the structure suffered from low light efficiency with the use of pinholes. Another integral imaging display using multi-Köhler illumination was developed, which provided the parallel illumination pattern for the elemental images [20]. The angle of divergence of light rays was constrained to the aperture of the corresponding microlens and thus eliminates flipped image. However, the display’s scale was constrained and it showed a limited FOV.

Here, a twice-imaging lens array is proposed to implement an integral imaging display without flipped images. It allows the light rays from each elemental image (EI) only pass through its corresponding lens unit to avoid flipping effect. In Section 2, we analyze the flipping effect in integral imaging and explain the principle of the twice-imaging lens array. Section 3 is devoted to the description of the design and optimization of the proposed lens arrangement. Finally, we present the performance of the proposed system with simulations as well as experiments in Section 5.

2. Principle

2.1 Flipping effect in integral imaging

The typical structure of the integral imaging display which can optically reconstruct 3D scenes is depicted in Fig. 1. The system consists of a liquid crystal display (LCD), a lens array, and a holographic functional screen (HFS) [21]. The LCD is responsible for delivering the EIA captured with CG integral imaging based on the backward ray-tracing [10]. The lens array is placed in front of the LCD to project the EIs onto the central depth plane (CDP). The distance between the lens array and the LCD is greater than the focal length of the lens to form a real 3D image, known as resolution priority integral imaging [22]. The HFS on the CDP is then introduced to diffuse light ray bundles and recompose the light rays. Points ${A_{n - 1}}$ , ${A_n}$ and ${A_{n + 1}}$ represent the pixels in the $E{I_{n - 1}}$ , $E{I_n}$ and $E{I_{n + 1}}$ . The light rays from ${A_{n - 1}}$ , ${A_n}$ and ${A_{n + 1}}$ are refracted by the corresponding lenses and then intersect in the reconstructed point ${A^{\prime}}$ . In this way, a full-parallax 3D image would be reconstructed in a specific FOV when the light rays from each EI are refracted by the corresponding lens. According to paraxial optics theory, the FOV is given by

$$\omega = 2\arctan ({{\raise0.7ex\hbox{$p$} \!\mathord{\left/ {\vphantom {p {2g}}} \right.}\!\lower0.7ex\hbox{${2g}$}}} ), $$
where $p$ is the size of the EI on the LCD, and $g$ refers to the distance between the LCD and the lens array. Beyond the FOV, note however that instead of refracted by its corresponding lens, the light rays emanating from the $E{I_n}$ are refracted by the adjacent lenses (${L_{n - 1}}$ , ${L_{n + 1}}$ , etc.) and form flipped images as shown in Fig. 1. The light rays from ${A_{n - 1}}$ , ${A_n}$ and ${A_{n + 1}}$ pass through the adjacent lenses and then reconstruct the flipped points ${B^{\prime}}$ and ${C^{\prime}}$ . It is evident that they exhibit the incorrect direction information compared with the point ${A^{\prime}}$ . Viewing positions ${P_2}$ , ${P_3}$ and ${P_4}$ are located within the FOV while ${P_1}$ and ${P_5}$ are out of the FOV. Figure 2 shows the experimental results of the 3D image observed on the viewing positions ${P_1} \sim {P_5}$ . To obtain these experimental results, a conventional integral imaging display with a single lens array is used, each lens having a focal length of 16.5 mm and 10 mm of lens diameter. It provides the FOV of 45 degree, the maximum lateral off-axis distance of 1.2 metre and the separation from the observer of 1.5 metre. On the central viewing position ${P_3}$ , the front side of the cube is observed. If the viewer moves from ${P_3}$ to the left, the left side of the cube (green area) is presented on ${P_2}$ . When the viewer continues to move to the left, the flipped image starts to appear so that the edge side of the 3D image is not integrated properly. On ${P_1}$ , the flipped image is fully observed and the right side of the cube (red area) is presented, showing the incorrect parallax relation. From ${P_2}$ to ${P_1}$ , the parallax of the reconstructed 3D image flips because of the existence of the flipped images. This phenomenon is known as flipping effect. The same effect happens again when the observer moves from ${P_3}$ to ${P_5}$ . Due to the flipping effect, the improper parallax relationship of the 3D object is reconstructed beyond the FOV, which can cause severely adverse effects on viewer’s judgment, especially in medical scenarios.

 figure: Fig. 1.

Fig. 1. The flipping effect in the conventional integral imaging display.

Download Full Size | PDF

 figure: Fig. 2.

Fig. 2. The experimental results of the 3D image observed on the viewing positions P1-P5 (see Visualization 1).

Download Full Size | PDF

2.2 Analysis of twice-imaging lens array

The twice-imaging lens array is proposed to avoid the flipping effect as illustrated in Fig. 3. The optical arrangement involves three major components, namely, a light-controlling lens array (LCLA), a field lens array (FLA), and an imaging lens array (ILA). The LCLA serves to firstly image the EIA to form the intermediate images onto the FLA plane and constrain the light rays. Here, the size of the intermediate image is supposed to equal to the pitch of the FL. The FLA acts as a set of field stops to block the stray light (dotted line in Fig. 3(a)) originally emerging from the adjacent EIs. Besides, it adjusts the direction of emergent rays from the LCLA to match the ILA. The ILA is used to image the intermediate image array and thus the 3D medical image is reconstructed in a FOV of 45 degree. Figure 3(b) illustrates the ray tracing of $E{I_n}$ with the proposed lens arrangement.${S_1}$ represents the upper edge pixel of $E{I_n}$ and ${S_2}$ represents the lower edge pixel of $E{I_n}$ .$Ra{y_1}$ denotes the upper marginal ray from ${S_1}$ and $Ra{y_2}$ denotes the lower marginal ray from ${S_2}$ . As the size of intermediate image of $E{I_n}$ is simply the size of $F{L_n}$ , $Ra{y_1}$ passes through the lower border of $F{L_n}$ to finally reconstruct the image point $S_1^{\prime}$ while $Ra{y_2}$ passes through the upper border of $F{L_n}$ to reproduce the image point $S_2^{\prime}$ . By following the path of $Ra{y_1}$ and $Ra{y_2}$ , the ray-passing region where the light rays can pass through the nth lens unit is determined (grey region in Fig. 3(b)). In other words, the proposed method allows the light rays from an EI only pass through its corresponding lens unit. When $Ra{y_3}$ , a marginal ray (purple dashed line in Fig. 3(b)) from the lower edge pixel ${S_3}$ of $E{I_{n - 1}}$ , is refracted by the $LCL{A_n}$ , it exceeds the lower edge of $F{L_n}$ and thus is blocked. Compared Fig. 3(a) and Fig. 1, we can clearly see that the twice-imaging lens array prevents the rays from passing through adjacent lenses so that the flipping effect is eliminated.

 figure: Fig. 3.

Fig. 3. (a) The principle of the proposed twice-imaging lens array. (b) The ray tracing of EIn with the twice-imaging lens array.

Download Full Size | PDF

To calculate the paraxial parameters of the proposed lens arrangement for its starting point in lens design, a single unit of the proposed lens arrangement is depicted in Fig. 4. Suppose the center of the LCL represents the origin of the coordinate system. A ray starting from marginal pixel $D$ with coordinates $(0,{y_1}, - {d_1})$ intersects the LCL at a height ${y_2}$ , FL at ${y_3}$ and IL at ${y_4}$ in sequence. After passing through the lens arrangement, the ray strikes the HFS at a height ${y_5}$ from the optical axis. ${d_1},{d_2},{d_3}$ and ${d_4}$ denote the distance between the corresponding components. ${u_1},{u_2},{u_3},{u_4}$ and ${u_5}$ represent the ray angles of each component. In the paraxial approximation, a Jones transfer matrix of the optical system arrangement along the light propagation paths can be expressed as follows:

$$\left( {\begin{matrix} {{y_5}}\\ {{u_5}} \end{matrix}} \right) = \left( {\begin{matrix}1 &{{d_4}}\\ 0 &1 \end{matrix}} \right)\left( {\begin{matrix} 1 &0\\ { - {\raise0.7ex\hbox{$1$} \!\mathord{\left/ {\vphantom {1 {{f_{LCL}}}}} \right.}\!\lower0.7ex\hbox{${{f_{LCL}}}$}}} &1 \end{matrix}} \right)\left( {\begin{matrix} 1 &{{d_3}}\\ 0 &1 \end{matrix}} \right)\left( {\begin{matrix}1 &0\\ { - {\raise0.7ex\hbox{$1$} \!\mathord{\left/ {\vphantom {1 {{f_{FL}}}}} \right.}\!\lower0.7ex\hbox{${{f_{FL}}}$}}} &1 \end{matrix}} \right)\left( {\begin{matrix} 1 &{{d_2}}\\ 0 &1 \end{matrix}} \right)\left( {\begin{matrix} 1 &0\\ { - {\raise0.7ex\hbox{$1$} \!\mathord{\left/ {\vphantom {1 {{f_{IL}}}}} \right.}\!\lower0.7ex\hbox{${{f_{IL}}}$}}} &1 \end{matrix}} \right)\left( {\begin{matrix} 1 &{{d_1}}\\ 0 &1 \end{matrix}} \right)\left( {\begin{matrix} {{y_1}}\\ {{u_1}} \end{matrix}} \right),$$
where ${f_{LCL}},\;{f_{FL}}$ and ${f_{IL}}$ denote the focal lengths of the LCL, FL and IL, respectively. For simplicity, the optical system is divided into three parts (red rectangular, green rectangular and purple rectangular in Fig. 4), whose Jones transfer matrix are in turn given by
$$\left( {\begin{array}{{c}} {{u_3}}\\ {{y_3}} \end{array}} \right) = \left( {\begin{array}{{cc}} 1 &{{d_2}}\\ 0 &1 \end{array}} \right)\left( {\begin{array}{{cc}} 1 &0\\ { - {\raise0.7ex\hbox{$1$} \!\mathord{\left/ {\vphantom {1 {{f_{IL}}}}} \right.}\!\lower0.7ex\hbox{${{f_{IL}}}$}}} &1 \end{array}} \right)\left( {\begin{array}{{cc}} 1 &{{d_1}}\\ 0 &1 \end{array}} \right)\left( {\begin{array}{{c}} {{u_1}}\\ {{y_1}} \end{array}} \right), $$
$$\left( {\begin{array}{{c}} {{u_4}}\\ {{y_4}} \end{array}} \right) = \left( {\begin{array}{{cc}} 1 &{{d_3}}\\ 0 &1 \end{array}} \right)\left( {\begin{array}{{cc}} 1 &0\\ { - {\raise0.7ex\hbox{$1$} \!\mathord{\left/ {\vphantom {1 {{f_{FL}}}}} \right.}\!\lower0.7ex\hbox{${{f_{FL}}}$}}} &1 \end{array}} \right)\left( {\begin{array}{{cc}} 1 &{{d_2}}\\ 0 &1 \end{array}} \right)\left( {\begin{array}{{c}} {{u_2}}\\ {{y_2}} \end{array}} \right), $$
$$\left( {\begin{array}{{c}} {{u_5}}\\ {{y_5}} \end{array}} \right) = \left( {\begin{array}{{cc}} 1 &{{d_4}}\\ 0 &1 \end{array}} \right)\left( {\begin{array}{{cc}} 1 &0\\ { - {\raise0.7ex\hbox{$1$} \!\mathord{\left/ {\vphantom {1 {{f_{LCL}}}}} \right.}\!\lower0.7ex\hbox{${{f_{LCL}}}$}}} &1 \end{array}} \right)\left( {\begin{array}{{cc}} 1 &{{d_3}}\\ 0 &1 \end{array}} \right)\left( {\begin{array}{{c}} {{u_3}}\\ {{y_3}} \end{array}} \right). $$
From the above equations, the focal lengths of each lens component are determined.

 figure: Fig. 4.

Fig. 4. A single unit of the proposed lens arrangement.

Download Full Size | PDF

Note that the image quality of the proposed lens arrangement is a factor that strongly influences the performance of the integral imaging display [23]. To evaluate the image quality of a lens, the usual method is to trace a large number of rays from a point source over the entrance pupil of the lens, and then plot a spot diagram of the points at which these rays pierce the image plane [24]. The RMS radius of the plotted spot diagram is used to decide whether the lens system is sufficiently well-designed for application. In our case, the RMS radius ${P_T}$ of the twice-imaging lens array yields that

$${P_T} \le {P_L}{M_{LCL}}{M_{FL}}{M_{IL}}, $$
where ${P_L}$ denotes the pixel size of the LCD, ${M_{LCL}},{M_{FL}}$ and ${M_{IL}}$ represent the longitudinal magnification of the LCL, the FL and the IL.

3. Design and optimization of the twice-imaging lens array

3.1 The design of the imaging lens

The IL of 10 mm pitch is designed to achieve a FOV of 45 degree ( $2{u_4} = 2{u_5} = {45^ \circ }$ ) and an image distance of 200 mm ( ${d_4} = 200mm$ ). The pitch of the FL is chosen as 9 mm ( $2{y_3} = 9mm$ ). According to Eq. (5), the focal length of IL is calculated as 9.7 mm ( ${f_{IL}} = 9.7mm$ ) and ${d_3}$ is 10.2 mm, respectively. Due to the reversibility of light principle, the HFS plane is regarded as the object plane and the FL plane is used as the image plane in designing the IL. In order to meet the image quality requirements, the spot radius of the IL on FL plane should be smaller than 101 $\mu m$ (the image size of LCD pixel on FL plane is 101 $\mu m$ ). However, it is hard to satisfy both small F number and small spot radius with only one lens in lens design. Consequently, a Cooke triplet is adopted as the starting point. In addition, surface curvatures and the distance between surfaces are chosen as optimization variables. The fields of 0°, 6.75°, 11.25°, 15.75° and 22.5° are selected. Since the pixels on the LCD in the proposed integral imaging system are made up of separate red, green and blue elements, the wavelengths of 486 nm (F light), 588 nm (d light), and 656 nm (C light) are selected for optimization to suppress chromatic aberrations. In addition, aspherical surfaces are used for a better lens performance. Figure 5(a) demonstrates the optical layout of the optimized IL, which is composed of two positive lenses and one negative lens. To evaluate the image quality, the spot RMS radius for five fields are calculated. As shown in Fig. 5(b), the image quality at wavelengths of 486 nm, 588 nm and 656 nm is evaluated. In the spot diagrams, symbols of blue cross, green square and red triangle are the positions in the image plane for the traced rays of 486 nm, 588 nm and 656 nm wavelengths respectively. The RMS radius of 0° field, 6.75° field, 11.25° field, 15.75° field and 22.5° field is 10.3 $\mu m$ , 10.4 $\mu m$ , 12.5 $\mu m$ , 17.4 $\mu m$ and 23.9 $\mu m$ , respectively. The optimized ILA satisfies the requirements of the image quality and the total spectral range for its application is from 486 nm to 656 nm.

 figure: Fig. 5.

Fig. 5. (a) The optical layout of the designed IL, (b) spot diagrams of the IL in 5 fields (RMS radius: 0° field: 10.3um; 6.75° field: 10.4um; 11.25° field: 12.5um; 15.75° field: 17.4um; 22.5° field: 23.9um).

Download Full Size | PDF

3.2 The design of the light-controlling lens and the field lens

The LCL is designed to image the EI and form an intermediate image on the FL plane so that the intermediate image acts as a new EI with limited angle of divergence. In order to block the light rays from other EIs and allow all the light rays from the corresponding EI, the size of the intermediate image should be the same as the pitch of the FL. Therefore, the magnification of the LCL is calculated as ${9 \mathord{\left/ {\vphantom {9 {12}}} \right.} {12}} = 0.75$ (the size of the EI is 12 mm and the size of the intermediate image is 9 mm). The distance between the EI and the LCL is set as 30 mm ( ${d_1} = 30mm$ ), and the distance between the LCL and the FL is 22.5 mm ( ${d_2} = 22.5mm$ ). According to Eq. (3), the focal length of the LCL is obtained as 12.9 mm ( ${f_{LCL}} = 12.9mm$ ). To obtain sufficient freedom in lens design, three lenses are chosen as the starting structure and aspherical surfaces are used. The designed LCL optical path diagram is shown in Fig. 6(a). It is straightforward to notice that the emergent rays would exceed the pitch of the IL. Thus, the FL is introduced to adjust the angle of emergence from the LCL. In addition, it acts as a field stop to block stray light. Here, a single Fresnel lens is used and its focal length is calculated as 7.0 mm ( ${f_{FL}} = 7.0mm$ ) based on Eq. (4). Figure 6(b) shows the optical path after introducing the FL. Note that when light rays emerge from the EI, pass from the ICL to the FL, the emergent rays can pass through the IL.

 figure: Fig. 6.

Fig. 6. (a) Optical path of the light rays refracted by the LCL, (b) optical path of the light rays for the LCL, the FL and the IL.

Download Full Size | PDF

3.3 The optimization of the proposed lens arrangement

To evaluate the image quality of the designed lens unit, the spot diagrams are shown in Fig. 7 (a). In each diagram, the dimensions of the plotted spot diagram are 20000 $\mu m$ on a side, and blue, green, and red symbols are the positions in the image plane for the traced rays of 486 nm, 588 nm, 656 nm wavelengths. Five field positions of 0°, 6.75°, 11.25°, 15.75°, 22.5° are selected and the RMS radii in these fields are 3249 $\mu m$ , 3358 $\mu m$ , 3439 $\mu m$ , 3461 $\mu m$ and 3145 $\mu m$ , respectively. According to Eq. (6), the RMS radius of the designed lens unit should satisfy that ${P_T} \le 2020\mu m$ ( ${P_L} = 135\mu m,{M_{LCL}} = 0.75,{M_{FL}} = 1,{M_{IL}} = 20$ ). From Fig. 7(a), it can be seen that the RMS radii in five field positions are greater than the 2020 $\mu m$ . Therefore, the LCL, the FL and the IL are combined altogether for optimization to enhance the imaging performance of the lens arrangement. The final specifications are summarized in Table 1. Additionally, more detailed parameters for defining aspherical surfaces are disclosed in Table 2. The spot diagrams of the final lens arrangement are shown in Fig. 7(b). The RMS radii of 0° field, 6.75° field, 11.25° field, 15.75° field and 22.5° field are reduced to 580 $\mu m$ , 480 $\mu m$ , 399 $\mu m$ , 413 $\mu m$ and 545 $\mu m$ , verifying that the aberrations are well suppressed with the optimized lens arrangement. The total spectral range for the application is from 486 nm to 656 nm.

 figure: Fig. 7.

Fig. 7. Spot diagrams of the designed lens arrangement before the final optimization (RMS spot size: 0° field: 3249um; 6.75° field: 3358um; 11.25° field: 3439um; 15.75° field: 3461um; 22.5° field: 3145um). (b) Spot diagrams of the designed lens arrangement after the final optimization (RMS spot size: 0° field: 580um; 6.75° field: 480um; 11.25° field: 400um; 15.75° field: 413um; 22.5° field: 545um).

Download Full Size | PDF

Tables Icon

Table 1. Specifications used for the lens arrangement

Tables Icon

Table 2. Specifications for the aspherical surfaces

4. Simulation and experiment

In this section, both simulation and experiment are implemented to show that the flipping effect is removed with the twice-imaging lens array. For comparison, the ray tracing of the conventional lens array and that of the proposed lens arrangement are simulated, as shown in Fig. 8. In Fig. 8(a), a single lens array with focal length of 16.5 mm and FOV of 45 degree are used. We can see that the light rays emerging from the EI pass through the adjacent lenses and thus cause the flipping effect. In addition, the image spots of the single lens are quite large, which degrades the image quality. In Fig. 8(b), the twice-imaging lens array blocks stray light from other EIs so as to avoid flipped images. Meanwhile, it provides a FOV of 45 degree and high image quality.

 figure: Fig. 8.

Fig. 8. Optical path for (a) the conventional lens array and (b) the twice-imaging lens array.

Download Full Size | PDF

The experimental implementation of the proposed system is shown in Fig. 9. It is composed of a LCD panel, the proposed lens arrays and a HFS. The 23.6-inch 4 K LCD panel with pixel pitch 135 $\mu m$ is used to load the EIA, which is captured by CG integral imaging method [10]. Each EI is composed of 88 ${\times}$ 88 pixels in a matric format from 88 ${\times}$ 88 viewpoint perspectives. The proposed lens arrays are manufactured and used in the system, each lens array containing 46 ${\times}$ 27 lens units. The 23.6-inch HFS is used to diffuse the light rays. The gap between the LCD panel and the lens array is 30 mm and the distance between the lens array and the HFS is 200 mm. The proposed system provides the maximum FOV of 45 degree. The maximum lateral off-axis distance is 1.2 metre and the separation from the observer is 1.5 metre. Figure 10 shows the experimental results of the 3D cube with the proposed display. We can clearly see the reconstructed 3D cube and the relative 3D position relation of different parts within the FOV of 45 degree (on viewing position ${P_2}$ , ${P_3}$ , ${P_4}$ ). More importantly, the flipped images are eliminated beyond the FOV (on viewing position ${P_1}$ , ${P_5}$ ) by the twice-imaging lens array, compared to Fig. 2.

 figure: Fig. 9.

Fig. 9. Optical configuration of the proposed display.

Download Full Size | PDF

 figure: Fig. 10.

Fig. 10. A 3D cube without flipping effect is reconstructed using the twice-imaging lens array (see Visualization 2).

Download Full Size | PDF

One of important applications of the integral imaging display without flipping effect is for medical analysis and diagnosis. Figures 11(a) and (b) shows the EIAs for human heart and sternum. To better evaluate the performance of the proposed lens array, Fig. 12 presents the reconstructed 3D medical images by the use of the conventional lens array (Figs. 12(a) and 12(c)) and of the proposed lens arrangement (Figs. 12(b) and 12(d)). In Figs. 12(a) and 12(c), the 3D medical scenes with the flipping effect are observed when the viewer moves horizontally from −30° to 30°. At the position of −22.5°, the images taken from this angle is the jump between the correct reconstructed image and the flipped image. These flipped images are more noticeable for larger angles, and the whole flipped image can be seen at −30°. The same effect happens when the observer moves towards positive angular deviations. In the proposed display, when the observer moves horizontally within the FOV (−22.5°, 22.5°), the different perspectives of the 3D medical scene can be seen continuously, as shown in Figs. 12 (b) and (d). Once out of the FOV, flipping doesn’t occur at all, as expected. Additionally, the display reproduces 3D medical scenes more clearly in the FOV of 45 degree.

 figure: Fig. 11.

Fig. 11. (a) Coded EIA for human heart, and (b) coded EIA for sternum.

Download Full Size | PDF

 figure: Fig. 12.

Fig. 12. The reconstructed 3D image of human heart with (a) the conventional lens array (see Visualization 3) and (b) the proposed lens arrangement (see Visualization 4). The reconstructed 3D image of sternum with (c) the conventional lens array (see Visualization 5) and (d) the proposed lens arrangement (see Visualization 6).

Download Full Size | PDF

To further verify the validity of the proposed method, the optical reconstruction of a real 3D object with an optically recorded EIA is implemented. Figure 13(a) shows a medical anatomical skull model. The proposed lens array is used in the optical pickup process and the optically recorded EIA is obtained as shown in Fig. 13(b). With the proposed display, the different perspectives of the reconstructed skull model can be seen on different viewing positions, as depicted in Fig. 14. Note that flipped images do not appear beyond the FOV of 45 degree, which implies that the flipping effect is eliminated with twice-imaging lens array.

 figure: Fig. 13.

Fig. 13. (a) The medical anatomical skull model, and (b) optically recorded EIA of the skull model.

Download Full Size | PDF

 figure: Fig. 14.

Fig. 14. The reconstructed 3D image of the skull model observed from different viewing positions.

Download Full Size | PDF

5. Conclusion

An integral imaging system with twice-imaging lens array is presented to avoid the flipping effect. Unlike other methods, the lens array has been designed to firstly image the EIA to produce the intermediate images, block the undesirable lights from other EIs and then further image the intermediate images into a 3D scene. The aberrations are considered in lens design and the enhanced spot diagrams are presented. In the experiment, full-parallax 3D medical images showing continuous viewpoint and volumetric information are reproduced in the 45-degree FOV, and flipped images are eliminated out of the FOV. With the increasing command for 3D medical imaging, the proposed display can find many potential applications in the related area.

Funding

National Key Research and Development Program (2017YFB1002900); National Natural Science Foundation of China (61575025); State Key Laboratory of Information Photonics and Optical Communications (IPOC2017ZZ02).

Disclosures

The authors declare no conflicts of interest.

References

1. G. Lippmann, “La photographie integrale,” C. R. Acad. Sci. 146, 446–451 (1908).

2. J. Hong, Y. Kim, H. Choi, J. Hahn, J. Park, H. Kim, S. Min, N. Chen, and B. Lee, “Three-dimensional display technologies of recent interest: principles, status, and issues [Invited],” Appl. Opt. 50(34), H87–H115 (2011). [CrossRef]  

3. M. M. Corral and B. Javidi, “Fundamentals of 3D imaging and displays: a tutorial on integral imaging, light-field, and plenoptic systems,” Adv. Opt. Photonics 10(3), 512–566 (2018). [CrossRef]  

4. X. Dong, L. Ai, and E. Kim, “Integral imaging-based large-scale full-color 3-D display of holographic data by using a commercial LCD panel,” Opt. Express 24(4), 3638–3651 (2016). [CrossRef]  

5. C. Ji, C. Luo, H. Deng, D. Li, and Q. Wang, “Tilted elemental image array generation method for moiré-reduced computer generated integral imaging display,” Opt. Express 21(17), 19816–19824 (2013). [CrossRef]  

6. J. Jung, S. Park, Y. Kim, and B. Lee, “Integral imaging using a color filter pinhole array on a display panel,” Opt. Express 20(17), 18744–18756 (2012). [CrossRef]  

7. B. Tavakoli, M. Daneshpanah, B. Javidi, and E. Watson, “Performance of 3D integral imaging with position uncertainty,” Opt. Express 15(19), 11889–11902 (2007). [CrossRef]  

8. Y. Igarishi, H. Murata, and M. Ueda, “3D display system using a computer generated integral photograph,” Jpn. J. Appl. Phys. 17(9), 1683–1684 (1978). [CrossRef]  

9. D. Kim, M. Erdenebat, K. Kwon, J. Jeong, J. Lee, K. Kim, N. Kim, and K. Yoo, “Real-time 3D display system based on computer-generated integral imaging technique using enhanced ISPP for hexagonal lens array,” Appl. Opt. 52(34), 8411–8418 (2013). [CrossRef]  

10. S. Xing, X. Sang, X. Yu, D. Chen, B. Pang, X. Gao, S. Yang, Y. Guan, B. Yan, J. Yuan, and K. Wang, “High-efficient computer-generated integral imaging based on the backward ray-tracing technique and optical reconstruction,” Opt. Express 25(1), 330–338 (2017). [CrossRef]  

11. M. Agus, F. Bettio, A. Giachetti, E. Gobbetti, J. Antonio, I. Guitian, F. Marton, J. Nilsson, and G. Pintore, “An interactive 3D medical visualization system based on a light field display,” Vis. Comput. 25(9), 883–893 (2009). [CrossRef]  

12. H. Liao, T. Inomata, I. Sakuma, and T. Dohi, “3-D augmented reality for MRI-guided surgery using integral videography autostereoscopic image overlay,” IEEE Trans. Biomed. Eng. 57(6), 1476–1486 (2010). [CrossRef]  

13. Y. Liang, B. Fang, Y. Wang, L. Wu, and P. Chen, “Design of an interactive 3D medical visualization system,” in Proceedings of IEEE Conference on Wavelet Analysis and Pattern Recognition (IEEE, 2011), pp. 60–64.

14. D. Zhao, B. Su, G. Chen, and H. Liao, “360 degree viewable floating autostereoscopic display using integral photography and multiple semitransparent mirrors,” Opt. Express 23(8), 9812–9823 (2015). [CrossRef]  

15. X. Sang, X. Gao, X. Yu, S. Xing, Y. Li, and Y. Wu, “Interactive floating full-parallax digital three-dimensional light-field display based on wavefront recomposing,” Opt. Express 26(7), 8883–8889 (2018). [CrossRef]  

16. G. Park, J. Hong, Y. Kim, and B. Lee, “Enhancement of viewing angle and viewing distance in integral imaging by head tracking,” in Digital Holography and Three-Dimensional Imaging, OSA Technical Digest (Optical Society of America, 2009), DWB27.

17. Z. Xiong, Q. Wang, S. Li, H. Deng, and C. Ji, “Partially-overlapped viewing zone based integral imaging system with super wide viewing angle,” Opt. Express 22(19), 22268–22277 (2014). [CrossRef]  

18. Y. Kim, J. Park, H. Choi, S. Jung, S. Min, and B. Lee, “Viewing-angle-enhanced integral imaging system using a curved lens array,” Opt. Express 12(3), 421–429 (2004). [CrossRef]  

19. R. M. Cuenca, H. Navarro, G. Saavedra, B. Javidi, and M. Martinez-Corral, “Enhanced viewing-angle integral imaging by multipleaxis telecentric relay system,” Opt. Express 15(24), 16255–16260 (2007). [CrossRef]  

20. Á. Tolosa, R. M. Cuenca, H. Navarro, G. Saavedra, M. M. Corral, B. Javidi, and A. Pons, “Enhanced field-of-view integral imaging display using multi-Köhler illumination,” Opt. Express 22(26), 31853–31863 (2014). [CrossRef]  

21. C. Yu, J. Yuan, F. Fan, C. Jiang, S. Choi, X. Sang, C. Lin, and D. Xu, “The modulation function and realizing method of holographic functional screen,” Opt. Express 18(26), 27820–27826 (2010). [CrossRef]  

22. J. S. Jang and B. Javidi, “Large depth-of-focus time-multiplexed three-dimensional integral imaging by use of lenslets with nonuniform focal lengths and aperture sizes,” Opt. Lett. 28(20), 1924–1926 (2003). [CrossRef]  

23. A. Karimzadeh, “Integral imaging system optical design with aberration consideration,” Appl. Opt. 54(7), 1765–1769 (2015). [CrossRef]  

24. R. Kingslake and R. Johnson, Lens Design Fundamentals (Academic, 2009), Chap. 1.

Supplementary Material (6)

NameDescription
Visualization 1       A reconstructed 3D cube showing the flipping effect by the conventional integral imaging display.
Visualization 2       A reconstructed 3D cube without flipping by the twice-imaging lens array based integral imaging display.
Visualization 3       The single lens array based integral imaging display results of a 3D image of human heart with the flipping effect.
Visualization 4       The twice-imaging lens array based integral imaging display results of a 3D image of human heart without flipping.
Visualization 5       The single lens array based integral imaging display results of a 3D image of sternum with the flipping effect.
Visualization 6       The twice-imaging lens array based integral imaging display results of a 3D image of sternum without flipping.

Cited By

Optica participates in Crossref's Cited-By Linking service. Citing articles from Optica Publishing Group journals and other participating publishers are listed here.

Alert me when this article is cited.


Figures (14)

Fig. 1.
Fig. 1. The flipping effect in the conventional integral imaging display.
Fig. 2.
Fig. 2. The experimental results of the 3D image observed on the viewing positions P1-P5 (see Visualization 1).
Fig. 3.
Fig. 3. (a) The principle of the proposed twice-imaging lens array. (b) The ray tracing of EIn with the twice-imaging lens array.
Fig. 4.
Fig. 4. A single unit of the proposed lens arrangement.
Fig. 5.
Fig. 5. (a) The optical layout of the designed IL, (b) spot diagrams of the IL in 5 fields (RMS radius: 0° field: 10.3um; 6.75° field: 10.4um; 11.25° field: 12.5um; 15.75° field: 17.4um; 22.5° field: 23.9um).
Fig. 6.
Fig. 6. (a) Optical path of the light rays refracted by the LCL, (b) optical path of the light rays for the LCL, the FL and the IL.
Fig. 7.
Fig. 7. Spot diagrams of the designed lens arrangement before the final optimization (RMS spot size: 0° field: 3249um; 6.75° field: 3358um; 11.25° field: 3439um; 15.75° field: 3461um; 22.5° field: 3145um). (b) Spot diagrams of the designed lens arrangement after the final optimization (RMS spot size: 0° field: 580um; 6.75° field: 480um; 11.25° field: 400um; 15.75° field: 413um; 22.5° field: 545um).
Fig. 8.
Fig. 8. Optical path for (a) the conventional lens array and (b) the twice-imaging lens array.
Fig. 9.
Fig. 9. Optical configuration of the proposed display.
Fig. 10.
Fig. 10. A 3D cube without flipping effect is reconstructed using the twice-imaging lens array (see Visualization 2).
Fig. 11.
Fig. 11. (a) Coded EIA for human heart, and (b) coded EIA for sternum.
Fig. 12.
Fig. 12. The reconstructed 3D image of human heart with (a) the conventional lens array (see Visualization 3) and (b) the proposed lens arrangement (see Visualization 4). The reconstructed 3D image of sternum with (c) the conventional lens array (see Visualization 5) and (d) the proposed lens arrangement (see Visualization 6).
Fig. 13.
Fig. 13. (a) The medical anatomical skull model, and (b) optically recorded EIA of the skull model.
Fig. 14.
Fig. 14. The reconstructed 3D image of the skull model observed from different viewing positions.

Tables (2)

Tables Icon

Table 1. Specifications used for the lens arrangement

Tables Icon

Table 2. Specifications for the aspherical surfaces

Equations (6)

Equations on this page are rendered with MathJax. Learn more.

ω = 2 arctan ( p / p 2 g 2 g ) ,
( y 5 u 5 ) = ( 1 d 4 0 1 ) ( 1 0 1 / 1 f L C L f L C L 1 ) ( 1 d 3 0 1 ) ( 1 0 1 / 1 f F L f F L 1 ) ( 1 d 2 0 1 ) ( 1 0 1 / 1 f I L f I L 1 ) ( 1 d 1 0 1 ) ( y 1 u 1 ) ,
( u 3 y 3 ) = ( 1 d 2 0 1 ) ( 1 0 1 / 1 f I L f I L 1 ) ( 1 d 1 0 1 ) ( u 1 y 1 ) ,
( u 4 y 4 ) = ( 1 d 3 0 1 ) ( 1 0 1 / 1 f F L f F L 1 ) ( 1 d 2 0 1 ) ( u 2 y 2 ) ,
( u 5 y 5 ) = ( 1 d 4 0 1 ) ( 1 0 1 / 1 f L C L f L C L 1 ) ( 1 d 3 0 1 ) ( u 3 y 3 ) .
P T P L M L C L M F L M I L ,
Select as filters


Select Topics Cancel
© Copyright 2024 | Optica Publishing Group. All rights reserved, including rights for text and data mining and training of artificial technologies or similar technologies.