Open Access
Add to BibSonomy
Abstract
A retroreflector array improves the visual quality of three-dimensional (3D) image in the pinhole-type integral imaging display. Conventionally, the limited size of the apertures of the pinhole array restricts the fill factor of the pixelated 3D images. We propose a system with a retroreflector, which consists of the array of corner cubes that increases the fill factor of the pixel by the diffraction. The pixel spreading model is developed by an equivalent corner cube structure. The simulation of pixel spreading by the Fraunhofer diffraction agrees well with experimental results, and proves the effectiveness of the proposed method to improve the visual quality while not sacrificing the viewing angle and the depth perception.
© 2017 Optical Society of America
1. Introduction
Integral imaging requires an array of the optical component, such as pinhole and/or lens arrays those array optics restricts directional views depending on the spatial positions of viewer, thus essential to capture the elemental images in image pickup process, as well as image reconstruct process for the integrated 3D display [1–10]. The pinhole and the lens have the complementary characteristics in integral imaging. The pinhole-type integral imaging has an advantage in cost, manufacturing, and in some case dynamic implementation of pinhole arrays. On the other hand the main demerit is the low optical efficiency due to the limited size of the aperture of the pinhole, which causes the low visibility of the reconstructed image in the display process [11, 12]. Due to such fundamental trade-off between aperture size and image quality, quantitative analysis of visibility in regard to the pinhole size are essential. The fill factor in the pinhole-type integral imaging, defined by the ratio of the active display region compared to the elemental area, is a useful figure of merit for the analysis. As previous research indicates, the fill factor can be regarded as the quantitative value of the visibility for each elemental area [13]. In the conventional pinhole-type integral imaging system, the fill factor is related to the size of pinhole array. To maintain the pinhole lens effect, the pinhole size has to be limited to increase pixel fill factor by diffraction effect. However, since the size of the pinhole array linearly scales with the total viewing angle of the reconstructed image, the enhancement of the fill factor through the reduction of the size is only feasible at the expense of the reduction of the total viewing angle [14]. Moreover, the depth expression is also affected by the number of the pixels per the elemental area and the size of the pinhole array.
To overcome limitations and trade-offs, we propose mathematically formulate and experimentally verified an effective usage of retro-reflector arrays to improve the fill factor without the degradations of the 3D effects while not sacrificing viewing angle and the depth expression.
The retroreflector is the array of the corner cube which consists of three mutually perpendicular reflecting surfaces. When an incident ray is projected onto the retroreflector, the ray is reflected three times and redirected back to the incident direction [15]. Therefore, the conjugated image can be reflected if the size of corner cube is small enough [16–18]. We benefit the requirement of such small size corner cube array to improve pixel fill factor by Fraunhofer diffraction by placing with the corner cube array at a distant location from the light source. In this arrangement, the diffraction effect increases the pixel fill factor without increasing the aperture ratio which causes the degradations of 3D viewing characteristics.
In this paper, we propose new pinhole-type integral imaging system using the retroreflector to improve the fill factor without increasing pinhole aperture size. Figure 1 shows the concept of the proposed system, which is the pinhole-type integral imaging system with the beam splitter and the retroreflector. The analysis about the conjugated image of the retroreflector is performed using an equivalent model of the corner cube structure based on the geometrical optics. The improvement of the fill factor in the proposed system is numerically simulated under consideration of the Fraunhofer diffraction effect, and verified by the comparison with the experimental results. The optimum design method of the proposed system is also discussed.
Fig. 1 Schematic diagram of proposed system which consists of pinhole-type integral imaging, a beam splitter and a retroreflector.
2. Principle
2.1 Effective aperture of the retroreflector
The shape of the effective aperture of a retroreflector is determined according to the incident direction. The ray tracing method is used for geometrical model of the retroreflector. There are one real and three imaginary corner cube structures in two-dimensional section as depicted in Fig. 2. For example, when an incoming ray is projected onto the real corner cube structure, the ray is reflected by two times and goes back to the incident direction. The imaginary ray can be drawn by introducing the equivalent relation in geometry, then the real and the imaginary rays make a straight line. In this way, the ray travels through two apertures, or the incident planes. The model, which we call it as an equivalent model, provides the same ray distribution after the retroreflection.
We apply the equivalent model to the retroreflector in three-dimensional space. There are two triangular apertures which are inverted shapes each other as presented in Fig. 3(a). Those apertures shares the same apex point and co-axis. The incident ray encounters an inversion in the vertical and the horizontal direction during the retroreflection. Therefore, the output aperture is also inverted with respect to the input aperture. The gap between two apertures is twice as large as the distance from the front face to the apex point of the retroreflector. The ray which passes through two apertures is the retroreflected ray. The ray, which passes through one of them, is non-retroreflected ray. The overlapped region between the input and output aperture, we define as an effective aperture of the retroreflector. Figure. 3(b) shows the effective aperture when the incoming light is normal to the retroreflector. In the proposed system, we assume that all incidence ray is almost normal of the retroreflector plane because the size of corner cube retroreflector is much smaller than the distance from the pinhole array to the retroreflector. Therefore, the effective aperture is assumed as the hexagonal shape when the Fraunhofer diffraction effect is applied to the proposed system.
Fig. 3 (a) Geometrical analysis of retroreflector in three-dimensional (b) Effective aperture of retroreflector when incoming light is normal to incident plane.
2.2 Fill factor enhancement by the Fraunhofer diffraction by a retroreflector
Definition of the fill factor in the pinhole-type integral imaging is the ratio of the pinhole area to the elemental area. There is a trade-off between the fill factor and the reconstructed three-dimensional image. Although the system with the low fill factor provides a wider viewing angle, the visibility of the system is lower due to the limited size of the aperture. By using the retroreflector, we prove that the system improves the visibility while maintaining the viewing angle and the depth expression. In the proposed system, the Fraunhofer diffraction, which is caused by retroreflector, is the main factor to increase the fill factor of the proposed system. And the diffraction is a reasonable approximation since the corner cube size of the retroreflector is too small compared to the distance from the pinhole array to the retroreflector. The effective aperture depending on the incident direction is discussed in section 2.1 and is also assumed as the hexagonal aperture. The effective aperture of the retroreflector can be divided into six regions in accordance with the reflection sequences.
One corner cube consists of three reflecting surfaces as shown in Fig. 4(a). There are six reflection sequences as presented in Fig. 4(b). The uncoated retroreflector, which is used in the proposed system, employs the total internal reflection due to the difference of refractive index. During three total internal reflections within one corner cube structure of the retroreflector, the phase of the incident wave is shifted because of the total internal reflection [19]. And the phase shift differs depending on the reflection sequence.
At each sequence, which is the field distribution at the image plane by the retroreflector follows as Eq. (1).
where is the field distribution of an incident wave, ,,,is the region number of retroreflector, is the distance from pinhole array to retroreflector,is the total phase shift during the retroreflection at each sequence, and is image plane and effective aperture of retroreflector coordinates, respectively. Based on Eq. (1), the intensity distribution of the Fraunhofer diffraction patterns is shown as Eq. (2).
We simulate to verify the theoretical analysis for the Fraunhofer diffraction with the phase shift and find the tendency depending on the distance from the pinhole array to the retroreflector. The simulation setup consists of the pinhole array, the retroreflector and a beam splitter. The one pinhole size of pinhole array is 0.276 mm and the interval between the pinholes is 1.564 mm. They are chosen with pixel size of the flat display for 3 by 3 and 17 by 17, respectively. The longest width of the effective hexagonal aperture of the retroreflector is 0.056 mm. One pinhole of pinhole array is modeled as incoherent point light sources within the pinhole. Therefore, we simulate the intensity distribution of the Fraunhofer diffraction by the retroreflector. For convenience, we choose the center wavelength of the incident light is 550 nm. And the incident light is unpolarized light. Figure 5 presents simulation results at each distance from the pinhole array to the retroreflector. When the distance from the pinhole array to the retroreflector is enough long compared as the effective aperture of the retroreflector, the diffraction pattern size becomes larger than the pinhole size. In addition, the shape of diffraction pattern is similar to the effective aperture as expected. To evaluate image quality of the proposed system at the gap between the patterns, the two-dimensional intensity distribution of the Fraunhofer diffraction patterns at each distance is simulated as shown in Fig. 6. At the distance is 80 mm, 1st peak of the intensity distribution starts overlapping. The intensity of the overlapped region is larger than 1st peak value of intensity distribution. The larger overlapped region becomes the blurred image. Therefore, it makes the image quality lower. As simulation results indicate, the Fraunhofer diffraction pattern size by the retroreflector increases the fill factor of the pinhole array until overlapping. We can also the optimize pinhole-type integral imaging with the various pinhole size by using the relation between the distance and the diffraction condition described in discussion section.
Fig. 5 Simulation results of Fraunhofer diffraction with phase shift of retroreflector. (a) Pinhole aray, (b) The distance from pinhole array to retroreflector is at 40 mm, (c) 50 mm, (d) 60 mm, (e) 70 mm, (f) 80 mm, (g) 90 mm and (h) 100 mm.
Fig. 6 2D simulation results of intensity distribution of Fraunhofer diffraction depending on the distance from the pinhole array to the retroreflector. There are two pinhole which gray box is the pinhole size and black box is the one elemental image size. (a) Intensity distribution of Fraunhofer diffraction by retroreflector in two pinholes. (b) There are two pinholes used in initial simulation setup. Total and pinhole size is 1.564 mm and 0.276 mm, respectively. (c) Magnification of (a) at near 0 mm. 1st peaks are overlapped from 80 mm to 100 mm. Red box R1 and R2 are 1st peaks of intensity distribution from 80 mm to 100 mm. R3 is the overlapped region of them.
3. Experimental results
The proposed system is basically aimed at displaying 3D image. Therefore, the first experiment is to confirm 3D imaging after the retroreflection. The experimental setup is depicted in Fig. 7(a). We displayed three characters: K, H and U that each depth of the image with respect to the distance from the display is designed to be 60 mm, 40 mm, and 20 mm, respectively. The retroreflector is the triangular type, and the longest width of the effective aperture size of the retroreflector is 0.056 mm. The specification of experimental setup is summarized in Table 1. At the distance from the pinhole-type integral imaging system to the retroreflector is 150 mm, the reconstructed character images are located at 90 mm, 110 mm and 130 mm from the retroreflector. Each experimental result presents one focused and two defocused images as shown in Figs. 7(b)-(c). The results indicate that each character image has the different depth information. And the image location before and after retroreflection are identical. According to the experimental results, the proposed system shows that the reconstructed image location by the retroreflector maintains the initial image location in the proposed system.
Fig. 7 (a) Experimental setup, (b) Experimental result located at character image ’K’, (c) ’H’ and (d) ’U’.
Table 1. Specification of experiment for image location
The second experiment confirms the increased fill factor by the Fraunhofer diffraction of the retroreflector. The experimental setup consists of a LED light source, an optical diffuser, a pinhole array which is 3 by 3, beam splitter and retroreflector. The specification of the pinhole array and the retoreflector are same as the first experiment. The experiment was performed by changing the distance from the pinhole array to the retroreflector. Figure 8 shows experimental results. According to the geometrical analysis, the size of the conjugated pinhole array image does not exceed two times the size of corner cube. However, it is much larger than two times of that due to the diffraction effect. At the distance 80 mm, the patterns begin to overlap each other. It makes the resolution of the imaging system lower. Therefore, in the specification of pinhole array, the distance 70 mm is the best distance for displaying the high resolution image and the increased fill factor.
Fig. 8 Experimental results of Fraunhofer diffraction with phase shift of retroreflector. (a) Pinhole array, (b) The distance from pinhole array to retroreflector is at 40 mm, (c) 50 mm, (d) 60 mm, (e) 70 mm, (f) 80 mm, (g) 90 mm and (h) 100 mm.
The third experiment confirms the feasibility of the proposed system. The experimental setup and conditions are same as the first experiment. We compare the fill factor and the visibility between the conventional and the proposed system. Upper line of Fig. 9 is the results of pinhole-type integral imaging and lower line is the results of the proposed system. The results of the proposed system show the increased fill factor compared as the conventional system. It provides better visibility for watching character image. In addition, they show horizontal parallax when observer moves within the viewing region. However, the depth locations of K, H and U of the proposed system are reversed. This occurs due to the change of the observer position. In the proposed system, it is same that the observer watches the backside of the 3D image of the pinhole-type integral imaging system.
Fig. 9 Experimental results of pinhole-type integral imaging and the proposed system (a) Left view of pinhole-type integral imaging, (b) center view, (c) right view (see Visualization 1),(d) Left view of the proposed system, (e) center view, (f) right view (see Visualization 2).
4. Discussion
The purpose of the proposed system is the improvement of the visibility with increased the fill factor by using the retroreflector. We consider two factors in order to achieve the optimized fill factor in the proposed system. There are the pinhole arrangement of the pinhole array and the distance from the pinhole array to the retroreflector. The rectangular arrangement is not effective arrangement considering the fill factor and the shape of the Fraunhofer diffraction pattern by the retroreflector. Therefore, we propose the honeycomb arrangement as the most effective arrangement in the proposed system. Note that the optimization for high fill factor does not mean the high resolution of the pinhole-type integral imaging because it cannot increase the number of pixels of the proposed system.
The Franhofer diffraction pattern size is the main factor to decide the fill factor of the proposed system. Two parameters, which are the viewing window size defined by a beam splitter size in the view from an observer and the distance from the pinhole array to the retroreflector, conclude the diffraction pattern size. The beam splitter is used to observe the reconstructed 3D image properly because the proposed system is the reflective-type. The size of beam splitter is same or bigger than the total size of the reconstructed 3D image in order to provide the whole viewing window. If the size of the beam splitter is decided, we can find the optimized distance from the pinhole array to the retroreflector based on the specification of the proposed system such as the pinhole size of the pinhole array and the corner cube size of the retroreflector. When the Fraunhofer diffraction patterns are overlapped each other, they yield the noise or the degradation of the resolution of the system. Therefore, we need to find the optimized distance. The method follows as
where is the optimized distance,is the distance from the pinhole array to the retroreflector,is the 1st peak of the intensity distribution of the Fraunhofer diffraction pattern, is the value of overlapped intensity distribution of the Fraunhofer diffraction pattern. Based on Eq. (3), the experiment for the optimization is conducted. The specifications of the experiment are summarized in Table 2. The size and the arrangement of the pinhole array are changed to adapt to the optimal conditions. Figure 10 shows the results compensated for the optimization. Comparing the results of Fig. 9, the overlapped region between the Fraunhofer diffraction patterns is decreased. However, in the use of the retroreflector, there is a drawback. It is difficult to reconstruct high resolution image. In ray optics, when a point light source projects onto the retroreflector, the retroreflected image is not converging at one point but forming some beam width. Therefore, it is hard to get clear sharp image. In addition, the increased diffraction pattern size affects the image resolution. As a future work, the research for the increased resolution of the system using the retroreflector is ongoing.
Table 2. Specification of experiment for optimization
Fig. 10 The optimized experimental results of pinhole-type integral imaging and the proposed system (a) Left view of pinhole-type integral imaging, (b) center view, (c) right view (see Visualization 3), (d) Left view of the proposed system, (e) center view, (f) right view (see Visualization 4).
5. Conclusions
The pinhole-type integral imaging system with the enhanced fill factor is proposed. The Fraunhofer diffraction effects from the retroreflector improves fill factor of pixelated image without noticeable degradations of the 3D viewing characteristics such as the viewing angle and the image depth. The diffraction phenomena of the proposed system can be calculated and simulated to design the optimum specifications of the system parameters like the interval and the size of pinhole sheet. Because the visibility problem should be severer for the nearsighted display systems like head mount display (HMD), the proposed method can be applied to the augmented reality and the virtual reality such as 3D HMD.
Funding
National Research Foundation of Korea (NRF) Grant funded by the Korean Government (MSIP) (No.2011-0030079).
References and links
1. G. Lippmann, “La photographic integrale,” CR Acad. Sci. 146, 446–451 (1908).
2. J.-H. Park, K. Hong, and B. Lee, “Recent progress in three-dimensional information processing based on integral imaging,” Appl. Opt. 48(34), H77–H94 (2009). [CrossRef] [PubMed]
3. J. Hong, Y. Kim, H.-J. Choi, J. Hahn, J.-H. Park, H. Kim, S.-W. Min, N. Chen, and B. Lee, “Three-dimensional display technologies of recent interest: principles, status, and issues,” Appl. Opt. 50(34), H87–H115 (2011). [CrossRef] [PubMed]
4. F. Okano, H. Hoshino, J. Arai, and I. Yuyama, “Real-time pickup method for a three-dimensional image based on integral photography,” Appl. Opt. 36(7), 1598–1603 (1997). [CrossRef] [PubMed]
5. R. Martinez-Cuenca, A. Pons, G. Saavedra, M. Martinez-Corral, and B. Javidi, “Optically-corrected elemental images for undistorted Integral image display,” Opt. Express 14(21), 9657–9663 (2006). [CrossRef] [PubMed]
6. K. Yamamoto, T. Mishina, R. Oi, T. Senoh, and M. Okui, “Cross talk elimination using an aperture for recording elemental images of integral photography,” J. Opt. Soc. Am. A 26(3), 680–690 (2009). [CrossRef] [PubMed]
7. J. Hahn, Y. Kim, E.-H. Kim, and B. Lee, “Undistorted pickup method of both virtual and real objects for integral imaging,” Opt. Express 16(18), 13969–13978 (2008). [CrossRef] [PubMed]
8. J. Arai, F. Okano, H. Hoshino, and I. Yuyama, “Gradient-index lens-array method based on real-time integral photography for three-dimensional images,” Appl. Opt. 37(11), 2034–2045 (1998). [CrossRef] [PubMed]
9. J. Arai, H. Kawai, and F. Okano, “Microlens arrays for integral imaging system,” Appl. Opt. 45(36), 9066–9078 (2006). [CrossRef] [PubMed]
10. S.-W. Min, J. Hong, and B. Lee, “Analysis of an optical depth converter used in a three-dimensional integral imaging system,” Appl. Opt. 43(23), 4539–4549 (2004). [CrossRef] [PubMed]
11. Y. Kim, J. Kim, J.-M. Kang, J.-H. Jung, H. Choi, and B. Lee, “Point light source integral imaging with improved resolution and viewing angle by the use of electrically movable pinhole array,” Opt. Express 15(26), 18253–18267 (2007). [CrossRef] [PubMed]
12. J.-S. Jang and B. Javidi, “Improvement of viewing angle in integral imaging by use of moving lenslet arrays with low fill factor,” Appl. Opt. 42(11), 1996–2002 (2003). [CrossRef] [PubMed]
13. S. Park, B.-S. Song, and S.-W. Min, “Analysis of image visibility in projection-type integral imaging system without diffuser,” J. Opt. Soc. Korea 14(2), 121–126 (2010). [CrossRef]
14. S.-W. Min, J. Kim, and B. Lee, “New characteristic equation of three-dimensional integral imaging system and its applications,” Jpn. J. Appl. Phys. 44(2), L71–L74 (2005). [CrossRef]
15. D. A. Arnold, “Method of calculating retroreflector-array transfer functions,” SAO Special Report 382 (1979).
16. K. Arrington and G. Geri, “Conjugate-optical retroreflector display system: Optical principles and perceptual issues,” J. Soc. Inf. Disp. 8(2), 123–128 (2000). [CrossRef]
17. H. Yamamoto, Y. Tomiyama, and S. Suyama, “Floating aerial LED signage based on aerial imaging by retro-reflection (airr),” Opt. Express 22(22), 26919–26924 (2014). [CrossRef] [PubMed]
18. B. Song, S. Choi, H. Sung, and S.-W. Min, “Reflection-type three-dimensional screen using retroreflector,” J. Opt. Soc. Korea 18(3), 225–229 (2014). [CrossRef]
19. J. Liu and R. M. Azzam, “Polarization properties of corner-cube retroreflectors: theory and experiment,” Appl. Opt. 36(7), 1553–1559 (1997). [CrossRef] [PubMed]
