Open Access
Add to BibSonomyAbstract
Three-dimensional projection integral imaging using micro-convex-mirror arrays is presented. In this scheme, a pseudoscopic to orthoscopic image conversion process is not required even if elemental images of direct camera pickup are used. In addition, a wide viewing angle without image flipping can be achieved even if optical barriers are not used. The feasibility of the proposed scheme was experimentally demonstrated, in which the viewing angle of more than 60 degrees was obtained. To the best of our knowledge, this is the largest viewing angle demonstrated by a single integral imaging system.
©2004 Optical Society of America
1. Introduction
Three-dimensional (3-D) imaging and display using two-dimensional (2-D) display devices have been subjects of much research due to their diverse benefits and applications [1–5]. Stereoscopic techniques have been most widely used to achieve 3-D displays so far, because it is relatively easy to realize a stereoscopic system that can display large images with high resolution [1]. However, stereoscopic techniques usually require supplementary glasses to evoke 3-D visual effect, and provide observers with only horizontal parallax and a limited number of viewpoints. Observation of stereoscopic images may cause visual fatigue because of convergence-accommodation conflict [2].
Holography has been used for 3-D displays, because true 3-D images can be produced by diffraction in gratings. In this case, the process to obtain proper gratings requires either coherent light when holograms are recorded or a large amount of computation time if computer-generated holograms are prepared [3–5]. Thus it may not be easy to realize large color 3-D displays based on holography.
For optical display and visualization of true 3-D images in space with incoherent light, integral imaging (II), or real-time integral photography, has been studied [6–10]. In II, 3-D images are formed by crossing the rays coming from two-dimensional (2-D) elemental images using a lenslet array. As in holography, II can provide observers with true 3-D images with full parallax and continuous viewing points. However, there are also drawbacks in II. For example, the viewing angle, depth-of-focus, and resolution of 3-D images are limited because lenslet arrays are used [11,12]. In addition, 3-D images produced in direct-pickup II are pseudoscopic (depth-reversed) images [7]. To overcome those limitations a number of techniques have been presented, which may make II systems more complex [9,13–18].
Recently, we developed a projection-type of II systems with a lenslet array to overcome the limited number of pixels in currently available devices [19,20]. We found that a simple modification of the projection II system, i.e., the replacement of the lenslet array with a micro-convex-mirror array, can also solve two critical problems in II: 1) Limited viewing angle problem, and 2) pseudoscopic to orthoscopic image conversion problem.
Therefore, in this paper, we present a projection II scheme with a micro-convex-mirror array. The use of a micro-convex-mirror array increases the viewing angle significantly, because it is easy to make micro-convex mirrors with a small f number with negligible aberration. In addition, when elemental images obtained from direct camera pickup with a lenslet array and a 2-D image sensor are used, 3-D orthoscopic virtual images are automatically displayed. The projection scheme allows flipping-free observations of 3-D images even if optical barriers are not used, because each elemental image can be projected onto only its corresponding micro-concave mirror. The feasibility of the proposed scheme is experimentally demonstrated, in which the viewing angle of over 60 degrees is achieved.
Fig. 1. Pickup and display in 3-D II. (a) Pickup of elemental images using a lenslet array. (b) Conventional image display using a lenslet array and a display panel. (c) Projection II with a lenslet array. (d) Projection II with a micro-concave-mirror array. 2-D image projectors are not shown. (e) 3-D projection II with a micro-convex-mirror array. 2-D image projectors are not shown.
2. Integral imaging: various schemes
2.1 Conventional II
Conventionally, lenslet arrays with positive focal lengths have been used in II. A set of elemental images of a 3-D object (i.e., direction and intensity information of the spatially sampled rays coming from the object) are obtained by use of a lenslet array and a 2-D image sensor such as a CCD or a CMOS image sensor, as depicted in Fig. 1(a). To reconstruct a 3-D image of the object, the set of 2-D elemental images are displayed in front of a lenslet array using a 2-D display panel, such as a liquid crystal display (LCD) panel, as depicted in Fig. 1(b). The rays coming from the elemental images converge to form a 3-D real image through the lenslet array. This 3-D image is a pseudoscopic (depth-reversed) image of the 3-D object. The pseudoscopic real image is converted into an orthoscopic virtual image when every elemental image is rotated by 180 degrees around its own center optic axis [10]. It is also possible to display an orthoscopic real image by introducing an additional imaging lens in front of the pickup lenslet array [21].
The pseudoscopic image to orthoscopic image conversion (P/O conversion), i.e., rotation of every elemental image by 180 degrees, is accomplished either optically or digitally. Optical conversion degrades 3-D image quality because either additional lenslet arrays or an array of graded-index rod lenses should be used [9,13]. In digital conversion, each elemental image is sequentially rotated by image processing in a computer. In this case, reconstruction of 3-D images from elemental images in real time is difficult [21].
The full viewing angle ψ is limited and determined approximately by 2×arctan[0.5/(f/#)], where f/# is the f number of the lenslet, when the fill factor of the lenslet array is close to 1 [10,15,16]. For example, even if f/# is as low as 1, ψ is limited by ~50 degrees. As the viewing angle decreases, the viewing region where the entire 3-D image can be seen becomes more restricted, especially when a wide 3-D image is displayed. There were a few studies to enhance the viewing angle by introducing some optical system modifications, which may be difficult in implementation of a large-scale II system [15–18].
2.2 3-D Projection II using a lenslet array
In projection II, high resolution elemental images can be displayed, and thus the resolution of reconstructed 3-D images can be improved [20]. This is because multiple 2-D image projectors can be used at the same time, as depicted in Fig. 1(c). Each projector casts only a subset of entire elemental images onto the corresponding lenslet array part. Of course the diverging angle θ of the projection beam should be close to 0. In this projection II scheme using a lenslet array, however, the P/O conversion is required and the viewing angle is narrow as in the conventional scheme.
2.3 3-D Projection II using a micro-concave-mirror array
The main advantage of 3-D projection II is that it allows the use of either micro-concave-mirror arrays or micro-convex-mirror arrays instead of lenslet arrays. The use of such mirror arrays is desirable, because it is easier to make diffraction-limited (or aberration-free) concave/convex mirrors with a small f/# than it is to make similar lenslets. If a micro-concavemirror array is used in 3-D projection II, pseudoscopic real images are displayed as depicted in Fig. 1(d), because the micro-concave-mirror array simply plays a role of a lenslet array. Thus, the use of a micro-concave-mirror array with a large f/# does not have additional advantages [22].
2.4 3-D Projection II using a micro-convex-mirror array
What we propose in this paper is a 3-D projection II scheme using a micro-convex-mirror array, as depicted in Fig. 1(e). In this case, orthoscopic virtual images are automatically reconstructed, when the elemental images obtained from direct camera pickup depicted in Fig. 1(a) are used. This is because each micro-convex mirror does not rotate the corresponding elemental image around its own center optic axis in the 3-D image reconstruction process. Therefore, the display of raw elemental images using a micro-convex mirror is exactly equivalent to the display of P/O-converted elemental images using either a lenslet array or a micro-concave-mirror array. Each convex mirror element could have an f/# smaller than 1. For example, if f/#=0.5, the viewing angle ψ becomes 90 degrees, which is acceptable for many practical applications.
Note that optical barriers [13] are not required in projection schemes, if 1) the focal length of micro-convex mirrors (or micro-concave mirrors) is shorter than the depth-of-focus of the relay optics, and 2) the diverging angle θ of the projection beam is small. These conditions are easily satisfied in projection II systems. Then, each elemental image is projected onto its own micro-convex mirror, i.e., elemental images are not displayed through their neighboring micro-convex mirrors. Therefore, as observers’ viewing direction deviates from the optical axis that is normal to the display lenslet array, they do not experience flipping of the reconstructed 3D image (that is, higher-order reconstructed images).
3. Experiments on a projection II system with a micro-convex-mirror array
3.1 System description
To demonstrate the feasibility of projection II using a micro-convex-mirror array, an optical setup depicted in Fig. 2 was used. The object to be imaged is composed of a toy car and a cone with white and yellow stripes, as shown in Fig. 3. The process to obtain elemental images for the object is not different from the conventional method, which is depicted in Fig. 1(a). The pickup lenslet array we used is made from acrylic, and has 53×53 plano-convex lenslets. Each lenslet element is square-shaped and has a uniform base size of 1.09 mm×1.09 mm, with less than 7.6 µm separating the lenslet elements. The focal length of the lenslets is approximately 3 mm. A total of 34×26 elemental images are used in the experiment.
A color LCD projector that has 3 (RGB) panels was used for the 2-D image projector. Each panel has 1024×768 square pixels with a pixel pitch of 18 µm. Each elemental image has 30×30 pixels. The magnification factor of the relay optics used in our system is 2. The diverging angle of the projection beam θ is approximately 1.4 degrees. The distance between the micro-convex-mirror array and the relay optics is approximately 0.9 m.
For the micro-convex-mirror array, we used the pickup lenslet array itself. This is possible, because approximately 4% of light energy is reflected for normally incident light at the surface of the acrylic lenslet array whose refractive index is approximately 1.5. Therefore, to produce the micro-convex-mirror array, we positioned the lenslet array so that the convex surfaces of lenslets face the 2-D image projector. Then, we observed the reflected light from the surface of the lenslet array. If we flip the lenslet array so that the opposite side faces the 2-D image projector, the effect of a micro-concave-mirror array is obtained.
Fig. 5. Reconstructed orthoscopic virtual 3-D images. (a) Left view. (b) Upper view. (c) Right view.
Because we are using plano-convex lenslets, it is possible to estimate the radius of surface curvature of the lenslet R from lens-maker’s formula: 1/f=(n-1)/R, where f is the focal length of the lenslet and n is the refractive index of the lenslet. In our case, R≈1.5 mm and thus the focal length of the micro-convex-mirror array is 0.75 mm in magnitude. The viewing angle ψ is expected to be 67 degrees.
3.2 Experimental results
The raw elemental images were obtained from direct camera pickup as shown in Fig. 4, They are projected onto the micro-convex-mirror array, to reconstruct a 3-D orthoscopic virtual image. Because the intensity of projected elemental images is strong, the reconstructed 3-D image is bright enough to be seen under the normal room lighting environment. Left, upper, and right views of the reconstructed 3-D image are shown in Fig. 5. Viewing directions for the three images are deviated from the optical axis by ~30 degrees, respectively. When we used the lenslet array as a micro-concave-mirror array by facing its plano side to the viewer, the reconstructed image became a pseudoscopic real image, as we expected.
We repeated the same experiment using P/O-converted elemental images. The P/O conversion was carried out digitally in a computer. When the P/O-converted elemental images were projected onto the micro-concave-mirror array, a 3-D orthoscopic virtual image was reconstructed, as we expected. On the other hand, when the P/O-converted elemental images were projected onto the micro-convex-mirror array, a 3-D pseudoscopic real image was reconstructed. This is a bit surprising result, because negative lenses and convex mirrors cannot produce real images in usual circumstance. In II, however, it is possible to get a real image from a micro-convex-mirror array (or negative lens array), because images are formed by ray crossing: The image becomes real if rays cross, and virtual if extended ray lines cross.
The measured viewing angle was 60~70 degrees, which agrees well with the predicted value. To observers who move beyond the viewing angle range, the entire reconstructed image disappears. Higher-order reconstructed images were hardly observed for a well-aligned system. To show how wide the viewing angle is, a movie for the reconstructed 3-D image, which was made as the viewing position changes continuously, is presented in Fig. 6.
4. Discussion and conclusion
Although the pitch of the micro-convex-mirror array is exactly equal to that of the pickup lenslet array, the focal length of the micro-convex-mirror array is four times shorter than that of the pickup lenslet array. In this case, the longitudinal size of the reconstructed 3-D image is 4 times smaller than that of the object (i.e., longitudinal demagnification takes place), while the lateral size does not change. For equal longitudinal and lateral magnification, elemental images should be picked up using a micro-concave-mirror array with the same f/#, as depicted in Fig. 7. In this case, the surface reflectivity of the micro-concave-mirror array should be much higher than 4%. It is also possible to pickup elemental images using a micro-convex-mirror array, and then to reconstruct 3-D images using a micro-concave-mirror array. To obtain large-scale high-resolution 3-D images in the projection II system with micro-convex-mirror array, a spatiotemporal multiplexing technique can be applied to this system [20].
In conclusion, we have presented a projection II system with micro-convex-mirror arrays, in which the pseudoscopic to orthoscopic image conversion is not necessary and the viewing angle can be improved dramatically for practical applications.
Acknowledgments
This work was supported in part by University ITRC project of Korea Ministry of Information and Communication.
References and links
1. S. A. Benton, ed., Selected Papers on Three-Dimensional Displays (SPIE Optical Engineering Press, Bellingham, WA, 2001).
2. T. Okoshi, “Three-dimensional display,” Proc. IEEE 68, 548–564 (1980). [CrossRef]
3. D. H. McMahon and H. J. Caulfield, “A technique for producing wide-angle holographic displays,” Appl. Opt. 9, 91–96 (1970). [CrossRef] [PubMed]
4. I. Yamaguchi and T. Zhang, “Phase-shifting digital holography,” Opt. Lett. 22, 1268–1270 (1997). [CrossRef] [PubMed]
5. P. Ambs, L. Bigue, R. Binet, J. Colineau, J.-C. Lehureau, and J.-P. Huignard, “Image reconstruction using electrooptic holography,” Proceedings of the 16th Annual Meeting of the IEEE Lasers and Electro-Optics Society, LEOS 2003, vol. 1 (IEEE, Piscataway, NJ, 2003) pp. 172–173.
6. G. Lippmann, “La photographie integrale,” Comptes-Rendus Academie des Sciences 146, 446–451 (1908).
7. H. E. Ives, “Optical properties of a Lippmann lenticulated sheet,” J. Opt. Soc. Am. 21, 171–176 (1931). [CrossRef]
8. C. B. Burckhardt, “Optimum parameters and resolution limitation of integral photography,” J. Opt. Soc. Am. 58, 71–76 (1968). [CrossRef]
9. N. Davies, M. McCormick, and M. Brewin, “Design and analysis of an image transfer system using microlens arrays,” Opt. Eng. 33, 3624–3633 (1994). [CrossRef]
10. F. Okano, H. Hoshino, J. Arai, and I. Yuyama, “Real-time pickup method for a three-dimensional image based on integral photography,” Appl. Opt. 36, 1598–1603 (1997). [CrossRef] [PubMed]
11. H. Hoshino, F. Okano, H. Isono, and I. Yuyama, “Analysis of resolution limitation of integral photography,” J. Opt. Soc. Am. A 15, 2059–2065 (1998). [CrossRef]
12. J.-S. Jang, F. Jin, and B. Javidi, “Three-dimensional integral imaging with large depth of focus using real and virtual image fields,” Opt. Lett. 28, 1421–1423 (2003). [CrossRef] [PubMed]
13. J. Arai, F. Okano, H. Hoshino, and I. Yuyama, “Gradient-index lens-array method based on real-time integral photography for three-dimensional images,” Appl. Opt. 37, 2034–2045 (1998). [CrossRef]
14. J.-S. Jang and B. Javidi, “Improved viewing resolution of three-dimensional integral imaging with nonstationary micro-optics,” Opt. Lett. 27, 324–326 (2002). [CrossRef]
15. J.-S. Jang and B. Javidi, “Improvement of viewing angle in integral imaging by use of moving lenslet arrays with low fill factor,” Appl. Opt. 42, 1996–2002 (2003). [CrossRef] [PubMed]
16. S. Jung, J.-H. Park, H. Choi, and B. Lee, “Wide-viewing integral three-dimensional imaging by use of orthogonal polarization switching,” Appl. Opt. 42, 2513–2520 (2003). [CrossRef] [PubMed]
17. J.-S. Jang and B. Javidi, “Large depth-of-focus time-multiplexed three-dimensional integral imaging using lenslets with non-uniform focal lengths and aperture sizes,” Opt. Lett. 28, 1924–1926 (2003). [CrossRef] [PubMed]
18. S.-W. Min, B. Javidi, and B. Lee, “Enhanced three-dimensional integral imaging system by use of double display devices,” Appl. Opt. 42, 4186–4195 (2003). [CrossRef] [PubMed]
19. J.-S. Jang and B. Javidi, “Real-time all-optical three-dimensional integral imaging projector,” Appl. Opt. 41, 4866–4869 (2002). [CrossRef] [PubMed]
20. J.-S. Jang and B. Javidi, “Spatiotemporally multiplexed integral imaging projector for large-scale highresolution three-dimensional display,” Opt. Express 12, 557–563 (2004), http://www.opticsexpress.org/abstract.cfm?URI=OPEX-12-4-557. [CrossRef] [PubMed]
21. J.-S. Jang and B. Javidi, “Formation of orthoscopic three-dimensional real images in direct pickup one-step integral imaging,” Opt. Eng. 42, 1869–1870 (2003). [CrossRef]
22. Y. Jeong, S. Jung, J.-H. Park, and B. Lee, “Reflection-type integral imaging scheme for displaying three-dimensional images,” Opt. Lett. 27, 704–706 (2002). [CrossRef]
