Open Access
Add to BibSonomyAbstract
We propose a new method for implementing 3D/2D convertible feature in the projection-type integral imaging by using concave half mirror array. The concave half mirror array has the partially reflective characteristic to the incident light. And the reflected term is modulated by the concave mirror array structure, while the transmitted term is unaffected. With such unique characteristic, 3D/2D conversion or even the simultaneous display of 3D and 2D images is also possible. The prototype was fabricated by the aluminum coating and the polydimethylsiloxane molding process. We could experimentally verify the 3D/2D conversion and the display of 3D image on 2D background with the fabricated prototype.
©2010 Optical Society of America
1. Introduction
Three-dimensional (3D) display is one of the attractive themes in the field of the display technology because it will definitely open a new market to the display industry [1–4]. But there is no leading standard for the implementation of 3D display yet. Integral imaging (InIm) is one of candidates that have been developed for long time since its first proposal in 1908 [5–7]. To be widespread in the market, one of key issues for the 3D display technique is whether it supports the display by projection. As we can see from the recent trend, the success in the movie market is the important factor for the new technology to be accepted in the display market. In that viewpoint, the display by projection is a necessary condition for the theater application because it is the easiest way to provide large-sized display. To satisfy such condition, researchers have also developed various ways to implement projection-type InIm. As shown in Fig. 1 , there are two categories of the projection-type InIm. Rear projection-type InIm is a very intuitive modification of the conventional InIm [8,9]. It simply replaces the display device of the conventional InIm with the combination of the rear-projection-screen and the projector. In this setup, the additional space is required behind the screen and it can be considered as cost-inefficient. Frontal projection-type InIm uses the concave mirror array instead of the lens array to integrate the 3D image – this is also called reflection-type InIm because it uses mirrors [10,11]. Because the concave mirror performs the same role as the convex lens, the principle of InIm is equally applied in this setup. In this case, the projection is done from the frontal side of the concave mirror array, so the additional room space is not needed. There was also a study reporting that the use of convex mirror array can implement the frontal projection-type InIm [12]. Instead of using concave or convex mirror array, the combination of the large aperture lens and the lens array also can implement the frontal projection-type InIm [13].
Fig. 1 Two categories of projection-type InIm. (a) Rear projection-type and (b) frontal projection-type.
In recent years, InIm became more practically useful because it can now provide 3D/two-dimensional (2D) convertible feature. In the early market of 3D display, it is important for the new 3D display device to be compatible with the 2D contents. In other words, 3D display device should operate as the 2D display while displaying 2D contents. Therefore, 3D/2D convertible feature is essential for the fast penetration of 3D display into the display market. Former researches of our group had proposed a number of methods to make it possible for InIm to be 3D/2D convertible. But all of them adopt the active devices as the core component for implementing 3D/2D conversion. The first proposal of 3D/2D conversion by Park et al. adopted polymer-dispersed liquid-crystal (PDLC) [14]. Choi el al. used polarization switcher for the implementation of thin panel system [15]. Many other active devices such as light emitting diode (LED) [16], organic light emitting diode (OLED) [17] and electroluminescent (EL) film have also been used [18]. But the use of active devices makes it difficult to provide 3D/2D convertible feature in the large-sized application.
In this paper, we propose a novel scheme that is capable of providing 3D/2D convertible feature for the projection-type InIm system. As discussed in the above, the major difficulty in realizing it comes from the use of the active device. Therefore, in our scheme, the new passive optical structure is proposed for providing 3D/2D convertible feature. Previously we presented the concept of our scheme at a conference, but it was difficult to verify our scheme experimentally because the fabrication method was not developed well [19]. In the following sections, we will describe the principle of the proposed scheme. And the experimental results will be provided to verify our method.
2. Principle of the proposed scheme
To achieve 3D/2D convertible feature in the projection-type InIm, we propose the novel optical structure named the concave/convex half mirror array (CHMA). As shown in Fig. 2(a) , the external appearance of the structure is simply the transparent plate. Any kind of transparent material can be used to form the entire shape. Inside this transparent plate, the thin concave/convex mirror array structure is buried in the middle as shown in the cross section of the structure. Ideally the thickness of it should be infinitesimal not to affect the transparent plate structure. For our purpose, the thin concave/convex mirror array inside the structure should have the half mirror characteristic. This is why we named our structure CHMA.
Fig. 2 The proposed structure of CHMA. (a) Cross sectional view of the CHMA. (b) Effect on the incident light when the incidental direction is the concave side and (c) the convex side of the CHMA.
CHMA affects the optical path of the incident light in unique way as shown in Fig. 2(b) and (c). If the light is incident from the right side of CHMA as shown in Fig. 2(b), a part of incident light will be reflected back by the concave mirror layer inside the transparent plate. Then it will propagate just as it is reflected by the ordinary concave mirror array except the refraction at the surface of the structure. On the other hand, the other part of the incident light will be transmitted through the concave mirror layer because it has the half mirror characteristic. Under the assumption that the concave mirror layer has infinitesimal thickness, there is no change of refractive index through the transmission, so the transmitted light will propagate just as it is passing through the transparent plate. Therefore the incident light is separated into two paths – one is reflected by the concave mirror array and the other is the path without any effect – by CHMA. For the case where the light is incident from the left side as shown in Fig. 2(c), the result is much similar but the reflection is made by the convex mirror array.
Figure 3 shows the system configuration that achieves the 3D/2D convertible feature for the projection-type InIm by adopting CHMA described above. For our system, we use two projectors – one is for the 3D integrated image and the other is for the 2D image. In our proposal, CHMA acts as the screen that shows the 3D integrated image. As shown in Fig. 3, CHMA is located in front of the observer and the projector for 3D integrated image projects image from the direction of observer. As mentioned before, both concave and convex mirror arrays can be used for the projection-type InIm, so there is no preference in determining which side of CHMA will be the frontal side. In Fig. 3, the concave mirror side was determined as the frontal side as an example. The rear projection-type screen is located behind CHMA for the display of 2D image. Then the projector for 2D image is located behind the screen to project 2D image from the back side.
Figure 4 describes the operation of the proposed system depicted in Fig. 3. To show the 3D integrated image to the observer, only the projector for 3D image is activated as shown in Fig. 4(a). Then the projected image is separated by CHMA into reflected and transmitted light components as discussed before, and the reflected part is shown to the observer. Because the reflected part is modulated by the concave mirror array structure, the 3D integrated image is shown to the observer if the proper elemental image is projected. In generating the proper elemental image, the refraction on the surface of CHMA should be considered. The refraction affects to the optical path of both the projection and integration. So the compensation of such refraction should be incorporated in the elemental image, and it can be easily done by using simple geometrical optics. Part of the transmitted term can be backscattered on the rear projection-type screen. Such backscattering can cause the emergence of halo or decrease the contrast of 3D image. The way to avoid such effect is to reduce the luminance of the backscattered light. The luminance of backscattering is reduced by the ratio of t 2 ε, where t is the transmittance of CHMA and ε is the scattering efficiency of the rear projection-type screen. So, if t or ε is lowered, the backscattering can be reduced. The holographic diffuser with transmittance over 90% is now available from several companies. When such diffuser is used for the rear projection-type screen, the backscattering effect can be sufficiently eliminated with low ε. The mode change can be easily done by alternating activated projectors. By activating the projector for 2D image with the projector for 3D image inactivated, 2D image projected on the rear projection-type screen will be shown to the observer as the transmitted light as shown in Fig. 4(b). Moreover, if we activate both projectors, it is possible to display 3D integrated image and 2D image in one scene together as shown in Fig. 4(c). We will call this ‘3D on 2D mode.’
3. Fabrication process
To show the feasibility of our method, we need to fabricate the proposed CHMA structure. Detailed procedures to fabricate CHMA are schematized in Fig. 5 . Fabrication is composed of two steps: one is to form the thin concave mirror array and the other is to cover the mirror layer with transparent material. As a result, CHMA will be constructed as the sandwich of three layers: base layer, thin concave mirror layer – or metallic layer - and the cover layer.
A 2D convex lens array is prepared as the base layer to support the shape of concave mirror array. As the first step, the base layer is coated with thin metallic layer to form the thin mirror layer. Then the 2D array of concave mirrors can be achieved following the shape of the base lens array. By using thin metallic layer, half mirror characteristic can also be achieved. The most suitable way to form the thin metallic layer can be the thermal evaporation to acquire acceptable uniformity. The ratio between transmittance and reflectance can be determined by controlling the thickness of metallic layer. Though the thickness of the metallic layer is not infinitesimal, there was no effect on the 2D image by the sufficiently thin metallic layer. Therefore we perform thermal evaporation as the first step. As the second step, we cover the result of the first step with the transparent material to adjust the outline of the entire structure to be the transparent plate. The most facile and inexpensive way for constructing transparent cover may be the molding process with polydimethylsiloxane (PDMS). The PDMS prepolymer is poured on the result of the first step. Then the vacuuming in the vacuum chamber is done for the elimination of air to increase the clearance of the PDMS layer. To harden the PDMS layer, 3 hours of baking at 80°C was performed and the PDMS molding process is completed.
4. Experimental results
For the verification of our proposed method, we fabricated prototype of CHMA for the several preliminary experiments. Figure 6 shows the camera captured image of the fabricated prototype. We can see that the external shape of the prototype is the plate. In this prototype, we determined the concave mirror side as the frontal surface. On the backside of the prototype, we attached the rear-projection-type screen for the display of the 2D image in 2D mode.
Fig. 6 Camera captured image of the prototype. (a) Frontal side of the prototype. The concave mirror side is used as the frontal side for this prototype. (b) Rear side of the prototype. On the rear side, the rear projection-type screen is attached. (c) Cross sectional view of the prototype. Three layers – the base, metallic and cover layers - can be investigated.
For the cover layer, we used Sylgard 184 base and curing agents in the PDMS molding process. Then they were mixed by the ratio of 10:1 for 10 minutes. As shown in Table 1 , the 2D lens array used as the base layer was made of polymethyl methacrylate (PMMA) whose refractive index is about 1.49. But the cover layer is made of PDMS and its refractive index is about 1.43. Because of such slight mismatch in refractive index, the prototype CHMA is not a perfectly transparent plate for the transmitted light. If the lens array made of PDMS is used as the base layer, the problem of index mismatching would be resolved. But the PDMS lens array cannot be used because of the thermal evaporation which is the first step of the process. To obtain the high surface uniformity of the mirror layer, the hardness of the base layer should be more than certain level. The base layer made of PMMA guarantees the uniform mirror surface, while the metallic layer evaporated on PDMS does not show the mirror characteristic because PDMS is damaged during thermal evaporation. Though the lens effect is not perfectly canceled because of the slight mismatch in refractive index, our fabrication is still valid for our purpose. The effective focal length of the prototype is sufficiently larger than the focal length of the base lens array, so the quality of 2D image is significantly enhanced by the existence of the cover layer. The effective focal length of each lenslet can be estimated as 26.95mm by using the formula f = R/Δn where R is the radius of curvature of the lenslet and Δn is the difference of the refractive index across the lenslet surface.
Table 1. Specification of the prototype
As a reference, the quality of 2D images shown through the CHMA and the base lens array are compared in Fig. 7 . The difference clearly appears for the highly textured image such as the text image compared in Fig. 7(a). The shape of each character is clearly recognizable only through the CHMA. As shown in Fig. 7(b), even the quality of the natural image where the low spatial frequency is dominant is noticeably degraded by the base lens array. Therefore the cover layer by PDMS molding is enough to provide the high quality image in the 2D mode of the proposed system.
Fig. 7 Comparison of the 2D images shown through the CHMA and the base lens array. (a) Highly textured image is used as the test 2D image. Part of the 2D image indicated as the red square box in the original image is magnified to show the details of the result. (b) A landscape image is used for the comparison of a natural image.
For the thin metallic layer, we evaporated aluminum with thickness about 150nm. We can estimate the transmittance of the mirror array as 30% approximately [11]. Then the reflectance of the prototype is about 70%. As a result, the luminance of the 2D image will be reduced to 30% as it is shown by the transmission.
Figure 8 shows the configuration of our experimental setup to implement the system described in Sec. 2. The configuration comprises two projectors, an objective lens, a beam splitter and a CHMA prototype. We used HS102 of LG Electronics whose resolution is 800Х600 for the both projectors. The elemental image is projected onto the concave mirror-side of CHMA by the projector 1. To enhance the performance of the integration, the pixel pitch of the projected elemental image should be much smaller than the pitch of one concave mirror of CHMA. Because the commercial projector used in the experiment is not designed for the intention of InIm, it is needed to reduce the size of the projected image by using the objective lens as shown in Fig. 8. As a result, the pixel pitch of the projected elemental image was reduced to 88µm. As the side effect of using the objective lens, the operating distance where the projected image is well focused was shortened to 60mm. To secure the enough watching distance, we located the beam splitter crossing the optical path of the projection to separate the watching direction from the projecting direction. In Fig. 8, the optical paths of the projection and watching in 3D mode are depicted. The projector 2 used for 2D mode is located behind the CHMA and the pixel pitch of the projected image is 133µm.
We performed a preliminary experiment to verify our proposed method as shown in Fig. 9 . In the 3D mode, two letters ‘S’ and ‘U’ are located at 7mm front of and 7mm behind of the CHMA. ‘N’ is located on the surface of CHMA. We generated the elemental image to display these integrated images by the computer aided calculation. Therefore the real image ‘S’, the surface image ‘N’ and the virtual image ‘U’ can be easily displayed together. In Fig. 9, the horizontal and vertical parallaxes are well expressed in the 3D image displayed by reflection-type InIm. Though we used the ordinary diffuser for the rear projection-type screen, the backscattering effect discussed in Sec. 2 is hardly noticed as image. In the experiment, t is about 0.3 and ε of the ordinary diffuser is generally under 0.5. Hence the luminance of backscattering is reduced below 5% of the projected image. It means that this level of reduction is enough to avoid backscattering effect in our experimental setup. Additionally, the fact that the projected image is focused near the surface of the concave mirror layer was also helpful in reducing backscattering effect.
Fig. 9 Camera captured images of the integrated images displayed by the 3D mode of our proposed system.
As described in Sec. 2, our proposed system is also able to display 3D integrated image and 2D image in one scene together. The experimental results shown in Fig. 10 illustrate this functionality. 3D integrated images were the same as the experiment in Fig. 10. For the 2D image we displayed the image indicated as ‘Image used for 2D image’ in Fig. 10. The 2D image is displayed with the pixel pitch of 133µm as described before, and there is no noticeable artifacts caused by index mismatch of the CHMA prototype as we discussed. But the luminance of the 2D image is reduced to 30% of the original image by the transmittance of the metallic layer. As we can see from Fig. 10, 3D integrated image and 2D image were successfully displayed in one scene. The luminance of the 2D image varies as the watching direction changes because the rear projection-type screen used in the experiment has directionally non-uniform diffusing characteristic.
Fig. 10 Camera captured image of the displayed images in ‘3D on 2D mode’ of the prototype. Original image used for 2D image is shown in the left side of the figure; the main gate of Seoul National University.
The operation of our system is demonstrated as a movie in Fig. 11 . In 3D mode, we continuously moved the position of camera to show the disparity of 3D integrated images. Then the conversion from 3D to 2D mode is easily done by simply activating the projector 2 with the projector 1 inactivated. At last, the movie shows the simultaneous display of 3D images and 2D images when two projectors are both activated.
5. Conclusion
In this paper, we have proposed the novel method to provide 3D/2D convertible feature to the projection-type InIm. To our knowledge, this is the first proposal that makes it possible for the projection-type InIm to provide 3D/2D convertible feature with the passive optical components. In fact, it is also the first proposal for the autostereoscopic 3D display implemented as projection-type to have 3D/2D convertible feature.
As we discussed, our prototype has the slight mismatch in the refractive index between the base and cover layers, but it is enough for our application as verified in Sec. 4. Moreover the PDMS molding process has advantage in that it is facile and inexpensive. Hence our fabrication method will be acceptable in most cases.
We believe that this study will be useful for the 3D industry because the 3D theaters will require autostereoscopic display as the next step. The advantage of adopting our proposed system is not only the saving in space of the theater. It will give a higher degree of freedom to 3D movie makers in doing their work because it is easy to mix up 3D images and 2D images with our proposed system. In conclusion, our proposed method will contribute to the widespread of autostereoscopic 3D display in 3D theaters.
Acknowledgment
This work was supported by the IT R&D program of MKE/KEIT. [KI002058, Signal Processing Elements and their SoC Developments to Realize the Integrated Service System for Interactive Digital Holograms].
References and links
1. P. Benzie, J. Watson, P. Surman, I. Rakkolainen, K. Hopf, H. Urey, V. Sainov, and C. von Kopylow, “A survey of 3DTV displays: techniques and technologies,” IEEE Trans. Circ. Syst. Video Tech. 17(11), 1647–1658 (2007). [CrossRef]
2. B. Lee, J.-H. Park, and S.-W. Min, Digital Holography and Three-Dimensional Display, T.-C. Poon, ed. (Springer US, 2006), Chap. 12.
3. B. Javidi, and F. Okano, eds., Three Dimensional Television, Video, and Display Technology (Springer, 2002).
4. T.-C. Poon, “Optical scanning holography - a review of recent progress,” J. Opt. Soc. Korea 13(4), 406–415 (2009). [CrossRef]
5. G. Lippmann, “La photographie integrale,” Acad. Sci., Paris, C. R. 146, 446–451 (1908).
6. J.-H. Park, K. Hong, and B. Lee, “Recent progress in three-dimensional information processing based on integral imaging,” Appl. Opt. 48(34), H77–H94 (2009). [CrossRef] [PubMed]
7. M.-O. Jeong, N. Kim, and J.-H. Park, “Elemental image synthesis for integral imaging using phase-shifting digital holography,” J. Opt. Soc. Korea 12(4), 275–280 (2008). [CrossRef]
8. M. Kawakita, H. Sasaki, J. Arai, F. Okano, K. Suehiro, Y. Haino, M. Yoshimura, and M. Sato, “Geometric analysis of spatial distortion in projection-type integral imaging,” Opt. Lett. 33(7), 684–686 (2008). [CrossRef] [PubMed]
9. S.-W. Min, J. Kim, and B. Lee, “Wide-viewing projection-type integral imaging system with an embossed screen,” Opt. Lett. 29(20), 2420–2422 (2004). [CrossRef] [PubMed]
10. Y. Jeong, S. Jung, J.-H. Park, and B. Lee, “Reflection-type integral imaging scheme for displaying three-dimensional images,” Opt. Lett. 27(9), 704–706 (2002). [CrossRef]
11. Y. Kim, S. G. Park, S.-W. Min, and B. Lee, “Integral imaging system using a dual-mode technique,” Appl. Opt. 48(34), H71–H76 (2009). [CrossRef] [PubMed]
12. J.-S. Jang and B. Javidi, “Three-dimensional projection integral imaging using micro-convex-mirror arrays,” Opt. Express 12(6), 1077–1083 (2004). [CrossRef] [PubMed]
13. M. Okui, J. Arai, Y. Nojiri, and F. Okano, “Optical screen for direct projection of integral imaging,” Appl. Opt. 45(36), 9132–9139 (2006). [CrossRef] [PubMed]
14. J.-H. Park, H.-R. Kim, Y. Kim, J. Kim, J. Hong, S.-D. Lee, and B. Lee, “Depth-enhanced three-dimensional-two-dimensional convertible display based on modified integral imaging,” Opt. Lett. 29(23), 2734–2736 (2004). [CrossRef] [PubMed]
15. H. Choi, S.-W. Cho, J. Kim, and B. Lee, “A thin 3D-2D convertible integral imaging system using a pinhole array on a polarizer,” Opt. Express 14(12), 5183–5190 (2006). [CrossRef] [PubMed]
16. S.-W. Cho, J.-H. Park, Y. Kim, H. Choi, J. Kim, and B. Lee, “Convertible two-dimensional-three-dimensional display using an LED array based on modified integral imaging,” Opt. Lett. 31(19), 2852–2854 (2006). [CrossRef] [PubMed]
17. Y. Kim, J. Kim, Y. Kim, H. Choi, J.-H. Jung, and B. Lee, “Thin-type integral imaging method with an organic light emitting diode panel,” Appl. Opt. 47(27), 4927–4934 (2008). [CrossRef] [PubMed]
18. J.-H. Jung, Y. Kim, Y. Kim, J. Kim, K. Hong, and B. Lee, “Integral imaging system using an electroluminescent film backlight for three-dimensional-two-dimensional convertibility and a curved structure,” Appl. Opt. 48(5), 998–1007 (2009). [CrossRef] [PubMed]
19. J. Hong, Y. Kim, S. Park, S. Min, and B. Lee, “3-D/2-D convertible projection-type integral imaging system by use of half convex mirror array,” in Digital Holography and Three-Dimensional Imaging, OSA Technical Digest (CD) (Optical Society of America, 2010), paper JMA17.
