Open Access
Add to BibSonomyAbstract
In order to realize three-dimensional (3D) displays, various multiplexing methods have been proposed to add the depth dimension to two-dimensional scenes. However, most of these methods have faced challenges such as the degradation of viewing qualities, the requirement of complicated equipment, and large amounts of data. In this paper, we further developed our previous concept, polarization distributed depth map, to propose the Lamina 3D display as a method for encoding and reconstructing depth information using the polarization status. By adopting projection optics to the depth encoding system, reconstructed 3D images can be scaled like images of 2D projection displays. 3D reconstruction characteristics of the polarization-encoded images are analyzed with simulation and experiment. The experimental system is also demonstrated to show feasibility of the proposed method.
© 2014 Optical Society of America
1. Introduction
Full three-dimensional (3D) displays, which can provide every aspect of depth cues, require tremendous amount of data [1]. The amount of the data capacity of the system is multiplied by the number-of-view of the conventional two-dimensional (2D) displays. For example, a nine-view 3D system requires nine times of data capacity for achieving the identical spatial resolution, or the spatial resolution of the system is divided by nine. Moreover, the realization of accommodation cue requires extremely narrow interval between viewpoints that results in increase of data capacity more than ten times of conventional multi-view displays.
As alternatives, 2.5D displays are used for providing partial depth perception cues with limited resources [1, 2]. For example, glasses-type stereoscopic methods provide only the binocular disparity which mainly induces the 3D perception. Perspectives are not provided although the observer’s position is changed, and the focus of the eyes should be fixed on the screen while the eyes naturally adjust focusing points according to the distance of objects in the real world. Consequently, the observed 3D images are unnatural to humans. Those unnatural characteristics of 3D systems cause the visual fatigue which reduces the viewing time of stereoscopic displays [3].
Another 2.5D display is a relief-type display. It is composed of a single surface which has a different depth position for every point [4, 5]. For example, a physically deformable screen can change its shape according to the displayed object, and the optical information such as color or brightness is provided by external 2D displays [6]. Although these systems can provide only one volumetric surface of an object in a limited range, it can satisfy every aspect of depth cues, such as binocular disparity, convergence, accommodation, and motion parallax [7]. However, physically deformable screens require complicated mechanical devices, and their expressible ranges are restricted.
Optically deformable screen can be a solution for overcoming the limitations of physically deformable screen. Depth Cube employed shutter screens which can be switched between transparent and scattering states [8]. By using the switchable screens, it can adjust the depth position of the projected images with a time-multiplexing technique. Similarly, depth-fused displays (DFD) can optically change the imaging position by the depth-fusing effect. The depth-fusing effect is an adjustment of focusing of the eye when observing two overlapped transparent images. The focusing position is decided by the intensity ratio of each layer [9–11]. However, most of the DFDs have limitations in size because available transparent flat-panel displays are confined to liquid crystal displays (LCDs).
In this paper, we propose Lamina 3D display, which can express relief-like volume by pixel-wise modulation of polarization of the projected image via a spatial light modulator (SLM). By adopting the polarization distributed depth map (PDDM) with projection optics, the scalability of the system is drastically improved, and viewing angle is also improved by adopting the multi-layered structure [12]. Also, the depth range of the system can be extended within the depth of focus of the projection optics. Although the projection-type dual-layered systems were proposed before, they required multiple projectors for displaying images for each layer [13]. However, the proposed system does not require additional projectors because the polarization modulator is embedded in the system. The feasibility of the system is analyzed by the point spread function (PSF) simulation and demonstrated by experiments.
2. System configuration
The Lamina 3D display is composed of encoding and decoding parts. In the encoding component, the 2D scene is overlaid with the depth information modulated by the PDDM, and the decoding component corresponds to the location at which relief-like 3D images are reconstructed in the special volume created by the laminated layers using the illumination characteristics of the polarization dependence of each layer. The conceptual diagrams illustrating the configuration and principle of the Lamina 3D display are presented in Fig. 1. The encoding component of the Lamina 3D display can be implemented using a simple 2D projector and SLM, such as a twisted nematic (TN)-LCD, which can rotate the polarization state of each pixel according to the grayscale of the input depth image [14, 15]. Consequently, two SLMs are needed for the encoding component: one for 2D imaging and another for depth encoding.
The 2D image modulated by the PDDM can be easily obtained and does not experience the temporal or spatial degradation during the depth multiplexing process because only the polarization of the image is modulated. The decoding component consists of polarization-sensitive optical devices. Because the human eye cannot detect the polarization of light but can detect the intensity of light, the special volume in which the polarization status is converted to the expressive depth must be defined within the decoding component of the Lamina 3D display. The special volume can be instrumented by laminating the volume with layers that can control the scattering and transmission ratio according to the polarization state of the projected light. The volume of the system consists of the multiple layers of the scattering polarizer films, which scatter and transmit the incident light according to the angle difference between the polarization axes of the light and the films. Like polarization screens which are commonly used as stereoscopic 3D screen, the polarized light is scattered without altering the polarization states. However, the amount of the scattered light of scattering polarizer is changed according to polarization states while the polarization screen always conserves the scattering amount of the light regardless of the polarization states.
The encoding component of the Lamina 3D display system can be assembled from the SLMs for the 2D imaging and depth encoding, linear polarizers, and relay optics including a projection lens set. The image from the SLM for the 2D imaging passes through the linear polarizer to ensure that the polarization state of the image is aligned with respect to the initial state for the modulation of the second SLM. This optical configuration is similar to the polarized light microscopy system. The only difference is that specimen is replaced with SLM which can control the polarization state. Like the polarized light microscope, the imaging optics does not alter the polarization [16]. To enable a pixel-by-pixel modulation of the polarization, the pixels of the 2D image should be overlaid with the corresponding pixels of the polarization-modulating SLM. For this purpose, the relay optics must be placed between the SLMs. By the relay optics, the image of the first SLM is located exactly on the second SLM. At this step, the size of the relayed image should be adjusted to fit to the size of the second SLM. With this configuration, the polarization status of each pixel of the projected image is modulated according to the grayscale of the relevant pixel of the depth map by the second SLM, which works as a polarization rotator composed of a TN-LCD with the polarizers on both sides removed. The input depth information of the second SLM is an 8-bit grayscale image with only one-third of the conventional 2D color information. This reduction in data is beneficial for the system implementation and information processing, including the acquisition and display processes. After passing the second SLM, the image has various polarization angles for each pixel distributed from 0° to 90° according to the given depth information of the second SLM as illustrated in the red box in Fig. 1. In our experimental setup, 0° corresponds to the farthest depth and 90° is assigned to the nearest depth position.
3. Volume rendition using laminated images
The decoding component of the Lamina 3D display is the laminated volume, where the image encoded by the PDDM is represented by a relief-like 3D image. The relief-like 3D image is a set of voxels located in the outermost part of the object such that each pixel of the image requires only one depth value. With the relief-like representation of the 3D object, the 3D data can be effectively reduced using the hidden point removal method while preserving the 3D volumetric characteristics [17]. The volume created by the passive screen is composed of arranged multiple scattering polarizers. Partially scattered images give the depth-fusing effect between layers resulting in the volumetric expression of images. The scattering characteristics of the multiple scattering layers can be expressed with the PSF which depends on the input polarization angle. The PSF of one layer can be approximately expressed as the Gaussian function shown in Eq. (1), whereas the PSF of the transmitted light is expressed by Eq. (2) according to the measured angular scattering properties of the films.
where θ represents the angle difference between the polarization state of the input light source and a scattering polarizer and ϕ is the deviation from the normal direction. ϕin represents the incident angle. aS(θ,ϕin), aT(θ,ϕin) and b are fitting parameters determined by the characteristics of the scattering polarizer and the conditions of the incident light ray. The coefficients aS and aT are highly dependent on the polarization state and incident angle, and their characteristics are estimated by measuring the property of the scattering polarizer.Because the scattering ratio of the light ray is a function of the angle difference between the polarization axes of light and the scattering polarizer, both the scattered and the transmitted portion of the light should be included when calculating the depth position of the pixels. The image at each layer is calculated as shown in Eqs. (3) and (4), where * represents the convolution operator. The experimental results of the images filtered by the scattering polarizer for each layer are shown in Fig. 2.
where imageSn and imageTn represents the planar intensity distributions of the scattered and the transmitted light at the n-th layer respectively. The initial conditions of scattering and transmitting portion of the image correspond to the projected 2D image which does not have scattering component. The observed images of each layer are transparency-weighted scattered images because the images are blurred as they pass through the layers. This behavior can be expressed as the following relationship in Eq. (5), where the transparency weight wn is given by Eq. (6), R represents a rotation matrix, P represents Jones matrix of a vertical linear polarizer, and αn denotes the polarization axis of the n-th scattering polarizer.
Fig. 2 Filtered images at each layer of the experimental system: (a) depth information, filtered images at (b) the nearest layer, (c) the middle layer, and (d) the farthest layer.
Thus, the reconstructed 3D image can be calculated by merging the retinal images of each layer with the calculated focusing position in Eq. (7) using the linear luminance weight [10, 11]. Thus, the amount of the degradation that occurs can be obtained from the degradation profile of the retinal image.
In Eq. (7), Z(x,y) indicates the observed depth position of the pixel (x, y), and zn does depth position of the n-th screen while N shows the total number of screens. Based on this analysis, we simulate the blur characteristics of the three-layer Lamina 3D display. We use an Imajor (Teijin DuPont, Japan) as the scattering polarizer, and the angular scattering characteristics of the scattering polarizer are measured with a luminance meter (LS-100, Konica Minolta, Japan) for normal and oblique light incidence. The linearly fitted coefficients for Eqs. (1) and (2) are as follows:
In the simulation, each layer is placed 4 mm from its neighbors, and the scatting axes of the scattering polarizer are 0°, 45°, and 90° in the order of the farthest to nearest layers with respect to the observer. The input image is the delta function, which has a width of one pixel. For the three cases of polarization corresponding to 0°, 45°, and 90°, the PSFs of the image are obtained under the focusing conditions of the observer according to the simulated depth calculated by Eq. (7). The resulted focusing positions are 0.4 mm, 2.8 mm, and 5.67 mm for input polarizations of 0°, 45°, and 90°, respectively. The nearest and farthest layers are located at 8 mm and 0 mm, respectively. Because of the leakage of the overlaid scattering polarizers, the selectivity of the layer is degraded. For this reason, although the polarization is set to a specific layer, some portion of lights are imaged on the other layers. As a result the luminance ratio of each layer is slightly degraded from the ideal condition and the observed focusing positions of the layers and their actual locations become different (Fig. 3).
Fig. 3 Actual and perceived position of each layer according to polarization state of projected images based on the simulation
The graph in Fig. 4(a) provides an example of the PSF for an input polarization of 45°. We assumed that the PSF is circularly symmetric so that the 2D PSF can be obtained from the one-dimensional (1D) profile of the PSF. The pixel pitch of the imaging device is assumed to be 1 µm. Thus, the 2D PSF presents a truncated image of the 1D profile.
Fig. 4 PSF of the Lamina 3D display with an input polarization of 45°: (a) 1D PSF of the system. The profile corresponds to the cross section of the 2D PSF along the red dashed line in (b). (b) 2D PSF of the system.
Based on the calculated PSF, we simulated a square image with a depth location set at the middle of the stacked layers. The input polarization was set to 45° to set the image at the middle layer. The simulated results are compared with the experimental results. The input 2D image is an entirely white square, as shown in Figs. 5(a) and 5(b). The blur observed for the experimental system is compared to the simulated data. For comparison, the images are normalized, and the difference between the original and the blurred image is determined to reduce the image noise or capturing offset of the experimental data. As shown in Fig. 5(g), the blur characteristics of the experimental and simulated results display similar trends. However, the image saturation, the uneven uniformity, the lens distortion and the oblique capturing direction of the imaging device also result in the distortion of the blur characteristics.
Fig. 5 Comparison of blur characteristics of Lamina 3D display: (a) Experimental and (b) simulated images, and (c) their horizontal intensity profiles. (d) Experimental and (e) simulated images of blur characteristics of white square obtained with a three-layer configuration, and (f) horizontal profiles of blurred image. (g) The difference between the original and the blurred profiles of the images for comparison of normalized intensity difference.
4. Experiment and result
The specifications of the experimental setup are summarized in Table 1. Another important characteristic of the Lamina 3D display is the number of laminated layers. A high number of layers are advantageous in this system because it widens the viewing angle and increases the depth range [9–11]. For the experimental setup, we measure the luminance distribution of each layer as shown in Table 2. In order to avoid the measurement of the transmitted portion, the measuring direction is deviated by 15 degrees from the normal direction of the layers. The Luminance is measured by a spectrometer (CA-210, Konica Minolta, Japan) and the measuring distance is set to 3 cm. Because the layers are closely located, we sequentially measure the luminance of each layer by inserting additional layers according to the input digital data. Because the luminance can be changed by the image data, the specification of light source, or transmittance of SLMs, the relative ratio of luminance of each layer is more important than the absolute value of luminance. For the case of three-layer configuration, the weight factor wn becomes 0.5, 0.71, and 1 for the 0̊, 45̊, and 90̊ layers, respectively. According to the Eqs. (5)–(7), the perceived positions of the layers become 3.59 mm, 4.6 mm, and 5.25 mm, respectively. The degradation of the depth representation occurs from the error of the weight factor and the leakage of the scattering polarizers which are not identical to the ideal polarizer. Also, the digital input of the system does not perfectly match to the polarization angle. Calibration of SLM and depth information can be helpful for undistorted visualization of depth.
Table 1. Specifications of the Experimental System
Table 2. Luminance Distribution over Layers
The system has a simple configuration based on the special polarizers that do not absorb but instead scatter the incident light according to the polarization angle. Because the polarization axes of the films are arranged rotationally, the amount scattered by each layer differs, producing the depth-directional intensity distribution for a certain polarization incidence. The change in intensity distribution in the depth direction can be observed as the difference in depth position. Therefore, the polarization-controlled image determined from the depth information can be represented as a relief-like 3D image. Figure 6 presents the basic scheme of the proposed system.
Fig. 6 Schematic diagram of experimental setup and prototype system: (a) conceptual diagram of system, (b) prototype system, and (c) scattering process at each layer.
The results for the experimental system are shown in Figs. 7 and 8. In Fig. 7, the stripe images used for depth calibration, the simple geometric images, and the car images are demonstrated, with side views demonstrating the parallax and reference depth images shown in the left column. The results indicate the feasibility of the proposed method with parallax in the side views. The decoding component of the system has a succinct configuration and does not require a complicated implementation. This device can also achieve scalability using large scattering polarizers, as in conventional projection-type displays. However, the scattering property of the volume produces a haze in the reconstructed images. Because the general polarization is composed of two orthogonal components, the modulated polarization states cannot be perfectly separated from each other except in the orthogonal case. This behavior results in the limited selectivity of the polarization and depth position, producing a blurry image that degrades the resolution of the reconstructed 3D image. The blur is aggravated if more scattering polarizers are inserted and sets the limits on the expressible depth range and viewing angle.
Fig. 7 Result images of Lamina 3D display. (a), (e), and (i) show the depth images for calibration, abstract depth, and computer-generated 3D car images, respectively. (b), (c), and (d) show images from left, center, and right views of the reconstructed 3D images, respectively. (f), (g), and (h) show left, center and right views of the abstract depth image, respectively. (j), (k), and (i) show left, center and right views of the computer-generated 3D object, respectively. Continuous perspectives of the reconstructed 3D images are shown in (Media 1).
Figure 8 shows additional experimental results which show the feasibility of full color expression. The parallaxes of reconstructed 3D images are more clearly observed in the result movies according to the different observing directions (Media 1, Media 2, and Media 3).
We also present the results of experimental systems with additional layers. Figure 9 compares the reconstructed images based on the number of layers in the system. The blur becomes severer as the number of layers increases in the Lamina 3D display. The polarization axes of each scattering polarizer are set to 0°, 45°, and 90° for the three-layer system; 0°, 30°, 60°, and 90° for the four-layer system; and 0°, 22.5°, 45°, 77.5°, and 90° for the five-layer system from the farthest to nearest layer with respect to the observer. The total depth is set to 12 mm for all setups, and the gap between each layer is 6 mm, 4 mm, and 3 mm for three, four and five layer systems, respectively. Although the systems with more layers achieve a wider viewing angle, the image degradation due to the blurring becomes worse as the number of layers increases. Thus, there is a limitation to the number of layers that should be employed in the system. For solving the blur characteristics, active screens which are used in Depth Cube [9] may be useful because only one scattering layer is used for imaging. However, active screens may increase the system complexity compared with the passive screens. For more effective visualization of the polarization-encoded depth, further research should be conducted.
Fig. 9 Comparison of image blur according to number of layers: (a) three layers, (b) four layers, and (c) five layers.
5. Conclusion
We proposed a projection-type depth-fused display using polarization-encoded depth information. By employing SLM and projection optics, the implemented DFD system can scale the reconstructed 3D images as a conventional projection-type display. The 2.5D characteristics of the reconstructed images are beneficial for the multi-user applications. Also, the moderated amount of the data alleviates the computation load and transmission of information as well as the acquisition of 3D information using a depth camera. We expect that the proposed method can be applied to various applications which require a large-sized screen and moderate data such as tele-conference or broadcasting systems.
Acknowledgment
This research was supported by Basic Science Research Program through the National Research Foundation of Korea (NRF) funded by the Ministry of Science, ICT & Future Planning (No. 2011-0030079). The computer graphic car objects shown in Figs. 1, 6, 8, and Media 1 are based on the model created by Natman. The human figure models shown in Figs. 5, 7 and Media 2 are based on the model created by Lunarmoonable and all images are used under Creative Commons Attribution 3.0.
References and links
1. B. Lee, “Three-dimensional displays, past and present,” Phys. Today 66(4), 36–41 (2013). [CrossRef]
2. S. A. Benton, Selected Papers on Three-Dimensional Displays (SPIE Optical Engineering Press, 2001).
3. D. M. Hoffman, A. R. Girshick, K. Akeley, and M. S. Banks, “Vergence-accommodation conflicts hinder visual performance and cause visual fatigue,” J. Vis. 8(3), 33 (2008) http://www.journalofvision.org/content/8/3/33. [CrossRef] [PubMed]
4. W. Fleming, “Vertical three-dimensional image screen,” US Patent 4,654,989 (1987).
5. M. Blackshaw, A. DeVincenzi, D. Lakatos, D. Leithinger, and H. Ishii, “Recompose: direct and gestural interaction with an actuated surface,” in Proceedings of CHI '11 Extended Abstracts on Human Factors in Computing Systems (ACM, 2011), pp. 1237–1242. [CrossRef]
6. S. Follmer, D. Leithinger, A. Olwal, A. Hogge, and H. Ishii, “inFORM: dynamic physical affordances and constraints through shape and object actuation,” in Proceedings of the 26th annual ACM symposium on User interface software and technology (ACM, 2013), pp. 417–426. [CrossRef]
7. E. B. Goldstein, Sensation and Perception (Wadsworth, 2013).
8. A. Sullivan, “DepthCube solid-state 3D volumetric display,” Proc. SPIE 5291, 279–284 (2004). [CrossRef]
9. S. Liu and H. Hua, “A systematic method for designing depth-fused multi-focal plane three-dimensional displays,” Opt. Express 18(11), 11562–11573 (2010). [CrossRef] [PubMed]
10. S. Ravikumar, K. Akeley, and M. S. Banks, “Creating effective focus cues in multi-plane 3D displays,” Opt. Express 19(21), 20940–20952 (2011). [CrossRef] [PubMed]
11. S. Suyama, S. Ohtsuka, H. Takada, K. Uehira, and S. Sakai, “Apparent 3-D image perceived from luminance-modulated two 2-D images displayed at different depths,” Vision Res. 44(8), 785–793 (2004). [CrossRef] [PubMed]
12. S. G. Park, J.-H. Kim, and S.-W. Min, “Polarization distributed depth map for depth-fused three-dimensional display,” Opt. Express 19(5), 4316–4323 (2011). [CrossRef] [PubMed]
13. J.-W. Seo and T. Kim, “Double-layer projection display system using scattering polarizer film,” Jpn. J. Appl. Phys. 47(3), 1602–1605 (2008). [CrossRef]
14. D. Armitage, I. Underwood, and S. T. Wu, Introduction to Microdisplays (Wiley, 2006).
15. P. Yeh and C. Gu, Optics of Liquid Crystal Displays (Wiley, 2010).
16. R. A. Carlton, Pharmaceutical Microscopy, (Springer, 2011), Chap. 2.
17. S. Katz, A. Tal, and R. Basri, “Direct visibility of point sets,” ACM Trans. Graph. 26(3), 24 (2007). [CrossRef]
