Open Access
Add to BibSonomyAbstract
A new super multi-view (SMV) display system that enables the number of views to be increased is proposed. All three-dimensional (3D) images generated by multiple multi-view flat-panel displays are superimposed on a common screen using a multi-projection system. The viewing zones of the flat-panel 3D display are produced in the pupils of the projection lenses and then imaged to the observation space by a screen lens. Sixteen flat-panel 3D displays having 16 views were used to construct a SMV display having 256 views. The 3D resolution was 256 × 192. The screen size was 10.3 inches. The horizontal interval of the viewing zones was 1.3 mm.
©2010 Optical Society of America
1. Introduction
A natural three-dimensional (3D) display is one that does not conflict with the human 3D perception so that it is free from visual fatigue. A super multi-view (SMV) display [1–4], which has a large number of views, was proposed as a glasses-free and natural 3D display. A high-density directional (HDD) display [5–10], where the viewing zones are generated at infinity, was also proposed. The important point regarding these two techniques is to increase the number of views to evoke the accommodation responses and to provide smooth motion parallax. In the present paper, a new SMV display system that allows the number of views to be increased is proposed.
Conventional 3D displays have two problems with respect to human 3D perception. One of these problems is the accommodation-vergence conflict [11,12]. Accommodation causes the eyes to focus on an object, and vergence perceives the depth of an object from the rotation angles of both eyes. Conventional two-view and multi-view 3D displays project different images to the left and right eyes. When two different images are presented to the left and right eyes, vergence correctly perceives the depth position of a 3D image. However, because both images are displayed on the display screen, accommodation makes the eyes focus on the display screen and not on the 3D image. Since there is a close interaction between vergence and accommodation, this conflict causes visual fatigue. The second problem is the absence or imperfection of motion parallax. Motion parallax is the change in a retinal image resulting from the movement of a viewer’s eye position. Two-view 3D displays do not generate motion parallax, and multi-view 3D displays generate discontinuous motion parallax because a retinal image does not change until the eye moves to an adjacent viewing zone. The detailed analysis of the viewing zones and the motion parallax of multi-view 3D displays is given in Ref. 13. This reduces the presence and realism of 3D images perceived by viewers, because humans unconsciously predict the retinal image change due to their movement. A natural 3D display is defined as free from these two problems.
The SMV display technique makes the interval between viewing zones smaller than the pupil diameter, so that two or more rays passing through the same point in space pass through the eye pupil simultaneously [1–3]. Therefore, the eyes can focus on that point. The HDD display technique samples ray proceeding directions with a small angle pitch to allow two or more rays to pass through the pupils simultaneously [7]. The SMV display technique produces a large number of parallax images (perspective projections of 3D scenes) into the corresponding viewing zones. The HDD display technique projects a large number of directional images (orthographic projections) with nearly parallel rays proceeding in the corresponding directions. The display systems developed to construct the SMV and HDD displays are explained in Sec. 2.
A head-mount-type SMV display also has been proposed [14]. Using this technique, the required number of views is not large. However, glasses-free observation is impossible. The SMV/HDD displays described above provide horizontal parallax. The possibility of realizing full-parallax natural 3D displays using the integral imaging technique was investigated [15].
The number of views of the glasses-free SMV/HDD displays has been increased. However, this increase results in difficulties with the previous display systems. In the present study, a new SMV display system that combines the multi-projection system and the flat-panel system is proposed to achieve a further increase in the number of views. A prototype SMV display with 256 viewpoints is also demonstrated.
2. Previous SMV/HDD display systems
Several systems have been used to construct autostereoscopic displays [16]. A multi-projection system and a flat-panel system have been used to construct the SMV and HDD displays. An SMV display with 30 views was constructed using a fan-like array of projection optics (FAPO) [4]. HDD displays with 64 and 128 ray directions [5,7,10] were constructed using a two-dimensional (2D) array of projection systems. HDD displays with 30 and 72 ray directions [6,8,9] were constructed using a flat-panel system that consists of a lenticular lens and a flat-panel display. Besides the above two systems, the display system using the focused light-source array (FLA) was developed to construct the first SMV display with 45 views [1–3]. However, development of the FLA system has not been continued.
The multi-projection system consists of a large number of projection optics and a common screen. The number of projectors is equal to the number of views. The advantage of the multi-projection system is that the resolution of 3D images and the number of views can be increased independently. The resolution can be increased by using higher resolution projectors. The number of views can be increased by using more projectors, i.e., the multi-projection system is scalable. Disadvantages of the multi-projection system include the system complexity and the system size. In addition, a large number of optical components are required, in addition to a long projection distance.
The flat-panel system consists of a high-resolution flat-panel display and a lenticular lens. Advantages of the flat-panel system include its simplicity and thickness. One disadvantage of the flat-panel system is the trade-off between the 3D resolution and the number of views. The resolution required for the flat-panel display is the product of the 3D resolution and the number of views.
In order to increase the number of views, a large number of projectors are required for the multi-projection system, and an ultra high-resolution flat-panel display is required for the flat-panel system.
3. Proposed SMV display system
In the present study, we propose a new SMV display system that does not require a large number of projectors and an ultra high-resolution flat-panel display in order to increase the number of views.
The proposed system is shown in Fig. 1 . Several flat-panel systems are combined by a multi-projection system. The flat-panel systems are arranged in a modified 2D arrangement. In the modified 2D arrangement, all of the projectors are arranged two-dimensionally with different horizontal positions. All 3D images produced by the numerous flat-panel systems are projected on a vertical diffuser, which is a common screen.
A lenticular lens of the flat-panel systems generates multiple viewing zones at a certain distance. In the proposed system, the multiple viewing zones of each flat-panel system are generated on an incident pupil plane of its corresponding projection lens. Each projection lens projects the display surface of its corresponding flat-panel system on the common screen. Each projection lens is appropriately shifted transversely along its optical axis so that all projected images are superimposed at the same position on the common screen. A screen lens, which is located on the common screen, images the exit pupils of all of the projection lenses at a certain distance from the common lens to generate viewing zones for observers. A vertical diffuser enlarges the viewing zones in the vertical direction.
Figure 2 shows the horizontal sectional view of the proposed system. The lenticular lens is shifted spatially on the flat-panel display in order to generate viewing zones on the incident pupil of a corresponding projection lens. As explained previously, the projection lenses are appropriately shifted to superimpose 3D images produced by all flat-panel systems on the common screen. The screen lens images the viewing zones of the exit pupils of the projection lenses onto the observation space to generate massive viewing zones for observers.
Figure 3 shows the vertical sectional view of the proposed system. The vertical diffuser on the common screen diffuses rays in the vertical direction so that the viewing zones generated by the projection lenses are enlarged vertically. The enlarged viewing zones overlap one another to produce a common vertical viewing zone for observers.
The arrangement of the projection lenses is illustrated in Fig. 4 . The projection lenses are also arranged in the modified 2D arrangement. The lenticular lens of the flat-panel system produces pseudoscopic viewing zones, i.e., the multiple viewing zones are repeated horizontally. The projection lenses of the proposed system have rectangular apertures to block the light that passes through the pseudoscopic viewing zones. The modified 2D arrangement makes the transparent areas of the pupils of all of the projection lenses to be continuous in the horizontal direction. Therefore, the multiple viewing zones produced by the flat-panel systems are imaged at a fixed distance from the common screen without any gaps in the horizontal direction.
The proposed SMV system is compared with the multi-projection system and the flat-panel system. The purpose of the proposed SMV system is to increase the number of views. The resolution and the number of flat-panel displays required for the three systems are shown in Table 1 . The target number of views of an SMV display is denoted by V, and the target 3D resolution is denoted by X × Y. The number of flat-panel displays used in the proposed system is denoted by L. For the multi-projection system, the required number of flat-panel displays is equal to the number of views, and the same number of projection optics is required. For the flat-panel system, despite the fact that only one flat-panel display is required and no projection optics is required, the resolution required for the flat-panel display increases in proportion to the number of views V. For the proposed SMV display system, the required number of flat-panel displays is less than the number of views, and the resolution required for flat-panel displays is proportional to V/L. Therefore, the proposed SMV display system can be constructed using a moderate number of flat-panel displays of moderate resolution.
Table 1. Requirements for flat-panel displays
In the above explanation, the flat-panel system uses a lenticular lens to produce multiple viewing zones. A parallax barrier can also be used instead of a lenticular lens. In the proposed system, the combination of offset multiple projectors and a vertical diffuser is used. This combination was previously proposed in Ref. 17 to construct multi-view displays. The proposed system uses a 2D array of lenticular displays. The combination of a one-dimensional array of time-sequential multi-view displays and a multi-projection system was previously proposed in Ref. 18.
The 3D displays that the authors previously developed employed either the multi-projection system [5,7,10] or the flat-panel system [6,8,9]. The new 3D display system proposed in this manuscript combines both systems in order to increase the number of views. To the authors’ knowledge, the combination of the multi-projection system and the lenticular system has not been proposed in any other study.
4. Prototype system
We constructed an SMV display with 256 views (SMV256) using the proposed SMV display system. Sixteen flat-panel systems with 16 views were combined by a multi-projection system.
The 16-view flat-panel system consisted of a lenticular lens and a liquid-crystal display (LCD) panel with a special subpixel layout referred to as the slanted subpixel arrangement [19]. The 2D resolution of the LCD panel was 1,024 × 768 and the screen size was 2.57 inches. The photograph of the subpixel structure of this flat-panel display is shown in Fig. 5 . Since the subpixel arrangement is slanted, the lenticular lens is not required to be slanted. The conventional multi-view displays are usually constructed by slanting the lenticular lens [20], because the subpixel layout of conventional flat-panel displays is generally the RGB stripe layout, i.e., the subpixel arrangement is not slanted. In the slanted subpixel arrangement, one of the vertical edges of one subpixel and the opposite vertical edge of another subpixel of the same color in the adjacent row occupy the same horizontal position. Therefore, ray-emitting areas of subpixels are continuous in the horizontal direction for each color. The use of the slanted subpixel arrangement has two advantages. One is that the viewing zones are produced along a horizontal line, and the other is that the crosstalk among viewing zones is theoretically zero. Using the slanted lenticular technique, the viewing zones are aligned along a slanted horizontal line, and there is considerable crosstalk among viewing zones.
In order to produce 16 viewing zones, a group of 12 × 4 subpixels (4 × 4 subpixels for each R, G, and B colors) corresponds to one of cylindrical lenses that constitute the lenticular lens. The 3D resolution of the flat-panel system was 256 × 192. The lenticular lens was designed to produce 16 viewing zones in the horizontal width of 21.0 mm at a distance of 200 mm from the lenticular lens. The horizontal pitch of the viewing zones was 1.31 mm. The lens pitch of the lenticular lens was 0.202 mm.
Next, the design issue of the multi-projection system is described. The display screen size and the pitch of the viewing zones are important parameters for designing the multi-projection system. Because the prototype system will be used for the study of suitable SMV display conditions, the pitch of the viewing zones should be less than the minimum pupil diameter (2 mm). The screen size is important because the screen size should be large enough to display 3D images in front of the display screen. The display screen size should be determined by considering the aberrations of the projection lenses because the lens shift brings about large aberrations. To evaluate the lens aberrations, the arrangement of the projection lens array was determined before designing the projection lenses by considering the size of the flat-panel systems. We decided that 16 projection lenses would be arranged in three rows. This arrangement is shown in Fig. 6 . The spot diagram of the projection lens with the maximum lens shift was evaluated to determine the screen size. The screen size was determined to be 10.3 inches, and so the magnification was 4.0. The diameter of the spot diagram was less than 0.5 mm to ensure the resolution of 256 × 192. The designed values are as follows: the focal length of the projection lens is 159.76 mm, the object length is 199.7 mm, the image length is 799.86 mm, and the image distortion is less than 0.5% at the maximum lens shift.
The horizontal pitch of the projection lenses is 21.0 mm. The projection lenses have rectangular apertures, and the widths of both the incident pupil and the exit pupil are 21.0 mm. The transparent areas are continuous in the horizontal direction.
The pitch of the viewing zones depends on the image formation by the screen lens. We determined that the observation distance is 800 mm. Thus, the magnification of the screen lens is 1.0, and the pitch of the viewing zones is 1.3 mm. The focal length of the screen is 400 mm.
The flat-panel systems were also arranged in three rows. As shown in Fig. 7 , the flat-panel systems in the upper row and the lower row were located behind the projection lenses. Behind the projection lenses in the middle row, a mirror was placed to bend the optical axes downward. The flat-panel systems in the middle row were placed on the horizontal plane. Photographs of the constructed optical engine are shown in Fig. 8 .
Fig. 8 Photographs of the constructed optical engine for SMV256: (a) projection lens array, (b) LCD arrays in the upper and lower rows, and (c) LCD array in the middle row.
A lenticular lens was used as the vertical diffuser placed at the common screen. The lenticular lens was placed so as to align cylindrical lenses in the vertical direction in order to diffuse rays vertically. A Fresnel lens was used as the screen lens. The focal length of the Fresnel lens was 400 mm.
The specifications of the SMV display with 256 views are shown in Table 2 . A photograph of the constructed SMV display is shown in Fig. 9 .
Table 2. Specifications of SMV256
To reduce the distortion and the defocus of 3D images, the multi-projection system should be aligned properly. The prototype system was designed to allow the positions of the lenticular displays and the projections lenses to be changed within ~1 mm. A grid pattern was displayed on the lenticular displays. By referring to images projected on the screen, the positions were adjusted. The alignment was performed to minimize the distortion and the defocus of each image and also to minimize the difference in all images. Figures 10(a) and 10(b) show photographs of the images projected by projection lens #7 located at the center of the lens array and projection lens #1 located at the periphery of the lens array, respectively. The lens numbers are shown in Fig. 6. The maximum image distortion in all of the projected images was 0.6%. The difference of the center positions among all projected images was 0.5 mm.
Fig. 10 Image distortion of the prototype system: (a) image projected by projection lens #7, and (b) that by projection lens #1.
5. Experimental results
In order to display 3D images using the prototype display, 256 parallax images were rendered using computer graphics software. Sixteen parallax images having consecutive horizontal parallaxes were combined in order to generate one image that was displayed on one of the flat-panel displays.
The photographs of 3D images produced by the prototype display are shown in Fig. 11 . The photographs were captured from three different horizontal positions. Movies of the 3D images are also provided to show the detailed changes of the 3D images depending on the viewing position.
The possibility of focusing on 3D images produced by the prototype display was verified. Five sets of three vertical lines were displayed at different depth positions. The depth positions of the upper three lines were 250 mm, 150 mm, and 0 mm in front of the display screen. The depth positions of the lower two lines were −400 mm and −800 mm behind the display screen. A camera was focused on each of the three lines. The aperture diameter of the camera lens was set to 5 mm, which is the average diameter of the pupil of the human eye. The captured images are shown in Fig. 12 . The minimum line width was obtained for lines on which the camera was focused. The results reveal that the prototype display has the possibility of producing 3D images on which the human eye can focus. Actually, the change in the blurring of the line patterns was observed by the human eye.
Fig. 12 Focusing on three lines at different depth positions produced by SMV256. The camera focuses on three lines at distances of (a) + 250 mm, (b) + 150 mm, (c) 0 mm, (d) −400 mm, and (e) −800 mm.
The intensity distributions of the generated viewing zones were measured. Figures 13(a) and 13(b) show the measured results for 16 viewing zones generated by projection lens #7 located at the center of the lens array and projection lens #1 located at the periphery of the lens array, respectively. The intensity distribution was measured by placing a cooled CCD camera on the observation plane. A white image was displayed to one viewing zone, in which the intensity distribution was measured, and black images were displayed to the other viewing zones.
Fig. 13 Intensity distributions of viewing zones: (a) 16 viewing zones generated by projection lens #7, and (b) those generated by projection lens #1.
The use of the vertical diffuser is essential for the proposed system. The diffusion angle of the lenticular lens used as the vertical diffuser is 35°. The difference in the vertical projection angles in the multi-projection system used for the prototype display is 4°. Therefore, the common viewing zone is not much limited by the difference of the vertical projection angles. The measured brightness of the screen is 170, 160, and 125 cd/m2 for the vertical viewing angles of 0°, 10°, and 20°, respectively.
6. Discussion
In the movies of the 3D images, variations in light intensity and color are observed in the 3D images when the viewpoint moves. Because the screen lens is a Fresnel lens, the transmittance and color depend on the ray incident angle. The images produced by the different projection lens incident on the Fresnel lens with a different vertical angle so that their light intensity and color are different. In addition, the widths of the 16 viewing zones produced by the lenticular lenses are slightly narrower than the widths of the incident apertures of the projection lenses. Therefore, additional apertures were attached in front of the projection lenses to decrease the widths of the incident apertures of the projection lenses. This also causes the drop in light intensity when the viewpoint moves.
From Fig. 13, there is considerable crosstalk among viewing zones. The crosstalk might be brought about primarily by the aberrations of the lenticular lens of the flat-panel systems and the Fresnel lens. The average pitches of the generated viewing zones were 1.2 mm and 1.1 mm for projection lenses #7 and #1, respectively. These values are smaller than the designed value. This difference might be caused by the shift of the lenticular lens.
When the slanted subpixel layout shown in Fig. 5 is used, the horizontal positions of the viewing zones are not identical for the R, G, and B colors. The difference of the horizontal positions of the viewing zones is one-fourth of the pitch of the viewing zones, because the 3D pixel consists of four rows of subpixels. However, the difference of the viewing zones is ~0.3 mm and is much smaller than the pupil diameter, so the difference did not affect the 3D images.
The 3D resolution of the prototype display is not very high. The resolution is, of course, an important issue for 3D displays. Recently, the resolution of LCD panels has been increasing. The development of LCD panels used for a super high-definition projector with the resolution of 8,000 × 4,000 was reported [21,22]. If such ultra high-resolution LCD panels were applied to the proposed system, the use of 16 projectors would provide a 3D display with a resolution of 2,000 × 1,000 and 256 views, or the use of four projectors would provide a 3D display with a resolution of 1,000 × 500 and 256 views.
The prototype display was constructed using flat-panel displays with a slanted subpixel arrangement. The rectangular apertures were placed in the projection lenses in order to eliminate the pseudoscopic viewing zones. Flat-panel displays with conventional subpixel arrangements can also be used to construct the proposed SMV display system. When flat-panel displays with the RGB stripe subpixel layout are used, lenticular lenses should be slanted [20] and parallelogram apertures should be used in the projection lenses.
7. Conclusion
A new SMV display system that combines multiple flat-panel 3D displays through a multi-projection system was proposed in order to increase the number of views of SMV displays. A large number of views can be generated using a moderate number of flat-panel displays of moderate resolution.
A prototype display with 256 views was constructed using 16 LCD panels and 16 projection lenses. The display screen size was 10.3 inches, and the horizontal pitch of the viewing zones was 1.31 mm. Three-dimensional images produced by the prototype display had smooth motion parallax. Moreover, it was possible to focus on the 3D images, which means that the accommodation function might work properly on the 3D images produced by the prototype display, so that the accommodation-vergence conflict might not occur. In the future, the accommodation responses to the prototype SMV display will be measured.
Acknolwedgements
The present study was supported by a Grant-in-Aid for Scientific Research from the Japan Society for the Promotion of Science (JSPS), No. (B) 20360153. The authors would like to thank Seiko Epson Corporation for providing the LCD panels.
References and links
1. Y. Kajiki, H. Yoshikawa, and T. Honda, “Hologram-like video images by 45-view stereoscopic display,” Proc. SPIE 3012, 154–166 (1997). [CrossRef]
2. T. Honda, Y. Kajiki, K. Susami, T. Hamaguchi, T. Endo, T. Hatada, and T. Fujii, “Three-dimensional display technologies satisfying ‘super multiview condition,” SPIE Crtical Reviews CR 76, 218–249 (2001).
3. T. Honda, Y. Kajiki, S. Susami, T. Hamaguchi, T. Endo, T. Hatada, and T. Fujii, “A display system for natural viewing of 3-D images,” in Three-dimensional television, video and display technologies, B. Javidi, F. Okano ed. (Springer-Verlag, Berlin Heidelberg, Germany, 2002) p.461–487.
4. T. Honda, D. Nagai, and M. Shimomatsu, “Development of 3-D display system by a fan-like array of projection optics,” Proc. SPIE 4660, 191–199 (2002). [CrossRef]
5. H. Nakanuma, H. Kamei, and Y. Takaki, “Natural 3D display with 128 directional images used for human-engineering evaluation,” Proc. SPIE 5664, 28–35 (2005). [CrossRef]
6. Y. Takaki, “Thin-type natural three-dimensional display with 72 directional images,” Proc. SPIE 5664, 56–63 (2005). [CrossRef]
7. Y. Takaki, “High-Density Directional Display for Generating Natural Three-Dimensional Images,” Proc. IEEE 94(3), 654–663 (2006). [CrossRef]
8. Y. Takaki, and T. Dairiki, “72-directional display having VGA resolution for high-appearance image generation,” Proc. SPIE 6055, 60550X–1-8 (2006).
9. M. Tsuboi, M. Fujioka, Y. Takaki, and T. Horikoshi, “Real Time Rendering for a Full Parallax 3D Display Using High-Density Directional Images,” in Proceedings of the 13th International Display Workshops (IDW’06), pp. 1379–1380 (2006).
10. K. Kikuta, and Y. Takaki, “Development of SVGA resolution 128-directional display,” Proc. SPIE 6490, 64900U–1 - 8 (2007).
11. L. Lipton, “Foundations of the Stereoscopic Cinema,” available for download from http://3d.curtin.edu.au/library/foundation.cfm, 1982, pp. 100–102.
12. W. A. Ijsselsteijn, H. de Ridder, and J. Vliegen, “Effects of stereoscopic filming parameters and display duration on the subjective assessment of eye strain,” Proc. SPIE 3957, 12–22 (2000). [CrossRef]
13. N. A. Dodgson, “Analysis of the viewing zone of the Cambridge autostereoscopic display,” Appl. Opt. 35(10), 1705–1710 (1996). [CrossRef] [PubMed]
14. S. K. Kim, D. W. Kim, Y. M. Kwon, and J. Y. Son, “Evaluation of the monocular depth cue in 3D displays,” Opt. Express 16(26), 21415–21422 (2008). [CrossRef] [PubMed]
15. Y. Kim, H. Choi, J. Kim, S. W. Cho, Y. Kim, G. Park, and B. Lee, “Depth-enhanced integral imaging display system with electrically variable image planes using polymer-dispersed liquid-crystal layers,” Appl. Opt. 46(18), 3766–3773 (2007). [CrossRef] [PubMed]
16. N. A. Dodgson, “Autostereoscopic 3D Displays,” Computer 38(8), 31–36 (2005). [CrossRef]
17. S. P. Hines, “Autostereoscopic video display with motion parallax,” Proc. SPIE 3012, 208–219 (1997). [CrossRef]
18. N. A. Dodgson, J. R. Moore, S. R. Lang, G. Martin, and P. Canepa, “A time-sequential multi-projector autostereoscopic 3D display,” J. Soc. Inf. Disp. 8(2), 169–176 (2000). [CrossRef]
19. Y. Takaki, O. Yokoyama, and G. Hamagishi, “Flat-panel display with slanted pixel arrangement for 16-view display,” Proc. SPIE 7237, 08–1-8 (2009).
20. C. van Berkel and J. A. Clarke, “Characterization and optimization of 3D-LCD module design,” Proc. SPIE 3012, 179–186 (1997). [CrossRef]
21. Y. Kusakabe, M. Kanazawa, Y. Nojiri, M. Furuya, and M. Yoshimura, “A high dynamic range and high-resolution projector with dual modulation,” Proc. SPIE 7241, 72410Q (2009). [CrossRef]
22. T. Nagoya, T. Kozakai, T. Suzuki, M. Furuya, and K. Iwase, “The D-ILA device for the world’s highest definition (8K4K) projection systems,” in Proceedings of International Display Workshop (IDW’08), pp. 203–206 (2008).
