1. Introduction

Through simultaneously presenting multiple perspective views (stereo-images) of a three-dimensional (3D) object, the glasses-free multiview 3D display creates multiple sub-viewing zones (SVZs), i.e. a spatial region around the viewpoint of each stereo-image. A pupil arriving at each SVZ can perceive the corresponding stereo-image, which makes the viewer perceive different stereo-image pairs as his/her left and right eyes moving into corresponding SVZs. Thus, the multiview 3D display evokes both stereo parallax and motion parallax depth cues of the viewers. This technology is compatible with existing flat two-dimensional (2D) display panels. So, the glasses-free multiview 3D display technology is developing very rapidly in recent years and begins to occupy a prominent position in 3D display area. According to mechanisms adopted for delivering each stereo-image to corresponding SVZ, the multiview 3D display techniques can be generally categorized as contact-type and projection-type [1, 2]. The contact-type system bases on thin optical plates, such as lenticular [3–5] and parallax barrier [6, 7] plates, to form SVZs, which makes it be suitable for portable or mobile 3D display products. But pixels of the flat display panel are shared by all the stereo-images, resulting in a rapid decrease of the display resolution when more SVZs are pursued. The projection-type system usually employs multiple projectors [8] or uses one fast switching projector being mated with a screen which provides optical functions for sequential stereo-images [9, 10]. Although it can provide a full display resolution, i.e. being equal to the adopted projector, it is difficult to be packaged into a small-size structure due to the necessary optical adjustment structure.

Inherently, both contact-type and projection-type multiview 3D display systems only provide a limited number of SVZs. So, the stereo-image viewed by a pupil does not change until the pupil moves to the adjacent SVZ. The motion parallax thus appears in a stepwise fashion, degrading the effectiveness of 3D displays. Overlapping adjacent SVZs by some extent may be able to alleviate this discontinuity as mentioned in ref [11], but such overlap will increase the blurriness of the displayed images [12], especially when the overlapping percent reaches 100% [13]. Furthermore, the presented light intensities in the overlapped zone are difficult to control accurately, resulting in an obvious light intensity fluctuation. The obtained continuity of motion parallax is thus at a coarse level. At present, super-multiview display technique and viewer-tracking technique have been proposed to bypass the discontinuous motion parallax issue. The super-multiview display, featured by a great number of SVZs with an interval smaller than the pupil diameter [14, 15], demands super high resolution or ultra large quantity of 2D display panels. Viewer-tracking technique [16–18] only accommodates limited viewers and also encounters difficulties for outdoor application due to complex environments.

In this paper, we propose a novel multiview 3D display system to realize continuous motion parallax by using only a moderate number of planar aligned OLED microdisplays without a viewer-tracking unit. Through appropriately offsetting multiple planar aligned lenses, stereo-images from different microdisplays overlap in a virtual displaying plane and share a common spatial-spectrum plane which serves as the observation plane P_observ. Through blocking partial microdisplay light-rays by the baffles, a complementary fusing zone (CFZ) gets generated between adjacent SVZs. When a pupil moves across the CFZ, continuously changing transitional stereo-images tiled by two segments from adjacent stereo-images are presented, and thus help to make the observed content evolving spatially from one stereo-image to its adjacent stereo-image gradually. So, discontinuous motion parallax gets overcome. Taking full advantages of the inherent large divergence angle of OLED pixels, very small light intensity fluctuation on the observing plane guarantees successful continuous motion parallax. The proposed system not only can present stereo-images with full display resolution, but also has a thin optical structure, thus offering great potentials for portable or mobile 3D display product applications.

The rest of this paper is organized as follows. In section 2, the theory of the proposed novel multiview 3D display technology for continuous motion parallax is explained. The experimental set up and results are shown in Section 3. Section 4 analyzes the spatial viewing zone along the z-direction that is provided by the proposed display system to the viewer. Section 5 provides conclusions.

2. Theory of the multiview 3D display system with continuous motion parallax

Figure 1 shows the optical structure of the proposed multiview 3D display system in the horizontal x-z plane. An array of planar aligned OLED microdisplays with a display area d_x × d_y are imaged by an array of planar aligned lenses (f). For simplicity, only two OLED microdisplays (OLED microdisplay k and k + 1) and two corresponding lenses (Lens k and k + 1) are drawn in the figure. The horizontal interval between adjacent microdisplays is denoted as d_x + 2Δ_∥. Through designing the specific offset value of each lens’s optical axis with respect to the corresponding microdisplay (δ_k for OLED microdisplay k), magnified virtual images of the messages loaded on different OLED microdisplays coincide on the P_disp as stereo-images. The magnification is determined by β = v/u, where u represents the object distance and –v denotes the image distance. Points E and F are the common marginal points of the stereo-images along the horizontal x-direction, or in other word, the EF zone represents the overlapping distribution area of the stereo-images projected from each microdisplay. The lenses are processed into rectangular shape with a horizontal size of d_x + 2Δ_∥ for seamless alignment along the x-direction. All the lenses and microdisplays are perpendicular with the z-direction. M_k-1, M_k and M_{k + 1} are the joined points of adjacent lenses. The common spatial-spectrum plane, i.e. the common focal plane of all the lenses, serves as the observation plane P_observ. A group of baffles are inserted between adjacent microdisplay-lens combination units to block the OLED microdisplay light-rays which exceed the corresponding lenses. Along the z-axis, they are confined in between the lenses and the microdisplays.

 

Fig. 1 Optical structure of the proposed multiview display system. Two OLED microdisplays are drawn here to demonstrate the proposed ideas.

Download Full Size | PDF

When only OLED microdisplay k is activated, the virtual stereo-image k is projected to the EF zone, as shown in Fig. 2(a). Due to usage of the baffles, partial light-rays of the microdisplay k which exceed the Lens k are blocked. It implies that the aperture of the Lens k plays the role of an optical pupil to the stereo-image k. The light-rays from the stereo-image k pass through the optical pupil freely, but get blocked once they exceed the optical pupil. According to geometric optics, the passing-through light-rays form two kinds of zones on the P_observ: V_(k)1V_(k)2 zone and V_(k)2V_{(k + 1)1} zone. Here V_(k)1, V_(k)2, and V_{(k + 1)1} are the intersection points of line EM_k-1, FM_k-1, and EM_k with the P_observ. For each observation points in the V_(k)1V_(k)2 zone, the whole stereo-image k is visible. This zone is denoted as sub-viewing zones k (SVZ_k). But for an observation point in the V_(k)2V_{(k + 1)1}, e.g. point A, only a segment of the stereo-image k, i.e. ED segment of the stereo-image k, is visible. That is to say, for any point in this zone, only a partial stereo-image k is visible. So, the V_(k)2V_{(k + 1)1} zone is denoted as partial-stereo-image viewing zone k (PVZ_k) here. Actually, there exist two PVZ_k zones adhering to the SVZ_k zone from two sides.

 

Fig. 2 Optical structures of the proposed multiview 3D display system with only two microdisplay-lens combination units for simplicity: (a) Only OLED microdisplay k being activated by stereo-image k; (b) Only OLED microdisplay k + 1 being activated by stereo-image k + 1.

Download Full Size | PDF

Similarly, when only OLED microdisplay k + 1 gets activated by the stereo-image k + 1, a SVZ_k+1 zone where the whole stereo-image k + 1 is visible and two PVZ_{k + 1} zones where only a partial stereo-image k + 1 is visible get generated on the P_observ, as shown in Fig. 2(b). Due to the seamless planar alignment of the Lenses k and k + 1, between SVZ_k and SVZ_k+1, a PVZ_k zone and a PVZ_{k + 1} zone overlap with each other completely. This overlapping zone is defined as the complementary fusing zone k~k + 1 (CFZ_{k~k + 1}) in this paper. For an observation point A in the CFZ_{k~k + 1}, the DF segment of stereo-image k + 1 gets visible.

Based on above discussions, when the microdisplays k and k + 1 are activated simultaneously by the stereo-images k and k + 1, respectively, the ED segment of stereo-image k and the DF segment of stereo-image k + 1 are presented to point A simultaneously, as shown in the left part of the Fig. 3. The two segments link up at the point D seamlessly, spatially tiling up a transitional stereo-image. The nomination of the CFZ is based on the spatially complementary characteristics of the two segments. Under this condition, two SVZs and one CFZ constitute a horizontal viewing zone in a spatial end to end manner. The center points of each SVZ, i.e. VP_k and VP_{k + 1} in Fig. 2, are taken as the viewpoints of the corresponding stereo-images. For example, the stereo-image k is the projection view of the target object converging to VP_k. As shown in Fig. 1, the target object is virtually located between point E and F around the P_disp. The horizontal sizes of the SVZ and CFZ zones are calculated geometrically as:

 

Fig. 3 The spatially changing transitional stereo-images observed by a moving observation point at different positions of a CFZ. The observation points A and A' are as denoted in the Fig. 2(b).

Download Full Size | PDF

For a point in the SVZ zone, light-rays passing through the point are from all displayed pixels of the corresponding stereo-image. For a point in the CFZ zone, light-rays passing through the point are from all displayed pixels of the corresponding transitional stereo-image. Because the transitional stereo-image is tiled by two complementary segments from two adjacent stereo-images, it has a resolution equal to the stereo-image and the number of light-rays passing through these two kinds of points is identical. The light emission from each OLED pixel inherently has a very large divergence angle. After transmitting through the lens with a limited aperture, each light ray presents an approximately homogeneous light intensity distribution on the P_observ. Thus, for any point in the horizontal viewing zone, a uniform light intensity distribution gets guaranteed.

The joint point D of the transitional stereo-image is in fact the intersection point of the line AM_k with the P_disp, which does linkage movement with the observation point but toward the reverse direction. For example, when the observation point A moves toward A' along the negative direction of x-axis, the joint point D shifts toward D' along the positive direction, as shown in Fig. 2(b). As to the observed transitional stereo-image, the segment from the stereo-image k shrinks to ED' zone and the segment from the stereo-image k + 1 expands to D'F zone, as shown in Fig. 3. That is to say, for an observation point moving along the CFZ zone, the spatial ratio of the observed two complementary segments from different stereo-images changes from 1:0 to 0:1 gradually. Consequently, a continuously changing transitional stereo-image gets realized for a moving observation point. With the help of the continuously changing transitional stereo-image, the observed image will evolve from a complete stereo-image k to a complete stereo-image k + 1 in a spatial “point by point” manner when the observation point moves from SVZ_k to SVZ_{k + 1}. As discussed above, the light intensity distribution in the horizontal viewing zone is approximately homogeneous. So the observed image shows no obvious light intensity fluctuation for a moving observation point.

Actually, the pupil of a viewer is an aperture, not a point as discussed above. Replacing the observation point by a pupil of diameter 2d, the joint point D will grow into a region, denoted as a blending region (BR) in this paper. As shown in Fig. 4, for a pupil in the CFZ_{k~k + 1} with marginal points B₁ and B₂ along the horizontal direction, the marginal points (D₁ and D₂) of the BR will be on the extension lines of B₁M_k and B₂M_k. Outside the BR, each point in the ED₂ or D₁F presents the image content of the stereo-image k or k + 1 to the pupil, respectively. However, within the BR, the presented content of each point is a hybrid of the two stereo-images’ contents by a spatially varied percent. Specifically, the presented content of a point Q in the BR is

 

Fig. 4 Optical diagram showing the observed transition stereo-image when the pupil is covered by a CFZ.

Download Full Size | PDF

In fact, when a pupil moves from the viewpoint k to its adjacent viewpoint k + 1, there are three kinds of situations. In situation I, the pupil is completely covered by the SVZ_k and the whole stereo-image k is observed. In situation II, the pupil straddles SVZ_k and CFZ_{k~k + 1}, as shown in Fig. 5. The intersection point of the line B₂M_k with P_disp is denoted as D₃. Here, B₂ is the marginal point of the pupil in the CFZ_{k~K + 1}. The transitional stereo-image is built up by two segments. A segment, denoted as ED₃, presents contents of the stereo-image k. The complementary segment, D₃F, works as the BR. For a point Q' in the BR, the presented content is expressed as:

 

Fig. 5 Optical diagram showing the observed transition stereo-image when the pupil straddles the adjacent SVZ and CFZ.

Download Full Size | PDF

Equations (2)–(5) mathematically indicate that the content of each display point presented to a moving pupil experiences a timed gradual process as the BR sweeps across. In other words, for example, the point D₃ in Fig. 5 is at the upper edge of the BR, it presents the content of stereo-image k. When the pupil translates a distance of 2d (the diameter of the pupil) along the negative direction of x-axis, the point D₃ becomes the lower edge of the BR, i.e. “D₁” in Fig. 4 of the situation III. The presented content changes to that of the stereo-image k + 1. With the pupil’s movement, the content presented by the point D₃ changes from the content of the stereo-image k to that of the stereo-image k + 1 timed gradually.

In summary, all above analysis clearly shows that the observed changes continuously from the stereo-image k to the stereo-image k + 1 when the pupil moves from a viewpoint k to the adjacent viewpoint k + 1. This evolution process takes place not only by a spatial “point by point” way for the whole observed image, but also by a timed gradual way for the presented content of each display point of the observed image.

Introducing more OLED microdisplays into the proposed system, more SVZs and CFZs will be formed. En route continuously changing transitional stereo-images, a multiview 3D display with continuous motion parallax will be implemented if only all linked SVZs and CFZs can cover two pupils of a viewer. In the case of 2m + 1 microdisplays with (2m + 1) + 1 baffles being inserted, numbered as $-m$, $-(m-1)$, ···, $0$, ···$(m-1)$, $m$, there will be 2m + 1 SVZs and 2m CFZs constructing the horizontal viewing zone (VZ) of the system. Symmetrically, lets the offset δ₀ = 0. As shown in the Fig. 6, the mid-point of the microdisplay 0, C₀, and the mid-point of the line-segment EF, C, are on the optical axis of the lens 0. To guarantee the overlapping of images of all OLED microdisplays, the mid-point of the marginal microdisplay m, C_m, should be imaged to the point C by the corresponding Lens m. According to the geometry relationship, the line CC_m passes through the optical center of the Lens m. The maximum offset of the microdisplay m with respect to the corresponding lenses can be expressed geometrically as:

 

Fig. 6 Optical diagram showing the offsetting of the optical axis of the Lens m with respect to the corresponding microdisplay.

Download Full Size | PDF

The rectangular lenses are cut down from a group of mother lenses with identical characteristics. To collect light-rays between adjacent baffles, along the horizontal direction, the aperture size of the mother lens should not be less than:

To obtain a thin optical structure, a small f is preferred. The parameters$2{\Delta }_{\parallel }$ and $\beta$ have small influences on the number 2m + 1 mathematically. So, the parameters N.A. and $d_x$ play key roles on the value of 2m + 1.

As discussed above, 2m + 1 SVZs and 2m CFZs construct a horizontal VZ with a size of (2m + 1)L_NF + 2mL_F. Thus, the viewing angle of the 3D object displayed by the proposed system, i.e. the field angle of the VZ to the EF zone, can be calculated approximately by:

3. Experiments and results

A display system is set up to implement the idea described above, as shown in Fig. 7. Nine white OLED microdisplays, with a display area d_x × d_y = 10.08 × 7.56mm² and a resolution 800 × 600, from OLEiD of China are used. To show the inner structure more clearly, four of the ten baffles are removed for photograph. Since each OLED microdisplay device is packaged with a driving board individually, its mechanical size is 22 × 17mm². Aligning microdisplays side by side, the value of 2Δ_∥ reaches 11.92mm. According to Eq. (1), the width of the sub-viewing zone L_NF is much larger than the pupil diameter, which is about 3mm at indoor environments. The pupil will receive an invariant stereo-image until it moves out of a SVZ, which will deteriorate continuous motion parallax. In order to reduce the width of the SVZ, nine OLED microdisplays are arranged into two parallel rows alternatively. The vertical interval between the two rows is set as d_y + 2Δ_⊥ = 17.6mm, a little larger than the vertical mechanical size of the microdisplay, which makes sure adjacent microdisplays (one in the upper row and another in the lower row) have no spatial conflict along the horizontal direction. Thus, the horizontal interval between adjacent microdisplays is reduced to 13mm in favor of a smaller width of the SVZ.

 

Fig. 7 Photograph of the experimental display system.

Download Full Size | PDF

To guarantee overlapping of all stereo-images, the optical axises of the two rows of lenses should be arranged by an offset δ_y = ± (8.8/(β-1))mm with respect to the vertical geometrical center of the corresponding microdisplays. The vertical viewing zones (VZ_u and VZ_l) of microdisplays belonging to different rows exhibit misplacement. Their overlapped region defines the vertical range (VZ_⊥) of the viewing zone where the whole stereo-image or transitional stereo-image are visible, as shown in Fig. 8.

 

Fig. 8 Schematic diagram showing the vertical viewing zone of the display system.

Download Full Size | PDF

Choosing achromatic lenses with an effective aperture 45mm, i.e. N.A = 0.75, as mother lenses, 9 kinds of rectangular parts, with a size of 13 × 36mm² to satisfy Eq. (7), are cut out from these mother lenses and processed into the rectangular lenses for usage in our experiment. Their geometrical centers are 0,$\pm (d_x+2{\Delta }_{\parallel })/(\beta -1)$,$\pm 2(d_x+2{\Delta }_{\parallel })/(\beta -1)$, $\pm 3(d_x+2{\Delta }_{\parallel })/(\beta -1)$ and $\pm 4(d_x+2{\Delta }_{\parallel })/(\beta -1)$ away from the geometric center of the mother lenses along the horizontal direction, respectively.

Other system parameters include f = 60mm, u = 160/3mm. Under this condition, β = 9, v = $-$480mm, L_NF = 3.285mm, L_F = 11.34mm, the vertical range of the VZ is 9.72mm and the available size of the stereo-image is 90.72 × 68.04mm². The thickness of the optical structure is as small as 65mm. For symmetric consideration, the displayed 3D object is set as 68 × 68 × 68mm³. The horizontal range of the VZ is 120.285mm, which is about 1.9 times as large as the average interocular distance (64mm) of a viewer. According to Eq. (9), a viewing angle of about 13° is reached by the proposed system.

A uniform light intensity distribution on the P_disp is a necessary condition for the proposed system. The inherent light emission characteristics of OLED, i.e. large divergence angle, guarantees this point. With all the microdisplays being active and all the pixels being set at the maximum intensity value, the light intensity distribution in the viewing zone is measured at a step interval of 3mm along the x-direction by a luminance meter CS-2000A from Konicaminolta. Figure 9 shows the measured values and a small light intensity fluctuation (<2%) is confirmed. So, intensities of the observed different stereo-image or transitional stereo-images are nearly uniform. The perceived image for a moving pupil is free from intensity fluctuation.

 

Fig. 9 Measured light intensities along the horizontal center line of the viewing zone.

Download Full Size | PDF

To exhibit how a transitional stereo-image is tiled spatially, a verification experiment is performed. A research-type CCD (SenSys 1602E) is put in the CFZ_0~1 to capture the presented image. For a transitional stereo-image from OLED microdisplay k and k + 1, if only one of them being activated at each time, the presented images are fragments of the transitional stereo-image. Figure 10(a) and 10(b) show the captured images when one of microdisplays 0 and 1 is activated by its corresponding stereo-image. Figure 10(c) shows the captured image when both of them are activated. Obviously, Fig. 10(c) is the spatial tilling of the former two. In our experiment, the diaphragm of the CCD is always set as 3mm, which is the average pupil diameter at indoor environments.

 

Fig. 10 Captured transitional stereo-images at a observation position in the CFZ_0~1 when (a)only the microdisplay 0 is active, (b) only the microdisplay 1 is active and (c)both of them are active.

Download Full Size | PDF

Two pyramids are displayed to demonstrate the proposed idea and system. Let the CCD be at ten positions with an equal interval along the horizontal direction in the VZ, captured images are shown in Fig. 11. A more intuitive feeling can be obtained through seeing the online multimedia recorded by a camera translating at a constant speed along the horizontal direction (Media 1). When the BR sweeps across the SVZs, the change rate of the observed transitional stereo-image becomes slow, as can be seen in the multimedia, which is due to the blank parts (i.e. no message) in the display zone for the stereo-images and the existence of the SVZs. This phenomenon can be alleviated by adopting full filled stereo-images and SVZs with a small size. Whatever, the change is still continuous. Five subjects observed the displayed 3D image in the lab environment and no obvious discontinuous motion parallax was perceived.

 

Fig. 11 Captured images with a spatial interval of 13mm along the horizontal direction when the proposed display system works. The labels on each image denote the shooting position, with 0mm representing the midpoint of the viewing zone. A more intuitive feeling can be found in the online multimedia (Media 1).

Download Full Size | PDF

4. Discussions on the spatial range of the VZ along the z-direction

In above paragraphs, the discussed SVZs and CFZs are confined in the observing plane P_observ. Actually, the SVZs and CFZs also occupy some zone space along the z-direction around the P_observ, as shown in Fig. 11. Here, only three lenses are drawn for simplicity. When the pupil deviates away from the P_observ along the negative z-direction, the SVZs get enlarged while the CFZs shrink. As a result, the changing rates of the observed transitional stereo-image will be sometimes faster and sometimes slower. This phenomenon can be seen more clearly in the online multimedia (Media 1). Inversely, if the pupil deviates away from the P_observ along the positive z-direction, this uneven changing rate gets remediated by some extent, but crosstalk noises from adjacent stereo-images will appear when the pupil is just centered at the viewpoints, i.e. the center points of each SVZ.

Once the pupil leaves away the plane determined by the points M'_k-1, M'_k, M'_{k + 1}, a new kind of CFZs gets generated. CFZ_k-1~k~k+1 in the Fig. 12 is shown as an example. For an observation point in the CFZ_{k-1~k~k + 1}, the observed transitional stereo-image is constituted by three segments from adjacent stereo-image k-1, k and k + 1, respectively. Although the presented transitional stereo-image still changes in a timed continuous manner under this condition, the difference between segments from the stereo-images k-1 and k + 1 becomes even more obvious, resulting in deterioration of the accuracy of the displayed 3D image. Therefore, in order to display the 3D image more accurately, the region between P_observ and the plane determined by the points M'_k-1, M'_k, M'_{k + 1} is taken as the preferred observing zone along the z-direction, which is about 20.3mm in our prototype system.

 

Fig. 12 Schematic diagram showing the spatial expansion of the SVZs and CFZs along the z-direction.

Download Full Size | PDF

Another problem that needs to be addressed is the wavefront aberrations from the processed lenses, especially when the lenses have larger offsets with respect to their corresponding microdisplays. Experimentally, anti-distortion through a correction Table [19] is performed to alleviate the wavefront aberration for the lenses with larger offsets. The procedure of the anti-distortion correction can be illustrated by Fig. 13. A dot pattern, as shown in Fig. 13(a), functions as the target image and is loaded onto one microdisplay with all other microdisplays being shut down at the same time. The projected virtual image, as shown in Fig. 13(b), is captured at the corresponding SVZ. Through analyzing the distortion of the captured image, a correction table is generated. An anti-distorted image is obtained based on the correction table, as shown in Fig. 13(c). Loading the anti-distorted image onto this microdisplay, a corrected image gets displayed, as shown in Fig. 13(d). This process is performed for each microdisplay-lens pairs and all stereo-images are anti-distorted through establishing their respective correction tables.

 

Fig. 13 Correction of the image distortion by the electronic method.

Download Full Size | PDF

5. Conclusions

In conclusion, a novel multiview 3D display with continuous motion parallax and full display resolution are realized through controllable spatial-spectrum fusing of light-rays from planar aligned OLED microdisplays. The proposed system needs only a thin optical structure (65mm) offering great potential for portable or mobile 3D display applications provided that the related driving and control systems can be integrated into a monolithic chip. Compared with existing technologies to bypass the discontinuous motion parallax issue in multiview 3D display, i.e. super-multiview display technique and viewer-tracking technique, the novel technology proposed in this paper can get implemented with a moderate number of 2D display panels without using viewer-tracking unit. This endows our system with higher practicability.

Limited by N.A = 0.75 of the used lenses and the mechanical size of the OLED microdisplay, 9 OLED microdisplays are used by the prototype system in this manuscript and a viewing angle of 13° is reached experimentally. If OLED microdisplays with a smaller size d_x are available and lenses with a larger N.A. are used, more natural motion parallax and wider viewing angles can be obtained through the proposed system. This is the focus of our future work.