Expand this Topic clickable element to expand a topic
Skip to content
Optica Publishing Group

Crosstalk reduction in stereoscopic 3D displays: Disparity adjustment using crosstalk visibility index for crosstalk cancellation

Open Access Open Access

Abstract

Stereoscopic displays provide viewers with a truly fascinating viewing experience. However, current stereoscopic displays suffer from crosstalk that is detrimental to image quality, depth quality, and visual comfort. In order to reduce the perceived crosstalk in stereoscopic displays, this paper proposes a crosstalk reduction method that combines disparity adjustment and crosstalk cancellation. The main idea of the proposed method is to displace the visible crosstalk using the disparity adjustment in a way that less amounts of intensity leakage occur on perceptually important regions in a scene. To this purpose, we estimate a crosstalk visibility index map for the scene that represents pixel-by-pixel importance values associated with the amount of perceived crosstalk and negative-after-effects of the crosstalk cancellation. Based on the crosstalk visibility index, we introduce a new disparity adjustment method that reduces the annoying crosstalk in processed images, which is followed by the crosstalk cancellation. The effectiveness of the proposed method has been successfully evaluated by subjective assessments of image quality and viewing preference. Experimental results demonstrate that the proposed method effectively improves the image quality and overall viewing quality of stereoscopic videos.

© 2014 Optical Society of America

1. Introduction

Stereoscopic 3D (S3D) display is a very powerful technology that provides the viewer with a truly unique visual experience. Since our eyes are located at different horizontal positions, the left- and right-eye images are slightly different. The differences in the horizontal positioning of objects in the left- and right-eye images provide disparities, which allow for the human visual system (HVS) to perceive the relative depth of objects. S3D display mimics this situation by presenting the viewer with two slightly different perspective images. The various S3D display technologies differ mainly in how the view separation is made [1,2].

When the view separation is imperfect in S3D display, a small proportion of one eye image is seen by the other eye as well. This unexpected light leakage from one image to the other view image (or the degree of the light leakage) is called as system crosstalk, which is related with the optical performance of S3D display [3,4]. Whereas, perceived crosstalk refers to the system crosstalk perceived by the viewer [3]. The system crosstalk results in mismatches between the left- and right-view images in terms of luminance, color, and structure, etc, which induce binocular asymmetries [5]. As a perceptual consequence of the crosstalk, human subjects perceive ghosts, shadows, or double contours [3,68].

It has been widely acknowledged that a high level of the system crosstalk significantly affects the overall viewing experience. In particular, the crosstalk negatively affects not only image quality, but also depth quality and visual comfort [3,611]. In literature, the crosstalk is known as one of the most detrimental factors that affects the image quality [3,6,7]. In addition, a high level of system crosstalk in S3D displays reduces binocular fusion limit [9], perceived depth magnitude [10], and even visual comfort [8]. In order to provide the viewer with a high viewing experience of S3D contents, reducing the perceived crosstalk is indeed a very essential task. Thus far, display manufacturers have dedicated significant efforts to minimize the system crosstalk as much as possible. However, the perfect separation of left- and right-view images is still not always possible given a diversity of S3D display characteristics [3,11,12]. Therefore, in order to provide a high viewing quality in S3D displays, the development of a tool mitigating the perceived crosstalk is essential.

In the field of image processing, a few number of methods have been proposed to reduce the perceived crosstalk, focusing on crosstalk cancellation (also known as anti-crosstalk) [3,1315]. The purpose of crosstalk cancellation is to hide the system crosstalk by a pre-distortion of S3D images before displaying them [3,14]. For instance, a simple crosstalk model [15,16] can be represented as M'I(x,y) = MI(x,y) + αMU(x,y), where MI(x,y) and MU(x,y) denote intensity values at (x, y) of intended and unintended view images, respectively. In addition, α denotes the level of system crosstalk (0 ≤ α ≤ 1). As such, αMU(x,y) quantifies the intensity leakage at (x, y). In this paper, we refer to αM as the leakage image. The crosstalk cancellation subtracts the amount of expected intensity leakage from the intended image as follows: MI(x,y) − αMU(x,y). In this way, the perceived crosstalk can be significantly suppressed, while the system crosstalk is still presented.

Naturally, the crosstalk cancellation fails when an intended view image is very dark and the amount of intensity leakage from an unintended view image is very large [14,15,17]. If the intensity of the intended view image is smaller than that of leakage, the system crosstalk cannot be compensated since intensity values cannot become any negative ones (see Figs. 1(a) and 1(b)). In this case, the crosstalk cancellation cannot hide the system crosstalk, and thus the crosstalk is still perceivable as shown in Fig. 1(c). Note that hereinafter uncorrectable regions refer to the regions where the system crosstalk cannot be compensated by the crosstalk cancellation.

 figure: Fig. 1

Fig. 1 Crosstalk cancellation with intensity mapping methods: (a) the original image with crosstalk, (b) uncorrectable regions (red color) by the crosstalk cancellation [3,1416], (c) crosstalk cancellation without an intensity mapping, (d) crosstalk cancellation with the global intensity mapping [14,15], and (e) crosstalk cancellation with the local intensity mapping [15].

Download Full Size | PDF

Existing crosstalk cancellation methods [14,15,17] have addressed such problems by manipulating the intensity of a scene, allowing for full compensation of the system crosstalk. Konrad et al. [14] proposed to raise the minimum intensity level of a whole image (i.e., global intensity mapping). Whereas, local intensity mapping methods [15,17] were also proposed to suppress the system crosstalk while preserving the dynamic range of the scene as much as possible. These approaches locally raise the intensity of uncorrectable regions. By raising the intensity, the system crosstalk can be fully compensated. However, this could negatively affect the overall viewing quality. As shown in Fig. 1(d), the global intensity mapping method can significantly reduce the dynamic range of a scene. In addition, the local intensity mapping methods can induce visible image artifacts such as halo artifacts around the uncorrectable regions (see the regions around the lamp in the Fig. 1(e)). Indeed, there exists a trade-off between the amount of perceived crosstalk and image artifacts caused by the crosstalk cancellation methods. For instance, any partial compensation of the system crosstalk may mitigate the loss of image quality caused by the intensity mapping while still lowering the visibility of system crosstalk to a certain acceptable level. However, balancing such a trade-off is not always feasible given a diversity of scene content characteristics.

A fundamental solution to mitigate the negative effects of crosstalk cancellation methods is to reduce the perceived crosstalk occurred on uncorrectable regions. Indeed, the intensity leakage on those regions can dominantly affect the overall viewing quality and is likely to cause the negative after-effects of crosstalk cancellation. In particular, the intensity leakage makes larger relative intensity changes on the uncorrectable regions than those on the other regions, and hence the visibility of system crosstalk could be more increased, which is in line with Weber’s law [18] (details will be discussed in Section 2.1). In addition, in the same sense, the intensity mapping for those regions can be more visible since the intensity mapping results in larger relative changes in intensity than other regions. For this reason, manipulating intensity of the uncorrectable regions can negatively affects the viewing quality of S3D contents as shown in Figs. 1(d) and 1(e).

Accordingly, in order to improve the viewing quality of S3D contents, it is essential to quantify the visibility of system crosstalk on the uncorrectable regions and manipulate the contents to mitigate their negative effects according to their crosstalk visibility. However, it is not a trivial problem since the visibility of crosstalk cannot be simply quantified by the level of system crosstalk. The visibility of crosstalk is strongly affected by scene content characteristics [3,6,7]. For instance, although the same level of system crosstalk is occurred on an entire image, the crosstalk is more visible around dark backgrounds of a 3D object with bright textures, possibly due to human’s intensity sensitivity (see Fig. 1(a)). Indeed, humans have a limited ability to distinguish the difference in intensities, well known as Weber’s law [18]. This implies that, given the level of system crosstalk, perceptual significance of the uncorrectable regions can be different according to the scene content characteristics. In addition, the extent of uncorrectable regions is also related with disparity magnitude (i.e., the amount of disparity) as shown in Figs. 2(a)-2(c). As such, by analyzing the content characteristics affecting the crosstalk visibility on the uncorrectable regions and by adjusting disparities of the regions according to their visibility, we can effectively reduce not only the amount of perceive crosstalk on those regions, but also the negative after-effects of crosstalk cancellation.

 figure: Fig. 2

Fig. 2 The effect of disparity magnitude on the extent of uncorrectable regions. Note that the uncorrectable regions are marked in red color. In these examples, disparity magnitude of the lamp decreases from the left to right figures.

Download Full Size | PDF

The main contribution of this paper is to propose a crosstalk reduction method that combines disparity adjustment and crosstalk cancellation for displaying S3D videos. In particular, we introduce a method to guide the disparity adjustment that reduces perceptually important regions, where the visibility of crosstalk is more apparent, by analyzing scene content characteristics. The proposed disparity adjustment aims at displacing the perceivable crosstalk regions in a scene in a way that less amounts of intensity leakage occur on the perceptually important regions. To this purpose, disparity shifting is used to adjust the extent of uncorrectable regions, particularly, which have dominant effect on the overall crosstalk visibility. Note that the disparity shifting alters the zero disparity plane of a scene while maintaining its disparity range [19]. In this way, we can reduce the amount of perceived crosstalk and the negative after-effects of crosstalk cancellation as well. To guide the disparity adjustment process, we introduce the crosstalk visibility index that quantifies the severity of crosstalk by considering the scene content characteristics. After the disparity adjustment process, crosstalk cancellation is further applied to fully compensate the remaining visible crosstalk.

The proposed crosstalk reduction method was also demonstrated by conducting subjective assessment experiments of the overall image quality and viewing preference. The results showed that the proposed method provided higher image quality than existing approaches of crosstalk cancellation while well preserving the overall viewing quality of a scene.

The rest of this paper is organized as follows: In Section 2, we present the overall process of the proposed crosstalk reduction method, which consists of an estimation of the crosstalk visibility index, disparity adjustment using the crosstalk visibility index, and crosstalk cancellation processes. Section 3 presents validation experiments that evaluate the performance of the proposed crosstalk reduction method. Finally, conclusions are drawn in Section 4.

2. Proposed crosstalk reduction method

As aforementioned, the main idea of the proposed crosstalk reduction method is to displace the visible crosstalk regions in a scene so that less amounts of intensity leakage occur on perceptually important regions before the crosstalk cancellation. In this paper, the perceptually important regions refer to the regions where the system crosstalk is highly likely to be more visible than other regions in the scene. In order to quantify the visibility of system crosstalk in a scene and displace the crosstalk regions according to the crosstalk visibility, the proposed crosstalk reduction method mainly consists of three consecutive processes: crosstalk visibility index estimation, disparity adjustment using the crosstalk visibility index, and crosstalk cancellation. Each processing step is described in the following sections.

2.1 Crosstalk visibility index estimation

In order to guide the disparity adjustment process, we first estimate the crosstalk visibility index. The crosstalk visibility index quantifies the visibility of system crosstalk occurred on uncorrectable regions in a scene, which may have the dominant influence on the overall perceived crosstalk and cause the negative after-effects of crosstalk cancellation. It is worthy to note that the visibility of system crosstalk (i.e., the amount of perceived crosstalk) is not completely dependent on the level of system crosstalk. Given the system crosstalk, the crosstalk visibility varies with scene content characteristics [3,6,7]. For instance, the crosstalk visibility can increase with decrease in the intensity of an intended image as discussed in the Introduction section. As such, in order to estimate the crosstalk visibility index, it is essential to consider not only the amount of intensity leakage, which is determined by the level of system crosstalk and intensity of the unintended view image, but also the influence of scene content characteristics on the visibility of system crosstalk. In this paper, a visibility weight is defined to consider the scene content characteristics in the estimation of the crosstalk visibility index. The visibility weights quantify the perceptual importance of regions according to their contribution to the crosstalk visibility. In particular, given the amount of leakage L at the n-th frame in a stereo video, the crosstalk visibility index Q(n) is estimated as follows:

Q(n)=1|R|x,yRL(x,y,n)V(x,y,n),
where Q(n) denotes the crosstalk visibility index at n-th frame in a stereo video, L(x,y,n) denotes the intensity leakage at a pixel position (x,y), and V(x,y,n) denotes the visibility weight at the position (x,y). This visibility weight quantifies how the visibility of system crosstalk differs according to the scene content characteristics. In addition, R denotes a set of pixel positions corresponding to the uncorrectable regions and |R| denotes the number of pixels in R. Notably, the crosstalk visibility index is used as a guidance of the disparity adjustment process to mitigate the negative after-effects of crosstalk cancellation, which is affected by the severity of perceived crosstalk on the uncorrectable regions. For this reason, the crosstalk visibility index is estimated from the uncorrectable regions R. In sum, the crosstalk visibility index Q(n) quantifies the amount of perceived crosstalk occurred on the uncorrectable regions.

In order to estimate the visibility weight map V, we consider two important content characteristics that affect the perceived crosstalk: intensity of the intended image and temporal changes of the crosstalk image, which we will describe as follows. Each of them is quantified as a specific feature map for the image and is combined into the visibility weight map.

Intensity of the intended image: It has been generally agreed that the system crosstalk is more visible around high contrast 3D objects/regions (e.g., bright textures against dark backgrounds) [3,6]. With the presence of disparity, the intensity leakage from the bright regions in the unintended view image is separated from the original image and makes the perceivable change of intensity as shown in Fig. 3. In addition, increase in the intensity of the bright regions also makes the system crosstalk more visible since the amount of intensity leakage occurred on those regions also becomes larger (see the examples in Figs. 3(a) and 3(b)). However, although the system crosstalk is occurred on an entire image, the crosstalk is not always visible. This is due to the fact that the visibility of system crosstalk is related with human’s ability to discriminate intensity differences. In literature, it has been well known that humans are more sensitive to relative intensity change than absolute change, which is in line with the Weber’s law [18]. This implies that, given an amount of intensity leakage, its visibility increases as the intensity of the intended image becomes lower, which makes larger relative intensity difference from the original intensity (see the examples in Figs. 3(a) and 3(c)). Accordingly, for the quantification of crosstalk visibility, both the amount of intensity leakage and intensity of the intended image should be considered.

 figure: Fig. 3

Fig. 3 Different visibility of system crosstalk: (a) high intensity object, (b) lower intensity object than (a), and (c) object with brighter background than (a). Note the objects in (a) and (c) have the same intensity, but the system crosstalk in (a) is more visible that that in (c).

Download Full Size | PDF

In literature, it has been well known that human’s ability to detect intensity changes is inversely proportional to the original intensity, and the subjective sensation of intensity is proportional to the logarithm of the intensity [18]. Inspired by the Weber’s law, we consider the intensity of the intended image as one of feature maps to estimate the crosstalk visibility index as follows:

VS(x,y,n)=log(MI(x,y,n)+γ),
where MI (x,y,n) denotes the intensity value at (x,y) in the n-th intended image of a stereo video. In addition, γ determines the slop of the log function. In our experiment, γ was set to 2.39 as used in [20]. Equation (2) implies that, given the amount of intensity leakage, the crosstalk visibility increases as the intensity of the intended image becomes lower.

Temporal changes of the leakage image: The temporal aspects of the perceived crosstalk are often neglected. However, temporal changes of the leakage image in stereo videos may strongly affect the visibility of system crosstalk (see the Introduction section for the definition of the leakage image). In particular, with the presence of system crosstalk, high-contrast moving 3D objects/regions are accompanied by the temporally changing leakage image across consecutive frames (see the crosstalk occurred on the tire in the examples of Fig. 4(a)). The fast change of the leakage image on the moving object may be perceived as flickering, and thus increase the visibility of system crosstalk. Indeed, in literature, temporal flickering (i.e., onset and offset of light intensity) is known as one of conspicuous features in a scene [21]. In addition, temporal changes of visible crosstalk regions in depth direction can also increase the visibility of system crosstalk. For instance, in-depth motion can cause sudden changes of the extent of visible crosstalk regions as shown in Fig. 4(b). Also, the dynamic increase in the object size (i.e., the crosstalk regions in our case) could make the object more conspicuous than other regions in a scene [22]. In sum, given a level of system crosstalk, the crosstalk visibility can be also affected by the temporal changes of leakage image across consecutive frames. In this paper, we quantify the temporal changes of leakage image as the absolute difference in intensities between consecutive frames as follows:

VT(x,y,n)=|MU(x,y,n)MU(x,y,n1)|,
where MU (x,y,n) and MU (x,y,n-1) denote the intensity values at (x,y) of the n-th and n-1-th unintended images in the stereo video, respectively.

 figure: Fig. 4

Fig. 4 The effect of temporal changes of leakage image: (a) change of leakage image on a moving object (see the tire) and (b) change of the extent of crosstalk regions.

Download Full Size | PDF

Lastly, the visibility weight map is estimated by the combination of the extracted feature maps as follows:

V=wSVS+wTVT,wherewS+wT=1,
where V'S and V'T denote the normalized feature maps of VS and VT, respectively. Note that each feature map has different extraction mechanisms and dynamic ranges. Hence, perceptually important regions in a feature map can be concealed by another map after the combination. For this reason, before combining the two different maps, we normalized each feature map using the normalization operator commonly used in visual saliency estimation [23], which promotes local maxima in the map. Also note that, in this paper the equal weights (i.e., wS = wT) were used under the assumption that different attributes contribute independently to the visibility of the crosstalk. Consequently, V quantifies the relative perceptual importance in a scene that affects the visibility of system crosstalk. Given an amount of intensity leakage, the regions with higher values in V represent that the system crosstalk occurred on those regions is highly likely to be more visible.

Figure 5 shows an example of the visibility weight map. Figure 5(a) is the original image with system crosstalk. From the figure, we can observe that the crosstalk occurred on the region around dark and/or moving objects (see the dashed-line rectangle) is more visible than the other regions (see the solid-line rectangle). The zoomed-in images of Fig. 5(a) and the corresponding original images are also presented in Figs. 5(b) and 5(c), respectively. As shown in the visibility weight map presented in Fig. 5(d), the dashed-line rectangle has relatively large visibility weights, while the solid-line rectangle has relatively small visibility weights in the map. This can be also observed in the crosstalk visibility index map shown in Fig. 5(e). Note that, in Fig. 5(e), the regions other than the uncorrectable regions were set to zero since the crosstalk visibility index is estimated from only those regions (see Eq. (1)). The details of the crosstalk visibility index will be explained with Eq. (6) in the following section.

 figure: Fig. 5

Fig. 5 Crosstalk visibility weight map: (a) image with system crosstalk, (b) zoomed-in images of (a), (c) original zoomed-in images corresponding to (b), (d) crosstalk visibility weight map, and (e) crosstalk visibility index map. Note that, in the visibility weight map, the larger values (i.e., white) represent higher visibility weights. In addition, in the visibility index map, the larger values (i.e., white) represent that the system crosstalk is more visible around those regions. In this figure, the dashed-line rectangles have larger visibility weights than that of the solid-line rectangle. As shown in (b), the crosstalk is more visible around larger visibility weights (see the short pants), which is also indicated in (e).

Download Full Size | PDF

2.2 Disparity adjustment using the crosstalk visibility index

Under the guidance of the crosstalk visibility index, the proposed disparity adjustment method displaces visible crosstalk regions in a scene in a way that high amounts of intensity leakage occur on perceptually less important regions. Specifically, the proposed disparity adjustment method performs the disparity shifting to reduce the uncorrectable regions, particularly, where high amounts of intensity leakage occur and where visibility weights are high. Recall that the visibility weights quantify the perceptual importance of the regions in a scene (see the explanation with Eq. (1)). The disparity shifting changes the maximum and minimum disparities of a scene while the disparity range of the scene is preserved. This is simply performed by shifting all pixels of the left- and right-view images in the opposite directions. Since the disparity shifting changes the position and extent of uncorrectable regions as shown in Figs. 6(a)-6(c), selecting an appropriate value of disparity shift can prevent that high amounts of intensity leakage occur on those regions.

 figure: Fig. 6

Fig. 6 Change of uncorrectable regions with varying amounts of disparity shifting: (a) original image (left-view), (b) the image shifted by 0.2 degrees towards the uncrossed disparity direction, and (c) the image shifted by 0.4 degrees towards the uncrossed disparity direction. Note that the uncorrectable regions are marked in light red color. The horizontal boundaries of the images in (b) and (c) were cropped due to the disparity shifting.

Download Full Size | PDF

To decide the proper disparity shift value, the proposed method varies the shift value and, at each time, estimates the crosstalk visibility index. Specifically, the disparity shift value for the n-th frame is obtained as follows:

H(n)=argminhQh(n),forhmin<h<hmax,
where Qh(n) denotes the crosstalk visibility index given a disparity shift value h at the n-th frame. hmin and hmax denote the pre-defined minimum and maximum shift values, respectively (More details are described in Section 3.1). The disparity shift values are searched within a limited range in order to ensure that excessive crossed and uncrossed screen disparities do not occur. Indeed, excessive screen disparity is one of major sources of visual discomfort in the viewing of S3D images [24,25]. Note that a positive shift value of H(n) represents the disparity shifting towards the crossed disparity direction, and a negative shift value represents the disparity shifting towards the uncrossed disparity direction.

The proposed method changes the zero disparity plane of a scene to the one where the uncorrectable regions are induced, particularly, where high amounts of intensity leakage occur and where crosstalk is highly likely to be more visible. Thus, if the uncorrectable regions have crossed disparities, the disparities of the scene are shifted toward the uncrossed disparity direction, which moves the scene depth range farther away from a viewer, and vice versa for the other case. The proposed method was motivated by that if such manipulation of depth information does not negatively affect the overall viewing quality, improvement of the image quality by the disparity adjustment can be beneficial for providing better overall viewing quality.

When there are several uncorrectable regions with different disparities in a frame, the proposed method performs the disparity shifting to change the zero disparity plane to the one which induces relatively large extent of uncorrectable regions and/or where the uncorrectable regions have the dominant effect on the overall crosstalk visibility. Due to the nature of the disparity shifting that displaces the crosstalk regions across different disparity planes (see Fig. 6), this could increase the amount of perceived crosstalk on other uncorrectable regions, but which have less effect on the overall perceived crosstalk. In order to ensure that such trade-off is beneficial, the proposed method estimates the crosstalk visibility index with varying disparity shift values and confirms that the disparity shifting can decrease the overall perceived crosstalk in the scene.

As mentioned in Section 2.1, the crosstalk visibility index quantifies the visibility of system crosstalk by considering the perceptual importance of regions in a scene. Given a disparity shift value h, the crosstalk visibility index at the n-th frame is quantified as follows:

Qh(n)=1|Rh|x,yRhαMUh(x,y,n)Vh(x,y,n),
where α denotes the level of system crosstalk (0 ≤ α ≤ 1), MUh(x,y,n) denotes the linearized intensity value (i.e., converted with 2.2 of gamma value as used in [15]) at (x,y,n) of the unintended image shifted by h pixels. Notably, αMUh(x,y,n) denotes the amount of intensity leakage at (x,y,n). In addition, Vh(x,y,n) denotes the visibility weight value at the corresponding position. Rh denotes the uncorrectable regions where the system crosstalk cannot be compensated. Also, | Rh | denotes the number of pixels of the regions.

In order to ensure continuous disparity shifting across consecutive frames, the shift values obtained using Eq. (5) need to be temporally smoothed, which is essential to provide a high viewing quality of S3D video [26]. However, it should be noted that the temporal smoothing can average out the shift values, and hence smoothing the shift values could hinder the reduction of perceived crosstalk using the disparity shifting. For this reason, the proposed method first applies a moving maximum filter, which is also known as the dilation filter [27], in order to enlarge the shift values before smoothing them. Specifically, for each frame, the moving maximum filter replaces the disparity shift value by the one that yields the largest amount of disparity shifting within a certain range from the current frame. Note that, in order to sufficiently smoothen the shift values, in this paper the size of filters was set to a large value (e.g., 30 frames in our experiment).

An example of the processed image using the proposed disparity adjustment is presented in Fig. 7. Figure 7(a) shows the original image with system crosstalk and Fig. 7(b) shows the processed image using the proposed disparity adjustment method. The disparity shifting that minimizes the crosstalk visibility index displaces the visible crosstalk occurred on perceptually important regions (see the dashed-line rectangle in the figure) to less important regions (see the solid-line rectangle in the figure). As shown in Fig. 7(b), the perceived crosstalk occurred on the less important region (i.e., balloon) is less visible and thus less annoying. More importantly, for those regions, the intensity raised for the crosstalk cancellation is also relatively less visible. Figures 7(c) and 7(d) also show the disparity ranges (i.e., maximum and minimum disparity) of the original stereo video and the processed videos, respectively. In these figures, the red vertical line represents the frame index where Figs. 7(a) and 7(b) were captured. The crosstalk visibility index of the original and processed videos is also presented in Fig. 7(e). In this figure, x-axis represents a frame index and y-axis represents the crosstalk visibility index. Apparently, from the figure, we can observe that the crosstalk visibility index of the processed video is relatively decreased compared to that of the original video.

 figure: Fig. 7

Fig. 7 Examples of the processed image (left-view): (a) the original image with perceived crosstalk (Media 1) and (b) processed image using the proposed disparity adjustment (Media 2), (c) disparity range of the original stereo video, (d) disparity range of the processed stereo video, and (e) comparison of the crosstalk visibility index between the original and processed stereo video. Note that, in (c) and (d), red vertical line represents the frame index where (a) and (b) were captured. Also, note that the disparity shifting in the proposed method displaces the perceived crosstalk occurred on perceptually important regions (see dashed-line rectangles in the figures) to less important regions (see solid-line rectangles in the figures). As shown in (b), the perceived crosstalk occurred on the less important region (i.e., balloon) is less visible and thus less annoying. Note that horizontal boundaries of the image in (b) were cropped due to the disparity shifting.

Download Full Size | PDF

2.3 Crosstalk cancellation

The purpose of the disparity adjustment process is to reduce the perceived crosstalk by displacing visible crosstalk regions, but not to fully suppress the crosstalk. As such, an additional processing step is further required to suppress the remaining visible crosstalk. For this purpose, the crosstalk cancellation process is applied to the stereo video produced using the disparity adjustment. In particular, in order to benefit from the proposed disparity adjustment method, we adopt the crosstalk cancellation with the local intensity mapping (i.e., local contrast reduction) [15]. As mentioned in the Introduction, this approach locally raises intensity of uncorrectable regions for crosstalk cancellation so that the dynamic range of an image can be preserved as much as possible. Note that since the disparity adjustment process does not completely eliminate the uncorrectable regions, the use of global intensity mapping [14] cannot elaborate the advantage of the disparity adjustment.

An example of the processed images is presented in Fig. 8. Figures 8(a) and 8(b) show the output images processed using the crosstalk cancellation methods with the global and local intensity mappings, respectively. In addition, Fig. 8(c) shows the image produced using the proposed method (i.e., the combined disparity adjustment and crosstalk cancellation). From the figures, we can observe that the proposed method provides much higher image quality by displacing crosstalk regions prior to the crosstalk cancellation. In addition, from Figs. 7(b) and 8(c), we can observe that reducing the uncorrectable regions significantly mitigates the negative effects of the intensity mapping for the crosstalk cancellation. Indeed, the intensity changes are more visible on the perceptually important regions with higher visibility weights (see the dashed-line rectangles in the figures) than other regions (see the solid-line rectangles in the figures).

 figure: Fig. 8

Fig. 8 Examples of the processed image: (a) crosstalk cancellation with the global intensity mapping [14], (b) crosstalk cancellation with the local intensity mapping [15], and (c) the proposed method that combines the disparity adjustment and crosstalk cancellation with the local intensity mapping. Note that horizontal boundaries of the image in (c) were cropped due to the disparity shifting.

Download Full Size | PDF

3. Experiment and result

In this section, we evaluate the validity of the proposed crosstalk reduction method. The main purpose of the evaluation is to demonstrate that the proposed disparity method can provide high quality of the viewing experience while reducing the perceived crosstalk. For the purpose of this study, we conducted subjective assessment experiments in which a panel of viewers assessed the overall viewing quality of S3D videos. In this section, we first present the subjective assessment procedure. Next, we present and discuss the results of the subjective assessment experiments.

3.1 Experimental environments

For the evaluation of the crosstalk reduction methods, we used 16 stereo videos that were highly likely to induce high amounts of perceived crosstalk. In particular, 11 synthetic and 5 natural scenes were used to cover a wide range of depth structure and image contrast, which affect the visibility of system crosstalk. The synthetic scenes were generated using a computer graphics tool (3DSMax®). The stereo videos were carefully generated to maximize the visibility of crosstalk. In particular, the objects appeared in the scenes have high levels of luminance/color contrast, large disparity magnitudes, and various motion characteristics. The synthetic scenes had a spatial resolution of 1280 × 720 pixels and were captured with 30 fps for 10 seconds. In addition, for the natural scenes, multi-view sequences from the MPEG [28,29] were used as in [6]. In our viewing environments, the maximum screen disparity of the stereo videos was 1.18 ± 0.36 degrees (mean ± standard deviation (SD)) and the range of the screen disparity was 0.93 ± 0.39 degrees. Figure 9 shows examples of the stereoscopic videos used in our experiments.

 figure: Fig. 9

Fig. 9 Stereoscopic 3D videos used in our experiments.

Download Full Size | PDF

All stereo videos were displayed on a half-mirror type S3D monitor (Redrover SDM-400®, a 40 inch linear polarized 3D display). The crosstalk levels of the S3D monitor were 0.75% and 0.27% for the left- and right-eye, respectively, as measured in [5]. In addition, the levels of gray-to-gray crosstalk [3, 3032], which is a measure of the system crosstalk with different gray levels, were 0.27 ± 0.22% and 0.20 ± 0.14% (mean ± SD) for the left- and right-eyes, respectively. As mentioned in [32], the crosstalk levels were measured with 6 × 6 gray-to-gray combinations. The gray levels used were 0, 50, 100, 150, 200, and 255 [32]. Note that the crosstalk levels were measured at the center point of the monitor by using a spectroradiometer (Minolta CS-1000®) placed behind the left and right lens of the glasses, based on previous studies [3,6,33]. Considering that the crosstalk level was lower than the visibility threshold reported in literature (about 1 to 2%) [3], [34], we assumed that the system crosstalk of the display did not affect our experimental results. A spatial resolution of the display was 1920 × 1080 pixels. All experimental environments followed the recommendations of ITU-R BT.2021 [35]. A total of 25 viewers participated in our experiments. Three subjects failed the Titmus Stereofly test and thus excluded from the experiments. As a result, 22 subjects participated subjective assessments. All of them had normal or corrected-to-normal vision and a minimum stereopsis of 60 arcsec (in the Stereofly test) [36]. Their average age was 25.4 years, ranging from 21 to 33.

For the use of crosstalk cancellation (see Section 2.3), each color channel of the input S3D videos was simply converted into linear values ranged from 0 to 1 using 2.2 of gamma value, which is commonly used in most displays. In addition, 5% of system crosstalk was simulated, as used in [15]. Note that the level of the crosstalk around 5% significantly affects the image quality and visual comfort [3]. For each stereo video, the search range of disparity shift values, i.e., hmin and hmax in Eq. (4), was selected for the maximum and minimum values of screen disparity to be fallen within ± 1.5 degrees. Given the disparity range, subjects did not report any visual discomfort due to excessive screen disparity magnitude as observed in [37]. The proposed method was implemented in MATLAB (version 7.13) on a 3.5 GHz dual-core PC. Given our experimental conditions and stereo videos, the average computation time of the proposed disparity adjustment was about 0.732 seconds per frame. Specifically, the computation time was about 0.018 seconds for the visibility weight map generation and 0.714 seconds for the disparity adjustment using the crosstalk visibility index. Most of computation time for the disparity adjustment was dedicated to a grid search of the disparity shift value. For real-time implementation of the proposed method in 3D display systems, the use of more sophisticated searching strategy can significantly reduce the computational load.

3.2 Subjective assessment method

In order to demonstrate the validity of the proposed crosstalk reduction method, we conducted subjective assessment experiments and compared the results with those of the existing crosstalk cancellation methods. To measure the viewing quality of the processed S3D videos, the subjects were instructed to assess two perceptual aspects: image quality and viewing preference.

The purpose of image quality assessment is to measure how well the proposed method preserves the image quality of stereo videos. The image quality was measured using the double stimulus continuous scale (DSCQS) method, as recommended in ITU-R BT.2021 [35]. The DSCQS method consists of randomly displaying two versions of the same video, where one is the reference video and the other is the test video. In our experiment, the reference video was always the processed stereo video with the current crosstalk cancellation methods, whereas the test image was the processed stereo video with the proposed method. The presentation of each video lasted for 10 seconds followed by 5 seconds of resting time with a mid-gray image. For the subjective assessment, the subjects were instructed to grade the image quality using a continuous scale divided into five segments, identified by five verbal labels (excellent, good, fair, poor, and bad) [35]. For analysis, the continuous ratings were converted to scores in the range of 0-100, where the higher values indicate better image quality.

For each test condition, the opinion scores were used to compute the difference mean opinion score (DMOS). DMOS is the average of all difference scores, where each difference score is obtained by subtracting the subjective score of the reference video from that of the corresponding test video. As such, positive values of DMOS indicate that the test video (i.e., the processed video with the proposed method) provided better image quality than the reference video.

In addition, the viewing preference of the processed stereo videos was also assessed to evaluate the overall viewing experience considering all perceptual quality aspects [38]. To measure the viewing preference, the subjects were instructed to answer “Which one do you prefer to see considering all quality aspects of viewing experience? Answer as A and/or B” [38].

3.3 Subjective assessment results

Figure 10 shows the DMOS of image quality. Figure 10(a) is the measured DMOS between the proposed method and the crosstalk cancellation with the global intensity mapping, and Fig. 10(b) is the measured DMOS between the proposed method and the crosstalk cancellation with the local intensity mapping. In these figures, the x-axis represents a video index and the y-axis represents the corresponding average DMOS. The reliability test for the subjective assessment results indicated a Cronbach’s α [39] of 0.965. Note that the reliability of psychometric test scores is generally regarded as acceptable if Cronbach’s α is higher than 0.7 [39].

 figure: Fig. 10

Fig. 10 Measured DMOS of image quality: (a) the proposed method vs. crosstalk cancellation with the global intensity mapping and (b) the proposed method vs. crosstalk cancellation with the local intensity mapping. Note that higher DMOS values represent that the proposed method provides higher image quality.

Download Full Size | PDF

From these figures, we can observe that the proposed method provided higher image quality than the existing crosstalk cancellation methods. The statistical results of the subjective assessment for visual comfort are also presented in Table 1.As shown in the table, compared with the existing crosstalk cancelation methods, the improvement of the overall image quality was statistically significant for both existing approaches. Note that the significance level was calculated using a paired t-test with a null hypothesis that there was no improvement of image quality of stereo videos. In addition, the improvement was higher against the crosstalk cancellation with the global intensity mapping method (p < 1.7e−3). In line with the previous study in [15], the global intensity mapping that reduces the dynamic range of a scene could more negatively affect the image quality when the system crosstalk is very severe.

Tables Icon

Table 1. Statistical results of subjective assessment for image quality.

In sum, the experimental results revealed that the proposed approach that combines the disparity adjustment and crosstalk cancellation provided a significantly higher improvement in the image quality. In particular, the results demonstrated that the proposed disparity adjustment method well mitigated the negative after-effects of the crosstalk cancellation under the guidance of the crosstalk visibility index.

In order to improve the image quality, the proposed crosstalk reduction method performs the disparity shifting ahead of the crosstalk cancellation. A reasonable concern is that this disparity adjustment process might negatively affect other perceptual aspects of stereo videos, such as visual comfort and depth quality. Accordingly, to verify the suitability of our approach, we measured the viewing preference of the processed videos, which is closely related to the overall viewing quality as mentioned in previous studies [38].

Figure 11 shows the assessment results of the viewing preference. Figure 11(a) shows the comparison between the proposed method and the crosstalk cancellation with the global intensity mapping, and Fig. 11(b) shows the comparison between the proposed method and the crosstalk cancellation with the local intensity mapping. In these figures, y-axis represents the percentage of answers for each condition. It can be seen from the figures that the proposed method was preferred in most cases (i.e., 90% and 82% against the crosstalk cancellation with the global and local intensity mappings, respectively). In particular, for the stereo videos with severe levels of system crosstalk like the ones used in this study, the disparity adjustment did not have significant negative effect on the overall viewing quality. Overall, the results in Figs. 10 and 11 indicate that the proposed crosstalk reduction method that combines the disparity adjustment and crosstalk cancellation is capable of improving image quality while preserving the overall viewing quality of stereo videos.

 figure: Fig. 11

Fig. 11 Comparison of the assessment results for viewing preference: (a) the proposed method vs. crosstalk cancellation with the global intensity mapping and (b) the proposed method vs. crosstalk cancellation with the local intensity mapping.

Download Full Size | PDF

Examples of the processed videos are presented in Fig. 12.As discussed earlier, the proposed method could achieve better image quality by avoiding the occurrence of severe crosstalk on perceptually important regions (e.g., see the man in these figures). As shown in the figures, the proposed method well mitigated the negative after-effects of the local intensity mapping used for the crosstalk cancellation. Additional examples of the processed video are presented in Fig. 13. When the backgrounds of a scene are very dark, the intensity changes produced by the intensity mapping methods are more visible and thus significantly degrade the image quality. As shown in the figure, the proposed approach can be more beneficial for such a case. The crosstalk visibility indices of the original and processed videos using the proposed method are also presented in Figs. 14(a) and 14(b), respectively. Note that Figs. 14(a) and 14(b) correspond to the stereo videos presented in Figs. 12 and 13, respectively. The figures show that the crosstalk visibility index of the processed video is relatively decreased compared to that of the original video.

 figure: Fig. 12

Fig. 12 Examples of the processed videos (left-view): (a) original video (Eiffel tower) (Media 3), (b) crosstalk cancellation with the global intensity mapping (Media 4), (c) crosstalk cancellation with the local intensity mapping (Media 5), and (d) the proposed method (Media 6). Note that horizontal boundaries of the images in (d) were cropped due to the disparity shifting.

Download Full Size | PDF

 figure: Fig. 13

Fig. 13 Additional examples of the processed videos (left-view): (a) original video (Pantomime) (Media 7), (b) crosstalk cancellation with the global intensity mapping (Media 8), (c) crosstalk cancellation with the local intensity mapping (Media 9), and (d) the proposed method (Media 10).

Download Full Size | PDF

 figure: Fig. 14

Fig. 14 Crosstalk visibility index of the processed stereo videos: (a) Eiffel tower and (b) Pantomime.

Download Full Size | PDF

For stereo videos with frequent and discontinuous disparity changes, the disparity shifting that minimizes the crosstalk visibility index of all of each frame can cause abrupt changes of the zero disparity plane across consecutive frames. This may lead to unnatural percepts of the processed stereo videos. As described in Section 2.2, in order to mitigate the negative effects of the proposed method, inherited from the disparity shifting, we applied a temporal smoothing filter to the disparity shift values before using them. In particular, as described in Section 2.2, the proposed method applied a moving maximum filter and average filter in order, using the filter size of 30 frames. In this way, given our experimental conditions, we could reduce the perceived crosstalk while well mitigating the negative effects. For the stereo videos that have more frequent and discontinuous disparity changes than those used in our study, enlarging the filter size can be effective to mitigate the negative effects caused by the disparity shifting.

3.4 Discussion

The purpose of the proposed disparity adjustment was to displace some crosstalk occurrence regions in a scene so that high amounts of system crosstalk mainly occur on perceptually less important regions in terms of the crosstalk visibility. To this purpose, we utilized the disparity shifting under the guidance of the crosstalk visibility index. In this way, we could effectively mitigate the negative after-effects of the existing crosstalk cancellation method. However, due to the nature of the disparity shifting that moves the crosstalk regions from one to another regions, there is a chance that the proposed method cannot improve the viewing quality of S3D contents if the same amount of the perceived crosstalk occurred on the all over the depth space. For further quality improvement in such an extreme case, non-linear disparity adjustment methods [2] could be also used under the guidance of the crosstalk visibility index. The use of non-linear disparity adjustment is capable of locally decreasing the disparity of perceptually important regions, while mitigating the perceived crosstalk and negative after-effects of the crosstalk cancellation. However, such application requires some complex optimization process to preserve the original depth structure of a scene and high computational complexity for rendering S3D videos. For this reason, in this paper, we utilized the disparity shifting as the disparity adjustment method, which might be a more practical and feasible solution for consumer electronic devices (such as S3D television and S3D mobile display) due to its low processing complexity.

Due to the proposed approach that adjusts disparities of the scene, the artistic creativity of a producer can be overruled, particularly in terms of a depth script of a scene. However, we would like to point out that providing high image quality is also very essential to maintain the artistic effect intended by the producer. For instance, halo artifacts and reduction in the dynamic range of a scene, caused by the existing crosstalk cancellation methods, can significantly reduce the overall viewing quality of a scene even though the original depth information of the scene is preserved. Indeed, with the existence of crosstalk in 3D displays, perfect maintenance of the artistic effect may not be possible. As such, the proposed method is motivated by that as long as the disparity adjustment does not negatively affect the overall viewing quality, improvement in the image quality can be beneficial for the overall viewing quality of stereo videos. That is, the proposed disparity adjustment implies the trade-off between preservation of image and depth information of the scene. This trade-off is beneficial, particularly for stereo videos with high levels of crosstalk like the ones used in this study as shown in the subjective assessment results for the viewing preference.

In this paper, in order to estimate the visibility weight map, the same weights (i.e., wS = wT in Eq. (4)) were used. However, the use of different weights for different contents may yield further improvement in the performance of the proposed crosstalk reduction method. This is because the perceptual significance of content characteristics that affect the amount of perceived crosstalk may be different depending on contents. For instance, for the stereo videos that hardly induce temporal changes of leakage image, the amount of perceived crosstalk could be dominantly affected by the intensity of intended image. In this case, the use of a larger weight for the intensity of intended image (i.e., wS) than that of the temporal change of leakage image (i.e., wT) may provide better prediction of the perceived crosstalk. As such, in order to improve the performance of the proposed method, it will be worthy of investigating a perceptual model that describes such relation with extensive psychophysical experiments.

4. Conclusions

In this paper, we proposed a crosstalk reduction method using a combined disparity adjustment and crosstalk cancellation for displaying S3D videos. In particular, we introduced the disparity adjustment that displaced perceivable crosstalk regions in a scene in a way that less amounts of the perceived crosstalk occurred on the perceptually important regions. To guide the disparity adjustment process, we quantified the severity of crosstalk occurred on uncorrectable regions, where system crosstalk cannot be compensated by the crosstalk cancellation, as the crosstalk visibility index by considering the scene content characteristics. After the disparity adjustment process, crosstalk cancellation process was performed to fully reduce the remaining visible crosstalk. In this way, we could reduce the amount of perceived crosstalk and effectively mitigate the negative after-effects of crosstalk cancellation as well. The proposed crosstalk reduction method was demonstrated by subjective assessments of image quality and viewing preference. The results showed that the proposed method could provide higher image quality than existing crosstalk cancellation approaches while preserving the overall viewing quality of S3D videos. We believe that the proposed method can be incorporated into various stereoscopic displays considering its simplicity and effectiveness in the reduction of perceived crosstalk.

References and links

1. P. Benzie, J. Watson, P. Surman, I. Rakkolainen, K. Hopf, H. Urey, V. Sainov, and C. Kopylow, “A survey of 3DTV displays: techniques and technologies,” IEEE Trans. Circ. Syst. Video Tech. 17(11), 1647–1658 (2007). [CrossRef]  

2. H. Sohn, Y. J. Jung, S.-i. Lee, F. Speranza, and Y. M. Ro, “Visual comfort amelioration technique for stereoscopic image: disparity remapping to mitigate global and local discomfort causes,” IEEE Trans. Circ. Syst. Video Tech. (to be published).

3. A. J. Woods, “Crosstalk in stereoscopic displays: a review,” J. Electron. Imaging 21(4), 040902 (2012). [CrossRef]  

4. K.-C. Huang, C.-H. Tsai, K. Lee, and W.-J. Hsueh, “Measurement of contrast ratios for 3D display,” Proc. SPIE 4080, 78–86 (2000). [CrossRef]  

5. Y. J. Jung, H. Sohn, S. I. Lee, Y. M. Ro, and H. W. Park, “Quantitative measurement of binocular color fusion limit for non-spectral colors,” Opt. Express 19(8), 7325–7338 (2011). [CrossRef]   [PubMed]  

6. L. Xing, J. You, T. Ebrahimi, and A. Perkis, “Assessment of stereoscopic crosstalk perception,” IEEE Trans. Multimed. 14(2), 326–337 (2012). [CrossRef]  

7. P. J. H. Seuntiëns, L. M. J. Meesters, and W. A. IJsselsteijn, “Perceptual attributes of crosstalk in 3D images,” Displays 26(4–5), 177–183 (2005). [CrossRef]  

8. F. L. Kooi and A. Toet, “Visual comfort of binocular and 3D displays,” Displays 25(2–3), 99–108 (2004). [CrossRef]  

9. Y.-Y. Yeh and L. D. Silverstein, “Limits of fusion and depth judgment in stereoscopic color displays,” Hum. Factors 32(1), 45–60 (1990). [CrossRef]   [PubMed]  

10. I. Tsirlin, L. M. Wilcox, and R. S. Allison, “The effect of crosstalk on the perceived depth from disparity and monocular occlusions,” IEEE Trans. Broadcast 57(2), 445–453 (2011). [CrossRef]  

11. M. Barkowsky, S. Tourancheau, K. Brunnström, K. Wang, and B. Andrén, “Crosstalk measurement of shutter glasses 3D displays,” in Proceedings of SID Symposium Dig. Tech. Pap. 42(1), 812–815 (2011). [CrossRef]  

12. C. Lee, G. Seo, J. Lee, T. H. Han, and J. G. Park, “Auto-stereoscopic 3D displays with reduced crosstalk,” Opt. Express 19(24), 24762–24774 (2011). [CrossRef]   [PubMed]  

13. J. Lipscomb and W. Wooten, “Reducing crosstalk between stereoscopic views,” Proc. SPIE 2177, 92–96 (1994). [CrossRef]  

14. J. Konrad, B. Lacotte, and E. Dubois, “Cancellation of image crosstalk in time-sequential displays of stereoscopic video,” IEEE Trans. Image Process. 9(5), 897–908 (2000). [CrossRef]   [PubMed]  

15. C. Doutre and P. Nasiopoulos, “Crosstalk cancellation in 3D video with local contrast reduction,” in Proceedings of European Signal Processing Conference (2011).

16. Y. C. Change, C. Y. M, and Y. P. Huang, “Crosstalk suppression by image processing in 3D display,” in Proceeding of SID Symposium.41(1), 124–127 (2010). [CrossRef]  

17. J. V. Baar, S. Poulakos, W. Jarosz, D. Nowrouzezahrai, R. Tamstorf, and M. Gross, “Perceptually-based compensation of light pollution in display systems,” in Proceedings of ACM Symposium on Applied Perception in Graphics and Visualization (2011).

18. M. W. Levine and J. M. Shefner, Fundamentals of Sensation and Perception (Brooks/Cole, 1991).

19. Y. J. Jung, H. Sohn, S.-i. Lee, and Y. M. Ro, “Visual comfort improvement in stereoscopic 3-D displays using perceptually plausible assessment metric of visual comfort,” IEEE Trans. Consum. Electron.submitted.

20. W. Frei, “Image enhancement by histogram hyperbolization,” Comput. Gr. Image Process. 6(3), 286–294 (1977). [CrossRef]  

21. L. Itti, “Automatic foveation for video compression using a neurobiological model of visual attention,” IEEE Trans. Image Process. 13(10), 1304–1318 (2004). [CrossRef]   [PubMed]  

22. S. L. Franconeri and D. J. Simons, “Moving and looming stimuli capture attention,” Percept. Psychophys. 65(7), 999–1010 (2003). [CrossRef]   [PubMed]  

23. L. Itti, C. Koch, and E. Niebur, “A model of saliency-based visual attention for rapid scene analysis,” IEEE Trans. Pattern Anal. Mach. Intell. 20(11), 1254–1259 (1998). [CrossRef]  

24. M. Lambooij, W. A. IJsselsteijn, and I. Heynderickx, “Visual discomfort and visual fatigue of stereoscopic displays: a review,” J. Imaging Sci. Technol. 53(3), 030201 (2009). [CrossRef]  

25. H. Sohn, Y. J. Jung, S. I. Lee, and Y. M. Ro, “Predicting visual discomfort using object size and disparity information in stereoscopic images,” IEEE Trans. Broadcast 59(1), 28–37 (2013).

26. M. Lang, A. Hornung, O. Wang, S. Poulakos, A. Smolic, and M. Gross, “Nonlinear disparity mapping for stereoscopic 3D,” ACM Trans. Graph. 29(4), 1–10 (2010). [CrossRef]  

27. I. Pitas and A. N. Venetsanopoulos, “Order statistics in digital image processing,” Proc. IEEE 80(12), 1893–1921 (1992). [CrossRef]  

28. ISO/IEC JTC1/SC29/WG11, “Description of exploration experiments in 3D video coding,” Doc. N9991 (2008).

29. Tanimoto Laboratory FTV test sequences, available: http://www.tanimoto.nuee.nagoya-u.ac.jp.

30. S. Shestak, D.-S. Kim, and S.-D. Hwang, “Measuring of gray-to-gray crosstalk in a LCD based time-sequential stereoscopic displays,” in Proceedings of SID Symposium Dig. Tech. Pap. 41(1), 132–135 (2010). [CrossRef]  

31. C.-C. Pan et al, “Cross-talk evaluation of shutter-type stereoscopic 3D display,” in Proceedings of SID Symposium Dig. Tech. Pap.41(1), 128–131 (2010). [CrossRef]  

32. D.-H. Kang, E.-J. Lee, J.-H. Lee, and J.-K. Song, “Perceptual strength of 3-D crosstalk in both achromatic and color images in stereoscopic 3-D displays,” IEEE Trans. Image Process. 21(7), 3253–3261 (2012). [CrossRef]   [PubMed]  

33. J.-C. Liou, K. Lee, and J.-F. Huang, “Low crosstalk multi-view tracking 3-D display of synchro-signal LED scanning backlight system,” J. Display Technol. 7(8), 411–419 (2011). [CrossRef]  

34. L. Chen, Y. Tu, W. Liu, and Q. Li, “Investigation of crosstalk in a 2-view 3D display,” in Proceedings of SID Symposium Dig. Tech. Pap. 39(1), 1138–1141 (2008). [CrossRef]  

35. ITU-R BT. 2021, “Subjective methods for the assessment of stereoscopic 3DTV systems,” (2012).

36. A. M. Norcia, J. Hale, M. W. Pettet, S. P. McKee, and R. A. Harrad, “Disparity tuning of binocular facilitation and suppression after normal versus abnormal visual development,” Invest. Ophthalmol. Vis. Sci. 50(3), 1168–1175 (2008). [CrossRef]   [PubMed]  

37. T. Shibata, J. Kim, D. M. Hoffman, and M. S. Banks, “The zone of comfort: Predicting visual discomfort with stereo displays,” J. Vis. 11(8), 11 (2011). [CrossRef]   [PubMed]  

38. Y. J. Jung, H. Sohn, S.-i. Lee, F. Speranza, and Y. M. Ro, “Visual importance- and discomfort region-selective low-pass filtering for reducing visual discomfort in stereoscopic displays,” IEEE Trans. Circ. Syst. Video Tech. 23(8), 1408–1421 (2013). [CrossRef]  

39. J.F. Hair, W.C. Black, B.J. Babin, R.E. Anderson, and R.L. Tatham, Multivariate Data Analysis (Pearson/Prentice-Hall, 2006).

Supplementary Material (10)

Media 1: MOV (895 KB)     
Media 2: MOV (885 KB)     
Media 3: MOV (265 KB)     
Media 4: MOV (246 KB)     
Media 5: MOV (284 KB)     
Media 6: MOV (313 KB)     
Media 7: MOV (1241 KB)     
Media 8: MOV (957 KB)     
Media 9: MOV (1121 KB)     
Media 10: MOV (1171 KB)     

Cited By

Optica participates in Crossref's Cited-By Linking service. Citing articles from Optica Publishing Group journals and other participating publishers are listed here.

Alert me when this article is cited.


Figures (14)

Fig. 1
Fig. 1 Crosstalk cancellation with intensity mapping methods: (a) the original image with crosstalk, (b) uncorrectable regions (red color) by the crosstalk cancellation [3,1416], (c) crosstalk cancellation without an intensity mapping, (d) crosstalk cancellation with the global intensity mapping [14,15], and (e) crosstalk cancellation with the local intensity mapping [15].
Fig. 2
Fig. 2 The effect of disparity magnitude on the extent of uncorrectable regions. Note that the uncorrectable regions are marked in red color. In these examples, disparity magnitude of the lamp decreases from the left to right figures.
Fig. 3
Fig. 3 Different visibility of system crosstalk: (a) high intensity object, (b) lower intensity object than (a), and (c) object with brighter background than (a). Note the objects in (a) and (c) have the same intensity, but the system crosstalk in (a) is more visible that that in (c).
Fig. 4
Fig. 4 The effect of temporal changes of leakage image: (a) change of leakage image on a moving object (see the tire) and (b) change of the extent of crosstalk regions.
Fig. 5
Fig. 5 Crosstalk visibility weight map: (a) image with system crosstalk, (b) zoomed-in images of (a), (c) original zoomed-in images corresponding to (b), (d) crosstalk visibility weight map, and (e) crosstalk visibility index map. Note that, in the visibility weight map, the larger values (i.e., white) represent higher visibility weights. In addition, in the visibility index map, the larger values (i.e., white) represent that the system crosstalk is more visible around those regions. In this figure, the dashed-line rectangles have larger visibility weights than that of the solid-line rectangle. As shown in (b), the crosstalk is more visible around larger visibility weights (see the short pants), which is also indicated in (e).
Fig. 6
Fig. 6 Change of uncorrectable regions with varying amounts of disparity shifting: (a) original image (left-view), (b) the image shifted by 0.2 degrees towards the uncrossed disparity direction, and (c) the image shifted by 0.4 degrees towards the uncrossed disparity direction. Note that the uncorrectable regions are marked in light red color. The horizontal boundaries of the images in (b) and (c) were cropped due to the disparity shifting.
Fig. 7
Fig. 7 Examples of the processed image (left-view): (a) the original image with perceived crosstalk (Media 1) and (b) processed image using the proposed disparity adjustment (Media 2), (c) disparity range of the original stereo video, (d) disparity range of the processed stereo video, and (e) comparison of the crosstalk visibility index between the original and processed stereo video. Note that, in (c) and (d), red vertical line represents the frame index where (a) and (b) were captured. Also, note that the disparity shifting in the proposed method displaces the perceived crosstalk occurred on perceptually important regions (see dashed-line rectangles in the figures) to less important regions (see solid-line rectangles in the figures). As shown in (b), the perceived crosstalk occurred on the less important region (i.e., balloon) is less visible and thus less annoying. Note that horizontal boundaries of the image in (b) were cropped due to the disparity shifting.
Fig. 8
Fig. 8 Examples of the processed image: (a) crosstalk cancellation with the global intensity mapping [14], (b) crosstalk cancellation with the local intensity mapping [15], and (c) the proposed method that combines the disparity adjustment and crosstalk cancellation with the local intensity mapping. Note that horizontal boundaries of the image in (c) were cropped due to the disparity shifting.
Fig. 9
Fig. 9 Stereoscopic 3D videos used in our experiments.
Fig. 10
Fig. 10 Measured DMOS of image quality: (a) the proposed method vs. crosstalk cancellation with the global intensity mapping and (b) the proposed method vs. crosstalk cancellation with the local intensity mapping. Note that higher DMOS values represent that the proposed method provides higher image quality.
Fig. 11
Fig. 11 Comparison of the assessment results for viewing preference: (a) the proposed method vs. crosstalk cancellation with the global intensity mapping and (b) the proposed method vs. crosstalk cancellation with the local intensity mapping.
Fig. 12
Fig. 12 Examples of the processed videos (left-view): (a) original video (Eiffel tower) (Media 3), (b) crosstalk cancellation with the global intensity mapping (Media 4), (c) crosstalk cancellation with the local intensity mapping (Media 5), and (d) the proposed method (Media 6). Note that horizontal boundaries of the images in (d) were cropped due to the disparity shifting.
Fig. 13
Fig. 13 Additional examples of the processed videos (left-view): (a) original video (Pantomime) (Media 7), (b) crosstalk cancellation with the global intensity mapping (Media 8), (c) crosstalk cancellation with the local intensity mapping (Media 9), and (d) the proposed method (Media 10).
Fig. 14
Fig. 14 Crosstalk visibility index of the processed stereo videos: (a) Eiffel tower and (b) Pantomime.

Tables (1)

Tables Icon

Table 1 Statistical results of subjective assessment for image quality.

Equations (6)

Equations on this page are rendered with MathJax. Learn more.

Q(n)= 1 | R | x,yR L(x,y,n)V(x,y,n),
V S (x,y,n)=log( M I (x,y,n)+γ),
V T ( x , y , n ) = | M U ( x , y , n ) M U ( x , y , n 1 ) | ,
V= w S V S + w T V T , where w S + w T =1,
H(n)= argmin h Q h (n), for h min <h< h max ,
Q h (n)= 1 | R h | x,y R h α M U h (x,y,n) V h (x,y,n),
Select as filters


Select Topics Cancel
© Copyright 2024 | Optica Publishing Group. All rights reserved, including rights for text and data mining and training of artificial technologies or similar technologies.