Open Access
Add to BibSonomy
Abstract
Considering the high complexity of local dimming backlight that is necessary to effectively suppress color breakups for field sequential color liquid crystal displays (FSC-LCDs), a global dimming-based solution is proposed. This solution involves considering that the color breakups mainly occur at object edges of an image. By introducing an algorithm to present the edge information in a single field, evaluating color breakup performances, and experimentally verifying based on a 240-Hz LCD, lighter color breakups are revealed compared with mainstream local dimming-based solutions. Therefore, the proposed solution can achieve FSC-LCDs with better performance and practicality for advanced display applications.
© 2019 Optical Society of America under the terms of the OSA Open Access Publishing Agreement
1. Introduction
A field sequential color liquid crystal display (FSC-LCD) does not require color filters to generate full-color images. Hence, compared with traditional LCDs, FSC-LCDs theoretically have three times the optical efficiency, a higher spatial resolution, and a lower cost of panel materials [1,2]. FSC-LCDs show great potential for use in ecologically friendly low-power displays [1,2], high-resolution displays (e.g., 4K/8K and 3D TVs [3,4]), augmented reality (AR) and virtual reality (VR) devices [5,6], etc. However, FSC-LCDs typically suffer from color breakups when a relative velocity exists between displayed objects and an observer’s eyes [7]. To suppress color breakups, several research groups have proposed effective solutions based on locally controlled backlights, i.e., local dimming backlights. For example, the Stencil-FSC method (denoted as “STENCIL”) [8–10] uses an additional field to present the averaged image content of each dimming block and then uses red (R), green (G), and blue (B) fields to reproduce a target image. Thus, the image content that tends to incur color breakups is concentrated in a single field. Another example is the local-primary-desaturation method (denoted as “LPD”) [11–13], which uses three new desaturated primary colors that can cover only the color gamut of each dimming block to replace the original primary colors (R, G, and B). The desaturated fields naturally lead to lighter color breakups.
These methods were developed for local dimming backlights [14,15] that incur high costs in terms of both hardware and software. Thus, to reduce the hardware loading and the software computations, a solution based on global dimming that performs comparably to local solutions is highly demanded. However, current global solutions, which are usually obtained by degenerating a certain local solution to a 1-by-1 dimming block configuration, have highly limited performances in terms of color breakup suppression because local solutions essentially take advantage of the homogenous image content of each dimming block rather than the entire image. As a result, global solutions directly degenerated from local solutions have little effect when a target image contains complex information.
In this study, we have observed that the image regions where most of the color breakups occur are usually high-contrast image boundaries (edges) [16]. For example, the color breakup phenomenon of an image Lily produced by the traditional red-green-blue (RGB) driving method is simply simulated, as shown in Fig. 1, where the color breakups in the edge regions are much more severe than those in the non-edge regions. Therefore, we consider using an additional field to present the critical edge information. Because the previously proposed STENCIL method also uses an additional field to present critical information, the proposed method is called the “Edge Stencil-FSC” method (denoted as “EDGE”). Nevertheless, compared with STENCIL, which considers the averaged image content in each dimming block critical, the edge information here is much more efficient in concentrating the image content that is likely to incur color breakups. Therefore, the proposed method is anticipated to achieve comparable performance by merely using a 1-by-1 dimming block (global dimming). The algorithm of the proposed method will be introduced in Sec. 2, and in Sec. 3, we will evaluate the algorithm using various images and compare it with previously developed solutions based on local dimming; both simulations and experimental results based on a 240-Hz LCD will be reported.
Fig. 1 (a) Lily image with color breakups produced by the traditional RGB driving method: (b) an edge region magnified; (c) a non-edge region magnified.
2. Algorithm
Inspired by the observation that most color breakups occur at high-contrast image boundaries (edges), as shown in Fig. 1, we consider the edge information in a target image rather than the entire image. The critical edge information is desired to be presented in the first field and then three mono-colored (R, G, and B) fields will be used to accurately reproduce the original image.
The algorithm begins with edge detection for a target image. To do so, the target image is first converted into the YCbCr color space to obtain a precise representation of the image’s luminance information (Y-channel). Next, because human eyes are more sensitive to luminance variations than to chromatic variations (Cb/Cr-channels) [17], the desired edges are detected from the Y-channel only. The edge detection process can be performed using an arbitrary available method. In this study, the famous Sobel edge detection method [18] is chosen, and the Sobel convolution kernels in the horizontal (Sbx) and vertical (Sby) directions are given in Eq. (1).
Next, to determine the backlight signal of the first field, i.e., the edge-dependent field, the R, G, and B components of the detected edges are each averaged. For example, a test image Blossom is shown in Fig. 2(a). Its edge intensity map is obtained using Eq. (1) and the edges are correspondingly extracted by multiplying the edge intensity map with the target image, as shown in Fig. 2(b). By averaging each of the R, G, and B components of the extracted edges, the global backlight signal of the edge-dependent field is determined to be a color close to magenta, as shown in Fig. 2(c). In contrast, if the global-version STENCIL method is adopted for the same image, the backlight of the first field will be more whitish because it is obtained by averaging the entire image, which contains the magenta blossom and the green leaves.
Fig. 2 Backlight signal determination using the proposed method: (a) target image; (b) edge intensity map and detected edges; (c) backlight signal determined by averaging the edges.
After determining the backlight signal of the edge-dependent field, the liquid crystal (LC) signals of this field are compensated for using Eq. (2) [14,15].
Here, corresponding to the R, G, and B channels, Ii denotes the intensity of the target image, BLedgei denotes the backlight signal of the edge-dependent field determined previously, and Tedgei denotes the LC signals derived from BLedgei. By choosing the minimum value among TedgeR, TedgeG, and TedgeB for each pixel, the LC signals Tedge1st of the edge-dependent field can be determined. By combining the global backlight signal with the LC signals, the mono-colored edge-dependent field is created in front of the screen to present most of the edge information.
In addition to the edge-dependent field, the remaining image information should be displayed in the following R, G, and B fields, whose LC signals can be solved by finding the differences between the target image and the edge-dependent field, as given by Eq. (3).
Here, BLi denotes the backlight signal of the R, G, or B field, which is determined according to the maximum value of the LC signals to prevent clippings, i.e., the conventional maximum method, and Ti’ denotes compensated LC signals.
The target image is finally produced by sequentially displaying the edge-dependent field and the following R, G, and B fields, as shown in Fig. 3. To demonstrate the color breakup performance, the Blossom image with color breakups accumulating in a frame time (1/60 s) is simulated by shifting the four fields, as shown in Fig. 4(a). Here, the shift distance is simply obtained by assuming the field of view (FOV) is 25 degrees and the speed of saccadic eye movements is 100 degrees/s. For comparison, the color breakup phenomenon produced by the global-version STENCIL method is also simulated for the same image under the same condition, as shown in Fig. 4(b). Because the magenta information is largely concentrated in the edge-dependent field by the proposed EDGE method, few color breakups at the blossom’s magenta edges are produced in Fig. 4(a). However, obvious color breakups are produced by the global-version STENCIL method in Fig. 4(b) because the backlight signal of the first field is obtained by averaging the entire image, thus causing the backlight signal to be far from the color of the blossom’s edges.
Fig. 3 Four fields produced by the proposed method for the Blossom image, including backlight signals (lower), compensated LC signals (middle), and front-of-screen field images (upper).
Fig. 4 Backlight signal of the first field (left) and corresponding simulated image with color breakups (right) for the Blossom image: (a) the proposed EDGE method; (b) the global-version STENCIL method.
3. Performance evaluation
3.1 Evaluation method
In our previous study, we proposed a metric that can predict subjective color breakup visibilities with a good linearity for not only certain patterns but also nature images; we also conducted a database dedicated to color breakup evaluation that contains 25 reference images with various contents, from very simple to very complicated, as shown in Fig. 5 [19,20]. Considering that eye-fixation movements, which are free while watching still images, are constrained while perceiving color breakups, this metric uses the so-called dominant visual saliency (DVS) regions of an image, i.e., the regions with visual saliency values larger than a threshold of 50%, as a weighting map to accumulate color differences between the original image and the image with color breakups.
Fig. 5 Database containing 25 reference images with various image content (numbered 1 to 25), dedicated to color breakup evaluation.
With the color breakup metric and the database, the color breakup solution of interest can be evaluated by simulating images with color breakups and working out the color breakup scores with the calculation flow in Fig. 6 for all 25 reference images. In addition, the color breakup scores need to be normalized by the score produced by a vertical white bar rendered by the RGB driving method, which is acknowledged to have much more severe color breakups than natural images [8–13].
Fig. 6 Calculation flow of the color breakup metric. Notations are listed as follows: CBU for color breakup, VS for visual saliency, DVS for dominant visual saliency, T for the threshold of 50%, Φ for the area of all DVS regions, ΔC* for chroma difference in CIELAB color space. Note the color fringe at the left of the image with CBU is produced by the real edge of the display panel and can be neglected as long as the display size is not too small.
Figure 7 shows the calculation flow starting from two retinal images, i.e., with and without color breakups. The image with color breakups can be easily simulated by shifting the front-of-screen field images according to the relative velocity between the displayed objects and the eyes. More implementation details of the metric and evaluation of the variety of the image database can be found in [19,20]. In addition, the database is provided in Dataset 1 [21].
Fig. 7 Evaluated color breakup scores (normalized) of the EDGE, STENCIL, LPD, and RGB methods for the 25 reference images, presented as: (a) boxplots with medians and red circles indicating the scores of individual images and a (b) heatmap.
3.2 Evaluation results and discussions
First, to simulate images with color breakups, color breakups are assumed to be induced by saccadic eye movements with the same condition used previously (FOV = 25 degrees, saccade speed = 100 degrees/s, and frame time = 1/60 s). Next, in addition to the proposed EDGE method, two representative solutions based on local dimming are adopted for comparison, i.e., STENCIL with 12 × 24 dimming blocks [8] (still denoted as “STENCIL”) and LPD with 10 × 18 dimming blocks [11] (still denoted as “LPD”). The configurations of the dimming block here are optimum with regard to image fidelity and color breakup suppression, as demonstrated in the literature [8,11]. The RGB method is also adopted as a reference. Note that the four involved methods (EDGE, STENCIL, LPD, and RGB) are configured to produce no distortions in terms of clipping [14,15] to provide a fair comparison.
Figure 7(a) shows evaluated color breakup scores corresponding to the four methods using boxplots that reflect the variations across the 25 images in the database. As simply seen from the boxes with their medians, the proposed EDGE method slightly surpasses the other two mainstream local solutions in terms of overall performance by merely using a global dimming backlight (not to mention RGB).
To reveal the results of individual images in detail, the same results are drawn as a heatmap in Fig. 7(b), where RGB (the rightmost column) certainly performs the worst for nearly all images. Moreover, under the premise that EDGE has the leading overall performance, the rank of the solutions depends on image content to a certain extent. In fact, the effectiveness of a certain solution (EDGE, STENCIL, or LPD) is largely affected by image content. To be specific, STENCIL calls for low local contrasts because it averages each dimming block to determine the backlight, and LPD calls for small local color gamuts because it shrinks the backlight’s color gamut to that of the image content in each dimming block. The proposed EDGE method requires the edge colors in an entire image to be homogeneous because the global backlight is determined by averaging the edge colors. In essence, from a statistical point of view, the requirement of the homogeneous edge colors of an image is no more difficult to be fulfilled than low local contrasts and small local color gamuts; thus, EDGE can be comparable to STENCIL and LPD by merely using a mono-colored field to present critical image information. To intuitively illustrate how the image content affects performance in addition to the quantitative evaluation results, five images with different performances, i.e., Image 1, 8, 9, 18, and 20, are selected, as summarized in Table 1, where the characteristics of these images are also provided. Referring to the preferences of the three solutions, the performances can be well explained by the image characteristics (local contrasts, local color gamuts, and homogeneity of edge colors). Furthermore, Figs. 8-12 show their simulated color breakup phenomena by magnifying the regions that typically reflect the image characteristics. To better understand the driving methods, the front-of-screen field images of a typical image that was frequently studied in previous research (Image 18: Lily [8–13]) are provided in Fig. 11 and elaborated as follows. (i) For EDGE, the highly homogenous edges in this image cause the mono-colored edge-dependent field to effectively present most of the edge information. (ii) For STENCIL, the flower’s petals are even better extracted in the first field than they are with EDGE, but the dark content around the petals’ edges considerably pulls down the backlight intensity of the dimming blocks. Thus, a large amount of the edge information remains in the R, G, and B fields, which causes higher color breakup visibility. (iii) For LPD, the low-saturation flower makes most of the dimming blocks have very small color gamuts, resulting in three low-saturation driving fields and thus a low color breakup visibility.
Table 1. Color breakup performances of five selected images and their image characteristics
Fig. 11 Simulated color breakup phenomena of Image 18 (Lily) with evaluated color breakup scores. The front-of-screen field images are provided and a black field is inserted for the methods using three fields.
3.3 Experimental verification
To experimentally verify the evaluation results, a 240-Hz 24” LCD (XL2540 from BENQ, FHD resolution) is adopted. Four front-of-screen field images of a frame are pre-calculated and successively displayed to emulate an FSC-LCD with a frame rate of 60 Hz. For methods using three fields, a black field is inserted (refer to Fig. 11). To imitate the saccadic eye movements and capture the color breakup phenomena, a camera (Nikon D750 with a 35-mm lens) is assembled on a rotating motor with a controllable rotation speed, as shown in Fig. 13. To create the condition used for the previous simulations, the camera is placed 1 m from the LCD to produce an FOV of 25 degrees, the motor’s rotation speed is set to 100 degrees/s, and the shutter speed is set to 1/60 s. To get the photographs starting from the field the same as the simulations, for example, from R, then G and B, many photographs are taken and the qualified ones are manually selected. In addition, considering insufficient gray-level responses while displaying ever-changing field images at such a high refresh rate, an overdrive process is performed to prevent color shift. More details about the apparatus setup and the calibrations that were done in our previous study can be found in [18].
Image 18 (Lily), which was elaborately studied in Fig. 11, is adopted again. Figure 14 shows the captured color breakup phenomena produced by the EDGE, STENCIL, LPD, and RGB methods. The color breakups are clearly demonstrated, and the photographs appear quite similar to the simulated images with color breakups, thus revealing that (i) the evaluation based on the image simulations is accurate and (ii) the conclusion from the evaluation that the proposed global EDGE method outperforms the local solutions is credible. Nevertheless, the photographs still appear slightly different from the simulated images, which is mainly because of two hardware limitations: (i) the gray-level response of the 240-Hz LCD is severely insufficient while emulating an FSC-LCD, and the overdrive process can only compensate to a limited extent, causing the color to be a little inconsistent across different methods; and (ii) the LCD motion blur [22] causes the photographs to appear more blurred than the images simulated by directly shifting the field images. In future work, the color breakup simulation may consider motion blur to further refine the evaluation.
Fig. 14 Color breakup phenomena of Image 18 (Lily) captured by the rotating camera, as produced by the EDGE, STENCIL, LPD, and RGB methods. Note that the power LED of the LCD is tailed (highlighted by a red circle in the right) because of the rotation of the camera while photographing.
4. Conclusion
Conventional color breakup solutions based on local dimming, e.g., STENCIL and LPD, take advantage of the homogeneous image content in each dimming block at the expense of a complicated backlight. To develop a new color breakup solution based on a simple global dimming backlight, we proposed the Edge Stencil-FSC (EDGE) method by utilizing the edges of an image to efficiently extract the critical image content that is likely to incur color breakups. Because the requirement of the proposed method, i.e., edge colors should be homogeneous, is even more likely to be fulfilled than low local contrasts (for STENCIL) and small local color gamuts (for LPD), the proposed method was proved to slightly surpass the two mainstream local solutions based on a sizeable image database. With superior color breakup performance and a simple demand for a global dimming backlight, the proposed method enables FSC-LCDs that feature higher optical efficiency, higher resolution, and wider color gamut to simultaneously achieve high performance and practicality for applications including 4K/8K and 3D TVs and AR/VR devices. In addition, the proposed concept of edge-dependent processing can be applied in more content-awareness color breakup solutions, thus providing a common idea for FSC display research. For example, if the cost of local dimming backlight is acceptable, the proposed method can also be combined with local dimming backlights by applying the proposed algorithm in each dimming block, and lighter color breakups are expected.
Funding
Ministry of Science and Technology (MOST) of R.O.C. project under grant number MOST107-2218-E-009-057-. AU Optronics, Taiwan.
References
1. H. Hasebe and S. Kobayashi, “A full-color field sequential LCD using modulated backlight,” SID Symp. Dig. Tech. 16, 81–83 (1985).
2. Y. Huang, H. Chen, G. Tan, H. Tobata, S. I. Yamamoto, E. Okabe, Y. F. Lan, C. Y. Tsai, and S. T. Wu, “Optimized blue-phase liquid crystal for field sequential-color displays,” Opt. Mater. Express 7(2), 641–650 (2017). [CrossRef]
3. K.-S. Peng, F.-C. Lin, and K.-T. Teng, “Efficient image resolution enhancement using edge-directed unsharp masking sharpening for a real-time ASIC application,” J. Comput. Sci. Syst. Biol. 8(3), 174–184 (2015). [CrossRef]
4. Y. Takaki, K. Tanaka, and J. Nakamura, “Super multi-view display with a lower resolution flat-panel display,” Opt. Express 19(5), 4129–4139 (2011). [CrossRef] [PubMed]
5. H. Chen, Y. Weng, D. Xu, N. V. Tabiryan, and S.-T. Wu, “Beam steering for virtual/augmented reality displays with a cycloidal diffractive waveplate,” Opt. Express 24(7), 7287–7298 (2016). [CrossRef] [PubMed]
6. F. Gou, H. Chen, M.-C. Li, S.-L. Lee, and S.-T. Wu, “Submillisecond-response liquid crystal for high-resolution virtual reality displays,” Opt. Express 25(7), 7984–7997 (2017). [CrossRef] [PubMed]
7. T. Järvenpää, “Measuring color breakup of stationary images in field-sequential-color displays,” J. Soc. Inf. Disp. 13(2), 139–144 (2005). [CrossRef]
8. F.-C. Lin, Y.-P. Huang, C.-M. Wei, and H.-P. D. Shieh, “Color-breakup suppression and low-power consumption by using the stencil-FSC method in field-sequential LCDs,” J. Soc. Inf. Disp. 17(3), 221–228 (2009). [CrossRef]
9. F.-C. Lin, Y.-P. Huang, C.-M. Wei, and H.-P. D. Shieh, “Color filter-less LCDs in achieving high contrast and low power consumption by stencil field-sequential- color method,” J. Disp. Technol. 6(3), 98–106 (2010). [CrossRef]
10. F.-C. Lin, Y.-P. Huang, and H.-P. D. Shieh, “Color breakup reduction by 180Hz stencil-FSC method in large-sized color filter-less LCDs,” J. Disp. Technol. 6(3), 107–112 (2010). [CrossRef]
11. F.-C. Lin, Y. Zhang, and E. H. A. Langendijk, “Color breakup suppression by local primary desaturation in field-sequential color LCDs,” J. Disp. Technol. 7(2), 55–61 (2011). [CrossRef]
12. Y. Zhang, F.-C. Lin, and E. H. A. Langendijk, “A field-sequential-color display with a local-primary-desaturation backlight scheme,” J. Soc. Inf. Disp. 19(3), 242–248 (2011). [CrossRef]
13. F.-C. Lin, C.-W. Chang, Y.-P. Huang, and H.-P. D. Shieh, “High-image reproduction by the low-field-rate stencil-LPD method for field-sequential-color LCDs,” J. Disp. Technol. 11(12), 1069–1075 (2015). [CrossRef]
14. G. Tan, Y. Huang, M.-C. Li, S.-L. Lee, and S.-T. Wu, “High dynamic range liquid crystal displays with a mini-LED backlight,” Opt. Express 26(13), 16572–16584 (2018). [CrossRef] [PubMed]
15. H. Chen, R. Zhu, M.-C. Li, S.-L. Lee, and S.-T. Wu, “Pixel-by-pixel local dimming for high-dynamic-range liquid crystal displays,” Opt. Express 25(3), 1973–1984 (2017). [CrossRef] [PubMed]
16. F.-C. Lin, K.-T. Teng, C.-W. Chang, C.-H. Lin, Y.-P. Huang, and H.-P. D. Shieh, “15-2: Effective color breakup suppression by a low-cost global dimming backlight for field-sequential-color displays,” SID Symp. Dig. Tech. 47(1), 171–174 (2016). [CrossRef]
17. K. T. Mullen, “The contrast sensitivity of human colour vision to red-green and blue-yellow chromatic gratings,” J. Physiol. 359(1), 381–400 (1985). [CrossRef] [PubMed]
18. A. P. Papliński, “Directional filtering in edge detection,” IEEE Trans. Image Process. 7(4), 611–615 (1998). [CrossRef] [PubMed]
19. Z. Qin, Y. J. Lin, F. C. Lin, C. W. Kuo, C. H. Lin, N. Sugiura, H. P. D. Shieh, and Y. P. Huang, “Image content adaptive color breakup index for field sequential color displays using a dominant visual saliency method,” J. Soc. Inf. Disp. 26(2), 85–97 (2018). [CrossRef]
20. Y.-J. Lin, Z. Qin, F.-C. Lin, H.-P. D. Shieh, and Y.-P. Huang, “85-3: Distinguished Student Paper: Image-content-adaptive color breakup index for field-sequential-color displays using dominant visual saliency method,” SID Symp. Dig. Tech. 49(1), 1159–1162 (2018). [CrossRef]
21. Z. Qin, “Color Breakup Image Database,” figshare (2018) [retrieved 1 Nov 2018], https://doi.org/10.6084/m9.figshare.7223039.
22. F. Peng, H. Chen, F. Gou, Y. H. Lee, M. Wand, M. C. Li, S. L. Lee, and S. T. Wu, “Analytical equation for the motion picture response time of display devices,” J. Appl. Phys. 121(2), 023108 (2017). [CrossRef]
