Open Access
Add to BibSonomy
Abstract
We present here an optical approach to boost the apparent pixel density by utilizing the superimposition of two shifted-pixel grids generated by a Pancharatnam-Berry deflector (PBD). The content of the two shifted pixel grids are presented to the observer’s eye simultaneously using a polarization-multiplexing method. Considering the compact and lightweight nature of PBD, this approach has potential applications in near-eye display systems. Moreover, the same concept can be extended to projection displays with proper modifications.
© 2019 Optical Society of America under the terms of the OSA Open Access Publishing Agreement
1. Introduction
Virtual reality (VR) offers eye-opening immersive experiences, where the artificial simulated environment can be as fascinating as the real one. Empowering the immersive user experience necessitates high-performance displays that could satisfy the fastidious requirements of the human visual system, including but not limited to high pixel density, large pixel number, wide field-of-view (FOV), and high refresh rate. Currently, mainstream commercial VR product manifests ~6 arcmins angular resolution with 100° FOV, while human eyes with 20/20 vision can resolve 1 arcmin. That is to say, a six-fold enhancement in resolution density is required. For a low-resolution display, the screen-door effect caused by the black matrix degrades the desired immersive experience considerably [1]. To mitigate the screen-door effect, display panels with reasonable size and a large number of pixels per inch (PPI) is urgently needed. Recently, display panels with over 1000 pixels per inch have been demonstrated [2], although not in mass production. They will reach the required human eye acuity within a factor of two. In spite of the challenges in manufacturing high PPI panels, as the pixel density increases, display luminance may decrease because of the reduced aperture ratio [3], and data flow rate in driving electronics will also increase accordingly. Foveated displays [4,5], making use of the varying resolution in the human visual system, provide an ultra-high resolution in the fovea region but a relatively low one in the peripheral. This approach offers an elegant design balance between FOV and panel pixel density, lowering the total pixel numbers while providing satisfactory high-resolution experience. However, it demands extra gaze-tracking apparatus [6] and beam steering module in the headset to make the high-resolution region following the eye movement.
Several approaches to boost the apparent pixel density of a display system have been proposed. For instance, a mechanical image shifter [7] can be utilized to generate spatially-displaced pixel grids in each sub-frame and then superimpose them to achieve higher resolution for projection displays. However, a vibrating image shifter in head-mounted near-eye displays is apparently not as tolerable as that in projection systems. Moreover, this approach also results in decreased framerate, which is unwanted in virtual reality contents that need fast motion picture response time in order to suppress image blurs [8]. To avoid mechanical vibration and framerate reduction, other approaches using optically overlaying and stacking displays have been developed [9–11]. Although they are able to offer higher pixel density, using multiple display panels usually causes decreased aperture ratio and lower optical efficiency.
In this paper, an optical approach based on polarization multiplexing instead of time multiplexing in our previous work [12] is proposed to enhance resolution density without compromising frame rate. This approach consists of separating each physical pixel spatially into two virtual pixels with subpixel precision using a passive polymer-based Pancharatnam-Berry phase [13,14] deflector (PBD) [15–18] and thus creating a new apparent pixel grid with smaller effective pixel pitch. Moreover, a pixelated polarization management layer is used to determine the separation ratio between two virtual pixels; thus both virtual pixel grids could display desired image content simultaneously without sacrificing frame rate. With our approach, the higher pixel density the display panel has, the more enhancement there will be. And it could also be easily integrated into foveated display systems due to its simple and compact optical structure.
2. Operation principle
The operation principle of our approach is to separate each original pixel into two virtual pixels optically and thus create a new pixel grid with higher pixel density, as Fig. 1 illustrates. For a display panel with pixel pitch , the diagonal shifting length from original pixel to virtual pixels should be , so that a new pixel grid with half a pixel pitch can be generated.
Fig. 1 Schematic diagram describing the principle of resolution enhancement by the shifted superimposition method. (a) From the perspective of pixel, each original pixel (green) with pitch P is separated diagonally into two virtual pixels (blue and yellow). (b) From the panel perspective, the original panel is split into two virtual panels. (c) A new pixel grid (orange) with half a pixel pitch (P/2) is formed by superimposition of two virtual panels.
In a VR system, the light from each pixel is collimated by a refractive lens before entering the observer’s eye, as Fig. 2 depicts. In other words, the different locations of each pixel are mapped onto different directions of a collimated light beam. Therefore, instead of mechanically shifting the display panel, it is also possible to get the desired image displacement by changing the direction of light after the collimating lens. A grating can achieve this functionality.
Fig. 2 A typical optical design for virtual reality. Pixel locations in the panel are mapped to different directions of nearly collimated beams by the lens.
Here, a PBD in the Raman-Nath regime [19] is chosen to function as the pixel separation component, considering its high diffraction efficiency in the ± 1 orders and polarization selective nature. Polymer based PBDs have shown great potential for display applications [20–22] because of its nearly 100% diffraction efficiency [23,24] and fabrication simplicity [25–31]. Figure 3 illustrates the liquid crystal orientation and characteristic polarization selectivity of a PBD. If an unpolarized light or linearly polarized light is incident on a PBD, then the left-hand and right-hand circularly polarized (LCP and RCP) component is deflected into + 1 and −1 order, respectively.
Fig. 3 Illustration of orientation distribution of local liquid crystal anisotropy director in a polymer PBD. The thickness d satisfies the half-wave plate condition and the period is determined by the desired separation angle of the two virtual pixel grids.
Making use of the polarization selective nature of PBD, the brightness separation ratio between two virtual pixels can be tuned by modulating the ratio between LCP and RCP fraction of input light using a pixelated polarization modulation layer (PML). Each pixel of the PML is a twisted nematic liquid crystal cell, applied to change the polarization state of the output beam. With pixelized intensity and polarization modulation from the display panel and PML, the pixel value of the two separated pixels from an original pixel can be assigned independently and simultaneously. Thus, the desired image content can be displayed on the two virtual pixel grids at the same time, as Fig. 4 shows. Without a pixelated PML, the PBD may still help to fill the black matrix but does not enhance the resolution because the two orthogonal polarizations share the same image content.
Fig. 4 Schematic illustration of the resolution enhanced near-eye display system based on polarization multiplexing. PBD: Pancharatnam-Berry deflector; L: lens; and PML: polarization modulation layer.
3. Experiment
3.1 Pancharatnam-Berry deflector
Photo-alignment method is applied to fabricate the PBD. A thin photo-alignment film (Brilliant Yellow, from Sigma-Aldrich) was spin-coated on a glass substrate. The coated substrate was directly exposed by the desired interference pattern generated by the circularly polarized beams with opposite handedness. To minimize environmental perturbations during the device fabrication process, a modified Sagnac interferometer was applied to fabricate the PBD [25], as Fig. 5 shows. The collimated linearly polarized laser beam (λ = 457 nm) is split into two beams with TE and TM polarizations by a polarizing beam splitter (PBS). Both beams emerging from the PBS travel around all three arms in opposite directions and recombine with each other at the PBS. Then the TE and TM beams are converted to LCP and RCP with only one quarter-wave plate. This common-path exposure design for PBD is robust to environmental vibrations, offering excellent contrast and fringe stability yet with a reduced requirement in laser coherence length, which increases the yield rate of high-quality PBD with a lower-cost laser.
After exposure, the substrate was coated with a diluted liquid crystal (LC) monomer (RM257), whose thickness satisfies the half-wave requirement at green wavelength, and then cured by a UV light, forming a cross-linked LC polymeric grating film. More details about PBD fabrication procedures are described in [25].
3.2 Image factorization
In order to generate two low-resolution images for the virtual panels, an optimization process is constructed to achieve the desired high-resolution image [12]:
where is a vector made up of the pixel values from the target high-resolution image to be displayed, is the mapping matrix between image contents on the two virtual panels (and ) and generated imagefrom the proposed system. Without losing generality and for a simple example, the mapping procedure is demonstrated in Fig. 6. The pixel values are rearranged as vectors in the manner shown in Fig. 6(a). In this case, the elements in the 9th row of mapping matrix, corresponding to the 9th pixel in the high-resolution image, are zeros except for the 2nd and the P + 6th columns, representing the pixel locations to be added in two virtual panels. For a high-resolution input image, it is factorized into two low-resolution images by Eq. (1). Then, the calculated pixel values from two virtual panels are compared to determine the luminance separation ratio for each pixel. The separation ratio is then passed to a calibrated PML to generate needed polarization states for each pixel. Meanwhile, the pixel values from two virtual panels are added together and then passed to the display panel to provide total needed luminance before pixel separation. The algorithm here is developed for the square pixel lattice, but the principle can be applied to other pixel arrangements such as PenTileTM matrix for OLED. Without the algorithm, the overlapping of shifted pixels may lead to considerable crosstalk between adjacent pixels, especially for the panels with a large aperture ratio.3.3 Polarization modulation layer
Several methods have been developed to modulate the polarization state (LCP/RCP ratio) of light from each pixel, among which the combination of a polarization rotator (PR) and a quarter-wave (λ/4) plate is commonly used, as shown in Fig. 7. Firstly, the orientation angle of linearly polarized light from the display is rotated by PR, such that the desired separation ratio is embedded as the TE/TM ratio. After that, all the polarized lights with different orientation angles are converted to elliptically polarized beams by the λ/4 plate. In this way, the TE/TM ratio are remapped as LCP/RCP ratio, which is the intrinsic orthogonal polarization pair of PBDs. As a proof of concept, in the demonstration system, a pixelated twisted-nematic (TN) LC panel and a λ/4 plate are combined to work as the PML. The TN LC cell is acquired by peeling off a polarizer from an LCD panel. TN LCD [32] has been widely used in the display industry for its broadband performance, and here it works as a PR to rotate the polarization orientation angle. With linearly polarized input, the polarization state of the output beam from a TN cell can be tuned by applying a different voltage across the cell. It should be noticed that, although the output light is no longer linearly polarized, the ratio between TE and TM polarizations can still be tuned through the TN polarization rotator.
Fig. 7 (a) Schematic illustration and (b) experimental implementation of the polarization modulation layer. The polarization orientation angle of the light from the display is rotated by the polarization rotator. Then a quarter-wave plate is utilized to convert the TE/TM ratio to RCP/LCP ratio, for the desired separation ratio after PBD.
3.4 Results and discussions
In the experiment, a resolution target ‘Siemens star’ was used as an example to demonstrate the resolution enhancement effect. The display panel and the PML have a resolution of 800x480 and a size of 5 inches. The focal length of the viewing lens is 55 mm, and the period of the fabricated PBD is controlled to be ~0.6 mm for the desired shift. In comparison with the apparent image captured at the original resolution [Fig. 8(a)], the resolution enhancement is clearly observed, especially at the sloped contours [Fig. 8(b)]. Moreover, the screen-door effect is significantly reduced, and the boundaries between pixels are not so apparent as the original image. It should be mentioned that the PML adapted from a commercial LCD panel is not designed for polarization modulation, which may lead to some crosstalk between adjacent pixels, depending on the distance between the definition layers of PML and the display panel. A display panel with relatively confined angular intensity distribution could be helpful to reduce the pixel crosstalk. Besides, as a common issue for cascaded panels, Moiré patterns induced by the similar spatial frequency of display panel and PML may decrease the image quality, although it is not very apparent in our experimental results. Additional optical engineering may be needed to eliminate the Moiré patterns, such as using a diffuser film in between two panels [33]. Moreover, the PBD can also be placed between the PML and the lens in Fig. 4, reducing the angle of incidence on it, which can be helpful to avoid reduced diffraction efficiency of PBD at oblique incidence. Since the grating period of PBD utilized here is around one thousand times longer than the visible wavelength, the chromatic dispersion of deflection angle induced by PBD is negligible. Although the off-axis shifting angle of PBD is different from the on-axis one, a corrected image is still achievable through an improved image rendering process together with the digital aberration correction for the eyepiece lens.
Fig. 8 Images captured through a camera with (a) original and (b) enhanced resolution of a “Siemens star” resolution target and an athletic shirt. The screen door effect is reduced, and the pixel density is doubled simultaneously.
4. Conclusion
A resolution enhancement method for near-eye display system is proposed and experimentally validated. This system, benefiting from the polarization selective nature of PBDs, can provide high-resolution images for viewers without sacrificing the display frame rate. The screen-door effect is also significantly reduced since the original black matrix is now filled by shifted pixels. The proposed resolution enhancement method has potential applications for projection and near-eye displays.
Funding
GoerTek Electronics.
Acknowledgments
The authors would like to thank Nannan Zhang for useful discussion.
References
1. C. Anthes, R. J. García-Hernández, M. Wiedemann, and D. Kranzlmüller, “State of the art of virtual reality technology,” in Proceedings of IEEE Aerospace Conference (IEEE, 2016), pp. 1–19. [CrossRef]
2. C. Vieri, G. Lee, N. Balram, S. H. Jung, J. Y. Yang, S. Y. Yoon, and I. B. Kang, “An 18 megapixel 4.3 ″1443 ppi 120 Hz OLED display for wide field of view high acuity head mounted displays,” J. Soc. Inf. Disp. 26(5), 314–324 (2018). [CrossRef]
3. S.-J. K. Park, M. Ryu, C.-S. Hwang, S. Yang, C. Byun, J.-I. Lee, J. Shin, S. M. Yoon, H. Y. Chu, K. I. Cho, L. Lee, M. S. Oh, and S. Im, “42.3: Transparent ZnO thin film transistor for the application of high aperture ratio bottom emission AM‐OLED display,” SID Int. Symp. Dig. Tech. Pap. 39(1), 629–632.
4. B. T. Mitchell, “Foveated display eye-tracking system and method,” U.S. patent 7,872,635 (Jan. 18, 2011).
5. G. Tan, Y. H. Lee, T. Zhan, J. Yang, S. Liu, D. Zhao, and S. T. Wu, “Foveated imaging for near-eye displays,” Opt. Express 26(19), 25076–25085 (2018). [CrossRef] [PubMed]
6. O. A. O. Sahlsten, K. Melakari, M. Ollila, and V. Miettinen, “Gaze-tracking system and method of tracking user’s gaze”, U.S. patent Application 2018/0157910A1 (Jun. 7,2018).
7. W. Allen and R. Ulichney, “47.4: Invited paper: Wobulation: Doubling the addressed resolution of projection displays,” SID Int. Symp. Dig. Tech. Pap. 36, 1514–1517 (2005). [CrossRef]
8. F. Peng, H. Chen, F. Gou, Y. H. Lee, M. Wand, M. C. Li, S. L. Lee, and S. T. Wu, “Analytical equation for the motion picture response time of display devices,” J. Appl. Phys. 121(2), 023108 (2017). [CrossRef]
9. F. Heide, D. Lanman, D. Reddy, J. Kautz, K. Pulli, and D. Luebke, “Cascaded displays: spatiotemporal superresolution using offset pixel layers,” ACM Trans. Graph. 33(4), 60 (2014). [CrossRef]
10. Y. Kusakabe, M. Kanazawa, Y. Nojiri, M. Furuya, and M. Yoshimura, “A YC-separation-type projector: High dynamic range with double modulation,” J. Soc. Inf. Disp. 16(2), 383–391 (2008). [CrossRef]
11. B. Sajadi, M. Gopi, and A. Majumder, “Edge-guided resolution enhancement in projectors via optical pixel sharing,” ACM Trans. Graph. 31(4), 79 (2012). [CrossRef]
12. Y. H. Lee, T. Zhan, and S. T. Wu, “Enhancing the resolution of a near-eye display with a Pancharatnam-Berry phase deflector,” Opt. Lett. 42(22), 4732–4735 (2017). [CrossRef] [PubMed]
13. S. Pancharatnam, “Generalized theory of interference, and its applications,” Proc. Indian Acad. Sci. Sect. A Phys. Sci. 44(5), 247–262 (1956). [CrossRef]
14. M. V. Berry, “Quantal phase factors accompanying adiabatic changes,” Proc. R. Soc. Lond. A Math. Phys. Sci. 392(1802), 45–57 (1984). [CrossRef]
15. N. V. Tabiryan, S. R. Nersisyan, D. M. Steeves, and B. R. Kimball, “The promise of diffractive waveplates,” Opt. Photonics News 21, 41–45 (2010).
16. N. Tabiryan, D. Roberts, D. Steeves, and B. Kimball, “4G optics: new technology extends limits to the extremes,” Photon. Spectra 51, 46–50 (2017).
17. Y. H. Lee, G. Tan, T. Zhan, Y. Weng, G. Liu, F. Gou, F. Peng, N. V. Tabiryan, S. Gauza, and S. T. Wu, “Recent progress in Pancharatnam-Berry phase optical elements and the applications for virtual/augmented realities,” Opt. Data Process. Storage 3(1), 79–88 (2017). [CrossRef]
18. T. Zhan, Y. H. Lee, G. Tan, J. Xiong, K. Yin, F. Gou, J. Zou, N. Zhang, D. Zhao, J. Yang, S. Liu, and S. T. Wu, “Pancharatnam-Berry optical elements for head-up and near-eye Displays,” J. Opt. Soc. Am. B 36(5), D52–D65 (2019). [CrossRef]
19. M. G. Moharam and L. Young, “Criterion for Bragg and Raman-Nath diffraction regimes,” Appl. Opt. 17(11), 1757–1759 (1978). [CrossRef] [PubMed]
20. H. Chen, Y. Weng, D. Xu, N. V. Tabiryan, and S. T. Wu, “Beam steering for virtual/augmented reality displays with a cycloidal diffractive waveplate,” Opt. Express 24(7), 7287–7298 (2016). [CrossRef] [PubMed]
21. T. Zhan, Y. H. Lee, and S. T. Wu, “High-resolution additive light field near-eye display by switchable Pancharatnam-Berry phase lenses,” Opt. Express 26(4), 4863–4872 (2018). [CrossRef] [PubMed]
22. Y. H. Lee, G. Tan, Y. Kun, T. Zhan, and S. T. Wu, “Compact see-through near-eye display with depth adaption,” J. Soc. Inf. Disp. 26(2), 64–70 (2018). [CrossRef]
23. C. Oh and M. J. Escuti, “Achromatic diffraction from polarization gratings with high efficiency,” Opt. Lett. 33(20), 2287–2289 (2008). [CrossRef] [PubMed]
24. S. R. Nersisyan, N. V. Tabiryan, D. M. Steeves, and B. R. Kimball, “Optical axis gratings in liquid crystals and their use for polarization insensitive optical switching,” J. Nonlinear Opt. Phys. Mater. 18(1), 1–47 (2009). [CrossRef]
25. T. Zhan, J. Xiong, Y. H. Lee, R. Chen, and S. T. Wu, “Fabrication of Pancharatnam-Berry phase optical elements with highly stable polarization holography,” Opt. Express 27(3), 2632–2642 (2019). [CrossRef] [PubMed]
26. N. V. Tabirian, S. R. Nersisyan, B. R. Kimball, and D. M. Steeves, “Fabrication of high efficiency, high quality, large area diffractive waveplates and arrays,” U.S. patent 9,983,479 (May 29, 2018).
27. J. Kim and M. J. Escuti, “Polarization grating exposure method with easily tunable period via dual rotating polarization grating masks,” J. Opt. Soc. Am. B 36(5), D42–D46 (2019). [CrossRef]
28. M. N. Miskiewicz, J. Kim, Y. Li, R. K. Komanduri, and M. J. Escuti, “Progress on large-area polarization grating fabrication,” Proc. SPIE 8395, 83950G (2012). [CrossRef]
29. J. Kim, Y. Li, M. N. Miskiewicz, C. Oh, M. W. Kudenov, and M. J. Escuti, “Fabrication of ideal geometric phase holograms with arbitrary wavefronts,” Optica 2(11), 958–964 (2015). [CrossRef]
30. N. V. Tabirian, S. R. Nersisyan, B. R. Kimball, and D. M. Steeves, “Cycloidal diffractive waveplate and method of manufacture,” U.S. patent 9,658,512 (May 23, 2017).
31. S. R. Nersisyan, N. V. Tabiryan, D. M. Steeves, and B. R. Kimball, “Characterization of optically imprinted polarization gratings,” Appl. Opt. 48(21), 4062–4067 (2009). [CrossRef] [PubMed]
32. M. Schadt and W. Helfrich, “Voltage-dependent optical activity of a twisted nematic liquid crystal,” Appl. Phys. Lett. 18(4), 127–128 (1971). [CrossRef]
33. G. P. Bell, R. Craig, R. Paxton, G. Wong, and D. Galbraith, 25.4: Invited paper: Beyond flat panels —Multi layer displays with real depth,” SID Int. Symp. Dig. Tech. Pap. 39(1), 352–355 (2008).
