1. Introduction

It is always desirable to enable eye-to-eye video conferencing. However, it is not yet possible to make this experience in a natural way by using the existing optical configurations because of the difficulty in physical placement of the display and the imaging apparatus in the same gaze line. Usually a video camera is either placed on above or below the screen bezel or built in at the edge of the display area. If the gaze of the speaking person is towards the screen, the observing person will see the gaze as away from themselves. Obviously, if they try to keep their eyes on the camera, they will simply not see the screen, hence the person’s face on the other side. Although in real life, people tend not to look directly at the eyes all the time, they normally converse facing each other. Such a feeling of being able to have eye contact is much more desirable in video conferencing for the people who are physically apart. Since it is not practically possible to place the cameras at the locations of the eyes of the person who is being viewed on the display (even that were the case, the cameras would have needed to move physically with the moving pictures), some hardware and software were developed to overcome these issues. Hardware solutions are mostly in the form of hidden cameras behind the screen or indirect imaging using cameras outside the display [1]. Software solutions re-render the images as if the eye gaze is towards the display [2–4]. A media wall system, which is a rear-projection tiled display with diffusive screens [6], was designed to allow eye-to-eye video conferencing by placing regularly distributed camera array. The main challenge there remains the same for the physical placement of cameras behind a diffuser screen.

To simplify the application without losing its general applicability, we define the problem as projecting pictures onto a diffusive media from rear and allow to image the scene on the observer’s side on a camera sensor behind of the diffusive screen at the same time, as shown in Fig. 1. Earlier hardware solutions use active shutters or half mirrors to achieve this [1]. Although carefully placed cameras behind a fast switching diffuser panel [5] can be a solution, the opening is normally large and disturbingly noticeable to the observers. Furthermore, the usual switchable polymer dispersed liquid crystal (PDLC) based diffusive screens or shutters draw considerable power to switch and requires constant power to maintain the clear state, which are undesirable.

 

Fig. 1 The concept of eye-to-eye video conferencing. A focusing element is embedded in the diffusive media which is used to project images from a rear projection source. The focusing element images the scene on the other side of the diffusive media to the same side as the projection source. Proper physical placement of one or more focusing elements will allow an eye-to-eye video transmission between similar units.

Download Full Size | PDF

In this work, we purposely designed a non-intrusive PS pattern, which is much less noticeable, and use a bi-stable diffusive material to provide a fully scattered diffusive screen or a diffusive PS pattern with zero power maintenance. Therefore, when it is required to image, the imaging device can be switched to PS state and it can stay at that state for an indefinite time or until the it is switched back to diffusive screen state. Hence, there is no need for high speed switching between these two states. In the diffusive screen state, it can only be used for displaying projected images. While in the PS state, the device can image the scene and display a projected picture on to it simultaneously. Since individual PS features can be minimized to a scale which is not detectable by human eye at a certain observation distance, it is possible to view a high-quality image without any ambiguity while imaging.

Diffractive lenses are popular for imaging applications. Since they are thin optical components, they can be embedded relatively easily in the optical systems or can be integrated as a component in the imaging optics [7]. In a previous study, it was shown that using photon sieve (PS) instead of Fresnel Zone Plate (FZP) would be more advantageous in terms of the optical quality and manufacturability [8, 9]. Furthermore, they can be used to image in the visible range [10]. It would be ideal to integrate PS with fine structures invisible to human eyes on diffusive screens.

For diffusive screens, a good alternative to the PDLC is Smectic A (SmA) liquid crystal (LC) materials used in bi-stable transreflective displays of zero power maintenance [15]. Unlike PDLC, it can be electrically switched between a dynamically scattered disordered state and an ordered state, which are both non-volatile. The scattered state is diffusive with transmitted and scattered lights similar to that of a scattered PDLC one while the ordered state is clear and transparent with no visible haze [15]. Because of the random nature of disorder at molecular level, the scattering is diffusive and almost perfect Lambertian and it is insensitive to the light polarization. It is possible to pattern the switching electrodes appropriately and use SmA LC to form a switchable PS on a diffusive screen.

2. Photon sieve design and fabrication

The switching PS is first designed as a chromium coated mask. The mask consists of a 10x10 array of PS structures which have pinholes with varying diameters in a concentric ring configuration. Figure 2 shows the 4X microscope image of the mask and measured diameter of a single PS is 1.624mm. The inset shows the image of the whole mask. Each PS structure is designed to have 50mm focal length at 532nm wavelength with diameter of 1.616mm. The diameter of the minimum feature size of the structure 15.625μm. The required pixel size of the Media-Wall application was ≈ 0.5mm, [6]. Therefore, we expect the pixel variation to be significantly low and cannot be detected by normal human eye at an observation distance of 2.5m.

 

Fig. 2 The image of photon sieve mask is shown on the left. The inset shows the detailed microscopic image. Diameter of a single PS is measured 1.624mm.

Download Full Size | PDF

In order to transfer this fixed structure to a switchable device configuration, following fabrication steps are performed: The electrode of the PS array was photolithographically patterned on an indium-tin-oxide (ITO, sheet resistance 120Ω/sq) coated on glass substrate of the size 26mm × 21mm × 1.1mm. The photomask is designed to present multiple opaque annular rings (interconnected) and transparent circles (isolated) of different sizes. Image reversal photoresist (AZ-5214E) is processed in such a condition as to allow the resist at the unmasked areas (isolated circles) exposed to UV light to be easily dissolved in a developer (AZ-351B), thus the unwanted ITO at those areas can be wet-etched away by a diluted hydrochloric acid (HCl). As a result, the masked area (interconnected annular rings) will function as a continuous electrode.

The patterning procedures and conditions are given as follows: Prior to photoresist coating, the substrate is first ultrasonically cleaned with acetone and isopropyl alcohol (IPA) sequentially (each for 5 minutes), rinsed in deionized (DI) water and then dehydrated on a hotplate at 90°C for three minutes. A 1.4μm-thick layer of AZ-5214E photoresist is spun on the substrate with 4000rpm for 30 seconds. To evaporate the coating solvent, the substrate is then soft-baked on the hotplate at 90°C for 1 minute and placed at room temperature for 5 minutes. Subsequently, the substrate is exposed on a UV light source for 5 seconds, and then developed in the AZ-351B (1 : 4 diluted in DI water) for up to 1 minute. The exposure time, development time and the aforementioned soft-bake temperature have been optimized for a sufficient radiation dose and a controllable dissolution rate, whilst inspecting feature sizes of the patterned photoresist under microscope against the design value. The substrate is then hard-baked on the hotplate at 120°C in 2 minutes for polymerization of the resist. The post-baked substrate is put into a 25% HCl solution on a hotplate at 50°C for etching in 65 seconds. The HCl concentration, etching temperature and time have been optimized by adjusting etching rate and surface roughness of the ITO film. The unexposed photoresist layer is finally stripped by acetone, followed by post-process cleaning. The fabricated patterns are well shaped and sized as observed in Fig. 3 below. The measured feature sizes of smallest isolated circles exhibit a maximum difference of 0.5μm in radius as compared to the designed value of 7.8μm, an accuracy of 6.4%.

 

Fig. 3 Patterned ITO electrode (peripheral region of two adjacent Fresnel lens) by photolithography. Measured diameters: a = 16.68μm, b = 16.62μm.

Download Full Size | PDF

After the ITO patterning is completed, the switching device is assembled with the following steps: SmA PS consists of a glass plate with photo-lithographically patterned ITO electrodes, and another piece of glass with continued ITO coating layer. A PS array is constructed in such a way that the two substrates are overlapped in parallel with a cell gap of 10μm, and then filled with SmA LC (supplied by Dow Corning Ltd.) in isotropic phase (≈ 100°) by capitally filling. Both substrates are pre-treated with homeotropic alignment agent (SE1211, Nissan Chemicals ltd). This is to ensure the none-conductive region (e.g. the central hole) will be pre-aligned to allow light to pass through (at transparent state), while the liquid crystal on conductive region is re-configured by E-field. Finally, when the device is switched into cleared state, a PS is formed. Otherwise, when switched into scattered state, the entire device area becomes diffusive, either to block light or allow additional image to be displayed. Figure 4 shows the switched states of the device. It does not need any power to keep the structure at its switched state.

 

Fig. 4 The images of PS device on a black background. When the device is in the scattering mode (a), all regions become diffusive. In the clear mode (b), the LC molecules are trapped under the circular photon sieve features. The light is diffracted around these diffusive regions. Inset shows a single PS structure.

Download Full Size | PDF

By using a PS array, we ensured the yield of the switching PS structures to be high. Some of the elements in the array performed worse than the others during the switching process. Figure 4(a) shows that the PS elements on the left are seen faint compared to the ones on the right. One reason is the variation in the thickness of the gap between the glass layers while the device is assembled.

3. Experimental setup and results

The optical setup shown in Fig. 5 is constructed to test the concept. We used the device to image a test object formed by placing a mask in front of a green LED source (Thorlabs M530L3), at a distance of 2f = 100mm, on the sensor of a Sony a7r2 camera at the imaging distance distance of 2f = 100mm while projecting an image on the sample from an Optoma ml750e DMD projector with LED light source. The LC PS device was switched outside of the setup and placed afterwards before testing of each state. This is another very important feature of the proposed solution. It does not need to draw constant power as in the case of PDLC material based solutions.

 

Fig. 5 The optical setup used to test the device in the concept described in Fig. 1.

Download Full Size | PDF

The working principle of the device as follows. When the device is in the normal display mode, it is in the fully scattering state and a picture is projected on to the screen as shown in Fig. 6(a). If the device is switched to the PS state for imaging by partially clearing the screen, the features of the PS in the cleared state will diffract the light forming the images on the camera sensor as shown in Fig. 6(a), while the projected picture on the remaining diffusive parts is still visible as shown in Fig. 6(b). The device can stay in this state indefinitely or until it is switched back to the fully scattering state.

 

Fig. 6 (a) and (b) are fully scattered and partially cleared PS states, respectively. The insets are the actual images captures on the corresponding screens. In the inset image of (b), the gaps between the photon sieves are due to the designed mask. It should be obvious to the reader that a square design for the photon sieve will prevent such gaps.

Download Full Size | PDF

The insets of Fig. 6(a) and (b) show projected images on the diffuser when the PS is switched on and off, respectively. The PS mask was previously designed for testing the image quality of PS with circular apertures. In this study, we used the mask for fabricating LC for demonstration purpose only. Hence, at the parts where the PS features are present, the projected image is displayed and observed. At other parts, the gaps between the structures are present. For real applications, the mask should be redesigned, e.g. with square aperture for each element such that the structure will be seen as a continuous diffuser since the minimum feature size is well below detectable size by human eyes at the given observation distance, which was 2.5m in our case. This will ensure that there is no ambiguity in the observed image on the screen. Furthermore, introducing more features at those parts can further improve the focusing power of the PS structures. Note that, the center pinhole, which shows a gap smaller than the pixel size, is not present in the design of the PS array by choice. In fact, this pinhole is removed since it increases the amount of undiffracted light even for the original mask structure [11]. The design also offers an option that if the PS structures are not needed for imaging, the device can be switched to full scattering state.

Using the device, we imaged the object while projecting a picture on it at the same time. In Fig. 7(a), the images captured by the camera in the PS state are shown. In the full scattering state, no imaging is possible since the diffuser blocks the light from the object as shown in Fig. 7(b). Some scattered light from the diffuser and the projection reaches to the sensor as background noise as illustrated in Fig. 6(b). The inset in Fig. 7(a) shows close up of an image of the object and the corresponding result when the background is removed. We also compared the imaging quality of the PS mask to off-the-shelf optical lenses in another study and we confirmed that the performance of the PS mask is comparable to the conventional optical lenses [11].

 

Fig. 7 The recorded image of the test object at the 2f imaging distance when PS is (a) on and (b) off. The PS structure on the diffuser images the test object when it is switched on. The inset shows close up of an image of the object and the corresponding result when the background is removed.

Download Full Size | PDF

4. Discussion

It is observed that the device has an acceptable imaging quality. One challenge is that a specific PS device would work the best at the designed wavelength. By designing similar PS devices for other wavelengths in the visible spectrum, a full color imaging is possible. The images of the same scene with say three different wavelengths for primary colors can be obtained and then combined to produce a color image. Hence, it would be possible to capture full color images of a scene with multiple built-in PSs on the diffuser screen while projecting an image on the same screen simultaneously.

When the sample is switched for purely image projection, the whole sample goes into a diffusive state as shown in Fig. 4(a). When it is switched to the partially cleared PS state for imaging, the PS features remain diffusive as shown in Fig. 4(b), scattering the projected lights which should not be confused with haze. It is possible in this case to use the device for imaging purpose according to the Babinet’s principle [12–14]. In this way, we can have a diffusive screen combined with embedded focusing PS features, which can be switched on and off.

When the sample is switched from the purely projection mode to the PS image mode, the projected lights on the PS features can in principle generate diffractive images of certain levels to the viewers, hence degrading the quality of the projected image. In practice, we did not observe such phenomenon. We attribute it to the facts that in our case the diffusive areas are larger than that of cleared ones and PS features are designed for a single wavelength only with moderate diffraction efficiencies.

The aim of embedding PS features on a diffusive screen for non-intrusive imaging during projection is for the applications like eye-to-eye video conferencing, which requires a medium or large field-of-view (FOV) to capture the scene. The work presented here is to demonstrate the feasibility of such a concept, but with a limited FOV. This can be improved in future by optimization of the PS design and tiling several PS imaging apparatuses in both horizontal and vertical directions.

The PS imaging method used here is intrinsically similar to that of a pinhole imaging approach. Although the depth of field is expected to be deep due to the small size of the focusing elements, its overall performance including the best spatial range of performance and its relationship with the feature size as well as the targeted wavelength range should be carefully analyzed and optimized in future.

It is also possible to change the design wavelength from visible to infra-red region. This way, hand or eye tracking applications can be embedded in the display systems where high-resolution imaging is not crucial.

It is also envisaged that these structures can be embedded on flexible substrates [15] rather than rigid glass substrates. This will introduce more degree of freedom which will open up other applications areas.

5. Conclusion

A LC PS device is assembled using SmA LC material which can switch between scattered and partially cleared PS states on a conventional projection screen. The device requires neither any power to keep its current state nor electrodes to activate individual features. Therefore, it is bi-stable. In the PS state, the device acts as part of the screen for an external picture source and as a focusing element to image the scene in front of the device on a camera sensor behind itself, simultaneously. When the PS structure is switched on, it was possible to view the displayed images on the the device almost without any detectable feature at the observation distance. Moreover, the device was capable of imaging with acceptable quality at the same time. It was also shown that the design of the PS does not need to consist of transparent pinholes, rather it can block light at the place of the pinholes and allow light to pass through elsewhere. Although, the proposed device is especially useful for eye-to-eye video conferencing setups, oher applications including eye or hand tracking, privacy, and surveillance can also benefit from this proposed solution.