Open Access
Add to BibSonomy
Abstract
Light-field imaging is a crucial and straightforward way of measuring and analyzing surrounding light worlds. In this paper, a dual-polarized light-field imaging micro-system based on a twisted nematic liquid-crystal microlens array (TN-LCMLA) for direct three-dimensional (3D) observation is fabricated and demonstrated. The prototyped camera has been constructed by integrating a TN-LCMLA with a common CMOS sensor array. By switching the working state of the TN-LCMLA, two orthogonally polarized light-field images can be remapped through the functioned imaging sensors. The imaging micro-system in conjunction with the electric-optical microstructure can be used to perform polarization and light-field imaging, simultaneously. Compared with conventional plenoptic cameras using liquid-crystal microlens array, the polarization-independent light-field images with a high image quality can be obtained in the arbitrary polarization state selected. We experimentally demonstrate characters including a relatively wide operation range in the manipulation of incident beams and the multiple imaging modes, such as conventional two-dimensional imaging, light-field imaging, and polarization imaging. Considering the obvious features of the TN-LCMLA, such as very low power consumption, providing multiple imaging modes mentioned, simple and low-cost manufacturing, the imaging micro-system integrated with this kind of liquid-crystal microstructure driven electrically presents the potential capability of directly observing a 3D object in typical scattering media.
© 2018 Optical Society of America under the terms of the OSA Open Access Publishing Agreement
1. Introduction
An electrically tunable light-field camera is a typical three-dimensional (3D) imaging system that captures both the intensity and direction of beams emanating and transmitting in typical scattering atmospherical circumstance from targets by inserting a liquid-crystal microlens array (LCMLA) between the main lens and imaging sensors [1,2]. So far, the application of the imaging system based on the LCMLA has been extended from the visible range to near infrared wavelength region [3]. Unlike common plenoptic cameras based on traditional silica microlenses [4,5], the depth-of-field of the light-field camera based on a LCMLA can be extended through electrically tuning the focal length of the LCMLA. Due to the intrinsic uniaxial anisotropy of LC molecules, the key issue of this electric-optical device is that it is polarization-sensitive to incident beams [6]. Hence, relatively strong stray light that cannot be completely focused by the LCMLA will surround the focal point of each microlens and then the final imaging quality will be decreased. Furthermore, it will reduce the ability of the LCMLA to detect targets with high polarization state or in dark environment [7].
To date, various researches such as using polymer dispersed liquid-crystal (PDLC), blue phase liquid-crystal (BPLC), and axially symmetric pre-alignment of LC molecules, have been suggested [8–12]. Although the methods of utilizing so-called polarization-independent LCMLA can improve the focusing efficiency, they also minimize the potential application through dehazing operation so as to sacrifice polarization features [13]. Recently, a multifunctional LCMLA composed of two layered nematic LC materials with orthogonal alignment has been proposed by Xin et al. in [14]. By adjusting the driving signal voltage sets, the device cannot only shape a focusing spot with no polarization dependent but also keep the polarization-sensitive property. It should be noted that the imaging quality can be further improved by decreasing the thickness of the separating layer fabricated, and those stray beams can also be employed effectively, which are still been wasted in polarization-sensitive state.
The polarization is a fundamental physical attribution of light. Polarization difference imaging (PDI), which can be used to enhance the visual contrast of objects, is another image information capturing means. The structural characters such as the shape and the shading and the texture of surface architectures, can be extracted from the polarization information of objects. Currently, the above clues have been widely used for improving imaging efficiency in scattering media, biomedical detection of cancerous tissue, and material identification [15–20]. As shown, the PDI across objects can reveal several particular features, which cannot be seen by conventional cameras. Generally, the common method of realizing PDI system is to insert a rotated polarizer into the optical path of a conventional imaging camera to sense orthogonal linearly polarized intensity [21]. However, the rotating part will bring measuring errors between two images of the same scene, and also the rotation rate is too small to achieve reasonable frame rate. Nowadays, the camera with pixelated polarization filters constructed by metal nanowires located at the focal plane has been built to achieve ideal polarization state of objects [22, 23]. But, the polarization-dependent property of the LCMLA cannot be solved according to the approach.
Extensive study of 3D measurement through plenoptic cameras based on the key LCMLA has been performed by Lei et al. in [24]. With the conventional LCMLA, the featured parameters including the focal depth, the positioning and the motion expression of 3D objects in scattering atmospherical media, can be sensed and recorded by elemental image cells and further calculated by imaging micro-system. Owing to the intrinsic polarization property of common LC materials, the stray light with an electric vector vibrating perpendicular to the optical axis of the LCMLA, will pass through the LC layer without beam converging [14]. Hence, the light-field camera based on conventional LCMLA without polarizer can only provide relatively inferior signal-to-noise ratio (SNR). So, the image quality will be decreased obviously. And, the prototype will diminish the 3D information of objects with highly partial polarized light or in complex scattering media. So, the polarization imaging combined with light-field information can be used to tackle this problem. Hence, it is necessary to build an imaging micro-system to sense and record both the wave vector or light field direction and the polarization of incident beams, simultaneously.
In this study, we present a twist nematic liquid-crystal microlens array (TN-LCMLA) for capturing the light fields and orthogonal polarization information of objects, which combines a twist nematic liquid-crystal (TNLC) cell, a polarizer and a LCMLA. Without additional optical and mechanical components, this kind of optoelectronic device can be directly integrated with common CMOS imaging sensors and then utilized to perform multiple functional imaging operation by directly manipulating incoming beams focused by the main lens. When the driving voltage signals applied on the LCMLA is turned off, the orthogonally two-dimensional (2D) polarized images are obtained continuously by adjusting the root mean square (RMS) voltage value over the TNLC cell. Turning on the LCMLA and regulating the signal voltage of the TNLC cell, light-field images consisted of horizontally polarized image and vertical polarized image, can be captured efficiently. Experiments demonstrate that the polarization light-field images can be effectively acquired via switching on or off the TNLC cell in sub-millisecond scale based on the performance of the liquid-crystal material used by us. Meanwhile, the polarization-insensitive images can also be obtained by relatively simple computation. Compared with conventional LCMLAs, the stray light will be eliminated and accordingly the image quality will thus be improved. Thus, the TN-LCMLA can be used to reveal the detailed information of objects according to the 2D or 3D imaging state of the TN-LCMLA selected only through loading a suitable set of driving voltage signal.
2. Structure and principles
2.1 Architecture
The detailed schematic of the TN-LCMLA designed by us is shown in Fig. 1. As a key component for construction a dual-polarized light-field imaging (DPLFI) micro-system, the TN-LCMLA involves two stacks of LC layer confined in both micro-cavities with corresponding electrodes shaped according to initial anchoring direction defined. As shown, the TN-LCMLA is composed of a twist nematic liquid-crystal cell (TNLC) and a polarizer and a LCMLA. Both micro-cavities in the TNLC and the LCMLA are fully filled with nematic LC materials (Merck E44: no = 1.5277 and ne = 1.7904). The depth of both micro-cavities is determined by the diameter of glass spacers used to maintain basic micro-cavity structure, is 10μm and 20μm, respectively. As shown in Fig. 1(a), two ~500μm silica substrates are pre-coated by a layer of indium thin oxide (ITO) film with a thickness of ~50nm over both end surfaces. Continuously, a thin polyimide (PI) layer with a typical thickness of ~2μm is spin-coated on the surface of each ITO-layer of the TNLC cell. After ~230°C baking for 30min, the PI layers are strongly rubbed to shape initial LC molecule anchoring grooves, which are perpendicular to each other, so as to form a twisted configuration between two substrates of the TNLC. The bottom PI layer of the TNLC is also rubbed paralleled to the optical axis of the LCMLA. Both ITO films of the TNLC act as planar electrodes to control the reorientation of LC molecules driven by external voltage signal applied on the TNLC.
The key functional structures of the LCMLA are both ~500μm silica substrates with different conductive film pre-coated over their end surface. The top substrate of the LCMLA is deposited by an aluminum film. After conventional UV-photolithography and wet-etching process, the patterned aluminum electrode with an arrayed micro-hole is formed. The structural parameters of micro-holes, which are shown in Fig. 1(b), is 128μm diameter and 160μm center-to-center spacing. A layer of ITO film is further deposited over the surface of another substrate of the LCMLA, which is referred to as a planar electrode. The similar PI layers are continuously spin-coated on the patterned and planar electrodes of the LCMLA and thus baked for 30min at 230°C. The fabricated PI layers are also rubbed anti-paralleled to each other, and then tightly sealed face to face to maintain the micro-cavity geometry. In the end, the polarizer is sandwiched between the TNLC and the LCMLA, and finally sealed together. It should be noted that the transmittance axis of the polarizer is paralleled to the rubbing direction of the bottom PI layer of the TNLC or the initial optical axis of the LCMLA.
2.2 Principle
The schematic of the TN-LCMLA coupled with an imaging sensor array is given in Fig. 2. As shown, incident beams can be decomposed into two linearly polarized light with orthogonal direction, which are represented by Ex (purple line) and Ey (red dot). The TNLC, which is referred to as a light rotator, is a key component of the TN-LCMLA to rotate sub-beams focused by the main lens. The twisted nematic effect (TN-effect) as a main property of the TNLC cell is used. In the off state of the TNLC, when external signal voltage applied on the TNLC cell is absence, a twisted configuration of LC molecules is formed by initially orthogonal alignment of PI layers fabricated. The polarization of incident beams will be rotated by 90°. In the on state of the TNLC, when a needed external signal voltage is applied on the TNLC cell, LC molecules between two planar electrodes of the TNLC will rearrange along the electric-field direction and finally vertical to the TNLC cell. So, the initial beam rotation of the TNLC will disappear. The polarizer is used to select corresponding polarized beams. Another important component of the TN-LCMLA is the LCMLA, which is used to form the light fields of imaging scene.
Fig. 2 The principle of the dual-polarized light-field imaging micro-system. (a) Schematic of the TN-LCMLA firstly interacting with incident beams. The beams transmit through the LC layer confined in the TNLC cell and their polarization is rotated in 90°. The polarizer selectively transmits light that presents the same direction with the optical axis of the LCMLA. The selected beams are continuously focused on the focal plane so as to further put onto the CMOS sensors. (b) The driving voltage signal across the TNLC cell will reorient LC molecules and thus keep the polarization state of incident beams. Only one polarized light is converged by the LCMLA.
The section view of a single TN-LCMLA cell is illustrated at the right in Fig. 2. The black dash line in the single TN-LCMLA cell indicates an equivalent reflective index distribution profile of a LC microlens formed under the needed driving voltage. When a driving voltage signal is applied on the LCMLA, a gradient refractive index distribution corresponding to a rearrangement of LC molecules under each micro-hole pattern is formed. Hence, incident beams can be converged and the focal length can be further adjusted by turning the RMS value of the driving voltage signal applied.
As shown in Fig. 2, incident beams impinging on the TN-LCMLA will firstly interact with the TNLC stack. The beams escaping from the TNLC, is then emerging and further passing through the polarizer so as to eliminate those unintended beams, which are perpendicular to the transmittance axis of the polarizer. Continuously, the beams with the same light vibrating direction as the polarizer will propagate through it, and then be converged by the LCMLA with a proper RMS voltage value of the driving signal loaded. Hence, the light intensity distribution will be sensed by the arrayed sensors of the camera. The pixel intensity distribution with orthogonal polarization can be represented by I∥(x, y) and I⊥(x, y), where (x, y) is the pixel position of sensors used, and both symbol ∥ and ⊥ indicate the polarization direction.
When turning off the TNLC in Fig. 2(a), the orthogonal field component Ex and Ey of incident light are rotated by 90°. Hence, the vertically polarized light will be picked out from incident beams and further converged by the TN-LCMLA, which means that the I⊥(x,y) is already sensed and recorded. When turning on the TNLC in Fig. 2(b), the orthogonal field component Ex and Ey of incident light will pass through the TNLC without rotation. So, the beams whose polarization direction is paralleled to the polarizer, will be converged by the TN-LCMLA, and then the component I∥(x,y) will be sensed and recorded.
3. 3D visualization of objects with different polarization
3.1 Depth estimation
The dual-polarized light-field imaging micro-system based on the key TN-LCMLA is built to sense and then record and finally retrieve two orthogonally polarized raw images of objects in sub-millisecond scale. The basic physical model used in this paper for calculating the depth information is proposed by our previous work in [24]. The irradiance sensed by the proposed DPLFI micro-system include both the polarization and light-field information. To estimate a precise depth of the object, the ballistic component of incident beams is a key parameter. The schematic of the depth evaluation based on Galilean model is depicted in Fig. 3. In this work, the pinhole model is used to estimate the depth information of the object.
As shown in Fig. 3, a virtual point P’ created by the main lens is located behind the imaging sensors. The rays from object point P between the main lens and two adjacent LC microlenses are refocused onto the point P1 and P2 of the focal plane or the imaging sensors, which is also the imaging plane of the TN-LCMLA.
Considering the main lens as an ideal lens, the object distance aL of object point P and the virtual image distance bL of virtual point P’ satisfy the following equation:
where fL represents the focal length of the main lens.According to Fig. 3, the following parameters: ei (i ∈ {1, 2}) and di (i ∈ {1, 2}) should be measured before estimating the object distance. The positive direction of these parameters is assumed to be upward. Hence, e1 and d1 are negative, whereas e2 and d2 are positive. According to the relationship of similar triangles, it yields:
andwhere ei represents the distance of the orthogonal projection of the virtual image point on the sensors with respect to the principal points of the micro-images, and di defines the distance of corresponding points with respect to the principal points of the micro-images, and b is the distance between the bottom LC microlens and the imaging sensors, and e is the center-to-center distance between two adjacent microlenses.If parameter is defined as a vertical disparity of two image point (P1, P2), which are remapped according to two adjacent microlenses, the Eq. (2) can be yielded as
Therefore, from the Eq. (4), the virtual point distance a can be calculated asSubstituting Eq. (4) into Eq. (1), the resolving progress can be written as:where l is the distance between the main lens and the bottom LC microlens.Due to parameters b, e, l and fL, are constant, we find that vertical disparity is a key parameter to obtain the depth of object P, and can be solved by the matching algorithm of the image template selected. The depth information of object point P is determined by multiple parameters from Eq. (6). With the increasing of the distance between the object point and the camera, the precise of the depth will be decreased gradually.
3.2 Evaluation of imaging quality
To quantitatively evaluate the clarity of images acquired, the RMS contrast [25] and a variance method are proposed in this section. The RMS contrast, which is a standard deviation of pixel intensity, is defined as Eq. (7). The pixel intensity should be normalized in the range [0, 1] for calculating a correct RMS contrast.
where is the intensity of pixel (i, j) in 2D image of size M by N. is the mean intensity of all pixel in the image.The variance method is utilized to evaluate the degree of the deviation between a set of discrete data and its expectation according to probability theory. The image with a high quality has a great gray-scale difference, which means that the variance of pixel intensity is large. Otherwise, the imaging quality is poor. The variance evaluation function is defined as
where is the intensity of pixel (i, j) in the gray image with a size of M by N. is the mean intensity of all pixel in a gray image.4. Experiments and discussions
4.1 Optical property of the TN-LCMLA
The point spread function (PSF) and the focal length of the TN-LCMLA are measured using an objective of × 40 and 0.65 numerical aperture, and a Standard Beam Profiling Camera (WinCamD of DataRay, Inc.). The collimated beams with a central wavelength of 671nm out from laser device (Changchun New Industries Optoelectronics Tech. Co. Ltd) are normalized by a linear polarizer from Thorlabs Inc. (operating wavelength range: 400-700nm). The tested sample is inserted between the polarizer and the objective.
To verify the impact of the polarization state on the PSF remapped at the focal plane, a conventional LCMLA and a fabricated TN-LCMLA are measured through changing the RMS voltage value applied on the devices but maintaining a constant focal length of 1.2mm, respectively. The same focal length of both samples means that the voltage signal of ~4.47 Vrms on the LCMLA is equal to that on the same part of the TN-LCMLA. The transmittance axis of the polarizer is adjusted at 45° with respect to the optical axis of the samples. Several typical non-axisymmetric PSFs are shown in Fig. 4. According to experiments, some problems associated with the non-axisymmetric PSF cannot be solved only by placing a polarizer just before the LCMLA used. It should be noted that the imperfect PSFs of tested samples can be attributed to the anisotropic character of liquid crystal molecules distributed in a specific electric-field stimulated in the LCMLA, which cannot be used to form an arrayed axisymmetric effective refractive index profile for shaping ideal focus of polarized incident beam. The fabrication and measurement errors will also bring into deviation so as to further degrade beam profile shaping, and then lead to final non-axisymmetric PSFs.
Fig. 4 The PSFs of the devices. (a)The converging pattern of the conventional LCMLA at ~4.47 Vrms and the focal length of 1.2mm. (b)Switching on the TNLC cell (~10 Vrms), the PSF of the TN-LCMLA at ~4.47 Vrms on the LCMLA and the focal length of 1.2mm. (c) Switching off the TNLC cell (0 Vrms), the PSF of the TN-LCMLA at ~4.47 Vrms on the LCMLA and the focal length of 1.2mm. (d) The relationship between the optical axis of the sample and the transmittance axis of the polarizer used.
As shown in Fig. 4(d), the linear 45° polarized light E from the polarizer is decomposed into two components of Ex and Ey. Where Ex and Ey are parallel and perpendicular to the optical axis of the sample, respectively. The converging patterns obtained and determined by the LC microlenses, show the influence of the polarized characters over the conventional LCMLA and the TN-LCMLA. At 4.47 Vrms applied on the conventional LCMLA, the lens effect is presented and the focusing patterns are recorded by the beam profiling camera [Fig. 4(a)]. Figure 4(a) shows that only incident beams with a polarization paralleled to the rubbed polyimide alignment is focused. In this circumstance, a set of concentrically circular rings was observed, simultaneously. The LC layer of conventional LCMLA becomes a phase retarder relative to the residual light. The residual beam with a polarization perpendicular to the optical axis of the conventional LCMLA distribute around the focusing spot in a fashion of interference fringe. This has been demonstrated by prior researchers in [26–28]. Compared to the PSF of the conventional LCMLA, two perpendicular components of incident beams can be converged by TN-LCMLA effectively. The stray light relative to the imaging micro-system has been completely eliminated through the TN-LCMLA via switching on or off the TNLC cell [Figs. 4(b) and 4(c)].
Another important property of the TN-LCMLA is that two orthogonal polarized PSFs are acquired in sub-millisecond scale. Generally, the response time of nematic liquid-crystal devices is linearly proportional to rotational viscosity (γ1). According to our experiments, the surface anchoring energy of liquid-crystal molecules and the thickness of the liquid-crystal layer also demonstrate remarkable impact on the response time of liquid-crystal molecules. Experimental verification of conventional nematic liquid-crystal (E44) with typical sub-millisecond scale response time has been reported in 1987 [29]. Loading a signal voltage on the TNLC cell, the twisted effect gradually vanishes because LC molecules will reorient according to the electric-fields generated in LC layer. Further increasing the signal voltage until 10 Vrms, LC molecules will be vertical to the inner surface of the TNLC cell, and then the twisted effect will eliminate, and finally the LC molecules' alignment is along the electric-field direction. As shown in Fig. 4(b), two components (Ex and Ey) of Linear 45° polarized light transmit through the TN-LCMLA without any rotation at ~10 Vrms on the TNLC cell. The Ex-component is selected to transmit the polarizer and then converged. The residual Ey-component is absorbed. Figure 4(c) shows a focusing pattern of Ey-component, where Ex and Ey are all rotated in 90° because the TNLC cell is already switched off.
To demonstrate the focusing performance of the TN-LCMLA, the relationship between the focal length and the driving signal voltage applied on the sample is shown in Fig. 5. The focal lengths of Ex-component and Ey-component of incident light are equal. As shown in Fig. 5, when the driving signal voltage applied on the LCMLA is more than ~1.0 Vrms but still less than ~4.5 Vrms, the focal length will be decreased fast with the increasing of the driving voltage. After exceeding an experimental value of ~4.5 Vrms, the focal length begins to increase gradually as the increasing of the signal voltage in an effective variance range from ~4.5 Vrms to ~9.0 Vrms. In the range of ~1.0 Vrms to ~9.0 Vrms, the focus-to-voltage curves present a parabola-shaped trend. Experiments demonstrate that two components of incident beams have the same focal length under the same RMS voltage.
4.2 Imaging application of the DPLFI micro-system
The measurement schematic of the dual-polarized light-field imaging micro-system, and the camera prototype, and the typical features of the TN-LCMLA fabricated, are illustrated in Fig. 6. The orthogonal light-fields and the polarized images were measured by the TN-LCMLA prototype. According to the proposed DPLFI micro-system, the object points are illuminated by a light source such as sun or other active illuminants. As shown in Fig. 6(a), the polarization and 3D information of the objects carried by reflected beams out from objects are totally recorded by the camera prototype located at the focal plane of the main lens. Hence, the proposed TN-LCMLA is built to create multimode imaging correlation according to the polarization state and the light-field characters of incident beams focused towards the focal plane of the main lens. Figure 6(b) depicts an imaging arrangement including the main lens, TN-LCMLA and the camera used. A main lens (M3520-MPW2) with a focal length of 35mm is used in the prototyped imaging micro-system, which consists of a TN-LCMLA [Fig. 6(c)] integrated onto a CMOS sensor array of MVC14KSAC-GE6-NOO of Microview. The size of the CMOS sensors is 4384 × 3288 with a pixel pitch of 1.4μm. The polarization light-field images are continuously obtained by adjusting the driving signal voltages applied on the TN and LCMLA cells. After relatively simple computation using our common algorithm, the correlated images with higher quality, which can be used to retrieve the polarization-independent or polarized 3D information of objects (e.g. depth or refocusing after a single exposure of the camera prototype), can be achieved. In this experiment, the F-number of the main lens is set as 5.6 to match that of the LC microlenses, which can effectively eliminate the crosstalk between sub-images under a larger aperture, and thus make a full use of the resolution of the imaging sensors.
Fig. 6 (a) Measurement schematic for performing light-field and polarization imaging based on the TN-LCMLA, and (b) dual light-field and polarization imaging prototype, and (c) basic structural features of the TN-LCMLA.
As shown, the dual-polarized light-field imaging micro-system can be used to obtain two orthogonal polarization light-field images by adjusting the driving signal voltage applied on the TNLC cell. One is a horizontally polarized image, and another show a vertically polarized image. Both images are denoted as I∥(x, y) and I⊥(x, y), respectively.
Once the optical properties of the TN-LCMLA are specified, the fabricated device can be combined with a CMOS sensor array to obtain polarized light-field images. A critical advantage of the proposed DPLFI micro-system is that two orthogonally polarized light-field images without any stray light can be acquired in sub-millisecond scale. In order to verify the capability mentioned, the orthogonally polarized light-field images of the conventional LCMLA and the TN-LCMLA are obtained, respectively, through substituting LC device in the same imaging micro-system. To make sure the imaging micro-system only receive needed incident light with particular polarization paralleled or perpendicular to the optical axis of the LC device, a polarizer is inserted between the main lens and the object.
Figures 7(a) and 7(b) show the horizontally and vertically polarized light-field images of the conventional LCMLA. The reflective beams of the target firstly pass through a polarizer, and then the main lens, and finally reshaped by the LCMLA. From the light-field images of a block in Figs. 7(a) and 7(b), it is evident that the conventional LCMLA is polarization-sensitive and a blurry light-field image will be shaped because of the stray light cannot being focused by the LC devices. As shown in Fig. 7(a), the light-field image of the object is very clear at the signal voltage of ~3.5 Vrms on the LCMLA. This can be attributed to the surface anchoring, which determines the optical axis of the LCMLA. The magnified image in blue frame, which already exhibits detailed beam intensity distribution and sharpness of the PSF, is shown in red frame. We can make sure that the transmittance axis of the polarizer is perpendicular to the optical axis of the LCMLA through rotating the polarizer. Due to the polarization-dependent behaviors of conventional LCMLA, the lens effect will disappear. This has been evidenced by the situation of the PSF in red frame inserted in Fig. 7(b). According to the principle of pinhole imaging, a blurry image is obtained because the patterned electrode of the LCMLA can be viewed as an arrayed micro-hole.
Fig. 7 Comparing the light-field images of the TN-LCMLA with that of the LCMLA under two orthogonally polarized states. (a) The 0° polarized light-field image of the conventional LCMLA at the signal voltage of ~3.5 Vrms. (b) The 90° polarized light-field image of the conventional LCMLA at ~3.5 Vrms. (c) Switching on the TNLC cell (~10 Vrms), the 0° polarized light-field image of the TN-LCMLA at ~3.5 Vrms. (d) Switching off the TNLC cell (0 Vrms), the 90° polarized light-field image of the TN-LCMLA at ~3.5 Vrms.
Compared with conventional LCMLA, the TN-LCMLA can export two orthogonally polarized light-field images without any stray light in sub-millisecond scale, as demonstrated by the light-field images in Figs. 7(c) and 7(d). Hence, the TN-LCMLA is polarization-independent. It should be noted that the two orthogonally polarized light-field images still rely on the driving signal voltage applied on the TNLC cell. This means that a fixed voltage signal should be applied on the TNLC cell so as to effectively adjust the status based on the twisted effect only by switching on or off the TNLC cell, as demonstrated by the magnified image in blue frame, and the sharpness of the PSFs in red frame shown in Figs. 7(c) and 7(d).
To estimate the depth of object under polarization condition, several experiments were performed, as shown in Fig. 8. An external polarizer was inserted between the object and the DPLFI micro-system to provide the ideal polarized beams reflected from the object. The angle between the transmittance axis of the polarizer and the optical axis of the TN-LCMLA is 70°, and the images were acquired by the DPLFI micro-system. In the context, the DPLFI micro-system presents an ability to acquire orthogonally polarized light-field images that can be combined with total intensity distribution. It should be indicated that the ability mentioned above can be continuously employed in such illumination circumstance with arbitrary polarization state.
Fig. 8 (a) Horizontally polarized light-field image, and (b) vertically polarized light-field image, and (c) the total intensity image, and (d) the rendering image.
Figures 8(a) and 8(b) depict the horizontally and vertically polarized raw images by switching on or off the TNLC cell, respectively. Due to the electric-vector in x-component is less than that in y-component, the intensity distribution of horizontally polarized light is lower than vertically polarized light [Figs. 8(a) and 8(b)]. From the PSFs inserted in the Figs. 8(a) and 8(b), it can be inferred that the stray light caused by the intrinsic property of LC materials has been eliminated completely. Compared with the image acquired by the conventional LCMLA, the image quality has been improved remarkably. Figure 8(c) gives a typical computational light-field image with total light intensity, which means that the capability can extend the application field of the LCMLA through adding some new functional microstructures.
The following parameters should be measured and calibrated before performing depth estimation. The approximate value of b, which equals the total thickness of the bottom silica and the protective glass window of the sensor array, is ~1.3mm. The value e, which is determined by the structure of the hole-patterned electrode, is 160μm. After calibration, the focal length fL of the main lens is 39.8mm, and the distance l between the main lens and each microlens of the TN-LCMLA is 38.5mm. According to image matching algorithm and Eq. (6), the depth of the object can be calculated. Using the rendering algorithm for image synthesis proposed in our prior research [1], the refocusing result has been illustrated in Fig. 8(d). Hence, the all-in-focus rendering image is shown in Fig. 8(d), where the reconstruction distance is 59cm according to the depth estimation algorithm denoted above.
In order to comparatively evaluate the imaging quality according to conventional LCMLA and TN-LCMLA imaging micro-system, several raw images indicating the same object is given in Fig. 9. As shown in Fig. 9(a), the light-field image, which is already blurry, is remapped on the COMS sensors through the polarization-sensitive LCMLA. The operation of adding two orthogonally polarized light-field images from the TN-LCMLA together, will lead to a polarization-insensitive imaging, as shown in Fig. 9(b). The clarity of images shown in Fig. 9 can be quantitatively evaluated by using an algorithm introduced in section 3.2. The values of the RMS contrast and variance are listed in Table 1. Raising the value of the RMS contrast and variance, the clarity of the images will be increased. As expected, Fig. 9(b) exhibits higher RMS contrast and image clarity because the imaging micro-system based on the TN-LCMLA has a sharper PSF without any stray light. The improvement ratio, which is defined to indicate the performance improving situation, is also calculated according to the value of the contrast and variance. The polarization-insensitive image obtained by the DPLFI micro-system, which has the same field of view (FOV) with Fig. 9(a), present an improvement contrast ratio of ~165.22% and an improvement variance ratio of ~377.62%, which means that the image quality is highly improved.
Fig. 9 (a) Raw images of 'E' block remapped by a conventional LCMLA, and (b) the polarization-insensitive raw image of 'E' block remapped by the TN-LCMLA developed.
Table 1. Evaluation of the imaging quality shown in Fig. 9
As demonstrated, beam polarization has a potential capability to enhance imaging contrast and also tell more detailed information under the shading. The improved performance of DPLFI micro-system over conventional intensity imaging system can be directly attributed to the particular polarization state emanating from a scene and then an ability to capture both the intensity and direction of light rays travelling in medium. To emphasize the polarization imaging capability of our prototype, the DPLFI micro-system is exploited to capture the light-field images with different polarization state under natural environment, which will lead to direct three-dimensional observation. Figure 10 presents a conventional intensity image and two orthogonal polarized light-field images of a scene with two toys (a fire truck and a tree) in front of buildings under sunset. To make the entire light-field images easier to understand, a conventional intensity image of the scene has been shown in Fig. 10(a) before we acquired corresponding light-field images of the same scene. From the polarized scene images, it is evident that the different polarization state can reduce the flare over the object surface, and then provide various detailed information of the targets. For example, the flare of the tree in red frame shown in Fig. 10(b), has been decreased relative to the same area shown in Fig. 10(c). This can be attributed to the specular reflection emanating from the tree, which has various polarized components in both orthogonal axes. As shown in Fig. 10, the magnified image of a fire truck is very obvious in two blue frames. The magnified image in the blue frame, as shown in Fig. 10(b), indicates a detailed surface information of a fire truck according to the horizontally polarized light-field photograph taken using our prototype. The magnified image in the blue frame, as shown in Fig. 10(c), demonstrates that the surface information of fire truck has almost been hidden.
Fig. 10 A complete light-field image captured by our prototype. The scene is illuminated by sunset. (a) The conventional intensity image of a scene with a fire truck and a tree in front of buildings. (b) Horizontally polarized light-field image remapped by our prototype, and (c) vertically polarized light-field image acquired by the same set-up.
Another important feature of the imaging micro-system is that it can be switched into 2D imaging mode at 0 Vrms. Decreasing the driving voltage of the LCMLA to 0 Vrms, the lens effect will disappear and the LC layer of the LCMLA will also be transformed into a phase retarder. In order to achieve two polarized images with obvious difference, a polarizer is placed in front of the object to create a polarization experimental condition. The angle between the transmittance beams of the polarizer and the optical axis of the TN-LCMLA is 60°.
Figure 11 demonstrates several 2D images recorded when the driving voltage of the TNLC varies from 0 to ~10 Vrms. As the increasing of the driving voltage, the intensity distribution of x-component is recorded, as shown in Fig. 11(a). Removing the voltage signal applied on the TNLC, the intensity distribution of y-component is shown in Fig. 11(b). Synthesizing two orthogonally polarized images, a 2D image having total incident beam intensity can be acquired, as shown in Fig. 11(c).
Fig. 11 2D polarized imaging based on the DPLFI. (a) Ex-component of 2D image at the signal voltage of ~10 Vrms applied on the TNLC. (b) Ey-component of the 2D image removing the signal voltage applied on the TNLC. (c) Computational image with total incident beam intensity.
5. Conclusion
In summary, we have developed a new TN-LCMLA architecture and further demonstrated a DPLFI micro-system based on this key device for effectively acquiring two orthogonally polarized light-field images in sub-millisecond scale. Compared with conventional LCMLAs, the imaging micro-system presents a polarization-independent character to incident beams and then the imaging quality has been remarkably improved through completely eliminating stray light surrounding the focuses of conventional LCMLAs. We can expect that the imaging micro-system with new typed TN-LCMLA will also highlight a rapid development of advanced light-field imaging under the circumstance of arbitrary polarization and low noisy-to-signal ratio, because two orthogonally polarized light-field images can be easily obtained through only switching on or off the TNLC cell. It should be noted that the current prototyped imaging architecture also indicate a potential capability of efficiently observing 3D objects in complex scattering media.
Funding
National Natural Science Foundation of China (61432007, 61176052); Major Technological Innovation Projects in Hubei Province (2016AAA010); Shanghai Aerospace Science and Technology Innovation Fund (2015081); China Aerospace Science and Technology Innovation Fund (CASC2015).
Acknowledgment
The authors would like to thank the Analytical and Testing Center of Huazhong University of Science and Technology for their valuable help.
References and links
1. Y. Lei, Q. Tong, X. Zhang, H. Sang, A. Ji, and C. Xie, “An electrically tunable plenoptic camera using a liquid crystal microlens array,” Rev. Sci. Instrum. 86(5), 053101 (2015). [CrossRef] [PubMed]
2. Q. Tong, Y. Lei, Z. Xin, X. Zhang, H. Sang, and C. Xie, “Dual-mode photosensitive arrays based on the integration of liquid crystal microlenses and CMOS sensors for obtaining the intensity images and wavefronts of objects,” Opt. Express 24(3), 1903–1923 (2016). [CrossRef] [PubMed]
3. Y. Wu, W. Hu, Q. Tong, Y. Lei, Z. Xin, D. Wei, X. Zhang, J. Liao, H. Wang, and C. Xie, “Graphene-based liquid-crystal microlens arrays for synthetic-aperture imaging,” J. Opt. 19(9), 095102 (2017). [CrossRef]
4. R. Ng, M. Levoy, M. Brédif, G. Duval, M. Horowitz, and P. Hanrahan, “Light field photography with a hand-held plenoptic camera,” Stanford Tech. Rep. CSTR 2(11), 1–11 (2005).
5. A. Lumsdaine and T. Georgiev, “The focused plenoptic camera,” in Proceedings of IEEE Conference on Computational Photography (IEEE, 2009), pp. 1–8.
6. S. Sato, “Liquid-crystal lens-cells with variable focal length,” Jpn. J. Appl. Phys. 18(9), 1679–1684 (1979). [CrossRef]
7. A. Markman, X. Shen, and B. Javidi, “Three-dimensional object visualization and detection in low light illumination using integral imaging,” Opt. Lett. 42(16), 3068–3071 (2017). [CrossRef] [PubMed]
8. H. Ren, Y. H. Lin, Y. H. Fan, and S. T. Wu, “Polarization-independent phase modulation using a polymer-dispersed liquid-crystal,” Appl. Phys. Lett. 86(14), 113505 (2005). [CrossRef]
9. Y. H. Lin, H. S. Chen, H. C. Lin, Y. S. Tsou, H. K. Hsu, and W. Y. Li, “Polarizer-free and fast response microlens arrays using polymer-stabilized blue phase liquid-crystals,” Appl. Phys. Lett. 96(11), 141110 (2010). [CrossRef]
10. H. Dai, L. Chen, B. Zhang, G. Si, and Y. J. Liu, “Optically isotropic, electrically tunable liquid crystal droplet arrays formed by photopolymerization-induced phase separation,” Opt. Lett. 40(12), 2723–2726 (2015). [CrossRef] [PubMed]
11. S. H. Lin, L. S. Huang, C. H. Lin, and C. T. Kuo, “Polarization-independent and fast tunable microlens array based on blue phase liquid crystals,” Opt. Express 22(1), 925–930 (2014). [CrossRef] [PubMed]
12. A. Y. Fuh, S. W. Ko, S. H. Huang, Y. Y. Chen, and T. H. Lin, “Polarization-independent liquid crystal lens based on axially symmetric photoalignment,” Opt. Express 19(3), 2294–2300 (2011). [CrossRef] [PubMed]
13. Y. Y. Schechner, S. G. Narasimhan, and S. K. Nayar, “Polarization-based vision through haze,” Appl. Opt. 42(3), 511–525 (2003). [CrossRef] [PubMed]
14. Z. Xin, Q. Tong, Y. Lei, D. Wei, X. Zhang, J. Liao, H. Wang, and C. Xie, “An electrically tunable polarization and polarization-independent liquid-crystal microlens array for imaging applications,” J. Opt. 19(9), 095602 (2017). [CrossRef]
15. R. Wu, J. Suo, F. Dai, Y. Zhang, and Q. Dai, “Scattering robust 3d reconstruction via polarized transient imaging,” Opt. Lett. 41(17), 3948–3951 (2016). [CrossRef] [PubMed]
16. Y. Y. Schechner and N. Karpel, “Clear Underwater Vision,” Computer Vision and Pattern Recognition, (CVPR) 1, 536–543 (2004).
17. P. C. Chang, J. C. Flitton, K. I. Hopcraft, E. Jakeman, D. L. Jordan, and J. G. Walker, “Improving visibility depth in passive underwater imaging by use of polarization,” Appl. Opt. 42(15), 2794–2803 (2003). [CrossRef] [PubMed]
18. N. K. Soni, R. V. Vinu, and R. K. Singh, “Polarization modulation for imaging behind the scattering medium,” Opt. Lett. 41(5), 906–909 (2016). [CrossRef] [PubMed]
19. S. Demos, H. Radousky, and R. Alfano, “Deep subsurface imaging in tissues using spectral and polarization filtering,” Opt. Express 7(1), 23–28 (2000). [CrossRef] [PubMed]
20. H. Shao, Y. He, W. Li, and H. Ma, “Polarization-degree imaging contrast in turbid media: a quantitative study,” Appl. Opt. 45(18), 4491–4496 (2006). [CrossRef] [PubMed]
21. S. P. Morgan, Q. Zhu, I. M. Stockford, and J. A. Crowe, “Rotating orthogonal polarization imaging,” Opt. Lett. 33(13), 1503–1505 (2008). [CrossRef] [PubMed]
22. C. K. Harnett and H. G. Craighead, “Liquid-crystal micropolarizer array for polarization-difference imaging,” Appl. Opt. 41(7), 1291–1296 (2002). [CrossRef] [PubMed]
23. V. Gruev, R. Perkins, and T. York, “CCD polarization imaging sensor with aluminum nanowire optical filters,” Opt. Express 18(18), 19087–19094 (2010). [CrossRef] [PubMed]
24. Y. Lei, Q. Tong, Z. Xin, D. Wei, X. Zhang, J. Liao, H. Wang, and C. Xie, “Three dimensional measurement with an electrically tunable focused plenoptic camera,” Rev. Sci. Instrum. 88(3), 033111 (2017). [CrossRef] [PubMed]
25. E. Peli, “Contrast in complex images,” J. Opt. Soc. Am. A 7(10), 2032–2040 (1990). [CrossRef] [PubMed]
26. H. Ren, Y. H. Fan, and S. T. Wu, “Liquid-crystal microlens arrays using patterned polymer networks,” Opt. Lett. 29(14), 1608–1610 (2004). [CrossRef] [PubMed]
27. S. J. Hwang, Y. X. Liu, and G. A. Porter, “Improvement of performance of liquid crystal microlens with polymer surface modification,” Opt. Express 22(4), 4620–4627 (2014). [CrossRef] [PubMed]
28. C. J. Hsu and C. R. Sheu, “Preventing occurrence of disclination lines in liquid crystal lenses with a large aperture by means of polymer stabilization,” Opt. Express 19(16), 14999–15008 (2011). [CrossRef] [PubMed]
29. D. J. A. Grant, B. K. Jones, and M. G. Clark, “Nematic LCD shutters with sub-millisecond switching times,” Proceedings of Eurodisplay 87, 67–70 (1987).
