Open Access
Add to BibSonomyAbstract
Spatial light modulator (SLM) is widely used in imaging applications for modulating light intensity and phase delay. In this paper, we report a novel device concept termed angular light modulator (ALM). Different from the SLM, the reported ALM employs a tunable blind structure to modulate the angular components of the incoming light waves. For spatial-domain light modulation, the ALM can be directly placed in front of an image sensor for selecting different angular light components. In this case, we can sweep the slat angle of the blind structure and capture multiple images corresponding to different perspectives. These images can then be back-projected for 3D tomographic refocusing. By using a fixed slat angle, we can also convert the incident-angle information into intensity variations for wavefront sensing or introduce a translational shift to the defocused object for high-speed autofocusing. For Fourier-domain light modulation, the ALM can be placed at the pupil plane of an optical system for reinforcing the light propagating trajectories. We show that a pupil-plane-modulated system is able to achieve a better resolution for out-of-focus objects while maintaining the same resolution for in-focus objects. The reported ALM can be fabricated on the chip level and controlled by an external magnetic field. It may provide new insights for developing novel imaging and vision devices.
© 2016 Optical Society of America
1. Introduction
Light modulation has long been a research focus for various imaging applications. Spatial light modulators (SLMs) are devices that can spatially modulate intensity or phase delay of an incoming light beam. Examples of SLM include digital micromirror device (DMD), deformable mirror, and different types of liquid crystal displays. For DMD and deformable mirror, an external voltage is used to mechanically drive individual actuator or membrane. For liquid crystal displays, an external voltage is used to align the molecules of a liquid crystal cell and control the polarization state of the light wave. The common thread through these different SLM devices is to modulate the light transparency in response to information-bearing electronic signals, as shown in Fig. 1(a). In recent years, SLMs have found many exciting applications in microscopy, including ultrafast imaging [1], optical sectioning [2], photostimulation [3], quantitative phase imaging [4], adaptive imaging [5–7], among others.
Fig. 1 The regular spatial light modulator (SLM) and the proposed angular light modulator (ALM). (a) The regular SLM modulates the intensity and/or phase delay of the income light beam. (b) The proposed ALM modulates the angular information of the incoming light beam. (c1) The spatial-domain light modulation using ALM, where the tunable blind structure is directly placed on top of an image sensor. (c2) The Fourier-domain light modulation using ALM, where the tunable blind structure is placed at the pupil plane of the optical system.
In this paper, we propose a new device concept termed angular light modulator (ALM) and report the development of a macroscale ALM prototype. Different from the SLM, the motivation of the proposed ALM is to modulate the angular information of the incoming light waves. As shown in Fig. 1(b), we use a tunable optical blind structure for this purpose and each pixel of the ALM contains a light-absorbing slat. Figure 1(c1) and 1(c2) demonstrate two implementations using the proposed ALM, one for light modulation in the spatial domain and the other for the Fourier domain.
For spatial-domain light modulation (Fig. 1(c1)), the ALM can be placed in front of an image sensor for selecting different angular light components. In this case, we can sweep the slat angle of the blind structure and capture multiple images corresponding to different perspectives. These images can then be back-projected for 3D tomographic refocusing [8,9]. By using a fixed slat angle in the device, we can convert the incident-angle information into intensity variations for wavefront sensing; we can also introduce a translational shift to the defocused object for high-speed autofocusing [10].
For Fourier-domain light modulation (Fig. 1(c2)), the ALM can be placed at the pupil plane of an optical system for reinforcing the light propagating trajectories. In this case, the light rays from out-of-focus positions will be rejected by the ALM (the red arrows in Fig. 1(c2)) while the light rays from the in-focus position remain unchanged. This novel configuration may be used for correcting aberrations in imaging and vision systems.
In the following, we will first explain the design and fabrication details of the macroscale ALM prototype in Section 2. In Section 3, we will discuss the use of ALM for spatial-domain light modulation and report the applications of 3D tomographic refocusing and single-shot autofocusing. In Section 4, we will discuss the use of ALM for Fourier-domain light modulation. We will show that the effective aperture size of the reported system reduces as the object move away from the in-focus position. As such, we can achieve a better resolution for out-of-focus objects while maintaining the same resolution for in-focus objects. Finally, we will summarize the results and discuss future directions, including how to make a microscale ALM on the chip level and control it using an external magnetic field.
2. Angular light modulator using optical blinds
The design and fabrication process of the macroscale ALM prototype is summarized in Figs. 2(a)-2(c) (Visualization 1). We used a standard photolithography process to fabricate the thin slats in the device. In particular, we etched a 125 µm-thick brass plate (McMaster-Carr) in Ferric Chloride solution at 40 °C for 20 minutes to form patterned brass slats. These slats were then dipped in the BrassBlack solution (Birchwood Casey) for 30 seconds for the oxidization process. After this process, the oxidized slats become a good light-absorbing material in the tunable blind structure. Figure 2(b) shows the photomask used in the fabrication process and the patterned brass slats. In Fig. 2(c), we placed the blind structure in front of an image sensor (Pointgrey, GS3-U3-50S5C-C). The angle of the slats was tuned using a stepper motor (Digikey, 1460-1074-ND). Figures 2(c1)-2(c3) show the blind structure at three different angles. In our prototype device, it takes ~0.1 seconds to sweep from −15 degrees to + 15 degrees and capture 10 images with different perspectives (Visualization 2).
Fig. 2 The design and testing of the macroscale ALM prototype. (a) (Visualization 1) The mechanical design of the movable frame. (b) A standard photolithography process was used to produce patterned brass slats. We oxidized the brass slats using the BrassBlack solution in the Petri dish. After this process, the oxidized slat became a good light-absorbing material in the tunable blind structure. (c) (Visualization 2) By tuning the slat angle of the device, we can capture images with different view angles for 3D tomographic reconstruction. (d) The transmitted intensity signal was measured as a function of the rotation angle of the entire setup. We achieved angular selectivity and tunability by using the ALM prototype.
We characterized the performance of the ALM prototype in Fig. 2(d). In this experiment, we mounted the system of Fig. 2(c) on a rotation stage and measured the light transmission. First, we removed the ALM and rotated the stage to different angles. The captured intensity signal was plotted as a function of the rotation angles in the dark red curve in Fig. 2(d). We then placed the ALM in front of the image sensor and repeated the experiment. In particular, we achieved two features by using the ALM: 1) angular selectivity–the full width at half maximum reduces from 30 degrees to 5 degrees, and 2) angular tunability–the peak transmission shifts as we adjust the slat angle. These two features can be used for wavefront sensing as we can convert the incident-angle information into intensity variation in Fig. 2(d) (a reference intensity value without the ALM is needed for normalization).
3. Spatial-domain light modulation using ALM
For spatial-domain light modulation, we can directly place the ALM in front of the image sensor for selecting different angular light components. In the photographic setup shown in Fig. 3(a), we captured 10 images corresponding to different slat angles and performed digital refocusing using tomographic back-projection [8,9]. Figures 3(b1)-3(b4) show the refocused images at the different z planes (Visualization 3) and we highlighted the in-focus regions using red arrows. In Fig. 3(c), we use two photographic lenses to form a 4f system. Figure 3(d) show the refocused images of 3 micro-objects (fruit flies) at different z-planes and we highlighted the in-focus regions using the red arrows.
Fig. 3 Spatial-domain light modulation using the ALM prototype (Visualization 3). (a)-(b) Photographic imaging and 3D refocusing of a scene. An f/1.8 lens was used in this system. (c)-(d) 3D refocusing of micro-objects. Two f/1.8 lenses were used to form a 4f system.
Whole slide imaging is an important tool for digital pathology and biomedical research. A key aspect of whole slide imaging is to determine the defocus distance of the sample in high speed. We have demonstrated the use of pinhole-modulated cameras for high-speed autofocusing [10,11]. In these previous demonstrations, the pinhole was placed at an off-axis position of the Fourier plane to achieve angular selectivity and we need to use a lens system to relay the pinhole to the Fourier plane of the objective lens. The reported ALM, on the other hand, is a spatial-domain solution for achieving angular selectivity. We can directly place the ALM with a tilted slat angle in front of an image sensor; no lens is needed for relaying the Fourier plane of the imaging system. Figure 4 demonstrates the use of the ALM prototype for the autofocusing application. In this experiment, we used the same setup in Fig. 3(c) for acquiring images. The slats of the ALM were adjusted to ~15 degrees. Figures 4(a1)-4(a3) show three images with three different defocus distances and the translational shift is clearly visible between them. In Fig. 4(b), we plotted the defocus distance as a function of the translational shifts of the captured images (the translational shift was identified by maximizing the phase correlation curve [10]). From this experiment, we can see that the reported ALM can be used for high-speed autofocusing.
Fig. 4 Recovering the defocus distance using the ALM. The slats of the ALM were adjusted to a fixed angle to achieve angular selectivity. This arrangement introduces a translation shift as the object moves away from the in-focus position. (a) The captured images with different defocus distances. (b) The relationship between the defocus distances and the translational shifts.
4. Fourier-domain light modulation using ALM
The ALM prototype can also be placed at the pupil plane of an imaging system to perform Fourier-domain light modulation. This configuration enables a new optical feature that, to the best of our knowledge, has not be reported before. We consider the example in Fig. 5(a), where the ALM is placed at the exit pupil plane of a lens. The light rays from the in-focus position remain unchanged in the system. However, the light ray from an out-of-focus position will be rejected by the ALM at the pupil plane (the red arrow in Fig. 5(a)). We analyze this effect as follows. First, we define the cutoff angle of the ALM to be θ0, which equals to the separation between two adjacent slats divided by the width of the slat. Next, we consider a defocused object placed at position ‘z’ in Fig. 5(a). Based on the Newtonian form of the thin-lens formula, the image distance can be expressed as f · (1 + f / z), where ‘f’ is the focal length of the thin lens. Under the small-angle approximation, the bending angle θ in Fig. 5(a) can be expressed as θ = h · z / (f · (z + f)), where ‘h’ is the height of the light ray at the thin lens’ plane. The condition for the light ray to pass through the ALM is θ < θ0 and we can express it as follows:
We can then define the cutoff height hcutoff = f·(z + f)· θ0 / z, which represents the effective aperture size for objects at the defocus distance ‘z’. As objects move away from the in-focus position, Eq. (1) implies that the effective aperture size ‘hcutoff’ of the lens will decrease. Therefore, we can extend the depth of field for out-of-focus objects while maintaining the same resolution performance for in-focus objects.Fig. 5 Fourier-domain light modulation using the ALM prototype. (a)-(b) The experimental setup, where we placed the ALM at the Fourier plane of a 4f system. (c) The images captured without using the ALM. (d) The images captured with the ALM. We maintain the same resolution performance for the in-focus object and achieve a better resolution performance for out-of-focus objects. (e) We can adjust the width or the density of the slats to adjust the cutoff angle at different heights. As such, we can design the relationship between the effective aperture size and the defocus distance.
We validated this novel optical feature using an experimental setup in Fig. 5(b), where we used a resolution target as the object and placed the ALM at the Fourier plane of a 4f system. In this experiment, the cutoff angle θ0 of our ALM prototype is about 6 degrees. Figure 5(c) shows the captured images without using the ALM and Fig. 5(d) shows the results using the ALM. As shown in Figs. 5(c1) and 5(d1), we get the same resolution performance for the images at the in-focus position. For out-of-focus positions, the captured images using the ALM have a higher resolution in the x direction, as shown in Figs. 5(d2) and 5(d3). The resolution improvement along the x direction is due to the ALM orientation in the y direction, causing an effective smaller aperture for out-of-focus objects in the x direction. To achieve an isotropic effect, we can design an ALM with 2D-grid slats that can modulate light in both x and y directions. We can also put two ALMs in series, one for modulating light in the x-direction and the other for the y-direction.
In our ALM prototype, the cutoff angle θ0 is a constant. We can further adjust the width or the density of the slats to adjust the cutoff angle at different heights, as shown in Fig. 5(e). In this case, θ0(h) is a function of the height h. Thus, we have extra degrees of freedom to design the relationship between the effective aperture size ‘hcutoff’ and the defocus distance ‘z’ in Eq. (1). The reported adaptive-aperture-size feature may enable the development of novel imaging and vision devices. For example, typical whole slide imaging platforms use linear CCDs to acquire images of the samples. The depth of field is limited due to the use of high NA objective. We can place the ALM at the pupil plane to extend the depth of field. We may also place the ALM at the exit pupil plane of eyes to extend the depth of field of human vision.
5. Summary and future directions
In summary, we have proposed a novel concept called angular light modulator (ALM) and reported the development of a macroscale ALM prototype. For spatial-domain light modulation using ALM, we can directly place the tunable blind structure in front of the image sensor. By changing the angle of the slats, we can selectively capture different angular components in the image acquisition process. We demonstrate the use of the reported device for 3D tomographic refocusing and single-shot autofocusing. For Fourier-domain light modulation using ALM, we can place the device at the pupil plane of an optical system. Different from conventional imaging systems, the effective aperture size of the reported system reduces as the object move away from the in-focus position. As such, we can achieve a better resolution for out-of-focus objects while maintaining the same resolution for in-focus objects. To the best of our knowledge, this effect has not been reported before. The relationship between the defocus distance and the effective aperture size can be designed using Eq. (1).
There are several future directions for further developing the reported ALM concept. 1) Out-of-plane thin slats can be directly fabricated on the chip level at microscale [12]. The standing micro-slat can be modeled as a combination of a rigid magnetic body and an elastic spring that supports the main body. The standing angle can be controlled by an external magnetic field applied in different directions with a high actuation speed and without direct mechanical contact. We can also explore the use of micro-structure in microlouver film for this purpose. 2) We demonstrated the use of the ALM for acquiring angular information. It can also be used for synthesizing the light field in a reverse manner and finds applications in head mounted displays. In this case, we can put the ALM in front of the display. For each slat angle, we show one image on the display with a corresponding view angle. By sweeping the slats to different angles and showing the corresponding images on the display, we can effectively synthesize the 3D view of the object and our eyes can focus on different planes without losing the focus cue. 3) We analyze the change of the effective aperture size using ray-optics calculation in Eq. (1). For microscale ALM, an analysis using wave-optics simulation is highly desired. 4) The extended-depth-of-field feature is appealing to many biomedical applications. We plan to investigate the use of ALM in microscopy and endoscopy applications.
Acknowledgment
This work was supported by NSF 1555986 and NIH R21EB022378. Z. Zhang acknowledges the support of the China Scholarship Council.
References and links
1. L. Gao, J. Liang, C. Li, and L. V. Wang, “Single-shot compressed ultrafast photography at one hundred billion frames per second,” Nature 516(7529), 74–77 (2014). [CrossRef] [PubMed]
2. Q. S. Hanley, P. J. Verveer, M. J. Gemkow, D. Arndt-Jovin, and T. M. Jovin, “An optical sectioning programmable array microscope implemented with a digital micromirror device,” J. Microsc. 196(3), 317–331 (1999). [CrossRef] [PubMed]
3. V. Nikolenko, B. O. Watson, R. Araya, A. Woodruff, D. S. Peterka, and R. Yuste, “SLM Microscopy: scanless two-photon imaging and photostimulation using spatial light modulators,” Front. Neural Circuits 2, 5 (2008). [CrossRef] [PubMed]
4. C. Maurer, A. Jesacher, S. Bernet, and M. Ritsch-Marte, “What spatial light modulators can do for optical microscopy,” Laser Photonics Rev. 5(1), 81–101 (2011). [CrossRef]
5. M. J. Booth, “Adaptive optical microscopy: the ongoing quest for a perfect image,” Light Sci. Appl. 3(4), e165 (2014). [CrossRef]
6. N. Ji, D. E. Milkie, and E. Betzig, “Adaptive optics via pupil segmentation for high-resolution imaging in biological tissues,” Nat. Methods 7(2), 141–147 (2010). [CrossRef] [PubMed]
7. I. M. Vellekoop, “Feedback-based wavefront shaping,” Opt. Express 23(9), 12189–12206 (2015). [CrossRef] [PubMed]
8. K. Guo, Z. Bian, S. Dong, P. Nanda, Y. M. Wang, and G. Zheng, “Microscopy illumination engineering using a low-cost liquid crystal display,” Biomed. Opt. Express 6(2), 574–579 (2015). [CrossRef] [PubMed]
9. G. Zheng, C. Kolner, and C. Yang, “Microscopy refocusing and dark-field imaging by using a simple LED array,” Opt. Lett. 36(20), 3987–3989 (2011). [CrossRef] [PubMed]
10. K. Guo, J. Liao, Z. Bian, X. Heng, and G. Zheng, “InstantScope: a low-cost whole slide imaging system with instant focal plane detection,” Biomed. Opt. Express 6(9), 3210–3216 (2015). [CrossRef] [PubMed]
11. J. Liao, L. Bian, Z. Bian, Z. Zhang, C. Patel, K. Hoshino, Y. C. Eldar, and G. Zheng, “Single-frame rapid autofocusing for brightfield and fluorescence whole slide imaging,” Biomed. Opt. Express 7(11), 4763–4768 (2016). [CrossRef]
12. K. Noda, K. Hoshino, K. Matsumoto, and I. Shimoyama, “A shear stress sensor for tactile sensing with the piezoresistive cantilever standing in elastic material,” Sens. Actuators A Phys. 127(2), 295–301 (2006). [CrossRef]
