1. Introduction

Projection-type spatial light modulators (SLM) have been hard-pressed to satisfy étendue requirements for head-up and head-mounted displays, as well as multi-perspective requirements for “no-glasses” 3D displays while maintaining small optical system package sizes [1]. In these systems, the étendue of the SLM dictates the field-of-view (FOV) of the observer and the viewing area or eye-box through which the observer can view the displayed image [1].

For multi-perspective 3D displays, a high number of perspectives across the FOV is necessary to achieve smooth transitions between individual perspectives while maintaining a high pixel count (i.e., spatial sampling) for each projected pattern. Conventionally, in a single-SLM display, the multiple perspectives are implemented at the expense of pixel count per perspective. For example, “subpixels” can be allocated toward different perspectives from each “superpixel” of a microlens array or grating array, resulting in lower pixel count patterns per perspective [2–4].

One approach to these challenges involves combining multiple projection systems to add the étendues of the multiple SLMs for a wider collective FOV and a larger collective eye-box while directly increasing the number of perspectives without compromising the pixel count per perspective [5–7]. Other methods have cascaded an SLM with a beam steering device to project the image of the SLM in multiple directions at different times for time-integration by the human eye [8,9]. Though these approaches are effective, increasing the number of SLMs or beam steering devices significantly increases system package size, power consumption, and cost.

Yet another approach increases projection-type SLMs’ pixel density in fabrication [10]. Though not increasing étendue, a smaller pixel pitch SLM under a microlens array or grating array enables more perspectives without loss to pixel count per perspective [2–4]. Increased pixel density can also be achieved by dithering a plane parallel plate after an SLM, offsetting time-multiplexed projected images in x- and y-directions by a half pixel, though the new quartered-pixels are not completely independent [11]. Resonant MEMS mirrors have been used to great effect, where pixel density is dependent on illumination pulse rate, angular spot size, and mirror angular selectivity (e.g., x-axis resolution determined by pulse count during single mirror sweep and spot size, and y-axis resolution determined by mirror angular selectivity in y-axis and spot size), but a steep speed-size tradeoff coupled with angular throw limitations leaves these devices in a similar étendue order of magnitude as other SLMs [12]. Pupil-segmenting techniques have been used for time-multiplexed illumination on an SLM to project different patterns in multiple directions for an increased number of perspectives, either by multiple sources or steering a single source [13–15]. In these cases, the overall SLM étendue is partitioned per source or source location. To increase viewing angle beyond the SLM’s étendue limit, these systems employ a vertical diffuser volume after the SLM to expand the viewing angle.

In this paper, we demonstrate projecting multiple, independent, native-resolution patterns across the 48° full-angle reflected output from a single Digital Micromirror Device (DMD)-type SLM using a single illumination source without an additional angular modulation element. The simultaneous Angular Spatial Light Modulation (ASLM) utilizes a programmable blazed grating beam steering technique reported previously while only transitioning selected micromirrors of a spatial distribution [16]. The 48° output effectively increases the overall F-number of an F/2.4 DMD to F/1.22, doubling the étendue of the DMD for increased FOV and viewing angle for displays. We also report a single element and multi-perspective display by time-multiplexed, pupil-segmented illumination synchronized to independent DMD patterns in an orthogonal direction for a 2D arrayed output.

Section 2 reviews the beam steering technique by programmable blazed grating. Section 3 discusses ASLM system optical design and configurations. Section 4 presents the application-specific experimental setups and results, including extended projection, light-field projection, and 3D display. Section 5 discusses design impacts on space-bandwidth product and étendue, and other considerations such as diffraction effects and current limitations. Section 6 outlines paths forward in the context of scalability, full-color and gray-scaling implementations, an application to holography, and techniques to improve speed and resolution.

2. DMD beam steering by programmable blazed grating

A DMD is most commonly considered a binary device, where each pixel is a micromirror which tilts between binary “on” and “off” positions. Light incident upon on-state pixels is reflected toward projection optics to project white (i.e., bright) pixels, and light incident upon off-state pixels is reflected toward a beam dump, equivalently projecting black pixels. Kilohertz switching of micromirror positions enables time-integrated bit-depth for gray-scaling. Interleaving red, green, and blue illumination on the DMD, or optically conjugating multiple DMDs with red, green, and blue illumination, enables full-color projections. Figure 1 shows the geometry and binary states of the DLP3000 DMD (manufactured by Texas Instruments).

 

Fig. 1 (a) DLP3000 DMD micromirror layout and rotation axes. (b) On-state and (c) off-state micromirrors reflecting light. The color scale shows height above DMD base surface.

Download Full Size | PDF

We previously published a time of flight lidar system using a DMD-based beam steering technique [16]. In the system, a DLP3000 DMD (manufactured by Texas Instruments) with a nominal pattern refresh of 4 kHz is illuminated with an 8 ns laser pulse during the 2.4 µs transition of the micromirrors. The micromirrors are effectively frozen between the ± 12° binary states because they rotate only about 0.08° during the duration of the 8 ns illumination pulse. The temporally frozen micromirrors form a blazed grating with a programmable blaze angle based on synchronization of the illumination pulse and the micromirror transition.

Equation (1), the diffraction grating equation, determines the output diffraction order angles for a DMD with square micromirrors which rotate along their respective corner-to-corner axes, where ${\theta }_{\text{inc}}$ is the source input angle of incidence, ${\theta }_n$ is the diffraction-order-specific output angle, $n$ is the diffraction order number, $\lambda$ is the wavelength, and $p$ is corner-to-corner pixel pitch. The 2 is due to the $p/2$ pitch between neighboring pixels’ axes of rotation in the arrayed structure [16]

The number of steerable diffraction orders is limited by the full angular throw of the micromirrors, ${\theta }_{\text{mirror}}.$ The output angles for highest and lowest steerable diffraction orders, ${\theta }_{n_{\text{max}}}$ and ${\theta }_{n_{\text{min}}},$ respectively, cannot exceed the input-directed half-reflected-sweep (equal to full angular throw) of the micromirrors, or ${\theta }_{\text{inc}}-{\theta }_{n_{\text{max}}}\le {\theta }_{\text{mirror}}$ and ${\theta }_{n_{\text{min}}}-{\theta }_{\text{inc}}\le {\theta }_{\text{mirror}}.$ Similarly, for the full reflected sweep, ${\theta }_{n_{\text{min}}}-{\theta }_{n_{\text{max}}}\le 2{\theta }_{\text{mirror}}$. Put more simply, the angular extent of the discrete output diffraction orders is limited to the reflected angular extent of the micromirrors (e.g., 48° for a ± 12° micromirror). For instance, for the DLP3000 DMD with a 24° full angular throw ( ± 12°), a 10.8 µm corner-to-corner pixel pitch, 905 nm illumination, and a 30° illumination angle of incidence, the DMD can sequentially steer across 5 diffraction orders (e.g., −2, −1, 0, 1, 2) by manipulating the diffraction efficiencies of multiple orders through controlling the programmable blaze angle [16].

3. ASLM design configurations

Our previous system produced an angular light modulation effect of directing light into selectable directions (i.e., diffraction orders), but involved no spatial light modulation across the illumination plane. The Angular Spatial Light Modulator system simultaneously modulates light in both the angular domain, defined by the global angle of the mid-transition micromirrors, and spatial domain, defined by selecting which micromirror pixels will transition. Figures 2(a) and 2(b) show two phase profiles across a DMD at two different moments of illumination. The selected transitioning mirrors of each profile, which form the respective spatial modulations, are at different angles. The respective, diffraction-order-dependent binary patterns are depicted in Figs. 2(c) and 2(d).

 

Fig. 2 (a), (b) Phase profiles of two different spatial patterns of micromirrors illuminated at different moments during the micromirror transition. (c), (d) The respective binary patterns projected into different diffraction orders.

Download Full Size | PDF

The following sub-sections cover DMD illumination schemes, projection schemes and applications, and the process necessary to achieve ASLM pattern steering.

3.1 ASLM illumination and pupil schemes

Conventional DMD illumination is dependent on the rotational throw of the micromirrors, and typical designs define illumination, on-state, flat-state, and off-state pupils [17]. For instance, the DLP3000 has a ± 12° micromirror rotation, so the input pupil is limited to F/2.4 $\left(F/\#=\frac{1}{2\sin {12}^{°}}\right)$ in order to maximize contrast of the projected pattern by preventing contamination between output pupils. Figure 3(a) shows a conventional illumination and projection scheme for spatial light modulation by DMD, both maximizing pupil NA (Numerical Aperture) while preventing crosstalk between pupils. The single-source illumination is reflected into binary output states, one toward a projection lens and one toward a light dump, for each of the two micromirror angles.

 

Fig. 3 DMD illumination schemes for (a) conventional binary state projection, (b) ASLM pattern steered output, (c) pupil-segmented illumination for conventional projection, and (d) vertical pupil-segmented illumination and horizontal ASLM pattern steering for a 2D output pupil array. Input and output pupils depicted for a single point, but an extended illumination area is employed so that multiple micromirrors are illuminated. Micromirrors (~10 µm) are not on the same scale as the illumination and projection optics (mm-cm) to demonstrate pupil-driven design forms. For the illustration purpose, only a single micromirror is schematically depicted.

Download Full Size | PDF

Figure 3(b) shows the illumination scheme for the proposed ASLM diffraction-based pattern-steered system. Similar to the conventional scheme of Fig. 3(a), the cone angle of the input illumination must be limited to prevent crosstalk across adjacent pupils of different output diffraction orders. In Fig. 3(b), five output diffraction order pupils, in directions as described by Eq. (1), and the respective five micromirror rotation angles (blaze angles) are depicted. Through time-multiplexing, each output diffraction order pupil can project an independent spatial pattern.

Figure 3(c) depicts a pupil-segmented illumination scheme to project multiple patterns, one per illumination source, into different directions by time-multiplexing [13,14]. There, multiple independent sources share the total horizontal angular extent equivalent to the illumination input pupil of Fig. 3(a) to similarly project on-state light toward a lens and off-state light toward a light dump. In general, the number of light sources is equal to number of perspectives.

In Fig. 3(d) a proposed 2-dimensional pupil-segmenting method by ASLM is depicted. There, a 5 × 3 array of output pupils is generated by example. Pupils steered in the horizontal direction are generated by diffraction orders (similar to Fig. 3(b)), whereas multiple vertical outputs are created by a light source distribution similar to Fig. 3(c).

For all Figs. 3(a)–3(d), the ray bundles for only a single illuminated point depicts the angular extents of the inputs and outputs, but through the series of demonstrations an extended area illumination is employed, especially for diffraction-based and discrete pattern steering which is reliant on illuminating a multitude of micromirrors. Furthermore, specular reflections occur from a cover glass window for all of these illumination schemes. The cover glass is parallel to the micromirrors while the DMD is not powered, or equivalently at an micromirror angle of 0°. In the binary state steering cases of Figs. 3(a) and 3(c), a “flat state” pupil is sequestered between the on-state and off-state pupils of the same angular extent (not shown) in order to maximize contrast of the on-state. In the case of the diffraction-based beam steering schemes of Figs. 3(b) and 3(d), the cover glass specular reflection propagates in the 0th diffraction order direction (or angular extent), reducing contrast in that diffraction order direction (or angular extent). However, the issue of the specular reflection could be solved by design options such as anti-reflection coatings designed to specific angles-of-incidence, or by adhering a small wedge prism with optical adhesive to the cover glass to reflect the specular reflection away from the projection directions. In either case, the specular reflection issue is fixable, elementary, and independent from the diffraction-based beam steering technique, so it will not be discussed further in this paper (with the exception of a correction offset in the 0^th diffraction order through the diffraction efficiency measurements in Section 5.3).

3.2 Illumination optics and sequencing

As previously discussed, and as evident in Figs. 3(a)–3(d), the illumination input pupil cone angle of each source must be limited to prevent crosstalk between adjacent output pupils. Furthermore, and similar to conventional DMD illumination design, homogenous illumination across the active area of the DMD is necessary for homogenous white and black levels across each projected output. A telecentric illumination scheme further prevents pupil contamination across output pupils. Three illumination schemes were separately employed: direct illumination, critical illumination [18], and Koehler illumination [18].

The direct illumination scheme uses an expanded collimated laser to illuminate the DMD. The center region of the Guassian intensity distribution covers the DMD active area and is sufficiently homogenous. For demonstration purposes, this scheme has the effect of a telecentric, very low NA (~0.01) system with high throughput. A very low NA illumination enables a very long depth-of-focus, enabling visual identification of patterns far from focus to distinguish independent pupils (see Fig. 5(b) and corresponding discussion). The direct illumination scheme is used in experimental demonstration #1 extended FOV display (Section 4.1) and #2 light-field projection (Section 4.2).

The critical illumination scheme uses a 532 nm expanded collimated laser, for the same homogeneous area, to illuminate a transmissive 5° Light Shaping Diffuser (Edmund Optics, #47-994) for a 5° output cone angle. The imaging relay of the critical illumination scheme uses an iris for telecentric and NA-adjustable illumination. Critical illumination schemes are very common for projection-type displays as the designs can independently tune homogeneity, NA, and telecentricity. An extended pupil (increased NA) is necessary for a multi-view display to form an extended eye-box. The critical illumination scheme is used in experimental demonstration #3 1D multi-view display (Section 4.3).

Finally, the Koehler illumination scheme uses three 5 mm diameter green LEDs (narrow line-width) to homogenously illuminate the DMD. The LEDs have built-in diffusers for an extended illumination pupil, and they are placed at the front focal point of the condensing lens of the Koehler illumination scheme for telecentric illumination. Koehler illumination schemes are also very common in projection-type displays, particularly when compact packaging is necessary, and these schemes also enable an extended eye-box. The Koehler illumination scheme is used in experimental demonstration #4 2D multi-view display (Section 4.4).

Each diffraction order output pupil is sequentially illuminated, by illumination pulse synchronization, while the pixels of a corresponding pattern are actuated in order to achieve binary pattern steering. Before illuminating each pattern, all pixels are first actuated to the “off” state, controlled by actuating the DMD to a fully “black” pattern. Upon actuation to the next programmed binary pattern, only select mirrors will flip, those pixels set to be “white” / “on”. The plane of the DMD will have the amplitude spatial distribution of the programmed pattern when observed from the programmed diffraction order direction.

4. Experimental setups and results

The four experimental ASLM setups are tabulated in Table 1 and were configured to demonstrate the broad capabilities of the multi-order diffraction-based ASLM pattern steering technique in the context of real-world applications. The illumination types are explained above in Section 3.2, where direct and critical illumination use a 532 nm laser and Koehler illumination uses green LEDs. Small lenses were used for projection in setup #1 and setup #3 to project individual diffraction orders, while the larger single lens of setup #2 captures multiple diffraction orders. No additional projection optics beyond a camera were used to capture the images for the “no optics (direct view)” case demonstrations (i.e., direct view by eye/camera).

Table 1. ASLM Projection Schemes and Applications

View Table | View all tables in this article

4.1 Extended FOV display demonstration

The increased pixel count of the ASLM output, across spatial and angular domains, can be mapped to a tiled projection for an extended FOV display. Figure 4(a) shows direct illumination from a collimated 532 nm laser (Coherent, VECTOR 532-1000-20) (for the effect of low-NA illumination) onto a DMD for a 9 diffraction order output where each diffraction order projects a different binary pattern through time-multiplexing. An array of 6.2 mm diameter plano-convex lenses of 38 mm focal length (Rolyn #10.0100) projects the DMD onto an observation screen. Each lens covers one diffraction order, and a spatial filter array is placed at a Fourier plane (a focal length away) after the lens array to filter out higher-order diffraction orders. Each higher-order, off-steering-plane diffraction order contains the same spatial information as the on-steering-plane output, but higher-order terms from different patterns overlap, leading to significant crosstalk if left unfiltered. The beam paths are schematically shown in Fig. 4(a), and the dotted-lines for the left-most diffraction order shows the marginal ray paths for an imaging condition. The experimental setup is shown in Fig. 4(b). Only one letter is displayed at a time, but all seven letters appear static due to a 1 kHz refresh rate of the DMD.

 

Fig. 4 (a) ASLM extended FOV display schematic with seven independent diffraction orders and off-state light. All output diffraction orders numbered. DMD shown is the larger structure, not the active area. (b) Experimental setup with higher diffraction orders visible on the spatial filter (i.e., card with the array of green dots) and seven independent binary patterns on an observation screen, each 2 cm in height and spaced by 6-7 cm.

Download Full Size | PDF

The off-state light of each diffraction order is sent into the off-state direction because those mirror segments were already, and remain, in the off-state rather than transitioning. In this setup, when the “landed” (non-transitioning) on-state mirrors direct light into the direction of the left-most diffraction order, the “landed” off-state mirrors direct light in between the two right-most diffraction orders. This is because the total number of diffraction orders for which light can accurately be steered across the rotational throw of the micromirrors, as discussed in Section 2, in this configuration is about 8.5. Rounding down, a total of 8 diffraction orders can be accurately steered, but the landed state inaccurately steers the light across two diffraction orders, making both off-state light directions. While originally tested in the conventional two-state implementation system, this spread of the single-state light across multiple diffraction orders of a DMD due to alignment was previously described for coherent illumination [17].

The angle-of-incidence can be adjusted to ensure the off-state light is directed into the right-most diffraction order, but the result is that light cannot be accurately directed the left-most diffraction order due to the limited rotational throw of the micromirrors in practice.

4.2 Light-field projection demonstration

Achieving simultaneous angular and spatial light modulation at a plane creates a light-field at the surface of the DMD which can be reproduced elsewhere by a lens. Figure 5(a) shows the same direct illumination setup and DMD output as Fig. 4(a), except a single 100 mm focal length, 25 mm diameter lens was placed to image the DMD onto an observation screen, capturing three diffraction orders (the “I”, “D”, and “E”) and filtering out all other diffraction orders with a spatial filter. The projected pattern images overlap at focus because each pattern object sequentially occurs on the same area of the DMD. Beam paths are schematically shown in Fig. 5(a).

 

Fig. 5 (a) ASLM light-field projection schematic capturing three diffraction orders and spatially filtering out other diffraction orders. All diffraction orders numbered. (b) Through-focus of ASLM light-field projection.

Download Full Size | PDF

The effective ~0.01 NA illumination increases the depth-of-focus of the projections. The increased depth-of-focus allows the observation screen distance, z, to be adjusted while the beam footprints maintain the approximate shape of the projected patterns. Figure 5(b) depicts the on-axis field-point ray bundles of the three diffraction orders going through focus, and captures of the observation screen at different locations (z₁, z₂, and z₃). The three patterns depict an “I”, a “D”, and an “E”. The “I” and “E” switch sides of the “D” through focus, demonstrating how the patterns are direction dependent, and how the projections cross as expected through focus. The light-field and patterns are in-focus when the patterns perfectly overlap.

4.3 1D multi-view display demonstration

Rather than projecting an SLM to a projection screen, many display systems project images directly onto a viewer’s retina, such as head-mounted displays and VR/AR (Virtual Reality and Augmented Reality) glasses. We applied the critical illumination scheme to achieve an array of output pupils forming an extended eye-box. Figure 6(a) shows critical illumination of a DMD for ASLM output across seven independent diffraction orders. Off-state light covers both the two right-most diffraction orders in the same manner as the extended display case in Fig. 4. Two lenses (Rolyn #10.0100) each covering a pupil to project an angle-dependent pattern onto an observation screen were utilized. The remaining outputs propagate to the observation screen to demonstrate the angular extent and separation of output pupils.

 

Fig. 6 (a) ASLM 1D multi-view display by 0.1 NA critical illumination with observation screen and two imaging lenses. Single-point illumination is shown to accurately depict independent pupil steering, but extended-area illumination is assumed for diffraction-based pattern steering. All diffraction orders numbered. (b) Photo of the of the observation screen showing five of the diffraction order outputs. The lenses form two angle-dependent patterns of Princess Leia. (c) Close-up views (left) and their respective input binary patterns (right).

Download Full Size | PDF

Figure 6(b) is a photo of the projection screen showing five of the diffraction orders. The left-most lens is not fully covering the area of the respective pupil, causing part of the left-most pupil to be visible. This demonstrates the concept of an eye-box, in which the human eye does not need to cover the full area of the eye-box, but rather only a portion to form an image. A higher-order diffraction order is only visible above and between the two left-most pupils due to a synchronization error of the first or second diffraction order from the left. The projections are shown close-up with their respective binary patterns in Fig. 6(c).

The extended pupil allows direct viewing of the patterns on the DMD by an eye or camera without additional optics or precise alignment. Figure 7(a) depicts the setup to directly view the ASLM output patterns on the surface of the DMD by a small-aperture smartphone camera in place of a human eye. Figure 7(b) includes photos of the DMD from the same directions as the two previously projected patterns. Ambient light illuminates the off-state pixels and the DMD package.

 

Fig. 7 (a) ASLM 1D multi-view display by 0.1 NA critical illumination with small-aperture camera for direct viewing of the angle-dependent patterns on the surface of the DMD demonstrating a successful angularly and spatially modulated light field at the DMD plane. Single-point illumination is shown to accurately depict independent pupil steering, but extended-area illumination is assumed for diffraction-based pattern steering. All diffraction orders numbered. (b) Photos of the DMD from two different angles with ambient light. The scale bar is only accurate for the vertical dimension in both photos due to the changing viewing angle. Single quadrant illuminated (Section 5.4).

Download Full Size | PDF

4.4 2D multi-view display demonstration

The 1D output pupil array was expanded to 2D by vertically stacked pupil segmentation. Three vertically-stacked LEDs in a 0.1 NA Koehler illumination scheme each output six independent output pupils for a total of 18 output pupils in a 2D array as shown in Fig. 8(a). The 2D arrayed output system was created using three green LEDs (Kingbright, WP154A4SEJ3VBDZGW/CA, 5 mm RGB, only green channel used), where each LED pulse has a duration of one clock cycle (62.5 ns) of the driving Arduino Uno. (The Arduino Uno microcontroller drives the DMD-based beam steering, see [16] for more details.) The DMD (DLP3000, Texas Instruments) was operated at 4 kHz and sequentially synchronized to LEDs pulsing at 2 kHz to accommodate the black pattern refreshes between multi-perspective patterns.

 

Fig. 8 (a) ASLM 2D multi-view display by 3-LED pupil segmented Koehler illumination. Each vertically-stacked LED has six independent horizontally steered output diffraction orders for a 3 × 6 array of outputs for direct pattern viewing on the surface of the DMD. Single-point illumination is shown to accurately depict independent pupil steering, but extended-area illumination is assumed for diffraction-based pattern steering. (b) Photos of the 18 angle-dependent patterns, arranged by viewing position corresponding to the 2D output pupil array.

Download Full Size | PDF

Binary, perspective-accurate patterns of a wire-grid cube were created in POV-Ray (Persistence of Vision Raytacer Pty. Ltd.) using the anticipated output directions, a 3 × 6 array extending approximately 20° × 40°. The binary patterns were loaded onto the DMD, with interleaved black patterns for micromirror reset, and cycled at 4 kHz in synchronization with the 3-LED illumination to steer the patterns in their corresponding directions. As long as both eyes of an observer are within the area of the 2D pupil array, the viewer observes accurate parallax of any two patterns.

Similar to the photos of Fig. 7(b), in order to showcase the direct-viewing capability of the configuration, a small-aperture smartphone camera in place of a human eye captured photos of the DMD surface directly (but without ambient light). The 18 photos of the 18 perspectives are arranged in Fig. 8(b) corresponding to the pupil array positions of Fig. 8(a).

5. Discussions

5.1 Impact on space-bandwidth product

The space-bandwidth product (SBP) is a performance metric which has been defined as the number of degrees of freedom of an optical system [19,20]. For instance, in an imaging system, the number of independent imaging points can be rudimentarily calculated by dividing the image area by the point-spread-function area (effectively multiplying the image area by the 2D spatial bandwidth). Similarly, in digital information projection systems using spatial light modulators (SLMs), the number of degrees of freedom, or independent channels counting projection complexity, is a count of the number of pixels on the SLM, since this is the resolution and complexity limit of projected images.

Typical SLM projection systems only require pixel sizes to be smaller than human acuity after projection and observation. However, the development of light-field and multi-perspective projection systems, where each pixel corresponds to one spatial coordinate and one angular coordinate, has defined a new challenge for SLM-based projection systems. This is demonstrated in microlens array displays and grating array displays, where spatial resolution is compromised for increased perspectives, and maintaining spatial resolution while achieving 10 or 100 perspectives would, respectively, require a 1 or 2 order of magnitude increase in pixel density [2–4].

While smaller pixels have been achieved [10], other multi-perspective display techniques, such as pupil-segmenting [13,14] and the proposed diffraction-based pattern steering method, offer means to increase perspectives without cost to spatial resolution. For equivalent comparison across these multi-perspective displays, SBP density $\left(\frac{\#\text{of}\text{channels}}{\text{active}\text{area}}\right)$ is a more valid comparison than pixel density $\left(\frac{\#\text{of}\text{pixels}}{\text{active}\text{area}}\right).$ A 3.015 µm pixel has been previously achieved for a projection SLM [10], equating to an SBP density of 1.1 × 10⁵ mm⁻¹. In our current 3-LED by 6 diffraction order demo, each 7.637 µm micromirror (edge) pitch outputs a pixel for each of the 18 output patterns, equating to a SBP density of 1 × 10⁵ mm⁻², with plenty of room to scale (Section 6). An additional orders of magnitude increase in the SBP density is required to achieve, without compromising the spatial and angular resolution, an accommodation blurring effect by Maxwellian image overlap for to a multi-focus display [21].

5.2 Étendue increase due to the ASLM

In near-eye and head-up displays, the FOVs and eye-box sizes are limited by SLM étendue. Projection optics can be tuned for wider FOV at cost to a smaller eye-box, and vice-versa. The eye-box size or FOV can only be increased, without consequence to the other, by either increasing the output cone angle or the size of the SLM, either for the effect of increasing system étendue.

The pupil-segmenting technique used for traditional binary projection (Fig. 3(c)) interleaves across different illumination sources to increase SBP at the cost of throughput per channel (e.g., each channel might be F/10 instead of the typical F/2.4 of ± 12° rotating micromirrors), but it does not increase overall system étendue. For instance, given a DMD with a fixed area and illumination F/# (limited to prevent pupil contamination in the steering axis), segmenting the input illumination into multiple sources will not increase the overall étendue unless the input sources breach the F/2.4 limit (which is an option in the axis orthogonal to the steering axis).

By implementing the diffraction-based multi-order ASLM beam steering technique, the angular throw of the output pupils has been increased from 24° to 48°, equivalently improving the F/2.4 system to F/1.22 (see 48° angular extent on Fig. 4(a)). However, the angular extent of the binary patterns is decreased because the diffraction order(s) with off-state light are not usable for independent pattern projection. For instance, the angular extent of the seven on-state diffraction orders of Fig. 4(a) is about 42°, corresponding to F/1.40. The étendue is “nearly doubled” from traditional DMD projection schemes since the angular extent doubles (e.g., from 24° to 48°), but is also decreased by the setup-dependent off-state diffraction order(s) (e.g., from 48° to 42°). The angular extent can be closer to the full 48° if a longer wavelength or smaller micromirror is used, thereby creating more on-state diffraction orders and slimming the angular extent of the off-state(s) by increased angular selectivity.

While we have not tested DMDs with different micromirror angular throws (e.g., ± 17°), our results suggest we would achieve a similar “nearly-doubled” étendue in these cases as well (e.g., improving from F/1.71 to nearly F/0.89).

5.3 Diffraction efficiency of the programmable DMD blazed grating

Diffraction efficiency and crosstalk measurements were taken for 532 nm illumination at an angle-of-incidence of 30°. The experimental setup is depicted in Fig. 9. A collimated beam is incident on a DMD and steered into diffraction order directions. A test APD (Hamamatsu, C12702-04, 3 mm diameter active area) on a swing arm rotates about the DMD to capture individual diffraction orders while maintaining propagation length, vertical alignment, and normal APD angle-of-incidence. A lens (10 mm diameter, 30 mm focal length) in housing before the test APD collects and focuses light from a single diffraction order at a time onto the test APD. (This is to accommodate for beam divergence due to typical diffraction and aberrations, not particularly due to the DMD.)

 

Fig. 9 Diffraction efficiency and crosstalk measurement experimental setup using a swing arm about the DMD and an active-normalization system of two APDs.

Download Full Size | PDF

We determined our particular well-aged 532 nm laser source unit (Coherent, Vector 532-1000-20) has significant pulse energy jitter from pulse-to-pulse and has average pulse energy walk at ~10-minute scales. To compensate for pulse-to-pulse energy jitter, the APDs’ signals were read by an oscilloscope (Rigol, DS1104, 1 GSa/s) using N = 128 averaging. A reference APD (same model, also averaged at N = 128) sampled the beam through a 50/50 beam splitter with fixed alignment to compensate for average pulse energy walk. The beam at the reference APD was small enough to fit onto the 3 mm diameter active area due to a shorter propagation distance from the beam splitter. However, even if the beam was larger than the active area, fixed alignment would have meant that an equivalent portion of the beam would always be captured by the reference APD, thereby making a steady normalizing value. Each N = 128 averaged test APD read-out was first normalized against a respective N = 128 averaged reference APD read-out, for the same 128 oscilloscope captures, before further analysis to ensure pulse-to-pulse energy jitter and pulse energy walk was fully compensated for. The stability in the following results shows the effectiveness of this active-normalization approach.

The illumination scheme is equivalent to the direct illumination scheme previously discussed (Section 3.2): a collimated beam illuminates the DMD. However, special care was taken to ensure less than a single quadrant was illuminated, for reasons discussed in Section 5.4 and to maintain alignment efficiency, by using a collimated beam of ~1 mm at the DMD (a single quadrant is 1.64 mm wide [22]).

The swing arm was fixed to the optimized angle for each diffraction order. The captured signal was not very sensitive to swing arm angle near each diffraction order’s peak since the lens diameter was 2-3 times greater than the beam diameter at the lens. This gave greater certainty that we were indeed collecting most of the photons at each position. Once the swing arm angle was fixed per diffraction order, the beam steering angle was incremented by incrementing the pulse delay between DMD reset trigger and laser pulse trigger on a driving Arduino Uno microcontroller (see [16] for more details about time delays, driving electronics, and DMD-based beam steering). APD peak values were captured for each time delay increment (equivalent to mirror angle increments since the micromirror transition is fairly linear [16]). Diffraction efficiency as a function of time delay, per diffraction order, is shown in Fig. 10(a). With a source wavelength of 532 nm, only 8 diffraction orders were used because the + 4 diffraction order is beyond the angular extent of the micromirror angular swing at 30° angle-of-incidence (i.e., some energy goes into + 4, but a peak cannot be captured). The 0th diffraction order signal contained the specular reflection due to the cover glass and made a signal floor of 0.156 (normalized pulse energy units). This value was subtracted from all points of the 0th diffraction order in Fig. 10(a) to accurately show the diffraction-based beam steering efficiency. The specular reflection is plotted separately by the dashed line on Fig. 10(a). The lower peak in Fig. 10(a) of diffraction order −4 at 500 ns is due to an initial compression of the micromirrors before transition (i.e., the micromirrors go the opposite direction at first, e.g., drop at 625 ns) [23].

 

Fig. 10 (a) Steering efficiency into each diffraction order direction as a function of illumination pulse delay (dictating mirror angle) beyond 215 µs at 62.5 ns increments. (b) Steering efficiency into diffraction order −2 at 2.5 ns increments.

Download Full Size | PDF

The process was repeated for diffraction order −2 (fixed optimized angle, incremented delay) at 2.5 ns increments using a programmable delay microchip (Maxim Integrated, DS1023, 8-bit in 250 ps increments) to determine the peak diffraction efficiency sensitivity and to demonstrate fine steering control. We verified 2.5 ns delay incrementing (by 10 increments of 250 ps on the DS1023) of the Arduino Uno with an oscilloscope. Figure 10(b) shows the high stability and fine delay tuning capability of the system for the peak of diffraction order −2 at 2.5 ns increments.

The diffraction efficiencies in each diffraction order, for each optimized delay, are shown in Table 2. Values of “-” were too small to distinguish from noise. A value measured in a diffraction order different from the value’s respective optimized order signifies crosstalk.

Table 2. Measured diffraction efficiency and crosstalk for 532 nm source at 30° angle-of-incidence.

View Table | View all tables in this article

These diffraction efficiency values generally concur with our previous measurements at 905 nm [16], including the trend of max diffraction efficiency per order to decrease from negative to positive diffraction orders, but the odd-order suppression is new to the 532 nm measurements. This may have been previously missed in the 905 nm measurements [16] due to fewer diffraction orders and a stronger cover glass reflection.

The diffraction efficiency of light into a particular discrete order from a diffraction grating is fill-factor dependent [20]. Individual diffraction orders may also be suppressed by fill-factor [20]. The odd-order suppression in the experimentally measured diffraction efficiency can be explained by a diffraction-order-dependent fill-factor. A diffraction-order-dependent fill-factor is best illustrated by Fig. 11, in which DMD micromirrors are tilted ± 12° and viewed from a 30° angle-of-incidence source.

 

Fig. 11 DLP3000 DMD geometry viewed from the source direction (30° angle-of-incidence) depicting fill-factor for micromirrors tilted (a) 12° toward source; (b) 12° away from source. (Parity: The source is setup to view the micromirrors 30° to the left.) Height h is measured normal from micromirror base surface (same as Fig. 1(b) and 1(c)).

Download Full Size | PDF

The projected area of each micromirror in Fig. 11(a) and 11(b) is angle-dependent due to the out-of-plane motion of the mirror around the torsion hinge, despite the projected micromirror pitch being static, so the projected fill-factor is mirror-angle-dependent. The fill-factor is therefore diffraction-order-dependent since light is steered into each diffraction order at different mirror angles.

The following outlines an analytical model for the diffraction efficiency of a DMD-based programmable blazed grating. A 1D model will be used for a simplified discussion. This 1D model is a valid comparison for the 45°-clocked, diamond-orientation DMD architecture due to the independent nature of orthogonal diffraction. That is, given a square (non-clocked) grid of micromirrors, the far-field diffraction pattern will also be a square grid of points (the Fourier transform of a 2D comb function is a 2D comb function [20]). If the square grid of micromirrors is clocked (rotated about a surface normal) by 45°, the far-field diffraction pattern will simply follow the 45° clocking. The axis of micromirror rotation (e.g., edge-to-edge vs. corner-to-corner) dictates the steering axis between diffraction orders (e.g., steer across neighboring orders vs. corner-to-corner orders). Therefore, the steering axis can be decently modeled by only considering the mirrors’ single rotating axis and keeping in mind the 45°-clocked diffraction order grid.

According to Goodman, the diffraction efficiency, ${\eta }_n$, of a 1D phase blazed grating into diffraction order $n$ can be calculated by the Fourier series coefficient, $c_n$, where${\eta }_n={\left|c_n\right|}^2$ [20]. The Fourier series coefficient is given by

The magnitude of the phase modulation across one period of the grating is dependent on the one-dimensional projected fill-factor, the micromirror angle, and the wavelength, $\lambda$, as shown here:

The analytical solution for diffraction efficiency, both order-dependent and mirror-angle-dependent, can now be determined as:

This solution demonstrates the compounded reliance of the diffraction efficiency on the mirror angle, even though Eq. (6) is simplified for a 1D array. The multiplied cosine and sine terms, most ${\theta }_{\text{mirror}}$ dependent, cause oscillations in diffraction efficiency for each diffraction order as each order has a different optimized mirror angle. This oscillation results in odd-order suppression, which can be compensated for in design by either using a variable pulse energy source or by diffraction-order-specific attenuation. Alternatively, and simpler for design, looking at the cosine quotients throughout, a reduction in the illumination angle-of-incidence would significantly increase the diffraction efficiency and maintain a more-stable domain.

5.4 Current limitations

The Lightcrafter Evaluation Module with the DLP3000 DMD uses a quadrant reset (also named “quad-block reset”) for which a phased micromirror reset occurs across four separate quadrants (i.e., blocks) of the DMD active area. The cascaded reset of the four quadrants causes a quadrant-dependent efficiency if the entire DMD is illuminated with a single short pulse into a transitional diffraction order. Figure 12 shows the quadrant-dependent efficiency change, where the micromirrors of each quadrant are at different transition points. Quadrants not steering light accurately into a diffraction order will spread energy across neighboring, high-order diffraction orders.

 

Fig. 12 Phased quadrant reset causing quadrant-dependent beam steering efficiency.

Download Full Size | PDF

The quadrant reset allows the DMD to actuate one quadrant of pixels while loading data to another quadrant of pixels. Other DMDs and drivers, such as the DLP7000 driven by the Discovery 4100, can use a global reset which actuates all pixels simultaneously. For convenience of the Lightcrafter Evaluation Module with the DLP3000, we only illuminated one quadrant and kept the micromirrors of the other three quadrants in the off-state through testing.

The DLP3000 has 608 × 684 micromirrors, though it should be noted that diamond orientation and unique addressing causes the effective pixel pitch in the vertical dimension to be half that of the horizontal dimension (e.g., projecting a white square of pixels in a 608 × 684 bitmap appears to have a rectangular output). A single quadrant has a resolution of 152 × 684, though an aspect ratio of 9:4 (from 152 × 342).

6. Concluding remarks

The DLP3000 at 632 nm illumination has 7 independent output diffraction orders (not counting off-state output) for a total SBP, as calculated by the total number of independent output pixels, of 152 × 684 × 7 ≈7.3 × 10⁵. Our three-illumination-source design increased this to152 × 684 × 7 × 3 ≈2.2 × 10⁶. The SBP of the ASLM can be scaled for each of these four factors. For instance, the DLP7000 has a corner-to-corner pixel pitch of 19.34 µm for 13 independent output diffraction orders, 1024 × 768 resolution, and global refresh for an SBP of 1024 × 768 × 13 ≈1.0 × 10⁷ [24]. Additionally, increasing the number of illumination sources and decreasing their sizes, for instance by a 2D individually addressable LED array or OLED device, can increase the independent output pupil density. By example, a 10 × 10 LED array, for the effect of a 10 × 130 output pupil array, or 1300 views, would have an SBP of 1.0 × 10⁹. For even more smooth transitions between output pupils, a 100 × 100 source array could be used for 130,000 views or an SBP of 1.0 × 10¹¹, the equivalent of a 100 giga-pixel display. The same system would have an SBP density 6.9 × 10⁸ mm⁻². Head-tracking and eye-tracking techniques can be employed to render only necessary pixels to comply with the 32 kHz refresh rate of the DLP7000. While the beam steering factor is only contributing one order of magnitude to the SBP, the dramatically increased SBP of a pupil-segmented system is more accessible due to the high angular extent of the ASLM system, as discussed in Section 5.2.

The diffraction-based output directions of the ASLM are wavelength dependent, which must be considered in a full-color implementation. Powered dispersive elements, such as highly chromatic lenses or diffractive optical elements, or an array of dichroic surfaces can be used to compensate for the wavelength dependence of the output pupil directions. Alternatively, source tiling can enable overlap across different diffraction orders. For instance, source position 1, wavelength 1, diffraction order 2 can be made to overlap source position 2, wavelength 2, diffraction order 3.

Gray-scaled ASLM output is possible by pulse width modulation (PWM), typical to DMD projectors, but enacting such gray-scaling is currently difficult on the Lightcrafter Evaluation Module due to hardware and software limitations.

Both bit-depth and multiple perspectives are results of high-speed DMD time-integration. The tradeoff between bit-depth and number of perspectives must therefore be noted, though varying pulse energy across binary patterns can offer higher time-integrated bit-depths than constant pulse energies. For instance, an ASLM multi-perspective display with 32 kHz refresh rate [24], 13 diffraction orders and perspectives, interleaved black patterns (factor of 2), full RGB color (factor of 3), and 30 FPS has 32,000 ÷ 13 ÷ 2 ÷ 3 ÷ 30 ≈13 binary patterns per color per frame per perspective. This example can either achieve 13-bit gray scaling per color with varying pulse energies, or 3-bit (13 levels rounded down to 2³ = 8 levels) without varying pulse energies.

The ASLM system can alternatively be described as a digitally controlled multi-order discrete phase-amplitude hologram. In volumetric holographic data storage, eigenmode and angular multiplexing techniques have been applied to increase storage density [25], and solid-state eigenmode multiplexing by phase-only SLM has been achieved [26]. Additionally, a DMD-based phase-only spatial modulation technique has been previously demonstrated [27]. The phase-amplitude modulation of the ASLM offers the potential for a near-solid-state, single-chip-driven holographic data storage system with spatial and angular multiplexing capabilities for higher storage densities.

We have previously reported a multi-pulse DMD lidar beam steering system, in which multiple pulses are incident on the DMD mirrors during one transition of the micromirrors to improve sampling rate [28]. We have also reported a multi-beam DMD lidar beam steering system, in which multiple horizontally-offset sources are used to increase horizontal angular resolution [29]. Both of these techniques can be implemented into the ASLM system for higher angular resolution and sampling, though driving limitations currently exist which prevent the pattern from being adjusted during a single transition of the DMD.

The systems presented demonstrate the single-plane angular and spatial modulation capability of the ASLM, the nearly-doubled étendue of the DMD, and the impact to applications such as display, 3D display, light-field projection, and holography. We have discussed the implications of this high-impact device, which relies solely on readily available and highly-developed components, and we have proposed a path forward to continue increasing SBP by orders of magnitude.