Open Access
Add to BibSonomy
Abstract
We propose a microlens array (MLA) construction method based on sub-channel optimal design and splicing, and an integrated projection imaging analysis method by using ray tracing and image warping. Our stop mask greatly improves imaging quality and eliminates crosstalk. We realize various projection distances, required projection imaging dimensions, and design optimization of sub-lens structures, providing freedom and possibility for MLA structure design requirements. Optical system chief ray tracing and sub-image generation is combined by using radial basis function forward image warping. Imaging distortion and overlap misalignment from short focal projection, multi-aperture offset, and complicated surfaces are perfectly corrected. Sub-image warping pixel mapping facilitates real-time replacement of projected images. We conduct substantial MLA integration imaging designs and precision analysis of different sub-aperture sizes, MLA sizes, and projection distances.
© 2019 Optical Society of America under the terms of the OSA Open Access Publishing Agreement
1. Introduction
The traditional single-aperture projection optical system, with a large lens aperture and field of view (FOV), thus causing the projection system to be large. Because of the development of ultra-short throw projection technology of freeform surfaces, short-distance projection imaging can now enter a new stage [1–3]. In the practical application for existing imaging systems, MLA has the great advantages of a large depth of focus, short imaging distance and miniaturization [4–7], as well as widespread use in three-dimensional integrated imaging display [8–11], light field imaging [12–14], and other micro-optics application fields [15–17]. An integrated projection system based on MLA can realize the projection imaging effects of ultra-short distance, large depth of focus, and high illumination uniformity through multi-channel projection imaging [6,7]. In the future, the MLA-based projection imaging device could be applied to ultra-short throw image projection, structural light illumination, free-form surface uniform illumination, advertising and so forth.
Figure 1 shows a sub-image of an individual MLA channel imaged on the projection plane. Every sub-image corresponds to its own MLA sub-channel and contains all the imaging information of the projection picture. This means that every imaging point S' on the projection plane could find its object point Si on every sub-image of every sub-channel in MLA. For imaging integration, it is certainly true that every sub-image suffers scaling, translation, inversion, and sometimes optical imaging distortions. Hence, the design method of a system with high-integrated projection imaging accuracy is very important. In this paper, we propose an integration projection imaging optical design and analysis methods based on MLA. The methods we proposed can be very flexible implementations of different apertures, structures, and sub-lenses of different surface types. By considering the projection distance, projection dimension, and the actual demand, we design and optimize the single channel microlens. The sub-lens is then spliced into an MLA of a certain size. Thereafter, through optical system ray tracing by optical design software (CODE V) and the radial basis function (RBF) image warping method [18–20], we obtain perfect correction of the inherent distortions of the sub-lens and the keystone distortion of the oblique projection position. Finally, the sub-images corresponding to each sub-channel of the MLA can be obtained.
MLA projector based on paraxial imaging theory was proposed and designed by M. Sieler [6,7]. However, the design method based on paraxial imaging theory is limited in several aspects, in particular when using aspheric or even freeform surface type lenses, as well for the development and application of complicated microlens systems. Especially, for high precision integration imaging, the image distortion due to the large FOV and the complicated system structure of the micro sub-lens cannot be neglected. Therefore, the high precision predistortion and sub-lens imaging design and optimization process are indispensable. The image warping method we adopted could accurately warp the sub-image including scaling, translation and predistortion. Furthermore, the stop array inserted by us not only eliminates channel crosstalk, but also maintain and strengthen the uniformity of illumination and the sharpness of the pattern. In addition, the shared projection area of each sub-channel in MLA is analyzed and considered, and the illumination uniformity and integrity of the projection image are guaranteed. Compared with the MLA imaging analysis of the paraxial method proposed by M. Sieler, the method presented could also provide flexible degrees of freedom in the microlens design.
Figure 2 shows the basic structure of ultra-short throw projection system proposed based on MLA. It is composed of three lens layers. The first and second lens layers constitute the projection front MLA group. The first and third lens layer is constituted of the plano-convex sub-lens (Lens 1 and Lens 3 in sub-channel). The second lens layer is a flat plate (Lens 2 in sub-channel), where one side is the stop array mask layer and the other side is the sub-image array mask layer. The stop array mask layer, which limits the size of the light beam but improve the imaging quality, also shields and eliminates the crosstalk stray light between neighboring sub-channels. The third lens layer is the condenser MLA layer, which is also constituted of plano-convex sub-lens (Lens 3). To obtain sufficient flux and a good imaging effect for each sub-channel, the condenser sub-lens, the sub-image mask, and the former sub-lens projector constitute a projection system based on the principle of Kohler illumination. As shown in Fig. 2, the imaging principle of MLA is briefly illustrated by taking the imaging optical path of a sub-channel as an example. The third condenser sub-lens of the sub-channel is illuminated by a collimated parallel beam, the light is then refracted to the sub-image mask layer, and the sub-image is projected onto the projection plane via the former projection MLA group. The image projected on the projection plane is made from the superposition of the imaging of each sub-channel. The MLA manufacturing method such as thermal reflow [21], gray scale lithography polymer [22], soft lithography [23] and femtosecond laser micro-nanofabrication via two photon polymerization [24], have been widely used in mass production. The mask layer introduced above can be fabricated by the regular mask exposure method or using an electron-beam written chromium mask in a lithography process [25]. The latter can achieve higher sub-image resolution.
In section 2 of this paper, we introduce the design optimization and analysis of the projection lens of the sub-channel. The MLA projector constitution method by splicing the sub-channel is also proposed. In section 3, the shared area of MLA far-field projection is analyzed. Substantial MLA integration imaging designs and precision analysis of different sub-aperture sizes, different MLA sizes, and different projection distances are conducted to explore thoroughly the influencing factors and internal rules of the microlens array projection imaging. In section 4, considering each sub-channel projection imaging offset and distortion problems, the RBF image warping algorithm is adopted to generate the sub-image array mask. Through system modeling and imaging simulation (LightTools), we verify the effectiveness of the proposed method.
2. Optical design method of MLA projector
We adopt chief ray tracing analysis of the optical system to realize the goal of MLA projection by splicing the optimized sub-lens. By chief ray tracing, the geometric characteristics of the sub-image source can be accurately analyzed, the RBF image warping method can accurately warp the initial sub-image, and the warping precision could be less than 1 pixel. The combination of the ray tracing method and image warping method guarantee the final imaging integration precision, including the inherent distortion of the sub-lens due to the complicated sub-lens structure and large FOV. The method proposed in this paper is applicable to different MLA imaging structures, making the design, application, and operation of the MLA structures more flexible and convenient.
The specific design flow chart is shown in Fig. 3. First, according to the basic parameters, such as the aperture and the focal length of the sub-lens, the initial structure of the sub-lens is established and optimized according to the projection FOV, geometrical size, and image quality requirements. The basic image analysis and image quality optimization of the sub-lens are realized through the variation of the lens structure arguments such as the lens tilt, lens surface parameters, and the control of the focal length. The sub-lens obtained by the above optimization is set to be the central sub-lens of the MLA, which means the optical axis of this sub-lens will also be the central axis of the whole MLA. Therefore, the offset amounts of the optimized sub-lens are set to zero in the X and Y directions. According to the sub-aperture size, we can obtain the required array form (rectangular array arrangement/hexagon array arrangement), the required array size, and the geometry-oriented offset data matrixes of the other sub-lenses relative to the central sub-lens in the required MLA. The offset and tilt matrixes obtained above can be used to loop addressing the position of each sub-lens in the required MLA. Therefore, the MLA projector could be designed just by arranging the optimized central sub-channel projector. The offset matrixes calculated above are used to locate the positions of each sub-channel. In order to maximize the projected area of a single sub-channel and eliminate crosstalk between sub-channels, the corresponding sub-image on sub-image plane should be located within the circular aperture of the sub-lens. By ray tracing with the maximum field aperture of the sub-image plane, we can analyze the specific geometric projection size of the MLA on the projection plane and the sub-imaging overlapping effect of each sub-channel. After ray tracing of each sub-channel, the shared projection area at the projection plane is selected and divided into a grid of field points. The chief rays of the grid of field points on the projection plane are traced in each sub-channel and the tracing points on the sub-image plane of sub-channel are obtained. All the sub-images masks from the different sub-channel of MLA could be obtained through the tracing points on the sub-image plane and by adopting the high-precision RBF image warping method. The sub-image array mask constitutes all these warped sub-image masks. Finally, the simulation of the projection MLA model and the simulation of MLA projection imaging are realized.
2.1 Optimization and analysis of the sub-lens structure design of MLA
To illustrate the design method in more detail, we provide a design example. In this paper, the design structure of projection sub-lens is conducted as an inverse procedure of actual imaging which means the left-most object plane is the actual projection imaging plane and right-most surface is the actual sub-image source plane. Figure 4(a) shows the initial projection sub-lens structure we created, where surface 1 is a sphere, surface 2 the plane of the stop, and surface 3 the sub-image plane, also called the sub-image mask layer plane. The aperture size of the sub-lens is p. The half FOV of the sub-lens is p/2 for individual imaging. In this paper, surface 2 of the stop is introduced to improve the image quality. In this design example, we adopt a relative larger sub-aperture size p = 1.40 mm of the sub-lens. Because the Fresnel Number FN is much larger than 200 (FN >> 1) in the visible spectrum, the imaging influence suffered by the micro-aperture diffraction effects could be neglected [17]. Figure 4(b) shows the result of the optimization of the initial structure in Fig. 4(a). The effective focal length f is 1.79 mm, the projective distance 500 mm, the overall length of the sub-lens 2.57 mm, and the diameter of the entry pupil 0.8 mm. In this optimization design, surfaces 1 and 3 stand for the aspheric surface, surface 2 for the planar surface, and surface 4 for the sub-image plane.
Fig. 4 (a) Initial projection sub-lens. (b) Optimized projection sub-lens. (c) Half-FOV MTF of optimized projection sub-lens. (d) Spot diagram of optimized projection sub-lens.
The introduction of the aspheric surface greatly improves the imaging quality of the projection sub-lens. In the design and optimization of the sub-lens, imaging quality and luminous flux are prioritized, because the scaling and distortion problem caused by the large FOV and offset of the sub-image will be solved by the image warping method in section 3. Figures 4(c) and 4(d) correspond to Modulation Transfer Function (MTF) and the spot diagram analysis results of the optimized sub-lens in Fig. 4(b). In order to analyze the influence of chromatic aberration in the visible band, Fig. 5(a) and 5(b) shows the ray aberration plot and the field curves diagram of the optimized sub-lens. It can be seen from the figure that chromatic aberration is well controlled, which has little influence on the projection imaging quality. In addition, in section 3.1 of this paper, we adopted the solution of projection imaging of each sub-channel to the shared area to ensure that the projection imaging has a uniform illumination, which to some extent reduced the actual effective imaging FOV in each sub-channel, and further decreased the lateral chromatic aberration, which is a variation of the chief ray location as a function of wavelength. As shown in Fig. 5(b) at the upper right corner, the curves of each wavelength distortion rate basically coincide, indicating that the lateral chromatic aberration is very small in this sub-channel system. After the optimization of the projection-imaging sub-lens, the back-end design of the condenser sub-lens (lens layer 3 of condenser MLA layer in Fig. 2) should be taken into account. For convenient splicing, we used a plano-convex lens as condenser sub-lens. The design of the condenser sub-lens needs to consider the divergence of the source and the Kohler illumination principle. For maximum light efficiency, the condenser and projection imaging sub-lenses constitute a Kohler illumination structure [6,17], i.e., the light source passing through the convergent sub-lens and the field aperture (position 4) is imaged on the stop of the projection sub-lens system (position 2).
Fig. 5 (a) Ray aberrations of optimized projection sub-lens. (b) Field curves of optimized projection sub-lens.
2.2 Splicing constitution of MLA
After the optimization and construction of a complete sub-channel, MLA can be obtained by splicing the sub-lens channel. The offset matrixes are generated for the required number of sub-lenses in MLA. According to these offset matrixes, we can achieve the addressing position of each sub-lens and further imaging projection analysis in the target MLA. Taking the rectangular array arrangement form as an example, the position relation between the adjacent lenses of the rectangular arrangement sub-lenses is shown in Fig. 6(a). px and py are the center interval size between the adjacent lenses in the MLA when considered as a rectangular arrangement and p is the aperture size of the sub-lens. Considering a close-packed rectangular arrangement array, px = py = p, Fig. 6(b) describes the splicing form of the central column sub-lens passing through the central sub-lens of the MLA in the YZ direction. Assuming that ML (icentral, jcentral) is the optimized central sub-lens, and it could have an inclination(the incline angle is determined in the previous sub-lens design optimization; the X-axis is the rotation axis). Corresponding to Fig. 6, the offset amounts of a sub-lens ML (i, j) in the rectangular splicing array are:
where XDE (i, j), YDE (i, j), and ZDE (i, j) are the offset amounts in the X, Y, and Z directions, respectively, of the ML (i, j) sub-lens in MLA relative to the central sub-lens obtained by optimization.Fig. 6 Sketch map of a rectangular array arrangement structure of MLA. (a) px and py are the center interval between the adjacent lenses in the MLA, while p is the aperture of the sub-lens. (b) Cross section view of MLA through the central sub-lens of the MLA in the YZ direction and the definition of YDE and ZDE. (c) Cross section view of MLA through the MLA plane and the definition of XDE.
Through the above MLA offset matrixes, the optimized central sub-lens can easily be shifted to all possible locations of the sub-lenses in the MLA. Through traversal addressing of the offset matrixes of MLA, we can achieve projection-imaging analysis for each sub-lens in MLA. In addition, MLA is realized by using a planar MLA with a circular aperture sub-lens. A hexagonal arrangement splicing of the sub-lens is also realized in a similar approach.
3. Projection area analysis of MLA
3.1 Paraxial imaging characteristics of MLA
Figure 7 shows the case of a planar MLA structure parallel to the projection plane. Assume that the neighboring sub-channels have no crosstalk and that each sub-channel is imaging independently. Because each sub-channel of the MLA is the same, the sub-image facet of each sub-channel has the same maximum field aperture, and the projection size of each sub-channel is the same on the projection plane. The paraxial imaging optics is introduced for brief projection area analysis. Here, the sub-aperture size of the sub-channel of the planar MLA is p, the focal length f, and the projection distance L. Then, the projection radial dimension size of a single sub-channel is approximately S:
To realize enough projection illumination uniformity by the superposition of multiple sub-channels in the same area, the projection imaging areas of all sub-channels of the planar MLA should coincide as much as possible. As shown in Fig. 7, assuming that the sub-channel number of the MLA in the maximum aperture direction is N, it is evident that the size of the shared projection area (the gray area in the right drawing in Fig. 7) of each sub-channel in the projection distance L can be expressed approximately as D:
According to the analysis of the above equation, when the projection distance L is short (L ≈Np), the total and shared projection area of the sub-channels will be significantly reduced, which could be regarded as MLA near-field imaging. On the other hand, when the sub-channel number N and the sub-aperture p of the sub-channel are constant and the projection distance is far greater than the maximum aperture size of the MLA Np (L >> Np), it could be regarded as MLA far-field imaging. In this paper, we focus on the problem of MLA far-field imaging. Therefore, the size of the MLA will have little influence on the overall projection area, but there is still a small portion lost at the edge of the projection area. Relative to the projection zone, the loss is (N−1)p, which is approximately equal to the size of the whole MLA aperture. In addition, for the microlens with the same F/#, when the projection distance L is set, the increase of the sub-channel number N and the sub-aperture size p will also shrink the shared projection area D. Moreover, the projection imaging aberration increases with the increase of the sub-aperture p, which might degrade the intensity of illumination uniformity and imaging quality.
3.2 The optical integration precision analysis of MLA integrated projection
As shown in Fig. 8, the boundary curves of the shared region can be obtained by chief ray tracing along the maximum field stop of each sub-channel projector. The boundary curves of the shared projection area are composed of the projection circle boundaries of the sub-channels at the maximum aperture of MLA (for planar MLA, the four projection circles correspond to red, green, magenta, and black in Fig. 8(a)). The boundary curves of the shared projection region could be fitted by polynomials, and the projection position in the shared projection area can easily be determined.
Fig. 8 (a) The sampling FOV points in the shared area. (b) RMS spots analysis of a 1.4-mm sub-aperture 12 × 12 MLA. (c) Analysis of average RMS spots in MLA imaging for different sub-apertures and different number of sub-channels at a projection distance of 500 mm.
Suppose that, along the boundary curves of the shared region, the red rectangle ends up in the shared projection region shown in Fig. 8(a) as the final projection region of all sub-channels. The projection sizes in dimensions X and Y are Xproj and Yproj, respectively. Because the sub-channel of the MLA system is a rotationally symmetric structure, the analysis is performed for half of the diagonal FOV of the entire projection area on projection plane. As shown in Fig. 8(a), a RMS spot analysis of the MLA projection imaging was carried out by sampling 21 field points on the projection plane uniformly along the red dashed line. To reflect the cumulative effect of the sub-channel imaging quality, the integral imaging evaluation of the MLA projection can be calculated by using:
where the imaging RMS spot radius of the MLA at the projection field of view is determined by the average of the RMS spots of the N sub-channels at the projection FOV. Here, is the object field corresponding to N−1 different sub-channels.For instance, as shown in Fig. 8(a), when the corresponding projection distance is 500 mm, Xproj and Yproj are equal to 250 mm. Figure 8(b) shows the RMS spots analysis of an 11 × 11 MLA with a 1.4-mm sub-aperture. The blue lines depict the RMS spot size variation of the different sub-lenses projectors accompanying the normalized field. The green line illustrates the variation of the average RMS spot size calculated by Eq. (5), while the red discrete small circles depicts the RMS spot size variation of the central sub-channel projector in the MLA. It is clear that the RMS-spot-size differences between the sub-channels are evidently varying in a limited range, and the average of RMS spot size of MLA is approximately the center sub-lens in MLA (the aforementioned optimized sub-lens). Therefore, the central sub-lens could reflect the overall average situation of a projection MLA. The varying range could illustrate the optical integration imaging deviation of MLA. In other words, the optical integration precision will be guaranteed if the RMS spots size of all the fields is much larger than the limited variation range. Figure 8(c) shows the RMS spots analysis graph of the MLA projection, depicted by different sub-aperture sizes and different number of sub-channels. The curves describe the average RMS spots of different sizes of the MLA cumulated imaging (the numbers of sub-channels are 6 × 6, 11 × 11, 22 × 22, 32 × 32, and 48 × 48) with sub-channels of sub-aperture sizes 0.6, 1.0, 1.4, 1.8, and 1.4 mm. The curves in Fig. 8(c) show that the differences in the average imaging aberration of MLA are very small with the same sub-aperture size and different MLA sizes. However, the marked differences of the average imaging aberration of MLA are presented with the different sub-aperture sizes and the same MLA size. This indicates that when the imaging distance is fixed, the optical integrated imaging aberration of MLA is mainly affected by the sub-aperture size of MLA, while the sub-channel number of MLA has a relatively small impact on the optical integrated imaging aberration of MLA. As the number of sub-channels of the MLA increases, the effect on the integrated aberration of the MLA system increases, evidently illustrated by the curves of p = 1.4, 1.8, and 2.2 mm. In fact, as the number of sub-channels increases, the shared area of the MLA projection will be reduced. To obtain a uniform and adequate projection size, the number of sub-channels in MLA should not be excessive.
Figure 9 depicts the integrated projection imaging aberration when the sub-lens of the above optimization design (500-mm projection distance and 1.4-mm sub-aperture size) is used to realize the same sub-aperture size (1.4-mm) with a different number of sub-channels (6 × 6, 11 × 11, 22 × 22, 32 × 32, and 48 × 48) and at different distances (300, 400, 500, 600, and 700 mm). From the five graphs in Fig. 9, it can be concluded that when the projected imaging distance is less than the projected design distance, the integrated imaging quality of the MLA projected images, corresponding to different MLA sub-channels, deteriorates. Moreover, the imaging quality corresponding to the various MLA sizes deteriorated further when the projection imaging distance was further shortened, and the integrated imaging quality of the larger MLA size deteriorated more seriously. When the projected imaging distance is greater than the projected design distance, the integrated precision of the MLA projected imaging corresponding to different MLA sub-channels remains good. When the projection imaging distance is further increased the integrated imaging aberrations of different MLA sizes gradually approach each other and still maintain good imaging quality. In a sense, the MLA projection imaging has a large depth of focus (DOF). We also analyze the same case as the optimized sub-lens, the integrated imaging analysis of MLA with different sub-aperture sizes (0.6, 1.0, 1.4, 1.8, and 1.4 mm) while at different imaging distances (300, 400, 500, 600, and 700 mm) for the same MLA size (11 × 11). As shown in Fig. 10, it is clear that the effect of the sub-aperture size on the overall integrated imaging of MLA is still strong due to different color curves bunches, and a larger sub-aperture size will undoubtedly introduce larger aberrations. In addition, it is observed that the projection imaging distance still has a certain influence on the overall integrated imaging, and the projection distance has a greater influence on the MLA integrated imaging with a larger sub-aperture. By analyzing the curves of different projection distances for the same sub-aperture size, we conclude that when the projected imaging distance is less than the designed imaging distance, the image quality of the integrated MLA imaging deteriorates more seriously.
Fig. 9 Average RMS spot size analysis of MLA with different sizes of MLA and projection distance by using the designed sub-aperture size and projection distance.
Fig. 10 Integrated imaging analysis of MLA with different sub-aperture sizes and different imaging distances in the same MLA size (11 × 11) by using the designed sub-aperture size and projection distance.
4. Generation of sub-images and simulation of MLA integrated projection
The sub-aperture size p of the sub-channel can directly affect the number of the solved imaging points of the sub-image on the image layer. The diagonal resolvable pixel number of the square sub-image can be determined as:
where is the average RMS spot size of the optimized sub-lens. According to the optimized example sub-lens with f = 1.79 mm, p = 1.4 mm, and (F/#) = 2.1 mm, the limiting situation of the resolvable points in the diagonal line of the full FOV of sub-lens is about 1000 (near to the diffraction limit). However, the factual situation calculated by Eq. (5) is about 350 resolvable points when equals 4 μm. This means that the diagonal resolution of a 1.4-mm sub-image mask needs to be higher than 350 pixels to achieve the highest practical projective resolution.
Once the pixel size in the sub-image mask is close to the average size of the RMS spot, the projection integration imaging resolution could be deduced approximately through Eq. (5). In addition, if the sub-image mask generated with the resolution is equal to or larger than the resolved points number Pix and warps the sub-images with a precision less than 1 pixel, the projection imaging could realize the best integration imaging effect. Therefore, the ideal integrated image quality depends not only on high optical integrated imaging precision, but also on the high image resolution and the image warping predistortion precision.
As shown in Fig. 11(a), after the ray tracing of the maximum field aperture above, the region of shared projection of all sub-channels on the projection plane is determined. A 21 × 21 grid of field points of the real projection height on the projection plane are uniformly sampled within Xproj and Yproj in the x and y directions, respectively. After that, the position of each sub-lens in the target MLA is selected through the offset matrixes addressing of the sub-lens. For each sub-lens in MLA, the chief ray tracing can be performed with the grid of field points according to Fig. 11(a). Figure 11(b) shows the chief ray tracing points generated at the image plane of four sub-channels corresponding to the four top-angle sub-lenses in the planar MLA of the 11 × 11 sub-channel rectangular arrangement. The blue regular arrangement points with 1.4 × 1.4 mm correspond to the initial full-size sub-image. After the offset addressing and ray tracing of the whole MLA are completed, the chief ray tracing points of all sampled grid points are obtained corresponding to the sub-images of all the different sub-channels. The positions of the sub-images are offset; the sub-images are scaled and distorted. Knowing the size of the initial sub-image and its sampling points (the blue points in Fig. 11(b)), the initial sub-images of each channel are offset and warped corresponding to the position of the blue regular points and the distorted ray tracing points, using the RBF image warping method. By the image warping method, all the elastic warping sub-images in MLA are generated corresponding to the tracing points’ locations of all the sub-channels with the warping precision less than 1 pixel. At the same time, the coordinate mapping of the pixel value from the original sub-image to the pre-distorted sub-image can be generated to facilitate the real-time and rapid generation of other projected images. The RBF mapping method uses n polynomial basis functions (the corresponding points are equal to the number of basis functions). Each original regular sampling point Mj' (xj', yj'), (blue points in Fig. 11(b)) and corresponding tracing predistortion points of chief rays Mj (xj, yj) can be described as:
where Ri represents the ith basis function, centered at (xcenter_i, ycenter_i); are the weights of the basis functions; pm (xj, yj) is a fitting polynomial of order m to guarantee the fitting accuracy of degree m; j is an integer from 1 to n; andis a scaling factor. The basis centers are located at the position of the original grid field of view (blue points in Fig. 11(b)), and the characteristic radius is equal to the minimum distance between all the original grid field of view points. In addition, = −2.Fig. 11 (a) Uniformly sampled view points on the projection plane. (b) Blue original grid field of view points and the distorted ray tracing points through chief ray tracing of the sub-lens ML (1,1), ML (1,11), ML (11,1), and ML (11,11) in the 11 × 11 MLA.
Figure 12(a) shows the generated 1.4-mm sub-aperture rectangular array mask of 15.4 × 15.4 mm with a resolution of the sub-image mask of 800 × 800 pixels. It can be seen that the sub-image has some pincushion distortion. Figures 12(b) and 12(c) show the LightTools simulation of the MLA structure and the rendering of the 2D illumination simulation result: a nearly 250 × 250 mm projection size is achieved while the projection distance is 500 mm. The projection area at the same distance (500mm) can achieve a projected area larger than 5.5 times that designed by M. Sieler [6,7]. The illumination uniformity is higher than 95%. Owing to the stop array inserted eliminates channel crosstalk and larger projection FOV, incident light does not need to be perfectly collimated, and the divergence angle could be larger, to make the backlight of MLA uniform and with sufficient brightness, the light source adopted is the 400 lm lambert-type plane extend source with a divergence angle of 120 degrees. The projection simulation result shows that the projection imaging illumination is greater than 100 lux and that the distortion at the edge field is less than 0.3%. In addition, for the hexagon array arrangement MLA structure, Fig. 13 shows the sub-image array mask and the stop array mask corresponding to oblique projection, as well as the structure and illumination simulation results. The sub-image of the predistortion in the oblique projection produces an obvious trapezoid distortion. The illumination simulation result shows that distortion at the edge field is less than 0.5%. Furthermore, although the far end of the oblique projection has greater defocusing and illumination decreasing, better image quality can still be obtained thanks to the large DOF of the MLA imaging.
Fig. 12 (a) Image (left) and aperture (right) array mask. (b) MLA and light source simulation structure. (c) Projection illuminance simulation result.
Fig. 13 (a) Hexagon array arrangement MLA structure. (b) Stop mask, sub-image array mask. (c) Projection simulation in LightTools. (d) Illumination simulation results corresponding to a hexagonal array configuration.
The projector based on MLA could be understood as a beam shaper. This means that a beam passing through the MLA device can be converted into an illumination distribution with a specific pattern. With the development of freeform surface theory, more and more attention has been paid to beam shaping based on freeform surface [26–31]. However, the freeform-based method always initially estimates the light source as a point, collimated beam, or the amplitude and phase distribution of the light source is known [28–31], which too ideal deviated from the actual situation, freeform surfaces machining and assembling errors can also worsen the shaping process, so that the final effect is greatly reduced. The contrast of the pattern obtained by beam shaping with freeform surface is low, because the freeform surface is continuous and smooth. For beam shaping based on multiple micro-apertures MLA we proposed, since the incident beam is divided by multiple micro sub-apertures, no strict parallel collimated beam and the profile machining tolerance of the microlens are required. Because the illumination distribution on the projection plane comes from the superposition of multiple apertures on the projection plane, it has a more stable and uniform illumination distribution. Figure 14 shows a multilayer MLA with a 500 mm projection distance, designed by the method we propose. The projection-imaging group has two layers of quadric surface MLAs, the sub-aperture size is 0.7mm. The method achieves a high resolution (the resolution of the sub-image is 800 × 800 pixels) and high imaging quality. The illumination is about 600 lux with a projection size of 120 × 110 mm and projection distance L = 400 mm while illuminated by a 400 lm lambert-type extend source. The illumination uniformity is higher than 95% and the parttern edges are very sharp.
Although the current implementation of MLA imaging resolution and imaging size is limited compared to some commercial products of ultra-throw projector, the MLA projector is aimed at the application that need high illumination uniformity and minimum volume and weight. There is no doubt that projection imaging based on MLA has higher illumination uniformity and smaller system volume. A brief comparison between single-aperture projection and multi-channel integrated projection based on MLA with the same aperture size are shown in Table 1.
Table 1. Comparison between integrated projector and single-channel projector with the same aperture size
Since the mask used in the simulation are set as full absorption, a great deal of simulated light is lost in the illumination simulation, accounting for more than 75%. The more transmission areas of the pattern, the less light loss. The factual transmittance of the mask would be relatively higher than that in the simulation. In addition, the Fresnel loss of the MLA itself could also produce about 5% loss. In the future, etching will be considered to generate a fixed mask on the plane base of the MLA. In the practical application, we may also choose the clear aperture of the stop mask appropriately, under the premise of weighing image quality and illumination. Sub-channel structure design, mask material, array filling factor, anti-reflection film and other methods to improve light efficiency will be further explored. The trapezoidal pre-distortion of the sub-image corresponding to the sub-channel at the image source is due to the large angle oblique projection, which leads to the attenuation of the remote illumination. We have been exploring more effective solutions to the problem caused by such tilt, including adjustment of the sub-channel design, shielding the near-end part in some sub-images, the filling factor of MLA and the splice structure of sub-lens.
Through the sub-image warping method in this paper, the calculation generation and preparation of sub-image array mask of an ordinary spherical MLA sample provided by the MLA manufacturer are realized. Figure 15 shows the MLA sample and masks used in this experiment, as well as the preliminary experimental results. The sub-image array mask and the stop array mask are film masks with a 2540dpi resolution. The experimental results as shown in Fig. 15(c), a projection size 8 × 10cm with a common collimating flashlight is achieved, which proves the correctness of the method proposed in this paper. Since this experiment is only used to verify the basic correctness of the design method and the sub-image generation method proposed in this paper, the sub-image array is calculated according to the parameters of the MLA given by the manufacturer, but the parameters of MLA are not given in accordance with the optimization method proposed in this paper, and the film mask resolution for validation is relatively low, and the light source is not designed strictly, so the stray light is not well suppressed.
Fig. 15 (a) Components of MLA projector. (b) Overall look of MLA projector. (c) Experimental effect.
5. Conclusion
In this paper, we propose a MLA construction method based on sub-channel design optimization and splicing, and a specific method of analyzing the MLA projection imaging by using the offset addressing of the sub-channels and the chief ray tracing method. Compared with the paraxial analytical method, this method can be applied to any MLA structure. It realizes high precision integrated projection imaging. The design method provides more freedom and possibility in the structure design of MLA and actual imaging projection requirements. A design process example is given such that, by designing and implementing the integrated projection imaging of an aspherical MLA structure, we can achieve the highest accuracy integration projection as well as a good imaging effect. The illumination simulation of the projection imaging verifies the feasibility and effectiveness of the method proposed in this paper. In addition, we discussed the tradeoff between image quality, projection size, resolution, and the MLA structure parameters in MLA projection imaging. The image processing method of the RBF image forward warping is used to generate the array image, whereas the high-precision image warping control is used to accurately correct the distortion caused by the short focus or aspheric surface, as well as the reduction of the size and position offset of the sub-image. The sub-image pixel mapping relationship is generated at the same time. In future development, we will apply a transparent liquid crystal display instead of the sub-image array mask to realize real-time dynamic pattern projection and conduct further discussions and improvements of the remaining problems such as the illumination uniformity in the situation of tilted projection. We will further the research work of projection by using MLA integrated imaging method and explore more efficient and compact MLA projection optical system in the future. Furthermore, the near-field projection imaging problem of MLA will be explored and the corresponding sub-image acquisition method will be further studied.
Funding
National Key Research and Development Program of China (No. 2018YFB0406801); National Natural Science Foundation of China (No. 61822502); Beijing Institute of Technology Research Fund Program for Young Scholars.
Acknowledgments
We would like to thank Synopsys for providing the education license of CODE V and LightTools. Thank optical department of Beijing NED Ltd for their participation and guidance.
Disclosures
The authors declare that there are no conflicts of interest related to this article.
References
1. F. Muñoz, P. Benítez, and J. C. Miñano, “High-order aspherics: the SMS nonimaging design method applied to imaging optics,” Proc. SPIE 7100, 71000K (2008).
2. Z. Zhuang, Y. Chen, F. Yu, and X. Sun, “Field curvature correction method for ultrashort throw ratio projection optics design using an odd polynomial mirror surface,” Appl. Opt. 53(22), E69–E76 (2014). [CrossRef] [PubMed]
3. B. Yang, K. Lu, W. Zhang, and F. Dai, “Design of a free-form lens system for short distance projection,” Proc. SPIE 8128, 81280E (2011). [CrossRef]
4. J. S. Jang, F. Jin, and B. Javidi, “Three-dimensional integral imaging with large depth of focus by use of real and virtual image fields,” Opt. Lett. 28(16), 1421–1423 (2003). [CrossRef] [PubMed]
5. J. Duparré, P. Dannberg, P. Schreiber, A. Bräuer, and A. Tünnermann, “Artificial apposition compound eye fabricated by micro-optics technology,” Appl. Opt. 43(22), 4303–4310 (2004). [CrossRef] [PubMed]
6. M. Sieler, S. Fischer, P. Schreiber, P. Dannberg, and A. Bräuer, “Microoptical array projectors for free-form screen applications,” Opt. Express 21(23), 28702–28709 (2013). [CrossRef] [PubMed]
7. M. Sieler, P. Schreiber, P. Dannberg, A. Bräuer, and A. Tünnermann, “Ultraslim fixed pattern projectors with inherent homogenization of illumination,” Appl. Opt. 51(1), 64–74 (2012). [CrossRef] [PubMed]
8. N. S. Holliman, N. A. Dodgson, G. E. Favalora, and L. Pockett, “Three-Dimensional Displays: A Review and Applications Analysis,” IEEE Trans. Broadcast 57(2), 362–371 (2011). [CrossRef]
9. J. H. Park, G. Baasantseren, N. Kim, G. Park, J. M. Kang, and B. Lee, “View image generation in perspective and orthographic projection geometry based on integral imaging,” Opt. Express 16(12), 8800–8813 (2008). [CrossRef] [PubMed]
10. J. F. Algorri, V. Urruchi, J. M. Sánchez-Pena, and J. M. Otón, “An Autostereoscopic Device for Mobile Applications Based on a Liquid Crystal Microlens Array and an OLED Display,” J. Disp. Technol. 10(9), 713–720 (2014). [CrossRef]
11. J. S. Jang and B. Javidi, “Three-dimensional integral imaging of micro-objects,” Opt. Lett. 29(11), 1230–1232 (2004). [CrossRef] [PubMed]
12. X. Liu and H. Li, “The progress of light field 3-D displays,” Inf. Disp. 30(6), 6–14 (2014). [CrossRef]
13. M. Levoy, R. Ng, A. Adams, M. Footer, and M. Horowitz, “Light field microscopy,” ACM Trans. Graph. 25(3), 924–934 (2006). [CrossRef]
14. M. Levoy, Z. Zhang, and I. McDowall, “Recording and controlling the 4D light field in a microscope using microlens arrays,” J. Microsc. 235(2), 144–162 (2009). [CrossRef] [PubMed]
15. Q. Deng, C. Du, C. Wang, C. Zhou, X. Dong, Y. Liu, and T. Zhou, “Microlens array for stacked laser diode beam collimation,” Proc. SPIE 5636, 666–670 (2005). [CrossRef]
16. M. Ares, S. Royo, and J. Caum, “Shack-Hartmann sensor based on a cylindrical microlens array,” Opt. Lett. 32(7), 769–771 (2007). [CrossRef] [PubMed]
17. R. Voelkel and K. J. Weible, “Laser beam homogenizing: limitations and constraints,” Proc. SPIE 7102, 71020J (2008). [CrossRef]
18. A. Bauer, S. Vo, K. Parkins, F. Rodriguez, O. Cakmakci, and J. P. Rolland, “Computational optical distortion correction using a radial basis function-based mapping method,” Opt. Express 20(14), 14906–14920 (2012). [CrossRef] [PubMed]
19. D. Ruprecht and H. Muller, “Image warping with scattered data interpolation,” IEEE Comput. Graph. Appl. 15(2), 37–43 (1995). [CrossRef]
20. B. Fornberg, E. Larsson, and N. Flyer, “Stable computations with Gaussian radial basis functions,” SIAM J. Sci. Comput. 33(2), 869–892 (2011). [CrossRef]
21. H. Yang, C. K. Chao, M. K. Wei, and C. P. Lin, “High fill-factor microlens array mold insert fabrication using a thermal reflow process,” J. Micromech. Microeng. 14(8), 1197–1204 (2004). [CrossRef]
22. H. Wu, T. W. Odom, and G. M. Whitesides, “Reduction photolithography using microlens arrays: applications in gray scale photolithography,” Anal. Chem. 74(14), 3267–3273 (2002). [CrossRef] [PubMed]
23. M. V. Kunnavakkam, F. M. Houlihan, M. Schlax, J. A. Liddle, P. Kolodner, O. Nalamasu, and J. A. Rogers, “Low-cost, low-loss microlens arrays fabricated by soft-lithography replication process,” Appl. Phys. Lett. 82(8), 1152–1154 (2003). [CrossRef]
24. D. Wu, Q. D. Chen, L. G. Niu, J. Jiao, H. Xia, J. F. Song, and H. B. Sun, “100% fill-factor aspheric microlenses arrays (AMLA) with sub-20 nm precision,” IEEE Photonics Technol. Lett. 21(20), 1535–1537 (2009). [CrossRef]
25. U. D. Zeitner and E. B. Kley, “Advanced lithography for microoptics,” Proc. SPIE 6290, 629009 (2006). [CrossRef]
26. R. Wu, L. Xu, P. Liu, Y. Zhang, Z. Zheng, H. Li, and X. Liu, “Freeform illumination design: a nonlinear boundary problem for the elliptic Monge-Ampére equation,” Opt. Lett. 38(2), 229–231 (2013). [CrossRef] [PubMed]
27. Y. Luo, Z. Feng, Y. Han, and H. Li, “Design of compact and smooth free-form optical system with uniform illuminance for LED source,” Opt. Express 18(9), 9055–9063 (2010). [CrossRef] [PubMed]
28. Z. Feng, B. D. Froese, R. Liang, D. Cheng, and Y. Wang, “Simplified freeform optics design for complicated laser beam shaping,” Appl. Opt. 56(33), 9308–9314 (2017). [CrossRef] [PubMed]
29. Z. Feng, L. Huang, G. Jin, and M. Gong, “Designing double freeform optical surfaces for controlling both irradiance and wavefront,” Opt. Express 21(23), 28693–28701 (2013). [CrossRef] [PubMed]
30. R. Wu, S. Chang, Z. Zheng, L. Zhao, and X. Liu, “Formulating the design of two freeform lens surfaces for point-like light sources,” Opt. Lett. 43(7), 1619–1622 (2018). [CrossRef] [PubMed]
31. R. Wu, P. Liu, Y. Zhang, Z. Zheng, H. Li, and X. Liu, “A mathematical model of the single freeform surface design for collimated beam shaping,” Opt. Express 21(18), 20974–20989 (2013). [CrossRef] [PubMed]
