Continuous zoom compound eye imaging system based on liquid lenses

Yi Zheng; Di Wang; Zhao Jiang; Chao Liu; Chao Liu; Qiong-Hua Wang; Qiong-Hua Wang

doi:10.1364/OE.444188

1. Introduction

In recent years, the compound eyes of insects in nature have aroused great interest of the researchers. The compound eyes which consist of hundreds of small units have many notable characteristics, such as large depth of field, large field of view (FOV), dynamic sensitivity, spatial sensitivity and high integration [1–5]. Researchers have got a lot of inspiration to design artificial compound eye systems from this exquisite natural optical system. So far, bionic artificial compound eyes have been widely used in photogrammetry [6–9], intelligent sensing [10–12], biomedical observation [13,14] and especially optical imaging [15–18]. In the field of artificial compound eye imaging systems, the planar artificial compound eye imaging system is first designed due to its simple structure, but the FOV is difficult to expand [13,19–21]. Thus, researchers mainly focus on the design, manufacture and applications of the curved artificial compound eye imaging systems [22–28]. However, the design and manufacture of the artificial curved compound eyes are still very challenging. What’s more, the lack of adjustability limits the manufacturing yield and applications of the photonic devices [29–33]. Fortunately, attention has been paid to the research of adjustable artificial compound eyes. The focal lengths of the artificial compound eyes can be adjusted through temperature adjustment [34], pH adjustment [35] or hydraulic adjustment [36,37]. Although these works are very innovative, there are still many issues to be solved. Some of the adjustable artificial compound eyes have long response time, low stability and poor practicability. Furthermore, most of the reported adjustable compound eyes do not constitute practical imaging systems. Researchers are more likely to study on adjustable compound eye components and test their imaging quality with commercial cameras. And it is often necessary to move the cameras to capture the image surface formed by adjustable compound eyes, which hinders the systematization and practicability of the adjustable compound eyes.

In this paper, a continuous zoom compound eye imaging system based on liquid lenses is proposed. The main imaging part of the system consists of a liquid compound eye, two liquid lenses and a planar image sensor. Compared with other works on artificial compound eyes, there are three key novelties of our proposed system: 1) The refractive elements of the system are all liquid lenses, which improves the adjustability and the manufacturing yield of the curved artificial compound eye system at a low cost. 2) The system is designed and optimized based on the real performance data of the liquid lenses under normal gravity conditions. We take into account the aspherical effects of the liquid compound eye and liquid lenses, rather than simply regard them as ideal spherical refractive elements. This enables the system to be optimized more reasonably and be controlled more accurately. 3) The system can realize two continuous zoom modes without any mechanical movement of imaging elements suggesting promising applications in a variety of shooting scenes. Through simulations and experiments, we prove that the system can change the imaging paraxial magnification at a fixed working distance or change the working distance at a fixed imaging paraxial magnification. The continuous zoom imaging at a fixed working distance makes it convenient to observe the details or panorama of an object, and the variable-working-distance imaging with a fixed magnification plays a vital role in the measurement of three-dimensional morphology. Besides, the system is designed based on the optical design idea of secondary transfer, which not only gives full play to the FOV advantage of the curved compound eye, but also avoid the movement of the image sensor.

2. Structure, mechanism and fabrication

2.1 Structure and mechanism

The continuous zoom imaging system is shown in Fig. 1. The main refractive elements consist of a liquid compound eye and two liquid lenses. A planar image sensor is used to capture images, and the object is illuminated by a uniform white backlight.

Fig. 1. Continuous zoom imaging system based on a liquid compound eye and two liquid lenses. (a) Schematic diagram of the system structure. (b) Zoom mechanism of the liquid lens. (c) Zoom mechanism of the liquid compound eye.

Download Full Size | PDF

The liquid lenses and the liquid compound eye are filled with transparent hydraulic agent. Besides, the hydraulic agent is sealed in the cavities with transparent membranes. By injecting or drawing out hydraulic agent through the injection channels, the surface shapes of the liquid lenses and liquid compound eye will be deformed, which causes the changes of focal power, as shown in Figs. 1(b)–1(c). Thus, the system can realize continuous zoom imaging by only adjusting the volumes of the liquid injection.

The optical design idea of secondary transfer is adopted in the system, as shown in Fig. 2. When the object is illuminated, a virtual relay image is formed through the liquid compound eye. Then, the virtual relay image is transferred to the image sensor.

Fig. 2. Schematic diagram of the system’s optical path with parameter annotation.

Download Full Size | PDF

The paraxial geometrical optics theory is used to calculate the initial structural parameters of the system at first. The focal power Φ of the system’s central optical path is expressed as:

(1)$$\Phi \textrm{ = }{\Phi _c} + {\Phi _1} + {\Phi _2} - {d_1}{\Phi _c}{\Phi _1} - {d_2}{\Phi _c}{\Phi _2} - {d_2}{\Phi _1}{\Phi _2} + {d_1}{d_2}{\Phi _c}{\Phi _1}{\Phi _2},$$

where Φ_c is the focal power of the liquid compound eye, Φ₁ is the focal power of the liquid lens-1, Φ₂ is the focal power of the liquid lens-2, d₁ is the central distance between the liquid compound eye and the liquid lens-1, and d₂ is the central distance between the liquid lens-1 and the liquid lens-2. When the working distance u varied, we can still adjust Φ_c, Φ₁ and Φ₂ to keep the system in focus with a fixed back working distance l, which can further improve the large depth of field of the liquid compound eye. Besides, when the working distance and back working distance are fixed, we can also adjust Φ_c, Φ₁ and Φ₂ to obtain variable paraxial magnifications. The paraxial magnification is defined as the ratio of the paraxial image height to the object height. When the liquid compound eye and liquid lenses are approximately regarded as a thin lens array and thin lenses respectively, the paraxial magnification β can be expressed as:

(2)$$\beta \textrm{ = }{\beta _c}{\beta _1}{\beta _2},$$

where β_c is the paraxial magnification of the liquid compound eye, β₁ is the paraxial magnification of the liquid lens-1, and β₂ is the paraxial magnification of the liquid lens-2. The β_c, β₁ and β₂ can be expressed as:

(3)$${\beta _c}\textrm{ = (}{\Phi _c}u + 1{)^{ - 1}},$$

(4)$${\beta _1}\textrm{ = }{[{\Phi _1}{({\Phi _c} + {u^{ - 1}})^{ - 1}} - {\Phi _1}{d_1} + 1]^{ - 1}},$$

(5)$${\beta _2}\textrm{ = }{\{ [{({\Phi _1} + \frac{{{\Phi _c}u + 1}}{{u - {d_1} - {d_1}u{\Phi _c}}})^{ - 1}} - {d_2}]{\Phi _2} + 1\} ^{ - 1}},$$

Considering that the liquid compound eye is a multi-aperture and multi-axis optical system, when the inter-ommatidial angle θ becomes larger, it is not appropriate to regard the relay image as a simple plane. When the focused object surface changes, the relay image plane will become a variable spherical or free-formed surface. The field curvature of the two liquid lenses should be designed to fit the relay image as much as possible, which can reduce the mismatch between the relay image and the planar image sensor. The field curvature S_fc of a system can be expressed as [38]:

(6)$${S_{fc}} = {J^2}\sum\limits_{k = 1}^K {\frac{{{\Phi _k}}}{{{n_k}}}} ,$$

where J is the Lagrange invariant of the system, Φ_k is the focal power of each lens, and n_k is the refractive index of each lens. In addition, we also need to match the projection height of the light beam to the sizes of the lenses, which can be controlled by operands in the simulation software.

In order to form a continuous total FOV, the FOV of each sub-eye should overlap. Thus, the following expression needs to be followed:

(7)$${\omega _1}(\Phi )\textrm{ + }{\omega _2}(\Phi ) > f({a_c},u,\theta ) \cdot \theta ,$$

where ω₁ is the half FOV of the central sub-eye, ω₂ is the inside half FOV of the edge sub-eye, and a_c is the aperture of each sub-eye of the liquid compound eye. If the working distance u is much larger than the size of the liquid compound eye, f (a_c, u, θ) is approximate to 1.

According to Eqs. (1)-(7), we can preliminarily design the parameters of the initial structure of the system. Then, we can optimize the system by using optical design software to calculate specific parameters.

2.2 Fabrication and test

The fabrication process of the proposed liquid compound eye and the liquid lens are shown in Figs. 3(a)–3(b), respectively. And the final structure of the system is shown in Fig. 3(c). The outer shells, inner shells, substrate of the liquid compound eye, the clamping lids, and the central shell of the liquid lens are fabricated by 3D printer whose printing accuracy is within 50µm. The material of the elastic transparent membrane is PDMS whose transmittance is above 98%, tensile strength is above 5MPa, refractive index is 1.41 and abbe number is 44.4. The thickness of the membrane is 100µm under the condition of relaxation. Considering safety and convenience, deionized water is selected as the hydraulic agent whose refractive index is 1.33 and abbe number is 33.1.

Fig. 3. (a) Fabrication process of the proposed liquid compound eye. (b) Fabrication process of the proposed liquid lens. (c) Final structure of the system.

Download Full Size | PDF

The cavity of the liquid compound eye consists of two outer shells, two inner shells and a substrate. A piece of elastic transparent membrane is clamped by the outer shells and another piece of elastic transparent membrane is clamped by the inner shells. Then, they are fixed and glued to the substrate. The outer shells of the liquid compound eye have two holes which are used to inject the hydraulic agent and exhaust air. When the hydraulic agent exactly fills the liquid cavity, the hole which is used to exhaust air can be sealed. At the same time, it is also necessary to apply glue into the edge of another hole which is connected to the liquid pump. In the proposed liquid compound eye, the aperture of each sub-eye is ∼7 mm, the edge thickness of each sub-eye is ∼4 mm, the inter-ommatidial angle θ is 8 degrees, and the curvature radius of the outer shell R_c is 71.85 mm.

The cavity of the liquid lens consists of two clamping lids and a central shell. Two pieces of elastic transparent membrane are clamped between the clamping lids and the central shell, respectively. Then, the gaps between the clamping lids and the central shell are sealed. Similar to the cavity of the liquid compound eye, the central shell of the liquid lens also contains two holes. When the hydraulic agent exactly fills the liquid cavity, the hole which is used to exhaust air can be sealed and the edge of another hole which is connected to the liquid pump can be apllied glue. The aperture of the liquid lens is ∼16 mm and the edge thickness of the liquid lens is ∼7 mm.

After the liquid compound eye and liquid lenses are manufactured, they are assembled with a lens barrel to form a whole camera. A spiral port is reserved at the end of the lens barrel for connection with the image sensor.

Before further design and imaging experiments, the optical properties of the liquid compound eye and the liquid lens are tested. After the liquid compound eye and the liquid lens are exactly filled with the hydraulic agent, we clearly observe that the surface changes with the increase of additional injection volume, as shown in Figs. 4(a)–4(h).

Fig. 4. Demos of the liquid compound eye and the liquid lens. (a) Liquid compound eye without hydraulic agent injection. (b) Liquid compound eye exactly filled with hydraulic agent. (c) Liquid compound eye injected with additional 130µL of hydraulic agent. (d) Liquid compound eye injected with additional 450µL of hydraulic agent. (e) Liquid lens without hydraulic agent injection. (f) Liquid lens exactly filled with hydraulic agent. (g) Liquid lens injected with additional 360µL of hydraulic agent. (h) Liquid lens injected with additional 572µL of hydraulic agent.

Download Full Size | PDF

Then, the paraxial focal lengths of the liquid compound eye and the liquid lens are measured. The relationships between their focal lengths and additional injection volumes are shown in Figs. 5(a)–5(b), respectively. The focal length of the liquid compound eye ranges from ∼22.5 mm to ∼10.7 mm when the additional injection volume ranges from ∼130µL to ∼450µL. And the focal length of the liquid lens ranges from ∼41.6mm to ∼26.9mm when the additional injection volume ranges from ∼360µL to ∼572µL.

Fig. 5. (a) Relationship between the focal lengths of the liquid compound eye and the additional injection volumes. (b) Relationship between the focal lengths of the liquid lens and the additional injection volumes.

Download Full Size | PDF

Next, we establish the mathematical model of the surfaces of the liquid compound eye and the liquid lens and calculate the surface parameters according to the experimental data. That is to say, in addition to the relationship between the paraxial focal lengths and additional injection volumes, the relationship between the surface parameters and additional injection volumes should also be obtained. The relationships will be used in simulations and optimizations.

If the surfaces of the liquid lens and the liquid compound eye were ideally spherical, we could get the following equation:

(8)$$V = 2m\pi {(\frac{1}{c} - \sqrt {\frac{1}{{{c^2}}} - \frac{{{a^2}}}{4}} )^2}[\frac{1}{c} - \frac{1}{3}(\frac{1}{c} - \sqrt {\frac{1}{{{c^2}}} - \frac{{{a^2}}}{4}} )],$$

where V is the additional injection volume, c is the vertex curvature, a is the aperture diameter, and m is the number of channels. This might be reasonable under small-scale conditions [39]. And the focal length f’ can be approximately expressed as [38]:

(9)$$f^{\prime} \approx \frac{1}{{2c(n - 1)}},$$

But we note that there is obvious deviation between the experimental fitted curve and the ideal spherical fitted curve. Although there might be errors in the measurement of the focal lengths, the obvious deviation exists in several repeated experiments. So, we recognize that the influence of gravity cannot be ignored at this scale. Studies have shown that the deviation is directly proportional to the fifth power of the aperture [40]. In fact, the deformation profile of the surface should be expressed as:

(10)$$z(R) = \frac{{c{R^2}}}{{1 + \sqrt {1 - (1 + K){c^2}{R^2}} }} + \sum\limits_{i = 2}^\infty {{\alpha _i}} {R^{2i}},$$

where z(R) is height loss of the surface, R is the horizontal distance from center, K is the conic coefficient and α_i are the higher aspheric coefficients. The mathematical model of the surface is shown in Fig. 6. Thus, the relationship between the focal length and the additional injection volume can be expressed as:

(11)$$V = 2m[\frac{{\pi {a^2} \cdot z(\frac{a}{2})}}{4} - \int_0^{\frac{a}{2}} {2\pi R \cdot z} (R)dR],$$

Fig. 6. Mathematical model of the surface.

Download Full Size | PDF

In Eq. (10), c, K and α_i will change with the variety of the additional injection volume V. And the surface shape is mostly affected by the conic coefficient and the vertex curvature. Thus, in order to calculate and simulate easily, we approximately regard α_i as 0 in this paper. Under this approximation, the actual face shape and the simulated face shape can also be basically consistent [41].

Since the conic coefficient corresponds to the vertex curvature one by one, we can get the vertex curvature by measuring the focal length, and then get the relationship between the vertex curvature and the conic coefficient, as shown in Figs. 7(a)–7(b). It can be observed that the conic coefficient decreases with the increase of the vertex curvature. The standard exponential function is used to fit the experimental data. The function can be expressed as:

(12)$$K = A{e^{ - tc}},$$

where A and t are constants. In Fig. 7(a), A is equal to 26.153 and t is equal to 15.768. In Fig. 7(b), A is equal to 55.579 and t is equal to 53.271.

Fig. 7. (a) Relationship between the vertex curvature of the liquid compound eye and the conic coefficient. (b) Relationship between the vertex curvature of the liquid lens and the conic coefficient.

Download Full Size | PDF

Based on the previous test, we can control the conic coefficients of the surfaces by writing a macro in the simulation software. Thus, we can simulate and optimize the imaging quality of the system more accurately. It will also make the control of the injection volumes more reliable.

3. Simulation and experiment

3.1 Simulation

We use the optical simulation software OpticStudio for simulations and optimizations after calculating the initial structure. We use biconvex lens array to simulate the liquid compound eye and use biconvex lens to simulate the liquid lens, as shown in Fig. 8. The conic coefficients K of surfaces are set to variables related to the vertex curvatures c according to Eq. (12). It can be observed that there is overlap between the FOV of each sub-eye. The size of the image sensor is 2/3”, and the total length of the system is ∼120mm.

Fig. 8. Simulation model of the system’s optical path by the optical simulation software

Download Full Size | PDF

We simulate two zoom modes totally. In the first mode, we simulate continuous zoom imaging at variable paraxial magnifications from 0.021 to 0.041. The working distance is set to 240mm and the back working distance is set to 14.7mm. We selecte modulation transfer function (MTF) as a standard of the image evaluation. The system model, MTF of the central optical channel, MTF of the edge optical channel at the paraxial magnifications of 0.041, 0.031 and 0.021 are shown in Figs. 9(a)–9(c). Simulations show that when the values of MTF is greater than ∼0.1, the spatial frequencies of the central optical channel’s central FOV can reach ∼30lp/mm, 21lp/mm, and 8lp/mm, respectively. And the spatial frequencies of the edge channel’s central FOV can reach ∼14lp/mm, 17lp/mm, and 9lp/mm, respectively.

Fig. 9. System model, MTF of the central optical channel, MTF of the edge optical channel at the paraxial magnifications of (a) 0.041, (b) 0.031 and (c) 0.021.

Download Full Size | PDF

In the second mode, we simulate continuous zoom imaging at variable working distances from 200mm to 300mm. The magnification is set to 0.03 and the back working distance is set to 14.7mm. The system model, MTF of the central optical channel, MTF of the edge optical channel at the working distances of 200mm, 240mm and 300mm are shown in Figs. 10(a)–10(c). Simulations show that when the value of MTF is greater than ∼0.1, the spatial frequencies of the central optical channel’s central FOV can reach ∼11lp/mm, 23lp/mm, and 27lp/mm, respectively. And the spatial frequencies of the edge optical channel’s central FOV can reach ∼9lp/mm, 15lp/mm, and 16lp/mm, respectively.

Fig. 10. System model, MTF of the central optical channel, MTF of the edge optical channel at the working distances of (a) 200mm, (b) 240mm and (c) 300mm.

Download Full Size | PDF

In order to prove the continuous zoom characteristics of the system, several groups of data are simulated and provided for the two zoom modes. The relationships between the paraxial magnifications and the curvatures of the refractive elements are shown in Fig. 11(a). The relationships between the working distances and the curvatures of the refractive elements are shown in Fig. 11(b).

Fig. 11. (a) Relationships between the paraxial magnifications and the curvatures of the liquid compound eye and the liquid lenses. (b) Relationships between the working distances and the curvatures of the liquid compound eye and the liquid lenses.

Download Full Size | PDF

According to Fig. 5, Fig. 11 and Eq. (9), the values of the corresponding injection volumes in the controller of the liquid pumps can be calculated and used in the experiments. Therefore, we can input the values of the corresponding injection volumes in the controller of the liquid pumps to control the curvatures of the refractive elements and realize continuous zoom imaging.

3.2 Experiment

The experiment setups are shown in Fig. 12. The proposed system is connected with three single-channel liquid pumps made by BIOTAOR with the type of RSP01-BD. The flow speed of the liquid pump can range from 0.112µL/min to 84.870 mL/min. The image sensor is made by HIKROBOT with the type of MV-CA050-12GC. The resolution of the image sensor is 2448×2048, the frame rate is 11.57fps and the pixel size is 3.45µm×3.45µm. The system is used to photograph a board marked with letters “A-Z” and numbers “1-31” illuminated by a uniform white light source. The size of the board is 160mm×160mm. The experiment results are shown in Figs. 13(a)–13(f).

Fig. 12. Schematic diagram of the experiment setups.

Download Full Size | PDF

Fig. 13. Experiment results at different magnifications or working distances. (a) Magnification: ∼0.019, working distance: 240mm. (b) Magnification: ∼0.027, working distance: 240mm. (c) Magnification: ∼0.037, working distance: 240mm. (d) Magnification: ∼0.027, working distance: 200mm. (e) Magnification: ∼0.027, working distance: 240mm. (f) Magnification: ∼0.027, working distance: 300mm.

Download Full Size | PDF

The system is controlled based on the simulation data as well as the previous test of the relationships between the curvatures and injection volumes. It can be measured that the paraxial magnification of the system can vary from ∼0.019 to ∼0.037 at a fixed working distance and a fixed back working distance. The zoom ratio can reach ∼2. Besides, when the working distance varies from 200mm to 300mm, the paraxial magnification can be fixed to ∼0.027. Although there are some deviations between the actual zoom range and the simulation data, the experimental results are basically consistent with the theory. The causes of deviation will be analyzed in the following discussion. In fact, the zoom range can be further expanded but there is a trade-off between imaging quality and zoom range.

In the experiment, obvious changes between adjacent frame images can be observed, which means the control speed is faster than the acquisition speed of the image sensor. Actually, the response time of the liquid pump is within 1 ms. However, the response time of the system is also determined by the properties of the hydraulic agent, such as viscosity. Although the control speed is fast, the pressure conduction of liquid might be delayed. And it is measured roughly that the response time of the total system is within 260 ms when the paraxial magnification of the system varies from ∼0.019 to ∼0.037.

Besides, it could be observed that there is overlap between the FOV of each sub-image. In order to prove that the FOV of the system can be extended, we try to correct and stitch the sub-images. After separating the sub-images, an improved algorithm of speeded up robust features (SURF) [42] is used to calculate the dislocation value of sub-images and compensate the vignetting. Then, the pixels of the sub-images are mapped to the complete pixel matrix. The mapping relationship can be expressed as:

(13)$${\textrm{P}_{(x^{\prime},y^{\prime})}} = \sum\limits_{i = 1}^m {{{\mathbf A}_{({x_i},{y_i})}}\cdot \textbf{F}({{\mathbf F}^{ - 1}}({\textrm{P}_{({x_i},{y_i})}})\cdot {{\mathbf S}_{({x_i},{y_i})}})} \cdot {{\mathbf B}_{({x_i},{y_i})}},$$

where P_(x’,y’) is the complete pixel matrix, P_(x_i_,y_i₎ is the sub-image pixel matrix, F is the response function matrix of the system, S_(x_i_,y_i₎ is the vignetting compensation coefficient matrix, A_(x_i_,y_i₎ and B_(x_i_,y_i₎ are the complementary pixel matrixes calculated from the dislocation values.

The reconstructed image of the object is shown in Fig. 14(b). Compared with the photographed object shown in Fig. 14(a), it can be seen that the reconstructed image includes all the main information of the object. The total FOV is more than 45 degrees. A certain number of overlap pixels are nedded to calculate the relative position of the sub-images, but excessive proportion of overlap will lead to the decline of image quality and the waste of photosensitive area. Thus, the overlap proportion is expected to be between ∼15% and ∼30% when designing the system.

Fig. 14. (a) Photographed object. (b) Reconstructed image of the photographed object.

Download Full Size | PDF

4. Discussion

The gravity effect on the liquid lenses and the liquid compound eye needs to be considered, otherwise there will be great differences between simulations and experiments, especially when the aperture sizes are larger than 7mm. In the design and simulations, we have taken the aspherical effect of the surfaces into account. However, if conditions permit, the actual measurement of surface shape can be carried out to obtain more accurate fitting data. In addition, the uniformity of the membranes may affect the shape of the surfaces [41]. In the manufacturing process, the membranes should be evenly spread and bonded on the components to avoid a certain deviation. Besides, the mismatch between the liquid compound eye and the liquid optical relay system, the machining accuracy and alignment accuracy will also affect the imaging quality.

Considering that each liquid optical elements will only provide a degree of freedom for optimization when the distances between the elements are fixed, it is necessary to use more than one liquid lens in the optical relay system to keep the relationship of the object-image conjugate and ensure the possibility of optimization in the two zoom modes. More degrees of freedom for optimization will be helpful to improve the quality of imaging. In the simulation and optimization, only the aberrations of the central optical channel’s central FOVs are corrected well, because the degrees of freedom for optimization of a liquid compound eye and two liquid lenses are limited. In the experiment, we only use deionized water as the refractive material, which is not enough for the correction of the chromatic aberration. However, if we use more liquid and solid lenses, better imaging quality can be acquired, as shown in Fig. 15. The original image in Fig. 15 are captured by the imaging system consisting of a liquid compound eye, a liquid lens and a commercial camera. The commercial camera which consists of several solid lenses is made by Shenzhen Qiyun Photoelectric Co., Ltd. with the type of 0550-2MP. The reconstructed image is obtained by correcting and stitching the sub-images through the image processing program. Besides, higher performance image processing algorithms can also improve the image quality. If a larger FOV and magnification variation range are needed, more optical channels and cycles should be set. In addition, the inter-ommatidial angles can likewise be adjusted. It requires a trade-off between system complexity, manufacturing costs and the desired imaging properties. In fact, the sub-images contain the three-dimensional information of the object, which can be used for spatial measurement. It will be the focus of our future work.

Fig. 15. (a) Photographed object [43]. (b) Original image captured by a more complex compound eye imaging system. (c) Reconstructed image of the photographed object.

Download Full Size | PDF

5. Conclusion

In this paper, we propose a continuous zoom compound eye imaging system. The main imaging part of the system consists of a liquid compound eye, two liquid lenses and a planar image sensor. By adjusting the liquid injection volumes of the liquid compound eye and liquid lenses, the system can realize continuous zoom imaging without any mechanical movement of imaging components. The simulation and experiment results show that the system can change the imaging magnification at a fixed working distance and realize variable-working-distance imaging at a fixed magnification. The zoom ration can reach ∼2 and the variation range of the working distance is more than 100mm. The system has strong adjustability, adaptability and extensibility. It can be expected to be used in panoramic photography, 3D measurement, dynamic measurement and so on.

Funding

National Natural Science Foundation of China (61927809, 62175006).

Disclosures

The authors declare that there are no conflicts of interest related to this article. The characters in Fig. 15 are property of DreamWorks Animation.

Data availability

Data underlying the results presented in this paper are not publicly available at this time but may be obtained from the authors upon reasonable request.

References

1. Y. M. Song, Y. Z. Xie, V. Malyarchuk, J. L. Xiao, I. Jung, K. J. Choi, Z. J. Liu, H. Park, C. F. Lu, R. H. Kim, R. Li, K. B. Crozier, Y. G. Huang, and J. A. Rogers, “Digital cameras with designs inspired by the arthropod eye,” Nature 497(7447), 95–99 (2013). [CrossRef]

2. K. H. Jeong, J. Kim, and L. P. Lee, “Biologically inspired artificial compound eyes,” Science 312(5773), 557–561 (2006). [CrossRef]

3. Y. Cheng, J. Cao, Y. K. Zhang, and Q. Hao, “Review of state-of-the-art artificial compound eye imaging systems,” Bioinspir. Biomim. 14(3), 031002 (2019). [CrossRef]

4. L. C. Kogos, Y. Z. Li, J. N. Liu, Y. Y. Li, L. Tian, and R. Paiella, “Plasmonic ommatidia for lensless compound-eye vision,” Nat. Commun. 11(1), 1637 (2020). [CrossRef]

5. L. Zhu, Y. L. Zhang, and H. B. Sun, “Miniaturising artificial compound eyes based on advanced micronanofabrication techniques,” Light Adv. Manuf. 1(1), 83–99 (2019). [CrossRef]

6. H. J. Jian, J. Z. He, X. Y. Jin, X. C. Chen, and K. Y. Wang, “Automatic geometric calibration and three-dimensional detecting with an artificial compound eye,” Appl. Opt. 56(5), 1296–1301 (2017). [CrossRef]

7. W. B. Lee and H. N. Lee, “Depth-estimation-enabled compound eyes,” Opt. Commun. 412, 178–185 (2018). [CrossRef]

8. M. C. Ma, F. Guo, Z. L. Cao, and K. Y. Wang, “Development of an artificial compound eye system for three-dimensional object detection,” Appl. Opt. 53(6), 1166–1172 (2014). [CrossRef]

9. C. Y. Shi, Y. Y. Wang, C. Y. Liu, T. S. Wang, H. X. Zhang, W. X. Liao, Z. J. Xu, and W. X. Yu, “SCECam: a spherical compound eye camera for fast location and recognition of objects at a large field of view,” Opt. Express 25(26), 32333–32345 (2017). [CrossRef]

10. L. Song, H. Zhu, Y. L. Zheng, M. R. Zhao, C. A. T. Tee, and F. Z. Fang, “Bionic compound eye-inspired high spatial and sensitive tactile sensor,” IEEE Trans. Instrum. Meas. 70, 1–8 (2021). [CrossRef]

11. T. C. Yang, Y. H. Liu, Q. Q. Mu, M. Zhu, D. L. Pu, L. S. Chen, and W. B. Huang, “Compact compound-eye imaging module based on the phase diffractive microlens array for biometric fingerprint capturing,” Opt. Express 27(5), 7513–7522 (2019). [CrossRef]

12. H. Dahmen and H. A. Mallot, “Odometry for ground moving agents by optic flow recorded with optical mouse chips,” Sensors 14(11), 21045–21064 (2014). [CrossRef]

13. J. Tanida, H. Mima, K. Kagawa, C. Ogata, and M. Umeda, “Application of a compound imaging system to odontotherapy,” Opt. Rev. 22(2), 322–328 (2015). [CrossRef]

14. J. T. Fan, J. L. Suo, J. M. Wu, H. Xie, Y. B. Shen, F. Chen, G. J. Wang, L. C. Cao, G. F. Jin, Q. S. He, T. F. Li, G. M. Luan, L. J. Kong, Z. R. Zheng, and Q. H. Dai, “Video-rate imaging of biological dynamics at centimetre scale and micrometre resolution,” Nat. Photonics 13(11), 809–816 (2019). [CrossRef]

15. D. J. Brady, M. E. Gehm, R. A. Stack, D. L. Marks, D. S. Kittle, D. R. Golish, E. M. Vera, and S. D. Feller, “Multiscale gigapixel photography,” Nature 486(7403), 386–389 (2012). [CrossRef]

16. W. W. Lu, S. Y. Chen, Y. P. Xiong, and J. F. Liu, “A single ball lens-based hybrid biomimetic fish eye/compound eye imaging system,” Opt. Commun. 480, 126458 (2021). [CrossRef]

17. M. Miyata, M. Nakajima, and T. Hashimoto, “Compound-eye metasurface optics enabling a high-sensitivity, ultra-thin polarization camera,” Opt. Express 28(7), 9996–10014 (2020). [CrossRef]

18. A. Cao, L. F. Shi, Q. L. Deng, H. Pang, M. Zhang, and C. L. Du, “Structural design and image processing of a spherical artificial compound eye,” Optik 126(21), 3099–3103 (2015). [CrossRef]

19. H. X. Deng, X. C. Gao, M. C. Ma, Y. Y. Li, H. Li, J. Zhang, and X. Zhong, “Catadioptric planar compound eye with large field of view,” Opt. Express 26(10), 12455–12468 (2018). [CrossRef]

20. A. Brückner, J. Duparré, P. Dannberg, A. Bräuer, and A. Tünnermann, “Artificial neural superposition eye,” Opt. Express 15(19), 11922–11933 (2007). [CrossRef]

21. H. Kawano, S. Takushima, Y. Toyoda, M. Sugano, and T. Matsuzawa, “Compound eye image scanner optimally arranged for image combining and suppressing axial chromatic aberration by wavefront coding,” Opt. Eng. 60(04), 043103 (2021). [CrossRef]

22. K. K. Tao, G. G. Lian, Y. S. Liu, H. M. Xing, Y. Xing, X. D. Su, X. Feng, and Y. H. Wu, “Design and integration of the single-lens curved multi-focusing compound eye camera,” Micromachines 12(3), 331 (2021). [CrossRef]

23. J. Li, W. J. Wang, X. S. Mei, D. X. Hou, A. F. Pan, B. Liu, and J. L. Cui, “Fabrication of artificial compound eye with controllable field of view and improved imaging,” ACS Appl. Mater. Interfaces 12(7), 8870–8878 (2020). [CrossRef]

24. F. Liu, H. Bian, F. Zhang, Q. Yang, C. Shan, M. J. Li, X. Hou, and F. Chen, “IR artificial compound eye,” Adv. Optical Mater. 8(4), 1901767 (2020). [CrossRef]

25. X. Q. Liu, S. N. Yang, L. Yu, Q. D. Chen, Y. L. Zhang, and H. B. Sun, “Rapid engraving of artificial compound eyes from curved sapphire substrate,” Adv. Funct. Mater. 29(18), 1900037 (2019). [CrossRef]

26. F. L. Liu, X. L. Diao, L. Li, and Y. P. Hao, “Calibration technique of a curved zoom compound eye imaging system,” Micromachines 10(11), 776 (2019). [CrossRef]

27. H. Zhang, L. Li, D. L. McCray, S. Scheiding, N. J. Naples, A. Gebhardt, S. Risse, R. Eberhardt, A. Tünnermann, and A. Y. Yi, “Development of a low cost high precision three-layer 3D artificial compound eye,” Opt. Express 21(19), 22232–22245 (2013). [CrossRef]

28. Y. Y. Wang, C. Y. Shi, H. R. Xu, Y. J. Zhang, and W. X. Yu, “A compact bionic compound eye camera for imaging in a large field of view,” Opt. Laser Technol. 135, 106705 (2021). [CrossRef]

29. D. Wang, C. Liu, C. Shen, Y. Xing, and Q. H. Wang, “Holographic capture and projection system of real object based on tunable zoom lens,” PhotoniX 1(1), 6 (2020). [CrossRef]

30. B. Dai, Z. Jiao, L. L. Zheng, H. Bachman, Y. F. Fu, X. J. Wan, Y. L. Zhang, Y. Huang, X. D. Han, C. L. Zhao, T. J. Huang, S. L. Zhuang, and D. W. Zhang, “Colour compound lenses for a portable fluorescence microscope,” Light: Sci. Appl. 8(1), 75 (2019). [CrossRef]

31. C. Liu, D. Wang, and Q. H. Wang, “Holographic display system with adjustable viewing angle based on multi-focus optofluidic lens,” Opt. Express 27(13), 18210–18221 (2019). [CrossRef]

32. Z. Wang, Y. Z. Liu, C. Y. Gong, Z. Y. Yuan, L. Shen, P. X. Chang, K. Liu, T. H. Xu, J. F. Jiang, Y. C. Chen, and T. G. Liu, “Liquid crystal-amplified optofluidic biosensor for ultra-highly sensitive and stable protein assay,” PhotoniX 2(1), 18 (2021). [CrossRef]

33. T. Zhan, J. H. Xiong, J. Y. Zou, and S. T. Wu, “Multifocal displays: review and prospect,” PhotoniX 1(1), 10 (2020). [CrossRef]

34. D. F. Zhu, C. H. Li, X. F. Zeng, and H. R. Jiang, “Tunable-focus microlens arrays on curved surfaces,” Appl. Phys. Lett. 96(8), 081111 (2010). [CrossRef]

35. Z. C. Ma, X. Y. Hu, Y. L. Zhang, X. Q. Liu, Z. S. Hou, L. G. Niu, L. Zhu, B. Han, Q. D. Chen, and H. B. Sun, “Smart compound eyes enable tunable imaging,” Adv. Funct. Mater. 29(38), 1903340 (2019). [CrossRef]

36. K. Wei, H. S. Zeng, and Y. Zhao, “Insect–Human Hybrid Eye (IHHE): an adaptive optofluidic lens combining the structural characteristics of insect and human eyes,” Lab Chip 14(18), 3594–3602 (2014). [CrossRef]

37. J. J. Cao, Z. S. Hou, Z. N. Tian, J. G. Hua, Y. L. Zhang, and Q. D. Chen, “Bioinspired zoom compound eyes enable variable-focus imaging,” ACS Appl. Mater. Interfaces 12(9), 10107–10117 (2020). [CrossRef]

38. H. Gross, H. Ziigge, M. Peschka, and F. Blechinger, “Handbook of optical systems,” (WILEY-VCH Verlag GmbH & Co. KGaA, 2007, vol. 3, pp. 41–45).

39. A. Mikš and P. Pokorný, “Hybrid liquid-membrane lenses made by one fixed and one or two active optical components,” J. Opt. Soc. Am. A 38(1), 99–107 (2021). [CrossRef]

40. S. T. Choi, B. S. Son, G. W. Seo, S. Y. Park, and K. S. Lee, “Opto-mechanical analysis of nonlinear elastomer membrane deformation under hydraulic pressure for variable-focus liquid-filled microlenses,” Opt. Express 22(5), 6133–6146 (2014). [CrossRef]

41. Z. Q. Ding, C. H. Wang, Z. X. Hu, Z. G. Cao, Z. Zhou, X. Y. Chen, H. Y. Chen, and W. Qiao, “Surface profiling of an aspherical liquid lens with a varied thickness membrane,” Opt. Express 25(4), 3122–3132 (2017). [CrossRef]

42. H. Bay, T. Tuyteplaars, and L. V. Gool, “SURF: speeded up robust features,” in Proc. the 9th European Conference on Computer Version (2006, pp. 404–417).

43. N. Taylor and M. Watts, Kung Fu Panda 3 (Scholastic Inc., 2016).

Continuous zoom compound eye imaging system based on liquid lenses

Abstract

1. Introduction

2. Structure, mechanism and fabrication

2.1 Structure and mechanism

2.2 Fabrication and test

3. Simulation and experiment

3.1 Simulation

3.2 Experiment

4. Discussion

5. Conclusion

Funding

Disclosures

Data availability

References

Data availability

Cited By

Figures (15)

Equations (13)

Optics Express