Expand this Topic clickable element to expand a topic
Skip to content
Optica Publishing Group
Journal Home About Issues in Progress Current Issue All Issues Early Posting Feature Issues

Simple and fast field curvature measurement by depth from defocus using electrowetting liquid lens

Gyu Suk Jung and Yong Hyub Won

Open Access Open Access

Abstract

Field curvature, also called “Petzval field curvature,” is a defect in the lens in which the object of a flat plane is not focused on the image surface. Field curvature measurement is important in lens performance tests. This paper presents an electrowetting liquid lens based on the depth from defocus method for measuring field curvature. This method uses only a pair of defocused images for a patterned flat object, which are captured using the focus tuning function of the electrowetting liquid lens. Image processing for calculating field curvature is carried out using MATLAB. The results of the measurement experiment demonstrated high accuracy of the 12 µm root mean square error between the captured image surface and fitted curved image surface. There is no need for complex equipment such as lasers, microscopes, and telecentric systems. Because this system has no mechanical movement for focus tuning, it is simple and shows fast measurement time compared with other conventional methods.

© 2020 Optical Society of America under the terms of the OSA Open Access Publishing Agreement

1. INTRODUCTION

Demand for high-resolution cameras has been increasing, as cognitive research on AI and robots is underway with the fourth industrial revolution. Recently, due to the development of the sensor industry, research on sensors with 1 billion pixels is progressing. Mobile devices such as smartphones and vision robots require higher-resolution cameras. In addition, the need for high-resolution images remains in biomedical fields such as microscopes, endoscopes, and surveillance systems such as CCTV and black boxes for automobiles. As such, image quality remains important to all photographic systems.

However, even if an image sensor is high resolution, optical aberration can degrade the image quality. Field curvature, also called “Petzval field curvature,” is a defect in the lens in which the object of a plane is focused not on the flat image surface but on the curved image surface [1]. Figure 1 represents a ray diagram of the Petzval field curvature. If the center of an image is sharp, the field boundary appears blurred and vice versa. Petzval curvature is given as $1/{r_k}={-}{n_k}\sum {K_j}/({{n_{j - 1}}{n_j}})$, where ${r_k}$ is the Petzval radius, ${K_j}$ is the surface power of the ${j}$th surface, and ${n_j}$ is the refractive index of the ${j}$th material. Thus, Petzval curvature depends on only radius of curvature of the lens and its refractive index. In visual observing, the field curvature is not a severe problem, thanks to the function of the eye to refocus. However, in a capturing system, such as a camera, the field curvature deteriorates image quality. The field curvature problem is especially severe with wide-angle cameras. Much research has been conducted to solve the field curvature aberration problem [2–4].

In order to correct the field curvature, it is required to accurately measure the field curvature of the lens. In addition, when a lens is manufactured, it is necessary to measure the field curvature to check whether the lens is well-formed. Generally, field curvature is measured by capturing the size of a magnified laser beam spot [5–7]. This method requires a laser beam, which increases the cost and complexity of the measuring system. There is also a method using an optical microscope and conventional resolution test patterns [6]. This method requires a fine focus increment of 0.4 µm to find the best focusing point. Similar to this method, there is depth from focus-based (DfF) method using a common CCD camera [7]. These methods also require dozens of images to find the best focusing point. It takes several seconds to acquire whole defocused images because those are taken by mechanically changing the focal plane. During the time of capturing the whole images, even a small change of the camera/object position and DC component-like illumination can cause depth error [8]. Also, those methods obtain multiple defocused images by adjusting the distance between the lens and the sensor. Therefore, the introduction of a telecentric system is essential to suppress the variation in the magnification caused by the change of distance between sensors and lenses (max shift is about 20 pixels) [9]. As a result, the use of a telecentric system induces the change of system design according to the focal length of the target lens and requires fine alignment of the whole system.

 figure: Fig. 1.

Fig. 1. Petzval field curvature aberration. (a) Focused in center. (b) Focused in field boundary.

Download Full Size | PDF

In this paper, we introduce a simple and quick method to measure the Petzval field curvature of a lens using a liquid lens. This method of using the liquid lens requires only two defocused images by fast focus tuning (${\sim}{7.5}\;{\rm ms}$) [10]. In addition, it only requires a combination of the target lens and liquid lens regardless of the type of target lens, thus simplifying the system design. Figure 2 shows a schematic diagram of the proposed method of measuring Petzval field curvature. We place an electrowetting liquid lens directly behind the lens of which we want to measure the field curvature. An electrowetting liquid lens is a lens that changes focal length quickly by changing the contact angle of a conductive liquid by applied voltage. No complex equipment such as lasers, telecentric systems, or microscopes are required for measurements, only liquid lenses and image sensors. Further, only two defocus images are used to measure the field curvature. Therefore, this method is compact and simple. The rest of this paper is organized as follows. Section 2 explains the principle of the proposed method. Section 3 shows the measuring result of our method. Section 4 summarizes this paper.

 figure: Fig. 2.

Fig. 2. Schematic diagram of the proposed method.

Download Full Size | PDF

 figure: Fig. 3.

Fig. 3. Image formation of DfD optic model using liquid lens.

Download Full Size | PDF

2. FIELD CURVATURE MEASUREMENT METHOD

In this section, we explain the principle of our approach. We exploit the DfD algorithm based on rational filters to measure the field curvature [8,11,12]. This algorithm is good for processing time and accuracy. The DfD optic model is shown as Fig. 3, where $u$ is the distance from the lens to the object, $a$ is radius of lens aperture, ${R_1}$ is the defocus radius when the lens diopter is maximum (${D_{\max}}$), ${R_2}$ is the defocus radius when the lens diopter is minimum (${D_{\min}}$), and $v$ is the distance between the lens and image sensor. Let “$D$” be the diopter of the lens that causes the object to be focused on the image sensor. Then, when $\nabla D$ is defined by ($({D_{\max}}-{D_{\min}})/2$, $D$, ${R_1},$ and ${R_2}$ are given by

$$\begin{split} D&=D_{\max }-(1-\alpha) \nabla D=D_{\min }+(1+\alpha) \nabla D \\ R_{1}&=a v(1-\alpha) \nabla D \\ R_{2}&=a v(1+\alpha) \nabla D ,\end{split}$$
where ${\alpha}$ is the normalized depth value and ranges ${-}{1}$ to 1. If the defocus distribution model is adopted as the pill-box, the defocus function is
$$h(x, y)=\frac{1}{\pi {R}^{2}} \prod\left(\frac{1}{2 R} \sqrt{x^{2}+y^{2}}\right),$$
where $\prod (r)$ is the rectangular function, $x$ and $y$ are pixel positions, and $R$ is the defocus radius. Further, the Fourier transform of this function is
$$H(p, q)=\frac{1}{\pi {R} \sqrt{{p}^{2}+{q}^{2}}} j_{1}\left(2 \pi {R} \sqrt{{p}^{2}+{q}^{2}}\right),$$
where ${j_1}$ is the first-order Bessel function, and $p$ and $q$ are the spatial frequencies. The normalized ratio is defined as
$$\frac{M(p, q)}{P(p, q)}=\frac{H_{1}(p, q)-H_{2}(p, q)}{H_{1}(p, q)+H_{2}(p, q)}.$$
Further, this normalized ratio is a monotonic function of ${ \alpha}$, provided the radial frequency is under maximum frequency ${1.22}/({2}av \times {2} \nabla D)$ [12]. Thus, the monotonic function of this normalized ratio can be fit by any polynomial model and the normalized value ${ \alpha}$ is easily calculated. This model can be written as
$$\frac{M\!\left(f_{r}\right)}{P\!\left(f_{r}\right)}=\frac{G_{P 1}\!\left(f_{r}\right)}{G_{M 1}\!\left(f_{r}\right)} \alpha+\frac{G_{P 2}\!\left(f_{r}\right)}{G_{M 1}\!\left(f_{r}\right)} \alpha^{3}.$$
After obtaining the value ${\alpha}$ from Eq. (1), the $D$ value can be found for all ${ \alpha}$ of the object. Then, the depth map on the object side is calculated by the simple lens equation $u = {1/}(D - {1/}v)$. However, in order to obtain the Petzval radius, we need to know the depth map of the image side. We can get the depth map of the image side via simple lens equation:
$$ u_{i}(x, y)= \frac{1}{D_{\max }(x, y)-\frac{1}{u(x, y)}} ,$$
where ${u_i}$ is the depth map of the image side. If the captured object is flat, the estimated depth map will appear in the form of a dome. That is because the lens has a field curvature aberration. The depth map of the image side is itself a curved image surface. In order to find the Petzval radius, we approximate the depth map of the image side as a sphere model:
$$ z(x, y)=\sqrt{r_{k}-\left\{(x-a)^{2}+(y-b)^{2}\right\}}+c , $$
where $a,\;b$, and $c$ are arbitrary constants, and ${r_k}$ is the Petzval radius. The values of $a,\;b,\;c$, and ${r_k}$ are obtained by fitting this sphere model to the depth map of the image side. The objective function here is
$$ \sum_{x, y}\left(u_{i}(x, y)-z(x, y)\right)^{2} .$$
This objective function can be minimized with any least-squares method. In this paper, we use the Levenberg–Marquardt algorithm since this objective function is nonlinear.

3. VERIFICATION OF THE PROPOSED METHOD

To verify the validity of our method, we measured the radius of the field curvature of a solid lens and compare it with a theoretical value. We used an electrowetting liquid lens A-25H0 from Corning Varioptic. The CCD camera used in the experiment is acA2500-14uc from the Basler ace company with a pixel size of 2.2 µm. Specifications of the target solid lens of which the field curvature was measured, along with our experiment setting parameters, are shown in Tables 1 and 2. In order to avoid the chromatic aberration problem, only the blue wavelength band is exploited. In addition, the lens aperture was set to 2.5 mm to minimize spherical aberration. The object to be captured was set to be patterned and flat. If this research is just about measuring the depth of an object, a more complex object such as a sphere should be adopted to ensure reliability. However, this paper shows the method of measuring the field curvature of a target lens not for the depth of the object but for the quality of the lens. Field curvature is a defect in the lens in which the object is focused not on the flat image surface but on the curved image surface. According to the definition of the field curvature, only the flattened object can be used to measure the field curvature of the lens. ${D_{\max}}$ and ${D_{\min}}$ are the synthetic optical power of the target solid lens and the liquid lens when the optical power of the liquid lens is high and low diopter, respectively. The object distance is set to be 35 cm. In this method, to minimize the aberration effects of the liquid lens, the optical powers of the liquid lens near 0D are required. In this experiment, we set the distance of the object to 35 cm so that the center of the object remained in focus when the optical powers of the liquid lens was near 0D. If the distance between the lens and image sensor is different from our experimental setting, the distance of the object should also be different.

Tables Icon

Table 1. Specification of the Target Lens

View Table | View all tables in this article
Tables Icon

Table 2. Experiment Setting Parameters

View Table | View all tables in this article

First, the far-focused image and near-focused image of a flat object were obtained when the diopter of the liquid lens was 0D and 2D, respectively. The captured region of the object is 10 cm by 10 cm, so the on-axis distance between the center of the object and the lens is 35 cm, and the off-axis distance between the edge of the object and the lens is 35.707 cm. As a result, the optical powers of 85.501D and 85.445D are required to focus the center and the edge of the object, respectively. However, when using this DfD algorithm to extract object depth, it is not necessary to have a focal plane at the front and rear ends of the object to be measured in depth. The object whose depth is measured only needs to exist between the camera system’s near-focal plane and the far-focal plane. In this experiment, to improve depth accuracy by using linearity of the normalized ratio [8], the diopter larger than that is required to focus the center of the object and the diopter smaller than that is required to focus the edge of the object selected. In this experiment, 2D optical power change of a liquid lens is used. Only the target solid lens’s optical power is about 83.3 D (focal length is 12 mm). Thus, according to Gaussian reduction theory [13], when the optical power of the liquid lens changes from 0D to 2D, the front principal plane moves back by 60 µm, and the rear principal plane moves forward by 0.7 µm. Thus, although the magnification change is led, the maximum pixel shift for a 10 cm by 10 cm object is only 0.25 pixels; thus, the DfD calculation is not affected.

Field curvature is the sum of the target lens and liquid lens. Therefore, the field curvature of the liquid lens can affect the measurement of the field curvature of the target lens. The field curvature of the electrowetting liquid lens is given, i.e., optical ${\rm power}/\{({\rm water}\;{\rm index})*({\rm oil}\;{\rm index})\}$. In this method, the liquid lens’s optical power 0D and 2D are used in the capturing process of the two defocused images. The field curvature and the radius of field curvature of only the liquid lens are just ${0}\sim- {0.9}$ and $\infty \sim{1.11}\;{\rm m}$ when the liquid lens’s optical power ranges from 0D to 2D. Thus, the field curvature of the liquid lens can be negligible. Also, there was an analytical study of the same liquid lens used in this paper, and it showed good optical performance results less than 0.369 (unit of He–Ne wavelength) for coma aberration and proved that the liquid lens is suitable for defocusing mechanisms [14]. Therefore, other effects caused by the liquid lens can be negligible for measuring the field curvature of the target lens. The object used in the experiment and a pair of defocus images taken to measure the field curvature are shown in Fig. 4. A KIMTEX wiper was used as flat patterned object to be captured. As shown in Figs. 4(b) and 4(c), even though a flat object was captured, the defocus between the center and the boundary is different. In Fig. 4(b), which is the near-focused image, the center field (red region) is sharper than the boundary field (blue region). On the other hand, in Fig. 4(c), which is the far-focused image, the center field (blue region) is blurred, but the boundary field (red region) is sharper. Figure 4(b) corresponds to Fig. 1(a), and Fig. 4(c) corresponds to Fig. 1(b). This phenomenon is because the target lens has severe field curvature aberration. If the field curvature of the lens is corrected, the defocus on a flat object will be the same in all areas.

 figure: Fig. 4.

Fig. 4. Flat object and two defocused images. (a) Flat patterned object used in this experiment. (b) Near-focused image (focused in center). (c) Far-focused image (focused in field boundary).

Download Full Size | PDF

 figure: Fig. 5.

Fig. 5. Result of the DFD algorithm on the flat object. (a) Depth map. (b) 3D reconstruction. (c) Cross section of the depth.

Download Full Size | PDF

 figure: Fig. 6.

Fig. 6. Image side depth map and its fitting model. (a) Depth map of image side. (b) 3D reconstruction of the depth map of image side. (c) Cross section of the depth map in image side (d) Top view of fitted sphere model. (e) 3D view of fitted sphere model. (f) Cross section of the fitted model.

Download Full Size | PDF

After both defocused images were obtained, the DfD algorithm was applied to these two defocused images using MATLAB; thus, a depth map in the object side was obtained. Figure 5 shows the obtained depth map and its cross section. In fact, the flat object was actually located 35 cm constantly from the lens; however, the estimated depth map of the object was in the range of 35 to 75 cm. This is because the defocus between the two images is different at the center and field boundary. This means that a curved focal plane was formed due to field curvature aberration. Then, the depth map in the object side was converted to the depth map in image side simply by Eq. (6). The depth map, 3D reconstruction of the depth map in the image side, and its profile are shown as Figs. 6(a)–6(c). The $z$ axis in Fig. 6(b) means image distance from the lens to image sensor. Next, we formulated our objective function according to Eqs. (7) and (8). This objective function can be minimized by any least-squares fitting algorithm. In this experiment, a Levenberg–Marquardt algorithm was used because the objective function is nonlinear [15]. Figures 6(d)–6(f) show the fitting model obtained by minimizing the objective function by Levenberg–Marquardt algorithm.

 figure: Fig. 7.

Fig. 7. Difference between depth map in image side and fitted sphere model. (a) 3D reconstruction of depth map in image side. (b) Profile of depth map of image side.

Download Full Size | PDF

The final result of the fitting resulted in a Petzval radius of 23.1 mm. The Petzval radius of the target lens is theoretically 21.4 mm. We calculated the RMSE to determine whether it is appropriate to model a Petzval surface as a sphere. Figure 7(a) represents the difference between depth map in the image side and fitted sphere model. Most values were found to be less than 0.01 mm, and the actual calculation showed that the RMSE value was small, i.e., 0.012 mm. Therefore, it is appropriate to model a Petzval surface as a sphere surface. The Petzval field curve was successfully measured with tiny errors by the proposed method.

4. SUMMARY

In summary, the electrowetting liquid lens-based DfD method for measuring field curvature was described. Since the diopter change of the liquid lens is small near 0D, other effects (e.g., field curvature and magnification change, etc.) from the liquid lens are negligible. Only two defocused images were required for field curvature measurement. Image processing was carried out by MATLAB. After the depth map in the object side is obtained based on a DfD algorithm, it was transformed to image the side depth map using the simple lens equation. Then, a Petzval surface is modeled as a sphere surface using the Levenberg–Marquardt algorithm. To confirm how well the sphere surface model is fitted to the Petzval surface, RMSE was calculated, and its value was 0.012 mm. The result of measuring the radius of the field curvature by this method shows 23.1 mm, which is only 1.7 mm apart from the theoretical value. There is no need for complex equipment such as lasers, microscopes, or telecentric systems, so this method does not cost much. Further, this system has no mechanical movement for focus tuning; thus, it is simple and shows fast measurement time compared with other conventional methods.

Funding

Institute for Information and Communications Technology Promotion (2017-0-01803); BK21+ (Brain Korea 21+).

Acknowledgment

The authors received an Institute for Information and Communications Technology Promotion grant funded by the Korean government, Development of Fundamental Technology of Core Components for Augmented and Virtual Reality Devices and Brain Korea 21+ (BK21+).

Disclosures

The authors declare no conflicts of interest.

REFERENCES

1. W. T. Welford, Aberrations of the Symmetrical Optical System (Academic, 1974).

2. S. Matsunaga and S. K. Nayar, “Field curvature correction using focal sweep,” IEEE Trans. Comput. Imaging 1, 259–269 (2015). [CrossRef]  

3. Y. Wang, “Advanced theory of field curvature,” Ph.D. thesis (University of Arizona, 2014).

4. E. Logean, T. Scharf, N. Bongard, H. P. Herzig, and M. Rossi, “Field curvature correction in multichannel miniature imaging systems suited for wafer-level production,” Opt. Eng. 52, 075101 (2013). [CrossRef]  

5. Y. Ishii, “Reflection volume holographic scanners with field-curvature corrections,” Appl. Opt. 22, 3491–3499 (1983). [CrossRef]  

6. J. P. Kirk, “Astigmatism and field curvature from pin-bars,” Proc. SPIE 1463, 282–291 (1991). [CrossRef]  

7. G. Blahusch, W. Eckstein, and C. Steger, “Calibration of curvature of field for depth from focus,” Int. Arch. Photogramm. Remote Sens. Spat. Inf. Sci. 34, 173–180 (2003).

8. M. Watanabe and S. K. Nayar, “Rational filters for passive depth from defocus,” Int. J. Comput. Vis. 27, 203–225 (1998). [CrossRef]  

9. M. Watanabe and S. K. Nayar, “Telecentric optics for computational vision,” in Proceedings of European Conference on Computer Vision (1996), pp. 439–451.

10. B. Burger, S. C. Meimon, C. Petit, and M. C. Nguyen, “Improvement of varioptic’s liquid lens based on electrowetting: how to obtain a short response time and its application in the design of a high resolution iris biometric system,” Proc. SPIE 9375, 93750S (2015). [CrossRef]  

11. A. N. J. Raj and R. C. Staunton, “Rational filter design for depth from defocus,” Pattern Recogn. 45, 198–207 (2012). [CrossRef]  

12. M. Ye, X. Chen, Q. Li, J. Zeng, and S. Yu, “Depth from defocus measurement method based on liquid crystal lens,” Opt. Express 26, 28413–28420 (2018). [CrossRef]  

13. V. N. Mahajan, Fundamentals of Geometrical Optics (SPIE, 2014).

14. J. Fuentes-Fernández, S. Cuevas, L. Alvarez, and A. Watson, “Tests and evaluation of a variable focus liquid lens for curvature wavefront sensors in astronomy,” Appl. Opt. 52, 7256–7264 (2013). [CrossRef]  

15. K. Levenberg, “A method for the solution of certain non-linear problems in least squares,” Quart. Appl. Math. 2, 164–168 (1944). [CrossRef]  

Cited By

Optica participates in Crossref's Cited-By Linking service. Citing articles from Optica Publishing Group journals and other participating publishers are listed here.

Alert me when this article is cited.


Figures (7)
Fig. 1.
Fig. 1. Petzval field curvature aberration. (a) Focused in center. (b) Focused in field boundary.

View in Article | Download Full Size | PDF

Fig. 2.
Fig. 2. Schematic diagram of the proposed method.

View in Article | Download Full Size | PDF

Fig. 3.
Fig. 3. Image formation of DfD optic model using liquid lens.

View in Article | Download Full Size | PDF

Fig. 4.
Fig. 4. Flat object and two defocused images. (a) Flat patterned object used in this experiment. (b) Near-focused image (focused in center). (c) Far-focused image (focused in field boundary).

View in Article | Download Full Size | PDF

Fig. 5.
Fig. 5. Result of the DFD algorithm on the flat object. (a) Depth map. (b) 3D reconstruction. (c) Cross section of the depth.

View in Article | Download Full Size | PDF

Fig. 6.
Fig. 6. Image side depth map and its fitting model. (a) Depth map of image side. (b) 3D reconstruction of the depth map of image side. (c) Cross section of the depth map in image side (d) Top view of fitted sphere model. (e) 3D view of fitted sphere model. (f) Cross section of the fitted model.

View in Article | Download Full Size | PDF

Fig. 7.
Fig. 7. Difference between depth map in image side and fitted sphere model. (a) 3D reconstruction of depth map in image side. (b) Profile of depth map of image side.

View in Article | Download Full Size | PDF

Tables (2)

Tables Icon

Table 1. Specification of the Target Lens

View Table | View all tables in this article
Tables Icon

Table 2. Experiment Setting Parameters

View Table | View all tables in this article

Equations (8)

Equations on this page are rendered with MathJax. Learn more.

D = D max − ( 1 − α ) ∇ D = D min + ( 1 + α ) ∇ D R 1 = a v ( 1 − α ) ∇ D R 2 = a v ( 1 + α ) ∇ D ,
h ( x , y ) = 1 π R 2 ∏ ( 1 2 R x 2 + y 2 ) ,
H ( p , q ) = 1 π R p 2 + q 2 j 1 ( 2 π R p 2 + q 2 ) ,
M ( p , q ) P ( p , q ) = H 1 ( p , q ) − H 2 ( p , q ) H 1 ( p , q ) + H 2 ( p , q ) .
M ( f r ) P ( f r ) = G P 1 ( f r ) G M 1 ( f r ) α + G P 2 ( f r ) G M 1 ( f r ) α 3 .
u i ( x , y ) = 1 D max ( x , y ) − 1 u ( x , y ) ,
z ( x , y ) = r k − { ( x − a ) 2 + ( y − b ) 2 } + c ,
∑ x , y ( u i ( x , y ) − z ( x , y ) ) 2 .
Select as filters
Select Topics Cancel
Top
       
© Copyright 2024 | Optica Publishing Group. All rights reserved, including rights for text and data mining and training of artificial technologies or similar technologies.
Privacy | Terms of Use