Expand this Topic clickable element to expand a topic
Skip to content
Optica Publishing Group

High-precision camera calibration based on a 1D target

Open Access Open Access

Abstract

In this paper, the problem of camera 1D calibration is well solved by our proposed high-precision algorithm, which can satisfy actual requirements. We present a viewpoint that the closed-form solution can simply achieve high calibration accuracy in the absence of distortion. So, we abandon the habitual strategy of global nonlinear optimization for all intrinsic and extrinsic parameters. The innovations of the proposed algorithm are three-fold: firstly, cyclic distortion correction method is introduced to ensure that the estimated distortion parameters gradually approach the exact values; secondly, a new criterion for the nonlinear optimization of distortion correction is developed; thirdly, we enhance the anti-noise ability of the closed-form solution by optimally weighting the constraint equations. Extensive experiments prove that the proposed algorithm provides the highest calibration accuracy and robustness, which is comparable to 2D calibration. In addition, our proposed algorithm provides a new approach for desirable distortion correction and an idea for 2D calibration of large field of view.

© 2022 Optica Publishing Group under the terms of the Optica Open Access Publishing Agreement

1. Introduction

Vision measurement technique plays an important role in the industrial measurement, in which camera calibration is an essential step to high precision. Nowadays, the 2D calibration technique, based on planar pattern of checkerboard or grid dot, is most widely used [1,2]. But in large-scale metrology, the 2D target, required to cover the large field of view (FOV), is difficult and expensive to manufacture [3,4]. 1D calibration target is characterized by simple structure, low production cost, and flexible operation, which makes it very suitable for calibration of large FOV. Zhang [5] was the first to propose a calibration algorithm based on 1D target of rotating around a fixed point. However, 1D target has the weaker spatial geometric constraints, which leads to the inaccuracy in the actual application of Zhang’s algorithm. It is necessary to improve 1D calibration technique for high precision.

1D calibration technique has attracted the interest of scholars and been widely studied. Hammarstedt [6] analyzed the critical motions of 1D target and simplified the closed-form solution for some special cases. Wu [7] proved that the 1D target of rotating around a fixed point is equivalent to 2D rectangular target essentially, and develop a calibration method while the 1D target undergoes a planar motion. Qi [8] further proposed a motion pattern that 1D target moves only under gravity. Peng [9] calibrated a fixed ideal pin-hole camera by moving a 1D target of 2 points on a fixed plane. Miyagawa [10] achieved the calibration just with a single image in which the special target is made of two orthogonal 1D target with one common point. Despite progress of 1D calibration has been made, but the studies just focused on the motion pattern and the target structure.

Some researchers concentrated their attention on improving the accuracy of 1D calibration. To reduce the condition number of the matrix in the closed-form solution of Zhang's method, Franca [11] preprocessed the image data by normalization. Shi [12] introduced a weighted similarity-invariant linear algorithm, which greatly improved the 1D calibration accuracy. The algorithm includes a new estimation of the relative depth and a new set of weighted constraint equations. Wang [13] gave an adaptively weighted 1D calibration algorithm with higher accuracy. In above three algorithms, the relative depth of the points is needed to estimate first, which introduces extra noise. Lv [14] presented a concept of 1D homography matrix, which makes more rational use of multiple marker points and no estimation of the relative depth. The constraint equations on account of the unit vector in rotation matrix can reduce calibration errors. However, the studies aimed at improving the anti-noise ability of the closed-form solution. The problem that the nonlinear optimization easily traps into local minimum, remains unsolved. Some scholars also applied 1D target in multi-camera calibration [1518], but the performance is not as good as that of 2D target. Although 1D calibration becomes more accurate, further improvement is still needed to meet the actual application.

In this paper, a high-precision 1D calibration algorithm is presented. The factors that greatly influence the calibration accuracy are analyzed, and we abandon the habitual strategy of global nonlinear optimization of all intrinsic and extrinsic parameters. Based on the viewpoint that precise 1D calibration can be obtained in the case of no distortion, a cyclic distortion correction method is introduced to gradually approach the ideal undistorted image. Besides, to achieve higher calibration accuracy, a new criterion for the nonlinear optimization of distortion correction is developed, and the closed-form solution is improved by optimally weighting the constraint equations.

2. Theoretical fundamentals

2.1. Camera model

Pinhole model is the most commonly used camera model in computer vision. Let ${\bf P}\textrm{ = [}x y z{\textrm{]}^\textrm{T}}$ be a point of the calibration target under the world coordinate system, and its corresponding image point be ${\bf m}\textrm{ = [}u v{\textrm{]}^\textrm{T}}$. Their relationship can be described as,

$$\dot{z}\left[ {\begin{array}{c} \textbf{m}\\ 1 \end{array}} \right] = \textbf{K}[{\textbf{R|T}} ]\left[ {\begin{array}{c} \textbf{P}\\ 1 \end{array}} \right]\textrm{, with }\textbf{K} = \left[ {\begin{array}{ccc} \alpha &0&{{u_0}}\\ 0&\beta &{{v_0}}\\ 0&0&1 \end{array}} \right], $$
where $\dot{z}$ is the depth of P under the camera coordinate system; R and T, regarded as the camera extrinsic parameters, respectively represent the rotation matrix and translation vector from the world coordinate system to the camera coordinate system; K is the camera intrinsic matrix with the focal length $(\alpha ,\textrm{ }\beta )$ and the image principal point ${[{u_0},{v_0}]^\textrm{T}}$.

Pinhole model is only the ideal linear model. Lens distortion should be taken into account in actual. For simplicity, only radial component is considered here, and the distortion model can be described as below,

$$\left\{ \begin{array}{l} {u_d} - {u_0} = (u - {u_0})(1 + {\rho_1}{r^2} + {\rho_2}{r^4})\\ {v_d} - {v_0} = (v - {v_0})(1 + {\rho_1}{r^2} + {\rho_2}{r^4})\\ {r^2} = {(u - {u_0})^2} + {(v - {v_0})^2} \end{array} \right., $$
where ${[u,v]^\textrm{T}}$ denotes the undistorted image point; ${[{u_d},{v_d}]^\textrm{T}}$ denotes the distorted image point; $\boldsymbol{\rho } = ({\rho _1},{\rho _2})$ are the first two coefficients of radial distortion; and ${[{u_0},{v_0}]^\textrm{T}}$ denotes both the image distortion center point and the principal point.

2.2. 1D camera calibration

Among the existing 1D calibration techniques, homography-based linear algorithm (HBLA) has the best performance [14]. Here, a brief introduction of HBLA is given. As shown in Fig. 1, the 1D target is composed of a set of collinear mark points, $({\textbf{P}_0},{\textbf{P}_1},{\textbf{P}_2}, \cdots ,{\textbf{P}_N})$, where ${\textbf{P}_0}$ and ${\textbf{P}_N}$ are the rotary fixed point and the end point respectively. To avoid ambiguity, the projection of the original space location of $({\textbf{P}_0},{\textbf{P}_1},{\textbf{P}_2}, \cdots ,{\textbf{P}_N})$ is not drawn. Under the action of the j-th rotation matrix ${\textbf{R}^j} = [\textbf{r}_1^j\textrm{ }{\mkern 1mu} {\mkern 1mu} \textbf{r}_2^j{\mkern 1mu} {\mkern 1mu} \textrm{ }\textbf{r}_3^j]\textrm{ }(j = 1,2,3, \cdots ,J)$ and the common translation vector $\textbf{T} = {[{t_x}\textrm{ }{t_y}\textrm{ }{t_z}]^\textrm{T}}$, the corresponding image point of ${\textbf{P}_n} = {[{x_n}\textrm{ 0 0}]^\textrm{T}}$ can be given according to Eq. (1).

$$\dot{z}_n^j\left[ {\begin{array}{c} {\textbf{m}_n^j}\\ 1 \end{array}} \right] = \textbf{K}[{\textbf{r}_1^j{\mkern 1mu} {\mkern 1mu} \textbf{T}} ]\left[ {\begin{array}{c} {{x_n}}\\ 1 \end{array}} \right]$$

 figure: Fig. 1.

Fig. 1. An illustration of 1D camera calibration

Download Full Size | PDF

By defining $\textbf{K}[\textbf{r}_1^j\textrm{ }\textbf{T}\textrm{]} = {t_z}{\textbf{H}^j}$, 1D homography matrix ${\textbf{H}^j}$ can be generated. Note that ${\textbf{H}^j}$ and $[\textbf{r}_1^j\textrm{ }\textbf{T}\textrm{]}$ are both 3×2 matrixes. Then, we obtain

$$\left\{ {\begin{array}{c} {\textbf{r}_1^j = {t_z}{\textbf{K}^{ - 1}}\textbf{h}_1^j}\\ {\textbf{T}\textrm{ } = {t_z}{\textbf{K}^{ - 1}}\textbf{h}_2^j} \end{array}} \right.,\textrm{ with }{\textbf{H}^j} = \left[ {\begin{array}{cc} {\textbf{h}_1^j}&{\textbf{h}_2^j} \end{array}} \right] = \left[ {\begin{array}{cc} {h_1^j}&{h_4^j}\\ {h_2^j}&{h_5^j}\\ {h_3^j}&1 \end{array}} \right]. $$
To compute the homography matrix, direct linear transformation (DLT) is generally used [19]. Putting ${\textbf{H}^j}$ into a vector $\textbf{h}_\textbf{H}^{\textrm{ }j} = {[h_1^j\,\,h_2^j\,\,h_3^j\,\,h_{\begin{array}{cccccc} 4 \end{array}}^j\,\,h_5^j\,\,1]^\textrm{T}}$, a pair of equations can be derived as,
$$\left[ {\begin{array}{cccccc} {{x_n}} &0 &{ - u_n^jx_n^{}} &1 &0 &{- u_n^j}\\ 0 &{x_n} &{ - v_n^jx_n^{}} &0 &1 &{- v_n^j} \end{array}} \right]\textbf{h}_\textbf{H}^{\textrm{ }j} = \textbf{0}. $$
Using the knowledge that $\textbf{r}_1^j$ is a unit vector, the linear constraint can be given as
$$t_z^2\textbf{h}{_1^{j\textrm{T}}}{\textbf{K}^{\textrm{ - T}}}{\textbf{K}^{\textrm{ - 1}}}\textbf{h}_1^j = 1\textrm{ }({j = 1,2,3 \cdots ,J} ). $$
Let
$$\textbf{W} = t_z^2{\textbf{K}^{ - \textrm{T}}}{\textbf{K}^{ - 1}} = \left[ {\begin{array}{ccc} {{W_{11}}}&0&{{W_{13}}}\\ 0&{{W_{22}}}&{{W_{23}}}\\ {{W_{13}}}&{{W_{23}}}&{{W_{33}}} \end{array}} \right], $$
Eq. (6) is equivalent to
$${\textbf{V}^j}\boldsymbol{\xi } = 1,\textrm{ with}\left\{ \begin{array}{l} {\textbf{V}^j} = [h{_1^{j2}}\textrm{ }h{_2^{j2}}\textrm{ }2h_1^jh_3^j\textrm{ }2h_2^jh_3^j\textrm{ }h{_3^{j2}}]\\ \textrm{ }\boldsymbol{\xi }\textrm{ } = {[{W_{11}}\,\,\textrm{ }{W_{22}}\,\,\textrm{ }{W_{13}}\,\,\textrm{ }{W_{23}}\,\,\textrm{ }{W_{33}}]^\textrm{T}} \end{array} \right.. $$
If five or more observations are taken, multiple constraints like Eq. (8) form a linear system of equations for $\boldsymbol{\xi }$. Once $\boldsymbol{\xi }$ is solved by the least squares method, the intrinsic matrix K can be well-determined. The process above is called the closed-form solution.

Owing to the existing of noise and distortion, the result of the closed-form solution is imprecision and should be refined. The habitual strategy is the global nonlinear optimization for all intrinsic and extrinsic parameters. The optimal target function can be defined as

$$\min f(\boldsymbol{\rho },\textbf{K},\textbf{r}_1^{j} , \textbf{T}) = {\sum\limits_{j = 1}^J {\sum\limits_{n = 0}^N {\left||{\textbf{m}_n^j - \hat{\textbf{m}}_n^j(\boldsymbol{\rho },\textbf{K},\textbf{r}_1^{j},\textbf{T},{\textbf{P}_n})} \right||} } ^2}, $$
where $\hat{\textbf{m}}_n^j(\boldsymbol{\rho },\textbf{K},\textbf{r}_1^{j},\textbf{T},{\textbf{P}_n})$ denotes the computed reprojection of ${\textbf{P}_n}$. Levenberg-Marquardt (LM) algorithm is usually used to solve such a nonlinear minimization problem. To reduce the number of optimized parameters, the unit vector $\textbf{r}_1^j$ with three parameters can be simplified by spherical coordinates parameters, denoted by $({\theta ^j}\textrm{, }{\varphi ^j})$, which define the orientation of the 1D target.

3. Proposed method

In terms of the existing 1D calibration techniques, the LM algorithm easily traps into local minima, duo to the imprecision initial value and the coupling of too much parameters. In this paper, the habitual strategy of global nonlinear optimizing is not adopted any more.

3.1. What greatly influence the accuracy of 1D calibration?

In order to improve the calibration performance, we try to identify the factors that greatly influence the accuracy. We compare the closed-form solutions of normalized Zhang’s linear algorithm (NZLA) [11], weighted similarity-invariant linear algorithm (WSILA) [12] and HBLA by computer simulation. It is worth noting that NZLA, WSILA and HBLA here do not include the step of global nonlinear optimization. The simulation settings are as follows.

The intrinsic matrix of the simulated camera is $\textbf{K} = \left[ {\begin{array}{ccc} {6510}&0&{2600}\\ 0&{6490}&{1700}\\ 0&0&1 \end{array}} \right]$. The image resolution is 5000 × 3600 pixels, and no distortion exists. The simulated 1D target of 31cm long has 20 mark points and rotates around the fixed point T = [−1cm, −16cm, 63cm]T. The orientations of 1D target are randomly generated by sampling the distributions ${\theta ^j}\sim \textrm{Uniform}(0.25\pi ,0.75\pi )$ and ${\varphi ^j}\sim \textrm{Uniform}(0.3\pi ,0.7\pi )$. Zero-mean Gaussian noise with the standard deviation of 0.2 pixel is added to each simulated image point. The number of the random simulated images for calibration varies from 10 to 80.

To reduce accidental error, we repeat the calibration trial 100 times and average the results for each test settings. Figure 2(a) gives an example of the image set for calibration, Fig. 2(b) shows the reprojection errors of different closed-form solutions, and Fig. 2(c)-(f) plot the relative errors of the intrinsic parameters. Note that, the relative error represents the percentage of root mean square error of the result, and the relative errors of (u0, v0) are computed with respect to the focus length [5].

 figure: Fig. 2.

Fig. 2. The 1D calibration in the case of no distortion: (a) an image set of the 1D target with 10 rotations;(b) the reprojection errors of different closed-form solutions; (c)-(f) the relative errors of the four intrinsic parameters

Download Full Size | PDF

When there are 20 images participating in calibration, the relative error of below 0.1% for all parameters can be achieved. With the increasing number of images, the error decreases. The reprojection errors with different images are almost equivalent to the noise level of 0.2 pixel which can be easily reached in image feature point extraction. Despite the differences between the algorithms, the calibration results are acceptable and practical in the accuracy of both intrinsic parameters and reprojection errors.

Taking these facts into account, we present a viewpoint that if perfect image distortion correction is done, the closed-form solution is good enough for calibration even without nonlinear optimization. Imprecise distortion correction or fitting will introduce extra noise, which should be the reason why the existing algorithms are not accurate. Therefore, the performance of 1D calibration can be greatly improved by correcting the distortion as precise as possible.

3.2. Correction of image distortion

Distortion correction should be carried out first in calibration. Good distortion correction helps reduce extra noise in the process of closed-form solution. Since the distortion center ${[{u_0},{v_0}]^\textrm{T}}$ is separately optimized by other proposed method, only distortion coefficients $\boldsymbol{\rho }$ are accurately estimated here.

Carlos [20] gave a nonlinear optimization method for distortion correction, which is based on two characters, the collinearity of points and the cross ratio invariance. Considering the including of these two characters in 1D homography [21], we develop a new optimal target function defined by,

$$\min \textrm{ }f(\boldsymbol{\rho }) = \sum\limits_{j = 1}^J {\sum\limits_{n = 1}^N {{{||{\textbf{m}_n^j - \hat{\textbf{m}}(\boldsymbol{\rho },[{u_0},{v_0}],{\textbf{P}_n},{\textbf{H}^j}(\boldsymbol{\rho }))} ||}^2}} }, $$
where $\hat{\textbf{m}}(\boldsymbol{\rho },[{u_0},{v_0}],{\textbf{P}_n},{{\textbf{H}}^j}(\boldsymbol{\rho }))$ is the 1D homography reprojection of ${\textbf{P}_n}$ under the distortion coefficients $\boldsymbol{\rho }$, the distortion center ${[{u_0},{v_0}]^\textrm{T}}$, and the 1D homography ${\textbf{H}^j}(\boldsymbol{\rho })$. Note that 1D homography reprojection is not the reprojection which is related to $[\textbf{r}_1^{j},\textbf{T}]$ and K in the general sense. We will discuss it in Section 3.4.2 further.

Here, ${\textbf{H}^j}(\boldsymbol{\rho })$ is a function instead of a fix value. It means that ${\textbf{H}^j}(\boldsymbol{\rho })$ should be recomputed while obtaining Jacobian matrix and residual at each iteration of LM algorithm. The ${\textbf{H}^j}(\boldsymbol{\rho })$ cannot be expressed in algebraic analytic form, and its calculation rules are as follows:

  • 1. Obtain the approximate undistorted image points $\textbf{m}^{{\prime}j}_n$ based on the current value of $\boldsymbol{\rho }$;
  • 2. Compute ${\textbf{H}^j}(\boldsymbol{\rho})$ from $(\textbf{m}^{{\prime}j}_1\textrm{,}\textbf{m}^{{\prime}j}_2\textrm{,} \cdots \textbf{m}^{{\prime}j}_N)$ by DLT according to Eq. (5).

If there are no effective values, the initial $\boldsymbol{\rho }$ is set to zero, and the distortion center is set as the image center. After the nonlinear minimization, the distortion parameters and corresponding corrected image can be obtained. Certainly, the distortion correction is not completely accurate. In the later part, we will discuss how to improve the distortion correction.

3.3. Improved closed-form solution of HBLA

Since HBLA performs best among the existing algorithms as shown in Fig. 2, the closed-form solution of it is used to compute the camera intrinsic matrix after the distortion correction. To enhance the anti-noise ability, we further improve this closed-form solution by optimally weighting the constraint equations.

Here, we start to deduce the optimal weighting coefficient based on 1D homography. The constraint equation to be weighted is as follows,

$$\textbf{h}{_1^{j\textrm{T}}}\textbf{Wh}_1^j = 1,\textrm{ with}\,\,\textbf{W} = t_z^2{\textbf{K}^{ - \textrm{T}}}{\textbf{K}^{ - 1}}. $$
Even with the same data noise, the error of Eq. (11) is not constant in different rotations. We build a residual expression to evaluate it.
$${\varepsilon ^j} = \textbf{h}{_1^{j\textrm{T}}}\textbf{Wh}_1^j - 1. $$
The variation of ${\varepsilon ^j}$ is determined by,
$$\textrm{d}{\varepsilon ^j} = 2\textbf{h}{_1^{j\textrm{T}}}\textbf{W} \cdot \textrm{d}\textbf{h}_1^j. $$
In addition, the image point $\textbf{m}_n^j = {[u_n^j\textrm{ }v_n^j]^\textrm{T}}$ and $\textbf{h}_1^j = {[h_1^j\,\,h_2^j\,\,h_3^j]^\textrm{T}}$ satisfy
$$\left\{ {\begin{array}{l} {u_n^j = \frac{{h_1^j{x_n} + h_4^j}}{{h_3^j{x_n} + 1}}}\\ {v_n^j = \frac{{h_2^j{x_n} + h_5^j}}{{h_3^j{x_n} + 1}}} \end{array}} \right.. $$
We have
$$\textrm{d}\textbf{m}_n^j = \left[ {\begin{array}{ccc} {\frac{{{x_n}}}{{h_3^j{x_n} + 1}}}&0&{ - \frac{{(h_1^j{x_n} + h_4^j){x_n}}}{{{{({h_3^j{x_n} + 1} )}^2}}}}\\ 0&{\frac{{{x_n}}}{{h_3^j{x_n} + 1}}}&{ - \frac{{(h_2^j{x_n} + h_5^j){x_n}}}{{{{({h_3^j{x_n} + 1} )}^2}}}} \end{array}} \right] \cdot \textrm{d}\textbf{h}_1^j, $$
In view of the combined action of all image points on $\textrm{d}\textbf{h}_1^j$, we construct a vector,
$$\textrm{d}\textbf{m}_{\boldsymbol{\Sigma }}^{j} = {[\begin{array}{ccccc}\textrm{d}\textbf{m}{_1^{j\textrm{T}}} &\textrm{ d}\textbf{m}{_2^{j\textrm{T}}} &\cdots &\textrm{ d}\textbf{m}{_n^{j\textrm{T}}}\end{array}]^\textrm{T}}. $$
Then, a linear equation system takes form as bellow,
$$\textrm{d}\textbf{m}_{\boldsymbol{\Sigma }}^{j} = {\textbf{S}^j} \cdot \textrm{d}\textbf{h}_1^j,\textrm{ with}\,\,{\textbf{S}^j} = \left[ {\begin{array}{ccc} {\frac{{{x_1}}}{{h_3^j{x_1} + 1}}}&0&{ - \frac{{(h_1^j{x_1} + h_4^j){x_1}}}{{{{({h_3^j{x_1} + 1} )}^2}}}}\\ 0&{\frac{{{x_1}}}{{h_3^j{x_1} + 1}}}&{ - \frac{{(h_2^j{x_1} + h_5^j){x_1}}}{{{{({h_3^j{x_1} + 1} )}^2}}}} \\ {\frac{{{x_2}}}{{h_3^j{x_2} + 1}}}&0&{ - \frac{{(h_1^j{x_2} + h_4^j){x_2}}}{{{{({h_3^j{x_2} + 1} )}^2}}}}\\ 0&{\frac{{{x_2}}}{{h_3^j{x_2} + 1}}}&{ - \frac{{(h_2^j{x_2} + h_5^j){x_2}}}{{{{({h_3^j{x_2} + 1} )}^2}}}}\\ &\cdots &\\ {\frac{{{x_N}}}{{h_3^j{x_N} + 1}}}&0&{ - \frac{{(h_1^j{x_N} + h_4^j){x_N}}}{{{{({h_3^j{x_N} + 1} )}^2}}}}\\ 0&{\frac{{{x_N}}}{{h_3^j{x_N} + 1}}}&{ - \frac{{(h_2^j{x_N} + h_5^j){x_N}}}{{{{({h_3^j{x_N} + 1} )}^2}}}} \end{array}} \right]. $$
The least-squares solution of $\textrm{d}\textbf{h}_1^j$ is
$$\textrm{d}\textbf{h}_1^j = {({\textbf{S}^j}^\textrm{T}{\textbf{S}^j})^{\textrm{ - 1}}}{\textbf{S}^j}^\textrm{T} \cdot \textrm{d}\textbf{m}_{\boldsymbol{\Sigma }}^{j}. $$
Combing Eq. (18) and Eq. (13) yields
$$\textrm{d}{\varepsilon ^j} = 2\textbf{h}{_1^{j\textrm{T}}}\textbf{W}{({\textbf{S}^j}^\textrm{T}{\textbf{S}^j})^{\textrm{ - 1}}}{\textbf{S}^j}^\textrm{T} \cdot \textrm{d}\textbf{m}_{\boldsymbol{\Sigma }}^{j}. $$
Equation (20) represents that how the deviation of the residual is influenced by image data noise. Assuming that the error of an arbitrary image point independently satisfies Gaussian distribution and its standard deviation is $\delta$, the standard deviation of the residual is
$$\textrm{SD}({\varepsilon ^j}) = ||2\textbf{h}{_1^{j\textrm{T}}}\textbf{W}{({\textbf{S}^j}^\textrm{T}{\textbf{S}^j})^{\textrm{ - 1}}}{\textbf{S}^j}^\textrm{T}||\cdot \frac{\delta }{{\sqrt 2 }}. $$
To equalize residual of each constraint and ignore the scaling factor, the optimal weighting coefficient can be redefined as Eq. (21), where W is from the original closed-form solution.
$${w^j} = \frac{1}{{||\textbf{h}{{_1^j}^\textrm{T}}\textbf{W}{{({\textbf{S}^j}^\textrm{T}{\textbf{S}^j})}^{\textrm{ - 1}}}{\textbf{S}^j}^\textrm{T}||}}. $$
By adding the weighting coefficient into the constraint equation, Eq. (12) becomes
$${w^j}{\textbf{V}^j}\boldsymbol{\xi } = {w^j}. $$
Then, $\boldsymbol{\xi }$ and K with higher accuracy can be solved by the improved closed-form solution.

To evaluate the performance of the improved closed-form solution, we compare it with the original closed-form solution of HBLA. The simulation parameters are set as same as that mentioned in Section 3.1. It is seen from Fig. 3 that the improved closed-form solution has better calibration accuracy.

 figure: Fig. 3.

Fig. 3. Comparison of calibration errors between the original and improved closed-form solutions

Download Full Size | PDF

3.4. Cyclic distortion correction

According to the strategy of the existing algorithm, global nonlinear optimization should be adopted to refine the result of the closed-form solution. But it is extremely easy to fall into local minimum. Here, we present a cyclic distortion correction method instead of the global nonlinear optimization.

3.4.1. How to perform cyclic distortion correction?

As is known, it is imprecise to correct the image distortion without any prior information of the distortion parameters, especially the distortion center [22]. The inaccurate distortion correction introduces extra data noise, which greatly decreases the calibration performance.

The distortion center, as the principal points in intrinsic matrix, also takes part in the projection transformation of the pinhole model. Clearly, good image distortion correction is conducive to the closed-form solution, and the intrinsic matrix estimated in the closed-form solution can provide the more prior information for distortion correction. This idea has been widely used in the two-step camera calibration. Therefore, we apply the strategy of cyclic distortion correction to 1D calibration, even if the distortion correction and the closed-form solution are different from those in [23,24]. The calibration procedure is as follows:

  • 1. Regard the two calibration steps of the distortion correction and the closed-form solution as a cycle, and execute it repeatedly;
  • 2. The distortion coefficient and the principal point both estimated in each cycle are taken as the initial values for the distortion correction in the next cycle.

By this cyclic operation, both the calibration result and the distortion correction will be refined gradually and accurately.

3.4.2. When to stop the cyclic distortion correction?

The cyclic distortion correction cannot run indefinitely. This introduces a new question, when to stop it. We naturally think about the stop condition of the traditional nonlinear optimization which is the arrival of the minimum reprojection error. However, it is found that the nonlinear minimization expression of Eq. (9) is unreasonable. Theoretically, the 1D target under different rotations should have the same translation vector all the time. But in actual, it has a small displacement due to the mechanical movement. The ${\textbf{T}^j}$ needs to be separately used in the j-th rotation instead of T. Consequently, a modification to the expression is,

$$\min f(\boldsymbol{\rho },\textbf{K},\textbf{r}_1^{j},{\textbf{T}^j}) = {\sum\limits_{j = 1}^J {\sum\limits_{n = 1}^N {\left|\left|{\textbf{m}_n^j - \hat{\textbf{m}}_n^j(\boldsymbol{\rho },\textbf{K},\textbf{r}_1^{j},{\textbf{T}^j},{\textbf{P}_n})} \right|\right|} } ^2}. $$

If an estimated K is given, $[\textbf{r}_1^{j},{\textbf{T}^j}]$ uniquely depends on a ${\textbf{H}^j}$, and vice versa. Thus, the 1D homography reprojection error of Eq. (10) has the same meaning with the reprojection errors of Eq. (23) in this situation. We regard the minimization of 1D homography reprojection error directly as the judgment of the end of the cyclic distortion correction. With merging K and $[\textbf{r}_1^{j},{\textbf{T}^j}]$ into ${\textbf{H}^j}$, the optimization criterion of Eq. (10) is almost equivalent to that of Eq. (23). Once Eq. (10) achieves a minimum in the whole calibration, the optimized $\boldsymbol{\rho }$ do not change anymore. At this time, the cyclic distortion correction can be stopped.

3.5. Summary

For simplicity, the proposed whole calibration algorithm is called cyclic distortion correction algorithm (CDCA), and summarized as follows:

  • Step 1: Distortion correction: estimate the distortion coefficients by LM algorithm, and correct the image distortion.
  • Step 2: Closed-form solution: calculate the 1D homography matrix for each corrected image and the optimal weighting coefficients; establish a system of weighted constraint equations and solve it to get the intrinsic matrix.
  • Step 3: Cyclic calibration: alternately run the procedures of the distortion correction and the closed-form solution until convergence.

Figure 4 shows the algorithm procedure of CDCA. Regarding CDCA as a kind of iterative optimization, the optimized parameters are the intrinsic parameters including $\boldsymbol{\rho }$ and K essentially. By this way, the habitual strategy of global nonlinear optimizing for all intrinsic and extrinsic parameters is abandoned.

 figure: Fig. 4.

Fig. 4. The algorithm procedure of CDCA

Download Full Size | PDF

4. Experiments

4.1. Computer simulation

In this computer simulation, we adopt the same settings of the simulated parameters mentioned in Section 3.1, but add the radial distortion to the simulated images. The above settings are used just as the basic simulation parameters as shown in Table 1, and some of them will be changed for the corresponding analysis in subsequent experiments.

Tables Icon

Table 1. Basic simulation settings

To evaluate the performance of CDCA, we also test NZLA, WSILA and HBLA for comparison. It is worth noting that NZLA, WSILA and HBLA here all include the step of global nonlinear optimization. To reduce accidental error, we repeat the calibration trial 100 times and average the results for each test settings.

4.1.1. Performance with respect to the number of images

This experiment is conducted to evaluate the algorithm performance concerning the number of images. We varies it from 10 to 80, and the rest of the basic simulation settings remains unchanged. The relative errors of CDCA, NZLA, WSILA and HBLA are shown in Fig. 5.

 figure: Fig. 5.

Fig. 5. The calibration errors with respect to the number of images

Download Full Size | PDF

It is obvious that the performance of the CDCA is the best and is even better than that of the HBLA in Fig. 2. Although the relative errors of all four algorithms decrease as the number of images increases, the convergence of the CDCA is significantly stronger than that of the others. In order to show the intermediate processes of CDCA in detail, one calibration trial involving 20 images is selected, and Table 2 gives the result. As the number of cycles increases, the calibration accuracy becomes higher gradually.

Tables Icon

Table 2. The calibration result of one calibration trial involving 20 images

4.1.2. Performance with respect to the noise level

This experiment is conducted to evaluate the algorithm performance concerning the noise level. The standard deviation of zero-mean Gaussian noise varies from 0.0 to 1.0 pixel, and the rest of the basic simulation settings remains unchanged. The relative errors are shown in Fig. 6.

 figure: Fig. 6.

Fig. 6. The calibration errors with respect to the noise level

Download Full Size | PDF

All relative errors of the four algorithm rise with the increasing noise. The CDCA maintains the best calibration performance under different noise level. Even in the case of no noise, the NZLA, WSILA and HBLA do not act well enough, which reflects the impact caused by the incorrect distortion fitting in global nonlinear optimization.

4.1.3. Performance with respect to the position of the principal point

In general, the principal point does not lie exactly at the center of image. This experiment is conducted to evaluate the calibration accuracy under the different offset of the principal point from the image center ${[2500,\textrm{ }1800]^\textrm{T}}$. We reset the principal point ${[{u_0},{v_0}]^\textrm{T}}$, so that the offset varies from 0 to 500 pixels with a step of 50. The rest of the basic simulation settings remains unchanged. At each offset level, 100 trials are randomly performed. The relative errors of the CDCA, NZLA, WSILA and HBLA are shown in Fig. 7.

 figure: Fig. 7.

Fig. 7. The calibration errors with respect to the offset between the principal point and the image center

Download Full Size | PDF

It is seen that the calibration accuracy of the CDCA with best performance is almost independent of the principal point, but the offset slightly affects the accuracies of the NZLA, WSILA and HBLA. The CDCA has better stability in calibration.

4.1.4. Performance with respect to the distortion level

This experiment is conducted to evaluate the calibration accuracy under the different distortion levels. We reset the distortion coefficients $\boldsymbol{\rho }$, so that the image distortion varies from 0.0% to 1.0% with a step of 0.1%. The rest of the basic simulation settings remains unchanged. At each distortion level, 100 trials are randomly performed. The relative errors are represented in Fig. 8.

 figure: Fig. 8.

Fig. 8. The calibration errors with respect to the distortion level

Download Full Size | PDF

Clearly, the CDCA is robust in calibration. The relative errors of the CDCA in different distortion levels vary slightly. But the distortion level strongly affects the accuracies of the other algorithms. Only in the case of no distortion, the performance of the HBLA is closed to that of the CDCA.

4.2. Real image data experiment

In the experiments with real image data, the test camera has a resolution of 5488×3672pixels and the focus length is 12 mm. The FOV is about 45×34cm. For comparison, we adopt 2D calibration technique to measure the camera, and capture 20 images of a checkerboard pattern. The calibration result, regard as the reference value, is $\textbf{K} = \left[ {\begin{array}{ccc} {6725.2}&0&{2592.5}\\ 0&{6721.8}&{1835.3}\\ 0&0&1 \end{array}} \right]$. The 1D target is placed 50cm in front of the camera as shown in Fig. 9. Twenty-seven collinear points represented by checkerboard corners are marked on it. The distance between adjacent points is 1cm. Then, the 1D target rotates around its sharp supporting point and 200 images are taken.

 figure: Fig. 9.

Fig. 9. The 1D target. (a) The full picture of the 1D target. (b) One captured image and the feature points detecting.

Download Full Size | PDF

We evaluate the calibration performance by varying the number of images from 10 to 80. For each number, 100 trials are executed with randomly choosing from the available image set.

The relative errors are reported in Fig. 10. When the number of images is over 40, all relative errors of the CDCA is below 0.1%. Distinctly, the CDCA provides highest-accuracy 1D camera calibration. Compared with the computer simulation result in Fig. 3, the relative errors of CDCA are slightly larger, and the convergence value of the CDCA has a little deviation from the reference value in this real data experiment. Even so, the CDCA is precise enough for the needs of practical application.

 figure: Fig. 10.

Fig. 10. The calibration errors with respect to the number of images in the real data experiment

Download Full Size | PDF

The reason may be that the fixed rotary point depth varies slightly due to the mechanical rotation, and there is still a bit of inaccuracy in 2D calibration. Hence, we evaluate the influence of the depth variation on calibration accuracy here. The simulation settings are set as same as the mentioned in Section 4.1.1, and the disturbance of Gaussian distribution with zero mean is added to the translation vector T. We test the calibration accuracy of CDCA under different standard deviation from 0.1mm to 0.5mm. As shown in Fig. 11, the displacement of fixed point greatly influences the calibration accuracy. The performance of CDCA in Fig. 10 is close to the simulated result with the standard deviation of 0.2mm. It is roughly consistent with the actual mechanical error of the supporting point on our self-made 1D target.

 figure: Fig. 11.

Fig. 11. The calibration errors under different variations of translation vector

Download Full Size | PDF

5. Conclusions

In this paper, a high-precision calibration algorithm CDCA is presented, so that 1D target can be actually utilized in monocular vision well. Considering that the accurate calibration can be achieved in the case of perfect distortion correction, we focus on the cyclic optimization of distortion parameters, instead of the habitual global nonlinear optimization strategy. This leads to that the final calibration accuracy depends almost exclusively on the closed-form solution. Therefore, we adopt the closed-form solution of HBLA and improve it further by optimally weighting the constraint equations. Extensive experiments show that the CDCA provides the highest calibration accuracy and robustness. The desirable distortion parameters can also be estimated for image correction by this proposed algorithm. Even though the CDCA is proposed for 1D target here, its idea can be extend to 2D calibration of large FOV.

Funding

National Natural Science Foundation of China (62073116).

Acknowledgments

We acknowledge Hefei University of Technology for the support in this research.

Disclosures

The authors declare no conflicts of interest.

Data availability

Data underlying the results presented in this paper are not publicly available at this time but may be obtained from the authors upon reasonable request.

References

1. Z. Zhang, “A flexible new technique for camera calibration,” IEEE Trans. Pattern Anal. Mach. Intell. 22(11), 1330–1334 (2000). [CrossRef]  

2. S. Ramalingam and P. Sturm, “A unifying model for camera calibration,” IEEE Trans. Pattern Anal. Mach. Intell. 39(7), 1309–1319 (2017). [CrossRef]  

3. Y. Zhang, W. Liu, and F. Wang, “Improved separated-parameter calibration method for binocular vision measurements with a large field of view,” Opt. Express 28(3), 2956–2974 (2020). [CrossRef]  

4. B. Chen and B. Pan, “Calibration-free single camera stereo-digital image correlation for small-scale underwater deformation measurement,” Opt. Express 27(8), 10509–10523 (2019). [CrossRef]  

5. Z. Zhang, “Camera calibration with one-dimensional objects,” IEEE Trans. Pattern Anal. Mach. Intell. 26(7), 892–899 (2004). [CrossRef]  

6. P. Hammarstedt, P. Sturm, and A. Heyden, “Degenerate cases and closed-form solutions for camera calibration with one-dimensional objects,” in Proceedings of the Tenth IEEE International Conference on Computer Vision (ICCV) (2005), pp. 317–324.

7. F. Wu, Z. Hu, and H. Zhu, “Camera calibration with moving one-dimensional objects,” Pattern Recognit. 38(5), 755–765 (2005). [CrossRef]  

8. F. Qi, Q. Li, Y. Luo, and D. Hu, “Camera calibration with one-dimensional objects moving under gravity,” Pattern Recognit. 40(1), 343–345 (2007). [CrossRef]  

9. E. Peng and L. Li, “Camera calibration using one-dimensional information and its applications in both controlled and uncontrolled environments,” Pattern Recognit. 43(3), 1188–1198 (2010). [CrossRef]  

10. I. Miyagawa, H. Arai, and H. Koike, “Simple camera calibration from a single image using five points on two orthogonal 1-D objects,” IEEE Trans. Image Process. 19(6), 1528–1538 (2010). [CrossRef]  

11. J. A. de França, M. R. Stemmerb, M. B. de M. França, and E. G. Alves, “Revisiting Zhang's 1D calibration algorithm,” Pattern Recognit. 43(3), 1180–1187 (2010). [CrossRef]  

12. K. Shi, Q. Dong, and F. Wu, “Weighted similarity-invariant linear algorithm for camera calibration with rotating 1-D objects,” IEEE Trans. Image Process. 21(8), 3806–3812 (2012). [CrossRef]  

13. L. Wang, F. Duan, and K. Lu, “An adaptively weighted algorithm for camera calibration with 1D objects,” Neurocomputing 149, 1552–1559 (2015). [CrossRef]  

14. Y. Lv, W. Liu, and X. Xu, “Methods based on 1D homography for camera calibration with 1D objects,” Appl. Opt. 57(9), 2155–2164 (2018). [CrossRef]  

15. J. Sun, Q. Liu, Z. Liu, and G. Zhang, “A calibration method for stereo vision sensor with large FOV based on 1D targets,” Opt. Lasers Eng. 49(11), 1245–1250 (2011). [CrossRef]  

16. Y. Li, J. Huo, J. Liu, and M. Yang, “Calibration of quad-camera measurement systems using a one-dimensional calibration object for three-dimensional point reconstruction,” Opt. Eng. 58(6), 064107 (2019). [CrossRef]  

17. T. Jiang, X. Cheng, and H. Cui, “Calibration method for binocular vision with large FOV based on normalized 1D homography,” Optik 202, 163556 (2020). [CrossRef]  

18. B. Halloran, P. Premaratne, and P. J. Vial, “Robust one-dimensional calibration and localisation of a distributed camera sensor network,” Pattern Recognit. 98, 107058 (2020). [CrossRef]  

19. J. Heikkila and O. Silvén, “A four-step camera calibration procedure with implicit image correction,” in Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR) (1997) pp. 1106–1112.

20. C. Ricolfe-Viala and A. J. Sanchez-Salmeron, “Robust metric calibration of non-linear camera lens distortion,” Pattern Recognit. 26(2), 269–277 (2010). [CrossRef]  

21. Y. Lv, W. Liu, B. Du, and X. Xu, “Further Study and Application for One-Dimensional Homography Matrix,” Acta Opt. Sin. 38(7), 0715003 (2018). [CrossRef]  

22. R. Hartley and S. B. Kang, “Parameter-free radial distortion correction with center of distortion estimation,” IEEE Trans. Pattern Anal. Mach. Intell. 29(8), 1309–1321 (2007). [CrossRef]  

23. J. Weng, P. Cohen, and M. Herniou, “Camera calibration with distortion models and accuracy evaluation,” IEEE Trans. Pattern Anal. Mach. Intell. 14(10), 965–980 (1992). [CrossRef]  

24. Z. Qi, L. Xiao, and S. Fu, “Two-step camera calibration method based on the SPGD algorithm,” Appl. Opt. 51(26), 6421–6428 (2012). [CrossRef]  

Data availability

Data underlying the results presented in this paper are not publicly available at this time but may be obtained from the authors upon reasonable request.

Cited By

Optica participates in Crossref's Cited-By Linking service. Citing articles from Optica Publishing Group journals and other participating publishers are listed here.

Alert me when this article is cited.


Figures (11)

Fig. 1.
Fig. 1. An illustration of 1D camera calibration
Fig. 2.
Fig. 2. The 1D calibration in the case of no distortion: (a) an image set of the 1D target with 10 rotations;(b) the reprojection errors of different closed-form solutions; (c)-(f) the relative errors of the four intrinsic parameters
Fig. 3.
Fig. 3. Comparison of calibration errors between the original and improved closed-form solutions
Fig. 4.
Fig. 4. The algorithm procedure of CDCA
Fig. 5.
Fig. 5. The calibration errors with respect to the number of images
Fig. 6.
Fig. 6. The calibration errors with respect to the noise level
Fig. 7.
Fig. 7. The calibration errors with respect to the offset between the principal point and the image center
Fig. 8.
Fig. 8. The calibration errors with respect to the distortion level
Fig. 9.
Fig. 9. The 1D target. (a) The full picture of the 1D target. (b) One captured image and the feature points detecting.
Fig. 10.
Fig. 10. The calibration errors with respect to the number of images in the real data experiment
Fig. 11.
Fig. 11. The calibration errors under different variations of translation vector

Tables (2)

Tables Icon

Table 1. Basic simulation settings

Tables Icon

Table 2. The calibration result of one calibration trial involving 20 images

Equations (23)

Equations on this page are rendered with MathJax. Learn more.

z ˙ [ m 1 ] = K [ R|T ] [ P 1 ] , with  K = [ α 0 u 0 0 β v 0 0 0 1 ] ,
{ u d u 0 = ( u u 0 ) ( 1 + ρ 1 r 2 + ρ 2 r 4 ) v d v 0 = ( v v 0 ) ( 1 + ρ 1 r 2 + ρ 2 r 4 ) r 2 = ( u u 0 ) 2 + ( v v 0 ) 2 ,
z ˙ n j [ m n j 1 ] = K [ r 1 j T ] [ x n 1 ]
{ r 1 j = t z K 1 h 1 j T   = t z K 1 h 2 j ,  with  H j = [ h 1 j h 2 j ] = [ h 1 j h 4 j h 2 j h 5 j h 3 j 1 ] .
[ x n 0 u n j x n 1 0 u n j 0 x n v n j x n 0 1 v n j ] h H   j = 0 .
t z 2 h 1 j T K  - T K  - 1 h 1 j = 1   ( j = 1 , 2 , 3 , J ) .
W = t z 2 K T K 1 = [ W 11 0 W 13 0 W 22 W 23 W 13 W 23 W 33 ] ,
V j ξ = 1 ,  with { V j = [ h 1 j 2   h 2 j 2   2 h 1 j h 3 j   2 h 2 j h 3 j   h 3 j 2 ]   ξ   = [ W 11   W 22   W 13   W 23   W 33 ] T .
min f ( ρ , K , r 1 j , T ) = j = 1 J n = 0 N | | m n j m ^ n j ( ρ , K , r 1 j , T , P n ) | | 2 ,
min   f ( ρ ) = j = 1 J n = 1 N | | m n j m ^ ( ρ , [ u 0 , v 0 ] , P n , H j ( ρ ) ) | | 2 ,
h 1 j T Wh 1 j = 1 ,  with W = t z 2 K T K 1 .
ε j = h 1 j T Wh 1 j 1.
d ε j = 2 h 1 j T W d h 1 j .
{ u n j = h 1 j x n + h 4 j h 3 j x n + 1 v n j = h 2 j x n + h 5 j h 3 j x n + 1 .
d m n j = [ x n h 3 j x n + 1 0 ( h 1 j x n + h 4 j ) x n ( h 3 j x n + 1 ) 2 0 x n h 3 j x n + 1 ( h 2 j x n + h 5 j ) x n ( h 3 j x n + 1 ) 2 ] d h 1 j ,
d m Σ j = [ d m 1 j T  d m 2 j T  d m n j T ] T .
d m Σ j = S j d h 1 j ,  with S j = [ x 1 h 3 j x 1 + 1 0 ( h 1 j x 1 + h 4 j ) x 1 ( h 3 j x 1 + 1 ) 2 0 x 1 h 3 j x 1 + 1 ( h 2 j x 1 + h 5 j ) x 1 ( h 3 j x 1 + 1 ) 2 x 2 h 3 j x 2 + 1 0 ( h 1 j x 2 + h 4 j ) x 2 ( h 3 j x 2 + 1 ) 2 0 x 2 h 3 j x 2 + 1 ( h 2 j x 2 + h 5 j ) x 2 ( h 3 j x 2 + 1 ) 2 x N h 3 j x N + 1 0 ( h 1 j x N + h 4 j ) x N ( h 3 j x N + 1 ) 2 0 x N h 3 j x N + 1 ( h 2 j x N + h 5 j ) x N ( h 3 j x N + 1 ) 2 ] .
d h 1 j = ( S j T S j )  - 1 S j T d m Σ j .
d ε j = 2 h 1 j T W ( S j T S j )  - 1 S j T d m Σ j .
SD ( ε j ) = | | 2 h 1 j T W ( S j T S j )  - 1 S j T | | δ 2 .
w j = 1 | | h 1 j T W ( S j T S j )  - 1 S j T | | .
w j V j ξ = w j .
min f ( ρ , K , r 1 j , T j ) = j = 1 J n = 1 N | | m n j m ^ n j ( ρ , K , r 1 j , T j , P n ) | | 2 .
Select as filters


Select Topics Cancel
© Copyright 2024 | Optica Publishing Group. All rights reserved, including rights for text and data mining and training of artificial technologies or similar technologies.