A high precision 3D reconstruction method for bend tube axis based on binocular stereo vision

Junhua Sun; Yue Zhang; Xiaoqi Cheng

doi:10.1364/OE.27.002292

1. Introduction

Pipelines are the key parts of electro-mechanical products in aeronautics and astronautics field. It commonly consists of complex metal bent tubes produced by tube benders. Although every tube is made according to its CAD model, guaranteeing its manufacture precision is a great challenge due to spring-back and many disturbances during the fabricating progress. For successful and non-stress assembly, processed tubes are required to be measured and compared with their CAD models to evaluate whether their assembly demands are satisfied. Therefore fast and precise 3D bent tube measurement is quite important.

Contact measuring instruments such as coordinate measuring machine [1] (CMM) and mechanical gauges [2] are the main approaches for tube measurement in early stage. CMM has high precision but its operation is quite complex. Besides, the contact between the probe and tube surface may lead to partial deformation. Modeling method needs to fabricate molds for tubes with their CAD models in advance causing high cost and poor universality. Metronor light pen [3] is a new 3D reconstruction equipment combing stereo vision and tactile measurement, but moving the probe to scan the whole surface leads to a complicated operation.

In the past few years, stereo vision has been widely studied [4] and applied in the field of non-contact measurement [5] owing to its multiple advantages such as high accuracy, high efficiency, non-contact and automation. Therefore, many methods of measuring bent tube based on stereo vision have emerged. These automatic propositions can mainly be divided into two parts, bent tube surface measurement and space axis reconstruction. GOM ATOS Core 3D scanner [6] is a typical representative of measuring tube surface based on binocular stereo vision with the accuracy of 0.02 mm. However, it has the problem of poor efficiency because of 3D data registration. Bosemann [7] mentioned that AICON [8] provided a monocular vision method using photogrammetry. Though this method has high accuracy, it requires a scale benchmarking to provide constraint condition and the automation of the measurement is weak.

Compared with surface measurement, tube axis reconstruction has the superiority of full automation. So it has become an important direction that the scholars and companies paid great attention to. AICON developed TubeInspect [9] pipeline measuring system with the accuracy of 0.1mm. T. Zhang and P. Jin [10–13] used the center lines of image contours together with stereo vision to reconstruct 3D axis within 0.2s and the accuracy is nearly equal to AICON. Liu [14] proposed an accurate way to measure the position of pipe end faces that is a supplement for T. Zhang. Nevertheless, the above methods regarded the center lines of tube contours on image plane as the projected position of axis. This assumption is just a kind of approximate calculation with a great systemic error, especially for tubes with large diameter. Figure 1 describes that there exists two rays passing through the optical center and intersecting with an arbitrary cross-section of bent tube at two points of tangency $P_{1}$ and $P_{2}$ . These two rays intersects the image plane at the points $p_{1}$ and $p_{2}$ , which are the projection points of $P_{1}$ and $P_{2}$ . In fact, the image plane is usually not perpendicular to the cross section, so the projection point $c$ of cross section center $C$ is not the midpoint of line $p_{1} p_{2}$ on image plane. Considering that the contour axis is the midpoint of each pair of projection points $p_{1}$ and $p_{2}$ , thus the projection position of tube axis is deviated from the image contour axis.

Fig. 1 Projection error of bent tube axis.

Download Full Size | PDF

Accurately modeling the projecting process of bent tube axis is an effective way to improve the reconstruction precision. Ascending to 1990s, plenty of researchers have studied the perspective projection model of bent tubes, while most results focused on straight homogeneous circular cylinders or surface of revolutions [15–17]. There were only a few researches dealing with bent tubes: Boseman [18] divided the tube axis into several portions, and treated each segment as straight lines. It caused a considerable margin of error because the major parameters are measured via junctions of straight lines. Lee [19,20] mapped the surface points to axis through local geometric properties around points and fitted axis, solving the problem of straight-line approximation. But it needs a point cloud from pip surface. Caglioti [21] came up with a way using the geometric properties of tube surface to reconstruct a full 3D reconstruction of the shape from a single perspective image or even a fully-uncalibrated image under exact perspective. It required the radius of tube to be known. Although the above-mentioned means provided some perspective models for straight or bent tubes, there exists some matters like large errors or demanding more constraint conditions.

Addressing the benefits and disadvantages of existing measurement methods, this work utilizes binocular stereo vision to reconstruct bent tube axis. An accurate model for projective process of space axis and a way for calculating the precise projected position of axis on image plane are proposed. The method can further improve the precision of axis measurement on the basis of keeping user-friendly control. Besides, the radius of tube is not needed in advance.

2. Proposed approach

The bent tube surface can be treated as a family of circular cross-section with constant radius R, and each cross-section center lies on the axis of tube. Therefore, constructing the perspective projection model of any cross section and calculating the projected position of every cross-section center can obtain the whole projected position of axis.

2.1 perspective projection of an arbitrary cross section

Figure 2 depicts the perspective projection of an arbitrary cross-section $π$ on a bent tube. $π$ is a circle located on a supporting plane P. $n$ is the normal vector of plane P, intersecting at the cross-section center $P_{c}$ . Planes $T_{1}$ and $T_{2}$ are the tangent planes to the tube surface at point $P_{1}$ and $P_{2}$ separately, passing through the optical center O. Point $P_{3}$ is the center of the chord $P_{1} P_{2}$ . $O Q$ is the intersection line of planes $T_{1}$ and $T_{2}$ .Plane $T_{m i d}$ bisects the two planes $T_{1}$ and $T_{2}$ , passing through the line $O Q$ and the point $P_{c}$ . The points $P_{1}^{'}$ and $P_{2}^{'}$ are the projections of the points $P_{1}$ and $P_{2}$ respectively. Likewise, the same correlation applies to the point $P_{3}^{'}$ and the point $P_{3}$ , as well as the point $P_{c}^{'}$ and the point $P_{c}$ . The planes $T_{1}$ and $T_{2}$ separately intersects the image plane at the lines $t_{1}^{'}$ and $t_{2}^{'}$ . $V_{h}$ is the intersection point of the lines $t_{1}^{'}$ and $t_{2}^{'}$ , that can also be regarded as the intersection point of the line $O Q$ and the image plane.

Fig. 2 Perspective projection model of arbitrary cross section $n$ .

Download Full Size | PDF

The plane P is given by ${(n^{T}, - d)}^{T}$ , where $n$ is ${(n_{1}, n_{2}, n_{3})}^{T}$ and d is the distance from the plane P to O. Each point $x_{i}$ lying on the cross-section $π$ satisfies the equation $n^{T} x_{i} = d$ . As shown in Fig. 2, since the planes $T_{1}$ and $T_{2}$ are the tangent planes to tube surface, they are perpendicular to the plane P and the line $O Q$ is perpendicular to the plane P. So the line $O Q$ and the vector $n$ have the same direction in space. According to the definition of vanishing point in ([21,22]), the intersecting point $V_{h}$ for the lines $t_{1}^{'}$ and $t_{2}^{'}$ is the vanishing point for vector $n$ on the image plane. The point $V_{h}$ is given by $V_{h} = t_{1}^{'} \times t_{2}^{'}$ , and the vector $n$ can be written as $n = O V_{h} = A^{- 1} {\tilde{V}}_{h}$ , where ${\tilde{V}}_{h} = {(V_{h}, 1)}^{T}$ .

On the basis of camera imaging model, the points $P_{1}$ and $P_{1}^{'}$ satisfy the relationship shown in Eq. (1),

{\tilde{P}}_{1} = k_{1} {[\begin{matrix} A^{- 1} & 0 \end{matrix}]}^{T} {\tilde{P}}_{1}^{'} + {[\begin{matrix} 0 & 0 & 0 & 1 \end{matrix}]}^{T}

where

k_{1}

stands for a scale factor,

Α

is the camera matrix,

{\tilde{P}}_{1} = {(X_{1}, Y_{1}, Z_{1}, 1)}^{T}

and

{\tilde{P}}_{1}^{'} = {(x_{1}, y_{1}, 1)}^{T}

.

B = A^{- 1} = [\begin{matrix} b_{11} & b_{12} & b_{13} \\ b_{21} & b_{22} & b_{23} \\ b_{31} & b_{32} & b_{33} \end{matrix}]

Let the inverse matrix of

Α

be written as Eq. (2). Then Eq. (1) is written as Eq. (3).

[\begin{array}{l} X_{1} \\ Y_{1} \\ Z_{1} \end{array}] = [\begin{array}{l} k_{1} (b_{11} x_{1} + b_{12} y_{1} + b_{13}) \\ k_{1} (b_{21} x_{1} + b_{22} y_{1} + b_{23}) \\ k_{1} (b_{31} x_{1} + b_{32} y_{1} + b_{33}) \end{array}] = [\begin{array}{l} k_{1} γ_{11} \\ k_{1} γ_{12} \\ k_{1} γ_{13} \end{array}]

Because the point

P_{1}

lies on the plane P, it also satisfies

n^{T} P_{1} = d

. Therefore the coordinates of

P_{1}

takes the form as Eq. (4).

[\begin{array}{l} X_{1} \\ Y_{1} \\ Z_{1} \end{array}] = [\begin{array}{l} γ_{11} / (n_{1} γ_{11} + n_{2} γ_{12} + n_{3} γ_{13}) d \\ γ_{12} / (n_{1} γ_{11} + n_{2} γ_{12} + n_{3} γ_{13}) d \\ γ_{13} / (n_{1} γ_{11} + n_{2} γ_{12} + n_{3} γ_{13}) d \end{array}] = [\begin{array}{l} η_{11} d \\ η_{12} d \\ η_{13} d \end{array}]

The point $P_{2}$ can be expressed as ${(η_{21} d, η_{22} d, η_{23} d)}^{T}$ due to symmetry. The above derivation process introduces the perspective projection model for the points $P_{1}$ and $P_{2}$ . In fact, each cross-section can only be projected to two points on image plane except for end faces. Considering there are some geometric constraints between the points $P_{1}, P_{2}$ and $P_{c}$ , the projected position on image plane of the point $P_{c}$ could be obtained by the points $P_{1}^{'}, P_{2}^{'}$ . So when we compute the projected position of an axis, the above model can represent the projective process of every cross-section, including two end faces.

2.2 Projective position of cross section center

The planes $T_{1}$ and $T_{2}$ can be regarded as the back projection planes of the lines $P_{1}^{'} V_{h}$ and $P_{2}^{'} V_{h}$ (Fig. 3) respectively, so the normal vectors of these two planes can be expressed as $n_{1} = A^{T} ({\tilde{P}}_{1}^{'} \times {\tilde{V}}_{h}) = {(T_{11}, T_{12}, T_{13})}^{T}$ and $n_{2} = A^{T} ({\tilde{P}}_{2}^{'} \times {\tilde{V}}_{h}) = {(T_{21}, T_{22}, T_{23})}^{T}$ . Then since the plane $T_{m i d}$ bisects the planes $T_{1}$ and $T_{2}$ , a Lagrange function is given by $T_{m i d} = T_{1} + κ T_{2}$ with $T_{1} = {[n_{1}, 0]}^{T}$ and $T_{2} = {[n_{2}, 0]}^{T}$ . We define the normal vector of plane $T_{m i d}$ as $n_{m i d} = {(T_{31}, T_{32}, T_{33})}^{T}$ , and solve the appropriate value of $κ$ that satisfies the equation $(n_{1} / ‖ n_{1} ‖ - n_{2} / ‖ n_{2} ‖) n_{m i d} = 0$ . Then the expression of the vector $n_{m i d}$ can be given as $n_{m i d} = n_{1} / ‖ n_{1} ‖ + n_{2} / ‖ n_{2} ‖$ .

Fig. 3 Projected process of coupled contour points.

Download Full Size | PDF

Similarly to Eq. (1), the point $P_{c}$ could be written as Eq. (5):

{\tilde{P}}_{c} = k_{c} {[\begin{matrix} B & 0 \end{matrix}]}^{T} {\tilde{P}}_{c}^{'} + {[\begin{matrix} 0 & 0 & 0 & 1 \end{matrix}]}^{T}

Located on the planes P and

T_{m i d}

, the point

P_{c}

satisfies

n^{T} P_{c} = d

and

T_{m i d} {\tilde{P}}_{c} = 0

, where

T_{m i d} = {[n_{m i d}, 0]}^{T}

. Moreover, since the point

P_{c}

is the center of the cross-section

π

, there exists the condition

‖ P_{c} - P_{1} ‖ = ‖ P_{c} - P_{2} ‖

. Combining Eq. (1)~(5), a new formula is obtained as Eq. (6),

[\begin{matrix} k_{c} n_{1} & k_{c} n_{2} & k_{c} n_{3} \\ T_{31} & T_{32} & T_{33} \\ k_{c} (η_{11} - η_{21}) & k_{c} (η_{12} - η_{22}) & k_{c} (η_{13} - η_{23}) \end{matrix}] [\begin{matrix} θ_{1} \\ θ_{2} \\ θ_{3} \end{matrix}] = d [\begin{matrix} 1 \\ 0 \\ Ω \end{matrix}]

where the definition of some parameters in Eq. (6) is shown in Eq. (7). Other definition of parameters

(η_{11}, η_{12}, η_{13}, η_{21}, η_{22}, η_{23})

are defined in Eq. (4) and its dual equation.

{\begin{cases} B {\tilde{P}}_{c}^{'} = {[\begin{matrix} θ_{1} & θ_{2} & θ_{3} \end{matrix}]}^{T} \\ 1 / 2 (η_{11}^{2} - η_{21}^{2} + η_{12}^{2} - η_{22}^{2} + η_{13}^{2} - η_{23}^{2}) = Ω \end{cases}

To further simplify Eq. (6), the variables of the last row in Eq. (6) is written as Eq. (8).

{[\begin{matrix} ψ_{1} & ψ_{2} & ψ_{3} \end{matrix}]}^{T} = n - \frac{1}{Ω} {[\begin{matrix} η_{11} - η_{21} & η_{12} - η_{22} & η_{13} - η_{23} \end{matrix}]}^{T}

Finally a succinct equation for solving the coordinates of the point $P_{c}$ is given by Eq. (9) with ${\tilde{P}}_{c} = {(x_{c}, y_{c}, 1)}^{T}$ .

([\begin{array}{l} \begin{matrix} ψ_{1} & ψ_{2} & ψ_{3} \end{matrix} \\ \begin{matrix} T_{31} & T_{32} & T_{33} \end{matrix} \end{array}] B) {\tilde{P}}_{c}^{'} = 0

2.3 Image properties of arbitrary cross section

Caglioti [19] has provided the coupling way for contour points, making it possible to determine which pairs of imaging points are actually coupled for a cross-section. The points $P_{1}^{'}$ and $P_{2}^{'}$ are coupled only if the angle $P_{1}^{'} \hat{o} V_{h}$ coincides with the angle $P_{2}^{'} \hat{o} V_{h}$ , so it could be enforced as Eq. (10),

\frac{{\tilde{P}}_{1}^{'}^{T} ω {\tilde{V}}_{h}}{\sqrt{{\tilde{P}}_{1}^{'}^{T} ω {\tilde{P}}_{1}^{'}} \sqrt{{\tilde{V}}_{h}^{T} ω {\tilde{V}}_{h}}} = \frac{{\tilde{P}}_{2}^{'}^{T} ω {\tilde{V}}_{h}}{\sqrt{{\tilde{P}}_{2}^{'}^{T} ω {\tilde{P}}_{2}^{'}} \sqrt{{\tilde{V}}_{h}^{T} ω {\tilde{V}}_{h}}}

where

ω

is the absolute conic of the image, with

ω = A^{- T} A^{- 1}

. Then a loss function to estimate the difference between the angles

P_{1}^{'} \hat{o} V_{h}

and

P_{2}^{'} \hat{o} V_{h}

is constructed as Eq. (11).

J_{c} (P_{1}^{'}, P_{2}^{'}) = {[arc \cos (\frac{{\tilde{P}}_{1}^{'}^{T} ω {\tilde{V}}_{h}}{\sqrt{{\tilde{P}}_{1}^{'}^{T} ω {\tilde{P}}_{1}^{'}} \sqrt{{\tilde{V}}_{h}^{T} ω {\tilde{V}}_{h}}}) - arc \cos (\frac{{\tilde{P}}_{2}^{'}^{T} ω {\tilde{V}}_{h}}{\sqrt{{\tilde{P}}_{2}^{'}^{T} ω {\tilde{P}}_{2}^{'}} \sqrt{{\tilde{V}}_{h}^{T} ω {\tilde{V}}_{h}}})]}^{2}

At first, applying the contour extracting method presented in literature [21], we can get both the sub-pixel contours of tube image and the normal vectors for each point on contours. After that we subdivid the edge points into two chains located on the facing contours and found an initial pair of coupled points. Starting from this pair, other pairs are found incrementally by searching along the points nearby the last pair and choosing the one minimizing Eq. (11).

2.4 3D reconstruction of bent tube axis

Figure 4 shows the construction process using binocular stereo. Firstly we calculate the projected positions for tube axis on two image planes. Then the epipolar geometry is used to search the corresponding image points in two views. Finally for each pair of correlative points, we reconstruct the point in space that is projected to these two image points and got the whole axis.

Fig. 4 Procedure of bent tube axis reconstruction.

Download Full Size | PDF

3. Experiments and analysis

The measurement system is shown in Fig. 5, consisting of binocular stereo visual sensor, a background light source and a computer. The binocular stereo visual sensor comprises two industrial GC1380H cameras with resolution of $1360 \times 1024$ pixels and two Schneider Cinegon 1.4/12mm lenses. In order to evaluate our method, every testing tube is measured by GOM ATOS core 300 scanner, with a precision of 0.02 mm. Each group of experiment results is compared with the data from the scanner. The calibration results of binocular stereo visual sensors are as Table 1.

Fig. 5 Binocular vision measurement system.

Download Full Size | PDF

Table 1. Calibration results of binocular stereo visual sensors

View Table | View all tables in this article

3.1 Result using method with axis projection model

Three tubes with different diameters but with two bending angles are measured for several times. To quantify our measuring results, we first reconstructed their axes by the approach described in Section 2, and then separated each axis into straight line sections and arc sections and fitted these sections separately. The angle between any two adjacent lines is regarded as the bending angle of tube, and the radius of arc is treated as the bending radius of tube. Based on this, bending angle and radius are the parameters to be measured in the following experiments.

3.1.1 Results for tube with diameter about 6 mm

In this section we measured a bent tube with diameter about 6 mm, and the tube is shown in Fig. 6(a). The metrical data of GOM scanner on two bending angles are 112.25° and 99.79°, and the results of two radius are 25.17 mm and 24.82 mm.

Fig. 6 Result for tube with diameter about 6 mm. (a) Tube to be measured; (b) Reconstruction result.

Download Full Size | PDF

The tube was repeatedly measured 10 times in the same surroundings but with different visual angles. The reconstruction result is shown in Fig. 6(b), and the detail data is listed in Table 2. From the second to the last line in Table 2, the root mean square error (RMSE) of two bending angles are 0.020° and 0.020°, and the RMSE of two radius are 0.022 mm and 0.021 mm. The above results vividly indicated the high measurement repeatability accuracy and stability of the system. Compared with the results given by the GOM scanner, the root mean square deviation (RMSD) of bending angles are 0.030° and 0.031°, and the RMSD of radius are 0.029 mm and 0.029 mm.

Table 2. Results of the tube with about 6 mm diameter

View Table | View all tables in this article

3.1.2 Result for tube with diameter about 15 mm

Next we measured a thicker tube with diameter about 15 mm displayed in Fig. 7(a).The GOM measuring data are 100.64° and 122.45° for bending angles and 63.23 mm and 65.69 mm for radius. Figure 7(b) and Table 3 show the reconstruction results together. Compared with the results of the GOM scanner, the RMSD of bending angles are 0.031° and 0.029°, and the RMSD of radius are 0.028 mm and 0.029 mm.

Fig. 7 Result for tube with diameter about 15 mm. (a) Tube to be measured; (b) Reconstruction result.

Download Full Size | PDF

Table 3. Results of the tube with about 15 mm diameter

View Table | View all tables in this article

3.1.3 Result for tube with diameter about 24 mm

Finally we measured the thickest tube with diameter about 24 mm shown in Fig. 8(a). The GOM results for bending angles are 87.95° and 108.90°, and radius are 72.32 mm and 72.67 mm. The specific measurements are shown in Fig. 8(b) and Table 4. The RMSD of bending angles are 0.027° and 0.031°, and the RMSD of radius are 0.026 mm and 0.029 mm.

Fig. 8 Result for tube with diameter about 24 mm. (a) Tube to be measured; (b) Reconstruction result.

Download Full Size | PDF

Table 4. Results of the tube with about 24 mm diameter

View Table | View all tables in this article

The results above dedicate that for bent tubes with different diameters, our method is not affected by the factor of tube diameter. Among these results, the RMSD of bending angles is within 0.031°, and the RMSD of radius is within 0.031 mm. So the measuring performance of our method is excellent.

3.2 Result using method without axis projection model

We also reconstructed those tubes by the method using contour center lines as the projected position of tube axis and made a comparison between the results gained from the traditional method and our method. According to Jin’s method [9] we got the contour center lines in images and reconstructed 3D axes of three tubes. A serious of results are presented as follows.

3.2.1 Result for tube with the diameter about 6 mm

Table 5 reveals the results of the first tube measured by the method without perspective projection model. Contrast with the data measured from the GOM scanner, the RMSD of two bending angles are 0.035° and 0.034°, and the RMSD of radius are 0.035 mm and 0.032 mm.

Table 5. Results of the tube with about 6 mm diameter by traditional method

View Table | View all tables in this article

It’s obvious that for this method the measurement precision are lower than our approach, while the repeatability accuracy is similar to our method. Table 6 lists the comparison results of two methods on the same tube, where the method 1 represents for our method and the method 2 stands for the traditional method. The last line in the table provides the percentage of precision improvement that indicates the measurement accuracy of the first tube can be raised by 14.4% in average after adding a perspective projection model.

Table 6. Comparison results of two methods

View Table | View all tables in this article

3.2.2 Result for tube with diameter about 15 mm

Then we did the same experiment as subsection 3.2.1 on the second tube and gained the results in Table 7. Two bending angles have the RMSD of 0.037° and 0.038°, and the RMSD of radius are 0.042 mm and 0.038 mm.

Table 7. Results of the tube with about 15mm diameter by traditional method

View Table | View all tables in this article

Similar to the conclusion from the last experiment, our method has a higher measurement precision than the traditional method while maintaining the same repeatability accuracy. Table 8 implies that the measuring accuracy can be improved by 30.4% on average.

Table 8. Comparison results of two methods

View Table | View all tables in this article

3.2.3 Result for tube with diameter about 24 mm

Finally the identical content is repeated on the third tube. From Table 9 the RMSD of bending angles are 0.040° and 0.044°, and the RMSD of radius are 0.041 mm and 0.041 mm.

Table 9. Results of the tube with about 24 mm diameter by traditional method

View Table | View all tables in this article

The similar conclusion is gained as the foregoing discussion. From Table 10 the measuring accuracy is increased by 47.3% on average.

Table 10. Comparison results of two methods

View Table | View all tables in this article

The above details turns out that some system error can be corrected by adding the perspective projection model. Contrast with the traditional method, our method is able to decrease the root mean square deviation of the measurements and ensures the high repeatability accuracy. Furthermore, the reconstruction precision decreases as the diameter increases in traditional method, but our method is not sensitive to the changing of diameter. So when the diameter is larger, the proposed method behaves better in improving the measuring precision.

3.3 The influence of image noise on reconstruction results

To the test images, we add several Gaussian noises with mean 0 and varying standard deviations (SD) in same pictures. Using these pictures, a series of new 3d reconstruction results can be obtained and the biases between GOM results and new results are used to evaluate the influence of noises. For each noise level, we repeated the experiment for ten times. The final average measurement results of four key parameters are shown in Fig. 9. As can be seen, the RMSD increases slightly as the SD increases when the SD is lesser than 20. But after 20, the RMSD increases significantly. This is mainly because the greater image noise can flush image details. Under the high noise level, the loss of small regions feature is great. So it is hard to detect the accurate contour, and the fuzzy contour will result in large deviation of the final measured value.

Fig. 9 Measurement results of bent tube images with noise.

Download Full Size | PDF

4. Conclusion

This paper proposes a high precision bent tube axis reconstruction method based on the accurate calculation of the projected image position of bent tube space axis. The method can reduce the error caused by approximately estimating the projected position of tube axis. Meanwhile, binocular stereo vision requires no external constraint conditions and has high precision compared with monocular reconstruction. The proposed method is able to calculate the 3D coordinates of points located on tube axis without knowing tube radius in advance.

In contrast experiments, three tubes with different diameters but two corners were measured. The experimental results show that the root mean square deviation to the GOM scanner is reduced after appending the projection model while keeping a high repeatability accuracy. The average measurement accuracy of the tube with the diameter about 6 mm could be increased by 14.4%. The similar results occurred on rest tubes: the tube with about 15 mm diameter was raised by 30.4% and the one with about 24 mm diameter was improved by 47.3%. A better conclusion comes about that the measurement precision of our method is practically independent of tube diameter. The proposed method will heighten the correction effect of reconstruction error as the diameter increases.

The method can also be applied to multi-camera vision systems based on reconstructing tube axis. This cannot only improve the veracity of bent tube reconstruction but also solve the problem that the information of tube can’t be obtained completely at some angles.

Funding

Aeronautical Science Foundation of China 2017ZE51062.

References

1. X. Ji and G. Gong, “Study of Realization Technologies for 3D Reconstruction of Superchargers′ Impellers,” J. Mech. Sci. Technol. 25(3), 322–325 (2006).

2. Y. Wei and A. Thornton, “Tube production and assembly systems: the impact of compliance and variability on yield,” presented at Design Engineering Technical Conferences and Computer and Information in Engineering Conference, Baltimore, Maryland, 10–13 Sept. (2000).

3. Extreme Networks white paper, “Virtual metropolitan area networks,” (Extreme Networks, 2001), http://www.extremenetworks.com/technology/whitepapers/vMAN.asp.

4. F. Gu, H. Zhao, X. Zhou, J. Li, P. Bu, and Z. Zhao, “Photometric invariant stereo matching method,” Opt. Express 23(25), 31779–31792 (2015). [CrossRef] [PubMed]

5. Z. Liu, X. Li, F. Li, and G. Zhang, “Flexible dynamic measurement method of three-dimensional surface profilometry based on multiple vision sensors,” Opt. Express 23(1), 384–400 (2015). [CrossRef] [PubMed]

6. A. S. Metronor, Homepage, “Metronor One,” (Metronor, 2011–2018), http://www.metronor.com.

7. Gesellschaft für optische Messtechnik mbH Homepage, “ATOS Core – Optical 3D Scanner,” (GOM, 2011–2018), https://www.gom.com/metrology-systems/atos/atos-core.html.

8. W. Bösemann, “Industrial photogrammetry-accepted METROLOGY tool or exotic niche,” presented at the International Archives of the Photogrammetry, Remote Sensing & Spatial Information Sciences, Prague, Czech Republic,12–19 July. (2016).

9. AICON 3D Systems GmbH Homepage, “Aicon-scanner,” (AICON, 2011–2018), https://www.aicon3d.com/de-DE/products/aicon-scanner.

10. AICON 3D Systems GmbH Homepage, “Tubeinspect,”(AICON, 2011–2018), https://www.aicon3d.com/de-DE/products/tubeinspect.

11. P. Jin, J. Liu, S. Liu, and X. Wang, “Automatic multi-stereo-vision reconstruction method of complicated tubes for industrial assembly,” Assem. Autom. 36(4), 362–375 (2016). [CrossRef]

12. S. Liu, J. Liu, P. Jin, and X. Wang, “Tube measurement based on stereo-vision: a review,” Int. J. Adv. Manuf. Technol. 92(5-8), 2017–2032 (2017). [CrossRef]

13. T. Zhang, J. Liu, S. Liu, C. Tang, and P. Jin, “A 3D reconstruction method for pipeline inspection based on multi-vision,” Measurement 98, 35–48 (2017). [CrossRef]

14. T. Zhang, J. Liu, C. Tang, and S. Liu, “3D reconstruction method of bent tube with NURBS curve,” in 2nd International Conference on Mechatronics and Applied Mechanics (ICMAM2012), C.K. Wang and J.Guo, ed. (Applied Mechanics and Materials, 2013), pp. 112–118. [CrossRef]

15. S. Liu, P. Jin, J. Liu, X. Wang, and P. Sun, “Accurate measurement method for tube’s endpoints based on machine vision,” Chin. J. Mech. Eng. 30(1), 152–163 (2017). [CrossRef]

16. C. Doignon and M. de Mathelin, “A degenerate conic-based method for a direct fitting and 3-d pose of cylinders with a single perspective view,” in Proceedings of IEEE International Conference on Robotics and Automation, (IEEE, 2007), pp. 4220–4225. [CrossRef]

17. C. Liu and W. Hu, “Relative pose estimation for cylinder-shaped spacecrafts using single image,” in IEEE Transactions on Aerospace and Electronic Systems, (IEEE, 2014), pp. 3036–3056.

18. K. Wong, P. Mendonça, and R. Cipolla, “Reconstruction of surfaces of revolution from single uncalibrated views,” Image Vis. Comput. 22(10), 829–836 (2004). [CrossRef]

19. W. Bösemann, “The optical tube measurement system OLM photogrammetric methods used for industrial automation and process control,” presented at International Archives of Photogrammetry and Remote Sensing, Part B5, Vienna (1996).

20. I.-K. Lee, “Curve reconstruction from unorganized points,” Comput. Aided Geom. Des. 17(2), 161–177 (2000). [CrossRef]

21. I.-K. Lee and K.-J. Kim, “Shrinking: Another method for surface reconstruction,” in Geometric Modeling and Processing, 2004. Proceedings, (IEEE, 2004), pp. 259–266.

22. Caglioti and A. Giusti, “Reconstruction of canal surfaces from single images under exact perspective,” in European Conference on Computer Vision, A. Prinz, ed. (Springer, 2006), pp. 289–300.

Parameters	Left camera	Right camera
Intrinsic matrix	$A_{l} = [\begin{matrix} 1964.23822 & 0 & 709.88099 \\ 0 & 1964.29461 & 548.68461 \\ 0 & 0 & 1 \end{matrix}]$	$A_{r} = [\begin{matrix} 1963.98789 & 0 & 699.9736 \\ 0 & 1964.0666 & 537.2184 \\ 0 & 0 & 1 \end{matrix}]$
Distortion coefficients	$k_{l 1} = -0 .1072$ $k_{l 2} = 0 .08346$ $k_{l 3} = 0.00001$ $k_{l 4} = - 0.00003$	$k_{r 1} = -0 .10365$ $k_{r 2} = 0 .13459$ $k_{r 3} = 0.00007$ $k_{r 4} = 0.00006$
Rotation vector	$O M = {[\begin{matrix} 0.02121 & 0.84063 & 0.02497 \end{matrix}]}^{T}$
Translation vector	$T_{0} = {[\begin{matrix} - 617.53569 & 10.44938 & 253.56603 \end{matrix}]}^{T}$

count	Angle1	Angle2	R1	R2
1	112.25	99.83	25.22	24.86
2	112.30	99.82	25.21	24.85
3	112.30	99.83	25.19	24.87
4	112.26	99.77	25.22	24.85
5	112.28	99.79	25.19	24.82
6	112.27	99.82	25.17	24.87
7	112.28	99.80	25.22	24.82
8	112.26	99.83	25.17	24.83
9	112.31	99.82	25.17	24.86
10	112.28	99.82	25.18	24.82
RMSE	0.020°	0.020°	0.022 mm	0.021 mm
RMSD	0.030°	0.031°	0.029 mm	0.029 mm

count	Angle1	Angle2	R1	R2
1	100.67	122.42	63.24	65.68
2	100.63	122.47	63.23	65.64
3	100.68	122.45	63.25	65.68
4	100.66	122.40	63.21	65.71
5	100.65	122.46	63.26	65.71
6	100.64	122.43	63.28	65.67
7	100.66	122.44	63.25	65.66
8	100.65	122.40	63.22	65.65
9	100.70	122.43	63.26	65.66
10	100.69	122.42	63.27	65.66
RMSE	0.022°	0.023°	0.019 mm	0.023 mm
RMSD	0.031°	0.029°	0.028 mm	0.029 mm

count	Angle1	Angle2	R1	R2
1	87.99	108.91	72.35	72.68
2	87.99	108.96	72.31	72.67
3	87.98	108.94	72.36	72.63
4	87.96	108.93	72.32	72.63
5	87.97	108.92	72.32	72.65
6	87.95	108.94	72.33	72.69
7	87.94	108.92	72.35	72.65
8	87.97	108.91	72.35	72.64
9	87.97	108.93	72.36	72.63
10	87.99	108.89	72.33	72.65
RMSE	0.017°	0.020°	0.018 mm	0.021 mm
RMSD	0.027°	0.031°	0.026 mm	0.029 mm

count	Angle1	Angle2	R1	R2
1	112.28	99.83	25.23	24.87
2	112.31	99.83	25.21	24.86
3	112.28	99.84	25.22	24.88
4	112.27	99.80	25.23	24.83
5	112.30	99.80	25.19	24.84
6	112.28	99.82	25.17	24.82
7	112.28	99.83	25.21	24.85
8	112.27	99.83	25.17	24.84
9	112.32	99.82	25.18	24.87
10	112.26	99.78	25.20	24.82
RMSE	0.019°	0.019°	0.023 mm	0.021 mm
RMSD	0.035°	0.034°	0.035 mm	0.032 mm

A high precision 3D reconstruction method for bend tube axis based on binocular stereo vision

Abstract

1. Introduction

2. Proposed approach

2.1 perspective projection of an arbitrary cross section

2.2 Projective position of cross section center

2.3 Image properties of arbitrary cross section

2.4 3D reconstruction of bent tube axis

3. Experiments and analysis

3.1 Result using method with axis projection model

3.1.1 Results for tube with diameter about 6 mm

3.1.2 Result for tube with diameter about 15 mm

3.1.3 Result for tube with diameter about 24 mm

3.2 Result using method without axis projection model

3.2.1 Result for tube with the diameter about 6 mm

3.2.2 Result for tube with diameter about 15 mm

3.2.3 Result for tube with diameter about 24 mm

3.3 The influence of image noise on reconstruction results

4. Conclusion

Funding

References

Cited By

Figures (9)

Tables (10)

Equations (11)

Optics Express