Open Access
Add to BibSonomy
Abstract
The binocular structured light 3D measurement system is widely used in situ industrial inspection and shape measurement, where the system structure is generally unstable due to mechanical loosening or environmental disturbance. Timely corrections to the changing structural parameters thus is an essential task for online high-accuracy measurement, which is difficult for traditional unidirectional fringe projection methods to self-correct the structural change. To this end, we propose an online self-correction method based on the investigation that orthogonal fringe projection can intrinsically relax the constraint on the epipolar geometry relationship and provide bidirectional phases for accurate corresponding point searching. Since orthogonal fringe projection may sacrifice the measurement efficiency, we further design a searching strategy by locally unwrapping one directional phase to reduce the number of projection patterns. Experimental results demonstrate that the proposed method is effective for online self-correction of unstable system structure to achieve high-accuracy 3D measurement under complex measurement environments.
© 2024 Optica Publishing Group under the terms of the Optica Open Access Publishing Agreement
1. Introduction
With remarkable advantages of non-contact, high accuracy, high speed, and dense data capture, binocular structured light three-dimensional (3D) measurement has found widespread applications in diverse fields such as industrial inspection, intelligent manufacturing, cultural heritage preservation, and medical diagnostics [1–6]. Phase encoding information from fringe projection is often used to generate active features for robust and accurate 3D reconstruction [7,8]. Traditionally, unidirectional phase information is combined with the epipolar geometry to guarantee uniform constraints for corresponding point searching [9–12].
In scenarios requiring continuous data acquisition, such as handheld or online 3D measurement, however, unstable system structure caused by deformations and collisions during system movement may introduce unpredictable structural parameter variation. Inaccurate structural parameters will compromise the epipolar constraints, which can no longer provide the correct geometrical relationship for corresponding point searching. Frequent recalibration is generally performed to update the system parameters, but this is a cumbersome and impractical procedure. Thus, online self-correction of the changing structural parameters becomes imperative for accurate and reliable 3D measurement.
In response to the challenge of unstable binocular structured light 3D measurement, we propose a phase-aided online self-correction method for binocular structured light 3D measurement systems. We find that using bidirectional phases can fundamentally relax the epipolar constraints, enabling the corresponding point searching to avoid the disturbance of system structure instability. Subsequently, the structural parameters can be optimized from the point-to-line distance model using those correct phase matching points, without requiring the whole system recalibration with an auxiliary target. Considering orthogonal fringe projection sacrifices the time cost, we further design a searching strategy by locally unwrapping one directional phase, which can reduce the number of projection patterns to improve the measurement efficiency. Experimental results demonstrate the validity and effectiveness of the proposed phase matching method and online self-correction functionality, improving the robustness in measurement tasks where the binocular system structure is susceptible to interference and achieving high-accuracy 3D measurement.
2. Background knowledge
As shown in Fig. 1, the binocular structured light system comprises two cameras with 3D imaging and a projector with phase encoding. The well-established pinhole imaging model [13] can be used to describe the imaging process of an individual camera, such that
In this model, an object point $\mathbf {X}_{w}=(X_w,Y_w,Z_w)^T$ in world coordinate system is transformed to $\mathbf {X_c}=(X_c,Y_c,Z_c)^\mathrm {T}$ in the camera coordinate system through a rotation matrix $\mathbf {R}_c$ and translation vector $\mathbf {T}_c$. The object point is projected to be an image point $\mathbf {x}=(x,y)^\mathrm {T}=(X_c/Z_c,Y_c/Z_c)^\mathrm {T}$, which is then transformed to pixel coordinates $\mathbf { {m}}=(m,n)^\mathrm {T}$ using the intrinsic matrix $\mathbf {K}$, where $(f_x,f_y)^\mathrm {T}$ represents the equivalent focal lengths in pixel units, $(C_x,C_y)^\mathrm {T}$ represents pixel coordinates of the principal point, and $\alpha$ represents the skew coefficient of the image axis. $D(\mathbf {x})$ represents nonlinear functions related to the lens distortion [14]. $\lambda$ is a scale factor, and the superscript $\tilde {\cdot }$ denotes a homogeneous coordinate.
One of the camera coordinate systems, for example, the left one, can be treated as the world coordinate system. In this case, imaging an object point $\mathbf {X}$ by the two cameras results in a corresponding point pair $(\mathbf {{m}}_l, \mathbf {{m}}_r)$, involving structural parameters $\mathbf {R}$ and $\mathbf {T}$ that represent the relative orientation between the two perspectives. Thus, the binocular system can be modeled as
When the binocular system parameters are determined, 3D coordinates of the object point can be reconstructed from the corresponding points by solving Eq. (2). With the help of the phase encoding capability of fringe projection, dense and accurate matching point pairs can be acquired to achieve full-frame, high-accuracy 3D reconstruction. The captured phase-shifting fringe image can be represented as
where $I$, $a$, and $b$ denote intensities of the image, background, and modulation, respectively. $\phi$ denotes the phase modulated by the object depth, and $\delta _i=2\pi i/N$ denotes the $i$-step phase-shifting amount associated with the number of phase-shifting steps $N$.By solving Eq. (3), the modulated phase can be solved as
Due to the characteristic of the arctan function, the resulting phase is wrapped between $-\pi$ and $\pi$. Additional coding patterns, for example, variable frequency fringes [15,16] or Gray codes [17–19] can be projected to determine the fringe order $k$ for phase unwrapping, i.e., $\phi (m,n)={{\varphi }}(m,n)+2\pi k(m,n)$. For a calibrated binocular system, an image point of one perspective is associated with an epipolar line of the other perspective. In traditional unidirectional fringe projection methods, one can search for a corresponding point with the same phase as the image point along the epipolar line to implement binocular matching and 3D reconstruction.
3. Method
Accurate system parameters and corresponding points are two crucial factors for 3D measurement. Unidirectional fringe projection methods generally rely on the epipolar geometry relationship to provide additional constraints for corresponding point searching. However, in practical applications, mechanical vibration or environmental disturbance may arise to change the system structure. The epipolar constraints are erroneous in this case, leading to inaccurate matching points. Moreover, the system cannot detect such errors, which often happen in handheld or online measurement and inspection tasks.
In this work, we make an investigation on orthogonal fringe projection to address the issue of changing system structure from which traditional methods suffer. Orthogonal fringe patterns can be projected onto measured objects in succession to obtain bidirectional phase information. This two-dimensional information can provide enough system structure-independent constraints for the corresponding point searching in the image plane. Since the constraint on the epipolar geometry relationship is relaxed, the phase matching points are always correct regardless of the changing system structure. Therefore, the structural parameters can be accurately self-corrected for online 3D measurement. Please note that, regardless of how the left camera moves, its represented reference coordinate system remains unchanged. For the convenience of description and experimental comparison, in the following context, we only move the right camera to induce changes in the structural parameters.
As depicted in Fig. 2, after system structural disturbance, the original epipolar plane marked by pink changes to the new one marked by blue. Self-correction aims to determine new structural parameters associated with the changed epipolar geometry relationship. In Fig. 2, $\mathbf {m}_l$ and $\mathbf {m}_r$ represent a correct matching point pair. When only using unidirectional phase information, the incorrect epipolar line marked by red is inevitably employed to search inaccurate corresponding point $\mathbf {m}_0$ with the same phase as $\mathbf {m}_l$, as shown by the second row in Fig. 2. When using the phase information distributed along another direction, it becomes evident that the phase of $\mathbf {m}_0$ differs from that of $\mathbf {m}_l$, as shown by the third row in Fig. 2. In contrast, with the aid of the consistency of bidirectional phase information, the accurate matching point $\mathbf {m}_r$ can be found independent of the system structure. Once an accurate matching point pair set is available, the structural parameters $\mathbf {R}$ and $\mathbf {T}$ can be optimized to obtain the correct epipolar lines, as shown by the blue lines in Fig. 2.
Fig. 2. The process of online self-correction. Top: Binocular Epipolar Plane Illustration. $\mathbf {m}_l$ and $\mathbf {m}_r$ are a pair of correct corresponding points. The red and blue planes represent the epipolar planes under correct and erroneous parameters, respectively. Bottom: Orthogonal Phase Matching Method for Binocular Vision. The red and blue paths depict the processes of unidirectional and orthogonal phase matching, respectively. Red points represent incorrect matching points, and blue (and light blue) points are the correct matching points. The red dashed box indicates the process of local unwrapping.
Typically, orthogonal fringe projection needs to project more patterns from another direction, thus sacrificing the measurement efficiency. To save the time cost in the proposed self-correction method, we further design a searching strategy that uses the wrapped phase from another direction to obtain the phase matching points. Since the position variation of the matching point caused by the changing system structure is generally located in a specific range, the wrapped phase in such range can be locally unwrapped to implement the corresponding point searching, as shown by the fourth row in Fig. 2, taking a two-order phase distribution range as an example. Such a searching strategy does not need to project those patterns used for phase unwrapping from another direction, therefore improving the measurement efficiency without sacrificing the measurement accuracy. Fig. 3 shows the flowchart of the proposed self-correction method and searching strategy. The process involves two main steps, detailed as follows.
3.1 Bidirectional phase point searching
3.1.1 Phase information acquirement
Project one-directional (here takes horizontal direction as an example) phase-shifting and phase-unwrapping patterns to obtain unwrapped phase maps ${\mathbf {\phi }_{l}}$ and ${\mathbf {\phi }_{r}}$ in the left and right perspectives. Project vertical phase-shifting patterns to obtain wrapped phase maps $\mathbf {\varphi }_{l}$ and $\mathbf {\varphi }_{r}$.
3.1.2 Initial corresponding point searching
For an image point ${\mathbf {m}_{l}}$ of the left camera with an orthogonal phase pair $[{\mathbf {\phi }_{l}}(\mathbf {m}_{l}),{\mathbf {\phi }_{r}}(\mathbf {m}_{l})]$, the candidate corresponding point set $\mathbf {{m}}_{set}$ of the right camera is given by $\{\mathbf {m}|{\mathbf {\tilde {m}}} \mathbf {F} {{\mathbf {\tilde {m}}}_{l}}^{T}=0,m\in {{N}_{+}},m<{{n}_{x}},1\le n\le {{n}_{y}}\}$, where ${N}_{+}$ represent positive integer, $n_x$ and $n_y$ represent the image width and height, respectively, $\mathbf {F}=\mathbf {K}_{r}^{-\intercal }{{[\mathbf {T}]}_{\times }}\mathbf {R}\mathbf {K}_{l}^{-1}$ is the fundamental matrix, ${[\mathbf {T}]}_{\times }$ is the cross product matrix of $\mathbf {T}$[20,21]. Then, the initial image point $\mathbf {m}_0$ can be found in $\mathbf {{m}}_{set}$ where ${\mathbf {\phi }_{r}}(\mathbf {m}_{0})$ is equal to ${\mathbf {\phi }_{l}}(\mathbf {m}_{l})$.
3.1.3 Local phase unwrapping
In the phase map $\mathbf {\varphi }_{r}$, a region of interest with $\mathbf {m}_0$ as the center is extracted to be $\mathbf {\varphi }_{win}$. The phase distribution of $\mathbf {\varphi }_{win}$ is within a certain range of one fringe order, and it can be locally unwrapped to be ${{\phi }_{win}}(m,n)=\varphi _{win}(m,n)+2\pi k_{win}(m,n)$, by determining a local fringe order such that
Here, $n_b(m)=\{n|\nabla _{\perp }\varphi _{win}(m,n)>\pi \}$ the vertical coordinates of the boundaries of wrapped phases for the two orders within $\varphi _{win}$, and $\nabla _{\perp }$ represents the vertical gradient operator.
If the selected region is in the same fringe order, there is no phase ambiguity, and local unwrapping is not required. When the boundary between two fringe orders is encountered, the choice of local fringe order is determined by $\varphi _{l}$ . When $\varphi _{l}$ is positive, the fringe order in the upper region at the boundary is 0, and that in the lower region is 1. Conversely, when $\varphi _{l}$ is negative, they become -1 and 0, respectively.
3.1.4 Accurate corresponding point searching
The phase variation is generally continuous and monotonous for the small-sized local region. For the right camera, the desired phase pair to be found is $\mathbf {R}=[{{\phi }_{l}}(\mathbf {m}_{l}),\varphi _{l}(\mathbf {m}_{l})]$, while the initial orthogonal phase at position $\mathbf {m}_{0}$ is $\mathbf {S}={{[{{\phi }_{r}(\mathbf {m}_{0})},{{\phi }_{win}(\mathbf {m}_{0})}]}}$. Considering the derivatives of the orthogonal phase $\mathbf {p}={{[{{\nabla }_{\bot }}{{\phi }_{r}{({\mathbf {m}_{0}})}},{{\nabla }_{\bot }}{{\phi }_{win}}({{\mathbf {m}_{0}}})]}}$ and $\mathbf {q}={{[{{\nabla }_{\parallel }}{{\phi }_{r}{({\mathbf {m}_{0}})}},{{\nabla }_{\parallel }}{{\phi }_{win}}{({\mathbf {m}_{0}})}]}}$, where $\nabla _{\parallel }$ representing the horizontal gradient operator. Then, the search vector $(\alpha,\beta )$ can be determined by solving the following equation:
In order to obtain accurate sub-pixel coordinates of the matching point, the search vector can be further refined to be $({\alpha }^{\prime },{\beta }^{\prime })$ by utilizing bilinear interpolation surrounding the search location. Finally, the coordinates of the matching point can be calculated to be ${{\mathbf {m}}_{r}}={{\mathbf {m}}_{0}}+({\alpha }^{\prime },{\beta }^{\prime })$.
3.2 Self-correction
In each 3D scanning cycle, a point set $\{\mathbf {m}_{l}^{i}\},i=1,\ldots,M$ including $M$ points is obtained by uniformly sampling the image points of the left camera. The corresponding point set $\{\mathbf {m}_{r}^{i}\}$ of the right camera can be obtained through the aforementioned steps of bidirectional phase matching. To avoid the computation of curved epipolar lines, these corresponding points are transformed to normalized plane coordinates $\{(\tilde {\mathbf {x}}_{l},\tilde {\mathbf {x}}_{r})^i\}$ using the intrinsic camera parameters. The distance between the point $\tilde {\mathbf {x}}_{r}^{i}$ and the corresponding epipolar line $L_{r}^{i}={{[\mathbf {T}]}_{\times }}\mathbf {R}{{\tilde {\mathbf {x}}_{l}}^{i\intercal }}$ is then calculated by
The average distance of all corresponding points to the epipolar lines represents the accuracy of the system parameters. Thus, a cost function is established to optimize the structural parameters:
This is a least squares nonlinear optimization problem, which can be solved using the Levenberg-Marquardt algorithm [22]. Note that due to the projective characteristic of epipolar lines, the obtained structural parameters have a scaling factor and a 4-fold ambiguity [23]. The scaling factor can be determined using a standard scale or by comparing the distance between two corresponding points in the previous frame. For the 4-fold ambiguity, since the final solution will converge to the local optimal solution closest to the given initial values, using the initial structural parameters as the starting input for optimization does not affect obtaining the correct solution.
To enhance the accuracy and effectiveness of the optimization, further filtering is necessary for the point set $\{(\mathbf {m}_{l},\mathbf {m}_{r})^i\}$ based on the following criteria:
Modulation degree: During phase unwrapping, select image points with higher modulation degrees to generate a mask.
Singular point: Compute gradients of the scene image and phase map to generate masks at the scene boundaries. Performing erosion operations on all masks can remove edge points and isolated scattering points.
Depth range: Perform initial 3D reconstruction on the point set and filter out points outside the binocular system’s measurement range.
Once the self-correction process is completed, 3D reconstruction can be achieved by utilizing Eq. (2) to combine the precisely matched points with the accurate structural parameters. The accuracy of the measurement is controlled in the above two steps. During the phase searching step, the epipolar lines are employed solely for initial matching, and the determination of matching positions relies on orthogonal phases unrestricted by epipolar lines. In the parameter correction step, the data used for optimization is derived from the precisely matched points from the preceding step. Simultaneously, a further refinement is applied by selectively filtering robust corresponding points, ensuring the precision of the correction. It is evident that their precision stems from the uniqueness of the matching process based on orthogonal phases. Consequently, this approach is not influenced by structural instabilities occurring during system usage, addressing challenges related to measurement instability, frequent recalibration needs, especially in tasks involving continuous motion, and large-scale measurements.
4. Experiments and analysis
To validate the effectiveness of the proposed method, a binocular structured light 3D measurement system was constructed, as shown in Fig. 4. The system consists of two industrial cameras (Hikrobot MV-CA050-11UM) with a resolution of $2448\times 2048$ pixels and a projector (Anhua Optoelectronics F11GS) with a resolution of $1920\times 1080$ pixels. One of the cameras was mounted on a movable platform to simulate the changing system structure. We employed Zhang’s method [13] for system calibration, complementary Gray code [19] for phase unwrapping, and the Ceres Solver library [24] for optimization of Eq. (8).
4.1 Measurement with different methods
To demonstrate the performance of the proposed method, we slightly modified the structural parameters through the movable platform after completing the system calibration and used different reconstruction algorithms for comparisons. As shown in Fig. 5, the results at the left, middle, and right sides correspond to the method using unidirectional phase along the epipolar line (UniP method), fully unwrapping bidirectional phases (FBiP method), and proposed locally unwrapping bidirectional phases (LBiP method), respectively.
Fig. 5. Point clouds obtained by different methods. From left to right correspond to three different matching methods: unidirectional phase, full unwrapping orthogonal phases, locally unwrapping orthogonal phase. From top to bottom: projected patterns, computed phases, reconstructed point clouds, and the error resulting from subtracting the point clouds.
The first row in Fig. 5 shows the projected patterns of different methods. The captured image sequences were used to compute the phases, which were then used for 3D reconstruction, as shown by the phase maps and 3D models in the second and third rows, respectively.
The FBiP method was taken as a reference to compare UniP and LBiP methods. The deviations of point clouds of UniP and LBiP methods relative to that of the FBiP method are shown in the fourth row in Fig. 5. Since the UniP method dependents on the epipolar geometric constraints to search the corresponding points, the reconstruction result had an obvious deviation when the system structure was changing. In contrast, the LBiP method produced a point cloud almost identical to the FBiP method. This indicates that even in the presence of structural parameter disturbances, the LBiP method can still accurately perform phase point matching for subsequent self-correction and 3D reconstruction.
4.2 Measurement error analysis
In order to quantitatively investigate the influence of changing system structure on measurement accuracy, the right camera was subjected to translations along and rotations around the X, Y, and Z axes, followed by tests of standard spheres. Two reference points were placed in the scene to provide scale factor calculation. After calibrating the system, 3D reconstruction was performed on ten ceramic standard spheres fixed on a board, as shown by the sample picture and reconstructed 3D model in Fig. 6(a) and Fig. 6(b), respectively. In the translation case, the right camera was translated by 0.1mm in each of the three axial directions.
In the rotation case, the center of the right camera was aligned with the rotation axis of the turntable as closely as possible and was rotated by 0.0075 degrees each time around the axes. In each measurement, sphere fitting was performed on the reconstructed point cloud with and without self-correcting the structural parameters. Subsequently, the centers of all the spheres were obtained to calculate the distance between each pair of spheres.
In this experiment, the results of center coordinates in the remaining measurements were compared with the first measurement, and 45 center distances could be obtained to compare with the ground truth. The distribution curves of root mean square errors (RMSE) of the center coordinates and distances related to the translation distances and rotations angles are shown in Fig. 7, respectively.
Fig. 7. Measurement error of the system under translation or rotation. (a) X-axis translation. (b) Y-axis translation. (c) Z-axis translation. (d) X-axis rotation. (e) Y-axis rotation. (f) Z-axis rotation.
Without correcting the changing structural parameters, the measurement deviations of the center coordinates and distances increase with the increased translation and rotation. Specifically, the Z-axis translation has a relatively significant side effect on the measured center coordinates, while the Y-axis rotation on the center distances. The measurement accuracy on the Z axis (toward the measured objects) is generally lower than that on the other two axes. Thus, the influence of the Z-axis translation is larger than that of the X and Y axes. Moreover, as shown in Fig. 1, the Y-axis rotation will change the intersection angle of the optical axes of the two cameras, which has a more significant effect on the measurement uncertainty. In contrast, with self-correction of system structure, the measurement accuracy remains consistent, regardless of translation along or rotation around which axis. It reveals that self-correction is necessary for online 3D measurement with changing structural parameters, especially when the axial distance toward the measured objects and the intersection angle between the perspectives are changed.
4.3 Online self-correction on arbitrary structural changes
To validate the effectiveness of the proposed method under arbitrary changing system structure, the right camera was installed on an unfixed gimbal stage, allowing for unrestricted movement by gently tapping the stage, as shown in Fig. 8(a).
Fig. 8. Standard sphere measured with and without correction. (a) Tapping the unfixed gimbal stage causes the right camera to move arbitrary(see Visualization 1). (b)Without correction, the position of the fitted ball before and after striking is significantly offset. (c)With correction, the position of the fitted sphere before and after striking is basically the same.
In this experiment, we tapped the stage twice with an appropriate force and used the proposed method to correct the structural parameters. Reconstruction of standard spheres before and after impact is conducted using the principles of binocular structured light. Prior to impact, the accurate point cloud reconstructed through the UniP method, with correct structural parameters, serves as the reference point cloud to validate the self-correction effectiveness. Subsequent to impact, alterations in structural parameters lead to a loss of precision in the UniP method. In response, the proposed LBiP method is initially employed in the phase matching stage, where bidirectional phase constraints ensure the accuracy of binocular matching data, and local phase unwrapping technology simultaneously accelerates the projection efficiency. Next, the self-correction method is utilized, whereby robust matching points are selected to participate in optimization, ultimately yielding accurate structural parameters. Under correct binocular matching data and structural parameters, reconstruction and fitting of all standard spheres are then performed, and measurement errors are calculated using the distance between sphere centers.
The central one of the standard spheres shown in Fig. 6(a) was taken as an experimental demonstration. Fig. 8(b) shows the fitting results without self-correction. It can be seen that the positions of the fitted sphere change significantly. In comparison, the fitted spheres are consistent after self-correction, as shown in Fig. 8(c).
Table 1 lists the updates of the structural parameters by the proposed method, and Table 2 lists the data of measurement accuracy with and without self-correction. In these tables, "State 0" is the initial scanning state, while "State 1" and "State 2" are 3D scanning with collisions at different force levels. Consistent with the results in Fig. 7, larger structural changes at a higher force level cause lower measurement accuracy without self-correction. The proposed method can effectively and consistently correct the structural parameters, ensuring high-accuracy 3D measurement without cumbersome recalibration when the system experiences collisions or structural deformations.
Table 1. Self-correction of structural parameters.
Table 2. Accuracy under different correction conditions (mm).
4.4 Robust matching point set selection
To study the influence of outlier matching point sets on structural parameter optimization, we further performed 3D measurement of a ceramic matte standard plane. As illustrated in Fig. 9(a), the standard plane lies within the measurement range, out of which a rear plane is situated. Figure 9(b) shows the reconstruction results with accurate and changing structural parameters marked green and purple, respectively. The former is taken as a reference plane for experimental comparison. It can be seen that the two reconstructed planes have an obvious displacement. Then, plane fitting was performed, and the fitting error maps are shown in Fig. 9(c). The right-side error map related to the changing system structure appears distorted, except for the displacement away from the reference plane.
Fig. 9. Elimination of point set of outliers. (a) Measured ceramic matte standard plane. (b) The green point cloud represents the reconstruction under correct parameters, while the purple point cloud corresponds to the reconstruction after the movement of the right camera. (c) Error distribution for plane fitting of both point clouds in (b). The second and third rows represent the correction results before and after point set filtering, respectively, from left to right: filtered points, comparison with the reference plane, and distribution of fitting errors for plane fitting.
First, in the self-correction procedure, all point pairs from the proposed bidirectional phase point searching, regardless of whether they were located at the object’s edges, corners, regions with low fringe modulation, or even beyond the measurement range, were used for structural parameter optimization. The comparison result with the reference plane demonstrates a two-plane intersection in Fig. 9(e), while the distortion of the fitting error map is somewhat improved, as shown in Fig. 9(f). Then, we selected an effective point set (see Fig. 9(g)) according to the criteria of modulation degree, singular point, and depth range to perform self-correction and 3D reconstruction again. As the ineffective point pairs were eliminated, the reconstruction error was reduced to a minimal range, as shown by the comparison result with the reference plane and the fitting error map in Fig. 9(h) and Fig. 9(i), respectively. Therefore, outlier removal plays an active role in improving the measurement accuracy, especially for applications with large field of view and moving stitching, where complex background usually makes the structure-disturbed 3D measurement system prone to misalignment or matching of low confidence points.
5. Conclusion
In this work, we present an online self-correction method for structured light 3D measurement with the aid of bidirectional phase point matching, which can relax the system-dependent constraints required by traditional unidirectional fringe projection methods. Experimental comparison and analysis demonstrate that the proposed method can adapt to arbitrary system structural changes to obtain reconstruction results consistent with orthogonal fringe projection methods. Furthermore, the measurement efficiency can be improved through the designed searching strategy, which locally unwraps one directional phase to reduce the number of projected patterns. These confirm the effectiveness of the proposed method, greatly enhancing the measurement robustness and accuracy in tackling tasks involving continuous movement, susceptibility to impacts, and long baselines.
Funding
National Natural Science Foundation of China (62175109, 62275173); Shenzhen Fundamental Research Program (JCYJ20220531101204010).
Disclosures
The authors declare that there are no conflicts of interest related to this article.
Data availability
Data underlying the results presented in this paper are not publicly available at this time but may be obtained from the authors upon reasonable request.
References
1. S. D. Cochran and G. Medioni, “3-d surface description from binocular stereo,” IEEE Trans. Pattern Anal. Machine Intell. 14(10), 981–994 (1992). [CrossRef]
2. Y. Yang, H. L. Chen, X. Liu, et al., “Real-time point cloud registration for flexible hand-held 3d scanning,” in Sixth Symposium on Novel Photoelectronic Detection Technology and Application, (2020).
3. L. Song, X. Li, Y.-G. Yang, et al., “Structured-light based 3d reconstruction system for cultural relic packaging,” Sensors 18(9), 2981 (2018). [CrossRef]
4. Z. Cai, X. Liu, A. Li, et al., “Phase-3d mapping method developed from back-projection stereovision model for fringe projection profilometry,” Opt. Express 25(2), 1262–1277 (2017). [CrossRef]
5. C. Zuo, S. Feng, L. Huang, et al., “Phase shifting algorithms for fringe projection profilometry: A review,” Opt. Lasers Eng. 109, 23–59 (2018). [CrossRef]
6. X. Liu, X. Peng, D. He, et al., “Automatic 3d imaging and modelling system with color information for cultural heritage digitization,” Fringe (2014).
7. Z. Cai, X. Liu, X. Peng, et al., “Ray calibration and phase mapping for structured-light-field 3d reconstruction,” Opt. Express 26(6), 7598 (2018). [CrossRef]
8. Y. Miao, Y. Yang, Q. Hou, et al., “High-efficiency 3d reconstruction with a uniaxial mems-based fringe projection profilometry,” Opt. Express 29(21), 34243–34257 (2021). [CrossRef]
9. R. Ishiyama, S. Sakamoto, J. Tajima, et al., “Absolute phase measurements using geometric constraints between multiple cameras and projectors,” Appl. Opt. 46(17), 3528–3538 (2007). [CrossRef]
10. C. Bruer-Burchardt, C. Munkelt, M. Heinze, et al., “Using geometric constraints to solve the point correspondence problem in fringe projection based 3d measuring systems,” Lecture Notes Comput. Sci. 6979, 265–274 (2011). [CrossRef]
11. C. Bräuer-Burchardt, C. Munkelt, M. Heinze, et al., “Fringe code reduction for 3d measurement systems using epipolar geometry,” Proc. PCVIA, ISPRS 38, 192–197 (2010).
12. Z. Li, K. Zhong, Y. F. Li, et al., “Multiview phase shifting: a full-resolution and high-speed 3d measurement framework for arbitrary shape dynamic objects,” Opt. Lett. 38(9), 1389–1391 (2013). [CrossRef]
13. Z. Zhang, “A flexible new technique for camera calibration,” IEEE Trans. Pattern Anal. Machine Intell. 22(11), 1330–1334 (2000). [CrossRef]
14. J. Kannala and S. S. Brandt, “A generic camera model and calibration method for conventional, wide-angle, and fish-eye lenses,” IEEE Trans. Pattern Anal. Machine Intell. 28(8), 1335–1340 (2006). [CrossRef]
15. J. M. Huntley and H. Saldner, “Temporal phase-unwrapping algorithm for automated interferogram analysis,” Appl. Opt. 32(17), 3047–3052 (1993). [CrossRef]
16. J. M. Huntley and H. O. Saldner, “Shape measurement by temporal phase unwrapping: comparison of unwrapping algorithms,” Meas. Sci. Technol. 8(9), 986–992 (1997). [CrossRef]
17. G. Sansoni, S. Corini, S. Lazzari, et al., “Three-dimensional imaging based on gray-code light projection: characterization of the measuring algorithm and development of a measuring system for industrial applications,” Appl. Opt. 36(19), 4463–4472 (1997). [CrossRef]
18. G. Sansoni, M. Carocci, and R. Rodella, “Three-dimensional vision based on a combination of gray-code and phase-shift light projection: analysis and compensation of the systematic errors,” Appl. Opt. 38(31), 6565–6573 (1999). [CrossRef]
19. S. Xuezhen, S. Xianyu, and Z. Xiaoping, “Phase unwrapping based on complementary structured light binary code,” Acta Opt. Sin. 28(10), 1947–1951 (2008). [CrossRef]
20. H. Zhao and J. Li, “Stereo image matching based on phase unwrapping,” in Fifth International Symposium on Instrumentation and Control Technology, vol. 5253 (SPIE, 2003), pp. 394–397.
21. D. Li, H. Zhao, and H. Jiang, “Fast phase-based stereo matching method for 3d shape measurement,” in 2010 International Symposium on Optomechatronic Technologies, (IEEE, 2010), pp. 1–5.
22. A. Ranganathan, “The Levenberg-Marquardt algorithm,” Tutoral on LM algorithm 11, 101–110 (2004).
23. R. Hartley and A. Zisserman, Multiple view geometry in computer vision (Cambridge University Press, 2003).
24. S. AgarwalK. Mierle, and T. C. S. Team, “Ceres Solver,” (2022).
