Open Access
Add to BibSonomy
Abstract
High-speed 3D measurement is receiving increasing attention. However, simultaneously achieving high computational efficiency, algorithmic robustness, and reconstructing ratio is challenging. Therefore, a dynamic phase-differencing profilometry (DPDP) is proposed. By capturing the minimum three phase-shifting sinusoidal deformed patterns and establishing a brand-new model, the phase difference between the object on the reference plane and the reference plane is directly resolved to effectively improve computational efficiency. Although it is wrapped, by using only two auxiliary complementary gratings with a purposely designed lower frequency, a DPDP-based number-theoretical temporal phase unwrapping (NT-TPU) algorithm is also proposed to unwrap the wrapped phase difference rather than the phase itself with high robustness. Furthermore, compared to existing PSP-based NT-TPU, the proposed NT-TPU can normally work under more relaxed restrictions. In order to accomplish a high reconstructing ratio, a pentabasic interleaved projection (PIP) strategy based on time division multiplexing is proposed. It can improve the reconstructing ratio from one reconstruction per every five patterns to an equivalent of one reconstruction per every 1.67 patterns. Experimental results demonstrate that the proposed method achieves high computational efficiency, high algorithmic robustness, and high reconstructing ratio simultaneously and has prospective application in high-speed 3D measurement.
© 2024 Optica Publishing Group under the terms of the Optica Open Access Publishing Agreement
1. Introduction
3D measurement has attracted much attention due to its promising applications [1–3]. Among the various 3D measurement techniques, fringe projection profilometry (FPP) has become one of the mainstream technologies due to its non-contact and full-field measurement capabilities [4,5]. Fourier transform profilometry (FTP) and phase-shifting profilometry (PSP) are the two most used techniques in FPP. FTP was proposed by M. Takeda et al. in 1983 [6], enabling 3D measurement by projecting one sinusoidal grating but sacrifices accuracy due to frequency domain filtering [7]. PSP was posited by V. Srinivasan et al. in 1984 [8] and is widely acknowledged to achieve the highest accuracy in FPP by projecting N (N ≥ 3) phase-shifting sinusoidal gratings [9].
High-speed 3D measurement has become increasingly popular with the advancement of science and technology. As known for us, PSP has been proved to achieve high-speed 3D measurement at the minimal N = 3 as long as the projection frame rate is sufficiently high [10]. Due to the arctangent operation, the resolved phase is wrapped within (-π, π], so phase unwrapping plays a crucial role. The common phase unwrapping includes spatial phase unwrapping (SPU) [11] and temporal phase unwrapping (TPU) [12]. SPU can obtain relative phase without any additional auxiliary pattern, but it is much dependent on the unwrapping path and may not be effective in noisy and discontinuous scenarios as it is prone to experiencing the accumulation of unwrapping errors [13]. Its robustness in challenging. TPU utilizes multiple additional auxiliary patterns to obtain absolute phase with high algorithmic robustness, and it is therefore suitable for complex scenarios due to its point-to-point phase unwrapping feature [14–16]. Widely used TPU methods mainly include Gray code TPU [17], phase-coding TPU [18], dual-frequency hierarchical TPU [19,20], dual-frequency heterodyne TPU [21,22] and number-theoretical TPU (NT-TPU) [23–27]. Without loss of generality, to obtain absolute phase unwrapping for 64 fringe orders, Gray code TPU must project six auxiliary binary Gray-coded patterns. Its sampling rate and computational efficiency are challenging in high-speed 3D measurement [28]. Phase-coding TPU achieves unwrapping by additionally projecting three phase-coded patterns. However, the larger codeword numbers might lead to difficulties in codeword identification [29]. Dual-frequency hierarchical TPU and dual-frequency heterodyne TPU achieve unwrapping by using three additional auxiliary unit-frequency gratings or their equivalents. However, they may result in a frequency ratio of 64 times between phase-shifting gratings and auxiliary unit-frequency gratings or their equivalents, leading to local unwrapping errors caused by amplified phase noise [30]. Leveraging the number-theoretical relationship between phase-shifting gratings and three additional auxiliary gratings with purposely designed lower frequency, NT-TPU achieves accurate unwrapping using fewer auxiliary gratings. Therefore, it has found successful applications in high-speed 3D measurement [31]. However, achieving high algorithmic robustness while further reducing the number of auxiliary gratings is a constantly striving goal.
The addition of auxiliary gratings does effectively improve the robustness of 3D reconstruction, but it inevitably reduces the sampling rate of dynamic 3D measurement, thereby affecting the high-speed feature of 3D measurement. As a result, how to reduce the steps of 3D reconstruction process and improve the computational efficiency of the algorithm itself under a given sampling rate is also a constantly striving goal.
Therefore, a dynamic phase-differencing profilometry (DPDP) is proposed. By capturing the minimum three phase-shifting sinusoidal deformed patterns and establishing a brand-new math model, the phase difference between the object on the reference plane and the reference plane is directly resolved. Though it is wrapped, by using only two auxiliary complementary gratings, a DPDP-based NT-TPU algorithm is proposed to directly unwrap it. Finally, the 3D reconstructed result is obtained by phase-to-height mapping [32–34]. Different from PSP, which must first obtain the wrapped phase caused by the measured object on the reference plane and the wrapped phase just caused by the reference plane separately, and unwrap them separately, and then indirectly resolve the corresponding absolute phase difference by subtracting between them. That is to say, PSP requires two phase resolutions, two times of phase unwrapping and one subtraction to obtain the absolute phase difference. The proposed DPDP not only significantly improves computational efficiency by directly resolving the wrapped phase difference rather than the phase itself, but also significantly simplifies the steps of obtaining the absolute phase difference with only one phase difference resolution and only one time of phase unwrapping on the premise of inheriting all the advantages of the PSP. Furthermore, leveraging the direct phase difference acquisition capability of proposed DPDP, DPDP-based NT-TPU directly unwraps the wrapped phase difference rather than the phase itself by using only two auxiliary complementary gratings. It not only improves the sampling rate from 6 to 5 patterns per sampling period compared to PSP-based NT-TPU, but also further improves the algorithmic robustness owing to the fringe carrier frequency independence of phase difference.
Finally, in order to further improve reconstructing ratio while maintaining a constant sampling rate, a pentabasic interleaved projection (PIP) strategy based on time division multiplexing is proposed. Through astute design of two series of pentabasic sequences encompassing both phase-shifting and auxiliary gratings, along with cyclically interleaved projections, every sampling period consisting of 10 consecutively captured patterns can yield 6 reconstructions. Therefore, proposed PIP strategy improves the reconstructing ratio from one reconstruction per every 5 patterns in normal strategy to an equivalent of one reconstruction per every 1.67 patterns. Through the harmonious integration of proposed DPDP theory, DPDP-based NT-TPU algorithm, and PIP strategy, the high-speed 3D measurement with high computational efficiency, high algorithmic robustness and high reconstructing ratio simultaneously can be achieved.
2. Methodology
2.1 Proposed DPDP theory
A FPP system is shown in Fig. 1. Firstly, the necessary preparations for measuring the reference plane are conducted. The minimal 3 phase-shifting sinusoidal gratings are projected onto the reference plane, the corresponding fringe patterns are captured and saved in advance as:
While measuring, the same 3 phase-shifting sinusoidal gratings are projected onto the measured object on the reference plane, the corresponding deformed patterns are captured as:
Due to the arctangent operation, the resolved phase difference just caused by the measured object itself is wrapped within (-π, π], which reveals unique feature much different from the wrapped phase obtained by PSP shown in Fig. 2. The wrapped phase obtained by PSP exhibits a sawtooth-like distribution, as shown in Fig. 2(a). While the wrapped phase difference just caused by the measured object itself obtained by proposed DPDP exhibits isophase closure feature of multiple isolated islands owing to the fringe carrier frequency independence of phase difference, as shown in Fig. 2(b).
Fig. 2. Wrapped phase and wrapped phase difference. (a) Wrapped phase; (b) Wrapped phase difference.
2.2 Proposed DPDP-based NT-TPU algorithm
In order to unwrap the wrapped phase difference just caused by the measured object itself with high algorithmic robustness, a novel DPDP-based NT-TPU algorithm is proposed. Prior to measuring, 3 auxiliary phase-shifting sinusoidal gratings with a purposely designed lower frequency are projected onto the reference plane, the corresponding auxiliary fringe patterns are captured and saved in advance as:
While measuring, only two auxiliary complementary sinusoidal gratings are added to the projection sequence and then projected onto the measured object on the reference plane, the corresponding auxiliary deformed patterns are captured as:
As known for us, in PSP-based NT-TPU [24], to ensure a sufficiently large unambiguous phase unwrapping range, the periods of phase-shifting gratings and auxiliary gratings Tp and Ta must be coprime and their product must be closest to the resolution of the projector [31]. In contrast, due to the isophase closure feature of the wrapped phase difference, proposed NT-TPU only requires selecting a pair of smaller coprime numbers Qp and Qa and a greatest common divisor g, with Tp = gQp and Ta = gQa, the number-theoretical relationship can be described as:
It is worth noting that since it is feasible to list all pairs (kpm, kam) that may appear and calculate their corresponding matched values before measuring according to Eq. (8), the corresponding look-up table (LUT) can be prepared in advance to further improve computational efficiency. Moreover, incorporating the inherent constraint between kpm and kam, the corresponding LUT data volume can be further reduced:
By excluding invalid pairs, the LUT data volume of proposed NT-TPU is minimized. In addition, as proposed method directly obtains the phase difference independent of the carrier frequency, it further relaxes the number-theoretical constraints between the phase-shifting gratings and auxiliary gratings. This allows proposed method to use relatively higher frequency auxiliary gratings, thereby reducing the frequency difference between the phase-shifting gratings and auxiliary gratings and enhancing the auxiliary phase accuracy. Therefore, proposed method can achieve high algorithmic robustness.
It is worth noting that other FPP methods such as FTP and PSP also require the reference plane. When using the same normal reference plane as in FTP or PSP, proposed method already ensures its high algorithmic robustness, so there is no need for special requirements or restrictions on the quality of the reference plane.
Furthermore, since proposed NT-TPU requires purposeful design of the values of Qp, Qa and g parameters, a parameter selection flow is developed, as shown in Fig. 3. The detailed flow description is outlined below:
1). Select the values of Qp and g. The values of Qp and g can be assumed based on the equation Tp = gQp for the determined period Tp. List all possible combinations of Qp and g, where Qp and g are both positive integers. Subsequently, a preliminary screening based on algorithmic robustness is carried out on all of these combinations. Since the phase unwrapping error immunity condition of proposed NT-TPU can be expressed as:
Where Δφ represents phase error. The maximum phase error can be defined as:Further derivation can be obtained from Eqs. (10) and (11) as:
As a result, the allowable maximum phase error range, which does not introduce phase unwrapping errors in proposed NT-TPU, can be derived as:
Obviously, if Qp is large and g is small, the larger Qp decreases error resistance and algorithmic robustness of proposed NT-TPU, as shown in Eq. (13). Conversely, if Qp is small and g is large, given that Ta = gQa, a larger value of Ta may result due to the larger value of g, leading to larger auxiliary phase errors due to the level of phase noise is proportional to the square of the period of captured patterns [37]. Therefore, the values of Qp and g should be close to each other. Based on this, the combinations with the least absolute difference between Qp and g are retained, while combinations that did not satisfy the criteria are eliminated. Notably, to minimize situations where large values of Qp or g may occur, it is recommended that Tp not be a prime number.
2). Calculate the depth range of 2π phase height. For the retained combinations, calculate the depth range of 2π phase height of the phase-shifting gratings generated by each of these combinations. The depth range of 2π phase height is shown in Fig. 4. Point H1 and point H2 have same imaging point in the captured patterns and have 2π phase variation. The distance ΔH2π represents the depth range of 2π phase height. Θ represents the angle between projector and camera. Δd represents the spatial span of one period of the phase-shifting gratings. Assuming that the resolution of the projector is R and the projection width is W, the depth range of 2π phase height ΔH2π can be calculated as:
3). Determine the value of Qa. For each combination of Qp and g, the value range of kpm within the measuring depth range Hmea can be expressed as [0, Ceil (Hmea /ΔH2π)-1], where Ceil() represents round towards plus infinity. Since the range of kpm that can be unique determined is 0 to Qa-1, Qa ≥ Ceil(Hmea /ΔH2π). Moreover, three constraints must be satisfied to determine the value of Qa: a. Since Ta is larger than Tp, Qa>Qp; b. Qa and Qp should be coprime; c. The value of Qa should be minimized to ensure that proposed NT-TPU achieves maximum error resistance while satisfying the above conditions. By screening based on these, the corresponding Qa for each combination can be determined. It should be noted that, in order to ensure proposed method achieves the highest possible algorithmic robustness and measuring accuracy, Hmea should fall within the measuring system's nearly focused range [38].
4). Determine the optimal combination of Qp, Qa, and g. After determining all combinations of Qp, Qa, and g that meet the criteria, one combination needs to be selected as the optimal combination. Since the error resistance of proposed NT-TPU is inversely proportional to the value of Qp + Qa shown in Eq. (13), the combination with the smallest value of Qp + Qa is considered the optimal combination. Moreover, g reflects the frequency difference between phase-shifting and auxiliary gratings, and a smaller g indicates a smaller frequency difference. Since gratings of different frequencies require different defocus levels, gratings with closer frequencies are more advantageous in reducing defocus errors. Therefore, if multiple combinations have the same smallest value of Qp + Qa, the combination with the smallest value of g among these combinations is considered the optimal combination.
It is worth emphasizing that proposed NT-TPU achieves high algorithmic robustness through several aspects: 1). Proposed NT-TPU is a TPU algorithm, the phase unwrapping result of each pixel does not affect the unwrapping of other pixels, thereby avoiding error accumulation frequently seen in SPU; 2). Proposed NT-TPU can achieve higher frequency auxiliary gratings than PSP-based NT-TPU [24] due to fewer restrictions on the number-theoretical relationship, leading to lower auxiliary phase errors than PSP-based NT-TPU [37]; 3). As proposed NT-TPU has Qp and Qa values smaller than Tp and Ta in PSP-based NT-TPU, it requires less LUT data volume. The reduced LUT data volume not only improves computational efficiency but also enhances phase unwrapping reliability; 4). The allowable maximum phase error of PSP-based NT-TPU must be less than π/(Tp + Ta) [39]. While the allowable maximum phase error of proposed NT-TPU must be less than π/(Qp + Qa), as shown in Eq. (13). Therefore, proposed NT-TPU achieves a larger allowable maximum phase error range due to the smaller value of Qp + Qa compared to Tp + Ta in PSP-based NT-TPU.
2.3 Proposed PIP strategy
While measuring, in the event that normal projection strategy is employed, three phase-shifting gratings and two auxiliary gratings are cyclically projected in the sequence of $I_1^{pm}$, $I_2^{pm}$, $I_3^{pm}$, $I_1^{am}$, $I_2^{am}$, and captured by a high-speed camera. Therefore, one reconstruction is obtained for each sampling period consisting of 5 corresponding patterns. While this already meets the demands of high-speed 3D measurement adequately, a PIP strategy is proposed to further improve reconstructing ratio while maintaining a constant sampling rate, as shown in Fig. 5. It utilizes time-division multiplexing technology to cyclically interleave the projection and capture of two series of different pentabasic sequences encompassing both phase-shifting and auxiliary gratings ($I_1^{am}$, $I_1^{pm}$, $I_2^{pm}$, $I_3^{pm}$, $I_1^{pm}$ and $I_2^{pm}$, $I_3^{pm}$, $I_1^{pm}$, $I_2^{pm}$, $I_2^{am}$). And two adjacent pentabasic sequences form a cycle comprising ten deformed patterns ($I_1^{am}$, $I_1^{pm}$, $I_2^{pm}$, $I_3^{pm}$, $I_1^{pm}$, $I_2^{pm}$, $I_3^{pm}$, $I_1^{pm}$, $I_2^{pm}$, $I_2^{am}$). In each cycle, eight phase-shifting deformed patterns are arranged consecutively to produce six required wrapped phase differences. The closest Am is regarded as the DC component of each auxiliary deformed pattern. Furthermore, the auxiliary deformed patterns from the current cycle are merged with those from the adjacent cycle to obtain two auxiliary wrapped phase differences. Due to the measuring system having a sufficiently high refresh frame rate and acquisition frame rate, the captured adjacent three phase-shifting deformed patterns or adjacent two auxiliary deformed patterns can be considered as being captured at the same moment. Therefore, the required and auxiliary wrapped phase differences with high accuracy can be obtained. Additionally, since the time gaps between the four required wrapped phase differences temporally closest to the auxiliary wrapped phase difference $\varphi _\textrm{1}^{pm}$, $\varphi _\textrm{2}^{pm}$, $\varphi _\textrm{5}^{pm}$, $\varphi _\textrm{6}^{pm}$ and the auxiliary wrapped phase difference are small enough, supported by the high algorithmic robustness provided by proposed NT-TPU, there are no 2π phase jump errors during the phase unwrapping. Consequently, each auxiliary wrapped phase difference assists in unwrapping the four closest required wrapped phase differences ($\varphi _\textrm{1}^{pm}$, $\varphi _\textrm{2}^{pm}$, $\varphi _\textrm{5}^{pm}$ and $\varphi _\textrm{6}^{pm}$), resulting in four accurate reconstructions. For $\varphi _3^{pm}$ and $\varphi _4^{pm}$ that have a larger time gap with the auxiliary wrapped phase differences, their respective closest absolute phase differences ($\Phi _2^{pm}$ and $\Phi _5^{pm}$) are used to unwrap these two required wrapped phase differences as:
It is important to emphasize that in different cycles, proposed PIP strategy uses different auxiliary wrapped phase differences to assist in unwrapping the required wrapped phase differences. Consequently, proposed PIP strategy does not show error propagation, ensuring the algorithmic robustness of the entire measuring process.
Furthermore, it is also worth noting that due to the contiguous arrangement of eight phase-shifting deformed patterns in each cycle, the time gap between the six reconstructions obtained from these eight phase-shifting deformed patterns remain consistent. The reconstructions have a slight variation in the time gap only between the last reconstruction of the current cycle and the first reconstruction of the next cycle compared to the time gaps between any other two consecutive reconstructions. However, due to the measuring system having a sufficiently high refresh frame rate and acquisition frame rate, the variation in this time gap is very small compared to other time gaps. Therefore, it is reasonable to approximate the time gaps between each reconstruction as equal, which will not affect the visual representation of proposed method.
3. Experiments
The FPP measuring system is displayed in Fig. 6. It mainly consists of a high-speed digital light projector (HDLP, Light Crafter 4500) with 1140 × 912 pixels and a high frame CMOS monochrome camera (HCX20) with the maximum resolution of 1088 × 2048 pixels. A color camera (IMAGINGSOURCE DFM 72AUC02) with 1080 × 1920 pixels is also used to validate the robustness of proposed method. Additionally, it is considered empirically to determine the period Tp of proposed NT-TPU is 16 pixels. Θ is set to 15°, R is 912 pixels, W is approximately 650 mm, and Hmea is set to 200 mm. Thus, according to developed parameter selection flow, the values of Qp, Qa, and g are determined as 4, 5, and 4 respectively, resulting in Ta = gQa = 20 pixels for two auxiliary complementary sinusoidal gratings. The binary defocusing [40] is adopted to increase the projection frame rate.
3.1 Feasibility verification of the proposed method
Firstly, in order to validate the feasibility of proposed method, a comparative experiment between the PSP-based NT-TPU [24] and proposed method is conducted, as shown in Fig. 7. Both the two methods use phase-shifting gratings with 16 pixels per period. In order to ensure a sufficiently large unambiguous phase unwrapping range, the PSP-based NT-TPU requires three auxiliary phase-shifting sinusoidal gratings of 57 pixels per period. One phase-shifting deformed pattern is shown in Fig. 7(a), while the auxiliary deformed patterns with 20 pixels per period and 57 pixels per period are depicted in Figs. 7(b)–7(c), respectively. The auxiliary and required wrapped phases obtained by PSP are shown in Figs. 7(d)–7(e), which exhibit sawtooth-like distribution. The absolute phase difference just caused by the measured object itself obtained by PSP-based NT-TPU is shown in Fig. 7(f). The larger period of auxiliary gratings decreases algorithmic robustness and worsens defocus errors due to the varying defocus levels required for different gratings, resulting in spiky and blocky jump errors. The auxiliary and required wrapped phase differences obtained by proposed DPDP are shown in Figs. 7(g)–7(h), exhibiting isophase closure feature of multiple isolated islands. And the absolute phase difference just caused by the measured object itself obtained by proposed NT-TPU is shown in Fig. 7(i), achieving accurate unwrapping. Evidently, proposed method achieves higher algorithmic robustness than PSP-based NT-TPU, which is primarily due to three factors:
Fig. 7. The first comparative experiment. (a)-(c) Deformed patterns at three different frequencies; (d)-(e) Wrapped phases by PSP; (f) Absolute phase difference by PSP-based NT-TPU; (g)-(h) Wrapped phase differences by proposed DPDP; (i) Absolute phase difference by proposed NT-TPU.
1). Proposed method uses higher frequency auxiliary gratings than PSP-based NT-TPU, leading to less auxiliary phase noise and improved auxiliary phase accuracy due to the level of phase noise is proportional to the square of the period of captured patterns [37];
2). In this experiment, the LUT data volume for PSP-based NT-TPU is 71, while proposed method only has 8, as shown in Table 1. The reduced LUT data volume improves phase unwrapping reliability of proposed method effectively;
Table 1. The LUT of proposed NT-TPU when Qp = 4 and Qa = 5.
3). The allowable maximum phase error of the PSP-based NT-TPU must be less than π/73, while the allowable maximum phase error of proposed NT-TPU can approach π/9, which is more than 8 times that of the PSP-based NT-TPU. In fact, due to factors mentioned earlier, proposed NT-TPU achieves error resistance more than 8 times that of the PSP-based NT-TPU.
The data comparison results between the two methods are summarized in Table 2 as:
Table 2. Data comparison results between the two methods.
Using the result obtained by 8-step PSP as ground truth, proposed method achieves a root mean square error (RMSE) of 0.0388 rad. In fact, proposed method achieves an accuracy level comparable to that of PSP-based NT-TPU because the accuracy is determined by the phase-shifting gratings, and proposed method only modifies the steps for obtaining the absolute phase difference just caused by the measured object itself without altering the phase-shifting gratings.
In order to verify the computational efficiency of proposed method, the above experiment is repeated 10 times. The computational times on the i7-12650 H CPU for the PSP-based NT-TPU and proposed method from phase resolution to obtain the absolute phase difference just caused by the measured object itself are listed in Table 3. The average computational time for the PSP-based NT-TPU is 1.033 seconds, while proposed method has an average computational time of only 0.069 seconds, thereby reducing the computational time by a factor of 14.971. Clearly, proposed method achieves higher computational efficiency than PSP-based NT-TPU.
Table 3. Computational efficiency comparison between the two methods (/s).
The computational efficiency advantage of proposed method is attributed to two factors:
1). Compared to PSP-based NT-TPU, proposed method further simplifies the steps for obtaining the absolute phase difference just caused by the measured object itself Φpm. In order to obtain Φpm, PSP-based NT-TPU must first separately obtain the absolute phase caused by the measured object on the reference plane and the absolute phase just caused by the reference plane using NT-TPU. Subsequently, by subtracting between them, Φpm can be obtained. In addition, the corresponding auxiliary phase caused by the measured object on the reference plane and the auxiliary phase just caused by the reference plane are also required to achieve NT-TPU. That is to say, PSP-based NT-TPU requires four phase resolutions, two times of phase unwrapping and one subtraction. While in proposed method, the phase difference just caused by the measured object itself φpm is directly resolved by capturing the minimum three phase-shifting deformed patterns and establishing a brand-new math mode. Although φpm is wrapped, proposed NT-TPU facilitates the unwrapping of φpm by using only two auxiliary gratings with a purposely designed lower frequency to provide an auxiliary phase difference φam. Therefore, proposed method only requires two phase resolutions and one time of phase unwrapping;
2). The LUT data volume for PSP-based NT-TPU is 71, while proposed method only has 8. This reduction further reduces the computational load and enhances computational efficiency.
3.2 Performance comparison with other TPU methods
Furthermore, a comparative experiment between dual-frequency hierarchical TPU [19], dual-frequency heterodyne TPU [21] and proposed method is conducted to showcase the advantages of proposed method compared to other TPU methods, as illustrated in Fig. 8. The phase-shifting gratings for all three methods comprise of 16 pixels per period. To achieve unambiguous phase unwrapping, dual-frequency hierarchical TPU requires the auxiliary gratings to be 912 pixels per period, and dual-frequency heterodyne TPU requires the auxiliary gratings to be 114/7 pixels per period. For dual-frequency hierarchical TPU, one auxiliary deformed pattern with 912 pixels per period is shown in Fig. 8(a), while the wrapped phase obtained by PSP is shown in Fig. 8(b). The absolute phase difference just caused by the measured object itself obtained by dual-frequency hierarchical TPU is depicted in Fig. 8(c). Due to the larger frequency difference between the phase-shifting and auxiliary gratings as well as the lower auxiliary phase accuracy, there are occurrences of jump and block errors in the absolute phase difference. Moreover, the average computational time of dual-frequency hierarchical TPU is 0.152 seconds.
Fig. 8. The second comparative experiment. (a) Auxiliary deformed pattern with 912 pixels per period; (b) Wrapped phase by PSP; (c) Absolute phase difference by dual-frequency hierarchical TPU; (d) Auxiliary deformed pattern with 114/7 pixels per period; (e) Wrapped phase by PSP; (f) Absolute phase difference by dual-frequency heterodyne TPU; (g) Auxiliary deformed pattern with 20 pixels per period; (h) Wrapped phase difference by proposed DPDP; (i) Absolute phase difference by proposed NT-TPU.
For dual-frequency heterodyne TPU, one auxiliary deformed pattern with 114/7 pixels per period is shown in Fig. 8(d), while the wrapped phase obtained by PSP is shown in Fig. 8(e). The absolute phase difference just caused by the measured object itself obtained by dual-frequency heterodyne TPU is depicted in Fig. 8(f). For reasons similar to dual-frequency hierarchical TPU, dual-frequency heterodyne TPU results in a notable frequency of jump and block errors in the absolute phase difference. And the average computational time of dual-frequency heterodyne TPU is calculated as 0.241 seconds.
For proposed method, one auxiliary deformed pattern with 20 pixels per period is shown in Fig. 8(g), while the wrapped phase difference obtained by proposed DPDP is shown in Fig. 8(h). Finally, the absolute phase difference just caused by the measured object itself obtained by proposed NT-TPU is shown in Fig. 8(i). Observations indicate that proposed method yields accurate result. The average computational time of proposed method is calculated as 0.103 seconds. Experimental results indicate that, when compared to other TPU methods, proposed method maintains its advantages in both algorithmic robustness and computational efficiency.
3.3 Measurement of multiple isolated colored objects
The multiple isolated colored objects are measured, as shown in Fig. 9. The measured objects are shown in Fig. 9(a), while the phase-shifting deformed pattern with 16 pixels per period and the auxiliary deformed pattern with 20 pixels per period are depicted in Figs. 9(b)-9(c), respectively. The auxiliary and required wrapped phase differences obtained by proposed DPDP are shown in Figs. 9(d)–9(e), respectively. The absolute phase difference just caused by the measured object itself obtained by proposed NT-TPU is illustrated in Fig. 9(f), avoiding unwrapping errors. Finally, the 3D reconstructed result is shown in Fig. 9(g). Even in complex scenarios involving multiple isolated colored objects with non-uniform reflectance distribution due to color variations, proposed method still keeps high algorithmic robustness.
Fig. 9. Color experiment. (a) Measured objects; (b)-(c) Deformed patterns at two frequencies; (d)-(e) Wrapped phase differences by DPDP; (f) Absolute phase difference by proposed NT-TPU; (g) 3D reconstructed result.
3.4 Feasibility verification of the proposed PIP strategy
A dynamic scenario involving twisting Rubik's cube is measured, as depicted in Fig. 10. The captured frame rate is 200 Hz, resulting in a reconstructed rate of 120 Hz for proposed PIP strategy. The captured deformed patterns at three different moments are shown in Figs. 10(a)-(c), and all deformed patterns can be viewed in Visualization 1. The 3D reconstructed results corresponding to these three moments are shown in Figs. 10(d)-(f). It is worth noting that a total of 805 deformed patterns are captured, leading to 483 3D reconstructed results. All the results can be viewed in Visualization 2. From the experimental results, it can be observed that even in complex dynamic scenarios involving simultaneous horizontal motion, vertical motion, and rotational motion, accurate 3D reconstructed results are achieved by proposed PIP strategy, supported by the high algorithmic robustness provided by proposed NT-TPU. This not only demonstrates the feasibility of the proposed PIP strategy but also showcases the harmonious integration between the proposed DPDP theory, DPDP-based NT-TPU, and PIP strategy.
Fig. 10. Feasibility validation of proposed PIP strategy. (a)-(c) Deformed patterns at three different moments (from Visualization 1); (d)-(f) The corresponding 3D reconstructed results (from Visualization 2).
3.5 High-speed 3D measurement comparison
Finally, a comparative experiment for high-speed 3D measurement is conducted between the PSP-based NT-TPU and proposed method, as shown in Fig. 11. The motion speed and direction of the industrial workpieces are consistent between the two experiments. The captured frame rate is set at 200 Hz, and the total number of captured frames is set as 810. For PSP-based NT-TPU, the phase-shifting gratings are 16 pixels per period, while the auxiliary gratings are 57 pixels per period. The deformed patterns and 3D reconstructed results at three different moments are shown in Figs. 11(a)–11(c), respectively. Due to the smaller allowable maximum phase error range and larger frequency difference between phase-shifting and auxiliary gratings, the 3D reconstructed results obtained by PSP-based NT-TPU exhibit jump and block errors. The average computational time is calculated as 0.345 seconds. Furthermore, since six designed gratings are cyclically projected and captured according to the sequence $I_1^{am}$, $I_\textrm{2}^{am}$, $I_\textrm{3}^{am}$, $I_1^{pm}$, $I_2^{pm}$, $I_3^{pm}$, PSP-based NT-TPU generates one reconstruction for every 6 patterns. Therefore, 135 3D reconstructed results are obtained for the captured 810 deformed patterns. All the deformed patterns and 3D reconstructed results can be viewed in Visualization 3 and Visualization 4, respectively.
Fig. 11. High-speed 3D measurement comparison. (a)-(c) By PSP-based NT TPU (from Visualization 3 and Visualization 4); (d)-(f) By proposed method (from Visualization 5 and Visualization 6).
For proposed method, the deformed patterns and 3D reconstructed results at three different moments are shown in Figs. 11(d)–11(f), respectively. It can be observed that accurate 3D reconstructed results are obtained. The average computational time for proposed method is only 0.050 seconds. Moreover, 486 3D reconstructed results are obtained from the captured 810 deformed patterns, marking a 3.6-fold increase compared to that of PSP-based NT-TPU. The increased number of reconstructions not only improves the reconstructing ratio but also enhances data analysis capabilities. Visualization 5 displays all deformed patterns, while Visualization 6 shows all 3D reconstructed results. The comparative experiment demonstrates that proposed method achieves high-speed 3D measurement with high computational efficiency, high algorithmic robustness and high reconstructing ratio simultaneously.
4. Conclusions
In this paper, an innovative DPDP is proposed. By capturing the minimum three phase-shifting deformed patterns and establishing a brand-new math model, proposed DPDP can improve the computation efficiency and simplify the steps for obtaining the absolute phase difference just caused by the measured object itself from two phase resolutions, two times of phase unwrapping, and one subtraction in PSP to only one phase resolution and one time of phase unwrapping. Furthermore, a novel DPDP-based NT-TPU is proposed to directly unwrap the wrapped phase difference rather than the phase itself with high algorithmic robustness by using only two auxiliary complementary gratings with purposely designed lower frequency. It not only can improve sampling rate from 6 to 5 patterns per period, but also can further improve algorithmic robustness owing to the fringe carrier frequency independence of phase difference. Finally, a PIP strategy is proposed to further improve reconstructing ratio from one reconstruction for every 5 patterns in normal strategy to an equivalent of one reconstruction for every 1.67 patterns. A series of experiments show that proposed method achieves high computational efficiency, high algorithmic robustness and high reconstructing ratio simultaneously. It holds promising potential for applications in high-speed 3D measurement.
Funding
National Natural Science Foundation of China (Grant No.62375188).
Disclosures
The authors declare no conflicts of interest.
Data availability
Data will be made available on request.
References
1. H. An, Y. Cao, H. Li, et al., “High-speed 3-D reconstruction based on phase shift coding and interleaved projection,” Exp. Syst. Appl. 234, 121067 (2023). [CrossRef]
2. Q. Bao, J. Li, X. Li, et al., “Dynamic 3D measurement based on orthogonal fringe projection and geometric constraints,” Opt. Lett. 47(21), 5541–5544 (2022). [CrossRef]
3. H. An, Y. Cao, H. Wu, et al., “Spatial-temporal phase unwrapping algorithm for fringe projection profilometry,” Opt. Express 29(13), 20657–20672 (2021). [CrossRef]
4. Z. Wang, K. Li, N. Gao, et al., “High dynamic range 3D shape measurement based on crosstalk characteristics of a color camera,” Opt. Express 31(23), 38318–38333 (2023). [CrossRef]
5. H. Wu, Y. Cao, Y. Dai, et al., “Dynamic 3-D measurement based on temporal fringe order recognition,” IEEE Trans. Instrum. Meas. 72(1-12), 1–12 (2023). [CrossRef]
6. M. Takeda and K. Mutoh, “Fourier transform profilometry for the automatic measurement of 3-D object shapes,” Appl. Opt. 22(24), 3977–3982 (1983). [CrossRef]
7. Z. Zhang, Z. Jing, Z. Wang, et al., “Comparison of Fourier transform, windowed Fourier transform, and wavelet transform methods for phase calculation at discontinuities in fringe projection profilometry,” Opt. Laser Eng. 50(8), 1152–1160 (2012). [CrossRef]
8. V. Srinivasan, H. C. Liu, and M. Halioua, “Automated phase-measuring profilometry of 3-D diffuse objects,” Appl. Opt. 23(18), 3105–3108 (1984). [CrossRef]
9. J. Wang, Y. Cao, H. Wu, et al., “Absolute phase retrieval based on fringe amplitude encoding without any additional auxiliary pattern,” Opt. Express 31(25), 41952–41966 (2023). [CrossRef]
10. S. Zhang, “High-speed 3D shape measurement with structured light methods: A review,” Opt. Laser Eng. 106, 119–131 (2018). [CrossRef]
11. X. Su and W. Chen, “Reliability-guided phase unwrapping algorithm: a review,” Opt. Laser Eng. 42(3), 245–261 (2004). [CrossRef]
12. X. He and K. Qian, “A comparative study on temporal phase unwrapping methods in high-speed fringe projection profilometry,” Opt. Laser Eng. 142, 106613 (2021). [CrossRef]
13. H. Zhang, Y. Cao, H. Li, et al., “Real-time computer-generated frequency-carrier Moiré profilometry with three-frequency heterodyne temporal phase unwrapping,” Opt. Laser Technol. 161, 109201 (2023). [CrossRef]
14. M. Zhang, Q. Chen, T. Tao, et al., “Robust and efficient multi-frequency temporal phase unwrapping: optimal fringe frequency and pattern sequence selection,” Opt. Express 25(17), 20381–20400 (2017). [CrossRef]
15. C. Zuo, Q. Chen, G. Gu, et al., “High-speed three-dimensional profilometry for multiple objects with complex shapes,” Opt. Express 20(17), 19493–19510 (2012). [CrossRef]
16. G. Garnica, M. Padilla, and M. Servin, “Dual-sensitivity profilometry with defocused projection of binary fringes,” Appl. Opt. 56(28), 7985–7989 (2017). [CrossRef]
17. G. Sansoni, M. Carocci, and R. Rodella, “Three-dimensional vision based on a combination of gray-code and phase-shift light projection: analysis and compensation of the systematic errors,” Appl. Opt. 38(31), 6565–6573 (1999). [CrossRef]
18. Y. Wang and S. Zhang, “Novel phase-coding method for absolute phase retrieval,” Opt. Lett. 37(11), 2067–2069 (2012). [CrossRef]
19. J. M. Huntley and H. Saldner, “Temporal phase-unwrapping algorithm for automated interferogram analysis,” Appl. Opt. 32(17), 3047–3052 (1993). [CrossRef]
20. J. Wang, Y. Yang, P. Xu, et al., “Noise-induced phase error comparison in multi-frequency phase-shifting profilometry based on few fringes,” Opt. Laser Technol. 159, 109034 (2023). [CrossRef]
21. K. S. Mustafin and V. A. Seleznev, “Methods of increasing the sensitivity of holographic interferrometry,” Sov. Phys. Usp. 13(3), 416 (1970). [CrossRef]
22. J. Wang and Y. Yang, “Phase extraction accuracy comparison based on multi-frequency phase-shifting method in fringe projection profilometry,” Measurement 199, 111525 (2022). [CrossRef]
23. V. I. Gushov and Y. N. Solodkin, “Automatic processing of fringe patterns in integer interferometers,” Opt. Laser Eng. 14(4-5), 311–324 (1991). [CrossRef]
24. J. Zhong and Y. Zhang, “Absolute phase-measurement technique based on number theory in multifrequency grating projection profilometry,” Appl. Opt. 40(4), 492–500 (2001). [CrossRef]
25. Z. Huang, Y. Cao, A. Zhai, et al., “A novel 3D measuring method for discontinuous object by non-integral twin-frequency grating,” Optik 123(21), 1915–1919 (2012). [CrossRef]
26. Y. Ding, J. Xi, Y. Yu, et al., “Recovering the absolute phase maps of two fringe patterns with selected frequencies,” Opt. Lett. 36(13), 2518–2520 (2011). [CrossRef]
27. J. Song, D. L. Lau, Y. Ho, et al., “Automatic look-up table based real-time phase unwrapping for phase measuring profilometry and optimal reference frequency selection,” Opt. Express 27(9), 13357–13371 (2019). [CrossRef]
28. S. Yang, G. Wu, Y. Wu, et al., “High-accuracy high-speed unconstrained fringe projection profilometry of 3D measurement,” Opt. Laser Technol. 125, 106063 (2020). [CrossRef]
29. D. Zheng and F. Da, “Phase coding method for absolute phase retrieval with a large number of codewords,” Opt. Express 20(22), 24139–24150 (2012). [CrossRef]
30. Y. Chen, P. Yao, S. Gai, et al., “A self-alignment XOR coding strategy resistant to global illumination,” Measurement 193, 110767 (2022). [CrossRef]
31. Z. Wu, W. Guo, and Q. Zhang, “Two-frequency phase-shifting method vs. Gray-coded-based method in dynamic fringe projection profilometry: A comparative review,” Opt. Laser Eng. 153, 106995 (2022). [CrossRef]
32. Q. Ma, Y. Cao, C. Chen, et al., “Intrinsic feature revelation of phase-to-height mapping in phase measuring profilometry,” Opt. Laser Technol. 108, 46–52 (2018). [CrossRef]
33. Y. Xiao, Y. Cao, and Y. Wu, “Improved algorithm for phase-to-height mapping in phase measuring profilometry,” Appl. Opt. 51(8), 1149–1155 (2012). [CrossRef]
34. Z. Zhang, S. Huang, S. Meng, et al., “A simple, flexible and automatic 3D calibration method for a phase calculation-based fringe projection imaging system,” Opt. Express 21(10), 12218–12227 (2013). [CrossRef]
35. C. Li, Y. Cao, C. Chen, et al., “Computer-generated Moiré profilometry,” Opt. Express 25(22), 26815–26824 (2017). [CrossRef]
36. K. H. Rosen, “Elementary number theory and its applications,” Math. Gaz. 72(460), 162–163 (1988).
37. S. Lv and K. Qian, “Modeling the measurement precision of fringe projection profilometry,” Light: Sci. Appl. 12(1), 257 (2023). [CrossRef]
38. J. Zhang, B. Luo, X. Su, et al., “Depth range enhancement of binary defocusing technique based on multi-frequency phase merging,” Opt. Express 27(25), 36717–36730 (2019). [CrossRef]
39. Y. Wang, C. Wang, J. Cai, et al., “Improved spatial-shifting two-wavelength algorithm for 3D shape measurement with a look-up table,” Appl. Opt. 60(16), 4878–4884 (2021). [CrossRef]
40. W. Lohry and S. Zhang, “Genetic method to optimize binary dithering technique for high-quality fringe generation,” Opt. Lett. 38(4), 540–542 (2013). [CrossRef]
