1. Introduction

Recently, three-dimensional (3D) shape measurement has become one of the most active research areas in optical metrology [1]. Among 3D shape measurement techniques, the structured light method possesses advantages of fast-speed, high-accuracy and non-destruction, and has been applied in diverse fields [2–4], including reverse engineering, hyperspectral imaging, and microscopic measurement.

In structured light technology, the phase recovery from the fringe patterns is required and the main popular methods include phase-shifting algorithm [5], Fourier transform method [6], windowed Fourier transform [7] and deep learning [8]. By the above fringe analysis methods only wrapped phase ranging from $-{\pi}$ to ${\pi}$ can be obtained, and the phase unwrapping is required to obtain a continuous phase distribution. Conventional phase unwrapping methods can be classified into two categories, the spatial phase unwrapping (SPU) and temporal phase unwrapping (TPU) [9]. SPU provides a relative phase by integrating the phase gradient over a path that covers the domain of interest. Although SPU has the advantages of simplicity and convenience, it fails in simultaneously measurement of multiple isolated objects. In contrast, TPU unwraps the wrapped phase by capturing additional images, and the retrieval of absolute phase is achieved. This method is competent for the measurement of multiple isolated objects and has the advantage of stronger suppression of noise than SPU. In a review paper, Zuo et al. [10] thoroughly studied three types of TPU methods, including multi-frequency, multi-wavelength and number-theoretical approaches.

Although TPU can retrieve the absolute unwrapped phase, it is not suitable for high-speed measurement applications due to the additional images’ acquisition. In order to solve aforementioned problems, An et al. [11] proposed an absolute phase unwrapping method based on geometric constraints of the structured light system. In the method, an artificial absolute phase map, ${\Phi _{\min }}$, at a given virtual depth plane $z = {z_{\min }}$, is firstly generated based on the internal and external parameters of the calibrated structured light system. Then, the fringe order of each pixel in the wrapped phase map is determined by computing the difference between the wrapped phase and the artificial absolute phase. Finally, with the determined fringe orders, the absolute phase is obtained from the wrapped phase. The geometric constraint method has the advantages of high-speed, simple system setup, simultaneous multiple objects’ measurement and robustness to noise. This method has also been applied to other phase recovery to improve the accuracy and speed of 3D measurement. For example, Hyun and Zhang [12] used this method to enhance two-frequency phase-shifting. Li et al. [13] proposed a single-shot method based on geometric constraint to overcome the limitation in Fourier transform profilometry and reduce the motion-induced error [14]. In recent years, researchers have also proposed many methods to improve its measurement depth [9,15,16].

However, we found in our actual experiments that the selection of closest depth distance for the virtual plane used in reference [11] to generate an artificial absolute phase map was not always correct, and for some systems the maximum depth distance should be selected. Therefore, how to determine the correct depth distance for the virtual plane in a given structured light system is the first important issue in the application of geometric constraint.

In order to solve this issue, in this paper, based on the theory of An et al. [11], the monotonicity of the depth $\textrm{z}({u^c},{v^c})$ with respect to projector pixel coordinate of ${u^p}$ by means of geometric constraint is theoretically investigated and a discriminant of $\Delta ({u^c},{v^c})$ determined by the internal and external parameters of the calibrated structured light system is presented. It is shown that as $\Delta ({u^c},{v^c}) \ge 0$ at arbitrary point on the CCD pixel coordinates, the minimum depth distance is selected for the virtual plane to create the artificial absolute phase map, and the maximum depth distance must be selected as $\Delta ({u^c},{v^c}) \le 0$. Several experiments with different structured light systems are conducted to verify the correctness of the proposed monotonic discriminant of $\Delta ({u^c},{v^c})$.

Section 2 explains principles of the proposed method. Section 3 presents several experimental results to validate the method. Section 4 gives the summary of this paper.

2. Principle

2.1. N-step phase-shifting algorithm

For the N-step phase-shifting algorithm with equal phase shifts, the nth fringe pattern can be described as

2.2. Structured light system model and calibration

In structured light system, a well-known linear pinhole model is usually used to describe the imaging process of a camera. Its mathematical expression is as follows,

In the actual calibration process, in order to make the projector be able to capture images, Zhang and Huang [18] proposed a phase assistant method to establish a one-to-one correspondence relationship between the camera and projector pixels using two sets of orthogonal sinusoidal fringe patterns. Then the calibration target images captured by CCD are mapped onto the DMD plane using this established correspondence.

2.3. Absolute phase unwrapping using geometric constraint

Once the structured light system is calibrated by the method described in section 2.2, the projector matrices ${P^c}$ and ${P^p}$ can be estimated. Then, there are six equations with seven unknowns $({s^c},{s^p},{x^w},{y^w},{z^w},{u^p},{v^p})$ provided by Eqs. (4) and (6). Generally, in order to acquire 3D shape information of an object, another constraint equation is provided by the retrieval of absolute phase from the captured fringe patterns. The projector pixel coordinates ${u^p}$ and ${v^p}$ can be determined by the following equations,

An et al. [11] proposed an alternative approach. It was suggested that if a virtual plane was placed at the closest distance from the camera coordinate system ${\textrm{z}^w} = z_{\min }^w$, the remaining six parameters could be solved uniquely. With Eq. (4), the correspondent $x_{\min }^w$ and $y_{\min }^w$ for each camera pixel $({u^c},{v^c})$ can be expressed as,

2.4. Monotonicity analysis

In this part, we theoretically investigate the monotonicity of absolute phase unwrapping by geometric constraint in detail, and a monotonic discriminant is derived which is used to determine the correct depth distance of the virtual plane for a given structured light system.

Since ceil is a function that gives the nearest upper integer number, the unwrapped phase $\Phi ({u^c},{v^c})$ is always greater than the generated phase $\Phi _{\min }^V({u^c},{v^c})$ for arbitrary point on the CCD pixel coordinates, and the mathematical expression is as

The discriminant gives the insight that there may be a monotonically decreasing form of the depth distance of $z({u^c},{v^c})$ with respect to the projector pixel coordinate of ${u^p}$. Obviously, the maximum depth distance, rather than the minimum depth distance, should be selected to generate the absolute phase of ${\Phi _{\max }}({u^c},{v^c})$ for the phase unwrapping in this case. In a summary, as $\Delta ({u^c},{v^c}) \ge 0$ at arbitrary point on the CCD pixel coordinates the minimum depth distance is selected for the virtual plane to create the artificial absolute phase map, and the maximum depth distance is selected as $\Delta ({u^c},{v^c}) \le 0$.

3. Experiment

To verify the performance of the proposed discriminant, we develop a structured light system, shown in Fig. 1. The hardware system includes a single CCD (DAHENG MER-131-210U3M-L) and a projector (DLP6500). The camera’s resolution is $\textrm{1024} \times \textrm{1280}\; pixels$ and the projector’s resolution is $\textrm{1080} \times \textrm{1920}\; pixels$. The camera has a focal length of $\textrm{12}\textrm{.5 }mm$ and sensor unit size of $\textrm{4}\textrm{.8} \mu m \times \textrm{4}\textrm{.8} \mu m$. The projector has a focal length of $\textrm{23}\; mm$ and sensor unit size of $7.6 \mu m \times 7.6 \mu m$.

 

Fig. 1. Structured light system setup which meets the condition of monotonic increase of depth distance $z({u^c},{v^c})$ as projector pixel coordinate ${u^p}$ ($\Delta ({u^c},{v^c}) \ge 0$).

Download Full Size | PDF

In this paper, the structured light system is calibrated by the methods proposed in reference [19]. The calibration results of the system are as follows. The intrinsic parameter matrices for the camera and the projector are, respectively,

 

Fig. 2. Calculation result of the monotonic discriminant of the structured light system shown in Fig. 1.

Download Full Size | PDF

In the 3D measurement experiments, a sculpture shown in Fig. 3(a) is measured by the three-frequency phase-shifting approach firstly. We adopt nine-, five-, and five-step phase-shifting method for ${T_1} = 80\; pixels$, ${T_2} = 120\; pixels$ and ${T_3} = 270\; pixels$, respectively. Figure 3(b) shows the acquired wrapped phase from the fringe patterns of ${T_1} = 80\; pixels$. The 3D measurement result of the sculpture is shown in Fig. 3(c), where the minimum and maximum distances of this sculpture depth are ${z_{\min }} = 546.7465\; mm$ and ${z_{\max }} = 595.6813\; mm$, respectively.

 

Fig. 3. 3D shape measurement by the three-frequency phase-shifting approach. (a) Photograph of the measured sculpture; (b) Wrapped phase acquired from the fringe patterns of ${T_1} = 80\; pixels$; (c) 3D measurement result by this approach.

Download Full Size | PDF

Then, the minimum and maximum distances of the object are separately adopted to generate the artificial phase map for the absolute phase unwrapping by geometric constraint. To be concrete, the artificial phase map of ${\Phi _{\min }}$ is created at the virtual plane of $z = 546\; mm$, as described in section 2.3. The wrapped phase shown in Fig. 3(b) is then unwrapped by substituting the minimum phase map into Eq. (13) and the reconstructed 3D shape of the sculpture is shown in Fig. 4(a) where the minimum and maximum depths of the sculpture are ${z_{\min }} = 546.7465\; mm$ and ${z_{\max }} = 595.6813\; mm$, respectively. This result is the same as that obtained by the three-frequency phase-shifting approach. In a comparison, as the artificial phase map is created at the virtual plane of $z = 596\; mm$ for the absolute phase unwrapping, the reconstructed 3D shape is as Fig. 4(b), where the minimum and maximum depths are ${z_{\min }} = {629}{.7983}\; mm$ and ${z_{\max }} = 69\textrm{7}.\textrm{218}6\; mm$, respectively, different from that obtained by the three-frequency phase-shifting approach. The above experimental results prove that as $\Delta ({u^c},{v^c}) \ge 0$ at arbitrary point on the CCD pixel coordinates the minimum depth distance should be selected for the virtual plane to create the artificial absolute phase map by geometric constraint approach.

 

Fig. 4. Measurement result by geometric constraint approach. (a) 3D shape reconstruction by geometric constraint approach with $z = 546\; mm$ to be the depth of virtual plane; (b) 3D shape reconstruction by geometric constraint approach with $z = 596\; mm$ to be the depth of virtual plane.

Download Full Size | PDF

To more intuitively verify the above conclusion, we randomly select a row on the CCD pixel coordinates and then compare the depth distributions at this row in Figs. 3(c), 4(a) and 4(b). The results are shown in Fig. 5 where the red dashed line, the blue solid line and the black solid line represent the depth distributions at the row in Figs. 3(c), 4(a) and 4(b), respectively. Obviously, the red dashed line, the blue solid line are overlapping well, and the black solid line deviates upwards from the red dashed line, further verifying that as $\Delta ({u^c},{v^c}) \ge 0$ at arbitrary point on the CCD pixel coordinates the minimum depth distance should be selected for the virtual plane to create the artificial absolute phase map.

 

Fig. 5. The depth distributions at a randomly selected row acquired by different absolute phase unwrapping approaches, with red dashed line, blue solid line and black solid line representing the depth distribution in Figs. 3(c), 4(a) and 4(b), respectively.

Download Full Size | PDF

For the case of monotonic decrease of depth distance, we develop another structured light system using the same hardware equipment as shown in Fig. 6.

 

Fig. 6. Structured light system setup which meets the condition of monotonic decrease of depth distance $z({u^c},{v^c})$ as projector pixel coordinate ${u^p}$ ($\Delta ({u^c},{v^c}) \le 0$).

Download Full Size | PDF

The calibration of the system is conducted and the result is as follows. The intrinsic parameter matrices for the camera and the projector are, respectively,

 

Fig. 7. Calculation result of the monotonic discriminant of the structured light system shown in Fig. 6.

Download Full Size | PDF

We then apply this structured light system to the 3D shape measurements of the sculpture by different phase unwrapping approaches. Figure 8(a) shows the reconstructed 3D shape by three-frequency phase-shifting approach, in which the minimum and maximum depth values are ${z_{\min }} = 517.9246\; mm$ and ${z_{\max }} = 579.4308\; mm$, respectively. Figure 8(b) shows the reconstructed 3D shape by geometric constraint approach with the artificial phase map of ${\Phi _{\min }}$ created at the virtual plane of $z = 517\; mm$. The minimum and maximum depths are ${z_{\min }} = 463.6201\; mm$ and ${z_{\max }} = 516.9674\; mm$, respectively. Obviously, these values are different from that obtained by three-frequency phase-shifting approach. On the contrary, as the artificial phase map of ${\Phi _{\max }}$ is created at the virtual plane of $z = 580\; mm$ for the absolute phase unwrapping, the reconstructed 3D shape is as Fig. 8(c), where the minimum and maximum depths are ${z_{\min }} = 517.9246\; mm$ and $ {z_{\max }} = 579.4308\; mm$, respectively. These values are the same as that obtained by three-frequency phase-shifting approach. The above experimental results prove that as $\Delta ({u^c},{v^c}) \le 0$ at arbitrary point on the CCD pixel coordinates the maximum depth distance should be selected to create the artificial absolute phase map by geometric constraint approach.

 

Fig. 8. The 3D shape measurement results by different methods. (a) 3D shape reconstruction by three-frequency phase-shifting approach; (b) 3D shape reconstruction by geometric constraint approach with $z = 517\; mm$ to be the depth of virtual plane; (c) 3D shape reconstruction by geometric constraint approach with $z = 580\; mm$ to be the depth of virtual plane.

Download Full Size | PDF

We also randomly select a row on the CCD pixel coordinates and then compare the depth distributions at this row in Figs. 8(a)–8(c). The results are shown in Fig. 9 where the red dashed line, the black solid line and the blue solid line represent the depth distributions at the row in Figs. 8(a)–8(c), respectively. It can be seen that the red dashed line and the blue solid line are overlapping well, and the black solid line deviates downwards from the red dashed line, further verifying that as $\Delta ({u^c},{v^c}) \le 0$ at arbitrary point on the CCD pixel coordinates the maximum depth distance should be selected for the virtual plane to create the artificial absolute phase map.

 

Fig. 9. The depth comparison in a randomly selected row acquired by different phase unwrapping approaches, with red dashed line, black solid line and blue solid line representing the depth distributions in Fig. 8(a), Fig. 8(b) and Fig. 8(c), respectively.

Download Full Size | PDF

4. Summary

We have theoretically investigated the monotonicity of depth $z({u^c},{v^c})$ with respect to projector pixel coordinate of ${u^p}$ in the absolute phase unwrapping by geometric constraint, and presented a monotonic discriminant used to choose correct depth distance of the virtual plane to create the artificial absolute phase map in this paper. The monotonic discriminant indicates that as $\Delta ({u^c},{v^c}) \ge 0$ at arbitrary point on the CCD pixel coordinates, the minimum depth distance is selected for the virtual plane, and on the contrary, as $\Delta ({u^c},{v^c}) \le 0$ the maximum depth distance must be selected. Experimental results with different structured light systems demonstrate the validity of the theoretical analysis.