Theoretical analysis and experimental investigation of the Floyd-Steinberg-based fringe binary method with offset compensation for accurate 3D measurement

Di You; ZhiSheng You; Pei Zhou; Pei Zhou; JiangPing Zhu; JiangPing Zhu

doi:10.1364/OE.460519

1. Introduction

In the last years, digital fringe projection (DFP) techniques have been increasingly used for high-quality 3D measurement due to its flexibility [1], but most commercially available video projectors will introduce measurement error because of the projector nonlinearity. Besides 8-bits sinusoidal patterns are usually used and that will limit the measurement speed (typically $\leq$ 120 Hz). To overcome these issues, binary defocusing techniques were proposed, with which a properly defocused binary pattern is considered as a sinusoidal pattern in a good approximation.

The squared binary pattern is the earliest used binary fringe pattern [2], but it requires a high defocusing level to get a good sinusoidal fringe, which will limit the effective depth of field and cause a decrease in contrast. Further, a variety of techniques based on pulse width modulation have been proposed [3–5]. They all achieved better measurement quality compared with the squared binary method but fail to produce a high-performance 3D measurement when the binary patterns have a wide pitch.

Nowadays, dithering techniques called half-toning were also introduced into the binary defocusing techniques, which could improve fringe quality for wider fringe stripes [6]. Various dithering techniques have been proposed over the years including random dithering [7], ordered dithering [8], and error diffusion [9–12] techniques. Besides, some new techniques [13–17] used the iterative optimization methods to further improve the performance of the dithering techniques, and they are optimized from different aspects, such as intensity accuracy, phase accuracy and so on. But due to the complexity and time-consuming nature of the iterative optimization methods, the FS dithering technique [18], an error diffusion technique, is still one of the most commonly used binary fringe coding methods for its acceptable measurement accuracy. However, as shown in Ref. [17], there is a relatively large phase error when the fringe pitch is small, and the phase root mean square error (RMSE) of the defocused FS binary fringe decreases as the fringe pitch increases, similar to Fig. 1.

Fig. 1. The phase rms error for the FS dithering technique and the phase rms error after phase compensation.

Download Full Size | PDF

And through careful analysis, the reason for this result is due to the certain spatial offset of the fringe caused by FS dithering technique (the spatial offset is fixed for all fringe pitches, so the phase error it brings also decreases relatively as the fringe pitch increases). And after compensating for this offset, the phase rms error is greatly reduced in the case of small fringe pitch and remains basically the same for all fringe pitches, as shown in Fig. 1.

Next are some details about our findings. First of all, we found that the phase error of the defocused FS binary fringe has a non-zero average, which indicates that there is a fixed error between the recovered phase and the true phase, in addition to some random errors. Furthermore, both theoretical analysis and experiments have proved that the phase mean error for binary FS fringe is substantially the same under different defocusing levels and various phase-shift steps, which means the FS dithering technique causes a spatial offset from the original sinusoidal fringe. Through simulation and experiment, the spatial offset was proven to be basically the same for various fringe pitches. According to the analysis of the stereo triangulation principle, we can see that the spatial offset will introduce distortion to the final 3D measurement for monocular systems relying on triangulation. Finally, some experiments verified that our proposed phase compensation method could eliminate the effect of fringe offset, resulting in the improvements of the recovered phase quality and the 3D measurement accuracy.

The principle of the proposed method is introduced in Section 2. And experiments and discussion are given in Section 3 and Section 4 respectively. Finally, we summarize the findings in this paper in Section 5.

2. Principle

2.1 Floyd–Steinberg dithering technique

FS dithering technique [18] is a commonly used error-diffusion dithering technique which was later introduced to binary defocusing technique, and achieved good results [19].

Every pixel is quantized to 0 or 255 according to the threshold, and the quantization errors of the processed pixels are diffused to the unprocessed pixels according to a certain scale factor, thereby the quantization errors caused by the entire encoding are reduced. This process can be formally expressed as:

(1)$$\left\{\begin{array}{c} I^{\prime}(i, j)=I(i, j)+\sum_{i, j \in s} h(x, y) e(i-x, j-y), \\ I_{b}(i, j)=\left\{\begin{array}{c} 0\; \textrm{if } I^{\prime}(i, j)<127.5 \\ 255\; \textrm{if } I^{\prime}(i, j) \geq 127.5 \end{array}\right., \\ e(i, j)=I^{\prime}(i, j)-I_{b}(i, j), \end{array}\right.$$

where $I(i, j)$ is the original image, $I^{\prime }(i, j)$ is the image after adding the diffused quantization error, and $I_{b}(i, j)$ is the final quantized image. And $e(i, j)$ is the quantization error that is diffused to the surrounding pixels by a kernel $h(x, y)$. There are so many choices for kernel $h(x, y)$, and the kernel for FS dithering technique is:

(2)$$h_{F}(x, y)=\frac{1}{16}\left[\begin{array}{lll} - & * & 7 \\ 3 & 5 & 1 \end{array}\right].$$

2.2 N-steps phase-shifting algorithm

The phase-shifting algorithm (PSA) is often used in the field of FPP for high-accuracy 3D shape measurement, which can achieve pixel-wise phase measurement with high resolution and accuracy [20]. And it requires three phase-shifted fringe patterns at least, and the phase-shifted patterns can be expressed as:

(3)$$I_{n}(x, y)=a(x, y)+b(x, y) \cos \left[\phi(x, y)-\frac{2 n \pi}{N}\right],$$

where n=0,1,2,…,N-1, representing the phase-shift index. And $a(x, y)$ and $b(x, y)$ represent background and modulation intensities. Besides $\phi (x, y)$ is the phase which contains the desired information which can be extracted as follows:

(4)$$\phi(x, y)=\arctan \left[\frac{\sum_{n=0}^{N-1} I_{n}(x, y) \sin (2 n \pi / N)}{\sum_{n=0}^{N-1} I_{n}(x, y) \cos (2 n \pi / N)}\right].$$

Because of the characteristics of the arctan operator, the calculated phase result is in the range $-\pi$ to $\pi$, and it needs to be unwrapped by a phase-unwrapping algorithm [21].

2.3 N-steps phase-shifting fringe pattern encoded by Floyd–Steinberg dithering technique

To minimize the error caused by binarization, the following principles are recommended when designing the FS fringe pattern.

Principle 1: Do not encode multiple phase-shifted fringes separately. There are two ways to generate the FS fringe pattern for N-steps phase-shifting. To better explain, we illustrate these two methods in Fig. 2. Suppose we want to generate the FS fringe patterns for 4-steps phase-shifting with a pitch of 60 pixels, we can first create 4 standard sinusoidal patterns phase-shifted by ${\pi }/{2}$ and encode them as binary patterns respectively with FS dithering method, as shown in Fig. 2(a). Another way is to binarize a sinusoidal fringe pattern which is expanded by one period at first. Then cutting out the binary fringe patterns which we want from the whole binary pattern with a space interval of $1/4$ period, as shown in Fig. 2(b). Furthermore, the phase errors of these two methods were calculated in Fig. 2, which shows that the second method is more recommended because it has less phase error.
Principle 2: Choose the fringe pitch properly to avoid the phase-shift error. The fringe pitch should be a multiple of N for N-steps phase-shifting because the phase-shifting is realized by spatially moving the binary patterns. For example, a ${\pi }/{2}$ phase shift can be realized by moving $1/4$ period of the FS pattern.

Fig. 2. Two ways generating the FS phase-shifting fringe pattern. (a) First way: binarizing the fringe one by one; (b) second way: binarizing a bigger fringe then cutting out with certain space interval.

Download Full Size | PDF

2.4 Phase error of the binary Floyd–Steinberg fringe pattern

The defocused binary sinusoidal fringe pattern always has intensity errors caused by binarization which will eventually cause phase errors. To analyze the phase error of the binary FS fringe pattern clearly, a simple simulation was conducted. According to the principles mentioned above, we adopted the FS dithering method to generate the binary FS fringe patterns for 4-steps phase-shifting with a pitch of 36 pixels, and the Gaussian filter with a size of $5 \times 5$ pixels and a deviation of $5/3$ pixels is used to simulate the defocus. Then the difference between the recovered phase and the true phase (recovered from ideal sinusoidal fringe patterns) is the phase error. As shown in Fig. 3, the average of the phase error is 0.0334 rad, significantly greater than 0. The none-zero average of the phase error can infer that there is a fixed error between the recovered phase and the true phase, in addition to some random errors.

Fig. 3. Intensity error and phase error of defocused FS binary fringe pattern.

Download Full Size | PDF

Next, it will be proved that the average of the phase error between the phase retrieved from defocused binary fringe patterns and the ideal sinusoidal fringe patterns is the same under different defocusing levels with N-steps PSA.

Assuming that the intensity error between the binary fringe pattern and the standard fringe pattern can be regarded as a slight disturbance under the slightly defocused condition, then according to the work proposed by Li et al. [22], the phase error of the standard N-steps phase-shifting method is defined as:

(5)$$\Delta \phi(x,y)=\frac{2}{NB} \sum_{n=0}^{N-1} \sin (2 \pi f_{0} x+2 \pi n / N) \Delta I_{n}(x,y),$$

where $\Delta I_{n}(x,y)$ is the intensity error of the nth fringe pattern (from 0 to N-1), $\Delta \phi (x,y)$ is the phase error, $(x,y)$ is the coordinate of one pixel, $f_{0}$ is the carrier frequency, and $B$ is the fringe modulation.

And using a Gaussian filter $g$ acting on the slightly defocused binary fringe pattern to increase the defocusing level. Then the intensity error $\Delta I_{n}(x,y)$ and the fringe modulation $B$ will be changed like this:

(6)$$\begin{gathered} \Delta I_{n}^{\prime}(x,y)=g(x,y) \otimes \Delta I_{n}(x,y), \\ B^{\prime}=t_{1} B, \end{gathered}$$

where $\Delta I_{n}^{\prime }(x,y)$ is the new intensity error, $B^{\prime }$ is the new fringe modulation. “$\otimes$” represents convolution operation. $t_{1}$ is a fixed scale factor dependent on g.

So, the phase error of the binary fringe pattern after further defocusing can be expressed as:

(7)$$\begin{aligned} \Delta \phi^{\prime}(x,y) = & \frac{2}{NB^{\prime}} \sum_{n=0}^{N-1} \sin (2 \pi f_{0} x+2 \pi n / N) \Delta I_{n}^{\prime}(x,y)\\ = & \frac{2}{Nt_{1} B} \sum_{n=0}^{N-1} \sin (2 \pi f_{0} x+2 \pi n / N) \times (g(x,y) \otimes \Delta I_{n}(x,y)). \end{aligned}$$

And the average of the phase error of the binary fringe pattern after further defocusing can be expressed as:

(8)$$\begin{aligned} E(\Delta \phi^{\prime}) = & \frac{\sum_{{y}=1}^{{S}} \sum_{{x}=1}^{{T}} \Delta \phi^{\prime}({x}, {y})}{{S} \times {T}}\\ = & \frac{\sum_{{y}=1}^{{S}} \sum_{{x}=1}^{{T}}\left(\frac{2}{{Nt}_{1} {~B}} \sum_{{n}=0}^{{N-1}} \sin \left(2 \pi {f}_{0} {x}+2 \pi {n} / {N}\right) \times {g}({x}, {y}) \otimes \Delta {I}_{{n}}({x}, {y})\right)}{{S} \times {T}}\\ = & \frac{\frac{2}{{Nt}_{1} {~B}} \sum_{{n}=0}^{{N-1}}\left(\sum_{{y}=1}^{{S}} \sum_{{x}=1}^{{T}} \sin \left(2 \pi {f}_{0} {x}+2 \pi {n} / {N}\right) \times {g}({x}, {y}) \otimes \Delta {I}_{{n}}({x}, {y})\right)}{{S} \times {T}}. \end{aligned}$$

Expanding the convolution operation, we can establish the following equation:

(9)$$\begin{aligned} & \sum_{y=1}^{S} \sum_{x=1}^{T} \sin \left(2 \pi f_{0} x+2 \pi n / N\right) \times g(x, y) \otimes \Delta I_{n}(x, y) \\ & =\sum_{y=1}^{S} \sum_{x=1}^{T} \sin \left(2 \pi f_{0} x+2 \pi n / N\right) \times \sum_{v=1}^{S} \sum_{u=1}^{T} \Delta I_{n}(u, v) \times g(x-u, y-v) \\ & =\sum_{v=1}^{S} \sum_{u=1}^{T} \Delta I_{n}(u, v) \times \sum_{y=1}^{S} \sum_{x=1}^{T} \sin \left(2 \pi f_{0} x+2 \pi n / N\right) \times g(x-u, y-v) \\ & =\sum_{v=1}^{S} \sum_{u=1}^{T} \Delta I_{n}(u, v) \times g({-}u,-v) \otimes \sin \left(2 \pi f_{0} u+2 \pi n / N\right). \end{aligned}$$

So, due to the symmetry of the Gaussian kernel, we can establish the following equation:

(10)$$\begin{aligned} & \sum_{y=1}^{S} \sum_{x=1}^{T} \sin \left(2 \pi f_{0} x+2 \pi n / N\right) \times g(x, y) \otimes \Delta I_{n}(x, y) \\ & \quad=\sum_{y=1}^{S} \sum_{x=1}^{T} \Delta I_{n}(x, y) \times g(x, y) \otimes \sin \left(2 \pi f_{0} x+2 \pi n / N\right) \\ & \quad=t_{1} \sum_{y=1}^{S} \sum_{x=1}^{T} \Delta I_{n}(x, y) \times \sin \left(2 \pi f_{0} x+2 \pi n / N\right). \end{aligned}$$

Substituting Eq. (10) into Eq. (8), we obtain:

(11)$${E}\left(\Delta \phi^{\prime}\right)=\frac{\sum_{{y}=1}^{{S}} \sum_{{x}=1}^{{T}}\left(\frac{2}{{NB}} \sum_{{n}=0}^{{N-1}} \sin \left(2 \pi {f}_{0} {x}+2 \pi {n} / {N}\right) \times \Delta {I}_{{n}}({x}, {y})\right)}{{S} \times {T}}={E}(\Delta \phi).$$

So, we proved that the change of the defocus will not affect the average of the phase error. Besides, a simple experiment using Gaussian filters with different sizes $S \times S$ pixels and deviation $s$ pixels (S = [5,9,13], s = S/3) to simulate the various defocusing levels also verified this conclusion, as shown in Fig. 4. Obviously, the average of the phase error for different defocusing levels is substantially the same, which are 0.0335, 0.0333, 0.0333 rad respectively. And the standard deviation (STD) of the phase error is reduced as the amount of defocus increases, which are 0.0255, 0.0099, 0.0055 rad respectively.

Fig. 4. The phase error of the binary fringe patterns with pitch T=36 pixels under different defocus simulated by Gaussian filters with different sizes (a) $5 \times 5$ pixels; (b) $9 \times 9$ pixels; (c) $13 \times 13$ pixels.

Download Full Size | PDF

Similarly, we will prove that the average of the phase error would not change for various phase-shift steps. According to Principle 1 in section 2.3, the intensity error of the nth fringe pattern $\Delta I_{n}(x,y)$ can be represented by intensity error of the first fringe pattern $\Delta I_{0}(x,y)$ as:

(12)$$\Delta {I}_{{n}}({x}, {y})=\Delta {I}_{0}\left({x}+\frac{{n}}{{Nf}_{0}}, {y}\right).$$

And we can infer that:

(13)$$\sin \left(2 \pi f_{0} x+\frac{2 \pi n}{N}\right) \Delta I_{n}(x, y)=\sin \left(2 \pi f_{0}\left(x+\frac{n}{N f_{0}}\right)\right) \Delta I_{0}\left(x+\frac{n}{N f_{0}}, y\right).$$

If the length T of the pattern in the x direction is large enough, the following equation holds for any $n$:

(14)$$\sum_{y=1}^{S} \sum_{x=1}^{T} \sin \left(2 \pi f_{0} x+\frac{2 \pi {n}}{N}\right) \Delta I_{n}(x, y)=\sum_{y=1}^{S} \sum_{x=1}^{T} \sin \left(2 \pi f_{0} x\right) \Delta I_{0}(x, y).$$

So, the average of the phase error can be expressed as:

(15)$$\begin{aligned} E(\Delta \phi) = & \frac{\sum_{{y}=1}^{{S}} \sum_{{x}=1}^{{T}}\left(\frac{2}{{NB}} \sum_{{n}=0}^{{N}-1} \sin \left(2 \pi {f}_{0} {x}+\frac{2 \pi {n}}{{N}}\right) \Delta {I}_{{n}}({x}, {y})\right)}{{S} \times {T}}\\ = & \frac{\frac{2}{B} \sum_{y=1}^{S} \sum_{x=1}^{T} \sin \left(2 \pi {f}_{0} {x}\right) \Delta {I}_{0}({x}, {y})}{{S} \times {T}}. \end{aligned}$$

From Eq. (15), it can be seen that the average of the phase error is independent of phase-shift steps $N$. And the different phase results of one defocused FS fringe pattern recovered by 4-steps and 6-steps PSA confirmed this point, as shown in Fig. 5. As we can see, the more steps can help reduce the phase STD error, but do not change the phase mean error.

Fig. 5. The phase error of the binary fringe patterns with pitch T=60 pixels recovered by 4-steps and 6-steps PSA. (a) The defocused binary sinusoidal fringe; (b) result of 4-steps PSA; (c) result of 6-steps PSA.

Download Full Size | PDF

2.5 Offset caused by Floyd–Steinberg dithering technique

From the above analysis, the non-zero phase mean error means that the FS dithering technique brings a certain phase shift to the binary fringe, which can also be regarded as a certain spatial offset. It is worth mentioning here that the spatial offset of fringe will be collectively referred to as fringe offset, which represents the movement of fringe in pixel space. For example, suppose fringe pitch T=48 pixels and phase shift is $\pi /2$, then fringe offset would be 12 pixels. Overall, the phase shift and fringe offset have the relationship as follows:

(16)$$\textrm{fringe offset }={T} \times \frac{\textrm{phase shift }}{2 \pi}.$$

In this section, we will focus on this offset. Firstly, we simulated to calculate the phase shifts caused by FS dithering technique for various pitches T=24,36,48,60,72,84,96,108 and 120 pixels and plotted them as a function of the fringe pitches in Fig. 6(a). It can be seen from this figure that the phase shift caused by FS technique decreases as the fringe pitch increases, but the relationship between the two is not linear. Furthermore, a phase shift could be converted into a spatial offset, according to Eq. (16). And it could be found that the spatial offset brought by FS is basically the same for different pitches, about 0.19 pixels, as shown in Fig. 6(b). Then to show the spatial offset more intuitively, the corresponding cross-sections of the defocused FS fringe and sinusoidal fringe were demonstrated in Fig. 6(c), from which a tiny offset can be seen.

Fig. 6. (a) The phase shifts caused by FS dithering technique for various pitches; (b) the fringe offsets caused by FS dithering technique for various pitches; (c) a tiny offset between the defocused FS fringe and sinusoidal fringe.

Download Full Size | PDF

2.6 3D distortion caused by Floyd–Steinberg dithering technique

In this paper, we mainly considered the monocular structured light measurement system containing one camera and one projector for its simple structure and widespread use. Besides, both projector and camera are modeled using pinhole model and the system could be calibrated using the accurate methods in [23,24], and the 3D point cloud is obtained by method in [25]. If the system has been accurately calibrated in advance, accurately matching the corresponding points of the projected image and the captured image will be a key factor affecting the accuracy of 3D measurement according to the principle of stereo triangulation. And we mainly use the phase value to match the corresponding points in DFP, so the phase shift or fringe offset caused by FS dithering technique mentioned above will lead to a mismatch and bring distortion to 3D measurement results, as shown in Fig. 7. In order to eliminate this distortion, the original phase $\varphi _{ori}$ retrieved from N-step phase-shifting FS binary fringes needs to be compensated as follows:

(17)$$\left\{\begin{array}{c} \varphi_{comp}=2 \pi \times \frac{0.19}{T}, \\ \varphi_{new}=\varphi_{ori}-\varphi_{comp}, \end{array}\right.$$

where $\varphi _{comp}$ represents phase compensation amount associated with fringe pitch $T$, and $\varphi _{new}$ is the new phase result after compensation.

Fig. 7. Schematic diagram of measurement distortion caused by fringe offset for a monocular structured light measurement system.

Download Full Size | PDF

3. Experiments

In this section, we verified the performance of the proposed method through a 3D measurement system. It has a projector (DLP 4500) and a digital camera (MER-160-227U3M/C). The resolution of the projector is 912*1140 and the camera is 1280*1024. And the baseline length of the system is about 155 mm and the distance of the measured object to the system is about 550 mm. The experiment setup is shown in Fig. 8. Besides, we chose the three-frequency temporal phase unwrapping method to avoid phase unwrapping errors affecting the experimental result.

Fig. 8. The experiment setup.

Download Full Size | PDF

3.1 Experiments to analyze phase error

In this experiment, we mainly verified that the phase compensation method proposed in this paper could effectively reduce the phase error of the defocused FS binary fringes. From the 3D distortion analysis mentioned above, the difference between the phase of the defocused binary fringe and the phase of the ideal sinusoidal fringe is a very important cause of 3D measurement error, so we can use the phase RMSE, expressed as follows:

(18)$${RMSE}=\left(\frac{1}{N \times M} \sum_{y=1}^{N} \sum_{x=1}^{M}\left[\varphi_{d}(x, y)-\varphi_{i}(x, y)\right]^{2}\right)^{1 / 2},$$

to judge whether our proposed method can effectively improve the phase accuracy of the FS binary fringe pattern. $\varphi _{i}(x, y)$ and $\varphi _{d}(x, y)$ are the continuous phase maps retrieved from the ideal sinusoidal and the defocused FS binary fringe patterns, respectively. Besides a white plate was used as the test object for this experiment for its ease of measurement and the reference ideal phase was obtained by the 12-step PSA with standard sinusoidal fringes. In order to ensure sufficient comprehensiveness, fringe patterns with various pitches T= 24,36,48,60,72,84,96,108 and 120 pixels were used and 3 different defocusing levels were considered, which was implemented by adjusting the defocus of the projector. Figure 9 shows one of the experimental results with fringe pitch T = 60 pixels under low defocusing level. Furthermore, in order to eliminate the influence of the background area, we chose to calculate the phase of the same rectangular area for different fringes, marked by dashed box in Figs. 9(a) and (d). And experimental results in Figs. 9(g) and (h) show that phase RMSE could be reduced from 0.0293 rad to 0.0218 rad with phase compensation. It is worth mentioning that the original phase of the captured deformed FS fringe was recovered by the 4-step PSA in this experiment and the new phase after compensation was obtained according to Eq. (17) in Section 2.6.

Fig. 9. Experimental results with fringe pitch T = 60 pixels under low defocusing level. (a)-(c) captured deformed standard sinusoidal fringes, corresponding wrapped and unwrapped phase maps; (d)-(f) captured deformed defocused FS binary fringes, corresponding wrapped and unwrapped phase maps; (g),(h) phase error maps before compensation and after compensation.

Download Full Size | PDF

In order to succinctly and effectively present the whole experimental results, the RMSE of the original phase before compensation and the new phase after compensation was demonstrated as a function of fringe pitches in Fig. 10, and Figs. 10(a)-(c) represents the result under low, medium, high defocusing levels, respectively. From the experimental results, we can clearly find that, under different defocusing levels, the phase RMSE between the defocused FS binary fringes and the standard sinusoidal fringes can be significantly reduced by phase compensation, especially for fringes with small pitches, which is beneficial to use high frequency fringes for more accurate measurements. So this experiment validated our proposed method can indeed effectively improve the phase quality of the FS binary fringes.

Fig. 10. The phase RMSE before compensation and after compensation under (a) low, (b) medium, and (c) high defocusing levels.

Download Full Size | PDF

In addition to this, we also used this experiment on a white plate to demonstrate that the offset of FS binary fringes is basically the same for different fringe pitches. First of all, under low defocusing level, we obtained the phase error maps for FS binary fringe patterns with various pitches T= 24,36,48,60,72,84,96,108 and 120 pixels. Then calculate the mean phase error for all phase error maps. Through the theoretical analysis in Section 2.4, the mean phase error could be regarded as the phase shift. Therefore, the phase shifts could be displayed as a function of the fringe pitches in Fig. 11(a). And according to Eq. (16), we also could convert the phase shifts to the spatial offsets and plotted them in Fig. 11(b). Obviously, there is the same conclusion as the simulation in Section 2.5 that the offset caused by FS is basically the same for different fringe pitches, about 0.19 pixels.

Fig. 11. (a) The phase shifts caused by FS for various pitches; (b) the fringe offsets caused by FS for various pitches.

Download Full Size | PDF

3.2 Experiments to analyze 3D error

In the previous section, we have experimentally verified that phase compensation could eliminate the offset of the fringes and improve the phase accuracy of FS binary fringes. And in this section, we will further demonstrate that phase compensation has the ability to correct the 3D distortion caused by fringe offset and improve the final 3D measurement accuracy.

First of all, to clearly demonstrate the 3D distortion caused by fringe offset, we measured a white plate with standard 12-step phase-shifting sinusoidal fringes with pitch T = 36 pixels to get the reference 3D result, as shown in Fig. 12(d), then added a phase increment to the original phase map to simulate fringe offset (the phase increment value is equal to the phase compensation value for FS binary fringes with the same pitch, about 0.034 rad), thus we will obtain a new 3D result that is displayed in Fig. 12(e). Intuitively, these two 3D results are almost the same, but when we display them in the same coordinate system and zoom in on the central area (marked with green and red squares), we can find that they do not coincide, as shown in Fig. 12(f). In order to measure the difference between the two 3D results more accurately, they need to be registered first. Since the correspondence between the points in the two 3D result is known, it is best to use the registration method in [26], which could find the least-squares solution of a rigid transformation that minimizes the sum of the distances between corresponding points. After registration, the central area of these two 3D results basically coincide, as shown in Fig. 12(g). What about other areas? Next we display the whole difference between the two results after registration in Fig. 12(h), which clearly shows that the 3D measurement affected by fringe offset is significantly distorted compared to the reference 3D result. Specifically, the 3D errors in the central area are generally smaller than those in the surrounding areas and the RMSE reaches 0.09 mm. Besides the farther from the center area, the greater the error. So the regular distribution of 3D errors indicates the existence of 3D distortion.

Fig. 12. Experiments on a white plate showing 3D distortion caused by fringe offset. (a) The photo of the white plate; (b),(c) the captured deformed fringe images with standard sinusoidal fringes and the corresponding phase map; (d),(e) the reference 3D result and the 3D result affected by fringe offset; (f),(g) the central area of these two 3D results before and after registration; (h) the whole 3D error distribution after registration.

Download Full Size | PDF

Next, the same white plate was also measured with 4-step phase-shifting FS binary fringes with pitch T = 36 pixels, and the 3D measurement is displayed in Fig. 13(b). Besides the new 3D result after phase compensation is shown in Fig. 13(c). Both 3D results need to be first registered with the reference 3D result according to the same method above, and reference 3D result is demonstrated in Fig. 13(a), which is obtained with standard 12-step phase-shifting sinusoidal fringes. After registration, it is easy to get the 3D error distributions before and after phase compensation, which are shown in Figs. 13(d),(e) respectively. Combining Fig. 12(h) and Fig. 13(d), we can find that 3D results obtained from FS binary fringes has similar 3D distortions to the results affected by fringe offset. Besides the 3D error becomes more evenly distributed in space after compensation, which means 3D distortion is eliminated, as shown in Fig. 13(e). In addition to this, the RMSE dropped from 0.23 mm to 0.21 mm with phase compensation, about 9%.

Fig. 13. Experimental results for a white plate before and after compensation. (a) The reference 3D result; (b),(c) 3D result before and after compensation; (d),(e) 3D error distributions before and after compensation.

Download Full Size | PDF

Further more, two separate plaster models were also used to experimentally analyze 3D errors. The photo of the plaster model is shown in Fig. 14(a), and it was measured with standard 12-step phase-shifting sinusoidal fringes to get the reference 3D result (see Fig. 14(d)), meanwhile with 4-step phase-shifting FS binary fringes to get the original 3D result (see Fig. 14(e)), and the new 3D result was obtained with the new retrieved phase after phase compensation (see Fig. 14(f)). Figures 14(b),(c) are some samples of the captured deformed fringe images with standard sinusoidal fringes and FS binary sinusoidal respectively. From these 3D results, it is difficult to distinguish that the 3D result after compensation is better than that before compensation. This is because the phase compensation operation only corrects the overall 3D distortion caused by the fringe offset but does not optimize the 3D surface quality. So in order to better demonstrate the 3D accuracy improvement, we need to first register the two 3D results before and after compensation with the reference 3D result as before. After registration, two 3D error distributions are demonstrated in Figs. 14(g),(h) respectively. Similar to the results of the plate experiments before, the 3D error distribution before compensation is not uniform. The 3D errors in the left model’s lower part and the right model’s hair are significantly larger than other regions, which is the result of 3D distortion. Meanwhile, the phase compensation helps to remove 3D distortion, making the error distribution even, as shown in Fig. 14(h). Not only that, but it also reduces RMSE from 0.28 mm to 0.26 mm, about 7%.

Fig. 14. The results for experiments on two separate plaster models. (a) The photo of the plaster models; (b),(c) the captured deformed fringe images with standard sinusoidal fringes and FS binary sinusoidal fringes; (d) the reference 3D result; (e),(f) 3D result before and after compensation; (g),(h) 3D error distributions before and after compensation.

Download Full Size | PDF

4. Discussion

There are several aspects to be explained and discussed:

1. We mainly discussed the 3D distortion caused by fringe offset to monocular systems relying on triangulation. However, in fact, the offset would not bring 3D errors to the monocular systems based on phase-to-height mapping algorithm [27,28] and the binocular systems containing two cameras and one projector. The phase used in the former is usually the phase difference of the object surface with respect to the reference plane, so the effect of fringe offset is eliminated. As for the latter, the fringes captured by the left and right cameras have the same offset, so it would not matter either.
2. We strongly recommend using STD to evaluate the phase error of the defocused binary fringe instead of RMSE. As we all know, RMSE can be represented by the STD and the mean of the error: $(19)$${RMSE}=\sqrt{\delta(\Delta \phi)^{2}+E(\Delta \phi)^{2}},$$$ where $\delta (\Delta \phi )$ is the STD and $E(\Delta \phi )$ is the mean. Through the previous analysis in Section 2.4, the average of the phase error $E(\Delta \phi )$ could be considered as fringe offset because it is invariable for various defocusing levels and various phase-shift steps, which can easily be eliminated by phase compensation, or in some cases does not affect the 3D measurement results according to the discussion above. Therefore, compared with RMSE, the STD is not affected by the mean phase error and could directly characterize the quality of the final 3D reconstructed surface.

5. Conclusion

In this paper, we found that FS dithering technique would cause a certain fringe offset for the encoded binary fringe pattern, which will distort the final 3D measurement for monocular systems relying on triangulation. Based on this finding, we proposed a method to compensate on the phase map, to eliminate the effect of the offset. The experimental results on a white plate indicated that under various defocusing levels and with multiple fringe pitches, the phase compensation can significantly reduce the phase RMSE of the FS binary fringes, especially for fringes with small pitches. Besides a 3D measurement experiment on a white plate and two separate plaster models further verified that phase compensation can effectively eliminate the 3D deformation caused by the fringe offset and improve the 3D measurement accuracy.

Funding

Key Laboratory of Fundamental Science for National Defense on Vision Synthesization and Graphic Image, Sichuan University (2020SCUVS007, 2021SCUVS006); Fundamental Research Funds for the Central Universities (2021SCU12049); China Postdoctoral Science Foundation (2021M692260); Sichuan Province Science and Technology Support Program (2018GZDZX0029, 2019ZDZX0039); Key Research and Development Program of Sichuan Province (2021YFG0195, 2022YFG0053); National Natural Science Foundation of China (61901287, 62101364).

Acknowledgments

We thank the Key Laboratory of Fundamental Science on Synthetic Vision members for their good advice.

Disclosures

The authors declare no conflicts of interest.

Data availability

Data underlying the results presented in this paper are not publicly available at this time but may be obtained from the authors upon reasonable request.

References

1. J. Geng, “Structured-light 3D surface imaging: a tutorial,” Adv. Opt. Photonics 3(2), 128 (2011). [CrossRef]

2. S. Lei and S. Zhang, “Flexible 3-D shape measurement using projector defocusing,” Opt. Lett. 34(20), 3080–3082 (2009). [CrossRef]

3. C. Zuo, Q. Chen, S. Feng, F. Feng, G. Gu, and X. Sui, “Optimized pulse width modulation pattern strategy for three-dimensional profilometry with projector defocusing,” Appl. Opt. 51(19), 4477 (2012). [CrossRef]

4. G. A. Ayubi, J. A. Ayubi, J. M. Di Martino, and J. A. Ferrari, “Pulse-width modulation in defocused three-dimensional fringe projection,” Opt. Lett. 35(21), 3682 (2010). [CrossRef]

5. Y. Wang and S. Zhang, “Optimal pulse width modulation for sinusoidal fringe generation with projector defocusing,” Opt. Lett. 35(24), 4121 (2010). [CrossRef]

6. Y. Wang and S. Zhang, “Three-dimensional shape measurement with binary dithered patterns,” Appl. Opt. 51(27), 6631–6636 (2012). [CrossRef]

7. W. Purgathofer, R. F. Tobler, and M. Geiler, “Forced random dithering: Improved threshold matrices for ordered dithering,” in Proceedings of 1st International Conference on Image Processing (IEEE, 1994), pp. 1032–1035.

8. B. Bayers, “An optimum method for two-level rendition of continuous-tone pictures,” in International Conference on Communications, vol. 1 (IEEE, 1973), pp. 2611–2615.

9. T. D. Kite, B. L. Evans, and A. C. Bovik, “Modeling and quality assessment of halftoning by error diffusion,” IEEE Trans. on Image Process. 9(5), 909–922 (2000). [CrossRef]

10. Z. Pei, Z. Jiangping, J. Hailong, D. Zhijuan, and G. Yanqiong, “Design and experimental study of sinusoidal structured light mask with S shaped binary coding,” Laser & Optoelectronics Prog. 57(15), 151203 (2020). [CrossRef]

11. J. Zhu, P. Zhou, X. Su, and Z. You, “Accurate and fast 3D surface measurement with temporal-spatial binary encoding structured illumination,” Opt. Express 24(25), 28549 (2016). [CrossRef]

12. P. Zhou, J. Zhu, X. Su, Z. You, H. Jing, C. Xiao, and M. Zhong, “Experimental study of temporal-spatial binary pattern projection for 3D shape acquisition,” Appl. Opt. 56(11), 2995 (2017). [CrossRef]

13. J. Dai and S. Zhang, “Phase-optimized dithering technique for high-quality 3D shape measurement,” Opt. Lasers Eng. 51(6), 790–795 (2013). [CrossRef]

14. J. Dai, B. Li, and S. Zhang, “High-quality fringe pattern generation using binary pattern optimization through symmetry and periodicity,” Opt. Lasers Eng. 52, 195–200 (2014). [CrossRef]

15. J. Dai, B. Li, and S. Zhang, “Intensity-optimized dithering technique for three-dimensional shape measurement with projector defocusing,” Opt. Lasers Eng. 53, 79–85 (2014). [CrossRef]

16. N. Cai, Z. B. Chen, X. Q. Cao, and B. Lin, “Optimized dithering technique in frequency domain for high-quality three-dimensional depth data acquisition,” Chin. Phys. B 28(8), 084202 (2019). [CrossRef]

17. P. Zhou, N. Cai, T. Wang, X.-Q. Cao, and B. Lin, “High-quality 3D shape measurement by kernel-optimized high sinusoidal similarity dither patterns,” Appl. Opt. 59(34), 10645 (2020). [CrossRef]

18. R. W. Floyd, “An adaptive algorithm for spatial gray-scale,” in Proc. Soc. Inf. Disp., vol. 17 (1976), pp. 75–77.

19. J. Cheng, X.-y. Su, and T. Xian, “Study and application of binary error diffusion grating in phase measuring profilometry,” Journal-Sichuan University Natural Science Edition 43, 560 (2006).

20. C. Zuo, S. Feng, L. Huang, T. Tao, W. Yin, and Q. Chen, “Phase shifting algorithms for fringe projection profilometry: A review,” Opt. Lasers Eng. 109, 23–59 (2018). [CrossRef]

21. C. Zuo, L. Huang, M. Zhang, Q. Chen, and A. Asundi, “Temporal phase unwrapping algorithms for fringe projection profilometry: A comparative review,” Opt. Lasers Eng. 85, 84–103 (2016). [CrossRef]

22. J. Li, L. G. Hassebrook, and C. Guan, “Optimized two-frequency phase-measuring-profilometry light-sensor temporal-noise sensitivity,” J. Opt. Soc. Am. A 20(1), 106 (2003). [CrossRef]

23. Z. Li, “Accurate calibration method for a structured light system,” Opt. Eng. 47(5), 053604 (2008). [CrossRef]

24. P. S. Huang, “Novel method for structured light system calibration,” Opt. Eng. 45(8), 083601 (2006). [CrossRef]

25. C. Zuo, Q. Chen, G. Gu, S. Feng, and F. Feng, “High-speed three-dimensional profilometry for multiple objects with complex shapes,” Opt. Express 20(17), 19493 (2012). [CrossRef]

26. K. S. Arun, T. S. Huang, and S. D. Blostein, “Least-squares fitting of two 3-D point sets,” IEEE Trans. Pattern Anal. Mach. Intell. PAMI-9(5), 698–700 (1987). [CrossRef]

27. W. S. Zhou and X. Y. Su, “A direct mapping algorithm for phase-measuring profilometry,” J. Mod. Opt. 41(1), 89–94 (1994). [CrossRef]

28. W. Song, “Phase-height mapping and coordinate calibration simultaneously in phase-measuring profilometry,” Opt. Eng. 43(3), 708 (2004). [CrossRef]

Theoretical analysis and experimental investigation of the Floyd-Steinberg-based fringe binary method with offset compensation for accurate 3D measurement

Abstract

1. Introduction

2. Principle

2.1 Floyd–Steinberg dithering technique

2.2 N-steps phase-shifting algorithm

2.3 N-steps phase-shifting fringe pattern encoded by Floyd–Steinberg dithering technique

2.4 Phase error of the binary Floyd–Steinberg fringe pattern

2.5 Offset caused by Floyd–Steinberg dithering technique

2.6 3D distortion caused by Floyd–Steinberg dithering technique

3. Experiments

3.1 Experiments to analyze phase error

3.2 Experiments to analyze 3D error

4. Discussion

5. Conclusion

Funding

Acknowledgments

Disclosures

Data availability

References

Data availability

Cited By

Figures (14)

Equations (19)

Optics Express