1. Introduction

At present, 3D measurement technologies are extensively used in many fields, including the manufacturing industry, architecture, medicine, computer science, and entertainment. As the application and measurement environment separates, high-speed measurement, which is one of the most essential issues related to 3D measurement, is presented because of the demands in real-time data acquisition, resistance to environmental interference, and reducing consumption of measurement.

Over the past few years, high-speed 3D measurement technologies [1–3] have been advancing rapidly. For high-speed 3D measurement, the 3D shapes have to be profiled promptly, that is, raw images should be acquired in a short time [4].

Meanwhile, reducing the projection patterns and increasing the frame speed of a projector are consistently correlated in recent studies. Many researchers have developed several pattern projecting approaches for absolute phase retrieval with less projected patterns such as dual-frequency phase-shifting approach [5], bi-frequency phase-shifting approach [6] and 2 + 2 phase-shifting approach [7]. All these approaches have been implemented in high-speed real-time systems where the fringe order information is carefully designed and embedded to conventional phase-shifting patterns to reduce the number of projected patterns. Heist [8,9] established an aperiodic sinusoidal fringes projection system using array projection and reduced the projection patterns. In the aforementioned study, an array projector was used, which can achieve frame rates of 10 kHz. However, the fringe width and working focus are fixed; thus, this projection system is inflexible for other expansion applications. Nguyen [10] developed a rapid projection method that drives all the RGB channels of a projector, thereby increasing the frame speed from 60 Hz to 180 Hz without setting up a novel optical system. Yang [11] combined a high-speed rotating polygon mirror and line-structured laser to produce stable and unambiguous stroboscopic fringe patterns. This system combines the rapidity of grating projection with the high accuracy of a line-structured laser light source. The 3D data frame speed of this system is 30 Hz. Pankow [12] developed a 3D full-field high-speed digital image correlation measurement system using a single camera. This system was devised to record images at ultrahigh speed using a single camera and a series of mirrors. These mirrors effectively converted a single camera into two virtual cameras that view a specimen surface from different angles and simultaneously capture two images. The period of one single shot of an image is below 120 us. Park [13] developed an ultrahigh-speed digital laser grating projection system using a high-power laser diode. A polygon mirror was used in this projection system to generate fringe patterns from a high-speed laser diode. The optical measurement required to determine the profile of a 3D object could performed out within 2.6 ms.

Among the aforementioned methods, the binary projection methods [14], including binary speckles projection [15] and binary pattern projection [16,17], can achieve the highest projection speed. Different from the physical grating projection [18], the defocusing dithered binary patterns projection proposed by Zhang [19,20] is a novel method that can achieve high-speed and dense point results. Zhang proposed to utilize the optimal pulse width modulation technique to generate high-frequency fringe patterns, and the error-diffusion dithering technique to produce low-frequency fringe patterns. Furthermore, the fringe patterns are produced with blue light, thereby providing high-quality measurements compared with the fringe patterns generated with red or green light. The minimum data acquisition speed for high-quality measurements is approximately 800 Hz for a rabbit heart beating at 180 beats per min.

The performance of defocusing dithered projection is discussed clearly in Zuo’s work [21] where SPWM(sinusoidal pulse width modulation) fringe patterns combined with 4-step phase shifting are used to minimize phase error and generate high-quality sinusoidal fringe patterns with projector defocusing to overcome the nonlinear response of the projector in a 3D shape measurement. Defocusing projection method could achieve a high projection speed, but the depth of field of projector is decreased in defocusing method.

This paper intends to develop a high-speed projection method with focused projector where the dithering is performed both spatially and temporally. The study presents a high-speed four-step triangular pattern phase-shifting 3D measurement using the motion blur method. Binary grayscale triangular patterns are projected and could dither vertically. Therefore, the image captured by the camera is blurred into grayscale-intensity triangular patterns. The proposed method based on binary pattern with motion blur dithering can achieve an improved measuring precision with high speed. The entire projection time is reduced compared with the 8 bit sinusoidal patterns using a DMD projector. The rest of this paper is organized as follows. Section 2 explains the principles of the proposed method. Section 3 discusses the measurement experiments. Section 4 presents the conclusions.

2. Principles

The mathematical model of binary projection with motion blur is initially analyzed in this study. Thereafter, the high-speed triangular pattern binary projection method is proposed. Lastly, the phase unwrapping algorithm of triangular pattern is proposed to reconstruct a 3D surface. The framework of the high-speed measurement system is also proposed in this section.

2.1 High-speed projection of triangular pattern

High-speed projection is achieved using the triangular pattern and motion blur methods. To provide a high-speed projection of grayscale-intensity patterns, a single binary pattern is initially projected using the DMD. The binary pattern comprises an array of basic blocks that will transform into grayscale-intensity patterns when motion is blurred. We let the basic (kernel) block be$f_{block}(x,y)$, where x and y are the image coordinate points on DMD.

Thereafter, the binary pattern $f_{block}(x,y)$ is dithered along a certain direction using the projector in a high frequency. The dithering length of each pattern should be equal to the height of the binary pattern to provide a smooth shape of grayscale-intensity patterns. A grayscale-intensity triangular pattern image is obtained when the exposure time of the camera is expanded to contain the entire dithering procedure in one image.

In general, vertically and horizontally blurred patterns are extensively used. The grayscale intensity $F_{block}(x,y)$ of the horizontally blurred patterns can be calculated as follows:

Similarly, the grayscale intensity $F_{block}(x,y)$ of the vertically blurred patterns can be calculated as follows:

 

Fig. 1 Binary block, blurred patterns, and wave form of (a) saw tooth wave, (b) half-circle wave, (c) half-sinusoidal wave, (d) sinusoidal wave, (e) triangular wave with triangular block, and (f) triangular wave with diamond-like block.

Download Full Size | PDF

To explain why this method could achieve a higher projection speed, firstly, the conventional DMD 8-bit grayscale projection method should be analyzed. In the conventional method, an 8-bit number is used to express the grayscale of one projection point. The micromirror on DMD is flipped in serial to give a time modulation of light. The 8-bit number expresses how the micromirror flips. The projection period is divided into 8 flip parts. The flip time of micromirror of each part is twice as much as previous flip time. In this way, any 8-bit number could be projected in 256 basic flip times. For example, when the 8-bit number is 174, which is 10101110 in binary, as shown in Fig. 2, the micromirror flips to complete the projection.

 

Fig. 2 The process of conventional DMD 8-bit grayscale projection method. The grayscale of projected point is 174 (10101100 in binary).

Download Full Size | PDF

Compared with conventional method, a frame of binary pattern projection only costs one basic flip time. The period of basic flip time is shown in Fig. 2. In the binary dithering projection method, a frame of grayscale pattern is comprised by blurred binary patterns. When the amount of binary patterns is less than 256, it could achieve a higher projecting speed.

There are several specific binary patterns could generate grayscale patterns by dithering vertically. As shown in Figs. 1(b), 1(c) and 1(d), because of the quantization error of DMD micromirrors, the half-circle binary pattern, half-sinusoidal binary pattern and sinusoidal binary pattern cannot be described ideally in small basic blocks. This means that a large amount of binary patterns should be used to project these 3 kinds of patterns. To achieve the same projection accuracy to conventional DMD 8-bit projection method, the basic block size and dithering length of binary patterns should be at 256 pixels. The projection speed will not be faster.

However, as shown in Figs. 1(a), 1(e) and 1(f), linear binary patterns could be described ideally in small basic blocks. For example, as shown in Fig. 3, a frame of triangular pattern with the period of 16 pixels only needs 16 dithering steps of diamond-like binary blocks. And the projection speed will be 64 times faster comparing with conventional method.

 

Fig. 3 The 16 dithering steps of diamond-like binary blocks and the grayscale pattern comprised by these 16 binary blocks. The width of fringe pattern is 16 pixels.

Download Full Size | PDF

In order to get its mathematical expression, as shown in Fig. 4, we can select one basic block of the binary triangular pattern. The width and height of the block are 2a and 2b, respectively.

 

Fig. 4 Basic block based on the binary triangular pattern. The width and height of each block are 2a and 2b, respectively.

Download Full Size | PDF

The greyscale of each point in the block can be expressed as follows:

When the triangular pattern is dithering vertically, the greyscale in each column would be the same. The greyscale in each row can be derived as follows:

To provide a normalization equation of $F_{block}(x,y)$, we can set a = 2 and b = 1/2, and vertically shift the function down in 1 to make $x\in \left[0,4\right)$ and $F_{block}(x,y)\in [-1,1]$. One period of $F_{block}(x,y)$ can be derived as follows:

A triangular pattern image of a single phase is acquired after the modulation of the optical projector and camera systems. Thereafter, the basic binary pattern is shifted horizontally into the succeeding phase. The traditional four-step phase shifting method based on sinusoidal patterns demands that the phase of the fringe moves $\pi /2$ at each step, which is a quarter of a fringe period. Similarly, the phase movement should be a quarter of the fringe width when triangular patterns are used.

The fringe pattern on DMD could be expressed as follows:

Considering the time-dividing projection method of DMD, one binary pattern is projected in one basic time cell, and the projection time of the 8 bit grayscale pattern contains 256 time cells. The entire projection time employed in this study depends on the width of the triangular patterns. For example, if we select a pattern serial with a width of 26 pixels, the projection time is 26 time cells, thereby providing a higher speed compared with the projection time of 256 time cells of traditional sinusoidal patterns.

2.2 Phase unwrapping algorithm of triangular patterns

Triangular patterns are extensively used in fringe pattern projection methods [22]. Cao [23] used triangular patterns of two different spatial frequencies to achieve high-speed measurement. A serial of two different spatial frequencies triangular patterns with phase shifting are projected and the intensity ratio of the adjacent pixels is calculated for general stereo matching. A shift map of considerably high frequency patterns can be calculated using a multiple-step triangular-pattern spatial shift estimation algorithm to assist the precise stereo matching. Yang [24] presented an intensity ratio approach for 3D object profilometry measurement based on the projection of triangular patterns. The intensity ratio is point-by-point-based, thereby avoiding the influence of the surrounding points. Jia [25] used a two-step triangular pattern phase-shifting method, where the triangular intensity ratio distribution is computed from two captured phase-shifted triangular pattern images. These methods that use triangular patterns focus on intensity ratio computing for 3D reconstruction, which is sensitive to surface reflectivity and ambient light.

In the proposed method, which corresponds to a traditional four-step sinusoidal patterns phase unwrapping algorithm, the phase unwrapping algorithm for triangular pattern could be provided. The traditional phase unwrapping method can be introduced briefly: the grayscale$g_i(x,y)$ in each point on the image obtained by the camera can be expressed as follows:

By focusing on triangular patterns, we can convert the core of the sinusoidal method, that is, function $\cos (x)$ and $\sin (x)$, into the triangular core function. For convenience of explanation, we named these functions $T_{\cos }(x)$ and $T_{\sin }(x)$ with a period of 4. The relation-ship between $T_{\cos }(x)$ and $\cos (x)$ is shown in Fig. 5. $T_{\cos }(x)$ and $T_{\sin }(x)$ can be written as follows:

 

Fig. 5 Relationship between T_cos(x) and cos(x): T_cos(x) is in period 4, where cos(x) is in period 2$\pi$. Only one period of these two functions is shown in the graph.

Download Full Size | PDF

The grayscale of each point in the image can be expressed as follows:

Corresponding to $\tan (x)$ in the sinusoidal patterns, $T_{tan}(x)$ can be written as follows:

The inverse function of $T_{tan}(x)$ can be written as $T_{\arctan }(x)$.

Figure 6 shows the simulation result of the phase unwrapping method for the triangular pattern. The unwrapping result is linear and periodic and could be used for 3D reconstruction. In the proposed method, the phase of the patterns are calculated, which is not influenced by surface reflectivity and ambient light.

 

Fig. 6 Simulation result of the proposed phase unwrapping method: (a) four-step phase shifting pattern images and their phase unwrapping result, (b) waveform of a pattern image (80 points in a row), and (c) waveform of the phase unwrapping result (80 points in a row).

Download Full Size | PDF

3. Experiments

3.1 Projection of triangular patterns

On the basis of the proposed method, a triangular pattern frame is obtained by the projection of dithered binary patterns with the expanded exposure time of cameras. Figure 7 shows that when the exposure time expands, the obtained fringe image becomes a triangular pattern form of binary blocks. The triangular pattern image remains intact when the exposure time is equal to the dithering period of the binary patterns. The experiment is conducted at 5000 Hz fps of the binary pattern projection. Figure 8 shows the implementation of the measurement system using one DMD projector and two cameras.

 

Fig. 7 Captured images in different exposure times: (a) single static binary pattern with 4000 us exposure, (b)–(g) dithering patterns with exposures at 200, 800, 1600, 2400, 3200, and 4000 us.

Download Full Size | PDF

 

Fig. 8 Measurement system using one DMD projector and two cameras.

Download Full Size | PDF

3.2 Experiments on triangular patterns phase unwrapping

A series of experiments is performed to estimate the results of the proposed triangular pattern phase unwrapping method. Figure 9 shows the results of this method. A piece of plaster jaw is used for this experiment.

 

Fig. 9 Results of the proposed triangular pattern phase unwrapping method for the experiment involving a plaster jaw: (a) Projected binary patterns, (b) one fringe pattern image, and (c) phase calculating result for the fringe width of 20; (d) Projected binary patterns, (e) one fringe pattern image, and (f) phase calculating result for the fringe width of 24; (g) Projected binary patterns, (h) one fringe pattern image, and (i) phase calculating result for the fringe width of 28; (j) Phase unwrapping result using (c), (f), and (i).

Download Full Size | PDF

3.3 Experimental results of the 3D reconstruction

The results of the triangular pattern phase unwrapping method can be used for 3D reconstruction. Figures 10-12 show the reconstruction results of the plaster jaw and rubber bottle. The experiment was conducted with a projection frequency of 5000 Hz. Two cameras are used to capture images in the exposure time of 5.6 ms. The period of acquisition of the 12 images is 67.2 ms.

 

Fig. 10 Experiment involving the plaster jaw using the unwrapping images of the triangular pattern phase unwrapping method and two calibrated cameras: (a) plaster jaw; (b) one fringe pattern image, (c) local parts of the fringe pattern image, and (d) phase unwrapping results of the left camera; (e) one fringe pattern image, (f) local parts of the fringe pattern image, and (g) phase unwrapping results of the right camera.

Download Full Size | PDF

 

Fig. 11 Experiment involving the rubber bottle using the unwrapping images of the triangular pattern phase unwrapping method and two calibrated cameras: (a) rubber bottle; (b) one fringe pattern image, (c) local parts of the fringe pattern image, and (d) phase unwrapping results of the left camera; (e) one fringe pattern image, (f) local parts of the fringe pattern image, and (g) phase unwrapping results of the right camera.

Download Full Size | PDF

 

Fig. 12 3D reconstruction result using the unwrapping images of the triangular pattern phase unwrapping method: (a) polygonized point cloud result of the plaster jaw; (b) polygonized point cloud result of the rubber bottle; (c) color representation of the plaster jaw image that shows depth (Z) values of the measured points; and (d) color representation of the rubber bottle image that shows depth (Z) values of the measured points.

Download Full Size | PDF

To validate the feasibility of this approach in actual high-speed measurement, an experiment of moving hand measurement is held. To achieve the measurement of dynamic object, the projection frequency of binary patterns is set at 4800 Hz and the triangular pattern acquisition frequency is 300 Hz. As shown in Fig. 13(a), researcher’s hand is moving while the high-speed measurement experiment is ongoing. In the whole experiment, 31 frames of 3D data are acquired in 1.24 second. Figure 13(b) shows 4 frames of 3D data.

 

Fig. 13 High-speed measurement of dynamic object: (a) on shot of triangular fringe pattern image acquired by left camera. (b) 4 frames selected from 31continuous frames of 3D data. See Visualization 1.

Download Full Size | PDF

The measurement accuracy of the system can be tested by measuring the standard plane. The reconstructed 3D point-cloud of plane board was fitted as a standard plane. The distances between the 3D points and the standard plane were calculated as the plane measurement errors used for evaluate the measurement accuracy. Figure 14 shows the measured results on a standard plane board using the triangular patterns. The absolute mean error is 0.054 mm, and standard deviation of errors is 0.072 mm.

 

Fig. 14 Measurement deviation estimation: measuring a standard plane board using the proposed method; The absolute mean error is 0.054 mm, and standard deviation of errors is 0.072 mm.

Download Full Size | PDF

3.4 Experimental comparison

There are several methods could achieve high speed 3D measurement. SPWM pattern projection with projector defocusing method is a representative approach [21]. Comparisons of measurement speed and accuracy of SPWM method and our method are carried out to analyze the features of each approach.

The experiments of these two methods are carried out with a same system. In the SPWM method, the fringe width is 48 pixels and the sampling comb frequency is six times the fringe spatial frequency. Theoretically, the projection speed of SPWM pattern could achieve 4800 Hz in our projector. Limited by the frame speed of cameras, the fringe image capturing speed of these two experiments is fixed at 300 Hz. The 3D measurement speed of SPWM method is 75 Hz using four-step phase-shifting algorithms. The 3D measurement speed of our proposed method is 25 Hz.

In the experiments, a standard plane is measured in five different distances. The measurement results of proposed method are shown in Figs. 15(a), 15(c) and 15(e). Figure 15(a) shows the five 3D reconstruction results of plane. The five measurement distances are 706 mm, 758 mm, 819 mm, 876 mm and 917 mm respectively. Figure 15(c) shows the result of 3D reconstruction in the measurement distance of 819 mm. Figure 15(e) shows the result of 3D reconstruction in the measurement distance of 706 mm.

 

Fig. 15 Measurement results of our proposed method and SPWM method: (a) the five results of 3D reconstruction of our proposed method whose measurement distances are 706 mm, 758 mm, 819 mm, 876 mm and 917 mm respectively. (b) the five results of 3D reconstruction of SPWM method whose measurement distances are 702 mm, 751 mm, 821 mm, 895 mm and 930 mm respectively. (c) 3D reconstruction of plane using our proposed method in measurement distance of 819 mm. (d) 3D reconstruction of plane using SPWM method in measurement distance of 821 mm. (e) 3D reconstruction of plane using our proposed method in measurement distance of 706 mm. (f) 3D reconstruction of plane using SPWM method in measurement distance of 702 mm.

Download Full Size | PDF

The measurement results of SPWM method are shown in Figs. 15(b), 15(d) and 15(f). Figure 15(b) shows the five results of 3D reconstruction. The five measurement distances are 702 mm, 751 mm, 821 mm, 895 mm and 930 mm respectively. Figure 15(d) shows the result of 3D reconstruction in the measurement distance of 821 mm. Figure 15(f) shows the result of 3D reconstruction in the measurement distance of 702 mm.

The reconstructed 3D point-cloud of plane board was fitted as a standard plane. The distances between the 3D points and the standard plane were calculated as the plane measurement errors used for evaluate the measurement accuracy. In the experiments of our proposed method, the standard deviations of the five planes are 0.066mm, 0.070mm, 0.071mm, 0.085mm and 0.103mm at the measurement distances of 706mm, 758mm, 819mm, 876mm and 917mm respectively. In the experiments of the SPWM pattern projection with projector defocusing method, the standard deviations of the five planes are 0.094mm, 0.099mm, 0.106mm, 0.161mm and 0.221mm at the measurement distances of 702mm, 751mm, 821mm, 895mm and 930mm respectively. These two sets of standard deviations are shown in Fig. 16.

 

Fig. 16 The standard deviations of plane measurements using our proposed method and SPWM method in different measurement distances.

Download Full Size | PDF

The above experimental comparisons indicate that the accuracy of our proposed method is more stable comparing with that of the SPWM method when the measurement distances changes.

4. Conclusion

This study presents a novel four-step triangular pattern phase-shifting 3D measurement using the motion blur method. The entire projection time is reduced compared with the 8 bit sinusoidal patterns using a DMD digital projector. Moreover, a four-step triangular phase-shifting unwrapping algorithm is presented to achieve 3D reconstruction. The experiments indicate that the proposed method can achieve high-speed 3D measurement and reconstruction. Measurement error varied cyclically with phase and may be partially caused by unwrapping error at the phase vertices of the triangular patterns.