Compensation of fringe distortion for phase-shifting three-dimensional shape measurement by inverse map estimation

Kohei Yatabe; Kenji Ishikawa; Yasuhiro Oikawa

doi:10.1364/AO.55.006017

1. INTRODUCTION

Measuring three-dimensional (3D) shapes is a quite fundamental and important task for many industrial and scientific situations. The fringe projection technique is a 3D shape measurement method that has been developed by many researchers [1–4]. It acquires 3D information from two-dimensional images by analyzing sinusoidal patterns, or fringes, projected on an object to be measured. The phase-shifting method is a popular branch of the fringe projection methods and utilizes several fringe patterns with shifted phase. It can accurately reconstruct the objective phase, which contains 3D information, and has been successfully applied to many practical measurement problems.

Nowadays, it has become more common to use commercially available projectors and cameras for fringe projection methods, because their costs have been greatly reduced. However, to improve the visual quality for humans, most commercial devices have a nonlinear intensity response, which distorts the projected fringes. Since phase-shifting algorithms are designed based on purely sinusoidal fringes, distorted fringes provide inaccurate results. Unfortunately, measurement error caused by this nonlinear effect cannot be reduced by a denoising method designed for random noise [5], because the error appears as a periodic pattern. Therefore, any nonlinearity between input signal and measured intensity must be compensated before or within a reconstruction step of the objective phase. This problem has been addressed by many researchers, and many methods for reducing the error have been proposed [6–11].

In the case of phase-shifting techniques, measurement error caused by the nonlinear effect can be decreased by increasing the shifting steps [12]. Therefore, accurate results are obtained by taking the time to acquire many images of fringes if the measuring object does not move. On the other hand, there are some situations in which acquiring many images is not possible: for example, in real-time phase-shifting methods [13,14]. Then the error must be compensated by software acting on either the input signal or the acquired images. In any case, it is better to have a signal-processing method that reduces error and improves accuracy, and thus it is worthwhile to investigate it further.

One popular approach for the compensation is the gamma-model-based methods [12,15–23]. They assume that nonlinearity can be approximated well by the gamma model: for a parameter $Γ > 0$ , nonlinearity is written as the $Γ$ th power of a function of the input signal. The main advantage of the gamma model is its simplicity, which allows detailed analysis of phase error through series expansion and its coefficients. Although the effectiveness of the gamma model has been proved, it cannot perfectly compensate nonlinearity, which results from not only the gamma effect but also some other causes. This modeling error has been reduced by generalizing the gamma model through including the effect of defocusing and ambient light [17–20]. Nevertheless, some situations where the generalized gamma models cannot perfectly represent the nonlinearity exist in practice.

A lookup-table-based method is another popular approach [13,24–27]. It consists of constructing a table that represents nonlinear error from reference data measured in advance, and the effect of the nonlinearity is directly compensated by the table. Since unknown nonlinear effects of a system are revealed by the table obtained from real data, no specific model of nonlinearity has to be assumed. Therefore, it can be applied to any nonlinear measurement system, including the gamma model. Some research utilized an artificial neural network, which also does not require any specific model, for obtaining the nonlinear relation of the input pattern and observed illumination [28]. Such neural-network-based methods may have the possibility of improving their performance using some recently developed learning techniques [29]. However, many of these methods require input data, which may not be available for some situations, as well as observed images. Moreover, a table or network must be constructed each time the setting of a measurement system is changed, and such construction may consume time.

In this paper, as an alternative method, we propose a general framework for approximating and compensating the nonlinear distortion of a fringe. The proposed method tries to estimate an inverse map of the nonlinearity from only observed images. Thus, it can be applied to any measurement system easily, because no information about the system is required. In addition, any model, such as polynomials, can be used for approximating nonlinearity. Our contributions include (1) proposing a simple objective function measuring the degree of nonlinearity, (2) modeling nonlinearity in a flexible form, and (3) obtaining an analytic solution of the model parameters when a linear combination model is applied. The aim of this paper, however, is not to compete with the current state-of-the-art methods, which may require many reference images, but rather to show how our framework can treat multiple types of nonlinearity in the same manner. The proposed method should be able to be integrated with the state-of-the-art methods without much effort because of its flexibility.

This paper is organized as follows. In Section 2, a mathematical model of distorted fringes is reviewed briefly. Then, our method of modeling and compensating the nonlinear effect is introduced in Section 3. The proposed method is formulated as an optimization problem minimizing the energy of harmonic components caused by distortion. Therefore, the objective function is introduced first, and then the modeling of a nonlinear map and solution to the optimization problem are described. In Section 4, the proposed method is evaluated by simulations and real data, and finally the paper is concluded in Section 5.

2. FRINGE DISTORTION

Let $\tilde{I} (x)$ be an obtained image of a fringe pattern, and $x \in R^{2}$ denotes the position vector. In addition, the nonlinearity of the observation process is represented by a pixel-wise nonlinear map $N$ as

\tilde{I} (x) = N (I (x)) = N (I_{0} + \cos [φ (x) + δ]),

where

φ (x)

is the objective phase,

δ

is a phase-shifting step, and

I_{0}

is the relative background illumination, whose intensity is chosen so that the coefficient of the cosine term is omitted. The index

n

indicating the phase-shifting step of

δ_{[n]}

and

{\tilde{I}}_{[n]}

is also omitted for simplicity, because it is not important in describing the proposed method.

Distortion of $\tilde{I}$ is often analyzed by its Fourier series,

\tilde{I} (x) = c_{0} + \sum_{n} c_{n} \cos [n (φ (x) + δ)],

where the terms corresponding to

n \geq 2

are the harmonic components generated by the nonlinearity. The model-based compensation methods mentioned in the previous section analyze relations among the Fourier coefficients

{c_{n}}

in order to obtain the information necessary for a compensation algorithm. It is obvious that the nondistorted fringe is perfectly recovered when all the coefficients of second or higher harmonics become zero. Therefore, roughly speaking, those algorithms are designed so that the Fourier coefficients related to distortion effect are minimized.

Although the concept of minimizing higher harmonic components is straightforward, separating them from the observed images is not an easy task because of the modulation caused by $φ$ . One strategy for avoiding this difficulty is to utilize a simple model, including the gamma model, which has only a few parameters to determine. On the other hand, we will not simplify the model of nonlinearity so much but simplify the objective function to be minimized. That is, we propose to approximate the coefficients of the harmonic components by the Fourier transform rather than the Fourier series in Eq. (2).

3. BLIND COMPENSATION OF A DISTORTED FRINGE

In this section, a general framework for compensating unknown nonlinearity is proposed. It is formulated as an optimization problem minimizing a criterion of fringe distortion. Here, the proposed criterion is introduced first, and then the optimization method is described.

A. Fourier-Transform-based Criterion of Distortion

The two-dimensional Fourier transform is defined by

(F I) (ω) = \int_{R^{2}} I (x) e^{- 2 π i ⟨ ω, x ⟩} d x,

where

ω \in R^{2}

is the frequency vector and

⟨ \cdot, \cdot ⟩

is the standard inner product.

Since the harmonic components caused by distortion are lying in the high-frequency region of the spectrum, the total power within this region,

\int_{Ω_{High}} {| (F \tilde{I}) (ω) |}^{2} d ω,

should provide an approximation of the power of the higher harmonic components, where

Ω_{High}

is the high-frequency region corresponding to the harmonics caused by the nonlinearity of the observation process

N

and

| \cdot |

denotes absolute value. However, this quantity depends on the magnitude of the whole image; i.e., multiplying a constant by an image can increase or decrease it.

A simple modification to overcome this dependency on the magnitude in Eq. (4) is to calculate the ratio of the power of the low- and high-frequency components,

R (\tilde{I}) = \int_{Ω_{High}} {| (F \tilde{I}) (ω) |}^{2} d ω / \int_{Ω_{Low}} {| (F \tilde{I}) (ω) |}^{2} d ω,

where

Ω_{Low}

is the low-frequency region corresponding to the fundamental frequency of the fringe (the DC component should not be included in

Ω_{Low}

, because it causes magnitude dependency). This criterion provides an approximation of the higher harmonic components to its fundamental frequency component ratio. Therefore, a purely sinusoidal fringe, which consists of only the fundamental frequency component, has less value than its distorted counterpart, which includes the higher harmonic components. Hence, some nonlinear mapping

Φ

that minimizes

R (Φ (\tilde{I}))

should compensate the distortion of

\tilde{I}

. This can be written in the form of an optimization problem: finding

Φ^{⋆} \in \arg \min_{Φ} [R (Φ (\tilde{I}))],

where the superscript

⋆

indicates the optimality. Here,

Φ

is a pixel-wise function (because

N

is assumed to be pixel-wise in the previous section) that maps the intensity values of an image to real numbers. In the ideal case,

Φ^{⋆}

should be equal to

N^{- 1}

, because it minimizes the higher harmonic components related to the distortion. Note that this nonlinear mapping does not affect the objective phase

φ

, unlike low-pass filtering, because

Φ^{⋆}

is applied directly to an image rather than applying it to some operation in the frequency domain.

B. Gamma-Model-based Compensation

Although the objective function, Eq. (5), can be calculated numerically without effort, minimizing it is not easy because the variable of Eq. (6) is $Φ$ , which is a nonlinear continuous function lying on an infinite-dimensional space. Therefore, it must be expressed by a finite set of parameters. As an illustrative example, the well-known gamma model of the simplest form is presented here, because it has been used for compensation in many articles.

Compared to the multivariate counterpart, a single-variable optimization problem is extremely easy. Therefore, if a simple gamma model,

\tilde{I} (x) = N (I (x)) = {(I (x))}^{Γ}, (Γ > 0),

is combined with Eq. (6), the optimization problem becomes tractable, because the variable to be optimized is a single parameter: finding

γ^{⋆} \in \arg \min_{γ} [R ({\tilde{I}}^{γ})] .

We empirically found that in most of situations, the objective function

R (\cdot)

becomes convex for this gamma model. Thus, a simple line search is enough to solve Eq. (8). Since

R (\cdot)

can be calculated by a single fast Fourier transform, solving Eq. (8) by a line search method, which requires the calculation of

R (\cdot)

only several times, is reasonably fast.

C. Compensation by Inverse Map Estimation

In the previous subsection, it is shown by example that a simple gamma model can be incorporated with the optimization problem. Any other single parameter model can also be used for Eq. (6) in the same manner. However, utilizing a model with multiple parameters results in a quite difficult problem, because it becomes a nonconvex optimization problem, which has a lot of local minima. Here, we show that a linear combination model can overcome this difficulty and provide a general framework for solving Eq. (6) globally.

A linear combination of a known set of functions ${ψ_{n}}$ ,

Φ_{α} (\tilde{I}) = \sum_{n = 1}^{N} α_{n} ψ_{n} (\tilde{I}),

can approximate a continuous function

Φ

arbitrarily well if the functions

{ψ_{n}}

are chosen properly.

α \in R^{N}

is the set of parameters written as a vector, and the subscript of

Φ_{α}

indicates dependency on the parameters. Note that the above model is linear with respect to

α

. This approximation allows us to express Eq. (6) as a finite-dimensional optimization problem: finding

α^{⋆} \in \arg \min_{α} [R (Φ_{α} (\tilde{I}))] .

In the above model, $ψ_{n} (\tilde{I})$ can be computed only from the observed image $\tilde{I}$ . Therefore, by constructing a matrix $Ψ$ as

Ψ = [vec (ψ_{1} (\tilde{I})), vec (ψ_{2} (\tilde{I})), \dots, vec (ψ_{N} (\tilde{I}))],

Eq. (9) can be expressed by a matrix–vector multiplication

Ψ α = Φ_{α} (\tilde{I}) = \sum_{n = 1}^{N} α_{n} ψ_{n} (\tilde{I}),

where

vec (\cdot)

denotes a vectorization operator that reorganizes a matrix into a column vector, and

[v_{1}, v_{2}, \dots, v_{N}]

denotes the horizontal concatenation of column vectors, yielding a matrix. Thus, Eq. (10) can be written as a minimization of

R (Ψ α)

with respect to

α

: finding

α^{⋆} \in \arg \min_{α} [R (Ψ α)] .

This simplification of $Φ$ allows us to further simplify the original problem as follows:

α^{⋆} \in \arg \min_{α} [\frac{{‖ S_{H} F Ψ α ‖}_{2}^{2}}{{‖ S_{L} F Ψ α ‖}_{2}^{2}}],

where

{‖ \cdot ‖}_{2}

is the standard

ℓ_{2}

norm,

F

is the discrete Fourier transform matrix,

S_{H}

is a selection matrix corresponding to the high-frequency region

Ω_{High}

, which contains 1 for the high-frequency part and 0 for the other part, and

S_{L}

is that of the low-frequency region

Ω_{Low}

. By calculating all the matrix–matrix multiplications as

A = S_{H} F Ψ, B = S_{L} F Ψ,

the minimization problem finally becomes a problem of finding

α^{⋆} \in \arg \min_{α} [\frac{{‖ A α ‖}_{2}^{2}}{{‖ B α ‖}_{2}^{2}}],

which can be written as

α^{⋆} \in \arg \min_{α} [\frac{α^{T} A^{T} A α}{α^{T} B^{T} B α}],

where

A^{T}

denotes the conjugate transpose of

A

. This is in the form of the well-known Rayleigh quotient, whose solution is obtained by solving the following generalized eigenvalue problem:

A^{T} A α = λ B^{T} B α,

where

α^{⋆}

minimizing Eq. (17) is an eigenvector corresponding to the smallest generalized eigenvalue

λ

. There are many eigenvalue solvers that can numerically solve Eq. (18) without much effort. After obtaining a set of optimal parameters

α^{⋆}

, Eq. (9) provides an estimated nonlinear map

Φ_{α^{⋆}} (\cdot)

compensating the distortion effect.

Although the main portion of the proposed method has been described, one slight modification of the above formulation is needed to obtain a meaningful result. As the matrices $A$ and $B$ are constructed through the Fourier transform, their elements are complex numbers in general. Then, directly solving Eq. (18) yields a complex solution that cannot be applied to calibrate distortion. Thus, the above formulation must be modified so that all solutions contain only real numbers.

For a complex scaler $z$ , ${‖ z ‖}_{2}^{2}$ is simply

{‖ z ‖}_{2}^{2} = {| z |}^{2} = Re {(z)}^{2} + Im {(z)}^{2},

where

Re (z)

and

Im (z)

, respectively, denote the real and imaginary parts of

z

. Similarly, by assuming that

α

contains only real numbers,

{‖ A α ‖}_{2}^{2} = {‖ \tilde{A} α ‖}_{2}^{2}, {‖ B α ‖}_{2}^{2} = {‖ \tilde{B} α ‖}_{2}^{2},

where

\tilde{A}

and

\tilde{B}

are real matrices obtained by concatenating the real and imaginary parts of

A

and

B

vertically as follows:

\tilde{A} = {[Re {(A)}^{T}, Im {(A)}^{T}]}^{T}, \tilde{B} = {[Re {(B)}^{T}, Im {(B)}^{T}]}^{T} .

Hence, by replacing

A

and

B

in Eq. (18) with

\tilde{A}

and

\tilde{B}

, we obtain the real counterpart of the generalized eigenvalue problem,

{\tilde{A}}^{T} \tilde{A} α = λ {\tilde{B}}^{T} \tilde{B} α .

D. Summary of the Proposed Algorithm Solving Eq. (10)

The proposed algorithm for compensating the nonlinear effect by solving Eq. (10), described in the previous subsection, can be summarized as the following procedure:

(1) Input a fringe image $\tilde{I}$ . Some preprocessing such as noise reduction might be applied for better estimation.
(2) Transform the image by a set of predefined approximating functions ${ψ_{n} (\tilde{I})}$ .
(3) Calculate the two-dimensional Fourier transform of each piece of transformed data as ${F ψ_{n} (\tilde{I})}$ .
(4) Vectorize and concatenate them horizontally to construct a matrix $M = [vec (F ψ_{1} (\tilde{I})), \dots, vec (F ψ_{N} (\tilde{I}))]$ .
(5) Split $M$ into two matrices $A$ and $B$ by selecting its rows corresponding to the high- and low-frequency regions.
(6) Separate the real and imaginary parts of $A$ and $B$ , and concatenate them vertically as $\tilde{A} = {[Re {(A)}^{T}, Im {(A)}^{T}]}^{T}$ and $\tilde{B} = {[Re {(B)}^{T}, Im {(B)}^{T}]}^{T}$ .
(7) Solve the generalized eigenvalue problem for the matrix pencil $({\tilde{A}}^{T} \tilde{A}, {\tilde{B}}^{T} \tilde{B})$ .
(8) Set the eigenvector corresponding to the smallest generalized eigenvalue as the optimal parameter $α^{⋆}$ .
(9) Compensate the distortion effect by calculating the linear combination in Eq. (9) with the obtained solution $α^{⋆}$ .

In the above construction of matrix $M$ , multiple images (or small patches of images whose size may differ from each other) can be taken into account simultaneously by stacking corresponding matrices vertically: $M = {[M_{1}^{T}, \dots, M_{ℓ}^{T}]}^{T}$ . Moreover, window functions can be utilized for calculating the Fourier transform in order to separate the fundamental and harmonic components better.

Note that the matrices ${\tilde{A}}^{T} \tilde{A}$ and ${\tilde{B}}^{T} \tilde{B}$ are both square and their size is same as the number of approximating functions $N$ in Eq. (9). Therefore, the computational cost of solving the above eigenvalue problem does not depend on the image size. This should be an advantage when working on a huge fringe image. Although image size has an impact on the construction of the matrices, the fast Fourier transform allows an efficient implementation. The main computational cost of the proposed algorithm may be the calculation of the matrix–matrix multiplications, ${\tilde{A}}^{T} \tilde{A}$ and ${\tilde{B}}^{T} \tilde{B}$ , which can be executed in parallel by multicore processors.

E. Choice of Approximating Functions ${ψ_{n}}$

In the above framework, there are many options for the choice of approximating functions ${ψ_{n}}$ in the linear combination model. If one knows some information about the inverse map $N^{- 1}$ , that knowledge can be used in choosing suitable functions. However, in many cases, there is no such prior information. Therefore, the set of functions has to be chosen so that any unknown map can be approximated reasonably well.

One popular choice for such approximation is the sequence of polynomials ${x, x^{2}, \dots, x^{N}}$ , or monomial, because the set of polynomials can approximate all continuous functions arbitrarily well. Its close relation to the Taylor series should also be a reason for its popularity. However, it is well known in the approximation theory that a monomial is exponentially ill conditioned, i.e., numerically unstable.

In this paper, for numerical stability, the normalized Legendre polynomial is chosen for ${ψ_{n}}$ ,

ψ_{n} (x) = \sqrt{\frac{2 n + 1}{2}} \frac{1}{2^{n} n!} \frac{d^{n}}{d x^{n}} {(x^{2} - 1)}^{n},

which is orthonormal and ensures numerically stable approximation. The Chebyshev polynomial, which has several desirable properties for approximation, may also be a choice. We should note that any combination of functions is acceptable for

{ψ_{n}}

. That is, one can mix the above polynomials and/or some other functions to construct a set of functions with desirable properties depending on the application. In addition, from a defined set of functions, one may construct a corresponding orthonormal function through orthogonalization as in [30].

4. EXPERIMENTS

In order to confirm the flexibility and effectiveness of the proposed framework, it was applied to four simulated data and three measured data. The simple gamma model in Eq. (7) and the linear combination model in Eq. (9) with the Legendre polynomials [Eq. (23)] were compared in each experiment by solving Eq. (8) and Eq. (10). For calculating the Fourier transform in this section, a tapered Hann window (smooth trapezoidal window) was used (at the edge of the data, about a half-period of fringes was multiplied by the half-part of the Hann window).

A. Simulation

As illustrative examples, four types of nonlinearity were simulated. For simplicity, one-dimensional fringes for the three-step phase-shifting method were considered.

Nonlinearly transformed fringes were obtained from

{\tilde{I}}_{[n]} = Ξ [N (0.5 + 0.5 \cos [2 π x + δ_{[n]}])],

where

x \in [0, 4]

,

δ_{[n]} \in {0, 2 π / 3, 4 π / 3}

,

Ξ [\cdot]

is a scaling operator that set the value of fringes within [0, 1],

Ξ [I] = (I - \min {I}) / \max {(I - \min {I})},

\min {I}

and

\max {I}

are, respectively, the minimum and maximum values of

I

, and

N

is one of the following nonlinear maps:

Case (a): simple gamma,

N (I) = I^{2},

Case (b): simple gamma with bias,

N (I) = {(I + 1)}^{4},

Case (c): arctangent,

N (I) = \arctan [20 (I - 0.5)],

Case (d): combination of gamma with bias and arctangent,

N (I) = \arctan [2 {{(I + 0.5)}^{5} - 0.5}] .

The parameters in the above maps were selected so that each reconstructed phase,

φ = Arg [(\sum_{n} {\tilde{I}}_{[n]} \cos [δ_{[n]}]) - i (\sum_{n} {\tilde{I}}_{[n]} \sin [δ_{[n]}])],

has an error whose magnitude is similar to the others (

Arg [z]

denotes the principal value of the complex argument of

z

, and

i = \sqrt{- 1}

).

Without any additional information, the distorted fringes ${\tilde{I}}_{[n]}$ were compensated by the proposed method. For comparison, the simple gamma model [Eq. (7)] and polynomial model [Eq. (9) with Eq. (23)] were applied to all four cases. For the polynomial model, fringes ${\tilde{I}}_{[n]}$ were scaled as $2 ({\tilde{I}}_{[n]} - 0.5)$ so that their values are in $[- 1, 1]$ , because the Legendre polynomials are defined in $[- 1, 1]$ . The maximum degree of the polynomial was chosen as 15. The boundary of low-frequency region $Ω_{Low}$ and high-frequency region $Ω_{High}$ was set to 1.5 times the fundamental frequency (as described in Section 3.A, the DC component is not included in $Ω_{Low}$ ).

Figure 1 shows the compensation results, where only the first quarter, $[0, 2 π]$ , is depicted. Figure 1(a) shows that the gamma model accurately compensated the gamma nonlinearity. This result confirms that the proposed criterion [Eq. (5)] can properly measure the degree of distortion and can be used for compensating fringe distortion. On the other hand, as in Figs. 1(b)–1(d), the gamma model cannot compensate the error caused by other types of nonlinearity because of model mismatch. The risk of such model mismatching is one drawback of a model-based compensation method.

Fig. 1. Phase reconstruction error of the simulated three-step phase-shifting method caused by the four nonlinear distortion effects. (a)–(d) Correspond to the cases in Eqs. (26)–(29). The blue dashed lines represent error without compensation. The red dotted lines are error after compensation by the gamma model, while the green solid lines are error after compensation by the polynomial model.

Download Full Size | PDF

In contrast to the gamma model, the polynomial model was able to compensate all four types of nonlinearity to some extent. Although the polynomial itself is also a model, it can approximate any continuous function arbitrarily well as the degree of the polynomial increases. Therefore, by using a polynomial of sufficiently high degree, the proposed method can compensate distortion caused by any type of (continuous) nonlinearity without knowledge of it. Table 1 shows the root-mean-square (RMS) error of each result. From the table, it is obvious that the gamma model can compensate only gamma distortion, while a polynomial model can compensate any nonlinear distortion.

Table 1. RMS and Percentage of Error for Each Model

View Table | View all tables in this article

B. Measured Flat Surface

To confirm performance in real situations, the proposed method was applied to real data. Measured data were obtained by an EPSON EB-1735W in projecting fringe patterns and a SIGMA dp2 Quattro in capturing monochromatic images.

In order to compare with the simulations in Fig. 1, a flat surface was measured by projecting two different three-step fringe patterns:

Case (a): sinusoidal patterns without defocus,

Case (b): binary patterns with defocus using the projector.

Defocusing acts as a low-pass filter on the binary fringe, which results in a sinusoidal-like waveform [25]. Parameters for compensation were chosen to be the same as for the simulations: the degree of the polynomial was 15, and the boundary of $Ω_{Low}$ and $Ω_{High}$ was 1.5 times the fundamental frequency.

Figure 2 shows single lines of the compensated results for the measured data, for which the RMS error values are shown in Table 2. Since the case in Fig. 2(a) is an ordinary situation in which the gamma model can be applied, both the gamma model and the polynomial model reduced similar amounts of error. On the other hand, the compensation result for the gamma model contains more error than the polynomial model for the case in Fig. 2(b). This should be caused by model mismatch of the gamma model, as illustrated in Fig. 1.

Table 2. RMS and Percentage of Error the Measured Data

View Table | View all tables in this article

Fig. 2. Phase reconstruction error of the three-step phase-shifting method of a flat surface measured by a commercial projector and camera. (a) was obtained by projecting sinusoidal fringes, while (b) was defocused by binary fringes. The blue dashed lines represent error without compensation. The red dotted lines are error after compensation by the gamma model, while the green solid lines are error after compensation by the polynomial model.

Download Full Size | PDF

Although the error was reduced by the proposed method for both cases, the amount of reduction was not as convincing as in the previous simulation. Possible reasons for these results are the following. First, the proposed method only account for phase reconstruction error related to the nonlinear distortion of fringes. However, there are other sources of error including vibration, illuminance variation, and random noise. Every nonideal situation not only contributes to error but also reduces the accuracy of estimation for the compensation algorithm. This could be a factor that is limiting the practical performance of the proposed method. Nevertheless, the proposed method can be combined with other methods that compensate error not caused by distortion to improve the result. Second, the illumination and nonlinearity of the observed images depend on its position in general. Therefore, a nonlinear map compensating distortion should also be position dependent. However, in this paper, the proposed method only considers a single nonlinear map uniformly applied to each pixel value. This global nature could be another factor limiting the performance. Although it must be possible to extend the proposed method to adapt to such nonuniformity, some effort is needed for the modification.

C. Application to Real Objects

In order to demonstrate the performance of the proposed method for a practical situation, with an actual application in mind, the ear pinna of a dummy head was measured. An ear pinna is an important part of the ear that supplies information regarding the position of a sound source to the human. Measuring the shape of an actual ear pinna enables the simulation of localized sound, which can be used for personalizing a 3D audio system [31,32].

The ear pinna of a dummy head (NEUMANN KU100) was measured by the four-step phase-shifting method. Figure 3 shows one of the four fringe images, and Fig. 4 shows the corresponding measured result with and without compensation. Since the illumination of the images varies according to the position, the top 20 rows of the images (nearly flat part of the images) were used for estimating optimal parameters. Also, the nonlinear map obtained for compensating reconstruction error was scaled at each position according to the illumination of the images. For better comparison, cross sections of the results are also depicted in Fig. 5.

Fig. 3. One of four fringe patterns of the ear pinna of a dummy head used as test data.

Download Full Size | PDF

Fig. 4. Measured ear pinna of a dummy head obtained by the four-step phase-shifting method. The viewing angle was adjusted so that the periodic error caused by nonlinearity became more apparent. White lines (a) and (b) correspond to Figs. 5(a) and 5(b), respectively.

Download Full Size | PDF

Fig. 5. Cross sections of the measured ear pinna. (a) and (b) correspond to the white lines in Fig. 4. The blue dashed lines represent the results without compensation, while the red dotted lines and the green solid lines are, respectively, the results after compensation with the gamma model and the polynomial model.

Download Full Size | PDF

It can be seen in Figs. 4 and 5 that the result obtained without compensation contains error as a periodic pattern, which could be problematic for later application. On the other hand, the result compensated by the proposed method contains less error. These figures indicate that the polynomial model suppressed more error than the gamma model, as in Fig. 2. Therefore, we conclude that the proposed method can compensate measurement error caused by nonlinearity that is not only related to the gamma model but also related to other models.

5. CONCLUSIONS

In this paper, a methodology of compensating nonlinear distortion of sinusoidal fringes was proposed. The proposed criterion of distortion allows us to formulate a compensation method as an optimization problem. Moreover, the linear combination model allows an analytic solution of the optimization problem, which can be obtained by a simple procedure. Through simulations and real measurements, it was confirmed that the proposed method can reduce error caused by multiple types of nonlinearity from observed data only.

As mentioned in Section 4.B, at least two points should be improved to obtain more accurate measurement results. Since there is not only nonlinearity but also several other sources of error, combining the proposed method with other compensation methods would be preferable. Moreover, position-adaptive modification of the proposed method should be investigated in the future to deal with nonlocal behavior of illumination and nonlinear distortion of fringes.

Funding

Japan Society for the Promotion of Science (JSPS) (15J08043, 16J06772).

REFERENCES

1. M. Takeda and K. Mutoh, “Fourier transform profilometry for the automatic measurement of 3-D object shapes,” Appl. Opt. 22, 3977–3982 (1983). [CrossRef]

2. P. S. Huang, F. Jin, and F. P. Chiang, “Quantitative evaluation of corrosion by a digital fringe projection technique,” Opt. Lasers Eng. 31, 371–380 (1999). [CrossRef]

3. S. Zhang, “Recent progresses on real-time 3D shape measurement using digital fringe projection techniques,” Opt. Lasers Eng. 48, 149–158 (2010). [CrossRef]

4. S. S. Gorthi and P. Rastogi, “Fringe projection techniques: whither we are?” Opt. Lasers Eng. 48, 133–140 (2010). [CrossRef]

5. K. Yatabe and Y. Oikawa, “Convex optimization based windowed Fourier filtering with multiple windows for wrapped phase denoising,” Appl. Opt. 55, 4632–4641 (2016). [CrossRef]

6. X. Y. Su, W. S. Zhou, G. von Bally, and D. Vukicevic, “Automated phase-measuring profilometry using defocused projection of a Ronchi grating,” Opt. Commun. 94, 561–573 (1992). [CrossRef]

7. K. Hibino, B. F. Oreb, D. I. Farrant, and K. G. Larkin, “Phase shifting for nonsinusoidal waveforms with phase-shift errors,” J. Opt. Soc. Am. A 12, 761–768 (1995). [CrossRef]

8. Y. Surrel, “Design of algorithms for phase measurements by the use of phase stepping,” Appl. Opt. 35, 51–60 (1996). [CrossRef]

9. Y. Hu, J. Xi, E. Li, J. Chicharo, and Z. Yang, “Three-dimensional profilometry based on shift estimation of projected fringe patterns,” Appl. Opt. 45, 678–687 (2006). [CrossRef]

10. M. J. Baker, J. Xi, and J. F. Chicharo, “Elimination of γ non-linear luminance effects for digital video projection phase measuring profilometers,” in Proceedings of IEEE International Symposium on Electronic Design, Test and Applications (DELTA) (IEEE, 2008), pp. 496–501.

11. B. Pan, Q. Kemao, L. Huang, and A. Asundi, “Phase error analysis and compensation for nonsinusoidal waveforms in phase-shifting digital fringe projection profilometry,” Opt. Lett. 34, 416–418 (2009). [CrossRef]

12. K. Liu, Y. Wang, D. L. Lau, Q. Hao, and L. G. Hassebrook, “Gamma model and its analysis for phase measuring profilometry,” J. Opt. Soc. Am. A 27, 553–562 (2010). [CrossRef]

13. P. S. Huang, C. Zhang, and F.-P. Chiang, “High-speed 3-D shape measurement based on digital fringe projection,” Opt. Eng. 42, 163–168 (2003). [CrossRef]

14. K. Ishikawa, K. Yatabe, N. Chitanont, Y. Ikeda, Y. Oikawa, T. Onuma, H. Niwa, and M. Yoshii, “High-speed imaging of sound using parallel phase-shifting interferometry,” Opt. Express 24, 12922–12932 (2016). [CrossRef]

15. H. Guo, H. He, and M. Chen, “Gamma correction for digital fringe projection profilometry,” Appl. Opt. 43, 2906–2914 (2004). [CrossRef]

16. T. Hoang, B. Pan, D. Nguyen, and Z. Wang, “Generic gamma correction for accuracy enhancement in fringe-projection profilometry,” Opt. Lett. 35, 1992–1994 (2010). [CrossRef]

17. Z. Li and Y. Li, “Gamma-distorted fringe image modeling and accurate gamma correction for fast phase measuring profilometry,” Opt. Lett. 36, 154–156 (2011). [CrossRef]

18. X. Zhang, L. Zhu, Y. Li, and D. Tu, “Generic nonsinusoidal fringe model and gamma calibration in phase measuring profilometry,” J. Opt. Soc. Am. A 29, 1047–1058 (2012). [CrossRef]

19. P. Zhou, X. Liu, Y. He, and T. Zhu, “Phase error analysis and compensation considering ambient light for phase measuring profilometry,” Opt. Lasers Eng. 55, 99–104 (2014). [CrossRef]

20. J. Yao, C. Xiong, Y. Zhou, H. Miao, and J. Chen, “Phase error elimination considering gamma nonlinearity, system vibration, and noise for fringe projection profilometry,” Opt. Eng. 53, 094102 (2014). [CrossRef]

21. S. Ma, C. Quan, R. Zhu, L. Chen, B. Li, and C. J. Tay, “A fast and accurate gamma correction based on Fourier spectrum analysis for digital fringe projection profilometry,” Opt. Commun. 285, 533–538 (2012). [CrossRef]

22. S. Zhang, “Comparative study on passive and active projector nonlinear gamma calibration,” Appl. Opt. 54, 3834–3841 (2015). [CrossRef]

23. Z. Cai, X. Liu, X. Peng, Z. Zhang, H. Jiang, Y. Yin, and S. Huang, “Phase error compensation methods for high-accuracy profile measurement,” Meas. Sci. Technol. 27, 045201 (2016). [CrossRef]

24. S. Zhang and S. Yau, “Generic nonsinusoidal phase error correction for three-dimensional shape measurement using a digital video projector,” Appl. Opt. 46, 36–43 (2007). [CrossRef]

25. Y. Xu, L. Ekstrand, J. Dai, and S. Zhang, “Phase error compensation for three-dimensional shape measurement with projector defocusing,” Appl. Opt. 50, 2572–2581 (2011). [CrossRef]

26. K. Liu, S. Wang, D. L. Lau, K. E. Barner, and F. Kiamilev, “Nonlinearity calibrating algorithm for structured light illumination,” Opt. Eng. 53, 050501 (2014). [CrossRef]

27. C. Zhang, H. Zhao, L. Zhang, and X. Wang, “Full-field phase error detection and compensation method for digital phase-shifting fringe projection profilometry,” Meas. Sci. Technol. 26, 035201 (2015). [CrossRef]

28. M. J. Baker, J. Xi, and J. F. Chicharo, “Neural network digital fringe calibration technique for structured light profilometers,” Appl. Opt. 46, 1233–1243 (2007). [CrossRef]

29. S. Sonoda and N. Murata, “Sampling hidden parameters from oracle distribution,” in Proceedings of International Conference on Artificial Neural Networks (ICANN) (Springer, 2014), pp. 539–546.

30. L. N. Trefethen, “Householder triangularization of a quasimatrix,” IMA J. Numer. Anal. 30, 887–897 (2010). [CrossRef]

31. M. Otani and S. Ise, “Fast calculation system specialized for head-related transfer function based on boundary element method,” J. Acoust. Soc. Am. 119, 2589–2598 (2006). [CrossRef]

32. M. Otani, T. Hirahara, and S. Ise, “Numerical study on source-distance dependency of head-related transfer functions,” J. Acoust. Soc. Am. 125, 3253–3261 (2009). [CrossRef]

	Original	Gamma	Polynomial
Case (a)	$1.78 E - 1$	$4.47 E - 4$ (0.25%)	$3.67 E - 3$ (2.06%)
Case (b)	$1.67 E - 1$	$2.38 E - 2$ (14.3%)	$8.10 E - 5$ (0.05%)
Case (c)	$1.56 E - 1$	$1.75 E - 1$ (112%)	$1.26 E - 3$ (0.80%)
Case (d)	$1.34 E - 1$	$1.10 E - 1$ (81.9%)	$4.08 E - 4$ (0.30%)

	Original	Gamma	Polynomial
Case (a)	$1.47 E - 1$	$1.71 E - 2$ (11.6%)	$1.07 E - 2$ (7.28%)
Case (b)	$1.37 E - 1$	$7.58 E - 2$ (55.2%)	$2.09 E - 2$ (15.3%)

	Original	Gamma	Polynomial
Case (a)	$1.78 E - 1$	$4.47 E - 4$ (0.25%)	$3.67 E - 3$ (2.06%)
Case (b)	$1.67 E - 1$	$2.38 E - 2$ (14.3%)	$8.10 E - 5$ (0.05%)
Case (c)	$1.56 E - 1$	$1.75 E - 1$ (112%)	$1.26 E - 3$ (0.80%)
Case (d)	$1.34 E - 1$	$1.10 E - 1$ (81.9%)	$4.08 E - 4$ (0.30%)

	Original	Gamma	Polynomial
Case (a)	$1.47 E - 1$	$1.71 E - 2$ (11.6%)	$1.07 E - 2$ (7.28%)
Case (b)	$1.37 E - 1$	$7.58 E - 2$ (55.2%)	$2.09 E - 2$ (15.3%)

Compensation of fringe distortion for phase-shifting three-dimensional shape measurement by inverse map estimation

Abstract

1. INTRODUCTION

2. FRINGE DISTORTION

3. BLIND COMPENSATION OF A DISTORTED FRINGE

A. Fourier-Transform-based Criterion of Distortion

B. Gamma-Model-based Compensation

C. Compensation by Inverse Map Estimation

D. Summary of the Proposed Algorithm Solving Eq. (10)

E. Choice of Approximating Functions ${ψ_{n}}$

4. EXPERIMENTS

A. Simulation

B. Measured Flat Surface

C. Application to Real Objects

5. CONCLUSIONS

Funding

REFERENCES

Cited By

Figures (5)

Tables (2)

Equations (30)

Applied Optics

Abstract

1. INTRODUCTION

2. FRINGE DISTORTION

3. BLIND COMPENSATION OF A DISTORTED FRINGE

A. Fourier-Transform-based Criterion of Distortion

B. Gamma-Model-based Compensation

C. Compensation by Inverse Map Estimation

D. Summary of the Proposed Algorithm Solving Eq. (10)

E. Choice of Approximating Functions {ψn}

4. EXPERIMENTS

A. Simulation

B. Measured Flat Surface

C. Application to Real Objects

5. CONCLUSIONS

Funding

REFERENCES

Cited By

Figures (5)

Tables (2)

Equations (30)

Applied Optics

E. Choice of Approximating Functions ${ψ_{n}}$