## Abstract

Absolute distance measurements could be achieved by the technique of frequency scanning interferometry (FSI), which could provide a micron-level accuracy over several tens of meters. However, as the scanning range and distance increasing, the measurement accuracy and resolution would be seriously influenced by the dispersion mismatch effect, which is mainly caused by the dispersion difference between optical fiber and air. A dispersion compensation method was proposed in this paper, and the mismatch effect would be suppressed effectively without complicated calculations for each single measurement. The chirp in resampling clock would be corrected with a pre-calibration procedure, and the compensation factor is demonstrated for different distances. A calibration method of dispersion compensation factor is also proposed, and the linear regression residual is employed as the criterion. Then, the compensation method is evaluated by experiments with different spatial distances and scanning ranges. Finally, a 60-m experiment is carried out for accuracy comparison, and the measurement accuracy is better than 15 µm by comparing with commercial interferometers.

© 2021 Optical Society of America under the terms of the OSA Open Access Publishing Agreement

## 1. Introduction

High-accuracy and high-resolution distance measurement plays an important role in the manufacturing industry, such as rocket assembly, ship building, and aerospace engineering [1–3]. These measurement tasks are characterized by the scale of several tens of meters, the complicated structure, and the microns errors, which are equivalent to a 10^{−6} relative accuracy. Although the traditional interferometer has a high relative accuracy of 10^{−7}, its incremental principle limits the application. Consequently, these challenges have promoted the research of high-accuracy absolute distance measurement (ADM) technology [4–6].

Frequency scanning interferometry (FSI) is an absolute ranging technique with high performance, which comes from the idea of frequency-modulated continuous-wave (FMCW) radar in the radio-frequency regions [7]. In the 1980s, the emergence of semiconductor lasers promoted the FSI development in the optics field [8]. Benefiting from the heterodyne interference and the tunable laser, the FSI possesses the advantages of strong anti-interference, non-ambiguity range, and high accuracy [9–11]. These features have made FSI widely applied in optical coherence tomography (OCT) for medical imaging [12], optical frequency domain reflectometry (OFDR) for characterizing optical fiber networks [13], and manufacturing industry for ADM [14–16]. However, in practical application, the non-linearity of laser frequency sweep has a great influence on the measurement accuracy [17,18]. Considering the convenience and economy, the most popular way to solve the non-linearity is the frequency-sampling method [17], which using a fiber reference interferometer as a sampling clock signal.

Theoretically, the measurement resolution and accuracy could be improved by increasing the scanning range in the FSI [5]. However, the fiber dispersion of the reference interferometer would cause the frequency chirp of sampling clock, especially in the case of large scanning range and long-distance measurement. Essentially, this dispersion error is caused by the dispersion coefficient difference between the fiber and the free space, so it usually be called the dispersion mismatch effect [19]. Moreover, the mismatch effect leads to broadening and shifting of the distance spectrum, which brings about tens or even hundreds micron of accuracy loss [20]. Using the dispersion compensation medium is the most direct and quick way to reduce the influence of dispersion [21], but it’s difficult to perfectly match the sizes of the compensation medium and the dispersion medium, the dispersion error could not be fully eliminated. Therefore, the most popular method of dispersion compensation is based on numerical compensation [19,20,22–25]. In 2004, Maciej Wojtkowski et al. [22] proposed a numerically compensated dispersion algorithm using in the OCT field, which finds the optimal dispersion compensation phase in an iterative manner. Although this method achieves automatic compensation, it needs to balance the relationship between the iteration step length and compensation performance. As for the dispersion compensation in the ADM field, researchers have proposed various methods to obtain the dispersion compensation phase, such as the evaluation function [20], iteration method [19], curve fitting [23], interpolation method [24], etc. However, the compensation phase in these methods is applied to the frequency-resampled signal and is related to the target distance, which means that the dispersion compensation phase must be recalculated every time the target distance changes. In the industrial application with numerous targets to be measured, this recalculation step will greatly increase the time of distance extraction. A solution has been proposed by Lu et al. [25], who use the chirp decomposition to separate the dispersion compensation factor and the target distance. However, the calculation of chirp decomposition algorithm is time-consuming, which is also difficult to meet the efficiency of industrial measurement.

In this paper, we first derived a mathematical model for analyzing dispersion mismatch effect in Section 2. The group velocity dispersion (GVD) coefficient of the fiber is considered as the underlying cause of the sampling clock error. Then, a numerical dispersion compensation method suitable for industrial measurement has been proposed, which can avoid the optimization step of compensation phase for every distance. Different from the compensation method above, this method is aimed at correcting the dispersion chirp of sampling clock signal before the frequency-sampling process, so the dispersion compensation factor can be separated from the target distance. Besides, the compensation factor is calibrated in advance, and the linear regression phase residual is used as the criterion. In Section 3, the compensation method has been evaluated under the conditions of different distances and scanning ranges. Furthermore, the absolute distance of 60 m range has been measured, and the accuracy after dispersion compensation is better than 15 µm. Finally, in section 4, the conclusion and future work are summarized.

## 2. Principles

#### 2.1 Basic principle of frequency sampling in FSI measurement system

The schematic diagram of our FSI measurement system is shown in Fig. 1, which is improved on the typical dual-path FSI measurement system. In this system, the frequency-sampling method is used to correct the nonlinear scanning of the tunable laser, the continue-wave single frequency laser is introduced to monitor the measuring optical path variation in free space [26], and the gas absorption cell is used to preliminarily calibrate the optical path difference (OPD) of the delay optical fiber, which is regarded as a length reference. Detailed calibration process can be found in [5].

The beat signal of the reference interferometer is detected by the Photodetector 2, which can be expressed as:

*ν*represents the instantaneous frequency of the tunable laser, ${R_{\textrm{ref}}}$ is the geometric length of reference interferometer, ${n_{\textrm{fiber}}}$ represents the group refraction index of the fiber, and

*c*is the speed of light in vacuum. Here the initial phase is ignored.

For the measurement interferometer, the measurement light travels both in the fiber and in free space. After suppressing the Doppler errors, the beat signal of the measurement interferometer is filtered by the low-passed filter [26]. Similarly, the measurement signal can be expressed as:

Then, the frequency-sampling method is used to remove scanning non-linearity of the measurement signal. The sampling clock signal formed by the reference interferometer could be written as:

*N*is the total number of sampling points.

With Eq. (3) and Eq. (2), the resampled signal (measurement interference signal after correcting the scanning non-linearity) can be expressed as:

Conclusively, by applying the fast Fourier transform (FFT) or Chirp-Z algorithm on the resampled signal, the measurement distance can be calculated as:

#### 2.2 Influence of fiber dispersion

It should be noted that the derivation above assumes that the refractive index of the fiber ${n_{\textrm{fiber}}}$ is constant, which is reasonable only at a narrow frequency tuning range. However, while using a broadband tuning laser to improve the resolution, the refractive index of the fiber varies linearly with the tuning range. This adds chromatic dispersion chirp to the sampling clock signal, which further leads to a systematic frequency-sampling error. So, the effects of fiber dispersion could not be ignored. In this case, the refractive index ${n_{\textrm{fiber}}}$ at the scanning frequency *ν* is transformed into ${n_{\textrm{fiber}}}(\nu )$, and the beat signal of the reference interferometer can be rewritten as:

Similarly, the measurement interferometer signal can be rewritten as:

To approximate the effects of fiber dispersion, the propagation coefficient, ${{\beta (\nu )\textrm{ = }2\pi \nu {n_{\textrm{fiber}}}(\nu )} / c}$, can be written as a Taylor expansion about the starting frequency ${\nu _0}$:

Substituting Eq. (8) and Eq. (9) into Eq. (6) yields:

In Eq. (10), it can be seen that the first term is a fixed phase, and the second term is a linear phase containing the information of the optical path difference, and the third term is a quadratic chirp term which is produced by the fiber dispersion. In order to analyze the influence of fiber dispersion on the sampling clock signal in the time domain, it is assumed that the sweep rate of the tunable laser is constant to ignore the scanning non-linearity on the sampling clock. Therefore, the instantaneous frequency of the light is $\nu = {\nu _0} + \mu t$, where *µ* is the tuning slope and *t* is the tuning time. Then, the frequency of the sampling clock can be obtained by deriving the beat signal phase of the reference interferometer:

It is shown that the fiber dispersion leads to a shift in the frequency of the sampling clock, where the shift factor is ${{2\pi \mu {\beta _2}} / {{\beta _1}}}$. Then, the group time delay of the long optical fiber in reference interferometer can be calculated by:

According to Eq. (12), it could be found that the frequency shift is result from the time delay of the long optical fiber changing with the scanning range. For the scanning wavelength width ranges from 5 nm to 40 nm, the relative deviation ${{2\pi \mu {\beta _2}t} / {{\beta _1}}}$ varies from tens to hundreds part per million. Furthermore, while this beat signal is used as a frequency-sampling clock, the resampled signal will be introduced into a chromatic dispersion chirp.

With Eq. (8) and Eq. (9), the signal of the measurement interferometer can be transformed into:

Then, the reference interferometer considering the influence of fiber dispersion is used as sampling clock, yielding Eq. (14).

Correspondingly, the resampled signal can be expressed as:

Comparing Eq. (15) with Eq. (4), it could be inferred that the first term in resampled signal is a linear term without considering the fiber dispersion effect, the second term is a quadratic term caused by the dispersion mismatch, and the third is a constant phase. In the second term, the factor ${{4{\pi ^2}{\beta _2}{R_{\textrm{mea}\_\textrm{air}}}{n_{\textrm{air}}}} / {\left( {{\beta _1}^2R_{\textrm{ref}}^2{n_{\textrm{fiber}}}} \right)}}$ is only proportional to the distance of the free space in measurement interferometer, which doesn’t contain the optical fiber difference. This is because both the fiber used as sampling clock and the fiber used as measurement signal have the same GVD coefficient. The dispersion effect is eliminated when the fiber sampling clock samples the fiber interference signal. But for the space part, there is a GVD coefficient difference between the air and the fiber reference interferometer, which creates the dispersion mismatch. The dispersion mismatch introduces a quadratic dispersion term into the frequency-sampling process. After performing the FFT or Chirp-Z algorithm on this signal, the quadratic term makes the signal spectrum become broadened, the resolution decline, the center distance shift.

Here, simulations were made to demonstrate the influence of the dispersion mismatch in FSI. The G652.D standard single mode fiber is selected, so the inverse group velocity ${\beta _1}$ is 4.9×10^{−9} s/m, the group velocity dispersion ${\beta _2}$ is −23 ps^{2}/km, and the effective group refractive index of 1550 nm is 1.4682. The geometric length difference of the fiber in reference interferometer is 110 m, and the fiber difference in measurement interference is 0 m. In Fig. 2(a), the laser tunes from 1520 to 1535 nm, and the target was moved from 5 m to 30 m optical path difference (OPD) in the free space. In Fig. 2(b), a target located at 15 m OPD was measured. The tuning range was set as 1522.5–1532.5 nm, 1520–1535 nm, 1517.5–1537.5 nm, 1515–1540 nm, 1512.5–1542.5 nm, and 1510–1545 nm. It is illustrated that the center distance shifts by about 113 µm for every 5 m increase at the scanning range of 15 nm. When the distance increases to 20 m, the distance spectrum splits into two peaks. At 15 m OPD, every increase of 5 nm range will broaden the spectrum by about 230 µm. It is impossible to accurately identify spectral peak when the scanning range is 20 nm. Therefore, it is necessary to compensate dispersion mismatch in the case of large scanning range or long-distance measurement.

#### 2.3 Compensation method for the dispersion mismatch effect

According to the discussion above, the dispersion mismatch comes from the fiber dispersion effect of the sampling clock signal. The usual compensation methods are performed on the resampled signal. At this time, the chirp slope is related to the spatial distance and the GVD coefficient of sampling clock signal, which could be proved by Eq. (15). Therefore, when the spatial distance changes, the dispersion compensation phase needs to be recalculated accordingly. In this paper, we propose a dispersion compensation method that is directly applied to the sampling clock. After compensation, there is no mismatch between the clock and the free space. It means that dispersion compensation needs to be performed before scanning non-linearity correction. According to the Eq. (10), the phase of the reference interference signal consists of three terms. Only the third term is caused by the dispersion effect, which is composed of a fixed coefficient $2 \pi^{2} R_{\text {ref }} \beta_{2}$ and the quadratic of the scanning frequency interval $\Delta v$. We could use a phase operation to subtract the third term, thereby compensating the dispersion effect. Therefore, the reference interference signal is multiplied by a complex phase compensation term, which can be simplified and expressed as:

Because the OPD in measurement interferometer also contains fiber distance, similar compensation is applied to measurement interference signal:

*A*,

*B*,

*C*,

*D*,

*E*, and

*F*could be written by Eq. (18), $\Delta {\nu _{\textrm{comp}}}$ is the estimate of actual scanning frequency interval $\Delta \nu$, ${B_{\textrm{comp}}}$ and ${E_{\textrm{comp}}}$ represent the compensation factors in reference interferometer and measurement interferometer respectively. Obviously, the dispersion mismatch will be eliminated when ${B_{\textrm{comp}}}\Delta \nu _{\textrm{comp}}^2$ equals to $B\Delta {\nu ^2}$ and ${E_{\textrm{comp}}}\Delta \nu _{\textrm{comp}}^2$ equals to $E\Delta {\nu ^2}$.

From Eq. (16) and Eq. (17), we could see that the compensation factors ${B_{\textrm{comp}}}$, ${E_{\textrm{comp}}}$ and the interval $\Delta {\nu _{\textrm{comp}}}$ are the only three parameters used to compensate dispersion. Particularly, because ${B_{\textrm{comp}}}$ and ${E_{\textrm{comp}}}$ are constants, they can be obtained through accurate calibration in advance and then used for dispersion compensation. The detailed extraction procedure of these three parameters and the process of dispersion compensation are described as follows:

- (1) The estimation of the scanning frequency interval $\Delta {\nu _{\textrm{comp}}}$. Both of the reference and the measurement signal are first processed by the Hilbert Transform to extract the instantaneous phase. According to Eq. (10), the scanning frequency interval could be monitored by the phase change of reference interferometer. Because ${{\pi {\beta _2}\Delta \nu } / {{\beta _1} \ll 1}}$, the quadratic term could be ignored with a little loss of phase accuracy. Therefore, the scanning frequency change could be estimated by:$$\Delta {\nu _{\textrm{comp}}} = \frac{{\Delta \varphi {^{\prime}_{\textrm{ref}}}}}{{2\pi {R_{\textrm{ref}}}{\beta _1}}}$$where $\Delta \varphi _{\textrm{ref}}^{\prime}$ is the phase change of the reference interferometer. In particular, this evaluation method can realize a frequency scanning range estimation point by point.
- (2) The accurate calibration of the compensation factors ${B_{\textrm{comp}}}$ and ${E_{\textrm{comp}}}$. According to the nominal parameters of the optical fiber and the previous calibration for the OPD of optical fiber, the initial value of compensation factors can be obtained. The relationship between ${B_{\textrm{comp}}}$ and ${E_{\textrm{comp}}}$ follows:

*B*and ${E_{\textrm{comp}}}$ equals to

*E*. Since the second-order coefficient is much smaller than the first-order, a linear regression was performed using least squares method to eliminate the first-order slope in Eq. (21). Then, the regression phase residual could be calculated by:

- (3) Dispersion compensation and distance calculation. With the three parameters ${B_{\textrm{comp}}}$, ${E_{\textrm{comp}}}$ and $\Delta {f_{\textrm{comp}}}$ obtained above, the dispersion mismatch could be corrected by Eq. (16) and Eq. (17). Finally, by the compensated clock signal sampling the compensated measurement signal, a resampled signal without dispersion is acquired. The target distance could be obtained then by the FFT or Chirp-Z algorithm from Eq. (5).

## 3. Experiments and results

#### 3.1 System configuration

An experimental setup of our measurement system was established, shown in Fig. 1. The system combines with a measurement interferometer, and a reference interferometer. The external cavity diode laser (81606A #116 series, Keysight) is applied as the light source to output the frequency-modulated light. A continue-wave (CW) single frequency laser (1550.12 nm, RIO Orion) is used to monitor the target variation. Moreover, polarization is controlled to make the polarization state of the tunable laser and single-frequency laser the same (to maximize the interference signal amplitude). Then, the frequency-modulated light is divided into 3 parts by two fiber couplers. Among them, 90% of the light goes into the measurement interferometer, 5% goes into the reference interferometer, and 5% goes into the gas absorption cell (TRI-H(80)-5-FCAPC, certified by NIST). The reference interferometer is designed as a fiber Mach-Zender form with about 110 m delay optical fiber. This interference signal is received by photodetector PD2 (PDB470C, Thorlabs), which is used as a sampling clock signal. It should be noted that all the optical fiber in this system was integrated in a shielding box to avoid environmental effects such as temperature variation and mechanical vibration. Also, the length of the fiber delay line was preliminarily calibrated by the hydrogen cyanide (H^{12}CN) gas absorption cell, whose absorption peaks are generated at known laser frequencies. The gas absorption peaks were detected by photodetector PD1 (PDA10CS-EC, Thorlabs). In the measurement interferometer, the frequency-modulated light combines with the single-frequency light. The mixed light then is split into two parts by a 90/10 fiber coupler. The 90% part goes through a circulator and emits from a collimator towards the target. Then, the reflection light passes through the Acousto-optical Modulator (T-M080-0.4C2J-3-F2S, Gooch & Housego) and the polarization controller. The AOM is used to generate 80 MHz frequency shifted. The interference signals of frequency-modulated light and single frequency light are separated by the Wavelength Division Multiplexer (WDM) and received by PD3 and PD4 (PDB470C, Thorlabs). Then, the measurement signal is acquired by using a mixer and a low-passed filter. All the signals are synchronously sampled and recorded by a digital acquisition system (PXIe-8840, NI) at a sampling rate of 10 MS/s. Then the further processing would be carried out on the computer.

#### 3.2 Calibration of the compensation factors

According to the compensation method discussed in the previous section, the compensation factors and should be calibrated first. In this section, a target located at approximately 13-m OPD was selected to calibrate factors.

Figure 3 shows the determination procedure of the scanning frequency interval and the phase regression residual. The original measurement and reference signals are shown in Fig. 3(a). After removing the signal envelope, the Hilbert transformation was used to obtain the instantaneous phases. The resolved phases curve after unwrapping are shown in Fig. 3(b). Then, we can estimate the scanning frequency interval from Eq. (19). In our experiment, the laser tunes from 1520 nm to 1535 nm, corresponding to a dropping optical frequency, as shown in Fig. 3(b). Due to the dispersion mismatch effect, the measurement phase is a quadratic function of the reference phase. However the second-order coefficient is much smaller than the first-order slope, shown in Fig. 3(c). In order to eliminate the first-order slope, a phase linear regression is implemented. The final phase regression residual is obtained by Eq. (22), as shown in Fig. 3(d). As previously predicted, the distribution of regression residual follows an upper convex function before dispersion compensation. The quadratic coefficient of the regression residual curve is obtained by polynomial fitting, and the value is −3.46E-14. Then, we choose the quadratic coefficient as the criterion to determine when the values of ${B_{\textrm{comp}}}$ and ${E_{\textrm{comp}}}$ are optimal. The OPD of two paths in reference interferometer ${R_{\textrm{ref}}}{n_{\textrm{fiber}}}$ is 162.1396 m, which was preliminarily calibrated by the gas absorption cell. Besides, the OPD of optical fiber in measurement interferometer ${R_{\textrm{mea}\_\textrm{fiber}}}{n_{\textrm{fiber}}}$ is 14.9215 m. Detailed description of the fiber length calibration method could be found in [5] and [25]. The ratio ${{{B_{\textrm{comp}}}} / {{E_{\textrm{comp}}}}}$ is calculated by the Eq. (20), and then compensation is performed according to Eqs. (16) and (17). By establishing a fixed step and increasing the compensation factors ${B_{\textrm{comp}}}$ and ${E_{\textrm{comp}}}$, we can acquire an optimal result.

Figure 4 shows the relationship between the compensation factors and the quadratic coefficient. Figure 4(a) and 4(b) respectively show that the regression residuals are under compensation and over compensation, and their quadratic coefficients are −4.59E-15 and 4.68E-15. In the Fig. 4(c), the regression residual is uniformly distributed with the sampling points, which is the optimal compensation. The quadratic coefficient is 4.31E-17 at this time. According to Eq. (21), we can obviously see that the quadratic coefficient changes linearly with the compensation factor ${B_{\textrm{comp}}}$, as shown in Fig. 4(d). It reveals that the optimal factors can also be calculated via a linear fitting method, which can reduce the number of iterations and improve calculation efficiency. Finally, in this FSI ranging system, the optimal dispersion compensation factor ${B_{\textrm{comp}}}$ is 4.12E-23 and ${E_{\textrm{comp}}}$ is 3.79E-24.

#### 3.3 Analysis of the FWHM

The full width at half maximum (FWHM) of the spectral peak is usually used to evaluate measurement resolution. According to Fig. 2, the dispersion mismatch effect could cause the spectrum broadening, especially in the case of long measuring range and large scanning range. In this section, the compensation effect would be verified by applying different distances and scanning range.

In order to obtain a long spatial distance, a folding light path is built on the vibration isolation optical table. The light path is shown in Fig. 5. A hollow retroreflector (UBBBR2.5-1S, Newport) and four plane mirrors (PF20-03-P01, Thorlabs) are used to form the light fold, which is four times the actual spatial distance. The maximum optical path difference (OPD) of this light path is about 27 m. Firstly, this compensation method is applied to different distances. 14 targets were measured from 1 m to 27 m OPD with 2 m interval. These targets are equivalent to the spatial distances from 0.5 m to 13.5 m without folding. We set the scanning range as 1520-1535 nm. 10 individual measurements were presented for each position, and each measurement was compensated by the proposed method. Figure 6 shows the comparison of the distance spectrums and regression residuals at 7 m OPD (Fig. 6(a) and Fig. 6(d)), 17 m OPD (Fig. 6(b) and Fig. 6(e)), 27 m OPD (Fig. 6(c) and Fig. 6(f)). The purple line is the result without dispersion compensation, and the orange line is the result after dispersion compensation. It illustrates clearly that as the distance increases, the peak broadening becomes more pronounced, and the quadratic coefficient of the regression residual becomes larger. When the OPD is 17 m or 27 m, the target distance couldn’t be extracted from the spectrum because the peak is not a single peak at this time. After performing dispersion compensation by the proposed method, the broadening of the spectral peak is well corrected, and the regression residuals are very close to zero. To obtain a theoretical value of the FWHM, we reconstructed an ideal resampled signal without dispersion based on the experimental data. By applying the Chirp-Z algorithm to the signal, the theoretical FWHM is 185.0 µm. The more detailed results are shown in Fig. 7, which shows the FWHM and residual quadratic coefficients for all 14 positions. It can be clearly seen that the FWHM changes from 186.2 µm to 1124.1 µm, and the value of residual quadratic coefficient changes from −3.4E-15 to −7.2E-14 before compensation. The FWHM value would jump at some positions because discontinuous broadening sites reach the FWHM in these positions. Benefits from the compensation of the proposed method, the FWHM is below 185.5 µm for every position, which is very close to the theoretical FWHM. Absolute value of residual quadratic coefficient with compensation is below 7E-16, which is about an order of magnitude smaller than before compensation.

Then, the compensation effect is verified by applying different scanning range. The target was measured at about 27 m OPD in the free space. We set the scanning center wavelength to 1527.5 nm, and the scanning range are 10 nm, 15 nm, 20 nm, 25 nm, 30 nm, and 35 nm respectively. Each range was measured 10 times. In theory, if there was no dispersion mismatch effect, the FWHM would decrease and the resolution improves as the scanning range increases. However, the dispersion mismatch would conversely increase the FWHM. The larger the range, the more severe the dispersion effect. The experimental results below just verify the view. Three similar comparisons of the distance spectrums and regression residuals are shown in Fig. 8, whose scanning range are 10 nm (Fig. 8(a) and Fig. 8(d)), 20 nm (Fig. 8(b) and Fig. 8(e)) and 30 nm (Fig. 8(c) and Fig. 8(f)).

Figure 9. shows the results of the FWHM and residual quadratic coefficient for 10 repetitive experiments. We can clearly see that the FWHM increases from 713.6 µm to 2.8 mm without dispersion compensation, and the FWHM decreases from 276.4 µm to 80.5 µm after compensation from Fig. 9(a). The theoretical FWHM is 276.2 µm, 185.0 µm, 139.2 µm, 111.6 µm, 93.2 µm, and 80.0 µm, which are also obtained by ideal resampled signals. All the FWHM after compensation are also close to the theoretical value. Figure 9(b) shows that the residual quadratic coefficient remains almost the same before compensation, because the distance of the target was unchanged. Using the above compensation method, the absolute value of coefficient drops approximately from 7E-14 to 3E-16. So, to sum up, all the experimental results show that this compensation method is effective for reducing the FWHM and improving resolution.

#### 3.4 Evaluation of the accuracy

Dispersion mismatch effect not only reduces the measurement resolution, but also leads to a shift of distance measurement result. In this section, we conducted an experiment to verify the accuracy compensation effect of this method.

Long-distance measurement experiment was carried out on the underground 80 m long rail in the National Institute of Metrology, and the experimental implementation is shown in Fig. 10. The FSI system measured the absolute distance of multiple points along the rail, and the distance between each measurement point is compared with three laser commercial interferometers (Agilent 5530). Three commercial interferometers are placed in an equilateral triangle distribution to correct the Abbe error. The target retroreflectors of the FSI system and the commercial interferometer are fixed on the mobile cart, and the relationship between prisms remains rigid. 31 measurement points were presented in a range of 60 m spatial distance with 2 m interval. 10 repetitive measurements were presented for each point by the FSI system. During the measurement, the temperature, relative humidity, pressure and CO_{2} concentration in the light path are recorded by environmental compensator.

The distance residuals compared with interferometers are shown in Fig. 11, which illustrates the trueness of the distance results. The refractive index of air is corrected by the Ciddor Equation. It is worth noting that there is a very small error in the OPD of delay optical fiber directly calibrated by the gas cell. This error is most probably due to the shift of the absorption peak with the pressure of the gas cell [27]. Therefore, in this section, the OPD of delay optical fiber is accurately calibrated by using commercial interferometers and linear guideway. Figure 11(a) and (b) shows the residuals before and after dispersion compensation, respectively. The standard deviation (STD) is used here for evaluating the system stability, which is presented as the blue dot in Fig. 11(a) and the blue bar in Fig. 11(b). The red dot represents the average residual of 10 repetitive measurements. Before the compensation, the distance residuals increase linearly with the increase of target distance, and the maximum residual was 1.4 mm at 60 m. The STD rises slowly with distance increase. After compensating the dispersion mismatch, a maximum residual about −15 µm within 60 m range has been achieved, shown in Fig. 11(b). The STD before and after compensation is less than 11 µm and almost unchanged, which means that this compensation method would not affect the stability of the system. It is worth to mention that the measurement results include the cosine error, which is caused by the non-parallelism of the FSI light path and the guide rail. The radius of the beam is about 15 mm at 60 meters, but the lateral deviation of the beam center will not exceed 5 mm. Then, the angle between the FSI light path and the guide rail is less than 18″, so the cosine error is below 0.2 µm in the 60 m range.

The measurement uncertainty can be theoretically analyzed according to Eq. (5). The uncertainty of the spatial distance ${R_{\textrm{mea}\_\textrm{air}}}$ is determined by the uncertainties of the group refraction index of air ${n_{\textrm{air}}}$, the frequency of the resampled signal ${f_{\textrm{resample}}}$, and the optical path difference of reference interferometer ${R_{\textrm{ref}}}{n_{\textrm{fiber}}}$, respectively. The uncertainty of ${n_{\textrm{air}}}$ is related to the environmental temperature, relative humidity, pressure and CO_{2} concentration [28], which are stable at 0.2 K, 2%, 15 Pa, and 60 ppm during the measurement. Then, the relative uncertainty of first term ${n_\textrm{air}}$ is about 0.16×10^{−6} based on the Ciddor Equation. The uncertainty of ${f_{\textrm{resample}}}$ is related to the frequency extraction algorithm and the signal-to-noise ratio, which will introduce a measurement standard uncertainty of 5.5 µm. The uncertainty of ${R_{\textrm{ref}}}{n_{\textrm{fiber}}}$ is related to the external vibration and temperature variation on optical fiber [29], the peak accuracy of gas absorption cell. Therefore, the third term ${R_{\textrm{ref}}}{n_{\textrm{fiber}}}$ will lead to a relative uncertainty about 0.21×10^{−6}. In conclusion, the combined measurement uncertainty of the system is 5.5 µm ± 0.37 µm/m.

## 4. Conclusion and future work

We have proposed an efficient and powerful dispersion compensation method for FSI measurement system. This method applies dispersion compensation phase directly to the sampling clock signal before the frequency-sampling procedure, which separates the dispersion coefficient from the target distance. Therefore, the optimization process of compensation factor is only implemented once in advance, which guarantees the efficiency of distance measurement and reduces additional optimization errors. The validity of the method is proved by experiments with different spatial distances and different scanning range. Based on the experimental results, the distance resolutions after dispersion compensation are all close to the theoretical value, which demonstrates that the method is suitable for different conditions. Furthermore, the accuracy after dispersion compensation is verified on the underground 80 m long rail in the National Institute of Metrology. The measurement residuals are less than ± 15 µm and the precision is below 11 µm in a range up to 60 m. In future work, this dispersion compensation method and data processing would be further improved to obtain higher distance extraction performance, and the compactness of the FSI measurement system would be well integrated. Then, the system would be used for the large volume coordinate measurements.

## Funding

National Natural Science Foundation of China (52127810, 51721003, 51835007).

## Disclosures

The authors declare no conflicts of interest.

## Data availability

Data underlying the results presented in this paper are not publicly available at this time but may be obtained from the authors upon reasonable request.

## References

**1. **R. H. Schmitt, M. Peterek, E. Morse, W. Knapp, M. Galetto, F. Härtig, G. Goch, B. Hughes, A. Forbes, and W. T. Estler, “Advances in large-scale metrology – review and future trends,” CIRP Ann. **65**(2), 643–665 (2016). [CrossRef]

**2. **L. Yan, J. Xie, B. Chen, Y. Lou, and S. Zhang, “Absolute distance measurement using laser interferometric wavelength leverage with a dynamic-sideband-locked synthetic wavelength generation,” Opt. Express **29**(6), 8344–8357 (2021). [CrossRef]

**3. **B. Hughes and M. S. Warden, “A novel coordinate measurement system based on frequency scanning interferometry,” in Proceedings of Coordinate Metrology Society Conference, vol. 8 (2013), pp. 18–24.

**4. **Z. Zhu, G. Xu, K. Ni, Q. Zhou, and G. Wu, “Synthetic-wavelength-based dual-comb interferometry for fast and precise absolute distance measurement,” Opt. Express **26**(5), 5747–5757 (2018). [CrossRef]

**5. **D. John, H. Ben, A. J. Lancaster, A. J. Lewis, A. J. H. Reichold, and M. S. Warden, “Multi-channel absolute distance measurement system with sub ppm-accuracy and 20 m range using frequency scanning interferometry and gas absorption cells,” Opt. Express **22**(20), 24869–24893 (2014). [CrossRef]

**6. **H. Wu, F. Zhang, T. Liu, F. Meng, J. Li, and X. Qu, “Absolute distance measurement by chirped pulse interferometry using a femtosecond pulse laser,” Opt. Express **23**(24), 31582–31593 (2015). [CrossRef]

**7. **D. N. Keep, “Frequency-modulation radar for use in the mercantile marine,” Proc. IEEE: Radio Electron. Eng. **103**(10), 519–523 (1956). [CrossRef]

**8. **B. Culshaw and I. P. Giles, “Frequency modulated heterodyne optical fiber Sagnac interferometer,” IEEE Trans. Microw. Theory Tech. **30**(4), 536–539 (1982). [CrossRef]

**9. **J. A. Stone, A. Stejskal, and L. Howard, “Absolute interferometry with a 670-nm external cavity diode laser,” Appl. Opt. **38**(28), 5981–5994 (1999). [CrossRef]

**10. **Z. Deng, Z. Liu, X. Jia, W. Deng, and X. Zhang, “Dynamic cascade-model-based frequency-scanning interferometry for real-time and rapid absolute optical ranging,” Opt. Express **27**(15), 21929–21945 (2019). [CrossRef]

**11. **H. Kikuta, K. Iwata, and R. Nagata, “Distance measurement by the wavelength shift of laser diode light,” Appl. Opt. **25**(17), 2976–2980 (1986). [CrossRef]

**12. **N. Lippok, S. Coen, P. Nielsen, and F. Vanholsbeeck, “Dispersion compensation in Fourier domain optical coherence tomography using the fractional Fourier transform,” Opt. Express **20**(21), 23398–23413 (2012). [CrossRef]

**13. **M. Jiang, D. Chen, and S. He, “Multiplexing scheme of Long-period grating sensors based on a modified optical frequency domain reflectometry,” IEEE Photonics Technol. Lett. **20**(23), 1962–1964 (2008). [CrossRef]

**14. **R. Schneider, P. Thurmel, and M. Stockmann, “Distance measurement of moving objects by frequency modulated laser radar,” Opt. Eng. **40**(1), 33–37 (2001). [CrossRef]

**15. **H. Yang, J. Deibel, S. Nyberg, and K. Riles, “High-precision absolute distance and vibration measurement with frequency scanned interferometry,” Appl. Opt. **44**(19), 3937–3944 (2005). [CrossRef]

**16. **X. Wu, H. Wei, H. Zhang, L. Ren, Y. Li, and J. Zhang, “Absolute distance measurement using frequency-sweeping heterodyne interferometer calibrated by an optical frequency comb,” Appl. Opt. **52**(10), 2042–2048 (2013). [CrossRef]

**17. **G. Shi, W. Wang, and F. Zhang, “Precision improvement of frequency-modulated continuous-wave laser ranging system with two auxiliary interferometers,” Opt. Commun. **411**(1), 152–157 (2018). [CrossRef]

**18. **T. J. Ahn, J. Y. Lee, and D. Y. Kim, “Suppression of nonlinear frequency sweep in an optical frequency-domain reflectometer by use of Hilbert transformation,” Appl. Opt. **44**(35), 7630–7634 (2005). [CrossRef]

**19. **X. Xu, G. Liu, B. Liu, F. Chen, Z. Zhuang, G. Yu, and L. Cheng, “Research on the fiber dispersion and compensation in large-scale high-resolution broadband frequency-modulated continuous wave laser measurement system,” Opt. Eng. **54**(7), 074102 (2015). [CrossRef]

**20. **G. Liu, X. Xu, B. Liu, F. Chen, T. Hu, C. Lu, and Y. Gan, “Dispersion compensation method based on focus definition evaluation functions for high-resolution laser frequency scanning interference measurement,” Opt. Commun. **386**(1), 57–64 (2017). [CrossRef]

**21. **K. Asaka and K. Ohbayashi, “Dispersion matching of sample and reference arms in optical frequency domain reflectometry-optical coherence tomography using a dispersion-shifted fiber,” Opt. Express **15**(8), 5030–5042 (2007). [CrossRef]

**22. **M. Wojtkowski, V. Srinivasan, T. Ko, J. Fujimoto, A. Kowalczyk, and J. Duker, “Ultrahigh-resolution, high-speed, Fourier domain optical coherence tomography and methods for dispersion compensation,” Opt. Express **12**(11), 2404–2422 (2004). [CrossRef]

**23. **P. Hao, X. Qu, and F. Zhang, “Micron-precision measurement using a combined frequency-modulated continuous wave ladar autofocusing system at 60 meters standoff distance,” Opt. Express **26**(12), 15186–15198 (2018). [CrossRef]

**24. **T. DiLazaro and G. Nehmetallah, “Multi-terahertz frequency sweeps for high-resolution, frequency-modulated continuous wave ladar using a distributed feedback laser array,” Opt. Express **25**(3), 2327–2340 (2017). [CrossRef]

**25. **C. Lu, G. Liu, B. Liu, F. Chen, T. Hu, Z. Zhuang, X. Xu, and Y. Gan, “Method based on chirp decomposition for dispersion mismatch compensation in precision absolute distance measurement using swept-wavelength interferometry,” Opt. Express **23**(25), 31662–31671 (2015). [CrossRef]

**26. **Y. Shang, J. Lin, L. Yang, Y. Liu, T. Wu, Q. Zhou, and J. Zhu, “Precision improvement in frequency-scanning interferometry based on suppression of magnification effect,” Opt. Express **28**(4), 5822–5834 (2020). [CrossRef]

**27. **W. C. Swann and S. L. Gilbert, “Line centers, pressure shift, and pressure broadening of 1530-1560 nm hydrogen cyanide wavelength calibration lines,” J. Opt. Soc. Am. B **22**(8), 1749–1756 (2005). [CrossRef]

**28. **P. E. Ciddor, “Refractive index of air: new equations for the visible and near infrared,” Appl. Opt. **35**(9), 1566–1573 (1996). [CrossRef]

**29. **J. Matsuoka, N. Kitamura, S. Fujinaga, T. Kitaoka, and H. Yamashita, “Temperature dependence of refractive index of SiO2 glass,” J. Non. Cryst. Solids **135**(1), 86–89 (1991). [CrossRef]