Open Access
Add to BibSonomy
Abstract
Terahertz spectrum is easily interfered by system noise and water-vapor absorption. In order to obtain high quality spectrum and better prediction accuracy in qualitative and quantitative analysis model, different wavelet basis functions and levels of decompositions are employed to perform denoising processing. In this study, the terahertz spectra of wheat samples are denoised using wavelet transform. The compound evaluation indicators (T) are used for systematically analyzing the quality effect of wavelet transform in terahertz spectrum preprocessing. By comparing the optimal denoising effects of different wavelet families, the wavelets of coiflets and symlets are more suitable for terahertz spectrum denoising processing than the wavelets of fejer-korovkin and daubechies, and the performance of symlets 8 wavelet basis function with 4-level decomposition is the optimum. The results show that the proposed method can select the optimal wavelet basis function and decomposition level of wavelet denoising processing in the field of terahertz spectrum analysis.
© 2024 Optica Publishing Group under the terms of the Optica Open Access Publishing Agreement
1. Introduction
Terahertz wave refers to the electromagnetic wave within a frequency range of 0.1-10THz. This waveband is between microwave and infrared waves. Compared with other wavelengths of electromagnetic waves, terahertz waves have the characteristics of low energy, high permeability, security, fingerprint spectrum and so on [1]. Measuring materials is one of the important application aspects by terahertz spectrum technologies. Both physical and chemical information of material structure can be obtained based on the analysis of the obtained terahertz spectrum through extracting the useful feature information. According to spectral features, the materials can be analyzed qualitatively or quantitatively for detection or classification [2]. Inevitably, the measurement results of the terahertz spectrum will be influenced by noises during the measuring process. Both different devices and measuring environment changes of the samples can lead to the changes in the noise source, in which the emitter noise is the most serious influential factor. During the detection process, it generates thermal noise (Johnson noise), and the background radiation of the detection process causes noise [3]. So, the interfering noise in the system needs to be removed or suppressed. Current studies indicate that the disruption and rearrangement of the water molecular network, which is composed of hydrogen bonds, occur within a timeframe on the order of picoseconds or subpicoseconds during molecular thermal motion, and the corresponding electromagnetic frequency range is exactly located in the terahertz band [4,5]. Therefore, water has a strong absorption property for terahertz radiation. Water interference in the experimental process also generates noise [6]. In view of the above problems, it needs to perform denoising processing for terahertz spectrum signals to extract useful features. Denoising terahertz spectra facilitates clearer observation of the characteristics and properties of the sample and analysis of the structure, composition and nature of the substance. It can provide high-quality spectra and better prediction accuracy for subsequent qualitative and quantitative analysis.
In the aspect of preprocessing of terahertz spectra, the denoising methods of signals mainly include Savitzky-Golay smoothing method [7], first and second derivative methods [8], orthogonal signal correction [9], and normalization [10], etc. In the Savitzky-Golay smoothing method, different setting of smoothing points will low the accuracy of the model. When the number of points is too small, new errors will be easily generated; when the number of points is too large, the spectrum data containing information will be easily lost. Although the first and second derivative methods can reduce the interference of baseline deviation and background noise effectively, the noise will increase during the derivative operation. In the orthogonal signal correction, the information contained in the part orthogonal to the effective signal can be filtered out, but all kinds of noise in the spectrum may not be orthogonal to the effective signal. So the orthogonal signal correction can not really remove all kinds of noise. The maximum and minimum values of the normalization are easily affected by outliers, so the robustness of this method is of poor performance.
The modeling effect is improved by using a suitable preprocessing method for the spectral signal data. Compared with above preprocessing methods, wavelet analysis has excellent time-domain localization and multi-resolution analysis capability as a time-frequency analysis method [11], which maximize the extraction of useful information from initial signal. The traditional Fourier transform was difficult to satisfy the requirements of local analysis, thus the concept of wavelet was proposed. In 1989, Mallat [12] proposed the concept of multi-resolution analysis, which made wavelets have the characteristics of band-pass filtering, so that wavelet decomposition and reconstruction was used to filter and reduced noise. Wavelets have a broad range of applications in various fields [13–15]. With the help of wavelet analysis and reconstruction techniques, the non-stationary characteristics of the signal is effectively portrayed while removing noise, and the detailed information in the spectral data is retained, which is more advantageous in the spectral preprocessing. Shi et al. [16] reported the uses of wavelet basis function daubechies 9 to carry out 6-level decomposition and reconstruction for the terahertz spectra of 12 kinds of biomedical compounds, which realized baseline correction, noise removal, and qualitative identification with an identification rate of nearly 100%. Du et al. [17] manifested the uses of daubechies 4 wavelet basis function to perform the preprocessing for spectrum data of 3 kinds of mixtures of saccharide isomers. In combination with PLS and SVR models, the prediction result of quantitative analysis was 0.989. Qu et al. [18] displayed the uses of symlets 4 wavelet basis function to conduct 4-level decomposition and reconstruction for the terahertz spectrum denoising processing of three kinds of pesticides, which effectively removed noise and kept original peak characteristics. The above three pieces of literature used different wavelet basis functions and decomposition levels during the wavelet denoising but failed to state the reasons for using. They only presented the denoising effect after wavelet decomposition and reconstruction in the results, and failed to explain the primary cause why the selected wavelet basis function and decomposition level were optimal.
The decomposition and reconstruction of spectral signals with different wavelet basis functions and decomposition levels lead to a certain degree of spectrum distortion, so it is necessary to find the best wavelet basis functions and decomposition levels to minimize the degree of spectrum distortion, higher signal-to-noise ratio and retained more effective information. To find the optimal wavelet basis function and optimal decomposition level, it needs to use wavelet denoising quality evaluation indicator. Traditional wavelet denoising quality evaluation indicators mainly include: root-mean-square error (RMSE) [19], signal to noise ratio (SNR) [20], correlation coefficient [21], and smoothness [22], etc. In 2001, Ferguson et al. [23] denoised terahertz spectra using four wavelet basis functions (Daubechies, Coiflet, Symslet and Meyer), achieving up to 10 dB improvement in SNR when applied to waveforms with initial SNRs of 3 dB. However, the traditional evaluation indicators based on the principle of statistics have inherent limitations. The wavelet denoising quality is evaluated only when the truth-value was accessible. In practical applications, the truth-value is unknown, and traditional indicators dissatisfy the demands of quality evaluation. Different traditional indicators have different characteristics, like signal detail information and approximation information. By selecting the traditional indicators with correlations from them and then merging the indicators with a certain method, the compound evaluation indicators are obtained to measure the wavelet denoising quality. In this paper, the compound evaluation indicators as described in Ref. [24] are selected for systematically analyzing the quality effect of wavelet transform in terahertz spectrum preprocessing. The indicator is simple, fast, highly accurate, and unaffected by varying degrees of noise pollution. Moreover, the terahertz spectra of wheat samples are used as the research object. In order to improve the effectiveness of terahertz spectral preprocessing, the optimal wavelet denoising method is employed to process the terahertz spectra.
2. Principles and methods
2.1 Terahertz experimental device
The Z3 terahertz time-domain spectrometer (Zomega Terahertz Corporation, USA) is taken as the experimental platform, whose spectral range is 0.1-3.5 THz; the resolution is <5 GHz; dynamic range is 70 dB (peak value); and the maximum delay time is 1.3 ns. The light source is the FFPRO780 fiber laser provided by TOPTICA company in Germany, whose central wavelength is 780 nm; average power is 150 mW; and pulse width is 100 fs. This THz-TDS system has two type methods: transmission type and reflection type. In this study, the former type has been adopted.
As shown in Fig. 1, this experimental platform adopts a femtosecond laser to generate a pulse laser beam. Through polarization beam splitter, it is divided into pump light and probe light. The probe light is transmitted to the ITO glass after passing through the time-delay device consisting of chopper and mirror. The pump light is focused onto the terahertz emitter, which when biased, emits a broadband terahertz pulse. After penetrating through samples and converging with probe light, it illuminates on the electro-optical crystal zinc telluride detector. When the terahertz electric field passes through the electro-optical crystal, the electro-optical crystal’s refractive indicator changes in an anisotropic pattern. These changes will influence the polarization of the detecting laser so that they can be detected by a differential probe.
2.2 Experimental method
The moisture content of wheat plays a significant role in determining the storage quality. The moisture content in the range of 11% and 13% of wheat is considered ideal storage environment, and a standard of 12.5% guarantees secure storage, further reducing the risk of grain quantity and storage costs [25]. In this experiment, the wheats from the same batch of grain reserve depot are used which have a moisture content of 12.5%. The pulverizer is used to smash the wheat. The smashed samples are put into the mortar for further refined grinding. After filtering with a 200-eye sieves, the wheat powder is prepared to reduce the generation of base tilt [26]. An electronic balance is used to weigh 200 mg samples, and the weighted samples are put into a pelleting mold at a pressure of 10 MPa and pressed for 2 minutes to prepare the tablets with a diameter of 13 mm. Therefore, the tested samples’ front and back surfaces are smooth and flawless. The spiral micrometer is adopted to measure and record the thicknesses of the samples. Subsequently, the tablet samples are put into sample bags and sealed. During the experiments, the temperature and the humidity are 23.5°C and 25% by measurements.
When the THz-TDS system is used to collect the time-domain spectrum of samples, it is necessary to detect the time-domain spectrum signal without samples, as a reference signal. Then, samples are randomly selected and 3 groups of spectral signals are obtained by repeating the detection 3 times for each sample, with the samples repositioned for each detection. Finally, the average value is calculated to obtain the sample spectral signal. The frequency domain spectra are obtained by performing fast Fourier transform.
2.3 Wavelet denoising principle
Wavelet transform changes the one-dimensional signal of the time domain into a two-dimensional space of time-scale (frequency). It features multiresolution, and also has the capabilities of local signal characterization both in the time domain and the frequency domain. Some signal characteristics that are difficult to observe in the original time domain due to aliasing can be separated and significantly reflected at some scale in the frequency domain. Thus, it can achieve the goal of extracting effective signal features.
Wavelet transform is to use a cluster of functions to approximate signals. Through dilation and translation of Function h, it can obtain a cluster of functions:
where, a is the dilation factor and b is the translation factor. According to h function and the selections of parameters a and b, it can realize continuous and discrete wavelet transform.The characteristics of wavelet basis are symmetry, compact support, regularity, and vanishing moment, etc. Symmetry relates to the bias of signal and standard filter. Compact support reflects the degree of excellence of localization characteristics. Regularity is used for describing the degree of function smoothness. The vanishing moment relates to the number of wavelet coefficients being 0. In general, a higher vanishing moment corresponds to a longer support length [27]. In this paper, the used wavelet basis are symlets N, daubechies N, fejer-korovkin N, and coiflets N. Wavelet basis determines a signal’s decomposition and reconstruction, which affects wavelet coefficient value and further affects the denoising effect.
Wavelet denoising mainly contains the following steps [28]:
- 1) Decomposing the signals with noise through wavelets. Relevant wavelet basis function and decomposition level N are selected to perform N-level decomposition for the signals with noise to obtain each scale’s wavelet coefficient ${\psi _{ji}}$.
- 2) Performing threshold estimation for each obtained scale of wavelet coefficients. Appropriate threshold and corresponding threshold function are selected to perform threshold estimation for each scale’s wavelet coefficient, to obtain each scale’s wavelet estimation coefficient $\widehat {{\psi _{ji}}}$.
- 3) Performing wavelet reconstruction for each scale’s wavelet estimation coefficient. According to wavelet decompositions, it can obtain 1-Nth level’s high-frequency wavelet estimation coefficient $\widehat {{\psi _{ji}}}\; $ and the Nth level’s low-frequency signal coefficient, and then performs wavelet reconstruction and obtains the denoising signals.
2.4 Compound evaluation indicators of wavelet denoising
Traditional wavelet denoising quality evaluation indicators mainly include RMSE, SNR, correlation coefficient, and smoothness. Generally, RMSE refers to the square root of the variance between the original signal and the denoising signal. Smoothness refers to the square root ratio of denoising signal first difference with original signal first difference. The two evaluation indicators mentioned above reveal the same principles, namely, the smaller their value are, the better the denoising effects show, and vice versa. SNR refers to the ratio of signal power with noise power. Correlation coefficient refers to the similarity of the original signal with the denoising signal. These two evaluation indicators show the reverse principles, namely, the larger their value are, the better the denoising effects show. However, the above indicators have some limitations. These indicators have different characteristics and can use multiple indicators to merge and meet the demand that any traditional single indicator is unsatisfied with the quality evaluation under the condition of an unknown truth-value.
In this paper, two indicators (RMSE and smoothness) are selected according to Ref. [24]:
- 1) Smoothness can be expressed as:$${r = \frac{{\mathop \sum \nolimits_{i = 1}^{n - 1} {{[{\hat{f}({i + 1} )- \hat{f}(i )} ]}^2}}}{{\mathop \sum \nolimits_{i = 1}^{n - 1} {{[{\hat{f}({i + 1} )- f(i )} ]}^2}}}}$$where, $f(i )$ is the original signal; $\hat{f}(i )$ is the reconstruction signal; n is the signal length. Smoothness refers to the square root ratio of signal first differences before and after denoising. The smaller value represents a better denoising effect.
RMSE represents the signal difference before and after denoising. The smaller value indicates a better denoising effect.
The simple combination of two indicators RMSE and smoothness, can easily lead to bias because the cardinal numbers of two indicators are different and the change ranges are different as well. To put these change into the same scale for comparison, normalization processing for these is performed at the beginning and the numerical values in the range of [0,1] is made into scalar quantities. The specific calculation formulas are as follow:
where, PRMSE and Pr are the RMSE and smoothness after standardization, respectively. RMSE and r are the RMSE and smoothness under the condition of unknown truth-values, respectively. max () and min () are the maximum operation and minimum operation, separately.During the merging process of the two indicators, assignment operations need to be performed due to different weights. This paper adopts the variable coefficient weighting method [29]. The coefficient of variability also refers to the standard deviation ratio, which equals the ratio of standard deviation with mean. It reflects the degree of variation of the indicator standard. The basic idea is that the indicator with a larger variable coefficient can better reflect the disparity of the estimated units. The specific weighting process can be described as:
where, CV denotes the coefficient of variability; W denotes the weight based on the coefficient of variable weighting; σ and μ are standard deviation operation and mean operation, correspondingly.Finally, the compound evaluation indicator T is obtained by the linear combination method, which is expressed as below:
According to the features and numerical characteristics of the two indicators, as well as the normalization principle and coefficient of variable weighting process, it can be concluded that the smaller numerical value shows, the better indicator assessment is.
2.5 Steps for implementing the algorithm
Compound evaluation indicator experiment of terahertz spectrum signal wavelet denoising quality is designed as follows:
- 1) Determine the optional parameters. Wavelet basis functions are symlets 2∼8, daubechies 1∼10, fejer-korovkin 4∼22, and coiflets 1∼5 wavelets. The decomposition levels are 1∼8, and the threshold is set up in accordance with the Universal Threshold criterion and soft threshold processing function.
- 2) Calculate the RMSE and smoothness of each family of different wavelet basis function and decomposition level under the condition of unknown truth-value, performance normalization, figuring out the value of WPRMSE and WPr weight coefficients.
- 3) The linear combination is carried out according to the weight coefficients of RMSE and smoothness, and the T value of compound evaluation indicator is solved. The smaller T value corresponds to the optimal optional parameters of wavelet basis function and decomposition levels of the corresponding wavelet families.
- 4) Analyze and compare the optimal denoising effects of different wavelet families, and obtain the optimal wavelet, which is most suitable for terahertz spectrum denoising.
3. Results and discussions
3.1 Analysis of compound evaluation indicators
Figure 2 (a), (b) shows the original terahertz time-domain spectra of the wheat samples and the corresponding frequency-domain spectra obtained using the fast Fourier transform. The effective frequency range is 0.1-1.6 THz. The decreasing signal-to-noise ratio of the wheat spectra at both higher and lower frequencies is observed, which can be attributed to the restricted dynamic range of the measurement system. And there are no discernible absorption peaks in the spectra, regression model is used to explore the relation between the spectra and the properties of wheat. The high-quality spectra are essential for the models. Denoising of the original terahertz signal is required due to system and environmental interference. According to the above-described algorithms, it can obtain the compound evaluation indicators T value of wheat sample’s terahertz spectrum signal on symlets wavelet family at 1∼8 decomposition levels. The results are shown in Table 1.
Fig. 2. Denoising effects comparison of different wavelet basis functions with same decomposition levels: (a) is the time domain of THz spectra and (b) is the frequency domain of THz spectra.
Table 1. Compound evaluation indicator of symlets wavelet family with multi-level decomposition
It can be observed from Table 1 that symlets 2∼8 wavelet basis functions are used to decompose the terahertz spectrum signals. When the decomposition level is 1, the obtained compound evaluation indicator T value is the largest, and compound evaluation indicators are all larger than 0.6. From 1-level decomposition to 2-level decomposition, the compound indicator decreases significantly; while its decreasing amplitude is small from 2-level decomposition to 8-level decomposition, basically in the range of [0.1, 0.3]. Apart from that symlets 2 and symlets 3 wavelet basis functions’ 7-level to 8-level decompositions are weak declines where the T value decreases initially and then increases with the increment in decomposition levels. When symlets 2 and symlets 5 wavelet basis functions are used to decompose the terahertz spectrum signal and the decomposition level is 3, the obtained compound evaluation indicator T value is the smallest. When other symlets wavelet basis functions are used for 4-level decomposition, the T value is the smallest. By comparing the minimum values of each symlets wavelet basis function’s compound evaluation indicator (bold numerical value in Table 1), it can be concluded that when symlets 8 wavelet basis function is used for 4-level decomposition, the T value is the smallest (T = 0.1646). It is the optimal wavelet basis function and decomposition level in the symlets family.
To verify whether the symlets 8 wavelet basis function and 4-level decomposition are optimal, different wavelet basis functions and decomposition levels are selected for comparing denoising effects.
Figure 2 compares the reconstruction denoising effects using different wavelet basis functions at a certain decomposition level. The spectra in Fig. 2 (a) and (b) are as follows: the time domain and frequency domain of the wheat sample’s terahertz spectrum original signal; the time domain and frequency domain of symlets 2 wavelet basis function with 4-level decomposition after denoising (T = 0.2333); the time domain and frequency domain of symlets 5 wavelet basis function with 4-level decomposition after denoising (T = 0.1772); and the time domain and frequency domain of symlets 8 wavelet basis function with 4-level decomposition after denoising (T = 0.1646). It can be observed that the original time-domain signal’s noise is rather severe, and the frequency domain waveform obtained by Fourier transform is also disordered. After the denoising of symlets 2 wavelet basis function with 4-level decomposition and reconstruction, compared with the original signal’s frequency domain, the denoising effect is more obvious, but the fluctuation of the frequency domain is still large. For the denoising effects of symlets 5 and symlets 8 wavelet functions with 4-level decomposition and reconstruction, the waveforms of the time domain and frequency domain will become smoother with the decrease of T value. However, within the frequency domain 0.8∼1.5THz, the symlets 8 wavelet basis function reserves part of frequency information compared with the symlets 5 wavelet basis function. In the time domain, the symlets 5 wavelet basis function’s part waveforms around 15 ps is rather sharp, so the denoising effect of the symlets 8 wavelet basis function is better. It indicates that when the decomposition level is fixed, a smaller compound evaluation indicator T value of wavelet basis function is indicative of better denoising.
Figure 3 compares the denoising effects at different decomposition levels when the wavelet basis functions are all symlets 8. The spectra in Fig. 3 (a) and (b) are as follows: the time domain and frequency domain of the wheat sample’s terahertz spectrum original signal; the time domain and frequency domain of symlets 8 wavelet basis function with 2-level decomposition after denoising (T = 0.2281); the time domain and frequency domain of symlets 8 wavelet basis function with 6-level decomposition after denoising (T = 0.1851); and the time domain and frequency domain of symlets 8 wavelet basis function with 4-level decomposition after denoising (T = 0.1646). As shown in Fig. 3, the denoising effect of symlets 8 wavelet basis function after 2-level decomposition and reconstruction is not perfect; the fluctuation of the waveform in time domain is rather severe; the waveform of the frequency domain is similar to the waveform of the original signal, and symlets 8 wavelet basis function’s time-domain wavelet is too smooth after 6-level decomposition and reconstruction and causes the loss of important information. The waveform in the frequency domain also has this kind of problem since it only has 2 peak values within the frequency range of 0.2∼0.8THz. The lost information leads to a poorly denoising effect. After the denoising of symlets 8 wavelet basis function with 4-level decomposition and reconstruction, it obtains a smooth time-domain waveform while retaining the initial signal information. When wavelet basis function is certain, a smaller compound evaluation indicator of different decomposition levels corresponds to a better denoising effect. Therefore, by calculating the compound evaluation indicator of different wavelet basis functions with different decomposition levels, it can be concluded that the minimum T value is the optimal wavelet basis and decomposition level.
Fig. 3. Denoising effect comparison of same wavelet basis function at different decomposition levels: (a) is the time domain of THz spectra and (b) is the frequency domain of THz spectra.
3.2 Analysis of optimal wavelet selection
In order to find out the optimal wavelet basis function and decomposition level of various families, four wavelet families, i.e., symlets, daubechies, fejer-korovkin, and coiflets, are selected in this experiment to calculate the compound evaluation indicator T values.
Table 2 lists the compound evaluation indicator T values of spectrum signal of daubechies 1∼10 wavelet basis functions from 1∼8 level decomposition and reconstruction. It can be observed from Table 2 that the minimum T value is 0.1746, suggesting that the corresponding daubechies 9 wavelet basis function with 3-level decomposition is the optimal wavelet.
Table 2. Compound evaluation indicator of daubechies wavelet family with multi-level decomposition
Table 3 lists the compound evaluation indicator T values of spectrum signal of fejer-korovkin 4∼22 wavelet basis functions from 1∼8 level decomposition and reconstruction. As shown in Table 3, the minimum T value is 0.1750, which indicates that the corresponding fejer-korovkin 22 wavelet basis function with 3-level decomposition is the optimal wavelet.
Table 3. Compound evaluation indicator of fejer-korovkin wavelet family with multi-level decomposition
Table 4 lists the compound evaluation indicator T values of spectrum signal of coiflets 1∼5 wavelet basis functions from 1∼8 level decomposition and reconstruction. It can be observed from Table 4 that the minimum T value is 0.1704, which means that the corresponding coiflets wavelet basis function with 4-level decomposition is the optimal wavelet. Through comparison, the change rules of compound evaluation indicator T values of wavelet basis with multilevel decomposition in daubechies, fejer-korovkin, and coiflets wavelet family are similar to the symlets wavelet family in Table 1. All of these show progressive decreasing trend at first and then progressive increasing trend.
Table 4. Compound evaluation indicator of coiflets wavelet family with multilevel decompositions
Given the optimal wavelets of various families in the above tables, the optimal wavelet’s denoising effects of each family are shown in Fig. 4.
Fig. 4. Denoising effect comparison of different families of optimal wavelet basis functions with decomposition levels: (a) the time domain of THz spectra and (b) the frequency domain of THz spectra.
The spectra in Fig. 4 is calculated as follows: the wheat sample’s THz original signal; the terahertz spectrum of fejer-korovkin 22 wavelet basis function with 3-level decomposition after denoising (T = 0.1750); the THz spectrum of daubechies 9 wavelet basis function with 3-level decomposition after denoising (T = 0.1746); the THz spectrum of coiflets 3 wavelet basis function with 4-level decomposition after denoising (T = 0.1704); the THz spectrum of symlets 8 wavelet basis function with 4-level decomposition after denoising (T = 0.1646); and the wheat sample’s THz reference signal. To further verify the denoising effect of the above wavelets, the wheat sample and THz-TDS are used to obtain a reference signal in the Nitrogen environment. From Fig. 4, compared with the fejer-korovkin 22’s 3-level, daubechies 9’s 3-level and coiflets 3’s 4-level, the symlets 8’s 4-level decomposition and reconstruction are significantly different, which show the performance of denoising is better. Meanwhile, the waveforms of symlets 8’s 4-level are closer to reference signal. The comparisons showed that the denoising effect of 4-level decomposition through symlets 8 wavelet basis function is the best, and the denoising effect is efficient.
The symlets 8 wavelet basis functions with 4-level decomposition and reconstruction are selected by the composite evaluation indicator T values to denoise the terahertz spectrum of wheat samples. The selection of wavelet basis function is usually based on the symmetry, compact support, regularity, vanishing moment and other factors combine with the actual scene. As a common wavelet basis function, daubechies wavelet has good performance in local time-frequency. While symlets wavelet is an improvement of daubechies wavelet, the performance is almost the same in other aspects. But symlets wavelet has better symmetry characteristics, which can reduce the distortion of the signal. Fejer-korovkin wavelet and coiflets wavelet are more symmetric than daubechies wavelet, but the weak regularity of Fejer-korovkin wavelet results in poor smoothness, while the long length of coiflets wavelet compact support set results in poor local time-frequency. In Fig. 5, coiflets 3 and symlets 8 wavelet functions with 4-level have similar waveforms in the time domain and frequency domain. Thus, in Fig. 4, the denoising effects of coiflets 3 and symlets 8 wavelet functions with 4-level are relatively close, the same as daubechies 9 and fejer-korovkin 22 wavelet functions with 3-level. In Fig. 5, the time domain diagram of coiflets 3 and symlets 8 wavelet functions is similar to the time domain spectrum of wheat samples, which can better fit the spectral data. The selection of wavelet basis and decomposition level according to T value accords with the selection principle of wavelet basis.
Fig. 5. Waveform of wavelet basis: (a) and (b) are the time and frequency domains of coiflets 3 and symlets 8 wavelet functions with 4-level; (c) and (d) are the time and frequency domains of daubechies 9 and fejer-korovkin 22 wavelet functions with 3-level.
To verify the validity of the experiments in this paper, we add different levels of noise to the original terahertz spectra and use several wavelet basis functions with optimal T values for denoising comparisons. The original terahertz signals with signal-to-noise ratios of 1 dB, 5 dB and 10 dB and the terahertz signals after denoising using several wavelet basis functions are shown in Fig. 6, while the terahertz signals in the frequency domain with signal-to-noise ratios of 40 dB, 45 dB and 50 dB and the terahertz signals in the frequency domain after denoising using several wavelet basis functions are shown in Fig. 7. In both time and frequency domains, the coiflets 3's 4-level and symlets 8's 4-level have similar denoising effects compared to the fejer-korovkin 22's 3-level and daubechies 9's 3-level, with symlets 8's 4-level showing the best denoising effect.
Fig. 6. Denoising effect comparison of different families of optimal wavelet basis functions with decomposition levels bases on terahertz time-domain signals with different signal-to-noise ratios: (a) SNR = 1 dB (b) SNR = 5 dB (c) SNR = 10 dB.
Fig. 7. Denoising effect comparison of different families of optimal wavelet basis functions with decomposition levels bases on terahertz frequency domain signals with different signal-to-noise ratios: (a) SNR = 40 dB (b) SNR = 45 dB (c) SNR = 50 dB.
Different from the traditional denoising evaluation index, this paper adopts the composite evaluation index T-value to evaluate the denoising effect of several wavelet basis functions, and selects the optimal wavelet basis function as well as the decomposition level. The experimental results show that the symlets 8's 4-level wavelet basis function selected according to the T value can realize the optimal denoising effect.
4. Conclusion
After processing of the wheat sample’s terahertz spectrum signals, the compound evaluation indicator T values of symlets wavelet family can be calculated. It is discovered that T values have a trend of increasing initially and then decreasing with the increment of decomposition levels. Through time and frequency domain, it has been verified that the smaller the T values in symlets wavelet family is, the better the denoising effect shows. By performing denoising processing for terahertz spectrum signals of daubechies, fejer-korovkin, coiflets and symlets wavelet family, the optimal wavelet basis and decomposition level of different families are selected based on the minimum T values. It is found out that fejer-korovkin and daubechies family of wavelet basis functions are not suitable for the denoising processing of terahertz spectrum signals. The denoising effects of coiflets and symlets family of wavelet basis functions exhibits better performance, and the denoising effect of symlets 8 wavelet basis function with 4-level decomposition and reconstruction is the most suitable. The experiments validate that terahertz spectrum signal preprocessing removes noise and extract effective signal information.
In this paper, the optimal wavelet basis function and decomposition level are selected based on compound evaluation indicators, which solves the problem of optimal wavelet selection and provides an algorithm foundation for the application of wavelet denoising in the terahertz technology fields.
Funding
the Cultivation Programme for Young Backbone Teachers in Henan University of Technology; the Open Fund Project of Key Laboratory of Grain Information Processing & Control, Ministry of Education, Henan University of Technology (KFJJ2020103, KFJJ2021102); the Program for Science & Technology Innovation Talents in Universities of Henan Province (22HASTIT017, 23HASTIT024); Key Science and Technology Program of Henan Province (222102110246, 222103810072); The Innovative Funds Plan of Henan University of Technology (2021ZKCJ04); Natural Science Foundation of Henan Province (222300420040); National Natural Science Foundation of China (61975053, 62271191).
Acknowledgment
We are deeply grateful to the reviewers and editors for their invaluable and constructive suggestions and comments that greatly improved the version of this article.
Disclosures
There are no conflicts of interest in this article.
Data Availability
Data underlying the results presented in this paper are not publicly available at this time but may be obtained from the authors upon reasonable request.
References
1. M. Tonouchi, “Cutting-edge terahertz technology,” Nat. Photonics 1(2), 97–105 (2007). [CrossRef]
2. L. Afsah-Hejri, E. Akbari, A. Toudeshki, et al., “Terahertz spectroscopy and imaging: A review on agricultural applications,” Computers and Electronics in Agriculture 177, 105628 (2020). [CrossRef]
3. B. Ferguson and D. Abbott, “Wavelet de-noising of optical terahertz pulse imaging data,” Fluct. Noise Lett. 01(02), L65–L69 (2001). [CrossRef]
4. T. Fukasawa, T. Sato, J. Watanabe, et al., “Relation between dielectric and low-frequency Raman spectra of hydrogen-bond liquids,” Phys. Rev. Lett. 95(19), 197802 (2005). [CrossRef]
5. R. T. Hall and J. M. Dowling, “Pure Rotational Spectrum of Water Vapor,” J. Chem. Phys. 47(7), 2454–2461 (1967). [CrossRef]
6. Y. H. Yang, A. Shutler, D. Grischkowsky, et al., “Measurement of the transmission of the atmosphere from 0.2 to 2 THz,” Opt. Express 19(9), 8830–8838 (2011). [CrossRef]
7. Y. D. Liu, X. Y. Du, B. Li, et al., “Detection of Purple Rice Adulteration by Terahertz Time Domain Spectroscopy,” Spectroscopy and Spectral Analysis 40(8), 2382–2387 (2020). [CrossRef]
8. X. Wei, W. Zheng, S. Zhu, et al., “Application of terahertz spectrum and interval partial least squares method in the identification of genetically modified soybeans,” Spectrochim. Acta, Part A 238, 118453 (2020). [CrossRef]
9. P. P. Liu, X. Zhang, B. Pan, et al., “Classification of Sand Grains by Terahertz Time-Domain Spectroscopy and Chemometrics,” Int. J. Environ. Res. 13(1), 143–160 (2019). [CrossRef]
10. A. G. Ouyang, Y. L. Zheng, B. Li, et al., “Study on the Detection Method of Alum Content in Sweet Potato Starch by Terahertz Spectroscopy,” Spectroscopy and Spectral Analysis 40(3), 727–732 (2020). [CrossRef]
11. M. K. Babouri, N. Ouelaa, T. Kebabsa, et al., “Diagnosis of mechanical defects using a hybrid method based on complete ensemble empirical mode decomposition with adaptive noise (CEEMDAN) and optimized wavelet multi-resolution analysis (OWMRA): experimental study,” Int. J. Adv. Manuf. Technol. 112(9-10), 2657–2681 (2021). [CrossRef]
12. S. G. Mallat, “A theory for multiresolution signal decomposition: the wavelet representation,” IEEE Trans. Pattern Anal. Machine Intell. 11(7), 674–693 (1989). [CrossRef]
13. E. Kang, W. Chang, J. Yoo, et al., “Deep Convolutional Framelet Denosing for Low-Dose CT via Wavelet Residual Network,” IEEE Trans. Med. Imaging 37(6), 1358–1369 (2018). [CrossRef]
14. X. B. Xu, M. Luo, Z. Tan, et al., “Echo signal extraction method of laser radar based on improved singular value decomposition and wavelet threshold denoising,” Infrared Phys. Technol. 92, 327–335 (2018). [CrossRef]
15. Z. Y. Bao, G. Zhang, B. Xiong, et al., “New image denoising algorithm using monogenic wavelet transform and improved deep convolutional neural network,” Multimed. Tools Appl. 79(11-12), 7401–7412 (2020). [CrossRef]
16. C. J. Shi, J. Zhu, M. Xu, et al., “An Approach of Spectra Standardization and Qualitative Identification for Biomedical Materials Based on Terahertz Spectroscopy,” Sci. Programming 2020(5), 1–8 (2020). [CrossRef]
17. C. M. Du, X. Zhang, Z. Zhang, et al., “Quantitative analysis of ternary isomer mixtures of saccharide by terahertz time domain spectroscopy combined with chemometrics,” Vib. Spectrosc. 100, 64–70 (2019). [CrossRef]
18. F. F. Qu, L. Lin, Y. He, et al., “Terahertz Multivariate Spectral Analysis and Molecular Dynamics Simulations of Three Pyrethroid Pesticides,” J. Infrared, Millimeter, Terahertz Waves 39(11), 1148–1161 (2018). [CrossRef]
19. Y. Y. Al-Aboosi and A. Z. Sha’ameri, “Improved signal de-noising in underwater acoustic noise using S-transform: A performance evaluation and comparison with the wavelet transform,” Journal of Ocean Engineering and Science 2(3), 172–185 (2017). [CrossRef]
20. Y. D. Zhang, Z. Ji, X. Tan, et al., “Noise Reduction of the Electrocardiography Signal and Thoracic Impedance Differential Signal Based on Adaptive EEMD and Wavelet Thresholding,” J. Med. Imaging Health Inform. 8(1), 140–144 (2018). [CrossRef]
21. S. Abasi, S. Minaei, B. Jamshidi, et al., “Rapid measurement of apple quality parameters using wavelet de-noising transform with Vis/NIR analysis,” Sci. Hortic. 252, 7–13 (2019). [CrossRef]
22. Z. H. Deng, J. Wang, X. Liang, et al., “Function Extension Based Real-Time Wavelet De-Noising Method for Projectile Attitude Measurement,” Sensors 20(1), 200 (2020). [CrossRef]
23. B. Ferguson and D. Abbott, “De-noising techniques for terahertz responses of biological samples,” Microelectron. J. 32(12), 943–953 (2001). [CrossRef]
24. Z. Jianjun, Z. Zhetao, K. Cuilin, et al., “A Reliable Evaluation Indicator of Wavelet De-noising,” Geomatics and Information Science of Wuhan University 40(5), 688–694 (2015).
25. Z. Wu, J. Luo, K. Rao, et al., “Estimation of wheat kernel moisture content based on hyperspectral reflectance and satellite multispectral imagery,” International Journal of Applied Earth Observation and Geoinformation 126, 103597 (2024). [CrossRef]
26. H. Y. Ge, Y. Jiang, Z. Xu, et al., “Identification of wheat quality using THz spectrum,” Opt. Express 22(10), 12533–12544 (2014). [CrossRef]
27. G. Garg, “A signal invariant wavelet function selection algorithm,” Med. Biol. Eng. Comput. 54(4), 629–642 (2016). [CrossRef]
28. L. Ebadi and H. Z. M. Shafri, “A stable and accurate wavelet-based method for noise reduction from hyperspectral vegetation spectrum,” Earth. Sci. Inform. 8(2), 411–425 (2015). [CrossRef]
29. J. P. Ferreira, E. R. Schmildt, O. Schmildt, et al., “Comparison of methods for classification of the coefficient of variation in papaya,” Rev. Ceres 63(2), 138–144 (2016). [CrossRef]
