Open Access
Add to BibSonomy
Abstract
A dual-model hybrid pattern recognition based on a fiber optic line-based sensor with a large amount of data is proposed. The vibration signals are converted to gray-level images to reduce the memory requirement. The ResNet18 model for classification is used. To reduce the false positive rate, the over-zero rate and short-time energy are extracted from the intrusion signal, and a support vector machine (SVM) is used. Finally, a discriminator is constructed to determine the types of events by combining the two models trained on the validation dataset. The results demonstrate the excellent average recognition accuracy of this method, which achieves the 97.1% for six events.
© 2022 Optica Publishing Group under the terms of the Optica Open Access Publishing Agreement
1. Introduction
It is well-known that the distributed Fiber Optic Sensing System (FOSS) has many merits, such as anti-electromagnetic interference, low-cost, and long-distance sensing. Thus, it has been widely applied in oil and gas pipeline detection, high-speed track intrusion detection, and perimeter security [1–4]. The technologies in distributed FOSS can be divided into two classes: the FOSS based on continuous laser, and the other based on pulsed laser. Due to the features of simplicity, long sensing distance and low cost, the Sagnac-based fiber optic sensing technology was widely used in intrusion detection [5,6].
Pattern recognition methods based on FOSS have been proposed in quite a few papers. Wang et al. used the wavelet decomposition to decompose the vibration signal to construct the Shannon entropy feature vector and used a radial basis neural network for classification, which achieved 85% accuracy for vibration signal classification [7]. Ma et al. extracted the Mel Frequency Cepstrum Coefficient of vibration signals as features and used Gaussian hidden Markov model for intrusion signal detection, achieving the 4% false alarm rate and 3% missed detection rate, respectively [8]. Wang et al. used an echo state network to identify vibration events of sensing fiber hanged on the fence and buried underground, through which the average recognition rate is 98.75% and 100%, respectively [9]. Bao et al. used the variational modal decomposition to decompose the vibration signal, and then extracted the spectral entropy, energy ratio, and kurtosis of the eigenmode function as features, finally resulting in recognition accuracy ∼ 98% by means of Support Vector Machine (SVM) [10]. Mi et al. proposed multilayer perceptron neural networks to detect substation fences and obtained an accuracy of 97.6% [11]. Although the methods proposed in the above papers can identify intrusion events well, the sampling frequency of distributed Sagnac FOSS used are relatively low (2-50 KHz) [7–9,11], which does not involve the cases with high sampling frequency (>100 KHz). Furthermore, when the model recognition errors happen, the above methods cannot correct them [10]. Liu et al. used SVM and radial basis function neural network in the dual Mach-Zehnder interferometer system to separately identify vibration signals [12]. Since the two models in the system can provide mutual correction for each other, it achieved average recognition accuracy of 97% for five vibration signals recognition. However, since the two models used the same features, they may fail when using the same features and models.
As one coin has two sides, the high sampling frequency of the distributed Sagnac FOSS is the good restoration of the vibration signal, while it will bring a large amount of sampling data and therefore require more memory to store the data. The disadvantage will make it difficult to analyze using deep learning. To solve this conflict, a novel dual-model hybrid pattern recognition method based on fiber optic line-based sensor is proposed and investigated for vibration intrusion signal recognition.
2. Classification principle
Figure 1 shows the structure of the proposed dual-model hybrid pattern recognition system. The vibration signals generated from the FOSS are collected to the dataset after denoising, which is divided into a training set, a validation set, and a test set.
The processes of data preprocessing include feature extraction and converting the signals to grayscale images. After preprocessing, the extracted features and the converted images use the inputs of the SVM and ResNet18, respectively. After both models are trained using the training set, the validation dataset is used for validating the trained model. The final discriminator is constructed according to the recognition accuracy of each vibration signal on the validation process.
2.1 Simplified fiber optic line-based sensor
To increase the sensitivity and the sensing distance of distributed Sagnac FOSS, a simplified fiber optic line-based sensor is used [13]. The structure of sensing system is shown in Fig. 2. The system uses a Super Light Emitting diode (SLD) (the central wavelength is 1550 nm, the bandwidth is 3 nm and the power is 3 mW, CW) as the system's light source. First, the SLD laser is connected to the isolater to eliminate the optical reflections. Then, the beam from the isolater is injected into one symmetric 3 × 3 fiber coupler (C1), one time delay fiber (TDF), and one symmetric 2 × 2 fiber coupler (C2). The Faraday Rotator Mirror (FRM) is used to form line-based configuration and compensate the polarization induced signal fading. Finally, the fiber couplers (C1, C2), and photodetectors (PD1, PD2) are used to receive the returned optical signals [13].
2.2 Data preprocessing
The data preprocessing includes feature vectors for SVM and image converting for ResNet18, respectively. The process of feature extraction is illustrated in Fig. 3. The vibration signal first is divided into frames, and then the frames are windowed to reduce the energy leakage. Finally, the short-time energy (SE) and the zero-crossing rate (ZCR) of each frame are extracted after windowing, which of them are computed as:
The extracted short-time energy [E1,…,En] and the over-zero rate [Z1,…,Zn] of the vibration signal are concatenated to a feature vector [E1,…,En,Z1,…,Zn]. The dimensionality is much smaller than the original signals by this way.
The advantages of using the gray image to represent the signal are that: 1) The gray image can clearly reflect the vibration in the vibration signal; 2) Compared with the “Gramian Angular Field” (GAF) map [14] and recursive map [15], the gray image converted fast for it only requires considering the dimensionality of the vibration signal. Both the GAF map and the recursive map are unsuitable to represent the signal with high dimensionality.
2.3 Recognition model
To correct the recognition results of ResNet18 and improve the recognition accuracy of vibration signals, the vibration signals are recognized for two models, respectively. And the recognition results of the two models are sent to the discriminator to output the final recognition results. Figure 4 illustrates the structure of the recognition model.
The signal images are fed into the ResNet18 model, which is based on the residual module. The residual model can well alleviate the problem of gradient disappearance and gradient explosion in the deep convolutional neural networks. Although the ResNet18 network can discriminate vibration grayscale images well, it cannot correct the recognition results when they are wrong. Therefore, it is necessary to introduce an additional model to correct the recognition results of the ResNet18 network by comparison. For the number of sampling points is large in the Sagnac FOSS with a high sampling rate, using 1D CNN may be inefficient [16]. So the low-dimensional feature vector is first extracted from the vibration signal and then classified using an SVM [17].
According to their prediction results, the discriminator can select the best model from two models, i.e., Resnete18 model and the SVM model.
The structure of the discriminator is illustrated in Fig. 5,
- 1) Two trained models are used to recognize the validation set data of vibration signals, respectively.
- 2) The recognition accuracy of the two models for the validation set are to form a matrix of [2,m], where m denotes the number of vibration types.
- 3) The discriminator first judges whether the prediction result of the ResNet18 model (x1_i) is the same as that of the SVM (X2_k), with i and k indicating the prediction results. If the two models have the same prediction results, the prediction result of the main model is selected as the final prediction result.
- 4) If the two models do not have the same prediction results, the recognition accuracy of the validation set corresponding to the vibration type of the ResNet18 model (Val_x1_i), and that of the SVM (Val_x2_k) are obtained from the constructed recognition accuracy matrix, respectively. Then, the Val_x1_i subtracts the Val_x2_k. Through the comparison of multiple groups of experiments, it is concluded that when the subtraction result < 0.05, the two recognition models can be considered as the same, then the main model is selected as the final output; when the subtraction result > 0.05, the magnitude of Val_x1_i and Val_x2_k is compared. If Val_x1_i > Val_x2_k, the prediction result of the main model is output. Otherwise, the second model is selected.
3. Experimental results
In the laboratory, we simulated different vibration events and collected their signals, such as blowing, shaking, shearing, knocking, branches flapping, walking, and other situations. The collection frequency of the system is 500 KHz. Each type of vibration event has 12 G of signal data. Since the vibration signal is collected continuously, we need to segment the long signal of each type of event into a short clip with a 1 s length. Each type of clip has 500000 data points. Table 1 lists the amount of vibration signal samples of each type of vibration event.
Table 1. Vibration data sheet
Figure 6 shows the samples of six types of vibration events. The vibration signal data set is partitioned into a training set, validation set, and test set according to the ratio of 7:2:1. The training set has 1260 samples, the validation set has 360 samples, and the test set has 180 samples. The vibration types of blowing, shaking, shearing, knocking, flapping branches, and walking are named 1, 2, 3, 4, 5, and 6, respectively.
Fig. 6. Vibration waveform: (a): blowing (b): shaking (c): shearing (d): knocking (e): the flapping of branches (f): walking
3.1 Feature extraction and grayscale map conversion
The length of each frame is set to 10-30 ms to make the signal be considered short-time smooth, so the frame length is chosen to be 5000 according to the vibration signal length. Considering the frameshift is usually 20%-30% of the frame length, the frameshift is chosen to be 1000. After the vibration signal is divided into frames, the Hamming windowed function is chosen to add a window to the signal. A vibration signal is decomposed into 496 frames after frame-splitting and windowing, and the short-time energy and zero rates of each frame are extracted separately. The feature dimension of a vibration signal becomes [1,992] after fusing the short-time energy and short-time zero rates. After feature extraction from the split frames, the dimension of the vibration signal is significantly reduced. Figure 7 illustrates the features of the average statistics of the short-time energy and zero rates of the six types of vibration signals.
Fig. 7. Average value: (a): ZCR (b): SE. (A: blowing, B: shaking, C: shearing, D: knocking, E: the flapping of branches, F: walking)
From Fig. 7(a), we can see that the over-zero rate of the vibration signal of blowing is the highest, the over-zero rate of the vibration signal of walking is the lowest, and the over-zero rates of shaking, shearing, knocking, and the flapping of branches are not very different. To well distinguish the characteristics of the vibration signal, we add the short-time energy characteristics. From Fig. 7(b), it can be seen that the short-time energy of shaking is the highest, the short-time energy of the flapping of branches is the lowest, and the short-time energy of shearing, knocking, and the flapping of branches differ greatly. Therefore, the feature vector formed can well distinguish the different vibration signals by combining the two features.
To convert the vibration signals to a gray image, we transform the one-dimensional vibration signal to a two-dimensional matrix according to the amplitude of the time and behavior [14,18]. Figure 8 shows the grayscale image converted from the one-dimensional vibration signal. We can see that when a vibration occurs, lines also appear in the grayscale image. One line in the image corresponds to one vibration. Figure 8(c) shows 2 vibrations in the signal: the higher the energy of the vibration is, the lighter of the line is. After converting to an image, the storage size of the image is significantly reduced. For example, the size of the original 1D vibration signal is 12.6 MB, although its corresponding images are only 149 KB. The image gray level is 0-255. Although the method of compressing different formats is the same, the number of data bits stored differs for different formats of images. We chose the Portable Network Graphic (PNG) format, which is lossless compression.
Fig. 8. Grayscale: (a): blowing (b): shake (c): shearing (d): knocking (e): the flapping of branches (f): walking
3.2 Recognition results
The ResNet18 network and SVM are trained separately in the dual-model hybrid recognition approach. For the ResNet18 network, we use the ‘SGD’ as the optimizer for training ResNet18 network, set the learning rate to 0.001, set the batch_size to 300, use the ‘sparse_categorical_crossentropy’ as loss function, and set the epochs to 500 times.
For the SVM, we use the RBF as the kernel function, set the penalty factor to 12, and set the degree to 3. The grid search method is used to set the hyperparameters of the SVM.
After the two models are trained, we use the validation dataset to validate the accuracy of the two models for selecting the optimal model of the discriminator. Figure 9 illustrates the confusion matrix of the two models on the validation set.
From Fig. 9(a), we can see that the recognition accuracy of ResNet18 network model for blowing, knocking, and walking is 100%, for shaking is 94%, for shearing is 99%, for the flapping of branches is 73%. From Fig. 9(b), the recognition accuracy of SVM model for blowing, the flapping of branches and walking are 100%, for shaking is 97%, and for knocking and shearing is 84%. Figure 9 demonstrates that the recognition accuracy of models differs for different vibration events. Since the accuracy of a single model is relatively low, the false alarm will be high if we only use a single model for vibration signal recognition.
After a discriminator is constructed using the trained ResNet18 network and SVM model on the validation set, we will use it to recognize the signals in the testing dataset. Figure 10 shows the confusion matrix derived from the ResNet18 network model and SVM on the testing.
From Fig. 10(a), we can see that when using the ResNet18 model for vibration signal recognition, the recognition accuracy only achieves 80% for the flapping of branches, and the average recognition accuracy is 96.2%. From Fig. 10(b), the recognition accuracy for shear vibration is only 70% using the SVM, and the average recognition accuracy is 91.6%. Figure 11 shows the confusion matrix of vibration signal recognition on the testing set, from which the accuracy of the proposed method for the flapping of branches and knocking is increased to 93%, and for shearing is increased to 97%. The average accuracy is 97.1%. The increment demonstrates that the proposed method can effectively improve the recognition accuracy of the model for each vibration signal.
Figure 12 shows the recognition accuracy and recognition time of the three models for the test set. We can see that the proposed method has the highest recognition accuracy of vibration signals. Although the recognition time of this method is longer than that of SVM and the Resent18 method, the recognition time is only 0.589 s, which still satisfies the requirement of real-time. From Table 2, we can see that the Recall, Precision, and F1 are improved using the proposed method for shearing and percussion vibration recognition.
Table 2. Model performance parameters
4. Conclusions
A novel dual-model method hybrid recognition method of distributed fiber optic line-based sensor is proposed in this paper. The memory requirement for storing the signal data is significantly reduced by converting the signal to a grayscale image. To correct the recognition error of a single model, we used a dual-model hybrid recognition system, employing ResNet18 network and SVM models to construct a discriminator based on the recognition accuracy of the validation set. Extensive experiments show that recognition accuracy of the proposed method can achieve 100%, 100%, 97%, 93%, 93%, and 100% for blowing, shaking, shearing, knocking, walking, and the flapping of branches, respectively. The average recognition accuracy of intrusion signals is increased to 97.1%, which effectively improves the recognition accuracy of the model and reduces the false positives of intrusion events. Although the proposed method increases the recognition time of vibration signals, the recognition time of vibration signals still meets the requirement of real-time. Belonging to continuous laser type, Sagnac FOSS’s cost is lower than that of optical distributed optical fiber sensor based on pulsed laser type. The sampling frequency of Sagnac FOSS in existing papers is almost concentrated in a few KHz. Using a sampling frequency of 500 KHz, it can be sampled more abundant vibration signals. The FOSS with a higher sampling frequency (>100 KHz) will generate a large amount of data and occupy a large amount of memory. It is difficult to conduct deep learning for pattern recognition. The method proposed in this paper has a great potential for analyzing vibration signals of distributed fiber optic line-based sensor with a large amount of data.
Disclosures
The authors declare no conflicts of interest.
Data availability
Data underlying the results presented in this paper are not publicly available at this time but may be obtained from the authors upon reasonable request.
References
1. S. Li, R. Peng, and Z. Liu, “A surveillance system for urban buried pipeline subject to third-party threats based on fiber optic sensing and convolutional neural network,” Struct Health Monit 20(4), 1704–1715 (2021). [CrossRef]
2. Z. Li, J. Zhang, M. Wang, Y. Zhong, and F. Peng, “Fiber distributed acoustic sensing using convolutional long short-term memory network: a field test on high-speed railway intrusion detection,” Opt. Express 28(3), 2925–2938 (2020). [CrossRef]
3. Z. Wang, the flapping of branchesqin Lou, X. Wang, S. Liang, X. Sheng, “Multi-branch long short-time memory convolution neural network for event identification in fiber-optic distributed disturbance sensor based on φ-OTDR,” Infrared Phys Techn 109, 103414 (2020). [CrossRef]
4. X. Huang, H. Zhang, K. Liu, and T. Liu, “Fully modelling based intrusion discrimination in optical fiber perimeter security system,” Opt. Fiber Tech 45, 64–70 (2018). [CrossRef]
5. G. Allwood, G. Wild, and S. Hinckley, “Optical Fiber sensors in physical intrusion detection systems: a review,” IEEE Sens. J. 16(14), 5497–5509 (2016). [CrossRef]
6. J.C. Juarez, E.W. Maier, Kyoo Nam Choi, and H.F. Taylor, “Distributed fiber-optic intrusion sensor system,” J. Lightwave Technol. 23(6), 2081–2087 (2005). [CrossRef]
7. B. Wang, S. Pi, Q. Sun, and B. Jia, “Improved wavelet packet classification algorithm for vibrational intrusions in distributed fiber-optic monitoring systems,” Opt. Eng. 54(5), 0551041 (2015). [CrossRef]
8. Z. Ma, L. Yao, and Z. Jie, “An optic-fiber fence intrusion recognition system using mixture Gaussian hidden Markov models,” IEICE Electron. Express 14(5), 20170023 (2017). [CrossRef]
9. N. Wang, N. Fang, and L. Wang, “Intrusion recognition method based on echo state network for optical fiber perimeter security systems,” Opt. Commun. 451, 301–306 (2019). [CrossRef]
10. J. Bao, J. Mo, L. Xu, Y. Liu, and X. Lv, “VMD-based vibrating fiber system intrusion signal recognition,” Optik 205, 163753 (2020). [CrossRef]
11. Q. Mi, H. Yu, Q. Xiao, and H. Wu, “Intrusion behavior classification method applied in a perimeter security monitoring system,” Opt. Express 29(6), 8592–8605 (2021). [CrossRef]
12. K. Liu, Z. Sun, J. Jiang, P. Ma, and T. Liu, “A Combined Events Recognition Scheme Using Hybrid Features in Distributed Optical Fiber Vibration Sensing System,” IEEE Access 7, 105609–105616 (2019). [CrossRef]
13. W. Yuan, B. Pang, J. Bo, and X. Qian, “Fiber Optic Line-Based Sensor Employing Time Delay Estimation for Disturbance Detection and Location,” J. Lightwave Technol. 32(5), 1032–1037 (2014). [CrossRef]
14. C. Lyu, Z. Huo, X. Cheng, J. Jiang, A. Alimasi, and H. Liu, “Distributed optical fiber sensing intrusion pattern recognition based on GAF and CNN,” J. Lightwave Technol. 38(15), 4174–4182 (2020). [CrossRef]
15. L. Yu, G. Chen, J. Jiang, and Z. Huo, “Open-set events identification based on deep metric-learning for DMZI perimeter system,” IEEE Sensors J. 20(22), 13620–13628 (2020). [CrossRef]
16. H. Wu, J. Chen, X. Liu, Y. Xiao, and Y. Rao, “One-Dimensional CNN based intelligent recognition of vibrations in pipeline monitoring with DAS,” J. Lightwave Technol. 37(17), 4359–4366 (2019). [CrossRef]
17. B. Marie, D. Han, B. An, X. Lu, and Q. Tian, “Events detection and recognition by the fiber vibration system based on power spectrum estimation,” Adv. Mech. Eng 10(11), 1–9 (2018). [CrossRef]
18. Q. Che, H. Wen, X. Li, Z. Peng, and C. K. P., “Partial Discharge Recognition Based on Optical Fiber Distributed Acoustic Sensing and a Convolutional Neural Network,” IEEE Access 7, 101758–101764 (2019). [CrossRef]
