Open Access
Add to BibSonomy
Abstract
In this study, we employ our developed instrument to obtain high-throughput multi-angle single-particle polarization scattering signals. Based on experimental results of a variety of samples with different chemical composition, particle size, morphology, and microstructure, we trained a deep convolutional network to identify the polarization signal characteristics during aerosol scattering processes, and then investigate the feasibility of multi-dimensional polarization characterization applied in the online and real-time fine and accurate aerosol recognition. Our model shows a high classification accuracy rate (>98%) and can achieve aerosol recognition at a very low proportion (<0.1%), and shows well generalization ability in the test set and the sample types not included in the training set. The above results indicate that that the time series pulses from multi-angle polarization scattering contain enough information related with microscopic characteristics of an individual particle, and the deep learning model shows its capability to extract features from these synchronous multi-dimensional polarization signals. Our investigations confirm a good prospect of aerosol attribute retrieval and identifying and classifying individual aerosols one by one by the combination of multi-dimensional polarization scattering indexes with deep learning method.
© 2021 Optical Society of America under the terms of the OSA Open Access Publishing Agreement
1. Introduction
In the past few years there has been growing interest in developing new technologies and methods to analyze and characterize suspended particles in the air, due to the importance of various aerosol particles in environmental [1], meteorological [2], marine [3], and public health areas [4,5]. Recently, real-time online aerosol identification and classification methods are getting more and more attention [6,7]. Usually the standard method of aerosol analysis generally employs microscopy examination or molecular analysis of samples collected by membrane [8]. For example, Brostrom used scanning electron microscopy combined with energy dispersive X-ray spectroscopy to characterize the physical and chemical properties of complex aerosols [9]. And Pan, M also described several standard air sampling methods and their performance requirements [10]. These methods need to transfer the aerosol sampled on site to the laboratory for manual inspection, resulting in low time resolution and relatively long analysis time. To meet the needs of real-time measurement in real applications, lidar and remote sensing technology are often applied. To characterize and classify aerosol types, Mukai Sonoyo used multispectral satellite remote sensing data combined with polarization information [11], and Liu Dong used data measured by atmospheric backscatter lidar [12]. However, the above methods can only analyze and distinguish aerosols in a macro scale and cannot identify individual aerosol one by one.
One way to achieve classification and identification of single particles is to use a technology similar to flow cytometry. The optical flow cytometer can measure the scattered light of cells in a real-time, high-throughput mode for cell counting, size and fluorescence measurement [13–15]. Similarly, Kaye P. H. has developed an instrument for the simultaneous measurement of spatial scattered light and intrinsic fluorescence [7]. The spatial scattered light can be used to evaluate the shape and size properties of particles, and then combined with fluorescence signals, biological and non-biological aerosols can be distinguished. Ding Lei has developed an instrument to measure scattered light from different angles, and then distinguish between spherical and irregular particles [16]. Potenza developed SPES (Single Particle Extinction and Scattering) [17] method which realizes effective characterization of particle size, refractive index and shape [17,18,19]. Related research has shown good performance in analyzing the dust in the East Antarctic ice core samples [20]. Meanwhile, these studies also showed the importance of multiparametric characterization and the advantage of single particle detection which allows to obtain further information on a statistical basis [21].
Aerosol particles, such as minerals and sulfates, have a wide size distribution, complex composition and morphology, and various composite microphysical characteristics [22,23]. In the interaction between light and particle, the change of polarization state contains abundant information of particle microphysical characteristics [24–26]. The polarization measurements greatly extent the optical analysis of aerosol scattering and then improve the dimensionality of information. Some research works extract polarized signals to distinguish particles with different morphology [11,16,27]. Wyatt, P. J. [28] and Liao Riwei [29] retrieve the aerosol complex refractive index based on their respective polarization measurement signals. Hiroshi Kobayashi has developed a polarized optical particle counter that combines the measurement of polarized and scattered light [27], which determines the particle size by the scattering signals, and extracts the morphological characteristics by polarization signals, and then qualitatively characterizes the types of particles.
Deep learning methods have developed rapidly in recent years. The CNN networks play an important role in it. The conception of convolution was firstly introduced into artificial neural network in 1980 by Kunihiko Fukishima [30]. to imitate the vision system of animals. Nowadays, neural networks based on CNN structure have achieved state-of-the-art results for many tasks such as image classification [31], face recognition [32], speech recognition [33] and so on. There are also quite a few applications in optics, such as DL assisted optimized structure design [34] image optical noise removal and resolution enhancement via DL methods [35] and optical signal processing and feature extraction [36]. There are also some studies using deep learning methods to assist aerosol classification. Li Lianfa proposed an autoencoders residual network to estimate PM2.5 concentration based on remote sensing data which achieved a high spatial and temporal resolution and accuracy [37]. Wu Yichen used CNNs to assist label-free bioaerosol image reconstruction and classification which achieved relatively high accuracy on training set (>94%) for a high throughput analysis of bioaerosol type aerosols [38]. Leskiewicz Maciej proposed a model consisting of 22 dense neural networks combined with a decision tree to classify 48 kinds of bioaerosol particles using scattered light and fluorescence signals at the 35° and 145° angles [39]. In the above developments, only the scattering and fluorescence signals were used for aerosol classification and showed an accuracy less than 80% on the test set, which is not enough in real applications. In our recent research work, the introduction of multidimensional polarization signals enhances the capability of aerosol classification [40].
The model in deep learning, such as CNN, can be regarded as a powerful feature extractor. The development of convolutional neural networks has also played a role in promoting the research of optical related technologies. For example, they can be used to handle the complicated mapping function between biased color and ground truth color for RBGN cameras which achieved satisfactory RGBN color restoration performance [41]. And they could also be applied to recover missing information in the multiband polarimetric imaging system which achieved outstanding quality [42].
In this paper, we apply deep learning method to analyze polarization response of scattered aerosols and then extract their composite microscopic characteristics to identify aerosols at a single particle level. Here we employ our developed multi-angle high-throughput polarization vector synchronous analysis device, which can obtain the multi-channel timing optical pulse of individual aerosols and analyze their polarization status in real time. During experiments, we measured a variety of suspended aerosol particles, and selected 6 types of aerosol samples to form the training set, validation set, and test set. The detailed processing method, experimental device and measured results are presented in the following sections.
2. Experiments and samples
2.1 Measurement setup
Figure 1 and Fig. 2 show our measuring device to obtain multi-dimensional polarization signals of individual aerosols, which has been presented in [29]. The instrument can be divided into four parts: light path, gas path, photovoltaic conversion and analog-digital conversion. The optical part employs a solid laser (532 nm, 100 mW, MSL-III-532, Changchun New Industries Optoelectronics Technology Co., Ltd.) and a polarization state generator to generate a 45° linear polarized light beam and then modulate the light spot to a light sheet focused on the center of the scatter chamber (the detection area for particles). The width of the laser sheet at the detection area is 1mm and the height is 0.04mm. On the gas path, the air flow carrying aerosol particles pass through the detection area within the protection of sheath flow. The sheath nozzle is optimally-designed to make sure particles passing by the detected area one by one. When a particle flies through the detection area, we can synchronously record scattered light signals at five angles (10°, 30°, 60°, 85°, 115°) for a fixed time. For each angle, we use a spatial filter module composed of a lens and an aperture at fixed location to filter stray light. We also use a light trap at the end of light path to eliminate the forward stray light. For scattered light at 10°, we only record the intensity as an indication of particles flying through the detection area. For the last four angles, we additionally apply a four-quadrant polarization state analyzer (0°, 90°, 45°, 135° linear polarizer) to split scattered light to four polarization signal channels per angle. Then for each measured particle, at most 16 polarization signal channels combined with one intensity signal channel, are transmitted to photovoltaic conversion unit (SiPMT) through optical fiber bundle, and then are recorded by analog to digital converter.
Fig. 1. (a) Overview of structure of measuring instrument; (b) Schematic diagram of experiment setup. PSG, polarization state generator (composed of a 45° linear polarizer with a quarter wave plate); C, cylindrical lens; L, spatial filter module (composed of a lens and an aperture); P1-P4, scattered light polarization signal channel at four angles (30°, 60°, 85°, 115°); I, scattered light intensity signal channel at 10°; PSA, polarization state analyzer; FP, film polarizer; PC, personal computer.
Our current instrument can obtain signals of up to around 20000 particles in about 1 second. With the controlling of the air flow velocity, a particle flies through the detection area one by one within 50µs. And the sample rate of our analog-digital conversion is 1MHz, that is, we sample a point every 1µs (around 50 points for a particle which contains enough information of the particle).
The instrument in the subsequent experiments only uses linear polarizing films due to the restrictions of the manufacture process, so this study does not involve the circular polarization characterization. Even so, the information extraction based on linear polarization signal processing can already perform an accurate classification of several types of aerosols (accuracy rate as high as 98%). In this study we focus on the generalization ability of the classification model in practical applications. In the real field observation, for convenience, we usually select two scattering angles (in this study, 30° and 85°) and the corresponding 8 polarization scattering signals as our polarization characterization index system. And the result shows that the signal of two angles provides enough information for classification. The further investigation is underway on the polarization characterization applied in aerosol measurements using full Stokes vector analysis at more angles simultaneously.
2.2 Experimental samples
We measured six types of particulate matter including disordered mesoporous carbon, carbon sphere, Arizona test dust, 1µm monodisperse polystyrene latex particles (PSL), sodium nitrate and sodium sulphate. Disordered mesoporous carbon and carbon sphere are carbonaceous particles with light absorption effect. Arizona test dust particles have complex irregular forms. Sodium nitrate and sodium sulphate are water soluble salts with different components. Both carbon sphere and PSL are regular spheres with different diameter. Sodium nitrate has a trigonal crystal system. Sodium sulfate has monoclinic, orthorhombic or hexagonal crystal system. And Arizona dust has an irregular shape. Disordered mesoporous carbon has a mesoporous microstructure while the result of the samples are solid structures. In terms of chemical composition, we employ two kinds of water-soluble salts, that is, nitrate and sulfate. In addition, considering the possible absorption of aerosols, two carbonaceous aerosol samples have been detected. Therefore, our experiments results can constitute a dataset to observe the characteristic polarization features from complicated aerosol types with various attributes. We train our model on this dataset to test whether our model can recognize various cases of aerosol attribute differences. The number of measured particles for each category in this study is shown in Table 1.
Table 1. The number of particles that we tested in this study
3. Theory and principle
3.1 Polarization measurement
Polarization status of scattered light is very sensitive to the property of particles. Previous researches have made clear that the polarization scattering characteristics of aerosol particles could be used for morphological sample classification [46], soot particle recognition [47] and source apportionment [40]. The Stokes parameters (S0, S1, S2, S3) are always used for representing the polarization status of light. They could be calculated as Eq. (1) [48]. I0, I90, I45, I135 are the intensity of linear polarization statues among 0°, 90°, 45°, 135°. IRCP and ILCP are the right-handed and left-handed circular polarization status respectively.
3.2 Convolution neural network
In this study, we use a convolutional neural network to extract features from multi-angle polarization signals to recognize various types of aerosol particles. CNNs often serve as powerful feature extractor. Basic convolutional neural network contains three basic parts: convolution layers, pooling layers and fully connected layers. Conventionally, a feature detection module of CNN contains multiple convolution layers followed by a pooling layer. A sequence of multiple feature detection modules and fully connected layers at the tail construct the basic architecture of convolutional neural network [49–51]. CNN architecture employs filters at different part of signals with shared weights thus it does a good job for translation invariance data. In our experiments, when the scattered intensity is over a certain threshold at forward scattering angle, we can confirm that a valid scattering response of a single aerosol is detected and then the multi-channel polarization signal acquisition is triggered. So, the obtained pulse signal is part of the timing response of interaction between light and a particle, the 1-d CNN architecture is well suited for dealing with such signals.
3.3 Connections between CNN and Stokes measurement
The computation process of one-dimensional convolution on multi-channel time series signals could be viewed as Fig. 3.
Fig. 3. One-dimensional convolution computation process on 4 channel time series signals with kernel size 3, stride 1.
The kernel does convolution operation sliding through multi-channel time series signals with same weights at different location to extract abstract features from raw signals. For our measurement method, we cumulate intensity of scattered light with a specific polarization state when we detect a particle. The instantaneous intensity for each channel could be calculated by subtracting the current signal by the signal at the last time step (e.g., the instantaneous intensity of 0° polarization status scattered light at time step t could be calculate as $I_0^t = CH_1^t - CH_1^{t - 1}$). Then by using kernel with weights like in Fig. 4(a), the convolution result at each location should be Stokes parameter S0 at each time step. Other convolution kernels used for calculating Stokes parameters are shown in Fig. 4. The detailed arithmetic for calculating S1 with I0 and I90 is shown in Eq. (2). The process of calculating Stokes parameters could be accomplished by convolution arithmetic. One of the major characteristics of CNN is that it uses self-learned kernels to extract more abundant features than using kernels with fixed weight which extracts features with specific meaning such as Sobel operator for detecting edges. In the development of computer vision, it has been proved that CNNs provides a more generalized way of extracting features. In our research, we also try CNNs to extract individual microscopic features hidden in particle polarization signals to classify suspend aerosols in the air.
Fig. 4. Convolution kernel weights for calculating Stokes parameters. (a) Kernel weights for calculating S0 with I0 and I90; (b) Kernel weights for calculating S1 with I0 and I90; (c) Kernel weights for calculating S0 with I45 and I135; (d) Kernel weights for calculating S2 with I45 and I135.
3.4 Data preprocessing and training
To ensure that the pulse signal of an individual particle can be obtained completely, we set a fixed maximum measurement time for each particle. At the beginning of data processing, we calculate the current environment background noise from each optical detection channel at the tail of each segment of the signal when no aerosol is analyzed and subtract it from the entire segment of the signal to eliminate its influence, as shown in Fig. 5. The six type preprocessed signals within a certain period are demonstrated in Figs. 6 and 7. From Fig. 7 it can be found that the signal of some particles partially exceeds the range of the instrument. Though these signals only account for a relatively small part, we keep them in our measurement data record during data training and testing to enhance the robustness of the model.
Fig. 5. (a) Signals of 1µm PSL before data preprocessing (using 2 angle and 8 channels as example). (b) Signals of 1µm PSL after data processing (using 2 angle and 8 channels as example).
Fig. 6. Measured signals of 6 kinds of aerosol samples within 1.5 millisecond sampling time (using 2 angle and 8 channels as example). (a). Disordered Mesoporous Carbon. (b). Carbon Sphere. (c). Arizona Test Dust. (d). 1µm PSL. (e). Sodium Nitrate. (f). Sodium Sulphate.
Fig. 7. Measured signals of 6 kinds of aerosol samples within 0.5 second sampling time. (a). Disordered Mesoporous Carbon. (b). Carbon Sphere. (c). Arizona Test Dust. (d). 1µm PSL. (e). Sodium Nitrate. (f). Sodium Sulphate.
For the construction of training set, we first randomly select a certain time point and then take the continuous 10000 segments of signals for each type of particulate matter. Next, we take some points at the end of each signal and concatenate them together to form a new class named empty signal. Samples of empty signals is shown in Fig. 8. Now we have about 65000 segments of signals in 7 categories. We random select 80% of them as the training set and the remaining 20% of them as the validation set. The remaining signals in the entire dataset are used as the test set. We use these timing signals from high throughput measured particles to test the stability of the instrument measurement and the robustness of the model. The manually constructed empty signal class can be also used to eliminate the influence of the noise on the classification accuracy.
For the design of the convolutional neural network, we construct a 1-d convolutional neural network adopting design method just like a VGG [49] with fewer layers. The specific network structure is shown in Table 2. We have also tried to use a structure similar to GoogleNet [50], but it turns out that this has limited improvement in accuracy, probably due to the better aerosol flow rate control where feature extraction on multiple scales may not make much sense. The data volume of training set we used is about one-tenth of that of the test set, and these two datasets do not overlap. That is, the above training dataset and test dataset are identically distributed, and the data distribution of training set and test set may be different, so the generalization ability of the trained model can be evaluated on the test set. It should be noted that, after 80 epochs of training, the model reached a high accuracy rate on the test set. This result implies the potential of the on-line accurate identification of suspended particles, especially the specific recognition of some special type of particulate matters with an ultra-low proportion.
Table 2. Structure of our network
3.5 Single aerosol online high throughput recognition
An algorithm of one by one recognition for individual aerosol particle can be applied to realize a real time classification and then possible source discrimination of detected suspended particulate matters. To distinguish different types of aerosol particles in the mixed state, the variable definitions and the mathematical process can be described as Table 3.
Table 3. Variable Definitions
There are 6 kinds of suspended particles in total in our experiments, that is, T is equal to 6. Their true proportions are defined as α1 to α6. The sum of them is 1. Obviously, the true number of each type of particulate matter is α1N to α6N. For a single particle classification algorithm, if its recall matrix on the test set is C, the estimated proportion of type j aerosols could be calculated as in Eq. (3). Therefore, we can get the proportion identification error of the j-th category as in Eq. (4). It can be found that the deviation between the true and estimated value is related to the proportion of various types of particles and the recall matrix, irrelevant to the total number of particles.
4. Results
4.1 Training results
After training for 80 epochs, the accuracy on training set reaches a quite high level and the model generalizes well on validation and test set. The confusion matrix is shown in Fig. 9, which shows the classification result on training set and test set for 7 kinds of particle signals including empty signals.
It can be easily seen that a real detected particle signal is almost never mistaken as an empty signal, and an empty signal is never mistakenly classified as a real signal generated by any real type of aerosol. Obviously, by adding the empty signal class for training, the model pays more attention to the characteristics of the pulse rather than the characteristics of the base noise when no particles pass through. So, the empty signal can be well recognized, that is, the model can accurately identify the real particle signals and then distinguish the aerosol type based on the pulse characteristics.
The prediction accuracy on training, validation and test set is shown in Table 4. Such results show that the classification accuracy of the trained model has reached a very high level, and it has a strong feature extraction capability. Such a high accuracy can be achieved on a test set ten times the training set, which also proves that the model we trained has a strong generalization ability. And it has been also proved that those signals beyond the measurement range still have rich features enough to determine the aerosol type. Tables 5 shows the recall rate and precision rate of various aerosol types on the test set.
Table 4. Accuracy on dataset
Table 5. Recall and precision for each category on test set
The precision matrix and recall matrix are shown in Fig. 10. Whether it is recall or precision, the classification result of 1µm PSL is the best among the 6 types of particulate matter samples, maybe since the 1µm PSL particles have a good consistency in terms of size, refractive index and morphology. Some misclassification can be observed between the two water-soluble salts, implying some similarities in microphysical characteristics for sodium nitrate and sodium sulfate samples. Part of the disordered mesoporous carbon is mistakenly classified as Arizona dust, considering the irregular morphology of these two types, which may be explained by the important influence of aerosol morphology on the discrimination of polarization scattering signal. In general, all types of recall and precision have reached a high level. Since we did not construct the empty signal class in the test set, about 56% of the predicted empty signal on the precision confusion matrix comes from disordered mesoporous carbon. From the confusion matrix in Fig. 9, only 9 out of 172,370 disordered mesoporous carbon signals are mistakenly classified as empty signals, meaning a quite low probability of misclassification.
4.2 Comparison with other network structures
We also tried to use different models to classify the polarization signals of aerosol samples. The accuracy of these different models on the training set as the number of iterations increases is shown in Fig. 11. The blue line represents the accuracy on the training set of CNN model. The orange line and the yellow line show the classification accuracy of the fully connected neural network, where the former takes the entire segment of signals as input and the latter takes the polarization parameter calculated only by each pulse peak as input. The accuracy of these three models on the training set, validation set and test set are shown in Table 6. The recall matrixes on the test set of two fully connected networks with different inputs are shown in Fig. 12.
Fig. 11. The classification accuracy of the three models during model training with the number of iterations.
Fig. 12. (a). Recall matrix of fully-connected neural network model with entire segment of signal as input. (b). Recall matrix of fully-connected neural network model with the Stokes parameters as input.
Table 6. Accuracy of three models on dataset
As shown in Table 6, the fully connected neural network using the entire segment of signals as input does better than another one using the polarization parameter calculated only by each pulse peak. It indicates that the pulse shape generated from aerosol polarization scattering processes can provide more abundant characteristic information than the pulse peak value, so by extracting the entire short time response waveform of measured aerosols, the accuracy of single particle recognition can be improved greatly. Also, CNN does better than the fully connected neural network, maybe because the method of convolution calculation makes it easier for CNN to learn the convolution kernel similar to the structure that we mentioned in Section 3.3, and then can extract more meaningful features. Additionally, weight sharing and sparse connections characteristics of convolution kernel may make the entire model easier to train. We have also tried to use structure similar to GoogleNet [50] to extract multi-scale structured information, but its effect is not much different from ordinary CNN. Therefore, the ordinary CNN has enough ability to distinguish different types of aerosol particles suspended in the air, that is, the error rate is near Bayesian optimal.
Regardless of CNN or these two fully connected neural networks with different inputs, the model always achieves the highest accuracy on PSL (whether in terms of recall or precision), followed by carbon sphere. Similar with PSL, black carbon spheres also have relatively consistent microphysical features including regular shape, a narrow size distribution and a relatively uniform complex refractive index. Some misclassifications between sodium nitrate and sodium sulfate and between some disordered mesoporous carbon and Arizona dust have been discussed in the above paragraph.
4.3 Aerosol recognition with different proportions
Then we evaluate the performance of the trained model by applying it to identify some specific aerosol type with different proportions. When the proportion of target type is between 10% and 100%, the proportion identification error is shown in Fig. 13. The orange line and the blue line are the estimated maximum and minimum error, and the green line is the estimated expectation. They are calculated according to Eqs. (5), (6) and (9) using the recall matrix on test set as shown in Fig. 10(a). Every black dot in the figure represents a target type proportion identification error on a randomly sampled dataset. Their corresponding x-axis values represent the actual proportion of the target category. Here, to construct a sampled dataset, we randomly sample 10000 pieces of signals from the test set where the proportion of the target category is given, and the proportions of the remaining categories are randomly generated. For example, assuming 1µm PSL is type j, to construct a randomly sampled dataset in which the proportion of target type is 10%, we randomly select 10000 pieces of signals from the test set in which 10% of them are signals generated by 1µm PSL and the rest of them are signals generated by other types with randomly generated proportions. That is, αj is equal to 0.1 and all other α are all randomly generated. The sum of all α is 1 and N is equal to 10000. Then we use our trained model to estimate the proportion of 1µm PSL on the sampled dataset to simulate a proportion identification process. The proportion identification error is defined as Errj. The black dot for this randomly sampled dataset should be at location of (10%, Errj). For each target type with a certain proportion, we construct 100 sampled datasets. And in Table 7 we show the theoretical expectation of the proportion identification error when the actual proportion of target type is 10%, 80% or 100%. We also show the increase of theoretical expectation of proportion identification error with the increasing proportion of target type aerosol by 1%, which can be described as $1 - C[j,j] + \left( {\sum\limits_{i = 1,i \ne j}^T {C[i,j]} } \right)/({T - 1} )$.
Fig. 13. Proportion identification error for six categories as target separately (relatively large proportion). (a) Disordered Mesoporous Carbon. (b) Arizona Test Dust. (c) Sodium Nitrate. (d) Carbon Sphere. (e) 1µm PSL. (f) Sodium Sulphate.
Table 7. Theoretical expectation of the proportion identification error (relatively large proportion)
In Fig. 13, we can see that most of the proportion identification error on sampled datasets are concentrated between the upper and lower limits of the estimation, and their average values are almost on the line for the estimated expected proportion identification error. However, for some cases of the target type aerosol with a certain proportion, the proportion identification errors on sampled datasets fluctuates around the mean value and some of them exceeds the upper and lower limits of theoretical estimates. The reason is that our theoretical error estimate is calculated based on the recall matrix on the entire test set (Fig. 10(a)), and the error represented by each black dot should be calculated based on the recall matrix on the sampled dataset. And as the total number of signals in the sampled dataset increases, the recall matrix on the sampled dataset gradually approaches the recall matrix on the entire test set. By comparing proportion identification error of target type on randomly sampled datasets with the theoretically estimated proportion identification error limits, we confirm the theoretical error estimation method mentioned in Section 3.5.
Furthermore, in Fig. 13, we can see that estimated maximum, minimum and expectation of the proportion identification errors for the target type all increase linearly and monotonously as the true proportion of the target category increases. When the target type aerosols account for 10% at the beginning, from Table 7, except for disordered mesoporous carbon with an estimated expected error close to 0, the estimated expected proportion identification errors for the other five types are all less than 0 which is calculated according to Fig. 10(a) and Eq. (9), meaning that the real proportion is less than the estimated ratio. As the proportion of target types increases, the estimates of errors gradually increase. And when the actual proportion of the target category is 100%, these three error estimate lines reach the maximum value and converge to one point representing the recall rate for the target type particles. For estimates of proportion identification error as shown in Eqs. (5), (6), (9), the proportion identification error could be viewed as a weighted average of self-misclassification and mistakenly classifying other categories as target type. The self-misclassification term is 1-C[j,j] weighted by αj. The term for mistakenly classifying other categories as target type is $- \arg {\min _{i,i \ne j}}C[i,j]$ in Eq. (5), $- \arg {\max _{i,i \ne j}}C[i,j]$ in Eq. (6) and $- \sum\limits_{i = 1,i \ne j}^T {C[i,j]} /({T - 1} )$ in Eq. (9) weighted by 1-αj. The self-misclassification term is always positive meaning the overestimation of proportion identification error and the term for mistakenly classifying other categories as target type is always negative meaning the underestimation of proportion identification error. The self-misclassification term is much bigger than other terms for our trained model which can be inferred from the confusion matrix in Fig. 10(a). So as the increase of proportion of the target type aerosols, the estimated errors gradually increases to self-misclassification term which represents the recall rate for the target type particles. Considering the counteracting attribution of two error sources, there should be a critical point for αj which may be different for different target types. When αj is bigger than that, the self-misclassification term should be the majority source of proportion identification error. When αj is smaller than that, the term for mistakenly classifying other categories as target type should be the majority source. And when αj is equal to the value, the estimate of error should be zero.
From Fig. 13 and Table 7, we can see that when the target category accounts for a relatively large proportion (greater than 10%), the trained model can well handle the task of identifying the target category proportion. At this time, most of the errors come from the missed recognition of the target category and a small part is due to other categories mistakenly classified as the target category. Among them, 1µm PSL has the lowest proportion identification error and the slowest increase rate, followed by carbon spheres, then disordered mesoporous carbon and Arizona Dust, and finally two water-soluble salts. When 1µm PSL is target type, the highest expected proportion identification error is only 0.848%. As the proportion of 1µm PSL increases by 1%, the proportion identification error is expected to increase by 0.00953%. The expected error for carbon spheres is slightly higher which is 1.148%. And for every 1% increase in actual content, the error for carbon spheres increases by 0.01422%. The similarity of the two water-soluble salts may explain the larger error in proportion identification, but the highest expected error is only about 2.7%.
It should be noted that when the actual proportion of the target aerosol type is extremely low (1%, 0.1% or even 0.01%), the performance of our model is shown in Table 8. In this case, the overwhelming majority source of error should be that other categories are mistakenly classified as the target category. Because when the proportion of target aerosol type is reduced to 0.01%, even if 10% of the target aerosols are mistakenly classified, its contribution to the proportion identification error is only 0.001%. However, the other types of aerosols misclassified as the target type at least contribute -0.105% to the proportion identification error (1µm PSL). Consequently, the key to realize a precise recognition to those aerosol type with a very low ratio is to prevent other type aerosols from being mistakenly classified as the target type.
Table 8. Theoretical expectation of the proportion identification error (low proportion)
From the above discussion, we can see that there are two main sources of aerosol identification error, self-misclassification and mistakenly classifying other categories as target type. Ideally for each category if the signals that are misclassified are evenly divided into other categories, then we can use Eq. (10) to describe the proportion identification error. $\sum\limits_{i = 1,i \ne j}^T {C[i,j]} /({T - 1} )$ can be roughly estimated by (1-C[j,j])/(T-1). It is easy to know that the improvement of the overall classification accuracy can increase the average of C[j,j] for all j, and then reduce the impact of both source of errors. When the proportion of target types is relatively low, we need to at least decrease the estimated intercept term of proportion identification error to an order of magnitude lower than the proportion of target types. This means that we need to roughly increase the overall accuracy to keep (1-C[j,j])/(T-1) less than 0.1αj for all j. If the model needs to be suitable for proportion identification under 0.1% or even 0.01%, the classification accuracy needs to be improved by orders of magnitude. That is, we need to increase the current classification accuracy rate (98.00% as shown in Table 6) to about 99.99% for the case of the target proportion at the 0.1% level, or to about 99.999% to for the target proportion identification at the 0.01% level, which is extremely difficult to achieve, no matter in terms of instruments or algorithms.
Next, for each category, we select the probability threshold when FPR is equal to 0.0001 on the test set as the determination threshold. Table 9 shows the proportion identification error of the target type aerosols with a proportion of 1%, 0.1% and 0.01%, respectively. We also list the determination threshold and the intercept term of the theoretical expected error under the corresponding determination threshold. It can be seen that the error is already greatly improved when the target type is at 1% and 0.1%, and is also acceptable even when the target type is at 0.01%.
Table 9. Theoretical expectation of the proportion identification error for selected determination threshold (low proportion)
We then use these six types as target type one by one and randomly sample from the entire test set to generate datasets with target types contents of 1%, 0.1% and 0.01% like what we did in Fig. 13. We predict the proportion of the target category in the dataset and calculate the error, and compare it with the error by our original method based on the statistical distribution of the measured polarization optical indexes for high throughput aerosol analysis as shown in Fig. 15 and Fig. 16. In our original classification recognition algorithm, we multiply and sum the probability density curve for polarization parameters of different types of aerosols by their proportions to fit the probability density curve for the whole sampled dataset. We use the least square method to get the optimal solution of proportions. For the case of the target type aerosols with a lower proportion, the error of the classification recognition algorithm based on probability density curve is shown in the red dots in (a), (b) and (c) in Figs. 15 and 16. The black dots in (d), (e) and (f) in Figs. 15 and 16 are the errors of improved algorithm based on single aerosol classification with selected determination threshold on sampled datasets and the three lines shows the estimates of maximum, minimum and expected proportion identification error. Notice that these lines are the same in (a) and (d), (b) and (e), (c) and (f) for both Figs. 15 and 16.
Fig. 15. Proportion identification error and comparison (low proportion). (a) Disordered Mesoporous Carbon by algorithm based on distribution spectrum method. (b) Carbon Sphere by algorithm based on distribution spectrum method. (c) Arizona Test Dust by algorithm based on distribution spectrum method. (d) Disordered Mesoporous Carbon by algorithm based on single particle classification. (e) Carbon Sphere by algorithm based on single particle classification. (f) Arizona Test Dust by algorithm based on single particle classification.
Fig. 16. Proportion identification error and comparison (low proportion). (a) 1µm PSL by algorithm based on distribution spectrum method. (b) Sodium Nitrate by algorithm based on distribution spectrum method. (c) Sodium Sulphate by algorithm based on distribution spectrum method. (d) 1µm PSL by algorithm based on single particle classification. (e) Sodium Nitrate by algorithm based on single particle classification. (f) Sodium Sulphate by algorithm based on single particle classification.
For example, when Arizona test dust is selected as the target type, that is, (c) and (f) in Fig. 15, the red points in (c) shows the proportion identification error of Arizona test dust using the statistical distribution spectrum algorithm, and the black points in (f) are for the improved single particle classification algorithm. The lines in (c) and (f) are all estimated maximum, minimum and expected proportion identification error for the improved single particle classification algorithm and x-axis is logarithmic scale. We draw the upper and lower subgraphs to compare the performance of two different algorithms on the identification of aerosols with a very low proportion. It can be seen that in our improved algorithm, the error of the target category still conforms to the result of our theoretical estimation, that is, most of them are within the upper and lower limits, and the average value is on the expected error estimation line. Therefore, our improved algorithm can achieve a much better accurate classification and recognition than the original statistic distribution method, especially for the target aerosol with low proportion.
Through the above discussion, the error estimation method in Section 3.5 has been confirmed. There are two main error sources. One is that the signal of target type aerosols is mistakenly classified into the other types, and the other is that the signal of other types aerosols is mistakenly classified as the target type. The former is the main error source when the proportion of target particles is not low, we can directly count the classification results of the trained model to obtain an accurate estimated proportion. Otherwise the latter is the main error source for the case of low proportion target particles, we can adjust the threshold for distinguishing the signal of the target category. Using our improved method, the error is more controllable and predictable.
4.3 Potentials in aerosol attribute retrieval and similarity analysis
We sort various types of signals in the entire test set according to their predicted likelihood and take the highest-ranked signal of each type as shown in Fig. 17. Among them, each signal is the one that is most likely to be determined by the trained model in the whole data set. For example, the disordered mesoporous carbon signal in this figure is the signal considered to be the most consistent with the characteristics of disordered mesoporous carbon by the trained model. Since the Mie scattering theory can calculate the scattering process of spheres with known size, we take the first 30 ranked signals of 1µm PSL to calculate the average polarization parameters and compare it with the theoretical results as shown in Fig. 18. The x-axis is the scattering angle and the red triangles represent the average polarization parameters calculated. It can be found that the measured values are in a good agreement with theoretical calculations. Here we also can estimate the complex refractive index of each PSL particle according to the method in [29], where the imaginary part is always zero and the real part is shown on the vertical axis on the right in Fig. 19. Theoretically, the complex refractive index of PSL aerosols is 1.59 to 1.60, which is very close to our calculations. The above results also support the accuracy of our measurements and the data validity and show the potential of our model in subsequent research on aerosol attribute inversion.
Fig. 18. Simulation and average of Stokes parameters measured for 1µm PSL for P incidence. (a) S1. (b) S2.
After training our model, we try to use our model to classify the signals whose type do not exist in the training set. The result is shown in Fig. 20. For the new aerosol type whose measured signals are not included in our training set, the aerosol recognition by the model implies the similarity discrimination ability of our classification algorithm.
As can be seen in the figure, it reflects that our model has indeed learned the relationship between aerosol attributes and polarization scattering features. We can see that carbonaceous particles can be roughly divided into two types: carbon spheres and disordered mesoporous carbon. Fly ash is mainly mistaken for carbon spheres and sodium nitrate. Two types of PSL with different particle sizes are also mainly classified as 1µm PSL, which proves the availability of our algorithm in discriminating the similarity. Most of sodium chloride aerosols are recognized as sodium sulfate or sodium nitrate, both of which belongs to water-soluble aerosols with sodium. Titanium oxide nanowires are mainly classified as disordered mesoporous carbon, carbon sphere and Arizona test dust, which may be due to morphological reasons. These results imply that our model may indeed learn some complex relationships between polarization signal characteristics and particle size, morphology and complex refractive index, which need to be further discussed later.
5. Conclusion
In this study, we employ our developed multi-angle polarization scattering measuring device and obtain the simultaneous pulse response of multi-dimensional polarization indexes of various types aerosol particles. Our technology is suitable for online high-throughput real-time single aerosol analysis and can record the multi-dimensional polarization indexes as a big data basis. This research focuses on the feature extraction and accurate recognition based on these polarization optical indexes of aerosols and investigates the feasibility and potential of deep learning methods.
We train an end-to-end deep learning model based on the CNN network to classify single-particle polarization signals. Here we use the polarization scattering pulse waveform group as the input signals of deep learning, instead of the integrated Stokes parameters. The trained model achieves a high accuracy rate on the test set, which confirms its capability to extract polarization characteristics of aerosols with a good generalization. Furthermore, because our model can provide the probability of an individual particle belonging to different types, this single aerosol algorithm can be applied in an accurate proportional prediction of multicomponent aerosol mixtures. In addition, we present how to estimate and control the error of proportion by this method, and propose an improved way to realize accurate aerosol recognition even for the case of the target type aerosol with a very low proportion.
Our studies indicate that the implicit inherent connection between aerosol properties and polarization features can be revealed and learned by deep learning method. With the combination of our polarization measurement system and our polarization characterization based on deep learning, the real time high throughput aerosol identification of suspended particulate matter can be applied in air quality evaluation, microbiological monitoring in health and medical institutions, and early warning of toxic, harmful, flammable and explosive particles in special environment.
Funding
National Key Program of Science and Technology Supporting Economy of China (2020YFF01014500ZL); Science and Technology Research Program of Shenzhen Grant (JCYJ20200109142820687).
Acknowledgments
The work has been supported by National Key Program of Science and Technology Supporting Economy of China (2020YFF01014500ZL) and Science and Technology Research Program of Shenzhen Grant (JCYJ20200109142820687).
Disclosures
The authors declare no conflicts of interest.
Data availability
Data underlying the results presented in this paper are not publicly available at this time but may be obtained from the authors upon reasonable request.
References
1. X. Xia, H. Che, H. Shi, H. Chen, X. Zhang, P. Wang, P. Goloub, and B. Holben, “Advances in sunphotometer-measured aerosol optical properties and related topics in China: Impetus and perspectives,” Atmos. Res. 249, 105286 (2021). [CrossRef]
2. Z. Yinchao, S. Zhuoran, C. Siying, C. He, G. Pan, C. Su, H. Jinxi, W. Jiaqi, and N. Xuan, “Classification and source analysis of low-altitude aerosols in Beijing using fluorescence-Mie polarization lidar,” Opt. Commun. 479, 126417 (2021). [CrossRef]
3. H. Li, Y. Chen, S. Zhou, F. Wang, T. Yang, Y. Zhu, and Q. Ma, “Change of dominant phytoplankton groups in the eutrophic coastal sea due to atmospheric deposition,” Sci. Total Environ. 753, 141961 (2021). [CrossRef]
4. J. L. Mauderly and J. C. Chow, “Health effects of organic aerosols,” Inhalation Toxicology 20(3), 257–288 (2008). [CrossRef]
5. D. V. Lim, J. M. Simpson, E. A. Kearns, and M. F. Kramer, “Current and developing technologies for monitoring agents of bioterrorism and biowarfare,” Clin. Microbiol. Rev. 18(4), 583–607 (2005). [CrossRef]
6. J. K. Zhang, Y. Sun, Z. R. Liu, D. S. Ji, B. Hu, Q. Liu, and Y. S. Wang, “Characterization of submicron aerosols during a month of serious pollution in Beijing, 2013,” Atmos. Chem. Phys. 14(6), 2887–2903 (2014). [CrossRef]
7. P. H. Kaye, J. E. Barton, E. Hirst, and J. M. Clark, “Simultaneous light scattering and intrinsic fluorescence measurement for the classification of airborne particles,” Appl. Opt. 39(21), 3738–3745 (2000). [CrossRef]
8. E. Miaskiewicz-Peska and M. Lebkowska, “Comparison of aerosol and bioaerosol collection on air filters,” Aerobiologia 28(2), 185–193 (2012). [CrossRef]
9. A. Brostrom, K. I. Kling, K. S. Hougaard, and K. Molhave, “Complex Aerosol Characterization by Scanning Electron Microscopy Coupled with Energy Dispersive X-ray Spectroscopy,” Sci. Rep. 10(1), 9150 (2020). [CrossRef]
10. M. Pan, J. A. Lednicky, and C. Y. Wu, “Collection, particle sizing and detection of airborne viruses,” J. Appl. Microbiol. 127(6), 1596–1611 (2019). [CrossRef]
11. S. Mukai, I. Sano, and M. Nakata, “Algorithms for the classification and characterization of aerosols: utility verification of near-UV satellite observations,” J. Appl. Remote Sens. 13(01), 1 (2019). [CrossRef]
12. D. Liu, Y. Yang, Y. Zhang, Z. Cheng, Z. Wang, J. Luo, L. Su, L. Yang, Y. Shen, J. Bai, and K. Wang, “Pattern recognition model for aerosol classification with atmospheric backscatter lidars: principles and simulations,” J. Appl. Remote Sens. 9(1), 096006 (2015). [CrossRef]
13. J. W. ParksH. Schmidt, and Ieee, “Optofluidic Chips with Integrated Waveguides and Active Microfluidics for Single Particle Detection,” in 2016 Ieee Photonics Conference (2016), pp. 370–371.
14. H.-T. Chen and Y.-N. Wang, “Optical microflow cytometer for particle counting, sizing and fluorescence detection,” Microfluid. Nanofluid. 6(4), 529–537 (2009). [CrossRef]
15. Q. Xiang, X. C. Xuan, B. Xu, and D. Q. Li, “Multi-functional particle detection with embedded optical fibers in a poly(dimethylsiloxane) chip,” Instrum. Sci. Technol. 33(5), 597–607 (2005). [CrossRef]
16. L. Ding, J. Zhang, H. Zheng, Y. Wang, and L. Fang, “A method of simultaneously measuring particle shape parameter and aerodynamic size,” Atmos. Environ. 139, 87–97 (2016). [CrossRef]
17. M. A. Potenza, T. Sanvito, and A. Pullia, “Measuring the complex field scattered by single submicron particles,” AIP Adv. 5(11), 117222 (2015). [CrossRef]
18. F. Mariani, V. Bernardoni, F. Riccobono, R. Vecchi, G. Valli, T. Sanvito, B. Paroli, A. Pullia, and M. A. Potenza, “Single particle extinction and scattering allows novel optical characterization of aerosols,” J. Nanopart. Res. 19(8), 291 (2017). [CrossRef]
19. M. Potenza, S. Albani, B. Delmonte, S. Villa, T. Sanvito, B. Paroli, A. Pullia, G. Baccolo, N. Mahowald, and V. Maggi, “Shape and size constraints on dust optical properties from the Dome C ice core, Antarctica,” Sci. Rep. 6(1), 28162–28169 (2016). [CrossRef]
20. M. Potenza, L. Cremonesi, B. Delmonte, T. Sanvito, B. Paroli, A. Pullia, G. Baccolo, and V. Maggi, “Single-particle extinction and scattering method allows for detection and characterization of aggregates of Aeolian dust grains in ice cores,” ACS Earth Space Chem. 1(5), 261–269 (2017). [CrossRef]
21. L. Cremonesi, A. Passerini, A. Tettamanti, B. Paroli, B. Delmonte, S. Albani, F. Cavaliere, D. Viganò, G. Bettega, and T. Sanvito, “Multiparametric optical characterization of airborne dust with single particle extinction and scattering,” Aerosol Sci. Technol. 54(4), 353–366 (2020). [CrossRef]
22. R. L. Gorny, “Filamentous microorganisms and their fragments in indoor air - A review,” Ann. Agric. Environ. Med. 11(2), 185–197 (2004).
23. C. Poehlker, J. A. Huffman, J. D. Foerster, and U. Poeschl, “Autofluorescence of atmospheric bioaerosols: spectral fingerprints and taxonomic trends of pollen,” Atmos. Meas. Tech. 6(12), 3369–3392 (2013). [CrossRef]
24. A. Kouzoubov, M. J. Brennan, J. C. Thomas, and R. H. Abbot, “Monte Carlo simulations of the influence of particle nonsphericity on remote sensing of ocean water,” J. Geophys. Res. 104(D24), 31731–31737 (1999). [CrossRef]
25. J. Liu, J. Liu, X. He, L. Tian, Y. Bai, T. Chen, Y. Wang, F. Zhu, and D. Pan, “Retrieval of marine inorganic particle concentrations in turbid waters using polarization signals,” Int. J. Remote Sensing 41(13), 4901–4922 (2020). [CrossRef]
26. P. Hamill, P. Piedra, and M. Giordano, “Simulated polarization as a signature of aerosol type,” Atmos. Environ. 224, 117348 (2020). [CrossRef]
27. H. Kobayashi, M. Hayashi, K. Shiraishi, Y. Nakura, T. Enomoto, K. Miura, H. Takahashi, Y. Igarashi, H. Naoe, N. Kaneyasu, T. Nishizawa, and N. Sugimoto, “Development of a polarization optical particle counter capable of aerosol type classification,” Atmos. Environ. 97, 486–492 (2014). [CrossRef]
28. P. J. Wyatt, “Some chemical, physical, and optical properties of fly ash particles,” Appl. Opt. 19(6), 975–983 (1980). [CrossRef]
29. R. Liao, N. Zeng, M. Zeng, Y. He, and H. Ma, “Estimation and extraction of the aerosol complex refractive index based on Stokes vector measurements,” Opt. Lett. 44(19), 4877–4880 (2019). [CrossRef]
30. K. Fukushima, “Neocognitron - A self-organizing neural network model for a mechanism of pattern recognition unaffected by shift in position,” Biol. Cybernetics 36(4), 193–202 (1980). [CrossRef]
31. A. Krizhevsky, I. Sutskever, and G. E. Hinton, “ImageNet classification with deep convolutional neural networks,” Commun. ACM 60(6), 84–90 (2017). [CrossRef]
32. Y. Taigman, M. Yang, M. A. Ranzato, L. Wolf, and Ieee, “DeepFace: Closing the Gap to Human-Level Performance in Face Verification,” in Ieee Conference on Computer Vision and Pattern Recognition (2014), pp. 1701–1708.
33. Y. Zhang, W. Chan, N. Jaitly, and Ieee, “VERY DEEP CONVOLUTIONAL NETWORKS FOR END-TO-END SPEECH RECOGNITION,” in Ieee International Conference on Acoustics, Speech and Signal Processing (2017), pp. 4845–4849.
34. J. Chen and H. Jiang, “Optimal Design of Gain-Flattened Raman Fiber Amplifiers Using a Hybrid Approach Combining Randomized Neural Networks and Differential Evolution Algorithm,” IEEE Photonics J. 10(2), 1–15 (2018). [CrossRef]
35. X. Kuang, X. Sui, Y. Liu, Q. Chen, and G. U. Guohua, “Single Infrared Image Optical Noise Removal Using a Deep Convolutional Neural Network,” IEEE Photonics J. 10(2), 1–15 (2018). [CrossRef]
36. D. Wang and L. Wang, “On OCT Image Classification via Deep Learning,” IEEE Photonics J. 11(5), 1–14 (2019). [CrossRef]
37. L. Li, “A Robust Deep Learning Approach for Spatiotemporal Estimation of Satellite AOD and PM2.5,” Remote Sens. 12(2), 264 (2020). [CrossRef]
38. Y. Wu, A. Calis, Y. Luo, C. Chen, M. Lutton, Y. Rivenson, X. Lin, H. C. Koydemir, Y. Zhang, H. Wang, Z. Gorocs, and A. Ozcan, “Label-Free Bioaerosol Sensing Using Mobile Microscopy and Deep Learning,” ACS Photonics 5(11), 4617–4627 (2018). [CrossRef]
39. M. Leskiewicz, M. Kaliszewski, M. Wlodarski, J. Mlynczak, Z. Mierczyk, and K. Kopczynski, “Improved real-time bio-aerosol classification using artificial neural networks,” Atmos. Meas. Tech. 11(11), 6259–6270 (2018). [CrossRef]
40. D. Li, F. Chen, N. Zeng, Z. Qiu, H. He, Y. He, and H. Ma, “Study on polarization scattering applied in aerosol recognition in the air,” Opt. Express 27(12), A581–A595 (2019). [CrossRef]
41. Z. Han, L. Li, W. Jin, X. Wang, G. Jiao, and H. Wang, “Convolutional Neural Network Training for RGBN Camera Color Restoration Using Generated Image Pairs,” IEEE Photonics J. 12(5), 1–15 (2020). [CrossRef]
42. Y. Zhao, M. Wang, G. Yang, and J. C. Chan, “FOV Expansion of Bioinspired Multiband Polarimetric Imagers With Convolutional Neural Networks,” IEEE Photonics J. 10(1), 1–14 (2018). [CrossRef]
43. D. T. L. Alexander, P. A. Crozier, and J. R. Anderson, “Brown carbon spheres in East Asian outflow and their optical properties,” Science 321(5890), 833–836 (2008). [CrossRef]
44. K. S. Lee, K. H. Lee, S. B. Kim, J. H. Jung, B. H. Ha, H. J. Sung, and S. S. Kim, “Refractive-index-based optofluidic particle manipulation,” Appl. Phys. Lett. 103(7), 073701 (2013). [CrossRef]
45. I. N. Tang, “Chemical and size effects of hygroscopic aerosols on light scattering coefficients,” J. Geophys. Res. 101(D14), 19245–19250 (1996). [CrossRef]
46. Y. Chen, N. Zeng, S. Chen, D. Zhan, Y. He, and H. Ma, “Study on on morphological analysis of suspended particles using single angle polarization scattering measurements,” J. Quant. Spectrosc. Radiat. Transfer 224, 556–565 (2019). [CrossRef]
47. D. Li, N. Zeng, D. Zhan, Y. Chen, M. Zeng, and H. Ma, “Differentiation of soot particulates in air using polarized light scattering method,” Appl. Opt. 56(14), 4123–4129 (2017). [CrossRef]
48. J. Bai, C. Wang, X. Chen, A. Basiri, C. Wang, and Y. Yao, “Chip-integrated plasmonic flat optics for mid-infrared full-Stokes polarization detection,” Photonics Res. 7(9), 1051–1060 (2019). [CrossRef]
49. K. Simonyan and A. Zisserman, “Very Deep Convolutional Networks for Large-Scale Image Recognition,” arXiv 1409.1556 (2014).
50. C. Szegedy, W. Liu, Y. Jia, P. Sermanet, S. Reed, D. Anguelov, D. Erhan, V. Vanhoucke, A. Rabinovich, and Ieee, “Going Deeper with Convolutions,” in Ieee Conference on Computer Vision and Pattern Recognition (2015), pp. 1–9.
51. K. He, X. Zhang, S. Ren, J. Sun, and Ieee, “Deep Residual Learning for Image Recognition,” in Ieee Conference on Computer Vision and Pattern Recognition (2016), pp. 770–778.
