1. Introduction

Agricultural resources investigations are the basis for obtaining important data in agricultural production [1]. In recent years, UAV photographs have been introduced to agricultural resources investigations because UAV images is convenient and extremely low cost, and it can satisfy large-scale high-precision mapping in local areas [2]. Therefore, it has become an ideal tool for surveying agricultural resources with high precision in local areas. As the application of regional agricultural resources investigations becomes more extensive and there is an ever-growing demand for accuracy as well, the existing algorithms cannot follow the new requirement in high precision. A novel approach should be explored to meet the practical applications of UAV photography.

Cropland is a fundamental resource in agriculture [3]. Accurate recognition and segmentation of cropland in UAV photos is widely recognized in agricultural production and has high application value. Therefore, high-precision recognition and segmentation of cropland in UAV photographs is worth in-depth research.

In the recognition and segmentation of agricultural tasks, previous scholars had done a lot of work [4]. Hyperspectral image (HSI) was introduced to these studies, which involved near-infrared band (NIR) images especially [5]. And they proposed a series of algorithms such as support vector machines(SVM) [6], the random forest [7], network analysis [8], linear representation-based classification(LRC) [9], and so on.

About research images, the references show that HSI seems to be the mainstream of recognition and segmentation in agricultural tasks because the different wavelengths can provide rich spectral characteristics and the near-infrared band (NIR) is sensitive to crops. However, hyperspectral cameras usually require longer exposure times to maintain the signal-to-noise ratio of each spectral channel. And its expensive cost is one of the main constraints that prevent HSI from being widely applied [10]. At the same time, high-precision recognition and segmentation require the image with a higher spatial resolution and a high spatial resolution image is beneficial for accurate manual labeling. However, the higher resolution of the image will magnify the noise that the NIR spectrum brings into it. So, the NIR has a negative impact on high spatial resolution images [11]. Compared to HSI, traditional RGB images have high utility [12]. It is promising research and will significantly improve the efficiency of agricultural resources investigations if the visible UAV photograph can be used to quickly and accurately identify the cropland.

On algorithms based on the visible photograph in agriculture, the references show that they have made impressive progress. To encourage research in computer vision for agriculture, Wang et al. reviewed the applications of machine vision in agriculture [13]. Syazwani et al. present an automatic method for identifying the pineapple’s crown images in the designated plot and they further count the detected images by machine learning classifiers [14]. A leaf area index (LAI) estimation model developed by Liu et al. was compared to other practical methods [15]. They noted that UAVs with RGB cameras are a low-cost and efficient tool. Biswas et al. developed a new method to delineate individual patches and to estimate mangrove cover from high-resolution aerial photography (0.08 m spatial resolution) with RGB channels [16]. Hasan et al. recognize the cropland with winter wheat in Xinjiang, China, via LAI from UAV RGB images [17]. However, in real application scenarios, many data sets are high-dimensional [18]. Classification of high-dimensional data sets cannot be solved merely by traditional methods. Gradually, sparse representation classification (SRC) algorithms entered the scholar’s vision [19]. In SRC, a high-dimensional test can be linearly reconstructed by atoms and its coefficient is sparse. To calculate the residuals between the original test and the reconstructed data in each subspace to classify it into the class in the subspace where its residual is least. While many sparse models have been proposed and several applications have emerged, the study of cropland recognition problems based on UAV RGB photographs is lacking. Thus, we will focus on the SRC study for cropland recognition based on UAV RGB photographs.

The rest of this paper is organized as follows. In section 2, we briefly overview the related work. Section 3 introduces the multi-feature sparse representation based on adaptive graph constraint (AMFSR). In this section, we optimize the objective of AMFSR and then obtain the analytic solution of the model based on strict derivation, which is the key to designing a greedy algorithm. Section 4 presents the experimental results and analyses.

2. Related works

Sparse representation classification(SRC) was proposed by Hui et al. for face recognition [20]. It indicates that the features of samples belonging to the same class are located in the same low-dimensional subspace. And it implies that the most elements in the coefficient vector are zero and SRC can only handle non-zero elements, which greatly reduces the complexity of matrix operations. Because of its excellent ’parse’ property and its high performance of processing high-dimensional data, SRC has been becoming a hotspot in the high-dimensional data classification, such as remote sensing images [21] and biological image recognition [22]. Xun et al. applies the sparse representation algorithm into agricultural aerial imagery classification in 2021 [23].

2.1 Original sparse representation classification

SRC supposes that there are M distinct classes and M dictionaries in the classifying task, in which the dictionary of the m-th class is marked as $D_{m}$ and there are $N_{m}$ atoms in $D_{m}$. If sample $x$ belongs to class m, it can be linearly approximated by the atoms in $D_{m}$.

All of $D_{m}$ constitutes the general dictionary $D = [D_{\rm 1} \ D_{\rm 2} \ \cdots \ D_{\rm M}]$, then sample $x$ can still be described as

There are only non-zero values in $\alpha _{m}$ of $\left [\alpha _{\rm 1} \ \alpha _{\rm 2} \ \cdots \ \alpha _{\rm M}\right ]^{T}$ and the others are zero, so $\left [\alpha _{\rm 1} \ \alpha _{\rm 2} \ \cdots \ \alpha _{\rm M}\right ]^{T}$ is sparse. For approximating sample $x$ with dictionary $D$ and coefficient $\alpha$, an optimization problem is constructed, which is regularized by the 0-norm of coefficient $\alpha$.

The optimization problem is solved to obtain the sparse vector $\alpha$. Then sample $x$ is classified.

2.2 Nonlinear sparse representation classification

With the in-depth study of SRC, some scholars hold that the SRC of high-dimensional data can not be simply considered as a linear one. To solve this, a kernel sparse representation classifier (KSRC) is proposed by mapping the samples into kernel space via kernel tricks to capture their nonlinear structure [24], which makes the data easier to group. Gan L et al. present a multiple-feature kernel sparse representation classifier (MFKSRC) to improve the classification effect of the model by combining the kernel method with SRC [25]. As the research progresses, they found that only using a single kernel function cannot be reflected in classification. For avoiding the above mentioned issues, Dan Li et al. propose a novel adaptive kernel sparse representation method based on multiple feature learning (AKSR-MFL) with multi-kernel functions [26]. Sample $x$ and dictionary $D$ are converted into kernel space to construct their optimization model as fellows.

2.3 New style of SRC research

Its time complex is so high that $\Phi (.)$ converts the high-dimensional $x$ and $D$ to a high-dimensional kernel space in Eq. (6). For high performance, some scholars focus on the correlation between similar samples, especially, their approximate linear relations in low-dimensional subspaces. Chen et al. are interested in the similarity of neighbor samples and unveil two studying paths [27]. One is that they pay close attention to the contextual information that comes from a linear combination of four neighbors of a test when obtaining sparse coefficient vectors. Another is that they exploit the correlation between neighboring tests to construct an adopting joint sparsity model, where tests in a small neighborhood are assumed to be represented by a few common training samples. The second path is named joint sparsity representation.

Joint sparsity representation is a prominent direction for SRC research now. Zhang et al. assume that all neighbors of the central test share the same atoms and draw joint sparse representation classification(JSRC) [28]. And Peng et al. improve its performance in dictionary construction and sparse representation to set up local adaptive joint sparse representation (LAJSR) [29]. Recently, Zhang et al. expand the neighbor sample to a 7 $\times$ 7 window and assign different weights to its neighbor sample to realize weight joint sparse representation classification [30]. It constructs the joint sample matrix $X = \left [x_1 \ x_2 \cdots x_{49} \right ]$ of the 7 $\times$ 7 window, whose first column is the test located in the central test of window and the rest of the columns are its neighbors that are placed randomly in $X$. It is named as nonlocal weight joint sparse representation classification (NLW-JSRC) because the window of neighbor samples is extended to 7 $\times$ 7. So, NLW-JSRC is described as fellows.

Solving the optimization objective of NLW-JSRC, the test $x$ can be labeled by the minimal residual.

There are non-uniform regions and the adjacent samples in a window may be usually imaged by different features in UAV photographs, which contradicts the correlation assumptions of JSRC. In addition, the coefficients are different, and samples in a window are fitted by the same atoms. JSRC needs to be improved.

And it is undeniable that RGB images can form a high-dimensional space after extracting features, and sparsity also exists within any high-dimensional space. However, there is still a vacancy in SRC for cropland delineation with the high-dimensional data extracted from RGB photographs. Thus, we focus on the SRC method of cropland delineation from RGB UAV photographs.

3. Proposed methodology

3.1 Framework of the methodology

Because there is no complete spectral information in RGB UAV photographs, extracting feature information to construct an original high-dimensional dictionary is the primary task of sparse representation classification. Then the original dictionary is trained by online dictionary learning for sparse coding (ODL) [31] and the atoms are obtained to compose classification space and a multi-feature association dictionary (MFD). After that, the neighboring samples of the same feature structures are adaptively found based on a test via a correlation operator to make up a window. Finally, a constraint term is built through the similarity relationship of neighbors in the window to reconstruct the optimization objective of SRC. Leveraging the aforementioned ideas, the framework of our algorithm, multi-feature sparse representation based on adaptive graph constraint (AMFSR), is shown in Fig. 1.

 

Fig. 1. Framework of AMFSR.

Download Full Size | PDF

3.2 Multi-feature association dictionary

The relatively low number of spectral bands means that algorithms cannot obtain sufficient information from RGB aerial images, so the mission of its delineation remains challenging. To get more characteristic information from the three channels, we extract the color characteristics of each channel in different color spaces and settle for 12 characteristics to reflect the original spectrum from different perspectives. The features are combined to replace the original RGB space, which forms a new multi-feature space for multi-feature cropland identification.

The features are highly correlated with each other, it is not reasonable to establish a sparse representation classifier in each feature space, independently. Moreover, the dimension of feature vectors extracted from each feature space is different. Forcing the feature vectors to the same dimension will lose information and lead to misclassification [32]. So the multi-feature association dictionary (MFD) is propose to combine the sparse representation models of diverse features into an SRC mode. We describe the i-th feature sub-dictionary as $D_{i} \left (i = 1, \ldots,N\right )$. $N$ is the number of features and it is set to 12 in this paper. Relevantly, sample $x$ is defined as a vector $[x_1 \cdots x_i \cdots x_N ]^T$ that consists of feature data $x_i$. The i-th feature of the m-th class is denoted as $D_{i,m}(m=1, \ldots,M)$. $M$ is the number of sample categories and $D_i=\left [D_{i,1} \cdots D_{i,m} \cdots D_{i,M} \right ]$.

Thus, multi-feature association sparse representation (MF-SR) for a single test can be described as fellows.

3.3 Shape-adaptive window

Descriptions of the hills and mountains are different in UAV photographs. Cropland is scattered and its shape is irregular. Consequently, the algorithms using a rectangular window, such as NLW-JSRC, to obtain spatial structure are invalid. The irregular shape windows of similar spatial structures need to be adaptively constructed, which satisfies that the samples in a shape window are similar or the same class.

Drawing support from the idea of breadth-first search, the algorithm for constructing the shape window is designed. The algorithm is a searching algorithm. A test block (such as a cropland block) is regarded as a primary center sample $I_{center}$ and the starting point of the algorithm. A neighbor $I_j$ of $I_{center}$ is searched, and it will be added to window $T$ if the neighbor $I_j$ satisfies the conditions that it has not been searched and has a strong correlation with the center sample $I_{center}$. This process is iteratively executed as Fig. 2.

 

Fig. 2. Process to build an adaptive window of a test in Algorithm 1.

Download Full Size | PDF

For controlling size of a window and ensuring that the samples in a window are similar, a similarity measure $\omega$ is defined as fellows.

Equation (11) manifests distance between a center sample and its neighbor is smaller, and $\omega$ is bigger. Meanwhile, the similarity of a center sample and its neighbor is higher, and $\omega$ is bigger too. Defining only the $\omega$ measure cannot control the growth of the window, and a threshold $\xi$ needs to be set Newly added neighbors will be changed for the center sample and the above operations are performed iteratively until $\omega < \xi$ . A shape window will be built after all iterations based on a test block are done.

The process of building a shape window is summarized as Algorithm 1.

Algorithm 1. Adaptive window construction algorithm based on breadth-first search

View Table | View all tables in this article

3.4 AMFSR

3.4.1 Improving MF-SR with adaptive graph constraint

After window T is built, samples in it are akin and share similar sparse patterns. However, the characteristic values of each sample are uneven, which means the representation coefficients of the same atom for each sample are different. To limit the difference of their coefficients, the weight that is calculated with Eq. (11) is introduced to construct an adaptive graph constraint and improve the optimization objective of MF-SR, which is named multi-feature sparse representation based on adaptive graph constraint (AMFSR). Thus, the optimization objective of AMFSR is established as follows.

A regularization term, $\sum _{i,j}\left \|\alpha ^{i}-\alpha ^{j}\right \|\omega _{i,j}$, is implanted into Eq. (12). When $\left \|\alpha ^{i}-\alpha ^{j}\right \|$ is bigger, a smaller $\omega _{i,j}$ reconciles that $\sum _{i,j}\left \|\alpha ^{i}-\alpha ^{j}\right \|\omega _{i,j}$ is smaller. When $\omega _{i,j}$ is smaller, the difference $\left \|\alpha ^{i}-\alpha ^{j}\right \|$ is passed. The reason is that $x^i$ and $x^j$ are 2 different samples and their coefficients should be unequal as well. When the optimization objective is minimized, this regularization term satisfies the requirement that the coefficient is various based on sharing the same atoms.

3.4.2 Solving the optimization problem

(1) Matrix form of Eq. (12)

Generally, $\left \|\alpha ^{l}\right \|_0$ will be converted to $\left \|\alpha ^{l}\right \|_1$ for solving each $\alpha ^l$ in Eq. (12) by the feature-sign search algorithm [33]. However, it is not suitable for large-scale matrix operations. For ease of computation, a Laplace matrix $L$ is defined for each test.

$L$ is a measure based on feature differences and spatial distances. So, Eq. (12) is converted into matrix form.

Then, in each sub-dictionary, the residual difference between the reconstructed data of the sample and the original data is used to realize the classification.

(2) Solving $A$ of Eq. (14)

Because samples $X$ can be fitted by the dictionary $MFD$ and it is $X = MFDA$, the following process holds true.

To define $\widetilde {A} =A^T (AA^T )^{-1}$, thus$MFD=X\widetilde {A}$ and the objective function is modified as fellows.

The objective function of Eq. (17) was derived and its derivation is set to zero. The optimal solution is

According to the matrix operation is easy to draw the relationship between $\widetilde {A}$ and $A$, their relationship proof is

Equation (19) cannot satisfy the constraint of $\lambda \left \|\widetilde {A}\right \|_{row,0}$, which can cause that analytical solution may not be sparse. The idea is based on [33], a greedy algorithm is re-designed to approximately solve the objective Eq. (17). Firstly, the correlation matrix $W$ and window T are obtained with Algorithm 1 for each test, and the times h of iteration is set to 1, and the optimal dictionary $D_0$ is $\Phi$ and $R_{h-1}$ is initialized to the samples in window T. Each atom $d_k$ in the dictionary $MFD$ is sequentially injected into the following Eq. (20).

At each iteration, a suitable atom $d_\mu$ is selected to add into the optimal dictionary $D_h$, which can be best matched to the current residual $R_{h-1}$. So $D_h$ is

When current $D_{h}$ is settled in one cycle iteration, $\widetilde {A}_{h}$ is solved with Eq. (18) and $A_{h}$ is obtained with Eq. (19). The residual $R_{h}$ is updated with the following formula.

Then, the next iteration is executed until the times of iteration h satisfy the sparsity parameter H. The reconstruction coefficient $A$ is $A_H$ and Atoms in the optimal dictionary $D_h$ are greedily selected, which can approximate the original sample matrix of window T.

(4) AMFSR

Above process is summarized as Algorithm 2.

Algorithm 2. AMFSR

View Table | View all tables in this article

3.5 Time cost analysis

There are two main steps in AMFSR. The first is to build an adaptive window for each test. Its process is to find out all satisfying samples from the near neighbors of the test with breadth-first search and similarity threshold. So, its time complexity is O(2N-1) according to the time complexity of breadth-first search and N is the number of samples that enter into the adaptive window. Its meaning is that a sample search is performed 2N-1 times. The second is to reconstruct the spare coefficients. Its loop body is to optimize the dictionary and solve the spare coefficients. The sparsity parameter H determines the times that its circle is executed. Its time complexity based on the loop is O(H).

For an Image with $L\times W$ and size of test is $l\times l$, the number of tests will be $(L\times W)/(l\times l)$. Therefore, time cost of AMFSR is to set up $(L\times W)/(l\times l)$ adaptive windows and the cycle to reconstruct the spare coefficients is done $(L\times W)/(l\times l)$ times.

4. Experimental results and analysis

4.1 Experimental area and data collection

This research was in the Bishan and Hechuan districts located in Chongqing, China. The topography of the study area is mostly mountainous and hilly and the land cover type of them is agricultural. The cropland area of the Bishan district is about 281.8601 square kilometers. Hechuan is known as the ’Grain Silo of Bayu’, whose cropland area is about 976 square kilometers.

In 2022, the data sets are captured from rural areas in the Bishan and Hechuan districts by UAV. We collected RGB images using the Zenith P1 camera mounted on a DJI M300RTK. The shooting parameters are in Table 1.

An overview of the experimental area for this study is shown in Fig. 3. And 5 RGB images were randomly selected to verify the performance of the algorithm.

 

Fig. 3. Overview of the experimental area of this study. (a) The experimental areas are located in the Bishan and Hechuan districts of Chongqing, China. (b) UAV photograph of the experimental area in Hechuan. (c) Data captured in Hechuan. (d) UAV photograph of the experimental area in Bishan. (e) Data captured in Bishan.

Download Full Size | PDF

Data 1: Its size is $620\times 1240$ pixels and it is extracted from Fig. 3 (d).

Data 2: Its size is $620\times 1240$ pixels and it is extracted from Fig. 3 (d).

Data 3: Its size is $360\times 780$ pixels and it is extracted from Fig. 3 (b).

Data 4: Its size is $400\times 780$ pixels and it is extracted from Fig. 3 (b).

Data 5: Its size is $400\times 780$ pixels and it is extracted from Fig. 3 (b)

4.2 Experimental design and environment

4.2.1 Experimental design

To verify the effectiveness of the proposed algorithm, six experiments were designed.

Experiment 1: It is the experiment of selecting AMFSR parameters, which are sparsity $H$ and similarity threshold $\xi$. AMFSR identifies cropland with Data 1 to compare their recognition accuracy under the different parameters.

Experiment 2: It is the experiment of selecting sample sizes. AMFSR identify cropland with the Data 1, Data 2 and Data 3 to compare their recognition accuracy under the different sample sizes.

Experiment 3: It is a comparative experiment on single-feature dictionaries and a multi-feature association dictionary. AMFSR identifies cropland with Data 1 and Data 2 to compare their recognition accuracy under the different dictionaries.

Experiment 4: It is an accuracy comparative experiment with Data 1 between AMFSR and the comparative algorithms that involve SVM [6], LRC [9], KNN, SP, DeeplabV3+,and NLW-JSRC [30].

Experiment 5: It is the repeating experiment of Experiment 3 and Experiment 4 with Data 3, which tests the adaptability of AMFSR.

Experiment 6: It is a generalization ability experiment of AMFSR. AMFSR is trained by Data 3 to identify the cropland in Data 4 and Data 5, and their recognition accuracy is compared.

4.2.2 Experimental environment

This experiment is based on Intel Xeon silver 4114 CPU @ 2.20GHz (2 processor), 64GB memory, NVIDIA Titan V graphics card, windows 10 professional workstation 64 bit, Python 3.6.8, and PyCharm 2021.1.1(Professional Edition).

4.3 Experiment results and analysis

4.3.1 Analysis of parameters

Setting different sparsity $H$ and similarity threshold $\xi$ parameters, and the overall accuracy (OA) of Experiment 1 is shown in Fig. 4.

 

Fig. 4. OA of Experiment 1 with the varying $H$ and $\xi$ parameter. (a) OA based on the varying $H$ parameter. (b) OA based on the varying $\xi$ parameter.

Download Full Size | PDF

Figure 4 shows that OA increases along with the increase of $H$ when $\xi$ is fixed and $H$ is relatively small. When the sparsity parameter $H$ increases to a certain extent and the atomic number satisfies the requirement that the sample can be reconstructed by the atom, OA of the recognition task will arrive at the highest point. Continuing to increase $H$ beyond the optimal value, OA of the recognition task cannot continue to be improved and even will be decreased due to the introduction of atoms from different classes.

When $H$ is fixed and $\xi$ increases, more similar samples are selected to constrain the test, which improves the accuracy of cropland identification. Then $\xi$ continues to go up, and the adaptive window shrinks to exclude the more relevant samples, which eventuates misclassification and OA of AMFSR will be decreased. Figure 4 indicates that $H=2400$ and $\xi =0.95$ are the better parameters. So, they are chosen as parameters for subsequent experiments.

4.3.2 Results and Analysis of Experiment 2

Experiment 2 is designed to explore the effect that is caused by the sample block size on iteration time and recognition accuracy. Sample sizes of $5\times 5$ px, $10\times 10$ px, $15\times 15$ px, $20\times 20$ px, $25\times 25$ px, $30\times 30$ px, $50\times 50$ px, and $100\times 100$ px will be taken, respectively. The image is segmented into samples of different sizes, then 10% of the samples is the training samples and the rest are test. AMFSR will be executed with the different size samples and their OA and iterations are shown in Fig. 5.

 

Fig. 5. The Impact of different size samples. (a) OA of different size samples. (b) The iteration time of different sample sizes.

Download Full Size | PDF

Figure 5 presents that OA tends to increase as the sample size increases, when the sample size of AMFSR is smaller than 20$\times$20px. At this condition, the OA is mainly affected by the quantity of features included in a training sample. When the sample size is larger than 20$\times$20px, OA reduces along the sample size increase. The iterating times of AMFSR will be decreased while the sample size increases. The highest recognition rate is achieved for identifying cropland with a sample size of 20$\times$20px, whilst the iteration of AMFSR is done 1922 and 1035 times in our experiment respectively.

More iteration times of AMFSR are required for excessively small size of the samples. The larger the size of the sample is set and the fewer samples can be segmented from the image, thereby the iteration time is reduced. However, as the sample size is increased, the different features will be divided into a block, which leads to a decrease in OA of the AMFSR. Therefore, the block of 20$\times$20 px is the better sample and it is chosen as the size of subsequent experiment samples.

4.3.3 Results and analysis of experiment 3

The histograms of RGB, HSV, and Lab as the dictionary of RGB (RGB-D), the dictionary of HSV (HSV-D), and the dictionary of Lab (Lab-D) are extracted respectively. The multi-feature association dictionary (MFD) is obtained by associating the features of the three color spaces. AMFSR recognizes cropland with these different dictionaries and the results are shown in Fig. 6 and Fig. 7.

 

Fig. 6. The image results of Experiment 3 with different dictionaries on Data 1. (a) Original image of Data 1. (b) Label image of Data 1. (c) MFD. (d) RGB-D. (e) Lab-D. (f) HSV-D.

Download Full Size | PDF

 

Fig. 7. The image results of Experiment 3 with different dictionaries on Data 2. (a) Original image of Data 2. (b) Label image of Data 2. (c) MFD. (d) RGB-D. (e) Lab-D. (f) HSV-D.

Download Full Size | PDF

In Table 2, Acc_Cro = (Cro+)/( Cro_all+ Oth2Cro). Cro+ is number of samples that cropland in data sets is correctly identified with AMFSR. Cro_all indicates number of samples that cropland in data sets Oth2Cro means number of samples that non-cropland in data sets is identified as cropland with AMFSR. Acc_Oth is defined as (Oth+)/( Oth_all+ Cro2Oth). Oth+ describes number of samples that non-cropland in the data sets is correctly identified with AMFSR. Oth_all and Cro2Oth are similar to Cro_all and Oth2Cro respectively. And Acc_Cro and Acc_Oth are same in subsequent experiments.

Table 1. The shooting parameters.

View Table | View all tables in this article

Table 2. Recognition Accuracy of AMFSR with different dictionaries on Data 1.

View Table | View all tables in this article

Acc_Of_Cro of AMFSR that exploits RGB-D is only 3.8845%, while the recognition accuracy can reach 83.6000% with MFD in Table 2. For the cropland of Data 2, Acc_Of_Oth of AMFSR with Lab-D is only 9.5859%, while the recognition accuracy of the classifier can increase to 80.7531% with MFD. Kappa of AMFSR with MFD are all greater than 0.8 in the above experiments, and their accuracies of identifying cropland and others are almost identical. The aforementioned result proves that the joint dictionary of multiple features improves recognition accuracy, which is shown in Table 2 and Table 3.

Table 3. Recognition Accuracy of AMFSR with different dictionaries on Data 2.

View Table | View all tables in this article

4.3.4 Results and analysis of experiment 4

The image results of Experiment 4 are shown in Fig. 8.

 

Fig. 8. The image results of Experiment 5 with AMFSR and its Comparison algorithms on Data 3. (a) Original image of Data 3. (b) Label image of Data 3. (c) AMFSR. (d) NLW-JSRC. (e) DeeplabV3+. (f) SP. (g) SVM. (h)LRC. (i)KNN.

Download Full Size | PDF

The parameters of AMFSR are $\xi =0.95$ and H=2400. The parameters of NLW-JSRC algorithm are $H=2400$ and the window size is $7\times 7$. The sigmoid function is chosen as the kernel function in SVM. The L2 regularization with coefficient $\lambda =0.01$ is used in LRC. The parameter of KNN is 3. The backbone of deeplabV3+ is deeplabv3_resnet50. Their parameters are same in subsequent experiments as well.Experiment 4 is done ten times repeatedly and averages of their recognition accuracies are compared in Fig. 9.

 

Fig. 9. Recognition Accuracy of AMFSR and comparison algorithms on Data 1 with different atom numbers. (a) Acc_Cro of AMFSR and comparison algorithms. (b) Acc_Oth of AMFSR and comparison algorithms. (c) OA of AMFSR and comparison algorithms. (d) Kappa of AMFSR and comparison algorithms.

Download Full Size | PDF

As shown in Fig. 9, the number of atoms can affect OA. The recognition accuracy of the seven algorithms is gradually improved along with increasing the number of atoms. AMFSR, NLW-JSRC and DeeplabV3+ have significant recognition results, and DeeplabV3+ is superior to the other compared algorithms. The highest accuracy of DeeplabV3+ is around 80% and the overall accuracy for AMFSR is 86.0810%. Even compared with NLW-JSRC and DeeplabV3+, the accuracy of the proposed method is 6.2076% and 6.0810% higher.

In the single-image recognition task, OA of DeeplabV3+ is slightly lower than OA of AMFSR. The performance of DeeplabV3+ for large crop-area is comparable to the performance of AMFSR, and it is worse than AMFSR for dispersed and fragmented areas. Due to NLW-JSRC considering the connections of the adjacent samples, its performance is superior to SVM. In contrast to NLW-JSRC, AMFSR not only performs similarity-based selection of near-neighbor samples but also allows the coefficients of similar samples to be varied. So OA of AMFSR is greater than NLW-JSRC.

4.3.5 Results and analysis of experiment 5

To test the adaptability of AMFSR, Data 3 is utilized to repeat Experiment 3 and Experiment 4. The parameters are same as Experiment 3 and Experiment 4. First, AMFSR with different dictionaries is applied to recognize cropland and the results are shown in Fig. 10 and Table 4. Acc_Oth of AMFSR with RGB-D is only 1.3890% on Data 3, while the recognition accuracy of the MFD-based algorithm can reach 85.5478%. The accuracy of Data 3 is slightly higher than that of Data 1 and Data 2.

 

Fig. 10. The image results of Experiment 5 with different dictionaries on Data 3. (a) Original image of Data 3. (b) Label image of Data 3. (c) MFD. (d) RGB-D. (e) Lab-D. (f) HSV-D.

Download Full Size | PDF

Table 4. Recognition Accuracy of AMFSR with different dictionaries on Data 3.

View Table | View all tables in this article

Then, the six algorithms in Experiment 5 are compared with AMFSR and Experiment 5 is done ten times with Data 3. Their image results and averages of accuracies are presented in Fig. 11 and Fig. 12 repeatedly.

 

Fig. 11. The image results of Experiment 5 with AMFSR and its Comparison algorithms on Data 3. (a) Original image of Data 3. (b) Label image of Data 3. (c) AMFSR. (d) NLW-JSRC. (e) DeeplabV3+. (f) SP. (g) SVM. (h)LRC. (i)KNN.

Download Full Size | PDF

 

Fig. 12. Recognition Accuracy of AMFSR and comparison algorithms on Data 1 with different atom numbers. (a) Acc_Cro of AMFSR and comparison algorithms. (b) Acc_Oth of AMFSR and comparison algorithms. (c) OA of AMFSR and comparison algorithms. (d) Kappa of AMFSR and comparison algorithms.

Download Full Size | PDF

As the number of atoms increases, the recognition effect of AMFSR improves. When the atom number reaches the number of samples that can be reconstructed, the recognition accuracy of AMFSR is better than its comparison algorithm. OA of AMFSR is 91.4432%, which is 18.5241% higher than NLW-JSRC. The experiment proves that the AMFSR is more effective in recognizing regions of concentrating cropland and its adaptability is strong. DeeplabV3+ is more accurate than the other comparison algorithms. However, based on our single-image cropland recognition task, its recognition accuracy is also lower than AMFSR.

4.3.6 Results and analysis of experiment 6

The left subgraph (Data 4) and right subgraph (Data 5) of Data 3 are chosen as the data for the generalization experiment. AMFSR_1 based on AMFSR is trained by the training samples from Data 3 and AMFSR_2 based on AMFSR is built by the training samples from Data 3 and its subgraphs. The cropland in Data 4 and Data 5 is recognized with AMFSR_1 and AMFSR_2 respectively, and their recognition accuracy compared to evaluate the generalization ability of AMFSR.

The image results with AMFSR_1 and AMFSR_2 are shown in Fig. 13 (a) and Fig. 13 (b). Their recognition accuracy is listed in Table 5.

 

Fig. 13. The image results of Experiment 6 with different trained AMFSR on the Data 4 and Data 5. (a) Recognition results with AMFSR_1. (b) Recognition results with AMFSR_2.

Download Full Size | PDF

Table 5. Recognition Accuracy of different trained AMFSR on Data 4 and Data 5.

View Table | View all tables in this article

Figure 13 and Table 5 show that OA of AMFSR_2 is 89.3618% on Data 4 and it is 4.7359% higher than that of AMFSR_1. OA of AMFSR_2 is 90.7418% on Data 5 and it is 1.9696% higher than that of AMFSR_1. Several conclusions can be made from Fig. 13 and Table 5. The cropland features of Data 5 are very similar to those of Data 3, so the generalization ability of Data 3 trained mode is superior in Data 4. While the cropland features of Data 4 are different from Data 3, thereby generalization ability of the model is poor. AMFSR_2 incorporates characteristics of cropland in Data 4, and its OA of Data 4 is significantly improved. Hence, the generalization ability of our proposed AMFSR model depends on the features of training samples in the dictionary. In other words, if the AMFSR model dictionary contains the characteristics of the current cropland, the model can be extended it to identify arable land. Otherwise, the recognition accuracy will be reduced.

5. Conclusion

A new sparse representation classification is proposed for the cropland delineation, which is a multi-feature sparse representation based on adaptive graph constraint (AMFSR). A multi-feature association dictionary is constructed with feature vectors that were extracted from aerial images. Then the adaptive window is established for each test sample, and graph regularization is built and a similarity weight measure is defined. The objective function based on MF-SR is reconstructed to obtain the optimization model of AMFSR.

Because the recognition sample granularity of AMFSR is coarse, a fine-grained algorithm for cropland recognition of RGB aerial images can be further researched and its recognition accuracy is yet to be improved in the future.