1. Introduction

With increasing bandwidth-intensive applications such as big data, cloud services, smart phones and network video, network traffic has grown rapidly [1]. In order to provide agile and affordable on-demand network service to large data flows, it is desirable to develop a cognitive optical network (CON) [2–5]. A CON will have good scalability and interoperability and would be able to provide features like auto-configuration, self-optimization and autonomous network operation through self-learning and cognitive decision-making [2]. A CON is also capable of coordinating transmitters and receivers for real-time dynamic adjustment of the modulation format, line rate, spectrum allocation and other parameters. Managing the transceivers in this way enables us to realize intelligent channel management, bandwidth allocation and other link functions at the network nodes, thereby improving service quality and transmission quality. In a cognitive heterogeneous optical network, the physical layer will have a variety of architectures and hardware and may include different types of optical signals with different modulation formats and symbol rates [5]. It is challenging to come up with a modulation-agnostic optical receiver capable of detecting automatically all types of optical signals since some indispensable digital signal processing (DSP) algorithms used in coherent optical receivers are highly dependent on the received signal’s modulation format. Therefore, in order to implement intelligent optical network management, the key is to develop a universal optical receiver, which can recognize the modulation format information and compensate the channel distortion in an optical fiber transmission by using suitable DSP algorithms. Such a universal receiver will incorporate different DSP algorithms for signals with different optical modulation formats, where the modulation format is identified prior to the modulation-format dependent DSP algorithms such as adaptive equalization, carrier phase recovery and symbol decision.

Modulation format identification has been demonstrated by various techniques. An artificial neural network (ANN) trained with the features extracted from the asynchronous amplitude histograms has been used to identify on-off keying (OOK), differential phase-shift keying (DPSK), optical duobinary (ODB), differential quadrature phase-shift keying (DQPSK), quadrature phase-shift keying (QPSK) and 16-quadrature amplitude modulation (QAM) signals in [6]. A magnitude method was demonstrated to identify OOK, BPSK and QPSK signals in [7]. Asynchronous single channel sampling was utilized to identify the OOK and DPSK signals in [8]. The peak-to-average-power ratio evaluation was proposed to identify the 4/16/64/256-QAM signals in [9]. A deep neural network (DNN) in combination with signals amplitude histograms was applied in [10] to identify 4/16/64-QAM signal. Stokes space mapping assisted by a clustering algorithm was utilized in [11] to classify and identify 4/8/16/32/64-QAM signals. A nonlinear power transformation was calculated to identify BPSK, QPSK, 8-phase-shift keying (PSK) and 16-QAM signals in [12]. A convolutional neural network (CNN) was proposed in [13,14] to detect the eye diagrams of OOK, DPSK and 4-pulse amplitude modulation (PAM) signals and the constellation diagrams of QPSK, 8-PSK, 8/16/32/64-QAM. In the above-mentioned modulation format identifications techniques, the ANN, DNN and CNN techniques are machine-learning algorithms relying mainly on artificial intelligence technology. The key of machine learning is that by collecting a large number of samples, learning and training, it can automate the matching operation relating the parameters and modes.

Compared with other machine-learning algorithms, such as support-vector machine (SVM) with the translation of high-dimensional feature space and maximum likelihood with the feature calculation of probability model, the CNN and DNN algorithms can do the feature extraction automatically. The CNN is frequently applied in the image classification and such an image-based identification technique recently has been used to identify the constellation diagrams of optical modulation signals [14]. Note that signals always suffer various transmission impairments, such as amplified spontaneous emission (ASE) noise and fiber nonlinearities. These cause distortions in the constellation diagrams and the distorted signals in different scenarios may have different constellation diagrams [15]. Using the CNN to process the clock-recovered diagrams will be less subject to transmission impairments than using constellation diagrams.

This paper proposes an intelligent adaptive coherent optical receiver based on CNN and a clustering algorithm. To simplify the identification procedure and increase the noise tolerance, we utilize the CNN to process the clock-recovered amplitude diagrams instead of the phase-recovered constellation diagrams for modulation format identification. To fast track the signal’s changes for accuracy symbol decision and signal distortion compensation, we use a Fast Search and Find of Density Peaks (FSFDP) algorithm to classify the symbols and develop a set of intelligent adaptive clustering algorithms (IACA) for equalization and demodulation. We find that our proposed intelligent adaptive coherent optical receiver can simultaneously identify the modulation format and compensate for signal impairments.

The rest of the paper is organized as follows. Section 2 introduces the intelligent DSP algorithms based on machine learning. This includes the CNN algorithm for modulation format identification and the IACA for intelligent adaptive equalization. Section 3 simulates the proposed intelligent adaptive coherent optical receiver. We explain the implementation procedure and evaluate the proposed high performance receiver by looking at its performance with modulation format estimation and BER calculation of 4/8/16/32/64-QAM signals with phase-rotated constellation diagrams and laser linewidth compensation. Section 4 experimentally demonstrates the proposed receiver and evaluates the performance improvement for the impaired signals when there are fiber nonlinearities and modulator I/Q skew. Finally, Section 5 concludes the paper.

2. Intelligent DSP algorithms based on machine learning

In this section, we introduce two intelligent DSP algorithms based on machine learning. In a CON, it is necessary to allocate adaptively the available network resources for spectral maximization and energy efficiency. This requires an intelligent coherent optical receiver, which is autonomously aware of the received signals’ modulation format without any prior knowledge. We, therefore, introduce the CNN algorithm to identify the modulation format before the carrier-phase-recovery algorithm. After the carrier-phase-recovery, we obtain the raw constellation diagrams of the signals with the distortions from the ASE noise and fiber nonlinearities in the fiber transmission. We apply the FSFDP clustering algorithm to classify symbols accurately in their respective constellation categories and propose the IACA equalizer to demodulate and decide symbols in order to mitigate the signal impairments.

Figure 1 illustrates the proposed intelligent adaptive coherent optical receiver. The acquired data from the hardware coherent optical receiver is first processed by the modulation-format-transparent DSP algorithms of chromatic dispersion (CD) compensation and clock recovery. The CNN is then used to identify the modulation format and extract the K value (K = M in the M-QAM signals). After that, the corresponding carrier-phase-recovery algorithms are performed for the modulation-format identified signals. Next, we preprocess the carrier-phase-recovery data by normalizing them based on the average energy and utilize the FSFDP algorithm to classify and label the data. If there are cluster halos without the labels, the k-nearest neighbors (KNN) algorithm is applied to label them. Finally, the BER is estimated by comparing the obtained labels with the pre-stored labels of the data.

 

Fig. 1 The flow chart of intelligent adaptive coherent optical receiver.

Download Full Size | PDF

2.1 CNN algorithm for modulation format identification

The CNN belongs to a branch of deep learning, which can naturally learn from deep sample images to obtain deeper image features without the need for artificial dominant feature extraction and expression. It generally consists of a convolution part and a fully connected layer part. The convolution part typically has two types of layers, i.e. the convolutional layer and the pooling layer, where the output of the previous layer can be referenced as a feature map. The convolutional layer is obtained by convolving several convolutional kernels.

The generated feature map can then be written as:

The essence of pooling is down sampling. Pooling reduces the dimension of the feature maps and thus effectively reduces the required parameters of the subsequent layers. Moreover, pooling has a translation invariance feature implying that even when the input pixels in the field undergo slight displacement, the output of the pooling layer stays unchanged. Therefore, the CNN is more robust to some disturbance and can avoid overfitting. Thus, pooling greatly reduces the complexity of parameters’ calculation to improve the statistical efficiency of the network.

Here, we explain the principle of CNN-based modulation format identification for QPSK, 8/16/32/64-QAM signals in Fig. 2. We utilize the clock-recovered figures as the original color images with the size pixel of 800 × 800. To save computation resource, the color images are transformed into gray images, which are then compressed to 28 × 28 pixels. The 28 × 28 pixels are used as the input of the CNN, which is convolved with six 5 × 5 neurons kernels to yield the feature maps composed by six 24 × 24 pixels, as shown by the C1 layer in Fig. 2. The next is a pooling layer with 2 × 2 pixels to subsample the C1 feature maps and six 12 × 12 pixels are obtained in the S2 layer in Fig. 2. The following is another convolutional layer to convolve the output of the pooling layer S2 with six 5 × 5 neurons kernels, where twelve 8 × 8 pixels are achieved, as shown by the C3 layer in Fig. 2. After that, the last pooling S4 is performed for sub-sampling to yield twelve 4 × 4 pixels, which is unwound into the final F5 with 192 values in Fig. 2. Finally, F5 is fully connected with the output layer with 5 nodes for modulation format identification.

 

Fig. 2 Schematic diagram of the CNN-based modulation format identification.

Download Full Size | PDF

The trial is necessary and hundreds of iterations are applied to optimize the parameters of the CNN. The errors transfer reversely in the convolution layer and the pooling layer. In the convolution layer, the error is transferred from the following pooling layer, which is a reverse procedure of the pooling. Actually, the input of the convolution layer has been processed by the sigmoid function and therefore the error from the pooling layer has been processed by the differential of the sigmoid function. In the pooling layer, the error is transferred from the following convolution layer, which is a reverse procedure of the convolution, i.e. de-convolution of the convolution layer’s error.

2.2 IACA for intelligent adaptive equalization

After the modulation format is identified, the raw constellation diagrams of signals can be obtained by choosing the corresponding carrier-phase-recovery algorithms. We then classify the symbols in these raw constellation diagrams, which would be extremely useful when the signals are distorted and have a messy constellation distribution. We had proposed the density-centroid-tracking K-means algorithm for symbol classification in our previous work [15]. In this work, in order to improve the nonlinear classification accuracy further of the K-means algorithm on the distorted signals, we utilize the FSFDP clustering algorithm, which can complete the clustering by fast search and find of density peaks instead of using random iterations [16]. The FSFDP algorithm assumes that the cluster centers with higher density are often surrounded by lower local points and have a relatively large distance from these points. Therefore, the proposed method could identify the genuine cluster centers regardless of their shape and size without any other prior knowledge [16].

Here, we simply depict the FSFDP clustering algorithm. We assume that the data set of N categories is composed of n data, represented by S = ${\left\{X_i\right\}}_{i=1}^n$ with the labels I_S = {1,2, …, N}. For each data point X_i in the data set S, we need to calculate two quantities, its density ρ_i and its distance δ_i away from the points of higher density. We use the Gaussian kernel function as the density ρ_i of point i, written as:

Suppose ${\left\{q_i\right\}}_{i=1}^n$ represents a descending order of ${\left\{{\rho }_i\right\}}_{i=1}^n$, the distance δ_i can be defined as:

Once the cluster centers are successfully obtained, the remaining data points are assigned to the corresponding cluster according to the nearest distance to the cluster center. Furthermore, in order to separate the noise in the cluster, a border region is defined according to its density ρ_𝑏 for each cluster. The higher density data points (higher than ρ_𝑏) in each cluster are considered as the cluster core and the other points are considered as the noise, i.e., the cluster halos. When the samples are classified into the different constellation clusters, the next significant step is to make a precise decision for data recovery. The proposed intelligent adaptive clustering equalization demodulation technique based on the FSFDP algorithm has the following procedure s.

Step1: Data preprocessing. We normalize the carrier-phase-recovered signal based on the average energy, expressed as:

Step 2:Distance d_ij calculation. We calculate the Euclidean distance d_ij in data set P_i (d_ij = d_ji, i<j) and arrange the calculated distance d_ij in an ascending order. The distance d_c denotes the cutoff distance, expressed as:

Step 3:Density ρ_i and distance δ_i calculation. According to Eq. (2) and Eq. (3), we calculate the data point X_i’s parameters to determine the cluster centers, i.e., its density ρ_i and its distance δ_i away from the points of higher density. The number of the cluster centers is M for the M-QAM signals. We yield the product γ_i of the density ρ_i and the distance δ_i

We use the 16-QAM signal as an example to illustrate these parameters, as shown in Fig. 3. Figure 3(a) shows the constellation diagram of the 16-QAM signal, where the different colors represent different clusters and the red circles are the centers of the clusters. The decision graph of the calculated density ρ versus distance δ is plotted in Fig. 3(b), where the 16 points falling within the red rectangle identified as the cluster centers. The calculated γ is plotted in a descending order in Fig. 3(c), where the first 16 points are chosen as the cluster centers. The results in Fig. 3 are as per the description in the previous subsection.

 

Fig. 3 (i) Constellation clusters of the 16-QAM signal classified by the IACA (ii) the decision graph ρ vs δ (iii) the calculated γ in a descending order of the 50 data in (i).

Download Full Size | PDF

Step 4: Data labeling. A good classification of the constellation clusters is not useful unless we can label the data correctly. In this work, we calculate the distances between the cluster centers and the conventional QAM decision points and utilize the nearest distance principle of the K-means algorithm to label the cluster centers [15]. Figure 4 shows the labelled results based on the blind K-means algorithm and the training-sequence assisted K-means algorithm, where the black points are the conventional QAM decision points and the red points are the cluster centers without labels. Then according to the clustering results in Step 3, the labels of the cluster centers are used as that of the corresponding clusters’ data. For the distorted signals and the higher-order modulation signals with lower noise tolerance, the training-sequence assisted labelling method has simpler architecture and fast calculation speed with the complexity of O(nkt), where n is the number of the testing samples, k is the modulation orders (k = M in the M-QAM signals) and t = 1 is the iteration time. Moreover, only 1%-2% training sequences are enough for accurate symbols’ labeling.

 

Fig. 4 (i) Blind K-means labeling (ii) Training-sequence assisted K-means labeling.

Download Full Size | PDF

Step 5: Cluster halos labeling. As described in the FSFDP algorithm, each constellation cluster is composed of the cluster core and the cluster halo. Figure 5(i) and (ii) show the constellation cluster of the 64-QAM signal with much noise and that of the 64-QAM signal with less noise, where colored points represent the cluster cores, the black points represent the cluster halos and the red circles are the achieved cluster centers based on the FSFDP algorithm. The cluster halos are far from the cluster centers and only appear in the noisy signal, as shown in Fig. 5. For the noisy signal, it is also important to label the cluster halos. In this work, we label the cluster halos based on the weighted-voting KNN algorithm [17,18]. The complexity of the FSFDP algorithm without the need of iteration is dominated by the complexities of the density and distance calculation of O(n²) and the distance sequencing of O(nlogn). Thus, the complexity of the algorithm is O(n²).

 

Fig. 5 Constellation diagrams of the 64-QAM signal (i) with more noise (ii) with less noise.

Download Full Size | PDF

Step 6. BER estimation. We calculate the BER by comparing the achieved labels of the constellation clusters including the cluster cores and the cluster halos with the pre-stored labels.

3. Intelligent adaptive receiver and simulation results

Figure 6 shows the proposed receiver architecture based on the CNN modulation format identification and intelligent adaptive clustering algorithm equalization. We first apply the modulation-format-transparent DSP algorithms of CD compensation and clock recovery to acquire the clock-recovered data. Then we utilize the CNN to identify the modulation format for choosing the proper carrier-phase-recovery algorithms and extract the parameters of the cluster centers for the clustering algorithm equalization. After the carrier phase recovery, the raw constellation clusters are classified and labelled, and finally the BER is estimated.

 

Fig. 6 Proposed intelligent receiver with the machine-learning algorithms.

Download Full Size | PDF

Compared with the fiber nonlinearities, the ASE noise has more effect on the clock-recovered amplitude data. Therefore, we use various signals with different OSNR values to verify the proposed scheme. Figure 7 shows constellation diagrams of the 4/8/16/32/64-QAM signals after clock recovery. The OSNRs of the 4/8/16/32/64-QAM signals are varied in the ranges of 8-23 dB, 10-25 dB, 12-27 dB, 15-30 dB and 17−32 dB with 1-dB step, respectively. As shown in Fig. 7, the main difference of the different modulation formats lies in the number of the recovered amplitude level, where 1, 2, 3, 5, and 9 are corresponding to the 4/8/16/32/64-QAM signals, respectively. With the increase of the ASE noise, the amplitude circles of various signals are becoming thicker but the CNN-based technique can still distinguish the differences. However, when the modulation level increases, the higher-order signals are more susceptible to the noise. Therefore, compared with QPSK with the OSNR range of 8-23 dB, the 64-QAM signal can be distinguished successfully only in the OSNR range of 17-32 dB. It should be noted that the proposed technique can still work for 4/8/16/32-QAM signals even when the OSNR is less than 15 dB.

 

Fig. 7 The collected constellation diagram images of 4/8/16/32/64-QAM signals with the OSNR variation range of 8-23 dB, 10-25 dB, 12-27 dB, 15-30 dB and 17−32 dB at 1-dB step.

Download Full Size | PDF

Furthermore, in the CNN amplitude-level-based modulation-format-identification system, 100 constellation diagrams for each OSNR of each modulation format are acquired as the training data. Thus, the training data set is composed of 8000 images, i.e., 100 × 16 (OSNR number) × 5 (modulation format number). In the training processing, it takes 15.6s on the every iteration with the I7 processor and hundreds of iterations are applied to yield good model parameters for high identification accuracy. Similarly, 800 constellation diagram images are acquired for each modulation format with 16 different OSNR values as the testing data and thus the testing data set is composed of 4000 (800 × 5) images in total. It takes 0.4s on each testing image, where each image includes the information of 20000 symbols. The training with the complexity of O(n) is performed prior to the testing with the complexity of O(1) and then the training results are saved. Therefore, the offline training is applied in the CNN to reduce the online computation cost.

For the 4000 testing data set, we measure the accuracies of various modulation format estimation with different epochs, as shown in Fig. 8, where the training data set is completely trained once in an epoch. It is easier to distinguish the 8-QAM signal with two amplitude levels, and therefore 10 epochs are sufficient to achieve 100% accuracy. For 4/16/32/64-QAM signals, more epochs are indispensable to avoid the underfitting issue, where 30 epochs are necessary for the 4/64-QAM signals to achieve 100% accuracy. From Fig. 7, it can be seen that when the OSNR is lower, the 16-QAM signal and the 32-QAM signal have a similar amplitude distribution, resulting in a wrong identification. Therefore, more epochs cannot improve the identification accuracy to 100% for the 16/32-QAM signals. Moreover, in order to save computation resource, the color images are transformed into gray images, which are then compressed to 28 × 28 pixels. The compressed gray images are used as the initial input for the CNN, which have lower resolution and will lead to wrong identifications. Therefore, the uncompressed images with higher resolution may enable us to improve the identification accuracy at the expense of the computation resources.

 

Fig. 8 Accuracy of modulation format estimation versus the number of epoch for 4/8/16/32/64-QAM signals.

Download Full Size | PDF

On the other hand, when the signal is worse, the estimated phase reference is not accurate. This leads to the rotated complex amplitude of the signal shown in Fig. 9. For the lower-modulation-level QPSK and 16-QAM signals, the influence of such a rotation on the BER is not serious, as shown by red and blue square-marked curves. However, for the higher-modulation-level 64/128/256-QAM signals, the phase rotation has a significant effect on the BER performance because the conventional linear symbol decision cannot track the signals’ changes, which leads to greater errors. The improvement obtained by using the proposed IACA is remarkable, as shown by the black, magenta and brown triangles.

 

Fig. 9 BER versus SNR for 4/16/64/128/256-QAM with slight phase rotation. Inserts are the constellation diagrams of 256-QAM signals and the corresponding decision graph of the clustering centers.

Download Full Size | PDF

Furthermore, the higher-order modulation signals are susceptible to the laser linewidth, which introduces severe phase noise. Generally, Kalman filtering is an effective method to compensate the signal impairments caused by the laser linewidth [19]. Figure 10 shows the deteriorated BER performance with the increasing laser linewidth. The proposed IACA outperforms the conventional Kalman filtering, as shown by the curves marked with triangles and squares.

 

Fig. 10 BER versus laser linewidth for 16-QAM signal. Insert is the corresponding constellation diagram.

Download Full Size | PDF

4. Experiment setup and results

The proposed intelligent adaptive coherent optical receiver has also been experimentally demonstrated and the experiment setup is shown in Fig. 11. In Fig. 11, we use the 4/16/64-QAM signal as examples to verify the proposed algorithms. In the transmitter part, the 2/4/8-level pseudo-random binary sequences (PRBS) of 2^15-1 are modulated using the tunable laser diode (LD) at 1550.116 nm with a linewidth of 100 KHz by the I/Q modulator to generate the optical 4/16/64-QAM signals. An erbium-doped optical fiber amplifier (EDFA) followed by a variable optical attenuator (VOA) is used to adjust the launched power into the standard single-mode-fiber (SSMF). Another EDFA followed by another VOA is utilized to vary the input power of the coherent optical receiver, where the free-running tunable external-cavity laser with a linewidth of 100 KHz is used as the local oscillator (LO). The real time oscilloscope used as the analog-to-digital converter (ADC) acquires the data for the DSP algorithms that follow. After chromatic dispersion (CD) compensation and clock recovery, the data is processed by the CNN and the modulation format is identified. Based on this modulation format identification, the corresponding carrier-phase-recovery algorithm is chosen and the signal is recovered. We then use the proposed IACA for making the symbol decisions and estimating the BER.

 

Fig. 11 Experimental setup: LD: laser diode; AWG: arbitrary waveform generator; EDFA: erbium-doped fiber amplifier; VOA: variable optical attenuator; LO: local oscillator.

Download Full Size | PDF

We first tested the CNN modulation format identification using this setup. We acquired 500 images for each modulation signals of the 4/16/64-QAM with the BER range from 2.5 × 10⁻³ to 10⁻⁵, where the signal noise dominated by the ASE. We can achieve 100% identification accuracy for the QPSK and 16-QAM signals and 99.8% identification accuracy for the 64-QAM signals.

Secondly, to see the tolerance of the proposed method to fiber nonlinearities, we measure the identification accuracy and the BER as functions of the signal power launched into the 130-km standard single-mode-fiber (SSMF), as shown by the curves marked with triangles and circles in Fig. 12. When the launched signal power is more than 0-dBm, although the fiber nonlinearities deteriorate the BER (as shown by descending circle-marked curve), the identification accuracy does not change (as shown by the triangle-marked curve) in Fig. 12. It is obvious that the CNN identification is based on the amplitude data and has strong robustness against fiber nonlinearities. The corresponding clock-recovery diagrams are also shown by the insets in Fig. 12. When the launched signal power is higher than −5.19 dBm, we can achieve 100% identification accuracy.

 

Fig. 12 Measured identification accuracy and BER curves of 75-Gb/s 64-QAM signal versus the launched signal power into 130-km SSMF. Insets are the clock-recovered diagrams with the signal power at (a) and (b).

Download Full Size | PDF

Next, we evaluate the BER performance with phase noise. Figure 13 shows the measured BER versus the signal power launched into the 130-km SSMF. The higher signal power leads to more severe phase noise from the fiber nonlinearities and the lower signal power causes more severe ASE noise. This results in an arch-shape curve, as shown in Fig. 13. Support-vector machine (SVM) with high computation complexity of O(n³) is an effective method to solve the problem of nonlinear classification and can effectively improve the bit-error ratio (BER) performance [20, 21]. However, log₂M SVMs are required for M-QAM signals and amounts of computation expense is necessary to store and calculate the larger matrix for the support vectors. The proposed IACA method with 2% training sequences outperforms the conventional SVM method with 10% training sequences, as shown by the square-marked and the triangle-marked curves. The insets in Fig. 13 are the constellation diagrams of the signal with the launched signal power of −7.17 dBm and 4.82 dBm. Compared with the amplitude diagrams in Fig. 12, the constellation diagrams are more susceptible to the fiber nonlinearities. Furthermore, the modulation format information is indispensable prior to the carrier-phase-recovery algorithm. Therefore, the amplitude-level-based CNN identification technique not only provides the necessary information for the followed modulation-format-dependent carrier-phase-recovery algorithms but can also be integrated with the equalization algorithms to mitigate the signal impairment affected by the fiber nonlinearities.

 

Fig. 13 Measured BER curves of 75-Gb/s 64-QAM signal versus the launched signal power into 130-km SSMF and insets are the constellation diagrams with the launched signal power of −7.17 dBm and 4.82 dBm.

Download Full Size | PDF

Finally, the proposed method is a package solution to improve the signals’ performance and independent of the noise sources and categories. For example, for the in-phase (I) and quadrature (Q) skew noise of the modulator, the proposed method also works well, as shown in Fig. 14.

 

Fig. 14 Measured BER versus I/Q phase skew of the modulator. Insets are the constellation diagrams of the corresponding skews.

Download Full Size | PDF

The CNN is a pre-training MFI technique, where the candidate modulation signals are trained and saved to yield the optimum model parameters. In the testing systems, the identification could be “blind.” The CNN-based image identification is suitable for other modulation formats provided these modulation signals are pre-trained and saved. However, in this work, the identification feature is the amplitude circle and the modulation signals with the same amplitude circles cannot be identified unless the additional procedure is inserted. The IACA algorithm is suitable for other modulation formats with various constellation diagrams. Limited by the available experimental components, we only demonstrate the proposed scheme for QAM signals.

5. Conclusions

We develop an intelligent adaptive coherent optical receiver based on the CNN modulation format identification and the clustering symbol decision. The modulation format information is significant for the carrier-phase-recovery algorithm and the clustering symbol decision improves the system performance. We demonstrate the proposed receiver by simulations and through actual hardware-based experiments to show significant system performance improvement. Our approach can mitigate the system impairments because of phase rotation, laser linewidth, fiber nonlinearities and the modulator skew noise. The proposed system is a promising one for use as the physical-layer architecture in cognitive heterogeneous optical networks.