Fast fiber nonlinearity compensation method for PDM coherent optical transmission systems based on the Fourier neural operator

Junling Huang; Anlin Yi; Lianshan Yan; Xingchen He; Lin Jiang; Hui Yang; Bin Luo; Wei Pan

doi:10.1364/OE.511951

1. Introduction

Compacity of optical fiber communication systems has been promoted dramatically by leveraging coherent detection techniques in association with digital signal processing (DSP) techniques and multi-level modulation formats (such as high-order QAM) [1–3]. Recently, high-order QAM combined with probabilistically-shaped (PS) techniques have been widely investigated to narrow the gap between the capacity of optical fiber communication systems and the Shannon limit in terms of optical signal-to-noise ratio (OSNR) as much as possible [4,5]. However, high-order QAM signals, whether uniformly or probabilistically-shaped distributed signals, are susceptible to fiber nonlinearity which limits further extending the transmission distance and increasing capacity per fiber [6–8]. Consequently, fiber nonlinearity compensation or mitigation techniques are likely to emerge as an indispensable component for future high-capacity and long-haul coherent optical communication systems [9–11].

To compensate or mitigate fiber nonlinearity, various DSP-based algorithms have been proposed for coherent optical communication systems [12–14]. Among all the solutions demonstrated in the literatures, digital back-propagation (DBP) arguably is one of the best methodologies concerning accuracy to address optical fiber nonlinearity compensation [15–18]. It works by numerically solving the inverse-propagation nonlinear Schrödinger equation (NLSE) leveraging the split step Fourier method (SSFM). However, the performance of the DBP largely relies on the number of steps per span (StPS) used in the SSFM. As the baud rates and modulation level increase, the number of StPS has to be large enough to achieve effective compensation performance, which would result in prohibitively heavy computational complexity or time consumption [19,20].

Alternatively, neural networks (NNs) based methods have been recently emerging as an attractive solution for fiber nonlinearity compensation or mitigation by learning the nonlinear transfer function directly from data [21,22]. Hitherto, the NNs-based methods can be mainly divided into two categories. One is derived from the NLSE which is dubbed as learned DBP (LDBP) [23,24]. The other is independent of the NLSE which is referred to as black-box function approximators [25]. The idea of LDBP is to exploit the fact that the SSMF has essentially the same functional form as multi-layer NNs, i.e., alternation of linear and nonlinear operators [26,27]. LDBP could achieve transmission impairment compensation performance as well as, or even better than, conventional DBP method in terms of computational complexity and performance metrics [28]. By parameterizing the SSFM, the compensation performance can be improved further. Similar to the conventional DBP, the compensation performance and processing speed of LDBP are also heavily dependent on the StPSs [29]. The black-box equalizer is a purely data-driven method that relies only on received data to mimic the transmission model without any prior knowledge of the link parameters. Therefore, it is versatile and robust enough for different transmission scenarios and can potentially lead to a large reduction in computational complexity or time consumption [30]. Additionally, a black-box equalizer can be extended to include the functionalities of traditional DSP modules, such as compensating impairments caused by the transmitter and receiver [31]. The black-box model can be designed by using deep neural networks (DNNs), convolutional neural networks (CNNs), recurrent neural networks (RNNs), and so on [32–34]. However, some of these methods work for specific modulation formats or waveforms. While, most of these methods remain limited in nonlinearity compensation except for the combination of an extra CD compensation module, e.g. CD compensated by an extra NNs or a traditional DSP module [35]. Consequently, a NNs-based CD and nonlinearity simultaneous compensation technique would be desired in future coherent optical communication systems.

In this work, we introduce a novel fast black-box NNs model based on FNO to compensate CD and nonlinearity simultaneously. Accurate and low complexity optical fiber modeling for single polarization channel coherent optical communication system based on FNO is demonstrated in our previous work [36]. Here, we extend such concepts to channel equalization for the PDM coherent optical communication systems by utilizing FNO with four-channels input to reduce the parameters of model. The performance assessment of the proposed algorithm has been carried out against conventional DBP algorithm using various modulation formats, such as PDM-16/32/64QAM and PDM-PS-16/32/64QAM. Compared to 5StPS-DBP, ∼0.31 dB, ∼0.34 dB, and 0.17 dB Q-factor improvements are achieved for PDM-16/32/64QAM signals over 1600 km, 800 km, and 400 km SSMF, respectively. ∼0.22 dB, ∼0.34 dB, and ∼0.26 dB Q-factor improvements are obtained for PDM-PS-16/32/64QAM signals over 1600 km, 800 km, and 400 km SSMF, respectively. Unlike the DBP-based algorithm compensating the nonlinearity step by step, the FNO-based black-box NLC performs the nonlinearity compensation using only four Fourier layers which are not various with the transmission distance. Therefore, the time consumption can reduce from 6.04 seconds to 1.69 seconds compared to 5StPS-DBP over 1600 km fiber transmission when a central processing unit (CPU) is used. Additionally, compared 1D complex operations in the DBP, the FNO involves a large number of real multi-dimensional matrix operations which is inherently suitable for GPU acceleration. The time is reduced as low as 0.03 second by utilizing a graphic processing unit (GPU). This scheme also demonstrates the capability of generalization ability in terms of launched power and modulation format.

2. Operation principle

2.1 Structure of FNO-based NLC

Figure 1 shows the structure of fiber nonlinearity compensation based on FNO for PDM coherent optical communication system. The whole structure is composed of one lifting operator $\mathrm{{\cal P}}$, n Fourier layers, and one projection operator $\mathrm{{\cal Q}}$. ${\mathbf E}(t,L) = {[{{E_x}(t,L),{E_y}(t,L)} ]^T}$and $\hat{{\mathbf E}}(t,0) = {[{{E_x}(t,0),{E_y}(t,0)} ]^T}$ are the received PDM waveforms and impairment compensated waveforms by the FNO-based NL$\hat{{\mathbf E}}(i,0)$C. ${\mathbf E}(i,L)$ and are their corresponding discretized counterparts with N points ($i = 1,2,3, \cdots ,N$), respectively. Where, L is the fiber length. ${E_x}(t,0)$ and ${E_y}(t,0)$ are transmitted electric field of the two orthogonal polarization components. ${E_x}(t,L)$ and ${E_y}(t,L)$ are received electric field after fiber transmission. We divide all the ${\mathbf E}(i,L)$ and $\hat{{\mathbf E}}(i,0)$ into several blocks with the length of M, i.e., ${{\mathbf E}^k}(L) = \{{{{\mathbf E}^k}(1,L),{{\mathbf E}^k}(2,L), \cdots ,{{\mathbf E}^k}(M,L)} \}$, ${\hat{{\mathbf E}}^k}(0) = \{{{{\hat{{\mathbf E}}}^k}(1,0),{{\hat{{\mathbf E}}}^k}(2,0), \cdots ,{{\hat{{\mathbf E}}}^k}(M,0)} \}$. Where $k = 1,2,3, \cdots ,N/M$ is the number of each block. As shown in Fig. 1, the calculation between the input and output of the FNO can be written as:

(1)$${\hat{{\mathbf E}}^k}(0) = (\mathrm{{\cal Q}} \circ {\mathrm{{\cal L}}^{(n)}} \circ \cdots \circ {\mathrm{{\cal L}}^{(1)}} \circ \mathrm{{\cal P}}){{\mathbf E}^k}(L)$$

where ${\circ}$ denotes function composition. L and n are the fiber length and number of Fourier layers, respectively. In order reduce complexity of the FNO and enable it promote to the PDM system, we have modified the FNO structure from single-channel input (which is demonstrated in [36]) to four-channels input. While, the input vectors ${{\mathbf E}^k}(L)$ is map to ${{\mathbb R}^{M \times H}}$ dimension space (which is the space size in [36]) by the lifting operator $\mathrm{{\cal P}}$. The projection operator $\mathrm{{\cal Q}}$ is used to conversely project the last hidden layer’s outputs ${z^{(n)}}$ to the original dimension space by a fully connected network layer and reconstruct the waveform $\hat{{\mathbf E}}(t,0)$ after the propagation. The detail of the Fourier layer $\mathrm{{\cal L}}$ is depicted in the inserted diagram of Fig. 1. The calculation of any Fourier layer $\ell$ can be expressed as:

(2)$${z^{(i + 1)}} = \sigma [{{{\cal W}^{(i)}}{z^{(i)}} + {\mathrm{{\cal K}}^{(i)}}({z^{(i)}})} ]$$

where $\sigma$ is a nonlinear activation function with point-wised approximation. ${\cal W}$ and $\mathrm{{\cal K}}$ represent the convolution operator in time-domain and integral kernel operator in frequency-domain as:

(3)$${{\cal W}^{(i)}}({{z^{(i)}}} )= {{\cal W}^{(i)}} \ast {z^{(i)}}$$

(4)$${\mathrm{{\cal K}}^{(i)}}({z^{(i)}}) = IFFT[{{{\cal R}^{(i)}} \cdot FFT({z^{(i)}})} ]$$

where ${\cal R}$ denotes frequency-domain weight matrices. As shown in Eq. (4), the frequency-domain operator $\mathrm{{\cal K}}$ is obtained by performing a fast Fourier transform (FFT), a fully connected network layer and an inverse fast Fourier transform (IFFT) successively. Attributing to the FNO learning the complex characteristics in the transmission link from the received signals in time-domain as well as frequency-domain, it can achieve CD and nonlinear impairments compensation simultaneously.

Fig. 1. The structure FNO-based NLC for PDM coherent optical communication system

Download Full Size | PDF

2.2 Training of FNO-based NLC

Figure 2(a) shows the training phase of the FNO-based NLC. The inputs to the FNO-based are derived from the coherently detected signals with 2 sample/symbol whose sampling rates are adjusted by an offline up sampling DSP block. For each modulation format, we collect 500,000 random symbols at every launched power, e.g. total of 45,000,000 symbols for 16QAM signals in the launched power range from -1dBm to 7 dBm in the step of 1dBm. In the training phase, we use the first 300,000 symbols of 16QAM signals at only three launched power points, e.g. -1dBm, 1dBm and 3dBm, which means that the training cost of the FNO-based NLC would be much lower than most other NNs-based NLC methods. The testing data includes the remaining 200,000 symbols with launched power of -1dBm, 1dBm, 3dBm, and all symbols with other launched power points. To avoid excessive memory overhead, each input data sequence is divided into several sub-sequences both in the training and test phase. Figure 2(b) shows the detailed organizing of the received signal sequence into data blocks for FNO-based NLC networks. Both in the training and testing phase, we divide the symbols sequence into sub-sequences with equal lengths of M (M = 2,000 symbols). Considering the time series, dispersion, and nonlinearity caused by inter-symbol interference (ISI), an overlap-and-save method is utilized to organize the adjacent input blocks with overlapping symbols. The number of overlaps of the header and tail for each block are 116 symbols (which is calculated according [37]). After blocks are divided, the symbols of the x- and y- polarization are reshaped to 4D vectors ${\mathbf S}_{}^k(L) = [{S_{{x_I}}^k(L),S_{{x_Q}}^k(L),S_{{y_I}}^k(L),S_{{y_Q}}^k(L)} ]$ and then sent into the FNO-based NLC networks. The mean-squared error (MSE) is selected to the optimization criterion of the loss function which is defined as:

(5)$${l_{FNO}} = \frac{1}{M}\mathop \sum \nolimits_{i = 1}^M {||{{{\mathbf E}^k}(i,0) - {{\hat{{\mathbf E}}}^k}(i,0)} ||_2}$$

where, M and k are the length and number of every block, respectively. ${\hat{{\mathbf E}}^k}(i,0)$ and ${{\mathbf E}^k}(i,0)$ are the output symbol vectors of the FNO-based networks and the target output symbol vectors, respectively.

Fig. 2. Detailed DSP flow for FNO-based NLC. (a) The training phase of the FNO-based NLC. (b) Organizing the received signals sequence into data blocks for FNO-based NLC networks.

Download Full Size | PDF

In our experimental test, ${{\mathbf E}^k}(i,0)$ is obtained by utilizing the DBP which is calculated by numerically solving the inverse-propagation Manakov equation [38]:

(6)$$\frac{{\partial {E_{x,y}}}}{{\partial z}} ={-} \frac{\alpha }{2}{E_{x,y}} - \frac{{j{\beta _2}}}{2}\frac{{{\partial ^2}}}{{\partial {t^2}}}{E_{x,y}} + j\gamma \frac{8}{9}({|{E_{x,y}}{|^2} + |{E_{y,x}}{|^2}} ){E_{x,y}}$$

where ${E_{x,y}}$ is the complex field envelope for x and y polarization mode, t is a vector function of time and z is transmission distance, $\alpha$, ${\beta _2}$, $\gamma$ denote fiber attenuation coefficient, group velocity dispersion, the nonlinear coefficient, respectively. As almost no further performance improvements can be obtained beyond 16StPS-DBP, ${{\mathbf E}^k}(i,0)$ is calculated by the fiber parameters optimized 16-StPS DBP considering the trade-off between the compensation performance and training cost.

2.3 Parameter optimization of FNO-based NLC network

Similar to our previous work [36], the FNO-based NLC network is constructed by 4-layer Fourier layers. To further optimize the network structure synthetically considering the trade-off between the compensation performance and computation complexity, a grid search is carried out for the decisive hyper-parameters of FNO-based networks, such as the hidden dimension h, the Fourier mode m (m = M/2) and the activation function. In the training phase, network is trained for 200 epochs with an initial learning rate of 0.001, which is halved every 100 epochs and use the Adam optimizer with configuration parameter ${\beta _1} = 0.9,{\beta _2} = 0.999,\epsilon = {10^{ - 8}}$ to update model parameters. Figure 3 (a) shows the Q-factor (${Q_{dB}} = 20{\log _{10}}\left[ {\sqrt 2 \textrm{erf}{\textrm{c}^{ - 1}}(2BER)} \right]$) improvement various with the Fourier mode m ($m \in \{ 200,400,600,800,1000,1200\}$) and hidden dimension h ($h \in \{ 8,16,32\}$). However, the Q-factor improvements will be very limited when the m and h are up to 1000 and 16, respectively. As larger m and h will result in much more training cost and computation complexity, the optimal value of m and h are fixed to 1000 and 16 consequentially. Figure 3(b) shows the convergence performance between two commonly used activation functions, i.e. the rectified linear unit (Relu) and Gaussian error linear unit (Gelu) activation functions. As the Gelu activation function reals faster convergence speed and higher convergence accuracy, it will be selected to build up the FNO-based NLC network.

Fig. 3. The network optimization results for FNO-based NLC. (a) The Q-factor performance of the FNO-based NLC with the hidden size h and Fourier mode m. (b) The convergence performance comparison between rectified linear unit (Relu) and Gaussian error linear unit (Gelu) activation functions.

Download Full Size | PDF

3. Experiment setup and results

3.1 Experiment setup and DSP stack

Experiments are conducted to evaluate the effectiveness of the FNO-based NLC in practice. The experimental setup is shown in Fig. 4. At the transmitter, a continuous optical wave with a linewidth of 100kHz and center wavelength of 1545 nm is divided into two orthogonal polarization state beams by a polarization beam splitter (PBS) firstly, and then modulated by their corresponding in-phase/quadrature (I/Q) modulators. Four 32-GBaud PAM-M electrical driving signals are generated by a 65GSa/s arbitrary waveform generator (AWG) with a length of 2¹⁶-1 pseudo-random binary sequence (PRBS) and shaped by squared raised cosine filter with a roll-off factor of 0.8. The two modulated optical waveforms are combined again by a polarization beam coupler (PBS) for the generation of a PDM signal and then launched into the fiber link. The fiber link consists of several spans of standard single-mode fiber (SSMF) with a length of 80 km. The attenuation, dispersion parameter, and nonlinear coefficient of each span 80-km fiber are $\alpha = 0.2dB/km$, $D = 16ps/nm$, and $\gamma = 1.3{W^{ - 1}}k{m^{ - 1}}$, respectively. The loss of each span is compensated by an EDFA with a fixed gain 16 dB and a noise figure of ∼6 dB. After multiple spans of fiber transmission, the signal is coherently detected and sampled by an 80 GSa/s digital oscilloscope with 33-GHz bandwidth. The power of the signal and local oscillator (LO) laser is fixed to be -3dBm and 8dBm, respectively. The sampled signals are finally processed by offline digital signal processing (DSP) modules, including resampling, FNO-based NLC, polarization demultiplexed by using the constant modulus and multi-modulus algorithm (CMA-MMA), frequency offset estimation and compensation (FOE), carrier-phase estimation and recovery (CPE), decision-directed least mean square equalization (DD-LMS), symbol de-mapping and error counting (as shown in Fig. 4).

Fig. 4. Experimental setup for PDM coherent optical communication. ECL: external cavity laser, PBS: polarization beam splitter, PBC: polarization beam coupler, AWG: arbitrary waveform generator, EDFA: Erbium-doped fiber amplifier, SSMF: standard single-mode fiber, LO: local oscillator laser, ADC: analog-to-digital converter, CMA-MMA: constant modulus and multi-modulus algorithm, FOE: frequency offset estimation, CPE: carrier-phase estimation, DD-LMS: decision-directed least mean square equalization.

Download Full Size | PDF

3.2 Nonlinearity compensation performance results and discussion

The feasibility of the FNO-based NLC is evaluated by applying both 32GBaud PDM-16/32/64QAM and PDM-PS-16/32/64QAM modulation formats. In these part of the performance evaluation process, only PDM-16QAM modulation format signals with launched power of -1dBm, 0dBm, and 3dBm are utilized in the training phase. Figure 5(a) shows the Q-factor versus the launched power for PDM-16QAM after 1600 km SSMF transmission. We consider 16StPS-DBP as the benchmark for optimal DBP performance, as almost no further Q-factor improvements can be obtained beyond 16StPS-DBP. It can be seen that the FNO-based NLC shows ∼1.03 dB and ∼0.31 dB Q-factor improvement at the optimal launched power of 4dBm compared to 1StPS-DBP and 5StPS-DBP, respectively. Figure 5(b) and (c) show the Q-factor results for PDM-32QAM transmission over 800 km and PDM-64QAM transmission over 400 km SSMF compensated by only CDC, DBP and FNO-based NLC. As shown in these figures, the FNO-based NLC outperforms 1StPS-DBP and 5StPS-DBP by ∼0.61 dB and ∼0.34 dB for PDM-32QAM, and by ∼0.59 dB and ∼0.17 dB for PDM-64QAM. Some typical eye-diagrams are shown in Fig. 6.The effectiveness of the FNO-based NLC is further verified in the 32GBaud PDM-PS-16/32/64QAM coherent optical system. The Q-factor versus the signal launched power compensated by only CDC, DBP and FNO-based NLC is shown in Fig. 7 and some typical corresponding eye-diagrams are shown in Fig. 8 as well. Compared to 1StPS-DBP and 5StPS-DBP, ∼0.49 dB and ∼0.20 dB Q-factor improvement for PDM-PS-16QAM (H = 3.5) are achieved after 1600 km SSMF at the optimal launched power of 3dBm. The gain of ∼ 0.34 dB and 0.29 dB over 5StPS-DBP for PDM-PS-32QAM (H = 4.7) after 800 km SSMF at the optimal launched power of 4dBm and for PDM-PS-64QAM (H = 5.8) after 400 km at the optimal launched power of 4dBm are obtained, respectively. Correspondingly, the gain as much as 0.95 dB and 2.25 dB over 1StPS-DBP are obtained for PDM-PS-32QAM and PDM-PS-64QAM, respectively. All the above results show that the FNO-based NLC can gain significant Q-factor improvement. The generalization ability and computational complexity will discussed in the following sections.

Fig. 5. The Q-factor versus the signal launched power. (a) Q-factor results for PDM-16QAM transmission over 1600 km. (b) Q-factor results for PDM-32QAM transmission over 800 km. (c) Q-factor results for PDM-64QAM transmission over 400 km.

Download Full Size | PDF

Fig. 6. Typical eye-diagrams. (a) and (b) eye-diagrams after CDC and FNO-based NLC for PDM-16QAM transmission over 1600 km with launched power of 4dBm, respectively. (c) and (d) eye-diagrams after CDC and FNO-based NLC for PDM-32QAM transmission over 800 km with launched power of 4dBm, respectively. (e) and (f) eye-diagrams after CDC and FNO-based NLC for PDM-64QAM transmission over 400 km with launched power of 4dBm, respectively.

Download Full Size | PDF

Fig. 7. The Q-factor versus the signal launched power. (a) Q-factor results for PDM-PS-16QAM (H = 3.5) transmission over 1600 km. (b) Q-factor results for PDM-PS-32QAM (H = 4.7) transmission over 800 km. (c) Q-factor results for PDM-PS-64QAM (H = 5.8) transmission over 400 km.

Download Full Size | PDF

Fig. 8. Typical eye-diagrams. (a) and (b) eye-diagrams after CDC and FNO-based NLC for PDM-PS-16QAM transmission over 1600 km with launched power of 4dBm, respectively. (c) and (d) eye-diagrams after CDC and FNO-based NLC for PDM-PS-32QAM transmission over 800 km with launched power of 4dBm, respectively. (e) and (f) eye-diagrams after CDC and FNO-based NLC for PDM-PS-64QAM transmission over 400 km with launched power of 4dBm, respectively.

Download Full Size | PDF

3.3 Generalization ability

In practical deployment, some system parameters, such as launched power and modulation format, may change over time. In this case, it will be very costly and time-consuming if the model has to be retained again. Figure 9(a) illustrates the compensation accuracy versus launched power with various transmission fiber length using the normalized MSE as the criterion:

(7)$$\textrm{N - MSE} = \frac{1}{M}\mathop \sum \limits_{i = 1}^M \frac{{\parallel {{\hat{y}}_i} - {y_i}{\parallel _2}}}{{\parallel {y_i}{\parallel _2}}}$$

where, $M$ is the number of symbols, $y$ is the DBP-16StPs output signal, $\hat{y}$ and is the compensated signal by the FNO-based model. Here, only PDM-16QAM signals are used in the training and testing process. In the training process, we randomly select three launched power data of each transmission fiber length. As shown in Fig. 9(a) the normalized MSEs increase extremely slow with increasing the launched power and are mostly below $1 \times {10^{\textrm{ - }4}}$ over 240 km and 400 km SSMF which means the FNO-based NLC model exhibits strong generalization ability in terms of launched power. Yet, the normalized MSE will increase fast with increasing the launched power in the case of long distances which means that different compensation will be obtained at different launched power. However, the normalized MSE can still always keep below 0.0016 when the launched power is at or below the optimal power point of 3dBm which would be enough for fiber nonlinearity compensation application.

Fig. 9. The Normalized MSEs between FNO and DBP 16-StPS outputs in time domain. (a) The Normalized MSEs versus launched power with various transmission distance. (b) The Normalized MSEs versus launched power by different trained approach.

Download Full Size | PDF

Figure 9(b) shows the generalization ability in the modulation format. The normalized MSEs are calculated for PDM-16QAM signals in the testing process with FNO-based models training by different strategies:

Case 1: only 16QAM signals with random three different launched power points are used in the training phase and each power point selected 200000 symbols.
Case 2: only PS-16QAM (H = 3.5) signals with random three different launched power points are used in the training phase and each power point selected 200000 symbols.
Case 3: 16QAM, 32QAM, and 64QAM signals are used in the training phase. Each modulation format signal selected 200000 symbols at one power point.
Case 4: 16QAM(H = 3.2),32QAM(H = 4.7),64QAM(H = 5.8) signals are used in the training phase. Each modulation format signal selected 200000 symbols at only one power point.

As shown in Fig. 9(b), the normalized MSEs are almost the same for all of above the four cases which indicates this model has a strong generalization ability in the modulation format. To sum up, the FNO-based NLC model shows very well generalization ability both in launched power and modulation format. however, it should be admitted that it is incapable of generalization ability in terms of transmission distance which means that the model should be retained for different transmission distance systems.

3.4 Computational complexity

As the computational complexity of the addition operation is much less than the real multiplications [37], only real multiplications (RMs) per symbol are taken account for comparing the complexity between the DBP and FNO-based NLC algorithm. For the SSFM-based DBP, the RMs per symbol can be calculated by the following equation [37]:

(8)$${C_{DBP}} = 4q{N_{Sp}}{N_{StPS}}\left[ {\frac{{{N_{FFT}}({\textrm{lo}{\textrm{g}_2}{N_{FFT}} + 1} )}}{{({{N_{FFT}} - {N_{CD}} + 1} )}} + 1} \right]$$

where ${N_{Sp}}$ is the number of fiber span. ${N_{StPS}}$ is the number of steps per span. ${N_{FFT}}$ is the FFT size and ${N_{CD}} = \left\lceil {\frac{{q{\tau_{CD}}}}{T}} \right\rceil$ is the minimum overlap size in the DBP process to avoid ISI. Where ${\tau _{CD}}$ corresponds to the dispersive channel impulse response and T is the symbol interval. q is the oversampling factor. $\left\lceil x \right\rceil$ denotes the smallest integer larger or equal to x.

The RMs for the FNO-based network come from three parts, including the one lifting operator $\mathrm{{\cal P}}$, 4 Fourier layers $\mathrm{{\cal L}}$ and one projection operator $\mathrm{{\cal Q}}$. Considering one single block with the number of samples M, The RMs for $\mathrm{{\cal P}}$ and $\mathrm{{\cal L}}$ are all $M \cdot h$. For one Fourier layer, the RMs come from two parts. The first comes from the time-domain convolution ${\cal W}$ with RMs $M \cdot h \cdot h$. The other part comes from the convolution in frequency domain ${\cal R}$ with RMs of $2hM{\log _2}M$ for real Fourier transform and inverse transform, and RMs of $k \cdot h \cdot h$($k = M/2$)for frequency domain convolution ${\cal R}$. So, the total of RMs required for n Fourier layers are $n\left[ {\frac{3}{2}M{h^2} + 2hMlo{g_2}M} \right]$. Therefore, the number of RMs per symbol per pol for FNO-based network with q samples per symbol can be summarized as:

(9)$${C_{FNO}} = r\left[ {qn\left( {\frac{3}{2}{h^2} + 2hlo{g_2}M} \right) + 2qh} \right]$$

where, r is the overlap rate.

As an example, Table 1 computational complexity comparison results in terms of RMs per symbol per pol and time consumption for PDM-16QAM transmission over 1600 km SSMF with launched power of 4dBm. As shown in this table, there is ∼0.31 dB Q-factor improvement achieved compared to traditional 5StPS-DBP, and the RMs per symbol are reduced to only 41.53%. Unlike the DBP-based algorithm compensating the nonlinearity step by step, the FNO-based black-box NLC performs the nonlinearity compensation using only four Fourier layers which are not various with the transmission distance. Therefore, the time consumption can reduce from 6.04 seconds to 1.69 seconds compared to 5StPS-DBP over 1600 km fiber transmission when CPU (Intel i9-12900 k) is used. Additionally, compared 1D complex operations in the DBP, the FNO involves a large number of real multi-dimensional matrix operations which is inherently suitable for GPU acceleration. Consequently, the time be further reduced as low as 0.03 second by utilizing a GPU (NVIDIA GeForce RTX 3080). Compared to 5StPS-DBP, the maximum processing speed of the FNO-based NLC has increased up to 51 times. Therefore, this algorithm would have a potential application in the DSP modules of coherent optical communication systems with GPU deployed.

Table 1. Computational complexity comparison results @PDM-16QAM over 1600 km

View Table

4. Conclusions

In conclusion, we introduced a novel fast single-step black-box NNs model based on Fourier FNO to compensate for the CD and nonlinearity simultaneously. The feasibility of the proposed approach is demonstrated in a 32GBaud coherent optical communication system with PDM-16/32/64QAM as well as PDM-PS-16/32/64QAM modulation format signals. The proposed scheme is highly parallelizable which can reduce the time consumption significantly compared to conventional DBP algorithm. This scheme also demonstrated strong capable of generalization ability in terms of launched power and modulation format which is particularly important in practical deployment.

Funding

Natural Science Foundation of Sichuan Province (2022NSFSC0539); Sichuan Province Science and Technology Support Program (2023YFG0102); National Natural Science Foundation of China (U22A2089).

Acknowledgment

This work was supported in part by the National Natural Science Foundation of China, in part by the Sichuan Science and Technology Program, in part by the National Natural Science Foundation of Sichuan Province, and in part by State Key Laboratory of Advanced Optical Communication Systems Networks, China.

Disclosures

The authors declare no conflicts of interests.

Data availability

Data underlying the results presented in this paper are not publicly available at this time but may be obtained from the authors upon reasonable request.

References

1. S. J. Savory, “Digital filters for coherent optical receivers,” Opt. Express 16(2), 804–817 (2008). [CrossRef]

2. P. J. Winzer, D. T. Neilson, and A. R. Chraplyvy, “Fiber-optic transmission and networking: the previous 20 and the next 20 years,” Opt. Express 26(18), 24190–24239 (2018). [CrossRef]

3. H. Liu, B. Pan, Y. Huang, et al., “Ultra-compact lithium niobate photonic chip for high-capacity and energy-efficient wavelength-division-multiplexing transmitters,” Light: Advanced Manufacturing 4(2), 1–10 (2023). [CrossRef]

4. A. D. Ellis, Z. Jian, and D. Cotter, “Approaching the non-linear Shannon limit,” J. Lightwave Technol. 28(4), 423–433 (2010). [CrossRef]

5. L. Schmalen, “Probabilistic constellation shaping: challenges and opportunities for forward error correction,” in Proc. Optical Fiber Communication Conference (OFC’, 2018), Paper 1–3.

6. H. Chien, J. Yu, Y. Cai, et al., “Approaching terabits per carrier metro-regional transmission using beyond-100GBd coherent optics with probabilistically shaped DP-64QAM modulation,” J. Lightwave Technol. 37(8), 1751–1755 (2019). [CrossRef]

7. H. Maeda, K. Saito, H. Kawahara, et al., “High spectral efficiency real-time 500-Gb/s/carrier long-haul transmission over field-installed fibers,” J. Lightwave Technol. 39(4), 933–939 (2021). [CrossRef]

8. J. X. Cai, H. G. Bayshon, M. V. Mazurczyk, et al., “70.4 Tb/s capacity over 7,600 km in C + L band using coded modulation with hybrid constellation shaping and nonlinearity compensation,” in Proc. Optical Fiber Communication Conference (OFC’, 2017), Paper 1–3.

9. T. Koike-Akino, D. S. Millar, K. Parsons, et al., “Fiber nonlinearity equalization with multi-label deep learning scalable to high-order DP-QAM,” in Proc. Signal Processing in Photonic Communications (SPPCom’, 2018), Paper SpM4G-1. [CrossRef]

10. X. Li, M. S. Faruk, and S. J. Savory, “Advanced nonlinear digital signal processing for short-reach applications,” in Proc.Optical Fiber Communication Conference (OFC’, 2021), Paper F2D–3.

11. V. Bajaj, F. Buchali, M. Chagnon, et al., “Single-channel 1.61 Tb/s optical coherent transmission enabled by neural network-based digital pre-distortion,” in Proc. European Conference on Optical Communication (ECOC’, 2020), Paper 1–4.

12. D. Zibar, O. Winther, N. Franceschi, et al., “Nonlinear impairment compensation using expectation maximization for dispersion managed and unmanaged PDM 16-QAM transmission,” Opt. Express 20(26), B181–B196 (2012). [CrossRef]

13. T. Gui, C. Lu, A. P. T. Lau, et al., “High-order modulation on a single discrete eigenvalue for optical communications based on nonlinear Fourier transform,” Opt. Express 25(17), 20286–20297 (2017). [CrossRef]

14. M. Fu, Y. Wu, Z. Yang, et al., “Low-complexity triplet-correlative perturbative fiber nonlinearity compensation for long-haul optical Transmission,” J. Lightwave Technol. 40(16), 5416–5425 (2022). [CrossRef]

15. E. Ip and J. M. Kahn, “Compensation of dispersion and nonlinear impairments using digital backpropagation,” J. Lightwave Technol. 26(20), 3416–3425 (2008). [CrossRef]

16. E. F. Mateo, F. Yaman, and G. Li, “Efficient compensation of inter-channel nonlinear effects via digital backward propagation in WDM optical transmission,” Opt. Express 18(14), 15144–15154 (2010). [CrossRef]

17. X. Liang and S. Kumar, “Correlated digital back propagation based on perturbation theory,” Opt. Express 23(11), 14655–14665 (2015). [CrossRef]

18. S. Civelli, E. Forestieri, A. Lotsmanov, et al., “Coupled-channel enhanced SSFM for digital backpropagation in WDM systems,” in Proc. Optical Fiber Communication Conference (OFC’, 2021), Paper M3 H.7.

19. M. Secondini, D. Marsella, and E. Forestieri, “Enhanced split-step Fourier method for digital backpropagation,” in Proc. European Conference on Optical Communication (ECOC’, 2014), Paper 1–3.

20. L. B. Du and A and J. Lowery, “Improved single channel backpropagation for intra-channel fiber nonlinearity compensation in long-haul optical communication systems,” Opt. Express 18(16), 17075–17088 (2010). [CrossRef]

21. S. Zhang, F. Yaman, K. Nakamura, et al., “Field and lab experimental demonstration of nonlinear impairment compensation using neural networks,” Nat. Commun. 10(1), 3033–3040 (2019). [CrossRef]

22. Q. Zhou, F. Zhang, and C. Yang, “AdaNN: adaptive neural network-based equalizer via online semi-supervised learning,” J. Lightwave Technol. 38(16), 4315–4324 (2020). [CrossRef]

23. C. Häger and H. D. Pfister, “Nonlinear interference mitigation via deep neural networks,” in Proc. Optical Fiber Communication Conference (OFC’, 2018), Paper 1–3.

24. V. Oliari, S. Goossens, C. Häger, et al., “Revisiting efficient multi-step nonlinearity compensation with machine learning: an experimental demonstration,” J. Lightwave Technol. 38(12), 3114–3124 (2020). [CrossRef]

25. O. Sidelnikov, A. Redyuk, S. Sygletos, et al., “Advanced convolutional neural networks for nonlinearity mitigation in long-haul WDM transmission systems,” J. Lightwave Technol. 39(8), 2397–2406 (2021). [CrossRef]

26. Q. Fan, G. Zhou, T. Gui, et al., “Advancing theoretical understanding and practical performance of signal processing for nonlinear optical communications through machine learning,” Nat. Commun. 11(1), 3694 (2020). [CrossRef]

27. F. Li, X. Zhou, Q. Fan, et al., “Double-effect DNN-based DBP scheme for integrated sensing and communications,” in Proc. Optical Fiber Communication Conference (OFC’, 2022), Paper 1–3.

28. B. I. Bitachon, A. Ghazisaeidi, M. Eppenberger, et al., “Deep learning based digital backpropagation demonstrating SNR gain at low complexity in a 1200 km transmission link,” Opt. Express 28(20), 29318–29334 (2020). [CrossRef]

29. X. Lin, S. Luo, S. K. O. Soman, et al., “Perturbation theory-aided learned digital back-propagation scheme for optical fiber nonlinearity compensation,” J. Lightwave Technol. 40(7), 1981–1988 (2022). [CrossRef]

30. P. J. Freire, Y. Osadchuk, B. Spinnler, et al., “Performance versus complexity study of neural network equalizers in coherent optical Systems,” J. Lightwave Technol. 39(19), 6085–6096 (2021). [CrossRef]

31. E. P. da Silva and M. P. Yankov, “Adaptive turbo equalization for nonlinearity compensation in WDM Systems,” J. Lightwave Technol. 39(22), 7124–7134 (2021). [CrossRef]

32. O. Sidelnikov, A. Redyuk, and S. Sygletos, “Equalization performance and complexity analysis of dynamic deep neural networks in long haul transmission systems,” Opt. Express 26(25), 32765–32776 (2018). [CrossRef]

33. C. Li, Y. Wang, J. Wang, et al., “Convolutional neural network-aided DP-64QAM coherent optical communication systems,” J. Lightwave Technol. 40(9), 2880–2889 (2022). [CrossRef]

34. S. Deligiannidis, C. Mesaritakis, and A. Bogris, “Performance and complexity analysis of bi-directional recurrent neural network models versus volterra nonlinear equalizers in digital coherent systems,” J. Lightwave Technol. 39(18), 5791–5798 (2021). [CrossRef]

35. H. Ming, X. Chen, X. Fang, et al., “Ultralow complexity long short-term memory network for fiber nonlinearity mitigation in coherent optical communication systems,” J. Lightwave Technol. 40(8), 2427–2434 (2022). [CrossRef]

36. X. He, L. Yan, L. Jiang, et al., “Fourier neural operator for accurate optical fiber modeling with low complexity,” J. Lightwave Technol. 41(8), 2301–2311 (2023). [CrossRef]

37. A. Napoli, Z. Maalej, V. J. M. Sleiffer, et al., “Reduced complexity digital back-propagation methods for optical communication systems,” J. Lightwave Technol. 32(7), 1351–1362 (2014). [CrossRef]

38. G. P. Agrawal, Nonlinear fiber optics.Academic Press:London, U. K, 2007.

	Q-factor	RMs per symbol per pol	Time consumption using CPU (Seconds)	Time consumption using GPU (Seconds)
1StPS-DBP	6.78	2647	1.16	0.31
5StPS-DBP	7.50	13236	6.04	1.54
16StPS-DBP	8.14	42357	18.22	4.79
FNO	7.81	7738	1.69	0.03

Fast fiber nonlinearity compensation method for PDM coherent optical transmission systems based on the Fourier neural operator

Abstract

1. Introduction

2. Operation principle

2.1 Structure of FNO-based NLC

2.2 Training of FNO-based NLC

2.3 Parameter optimization of FNO-based NLC network

3. Experiment setup and results

3.1 Experiment setup and DSP stack

3.2 Nonlinearity compensation performance results and discussion

3.3 Generalization ability

3.4 Computational complexity

4. Conclusions

Funding

Acknowledgment

Disclosures

Data availability

References

Data availability

Cited By

Figures (9)

Tables (1)

Equations (9)

Optics Express