1. Introduction

The open optical network paradigm is a fully disaggregated network in which each network element can be abstracted inside the controller through standard protocols of different vendors [1]. Open optical networks aim to enhance the interoperability and software control at the optical layer, thereby maximally improving network flexibility, economics, and performance. Thus, it is convenient to achieve unified control of various transmission devices from different vendors in open optical networks.

Multi-channel and multi optical line system (OLS) transmission is a typical scenario in practical optical networks. However, each wavelength channel may have significant channel power excursion owing to the non-flat gain profile and noise figure (NF) of the Erbium-doped fiber amplifier (EDFA), wavelength-dependent attenuation of the reconfigurable optical add-drop multiplexer (ROADM), stimulated Raman scattering (SRS), and Kerr nonlinearity-induced interference (NLI) in the fiber [2]. Specifically, the non-uniform gain spectrum of the EDFA leads to uneven amplification of optical signals for different channels, which eventually results in the channel power excursions. The power excursion introduced by ROADM is mainly caused by transient effects during wavelength switching and filtering. Specifically, fast power overshoots and undershoots arise from sudden changes in power due to wavelength switching [3]. In cascaded wavelength selective switches (WSS) or ROADMs, the independent attenuation of each ROADM enhances power ripple and excursion [3,4]. In current optical transmission systems, the existing channel power equalization algorithms focus more on the performance of an optical multiplex section (OMS), without considering the improvement of the global end-to-end (E2E) transmission performance. Moreover, most methods optimize the gain spectrum of the EDFA and ROADM attenuation to equalize the channel power, respectively. However, it is necessary to optimize the global performance using a comprehensive and collaborative method rather than optimizing the components or equalizing a single OMS in an ultra-long multi-OLS and multi-channel open optical transmission system. Therefore, equalizing the power per channel of the transmission system using a global and collaborative approach is an urgent challenge.

In this study, we propose reinforcement learning (RL)-based channel power equalization (RL-PE) and channel power equalization (CMA-PE) based on a covariance matrix adaptive evolution strategy (CMA-ES) to jointly optimize the configuration of the EDFA and ROADM. The gain achieves a near-optimum value by adjusting the EDFA gain to minimize the amplified spontaneous emission (ASE) noise and NLI noise in the optical signal. Second, the slope caused by EDFA amplification is compensated by changing the EDFA tilt to obtain a relatively flat power of different channels. In addition, the signal power of different wavelength channels is adjusted by changing the attenuation of the ROADM, thereby attenuating some high-power channels to achieve an overall flat total power. Thus, the target can be achieved by maximizing the average and flatness of the generalized signal-to-noise ratio (GSNR). The main contributions of this study can be summarized as follows:

The remainder of this paper is organized as follows. Section 2 provides a brief review of related work on channel power equalization. Section 3 introduces the modeling of an open optical network system with the GNPy. In Section 4, we introduce the CMA-PE framework and algorithm procedures. Section 5 presents the design, modeling, and process of the RL-PE algorithm. Section 6 presents the simulation verification of the transmission system and comparison results of the two algorithms. Finally, we summarize our conclusions in Section 7.

2. Related work

This section summarizes the existing methods for EDFA gain flattening and ROADM channel power equalization.

2.1 EDFA gain flattening

Methods for EDFA gain flattening mainly include optimizing the internal structure parameters, changing the input power, introducing hardware devices or special fibers, and adjusting the operating point of the EDFA to compensate for non-flat gain. Specifically, an EDFA with flat gain can be designed by optimizing EDFA structural parameters, such as pump mode, pump power allocation, and Erbium-doped fiber (EDF) length in the static network [5,6]. In addition, the application of a gain flattening filter (GFF) after the EDFA to compensate for the uneven EDFA gain spectrum is a common approach verified in previous studies [7–9].

The static GFF is ineffective in improving the SNR of the cascaded EDFA system when part of the optical wavelength at the input of the amplifier is lost or the input optical power changes significantly. Therefore, it is necessary to adopt dynamic gain equalization technology. In addition to using hardware devices, the dynamic equalization technique can also define the best operating point by setting the parameters of the optical amplifier. In [10], Barboza et al. defined this problem as optical amplifier adaptive control of the operating point (ACOP), where the operating point control optimizes a set of parameters of the amplifier (e.g., gain, tilt, and NF). Subsequently, some ACOP methods were proposed in [10–14] to search for the optimal operating point of the EDFA. Borraccini et al. proposed a CMA-ES algorithm in [15] to define the parameters of the EDFA in each OMS to improve the average and flatness of the GSNR. However, these methods only determine the optimal gain and tilt of the cascaded EDFA or introduce hardware devices without considering the ROADM in the transmission system.

2.2 ROADM channel power equalization

The above methods improve the gain flatness or compensate for the channel power by adjusting the EDFA. In addition, channel power equalization by adjusting the ROADM is also a relevant method. In [16] the authors designed a WSS-based add/drop module, and the power equalization of each channel was achieved through WSS filtering and single-wavelength channel operation. Similarly, Sambo et al. took advantage of the WSS to switch and equalize single wavelengths and achieved signal equalization by introducing channel-independent attenuation [17]. In [18], the authors proposed a heuristic method to allocate channel power through the WSS of each OMS to improve the worst-case SNR of the transmission system. The authors in [19] proposed a network automation framework to obtain the optimal power for each OMS by modifying the attenuation of the WSS through the digital twin technique.

2.3 Joint optimization of ROADM and EDFA

In a network with cascaded EDFA and ROADM, power equalization is also achieved by optimizing both the EDFA and ROADM simultaneously. Mo et al. [20] used automatic gain control to optimize an EDFA, which maintains a constant average gain for all active channels. The channel power excursion is mitigated while the gain curve remains unchanged by using the ROADM to add a new dual-wavelength signal.

In [21], the authors proposed a fast and simple heuristic optimization of link power by adjusting the EDFA setting and using the WSS for channel equalization to achieve optimal power. However, this approach differs from ours in the following aspects. The first difference is that they equalize the channel power through local optimization, whereas we consider global equalization. To achieve global performance gains, we prioritize improving the E2E GSNR rather than optimizing each OMS individually. In addition, we propose a more comprehensive and explicit ROADM architecture and attenuation mode compared to that of [21], which the attenuation depends on the average value of the signal power to be more adaptable to open decoupled transmission systems.

However, it is imperative to adopt a joint optimization strategy for both the EDFA and ROADM, given that the respective optimization of each component is unable to achieve optimal channel power equalization and impacts the transmission performance. Additionally, the optimization based on a single OMS does not consider the global characteristics of the entire network, because the optimization in a single section will affect other OMSs, thereby leading to optimization results that are unable to maximize the global performance. Therefore, it is necessary to propose a joint and global equalization solution.

3. Modeling of open optical networks

This study uses the GNPy platform to model and design open optical networks. GNPy [22] is an open-source Python software simulation platform with an open interface for modeling and designing optical networks. In essence, it is a quality of transmission (QoT) estimator of the DWDM system that can evaluate the GSNR of the optical propagation path under a given network configuration. The platform models each network element and device independently to realize disaggregated network management. To perform practical simulations of open optical network systems, the GNPy platform targets open network planning and management through compatibility with multiple vendor devices.

3.1 Physical layer modeling

The GNPy platform adopts a partial disaggregated optical network that uses an optical network controller (ONC) to control the optical layer, as shown in Fig. 1. In addition, the GNPy platform simulates the main optical network elements, including the optical amplifier (OA) for optical signal amplification, transponder to deploy the optical signal, and ROADM for optical switching operations. This platform also models passive network components, such as optical fibers. Next, we elaborate on the adjustable parameters for each device available in the platform.

 

Fig. 1. Physical layer modeling of open optical networks

Download Full Size | PDF

3.2 Device adjustable parameters

The adjustable parameters of open optical networks simulated by GNPy include parameters of each device and spectral information. Fig. 1 also shows the adjustable parameters that are closely associated with the algorithm.

 

Fig. 2. ROADM internal structure diagram: (a) input signal power, (b) output power of 40-79 channels, (c) amplified power of 0-39 channels, (d) output power of 0-39 channels.

Download Full Size | PDF

3.3 GSNR estimator

GSNR has proven to be useful to evaluate the performance of optical transmission systems. Thus, a crucial feature of the GNPy platform is the GSNR estimator. The connection of each network element is first detected to check whether it is correct and whether the network topology is complete when the network topology is input to the GNPy platform. Next, an optical signal is transmitted from the transmitting node along each network element, which will cause a certain effect on the signal; e.g., ASE noise or NLI noise may be added to the signal. For example, ASE noise is introduced when the amplifier amplifies the optical signal. ASE noise is calculated using the following formula:

4. CMA-PE algorithm principle

4.1 Basic framework

The CMA-PE algorithm primarily uses CMA-ES for channel power equalization, which is a non-linear parameter optimization algorithm that realizes global optimization by updating the covariance matrix. The CMA-ES is extremely effective in case the GSNR under the corresponding parameter is known while the specific formula and gradient of the function are unknown. It is necessary to note that the improved CMA-ES algorithm can also be applied to solve this problem, such as Least Squares CMA-ES. As for the parameters in transmission system, the EDFA gain, EDFA tilt, and ROADM attenuations are adjustable in the optimization. The other parameters are fixed and determined by the network design and deployment, such as the parameters of the transponder, fiber, and the insertion loss of the ROADM.

The main framework of the algorithm is illustrated in Fig. 3, which shows that the parameter values of the transmission system are changed through continuous interaction with the environment. Therefore, the EDFA gain, tilt, and ROADM attenuation change within the given Gaussian distribution range to achieve the population with the optimal objective function. During the optimization process, both the CMA-PE and RL-PE algorithms obtain a larger and flatter GSNR distribution by constantly interacting with the environment and changing the transmission system parameters, although the two algorithms follow different steps and iterative methods. Given that our aim is to maximize the average and flatness of the GSNR, we define the objective function as maximizing the average with minimum standard deviation of the GSNR for all wavelengths. The CMA-PE objective function is given by the following formula:

 

Fig. 3. CMA-PE and RL-PE algorithm main framework

Download Full Size | PDF

4.2 Algorithm principle

Algorithm 1 describes the CMA-PE algorithm procedure. The algorithm guarantees proper operation by entering the initial value, range, and step size of each device parameter.

First, the initial population is obtained by initializing the population parameters and range of each parameter (line 1). Second, the maximum number of iterations t^max of the optimization algorithm is set to ensure that the algorithm will not operate indefinitely; the population termination standard deviation σ is also set to ensure that the algorithm jumps out as required (line 2). The algorithm then performs an update iteration to calculate the parameter value with best channel power equalization effect by updating the device parameters of the population. A random Gaussian disturbance is generated for the existing device parameters with a new population, which is added to the random disturbance for the device parameters (lines 4-5). The top µ optimal device parameters are selected from the newly constructed population to reduce the population (line 6). The population device parameters require updating during population generation by estimating the population expectation, covariance matrix, and step size. The optimal objective function is then obtained by calculating the GSNR of the population (lines 9-10). The population is updated to a smaller one if the optimal objective function of the population is smaller than the previous one; otherwise, the new population equipment parameters are searched again (lines 11-12). The standard deviation of the population is calculated to check whether the conditions jump out when the population is refreshed (lines 14-15). Finally, the algorithm returns and outputs the optimized device parameters and optimal income (line 16).

5. RL-PE algorithm principle

In the RL-PE algorithm model, the agent and environment are the device adjustment module and transmission system to be optimized, respectively. The device adjustment module changes the EDFA gain, tilt, and ROADM attenuation to obtain the reward value of this action, which is used to determine whether the action is beneficial to the QoT. Therefore, device parameter values can be obtained to make the GSNR of all wavelength channels larger and flatter.

5.1 RL-PE framework

Fig. 4 shows the main framework of the RL-PE algorithm, which uses a device adjustment module to control the system automatically. The device adjustment module takes the network topology, device parameters, and parameters to be optimized into the transmission network database when the transmission system is input (step 1). The state awareness module is then used to generate state data St for the RL model by obtaining network nodes, device parameters, and parameters to be optimized (step 2). The policy $\pi t(A|St,\theta )$ is obtained by feeding the state data into a deep neural network (DNN), where A and θ are the set of all actions and parameters in the DNN, respectively (step 3); πt is a probability distribution that provides the probability distribution of the set of all actions, which is then input to the action selection module. Each action will determine a set of parameters for all the EDFAs and ROADMs in the transmission system. Specifically, the parameters of EDFA include gain and tilt values while the parameters of ROADM refer to the attenuation values. The module selects a set of actions based on the given πt, which determines the parameters of the device (step 4). The selected action is delivered to the device adjustment module to change the device parameters in the network topology (step 5). Next, the device adjustment module feeds back the acquired GSNR to the reward module and calculates the immediate reward (step 6). The algorithm stores the state St, action at, and immediate reward rt in the experience buffer (step 7). The DNN is retrained and updated to maximize the long-term cumulative reward when there are N samples in the buffer (step 9). However, it is important to note that if there are significant changes in the channel distribution, it is necessary to retrain DNN to adapt to the new distribution. In this case, transfer learning can be utilized to accelerate the retraining process and reduce the retraining cost. In addition, it may be necessary to increase the number of neurons in the DNN and retrain the DNN, as the transmission system length expands such as adding more OMSs or devices.

 

Fig. 4. RL-PE algorithm framework

Download Full Size | PDF

5.2 Modeling

 

Fig. 5. An illustrative example of state representation.

Download Full Size | PDF

5.3 Training mechanism

Algorithm 2 describes the procedure of the RL-PE scheme. The starting values are set by entering the transmission system to be optimized and initial values of the required optimization parameters (EDFA gain, tilt, and ROADM attenuation) before running the algorithm. First, the buffer Buf is initialized and the action selection probability γ is set to 0.5 (line 1). The algorithm first checks whether it is the first training optimization (i.e., whether the buffer is empty). If there are no elements in the buffer, the parameters of the DNN are synchronized (lines 2-4). Next, the device adjustment module obtains the state from the transmission system, as described in the previous section (line 5). In line 6, the probability distribution πt is generated according to θ_c in the action space. Subsequently, different action generation schemes are selected based on the specific values of πt and γ (lines 7-10). The maximum strategy in the probability distribution is chosen when the minimum value of πt is greater than γ; otherwise, the strategy adopts random policies. In line 11, the algorithm selects all EDFA parameters ${E^a}(G,T)$ and ROADM parameters ${R^D}(A)$ in the transmission system. The status St, action at, policy πt, and reward rt are stored in the buffer after receiving an immediate reward for this action (line 14). The policy and value gradient are updated according to the method reported in [24] until the sample size reaches N (lines 15,16). Training algorithms, such as RMSProp or Adam [25], can be used to adjust the global parameters of the DNN. Next, γ is updated, and the buffer Buf is emptied to prepare for the next device optimization (line 17). The algorithm will output the optimized device parameters when the global parameter converges.

6. Simulation evaluation

6.1 Simulation setup

A transmission system comprises 7 OLSs in the simulation, as shown in Fig. 6(a). Each OLS was amplified by four line amplifiers (LAs), a boost amplifier (BA), and a preamp amplifier (PA), where BA and PA were the internal amplifiers of the ROADM. In the simulation, the filtering penalties of WSS is not considered in this work due to the GNPy does not support the accurate modeling of WSS currently. In addition, each OLS comprised five optical fibers; the length and loss of each optical fiber can be adapted to particular user requirements. In this simulation, we considered a standard single-mode fiber (SSMF); each fiber length was randomly established, and the loss coefficient was set to 0.2 dB/km with 4 dB fixed insertion loss. Fig. 6(c) shows the specific length and loss of each section of the optical fiber. The channel frequency ranged from 192.50 THz to 196.00 THz, centered at 194.00 THz with a channel grid interval of 50 GHz. The transmitter sent optical signals at a baud rate of 32 GBaud in QPSK modulation format and reached the receiver through optical fiber loss, EDFA amplification, and ROADM regulation to generate 80-channel WDM comb waves. Table 1. lists the simulation settings.

 

Fig. 6. (a) Simulated transmission system, (b)gain profile, (c) fiber length and loss setting.

Download Full Size | PDF

Table 1. Simulation setting

View Table

We employed the generalized Gaussian noise (GN) model in GNPy to incorporate the impact of SPM, XPM, and SRS on the overall NLI in transmission systems. We simplified the EDFA noise figure by varying the LA noise figure in the range [4.5-5], with an average of 4.75 [26,27], while the PA and BA noise figures were set to 3.65 dB [21]. Several commercial types of EDFAs are included in the GNPy platform. We represented the non-flat gain of EDFA by employing a specific gain ripple profile. As shown in Fig. 6(b), we assume that all EDFAs within the system have an identical gain ripple profile. The initial gain of each EDFA was set to compensate for the previous optical fiber loss. The default value of α was set to 1 based on the convergence speed and optimization performance in simulation.

6.2 Simulation results

In this simulation, node Trx A and node Trx H were considered as the source and destination nodes, respectively . The optimization space was the average gain and gain tilt configurations of 42 EDFAs and attenuation values of 7 ROADMs in 7 OMSs. CMA-PE and RL-PE were used to achieve EDFA optimization and channel power equalization based on the above simulation settings. Fig. 7 illustrates the obtained EDFA and ROADM parameters before and after optimization.

 

Fig. 7. Device parameters before and after optimization ((a) EDFA gain, (b) EDFA tilt, (c) ROADM attenuation).

Download Full Size | PDF

In Fig. 7(a), the gain of the EDFA did not change significantly since the initial value of the EDFA gain was set to exactly compensate for the fiber loss. However, the change in the EDFA tilt was relatively large to compensate for the slope of different channel power. The initial and optimized parameters were input to the transmission system after optimization to obtain the GSNR status of each channel to attain the optimization effect, as shown in Fig. 8(a). As expected, with respect to the initial status, the GSNR was improved after optimization of the CMA-PE and RL-PE algorithms and maintained at a higher level with less fluctuation. In addition, both the initial and RL-PE values were significantly reduced around channel 23 due to the gain ripple of EDFA.

 

Fig. 8. GSNR variation for each channel before and after optimization.

Download Full Size | PDF

Specific GSNR values are listed in Fig 8.(b), showing that both CMA-PE and RL-PE improved the GSNR of each wavelength channel compared to the initial configuration. The CMA-PE algorithm achieved near optimum results, with 1.3 dB GSNR average increase and 0.5 dB GSNR standard deviation decrease, making it suitable for adaptation to ultra-long multi-OLS transmission systems. Similarly, the RL-PE algorithm exhibited an improved performance compared to the initial configuration, increasing the average GSNR and reducing the GSNR standard deviation by 1 dB and 0.4 dB, respectively. Observing the indicators in Fig. 8, it is possible to conclude that the CMA-ES algorithm produces excellent results in terms of accomplishing maximum GSNR average and flatness.

We demonstrated its effectiveness by recording the signal power per channel for each OLS. The results of the signal distribution for each channel after each OLS depicted in Fig. 9 show that, with respect to the initial configurations, both CMA-PE and RL-PE improved the signal power after each OLS and flattened each channel power. The variation in the signal power per channel with increasing system length and number of devices is shown in the three plots at the top of Fig. 9. In the initial situation, the signal evolution was systematic because the gain of the EDFA compensated exactly for the loss of the previous section of the fiber. The trend of the optical signal variation illustrates the gain profile of the EDFA, which had less amplification capability at intermediate wavelengths.

 

Fig. 9. Channel signal power distribution after each OLS.

Download Full Size | PDF

The channels power of the last three OLSs are compared in the three graphs at the bottom of Fig. 9, which show the fifth, sixth, and seventh OLSs, respectively. Note that the trend of the signal power generally rose with increasing wavelength channels. This was caused by the power excursion towards higher wavelength channels for optical signal. However, the optimization effect was not evident when the transmission system was short because the optical signal passed through a small number of optical fibers, EDFAs, and ROADMs. The optimization effect also increased as the transmission system length, loss, and number of devices increased. For example, the signal power horizontal was distributed approximately 9 dB after the seventh EDFA of the initial configuration. In comparison, the signal power decreased to 3 dB after passing 36 EDFAs. In particular, the GSNR deteriorated significantly in the last two OLSs with increasing transmission length and the EDFA cascade.

From the observation of the signal power per channel in Fig. 9, it is evident that, regardless of the initial signal power, RL-PE or CMA-PE exhibited a significant decrease around Channel 23; this was mainly caused by the EDFA gain profile. Considering that ROADM attenuates based on the average power of all channels, it is possible that some channels with power below the average may not be attenuated, resulting in some channels retaining the original channel power. In the simulation, we used the EDFA whose noise coefficient increased with the wavelength channel, so the large-wavelength channel featured low power. Moreover, the algorithm changed the tilt of the EDFA to balance the different channel power as much as possible.

On this basis, the CMA-PE and RL-PE algorithms were used to optimize the transmission system from 1 to 7 OLSs. The results reported in Fig. 10 show that CMA-PE achieved a higher gain of objective function (GOF) in the optimization of each OLS, while the time cost grew significantly with increasing OLS. Although the GOF of the RL-PE algorithm was not as good as that of the CMA-PE in each OLS, the optimization time of the RL-PE algorithm was much smaller than that of the CMA-PE algorithm as the number of OLS increased. Therefore, the RL-PE algorithm achieved a higher GOF per hour than CMA-PE for multi-OLS transmission system optimization.

 

Fig. 10. Optimization results of 7 OLSs ((a) GOF, (b) optimization time cost, (c) GOF per hour).

Download Full Size | PDF

For example, it took 4.3 hours for the CMA-PE algorithm to obtain a 1.9 dB GOF improvement in the optimization of 7 OLSs. In comparison, RL-PE achieved a GOF improvement of 1.5 dB with only 1.5 hours, making it effective for fast design and optimization in ultra-long multi-OLS transmission systems. However, CMA-PE required 0.2 hours to improve 0.6 dB compared to the initial objective function, while RL-PE needed 0.1 hours to improve the GOF by 0.4 dB when optimizing 1 OLS. The CMA-PE algorithm was more efficient when the number of OLS was short, and the GOF per hour was 0.5 dB/h greater than that of RL-PE. The convergence rate decreased as the search space of CMA-PE became larger because the algorithm needed to optimize more device parameters when the number of OLSs increased. In addition, RL showed an advantage in convergence speed; thus, the optimization time of RL-PE was shorter. Therefore, the effectiveness of the RL-PE algorithm was prominent with increasing OLSs and optimization parameters.

7. Conclusion

In this paper, we propose the CMA-PE and RL-PE algorithms to realize channel power equalization based on the joint optimization of EDFA and ROADM configurations in an open optical transmission system. These algorithms optimize the gain, tilt of EDFA, and ROADM attenuation to maximize the average and flatness of the GSNR. We constructed a network with 80 wavelengths and 7 OMSs using GNPy for simulation and optimization. In terms of the average GSNR of all channels, the CMA-PE and RL-PE algorithms demonstrated an enhancement of 1.3 dB and 1 dB, respectively, while the reduction of the standard deviation for all channels was approximately similar. In general, both algorithms can improve the GSNR of each channel; in particular, the CMA-PE algorithm achieves a higher GOF in the optimization of each OLS, whereas the RL-PE algorithm accomplishes a lower time complexity and higher GOF per hour in the optimization of a multi-OLS transmission system.