Abstract

We investigate the complexity and performance of recurrent neural network (RNN) models as post-processing units for the compensation of fibre nonlinearities in digital coherent systems carrying polarization multiplexed 16-QAM and 32-QAM signals. We evaluate three bi-directional RNN models, namely the bi-LSTM, bi-GRU and bi-Vanilla-RNN and show that all of them are promising nonlinearity compensators especially in dispersion unmanaged systems. Αs far as inference is concerned, οur simulations show that the three models provide similar compensation performance, therefore, in real-life systems, the simplest scheme based on Vanilla-RNN units should be preferred. We compare bi-Vanilla-RNN in its many-to-many form with Volterra nonlinear equalizers and exhibit its superiority both in terms of performance and complexity, thus highlighting that RNN processing is a very promising pathway for the upgrade of long-haul optical communication systems utilizing coherent detection.

Bi-directional gated recurrent unit neural network based nonlinear equalizer for coherent optical communication system

Xinyu Liu, Yongjun Wang, Xishuo Wang, Hui Xu, Chao Li, and Xiangjun Xin
Opt. Express 29(4) 5923-5933 (2021)

Attention-aided partial bidirectional RNN-based nonlinear equalizer in coherent optical systems

Yifan Liu, Victor Sanchez, Pedro J. Freire, Jaroslaw E. Prilepsky, Mahyar J. Koshkouei, and Matthew D. Higgins
Opt. Express 30(18) 32908-32923 (2022)

81-GHz W-band 60-Gbps 64-QAM wireless transmission based on a dual-GRU equalizer

Cuiwei Liu, Chen Wang, Wen Zhou, Feng Wang, Miao Kong, and Jianjun Yu
Opt. Express 30(2) 2364-2377 (2022)

Equalization performance and complexity analysis of dynamic deep neural networks in long haul transmission systems

Oleg Sidelnikov, Alexey Redyuk, and Stylianos Sygletos
Opt. Express 26(25) 32765-32776 (2018)

Cascade recurrent neural network-assisted nonlinear equalization for a 100 Gb/s PAM4 short-reach direct detection system

Zhaopeng Xu, Chuanbowen Sun, Tonghui Ji, Jonathan H. Manton, and William Shieh
Opt. Lett. 45(15) 4216-4219 (2020)