## Abstract

We demonstrate a polarization-managed 8-dimensional modulation format that is time domain coded to reduce inter-channel nonlinearity. Simulation results show a 2.3 dB improvement in maximum net system margin (NSM) relative to polarization multiplexed (PM)-BPSK, and a 1.0 dB improvement relative to time interleaved return-to-zero (RZ)-PM-BPSK, for five WDM channels propagating over 1600 km ELEAF with 90% inline optical dispersion compensation. In contrast to the other modulations considered, the new 8-dimensional format has negligible sensitivity to the polarization states of the neighboring WDM channels. High-density WDM (HD-WDM) measurements on a 5000 km dispersion-managed link show a 1.0 dB improvement in net system margin relative to PM-BPSK.

© 2014 Optical Society of America

## 1. Introduction

The high cost of capacity for transoceanic optical transmission has spurred development of new terminal equipment solutions for best effect on existing dispersion managed submarine links. For ultra-long haul applications, coherent detection with PM-BPSK [1] and time interleaved return-to-zero (RZ) BPSK [2, 3] are state of the art solutions. In many cases, the total fiber capacity is limited by power and polarization dependent Kerr nonlinear (NL) interactions within and between co-propagating WDM channels. Eventually, NL interference will be partially compensated with data-dependent compensation fields applied either at the transmitter or receiver through appropriate digital signal processing (DSP) [4, 5]. At present, the complexities of such solutions limit the commercial applicability of this approach.

Current implementable solutions for mitigating NL interference include the use of linear pre-dispersion applied at the transmitter [6], exploiting the dispersion map symmetries to partially cancel the NL interference after detection [7], and rapidly alternating the transmit polarization state [8] to minimize nonlinear polarization effects in dispersion managed links. For dispersion uncompensated systems with ~50% linear pre-dispersion applied electronically at the transmitter, the resulting dispersion map symmetry with PM-BPSK modulation acts to squeeze NL interference into the phase domain significantly reducing the received BER [6, 7]. Recent phase conjugated twin wave (PCTW) methods [7], which transmit a complex optical field together with its conjugate, further exploit this symmetry to realize a partial cancellation of the NL interference through straightforward modifications of the receiver DSP. In dispersion managed links, however, the optimal linear pre-dispersion must often be determined numerically or through a manual search, and the dispersion map may not necessarily satisfy the symmetry conditions required for PCTW. In such systems, RZ modulation with a half-baud temporal delay between polarizations has been shown [3] to suppress nonlinear effects by minimizing optical power variations and inducing rapid changes in the optical field polarization that act to suppress cross-polarization modulation (XPolM). This approach requires additional optical hardware including RZ pulse carvers and support for the increased spectral bandwidth of the RZ format.

The linear and nonlinear performance of a channel is also strongly influenced by the constellation design. In the linear propagation regime, noise tolerance is improved through coding techniques which minimize the symbol error rate by exploiting the dimensionality of the signaling space [1]. The development of high spectral efficiency formats remains an active area of research [9], however most cases focus on minimizing sensitivity to noise rather than suppressing the generation of NL interference all together.

In this paper, we propose and demonstrate a novel method for time-domain coding that substantially reduces inter-channel NL interference. Constellation symbols span the 8-dimensional space defined by two adjacent signaling intervals (time slots), and the four dimensions of the complex optical field within each time slot. The encoding of the cross-polarized, ‘X’ modulation format maintains the same spectral efficiency as PM-BPSK, while increasing the minimum Euclidean distance relative to PM-BPSK by encoding four bits across two consecutive time slots. The format is power balanced with polarization symmetry that reduces polarization scattering effects [10]. The constellation alphabet is chosen so that within each symbol the polarization Stokes vectors for the two consecutive time slots are equal and opposite. Thus the degree of polarization (DOP) for each 8-dimensional symbol is zero. Depending on the application, the X-constellation is shown to provide up to a 2.3 dB increase in net system margin at an un-coded BER of 3.5% relative to PM-BPSK. We report a measured 1.0 dB improvement for an HD-WDM configuration, with 37.5 GHz channel spacing, propagating on a 5000 km optically dispersion managed link.

## 2. Modulation format design and nonlinear propagation characteristics

We compare modulation formats in terms of net system margin (NSM), which is defined as the difference between the channel’s optical signal-to-noise ratio (OSNR) and the OSNR required to maintain a specified un-coded BER prior to forward error correction (FEC). To motivate our subsequent discussion, we note that for high net dispersion (uncompensated) links a simple Gaussian noise (GN) model [11] can be used to estimate the performance difference between two modulation formats at the optical powers that maximize their respective NSMs. The difference in maximum NSM, $\text{\Delta}NS{M}_{max}^{dB}$, between formats can be formulated in terms of the signal-to-noise ratio (SNR) of the received symbols according to

Recently, a 4-ary frequency and polarization switched QPSK format (4FPS-QPSK) was demonstrated [12] where symbols were encoded across an 8-dimensional space comprising two optical subcarriers. The 4-bit per symbol solution was compared with dual carrier PM-QPSK (8 bits per symbol) and achieved $\text{\Delta}SN{R}_{b2b}^{dB}\approx 4.3\text{\hspace{0.17em}}\text{dB}$ at a BER of 10^{−3} (3 dB due to the differing spectral efficiencies). The format was propagated over spans of uncompensated, standard single mode fiber where, from the first term in (1), we would expect a ~6.5 dB improvement in maximum NSM if both formats engender the same degree of NL interference during propagation. However, a reach increase of 84% ($\text{\Delta}NS{M}_{\mathrm{max}}^{dB}\approx 4.0\text{\hspace{0.17em}}\text{dB}$) was measured, which suggests that the NL propagation penalty of 4FPS-QPSK overwhelmed the back-to-back advantage. The 4FPS-QPSK performance shortfall was attributed to difficulty with phase tracking as well as a possible difference in NL propagation performance.

The X-constellation is a particular variant of a standard, 8-dimensional bi-orthogonal constellation [12]. We start with an 8-dimensional bi-orthogonal constellation defined as all coordinate permutations of [±1,0,0,0,0,0,0,0], in which the first four coordinates are transmitted in the four optical dimensions comprising time slot (A) and the second set of four coordinates are transmitted in the subsequent time slot (B). Through minimum Euclidean distance decoding this format achieves a 0.53 dB coding gain at an un-coded BER of 3.5% as compared to conventional PM-BPSK, which shares the same 2 bit per time slot spectral efficiency, c.f. Fig. 1(a). Alternatively, 8-dimensional bi-orthogonal modulation can be viewed as a polarization- and time-switched QPSK format, in which two bits are used to select the polarization (either X-polarization or Y-polarization) and time slot (either Slot-A or Slot-B) in which to transmit a (2 bit) QPSK symbol. Since the QPSK symbol is transmitted on only one polarization of a single time slot, with zeroes transmitted elsewhere, the bi-orthogonal constellation is amplitude modulated, with a large variation in the optical power occurring between time slots.

Noting that cross-phase modulation (XPM) and self-phase modulation (SPM) processes are governed by time dependent changes in the optical power, and XPolM additionally by changes of the polarization state, it is clear that the power variations of the standard 8-dimensional bi-orthogonal constellation as defined above will be disadvantageous for NL propagation. To improve its NL characteristics, we apply a series of 8-dimensional rotations (i.e. real-valued, orthogonal matrices) to the bi-orthogonal constellation. In doing so, we take advantage of the fact that, while the Euclidean distance between 8-dimensional constellation points is invariant under orthogonal transformations (leaving the back-to-back required OSNR unaffected), such a rotation can change the characteristics of the optical power and polarization within and between the symbol’s two time slots.

First, we rotate the constellation to ensure constant symbol energy for both time slots. This rotation acts to reduce XPM and SPM by eliminating the large power variations of the standard bi-orthogonal format. Next, to address XPolM, we identify a rotation such that the two time slots that define the 8-dimensional symbol have equal and opposite polarization Stokes vectors, guaranteeing that the symbol’s DOP is zero. The resulting optical field polarization switches rapidly between two orthogonal states on the time scale of the signaling rate, strongly suppressing nonlinear polarization effects in dispersion managed links [8]. While the nonlinear benefit of this rapid polarization switching will diminish on dispersion uncompensated systems, the X-constellation’s 0.53 dB back-to-back coding gain relative to PM-BPSK is expected from (1) to provide up to a ~0.8 dB increase in maximum system margin.

The resulting X-constellation alphabet is summarized in Table 1, together with the corresponding binary symbol labels. Figure**1(a)**** shows the back-to-back BER over a range of OSNRs for the X-constellation as compared to conventional PM-BPSK. While the energy and polarization symmetry properties are shared with other space-time block codes, for example, the Alamouti-QPSK format [13], the X-constellation provides an additional 0.53 dB coding gain over existing methods. Further, we will demonstrate in the next section that the X-constellation, unlike PM-BPSK, is insensitive to the relative polarization states of neighboring WDM channels.**

**It is important to note that the polarization switching characteristic of the X-constellation is encoded within the symbol constellation as a result of our selection of the 8-dimensional rotations, and does not require additional optical hardware, such as pulse carvers with polarization temporal interleaving or external optical polarization scrambling. Realizing the improved linear (and nonlinear) noise characteristics of the X-constellation however requires a slight increase in DSP complexity at both the transmitter and the receiver. In comparison with 4-dimensional modulation, 8-dimensional modulation transceivers must additionally map data onto the optical fields of two time slots, frame on symbols and decode data from field estimates of two (instead of one) time slots. In practice, this added complexity is small compared with other aspects of a coherent modem such as forward error correction and channel equalization.**

**3. Simulation results**

**Our simulations compare the NSM of the X-constellation to that of PM-BPSK and time interleaved RZ-PM-BPSK over 1600 km of dispersion managed ELEAF fiber [14]. Here, we have employed RZ-PM-BPSK, which was adapted from the RZ-PM-QPSK format described in [3], and similar to the solution illustrated in [2], to facilitate a comparison between modulation formats at equivalent capacities. Our channel configuration consisted of five co-propagating WDM channels spaced by 50 GHz, with a probe channel at 1550 nm surrounded by two interfering, “pump” channels located on either side. The polarization states of the four pump channels were initially aligned at a specified orientation with respect to the polarization state of the probe. Further, for each WDM channel we applied the optimal linear pre-dispersion that, when applied electronically at the transmitter, minimizes the BER at the receiver. This channel configuration was then propagated over 20 x 80 km spans of ELEAF fiber [14] through a split-step Fourier solution of the nonlinear Manakov equation [15, 16]. Following each fiber span, an ideal dispersion element was applied that compensated precisely 90% of the chromatic dispersion developed in the preceding span. For comparison with RZ-PM-BPSK, we set the signaling rate for all formats to 28 GHz and optically filter each channel at the transmitter with a 42 GHz (FWHM) 4th order super-Gaussian filter. This transmit optical filtering limits to some extent the power and polarization balanced properties of the RZ-PM-BPSK format because the RZ spectrum extends past the 42 GHz filter bandwidth. Following propagation, we optically noise load at the receiver and subsequently apply a filter optimally matched to the transmitter impulse response. A 31 tap, LMS trained MIMO filter is used to suppress residual inter-symbol interference prior to decoding.**

**In Fig. 1(b), we illustrate the NSM, relative to the maximum NSM of PM-BPSK, corresponding to the best and worst case relative polarization alignments between the pump channels and probe. For this dispersion managed test case, the X-constellation improves the NSM by 2.3 dB and 1.0 dB relative to the best-case polarization alignments of PM-BPSK and RZ-PM-BPSK, respectively, at an un-coded BER of 3.5%. As shown in Fig. 1(b), the advantage of the X-constellation is considerably greater when budgeting for the worst case polarization alignment. In fact, we observe a ~1.5 dB variation in maximum NSM for PM-BPSK, and 1.0 dB for RZ-PM-BPSK, as the relative polarization alignment between the pump channels and probe is varied; an effect we attribute to XPolM in this dispersion managed link. With BPSK modulated pumps, we find that the distribution of the nonlinear noise on the received BPSK symbols changes from predominantly phase noise to predominantly amplitude (radial) noise as the polarization of the pumps is varied. Since BPSK modulation is less tolerant to noise in the amplitude (radial) dimension, the NSM ranges between minimum and maximum values corresponding to the maximum and minimum values of the nonlinear amplitude noise variance, respectively. Further, as shown in Fig. 1(b), this polarization dependence is not observed when the pump channels and probe are modulated with formats possessing higher order PSK symmetries, such as the X-constellation. This polarization dependence of the BPSK NSM will be reduced in either dispersion uncompensated systems or dispersion managed links in which the polarization mode dispersion (PMD) correlation bandwidth [17] is less than the WDM channel spacing.**

**4. Measurement results**

**We measured the NSM for the X-constellation and PM-BPSK on the 5000 km dispersion and dispersion-slope managed link shown in Fig. 2(a), consisting of 76 spans of Corning Vascade® L1000, S1000 and EX2000 fiber [18] with a residual end-of-link dispersion of −1500 ps/nm. Erbium doped fiber amplifiers between each span operate in power control mode with a nominal total output power of 15 dBm. As shown in Fig. 2(b), our test channel at 1550.1 nm was located in the center of 9 HD-WDM channels with 37.5 GHz spacing, in which each channel was modulated with an independent, commercially available transmitter (Ciena WaveLogic 3) operating at a signaling rate of 35 GHz. The system configuration is shown in Fig. 3. The optimum linear pre-dispersion minimizing the received BER was determined manually for each modulation format and applied electronically to each HD-WDM channel. The remaining 51 channels, with 50 GHz spacing, were bulk modulated by a single modified transmitter followed by an optical channel decorrelator. Eight continuous wave (CW) polarization scrambled lasers were used for power control. At the receiver, the test channel was optically noise loaded, detected with a coherent receiver [19] and processed offline. Figure 4(a) shows the resulting NSM for X-constellation and PM-BPSK at an un-coded BER of 3.5%. At low launch powers we observe a ~0.5 dB improvement in NSM associated with the Euclidean distance gain of the X-constellation. Further, the X-constellation improved the maximum NSM by 1.0 dB relative to PM-BPSK after 5000 km of propagation at a spectral efficiency 33% higher than that afforded by conventional 50 GHz channel separations. Figure 4(b) shows a measured 2.0 dBQ improvement in the Q-Factor for the X-constellation relative to PM-BPSK at the end of the link, where the Q is defined in terms of the inverse error function (erfcinv) as $Q{|}_{dBQ}=20{\text{log}}_{10}\left(\sqrt{2}\cdot \text{erfcinv(2}\text{BER})\right)$.**

**5. Conclusion**

**Whereas electronic NL pre-compensation cancels the NL field that is generated during propagation, we have shown that coding can be used to reduce the generation of NL noise, while simultaneously increasing Euclidean distance, without sacrificing spectral efficiency. The implementation complexity of our solution is similar to that of PM-BPSK with the additional requirement that pairs of time slots must be encoded and decoded together. Experimentally, we realized a 1.0 dB improvement in NSM at 3.5% un-coded BER after 5000 km of HD-WDM propagation when comparing the X-constellation to PM-BPSK under identical propagation conditions and data rates. Simulation results show that the X-constellation outperforms PM-BPSK by 2.3 dB and RZ-PM-BPSK by 1.0 dB with greatly reduced sensitivity to the polarization states of neighboring channels. It has not escaped our notice that the method of encoding symbols across adjacent time slots and identifying transformations that minimize power variations and polarization asymmetry can be extended to higher spectral efficiency modulation formats.**

**Appendix A. Derivation of net system margin**

**The GN approximation [11] can be used to compare the net system margin for two modulation formats propagating on a dispersion uncompensated link. The discussion below is formulated in terms of the electrical signal-to-noise ratio (SNR), defined as the ratio of the constellation symbol energy to the noise variance as measured in the electrical domain at the symbol rate. The SNR due to amplified spontaneous emission (ASE) noise, SNR_{ASE}, is related to the optical signal-to-noise ratio (OSNR) through the ratio of the symbol rate (B_{e} = 35 GHz) to the 0.1 nm optical reference bandwidth (B_{o} = 12.5 GHz) according to:**

**in which the OSNR and$$\frac{1}{SN{R}_{req}^{(k)}}=\frac{1}{SN{R}_{b2b}^{(k)}}-{\left(\frac{P}{{P}_{ref}}\right)}^{2}\frac{1}{SN{R}_{NL}^{(k)}({P}_{ref})},$$where$SN{R}_{b2b}^{(k)}$ is the back-to-back SNR yielding the specified un-coded BER in the absence of propagation impairments. Next, we assume that the link delivered OSNR increases linearly with optical power relative to its value at${P}_{ref}$. It follows that, in terms of the electrical SNR, the NSM in linear units is given by the ratio of $SN{R}_{ASE}$to $SN{R}_{req}^{(k)}$:$$NS{M}^{(k)}=SN{R}_{ASE}({P}_{ref})\left(\frac{P}{{P}_{ref}}\right)\left[\frac{1}{SN{R}_{b2b}^{(k)}}-{\left(\frac{P}{{P}_{ref}}\right)}^{2}\frac{1}{SN{R}_{NL}^{(k)}({P}_{ref})}\right],$$which reaches a maximum value:$$NS{M}_{\mathrm{max}}^{(k)}=\frac{2}{3}\frac{SN{R}_{ASE}({P}_{ref})}{SN{R}_{b2b}^{(k)}}\frac{{P}_{fib}^{(k)}}{{P}_{ref}}$$at a power defined as ${P}_{fib}$:$${P}_{fib}^{(k)}={P}_{ref}^{}{\left(\frac{1}{3}\frac{SN{R}_{NL}^{(k)}({P}_{ref})}{SN{R}_{b2b}^{(k)}}\right)}^{1/2}.$$The optimum launch powers for two modulation formats labeled (b) and (x) are then related by:$$\begin{array}{l}{P}_{fib}^{(x)}={\left(\frac{SN{R}_{NL}^{(x)}({P}_{ref})}{SN{R}_{NL}^{(b)}({P}_{ref})}\frac{SN{R}_{b2b}^{(b)}}{SN{R}_{b2b}^{(x)}}\right)}^{1/2}{P}_{fib}^{(b)}.\\ \end{array}$$Combining with (6) to solve for the ratio of maximum NSMs for the two formats we find that:$$\frac{NS{M}_{\mathrm{max}}^{(x)}}{NS{M}_{\mathrm{max}}^{(b)}}={\left(\frac{SN{R}_{NL}^{(x)}({P}_{ref})}{SN{R}_{NL}^{(b)}({P}_{ref})}\right)}^{1/2}{\left(\frac{SN{R}_{b2b}^{(b)}}{SN{R}_{b2b}^{(x)}}\right)}^{3/2}.$$Expressed in dB:where,and,**

*SNR*are expressed in linear units. The nonlinear SNR, $SN{R}_{NL}^{(k)}$, for a modulation format labeled with index_{ASE}*k*, scales with optical power,*P*, as:relative to the nonlinear SNR evaluated at an arbitrary reference optical power,*P*. Following nonlinear propagation, the required SNR, $SN{R}_{req}^{(k)}$ for a specified un-coded BER is given by:_{ref}**References and links**

**1. **E. Agrell and M. Karlsson, “Power-efficient modulation formats in coherent transmission systems,” J. Lightwave Technol. **27**(22), 5115–5126 (2009). [CrossRef]

**2. **M. Salsi, “Ultra-long-haul coherent transmission for submarine applications,” in *36th European Conference and Exhibition on Optical Communication (ECOC 2010)* (2010), We.7.C.1. [CrossRef]

**3. **C. Xie, “Interchannel nonlinearities in coherent polarization-division-multiplexed quadrature-phase-shift-keying systems,” Photon. Technol. Lett. **21**(5), 274–276 (2009). [CrossRef]

**4. **E. Ip and J. M. Kahn, “Compensation of dispersion and nonlinear impairments using digital backpropagation,” J. Lightwave Technol. **26**(20), 3416–3425 (2008). [CrossRef]

**5. **Y. Gao, J. C. Cartledge, A. S. Karar, and S. S.-H. Yam, “Reducing the complexity of nonlinearity pre-compensation using symmetric EDC and pulse shaping,” in *39th European Conference and Exhibition on Optical Communication (ECOC 2013)* (2013), PD3.E.5. [CrossRef]

**6. **A. Mecozzi, C. B. Clausen, M. Shtaif, S.-G. Park, and A. H. Gnauck, “Cancellation of timing and amplitude jitter in symmetric links using highly dispersed pulses,” Photon. Technol. Lett. **13**(5), 445–447 (2001). [CrossRef]

**7. **X. Liu, A. R. Chraplyvy, P. J. Winzer, R. W. Tkach, and S. Chandrasekhar, “Phase-conjugated twin waves for communication beyond the Kerr nonlinearity limit,” Nat. Photon. **7**(7), 560–568 (2013). [CrossRef]

**8. **C. Xie, I. Kang, A. H. Gnauck, L. Möller, L. F. Mollenauer, and A. R. Grant, “Suppression of intrachannel nonlinear effects with alternate-polarization formats,” J. Lightwave Technol. **22**(3), 806–812 (2004). [CrossRef]

**9. **R. Rios-Muller, J. Renaudier, O. Bertran-Pardo, A. Ghazisaeidi, P. Tran, G. Charlet, and S. Bigo, “Experimental comparison between Hybrid-QPSK/8QAM and 4D-32SP-16QAM formats at 31.2 GBaud using Nyquist pulse shaping,” in *39th European Conference and Exhibition on Optical Communication (ECOC 2013)* (2013), Th.2.D.2. [CrossRef]

**10. **C. Xie, L. Möller, D. C. Kilper, and L. F. Mollenauer, “Effect of cross-phase-modulation-induced polarization scattering on optical polarization mode dispersion compensation in wavelength-division-multiplexed systems,” Opt. Lett. **28**(23), 2303–2305 (2003). [CrossRef] [PubMed]

**11. **P. Poggiolini, “The GN model of non-linear propagation in uncompensated coherent optical systems,” J. Lightwave Technol. **30**(24), 3857–3879 (2012). [CrossRef]

**12. **T. A. Eriksson, P. Johannisson, M. Sjodin, E. Agrell, P. A. Andrekson, and M. Karlsson, “Frequency and polarization switched QPSK,” in *39th European Conference and Exhibition on Optical Communication (ECOC 2013)* (2013), Th2.D.4. [CrossRef]

**13. **E. Meron, A. Andrusier, M. Feder, and M. Shtaif, “Use of space-time coding in coherent polarization-multiplexed systems suffering from polarization-dependent loss,” Opt. Lett. **35**(21), 3547–3549 (2010). [CrossRef] [PubMed]

**14. **S. Pachnicke, *Fiber-Optic Transmission Networks* (Springer, 2012), 157.

**15. **D. Marcuse, C. R. Manyuk, and P. K. A. Wai, “Application of the Manakov-PMD equation to studies of signal propagation in optical fibers with randomly varying birefringence,” J. Lightwave Technol. **15**(9), 1735–1746 (1997). [CrossRef]

**16. **M. Reimer, A. Borowiec, X. Tang, C. Laperle, and M. O'Sullivan, “Direct measurement of nonlinear WDM crosstalk using coherent optical detection,” in *IEEE Photonics Conference (IPC)* (2012), 322–323. [CrossRef]

**17. **M. Karlsson and J. Brentel, “Autocorrelation function of the polarization-mode dispersion vector,” Opt. Lett. **24**(14), 939–941 (1999). [CrossRef] [PubMed]

**18. **Corning, “Vascade Optical Fibers, Product Information,” August 2012 (available online).

**19. **K. Roberts, M. O’Sullivan, K.-T. Wu, H. Sun, A. Awadalla, D. J. Krause, and C. Laperle, “Performance of dual-polarization QPSK for optical transport systems,” J. Lightwave Technol. **27**(16), 3546–3559 (2009). [CrossRef]