1. Introduction

The response of a central symmetric medium to incident light fields can often be expanded as $P={\Sigma }_{m\ge 0}{\chi }^{(2m+1)}{E}^{(2m+1)}$, in which $E^{(2m+1)}$ ($m=1,2,\mathrm{...}$) can be a mixing product of various light fields [1]. Typically, only the linear ($m=0$) and third-order ($m=1$) terms are considered, since the magnitude of ${\chi }^{(2m+1)}$ decreases rapidly with larger m. During the past decade, many techniques were exploited to enhance higher-order nonlinearities, such as by making use of the effect of electromagnetically induced transparency (EIT) [2]. As a fascinating phenomenon in nonlinear systems, optical solitons have been investigated intensively, and different types of solitons with different forms of nonlinearities have been demonstrated [3–8]. It is worth noting that the multi-dimensional spatial solitons are not stable just with third-order nonlinearity because of the catastrophic self-focusing effect [3,4]. In order to avoid this undesirable effect, several types of nonlinearities [9,10] or elliptic beam with orbital angular momentum [11] were proposed to overcome such instability. One of the schemes is to consider the focusing third-order and defocusing fifth-order nonlinear effects simultaneously with relatively high beam intensity, i.e., employ the competing cubic-quintic (CQ) nonlinearities [12], which was experimentally reported in photoconductivity and degenerate four-wave-mixing (FWM) measurements [13,14], π-conjugated polymer [15], chalcogenide glasses [16], and organic materials [17]. Unfortunately, such nonlinearity is always accompanied by significant higher-order multiphoton processes [6] that will affect the beam propagation. In [19,20], the multi-dimensional solitons and light condensates were theoretically studied in atomic system with giant CQ nonlinearity enhanced by EIT. Nevertheless, no further experimental investigation was reported so far.

In this letter, we first experimentally demonstrate controllable soliton formation and dynamics of the FWM signal and probe beams with competing CQ nonlinearities, in which the competitions between nonlinear cubic gain and linear as well as quintic absorptions are also involved. Comparing with pervious literatures, the main novelties of this letter can be summarized as the following two points: (I) The CQ nonlinear competition not only shows up in the real parts (refraction) but also in imaginary parts (absorption/gain). What’s more, the competitive coefficients can be controlled with multiple parameters (e.g., the frequency detunings, powers of pump fields, etc.) in the chosen atomic medium. (II) The EIT-enhanced competitions are investigated in stabilizing fundamental soliton, and high-order dipole and azimuthally-modulated vortex (AMV) soliton of the FWM beam by controlling experiment configuration. Especially for the AMV case with the configuration being fixed, it is found that the CQ competition mechanism also can cause the transformation among fundamental, dipole and AMV solitons, which can be stabilized by using the EIT-enhanced fifth-order nonlinearity with relatively low intensity.

The paper is organized as follows. In section 2, we present the theoretical model and experiment scheme. In Section 3, the competition between CQ nonlinear in soliton formation and dynamics of the FWM signal and probe beams are investigated. In Section 4, we conclude the paper.

2. Basic theory and experimental scheme

The experiments are carried out in a Na atomic vapor oven. As shown in Fig. 1(a), a three-level atomic system is formed by energy levels $|0〉$ ($3S_{1/2}$), $|1〉$ ($3P_{3/2}$) and $|2〉$ ($4D_{3/2}$) and five laser beams are adopted. The laser beams $E_1$, ${E^{\prime }}_1$ and $E_p$ ($E_2$ and ${E^{\prime }}_2$) connect the transition between levels $|0〉$ and $|1〉$ ($|2〉$) with the atomic resonant frequency ${\Omega }_1$ (${\Omega }_2$). The setup is shown in Fig. 1(b), and the configuration of the laser beams is given by the Fig. 1(c). The pump beams $E_1$ and ${E^{\prime }}_1$ with wavevectors $k_1$ and ${k^{\prime }}_1$ (with nearly the same power $P_1$, frequency ${\omega }_1$) propagate in the opposite direction to the weak probe beam $E_p$ ($k_p$,$P_3$, frequency ${\omega }_3$) with small angles (${\theta }_1={0.3}^o$) between them. The three beams come from the same dye laser DL1 (10 Hz repetition rate, 5 ns pulse width, and $0.04c{m}^{-1}$ linewidth) with frequency detuning ${\Delta }_1={\omega }_1-{\Omega }_1$, and their wave-vectors are in the x-o-z plane. The other two pump beams $E_2$ and $E_2\text{'}$ ($k_2$ and ${k^{\prime }}_2$, $P_2$ and ${P^{\prime }}_2$, ${\omega }_2$) from another dye laser DL2 (which has the same characteristics as the DL1) with ${\Delta }_2={\omega }_2-{\Omega }_2$ also propagate along the opposite direction of $E_p$ with small angles (${\theta }_2={0.3}^o$) between them, and their wave-vectors are in the y-o-z plane. In this system, there will be one FWM process, that the one-photon resonant degenerate FWM process (Fig. 1(a)) satisfies the phase-matching condition $k_F=k_p+k_1-{k^{\prime }}_1$ with the generated signal field $E_F$ propagating nearly in the opposite direction to the beam ${E^{\prime }}_1$. In the experiments, $E_p$ and $E_F$ are split by beam-splitter mirrors and recorded by charge coupled device (CCD) cameras and photo-multiply tubes (PMTs) simultaneously, as shown in Fig. 1(b).

 

Fig. 1 (a) The cascade three-level atomic system and the involved light fields used in the experiment. (b) Experimental setup. HR (high reflective mirror), BS (beam splitter$50\%$), PMT (photomultiplier tube), CCD (charge coupled device). (c) Spatial beam geometric drawing used in the experiment.

Download Full Size | PDF

In such experimental arrangement, several input laser beams with spatially non-uniform intensity and generated FWM signals interact with each other in a large spatial region, therefore spatially-varying phase modulation can affect the propagation and spatial patterns of the probe and FWM signals, which are relatively weak compared to the pump beams. The propagations of the probe and FWM signals can be described by the set of coupled equations [8]:

In Fig. 2(a), we present the theoretical nonlinear refractive index of the system versus ${\Delta }_2$ and the total pump intensity. In Figs. 2(b) and (c), the dependences of the real and imaginary parts of the linear term ${\chi }_{p,F}^{(1)}$, third-order one ${\chi }_{p,F}^{(3)X_2}\left({\left|E_2\right|}^2+{\left|{E^{\prime }}_2\right|}^2\right)$ and fifth-order one ${\chi }_{p,F}^{(5)X_2}\left({\left|E_2\right|}^4+{\left|{E^{\prime }}_2\right|}^4+{\left|E_2\right|}^2{\left|{E^{\prime }}_2\right|}^2\right)$ on the total pump intensity $\left({\left|E_2\right|}^2+{\left|{E^{\prime }}_2\right|}^2\right)$ are shown. It is obvious that ${\chi }_{p,F}^{(1)}$ keeps constant with varying $G_2$ (${G^{\prime }}_2$). Determined by the real parts of the third- and fifth-order susceptibilities, respectively, the focusing and defocusing nonlinearities follow cubic- and quintic-dependent laws, but the former case is positive and the latter case negative. The total refractive index composed of the linear part and the XPM nonlinear parts due to $E_2$ and ${E^{\prime }}_2$ can be further obtained as $\Delta n=n_1+n_2^{X_2}(I_2+{I^{\prime }}_2)+n_4^{X_2}(I_2^2+{I^{\prime }}^2{}_2)$, where $n_2^{X_2}=\mathrm{Re}[{\chi }_{p,F}^{(3)X_2}]/({є}_{0}c{n}_1)>0$ and $n_4{}^{X_2}=\{4\mathrm{Re}[{\chi }_{p,F}^{(5)X_2}]-{є}_0^2{c}^2{(n_2^{X_2})}^2\}/(2{є}_0^2{c}^2{n}_1)<0$ [22] are the corresponding Kerr nonlinear coefficients, respectively, with $I_2={є}_{0}c{\left|E_2\right|}^2/2$ (${I^{\prime }}_2={є}_{0}c{\left|{E^{\prime }}_2\right|}^2/2$) being the intensity determined by the power $P_2$ (${P^{\prime }}_2$). So when $I_2$ (${I^{\prime }}_2$) is not large enough, the refractive index $\Delta n$ will increase with $I_2$ (${I^{\prime }}_2$) due to the dominant positive third-order nonlinearity increasing in cubic law, but as $I_2$ (${I^{\prime }}_2$) exceeds a threshold value and gets into high-power region, it turns to decrease because that the negative fifth-order nonlinearity will play a dominant role on $\Delta n$. As shown in Fig. 2(a), this competition between different nonlinearities can be controlled by both the pump field detuning ${\Delta }_2$ and $I_2$ (${I^{\prime }}_2$). Thus, in the propagation of the probe or FWM beams, the diffraction will accompany competing nonlinearities between focusing and defocusing for certain pump field intensity, which is crucial for yielding high-order stable spatial solitons. The imaginary parts of the susceptibilities that offer gain or loss to the probe or FWM signals also have a competition which can influence the propagation of these beams. As presented in Fig. 2(c), when the pump intensities are small to satisfy $\left|Im\left\{{\chi }_{p,F}^{(1)}\right\}\right|\gg \left|Im\left\{{\chi }_{p,F}^{(3)X_2}{\left|E\right|}^2\right\}\right|\gg \left|Im\left\{{\chi }_{p,F}^{(5)X_2}{\left|E\right|}^4\right\}\right|$, the linear dissipation will be the dominant one. However, with $I_2$ (${I^{\prime }}_2$) increasing, the satisfied conditions of $\left|Im\left\{{\chi }_{p,F}^{(3)X_2}{\left|E\right|}^2\right\}\right|\gg \left|Im\left\{{\chi }_{p,F}^{(1)}\right\}\right|$ and $Im\left\{{\chi }_{p,F}^{(3)X_2}{\left|E\right|}^2\right\}<0$ can make the third-order nonlinear gain be dominant. If $I_2$ (${I^{\prime }}_2$) further increases, $Im\left\{{\chi }_{p,F}^{(5)X_2}{\left|E\right|}^4\right\}>0$ will make the fifth-order nonlinear dissipation be dominant. So, competitions exist not only among the real parts but also among the imaginary parts of the third- and fifth-order nonlinearities.

 

Fig. 2 (a)The theoretically calculated nonlinear refractive index as a function of ${\Delta }_2$ and the total pump intensity. The theoretically calculated real (b) and imaginary (c) parts of ${\chi }_{p,F}^{(1)}$, ${\chi }_{p,F}^{(3)X_2}\left({\left|E_2\right|}^2+{\left|{E^{\prime }}_2\right|}^2\right)$, ${\chi }_{p,F}^{(5)X_2}\left({\left|E_2\right|}^4+{\left|{E^{\prime }}_2\right|}^4+{\left|E_2\right|}^2{\left|{E^{\prime }}_2\right|}^2\right)$ and their sum versus the total pump intensity.

Download Full Size | PDF

In the experiment, the power of ${E^{\prime }}_{1,2}$ is approximately 2 times stronger than that of $E_{1,2}$, and 10 times stronger than that of $E_{p,F}$, therefore the SPM due to $E_{p,F}$ can be neglected. Furthermore, neglecting diffraction, we can obtain a set of solutions of $E_{p,F}$, in which the nonlinear phase shifts introduced in the propagation are ${\varphi }_{p,F}=2k_{p,F}z{\Sigma }_{m=1,2}{\Sigma }_{j=1,2}[n_{2m}^{X_j}(I_j^m+{I^{\prime }}_m{}^m)]/(n_0{I}_{p,F})$. The additional transverse propagation wave-vector can be obtained as $\delta k_x=(\partial {\varphi }_{p,F}/\partial x)\widehat{x}$, where $\widehat{x}$ is the unit vector along the horizontal axes. The direction of $\delta k_x$ determines the horizontal propagation of the beam, while the phase-front curvature ${\partial }^2{\varphi }_{p,F}/\partial x^2<0$ (${\partial }^2{\varphi }_{p,F}/\partial x^2>0$) lead to local focusing (defocusing) of the beams imposed on the weak field by the strong filed. Under balanced diffraction and XPM, the probe and FWM signals can form solitons with the electrical field intensity profile generally expressed as

3. Experimental observation of cubic-quintic solitons

First, the images of the transmission of the probe beam versus ${\Delta }_2$, captured by CCD cameras with different beams or combinations turned on, are shown in Figs. 3(a1)-(a4), and the corresponding intensities versus ${\Delta }_2$ recorded by PMT are shown in Fig. 3(a5). When one of or both of ${E^{\prime }}_2$ and $E_2$ are on, it is obvious that the probe suffers more significant focusing around the two-photon resonant (TPR) point (${\Delta }_1+{\Delta }_2=0$) than in other regions along the ${\Delta }_2$-axis, since the large XPM nonlinearity from ${E^{\prime }}_2$ and $E_2$ can bring strong focusing. The total nonlinear refractive index $\Delta n^{X2}=\Delta n_2^{X2}+\Delta n_4^{X2}$, i.e., the sum of third- and fifth-order nonlinear refractive indices, is positive and very large when ${\Delta }_2є[3050]GHz$, which undergoes sharp decrease when ${\Delta }_2$ is far away from TPR. This explanation can be further verified by the images with ${E^{\prime }}_2$ and $E_2$ both off, which shows no variation of the probe transmission images versus ${\Delta }_2$ due to the absence of ${\Delta }_2$-dependent Kerr nonlinearity (Fig. 3(a4)). It is interesting that the image focusing is enhanced in Fig. 3(a2) compared with Figs. 3(a1), and the focusing in Fig. 3(a3) is almost the same as that in Fig. 3(a1), though the total pump power in Fig. 3(a1) with ${E^{\prime }}_2$ and $E_2$ both turned on is larger than those in Fig. 3(a3) with one of the beams turned on. To explain this abnormal phenomenon, the fifth-order nonlinearity should be considered, i.e., the competition between CQ nonlinearities mentioned above. Specifically, in Fig. 3(a1), with both ${E^{\prime }}_2$ and $E_2$ turned on, the third-order nonlinear refractive index $\Delta n_2^{X2}=n_2^{X2}(I_2^{}+{I^{\prime }}_2^{})$ can be effectively weakened by the fifth-order one $\Delta n_4^{X2}=n_4^{X2}(I_2^2+{I^{\prime }}_2^2)$. Compared to the case in Fig. 3(a1), we find $\Delta n_4^{X2}$ suffers more reduction than $\Delta n_2^{X2}$ when $E_2$ is turned off (Fig. 3(a2)), which weakens the compensative effect of $\Delta n_4^{X2}$ on $\Delta n_2^{X2}$, thus $\Delta n^{X2}$ increases and results in more significant focusing. In Fig. 3(a3), $\left|\Delta n_2^{X2}\right|=\left|n_2^{X2}{I}_2\right|$ and $\left|\Delta n_4^{X2}\right|=\left|n_4^{X2}{I}_2^2\right|$ have nearly equal reductions as compared to the ones in Fig. 3(a1), so $\Delta n^{X2}$ has nearly no variation and the nonlinear focusing keeps. The three dots on the solid curve in Fig. 2(b) can elucidate the changing of focusing strength in Figs. 3(a1)-(a3). This explanation can be further verified by the more significant defocusing effect around TPR when ${p^{\prime }}_2$ is further increased to a higher value as shown in Fig. 3(b) from bottom to top. Satisfying the quintic-law, the decreasing factor of $\Delta n_4^{X2}=n_4^{X2}(I_2^2+{I^{\prime }}_2^2)$ is larger than that of $\Delta n_2^{X2}=n_2^{X2}(I_2^2+{I^{\prime }}_2)$, so $\Delta n^{X2}$ increases, which leads to the more significant focusing as shown in Figs. 3(b1)-(b6).

 

Fig. 3 (a) The probe images versus ${\Delta }_2$. (b) The probe images with ${P^{\prime }}_2$ decreasing from top to bottom. (c) and (d) are similar to (a) and (b), respectively, but show images of $E_F$. (e) The iso-surface plot of the beam with increasing ${P^{\prime }}_2$, and the xtick labels mark the cases shown in (a). (f) The iso-surface of the beam versus ${\Delta }_2$. Images from (a1) to (a4) as well as from (c1) to (c4) correspond to no beam blocked, $E_2$ blocked, ${E^{\prime }}_2$ blocked, and $E_2$ &${E^{\prime }}_2$ blocked, respectively, with the corresponding intensity curves recorded by PMT shown in (a5) and (c5). For (b), (d), and (f), no beam is blocked. For all the cases, ${\Delta }_1=-40GHz$.

Download Full Size | PDF

The intensity curves also reveal anomaly as shown in Fig. 3(a5), in which the EIT peak with ${E^{\prime }}_2$ or $E_2$ individually on is higher than that with both ${E^{\prime }}_2$ and $E_2$ on, though the total pump fields intensity is lower in the former case than in the latter case. The reason is that according to Eq. (3c) and the fifth-order curve (the dashdot curve) in Fig. 2(c), the fifth-order nonlinear loss $\mathrm{Im}\left\{{\chi }_{p,F}^{(5)X_2}{\left|E\right|}^4\right\}$ is larger with both ${E^{\prime }}_2$ and $E_2$ on than with ${E^{\prime }}_2$ or $E_2$ individually on, which can effectively lower the transmission peak. When both ${E^{\prime }}_2$ and $E_2$ are off, there is only the linear susceptibility and the nonlinear absorption nearly is eliminated, so the EIT peak disappears.

Also, the spatial shift of probe beam due to the cross nonlinear modulation of ${E^{\prime }}_2$ and $E_2$ can be observed. In Fig. 3(a1), when ${E^{\prime }}_2$ and $E_2$ are both on, the nonlinear phase shift introduced by the two fields is ${\varphi }_p=2\Delta n^{X2}{k}_{p}z{e}^{-r^2/2}/(n_0{I}_p)$ and the transverse nonlinear wave-vector ${\delta }_r=-{\nabla }_r{\varphi }_p$ is the gradient of nonlinear phase shift [23,24]. Because the probe beam spot is within the wings of ${E^{\prime }}_2$ and $E_2$ along the y-axis, the ${\delta }_r$ can be approximated by ${\delta }_r=2\Delta n^{X2}{k}_{p}rz{e}^{-r^2/2}/(n_0{I}_p)$ which leads to the upward shift of probe spots, due to positive $\Delta n^{X2}$. When ${E^{\prime }}_2$ is turned off (Fig. 3(a2)), similar to the nonlinear focusing mentioned above, $\Delta n^{X2}$ is larger than that in Fig. 3(a1), so the upward shift displacement becomes larger. With $E_2$ off in Fig. 3(a3), $\Delta n^{X2}$ is almost the same as that in Fig. 3(a1), but smaller than that in Fig. 3(a2), so the shift displacement is similar to that in Fig. 3(a1). When ${E^{\prime }}_2$ and $E_2$ are both off, ${\delta }_y=0$ for the XPM effect supplied by strong sufficiently pump fields diminishes and the spatial shift disappears (Fig. 3(a4)). Also, in the decreasing of ${P^{\prime }}_2$ from Figs. 3(b1)-(b6), the spatial shift displacement gradually increases due to the increasing $\Delta n^{X2}$.

Next, we adjust the beam configuration to make the FWM signals form dipole solitons and investigate their dynamics under competing third- and fifth-order nonlinearities. When ${\Delta }_2$ is scanned in the region far away from TPR, the nonlinearity induced by ${E^{\prime }}_2$ and $E_2$ is very weak, so the dipole modulation effect of electromagnetically induced grating (EIG) induced by ${E^{\prime }}_1$ and $E_1$ is dominant and $E_F$ gets dipole pattern [8]. This is verified by the almost unchanged dipole soliton patterns of $E_F$ when ${\Delta }_2$ is within the regions of $[-8010]GHz$ and $[70160]GHz$ in Figs. 3(c1)-(c4). When ${\Delta }_2$ is set within the region of $[1070]GHz$, i.e., around TPR, the nonlinear refractive index induced by ${E^{\prime }}_2$ and $E_2$ on the FWM signal is positive, which causes the dipole mode of $E_F$ to collapse into a single spot due to the strong focusing effect. When ${E^{\prime }}_2$ and $E_2$ are blocked (Fig. 3(c4)), $E_F$ remains to be in dipole mode even around TPR due to the vanishing nonlinear focusing effect. Compared to the nonlinear focusing in Fig. 3(c1), the focusing in Fig. 3(c2) with $E_2$ blocked is stronger and that in Fig. 3(c3) with ${E^{\prime }}_2$ blocked weaker. The reason is also that the total refractive index with ${E^{\prime }}_2$ off is smaller than that with $E_2$ off, but larger than that with both beams on. Due to the enhancement effect of the pump fields, the intensity of the FWM signal shows a peak higher than the background when ${E^{\prime }}_2$ and $E_2$ are blocked. With ${E^{\prime }}_2$ off, the enhancement effect is weakened because of the decreasing of the pump field intensity, so the peak becomes lower; when $E_2$ is off, the enhancement effect is weaker than that with ${E^{\prime }}_2$ and $E_2$ both on, but stronger than that with ${E^{\prime }}_2$ off, however the peak keeps almost the same height with that when ${E^{\prime }}_2$ is off. The main reason is that the magnitude of fifth-order nonlinear loss with $E_2$ off is much larger than that with ${E^{\prime }}_2$ off, which can bring considerable nonlinear loss and effectively limit the rising of peak. This variation reveals that the competition between the enhancement and nonlinear absorption plays an important role on the final received FWM signal.

With ${E^{\prime }}_2$ and $E_2$ both on, we investigate the images of the FWM signal with an increasing power ${P^{\prime }}_2$. The experimental results are summarized in Fig. 3(d). It is obvious that generated FWM beam is characterized by the dipole mode and single-spot at far away from TPR and around the resonance remain, respectively. But we find that the single-spot focuses more significantly with ${P^{\prime }}_2$ increasing (Fig. 3(d)), though the magnitude of $\Delta n_4^{X2}$ increases more rapidly than $\Delta n_2^{X2}$, which will decrease the spot focusing. The reason for this contradiction is that the imaginary parts of ${\chi }^{(3)}$ and ${\chi }^{(1),(5)}$ are negative and positive, respectively. The nonlinearity of ${E^{\prime }}_2$ is strong around TPR, and the positive imaginary part of susceptibilities will lead to a considerable dissipation of the FWM signal, which will be easily affected by the nonlinearity according to ${\varphi }_F=2\Delta n^{X2}{k}_{F}z{e}^{-r^2/2}/(n_0{I}_F)$. In Fig. 3(d), the total imaginary part of the linear and nonlinear responses is positive and increases with increasing ${P^{\prime }}_2$ because the initial ${P^{\prime }}_2$ is set to achieve $\left|Im[{\chi }_F^{(5)}{\left|E_2\right|}^4]\right|>\left|Im[{\chi }_F^{(3)}{\left|E_2\right|}^2]\right|$, which will lead to the decrease of the FWM signal intensity. Thus, $I_F$ decreases due to increasing loss with increasing ${P^{\prime }}_2$, and the decreasing rate is larger than the increasing rate of $\Delta n^{X2}$, which leads to a larger ${\varphi }_F$ that the FWM signal around TPR exhibit more significant nonlinear focusing as shown in Figs. 3(d1)-(d7). The iso-surfaces shown in Figs. 3(e) and (f) display the fundamental soliton changing with blockage of different beams and the transformation between dipole and fundamental solitons versus ${\Delta }_2$ based on Eq. (4) due to the competitions of nonlinearities and gain/loss mechanisms.

The influence of nonlinear competition on the transformation among fundamental, dipole and AMV solitons is also investigated. If we adjust the spatial configuration of the beams, the AMV mode of the $E_F$ signal can be obtained. The formation such AMV mode is due to the spatial modulation of the interference patterns [25] among $E_1$, ${E^{\prime }}_1$ and $E_p$. The AMV FWM field can be rewritten from Eq. (4) as

 

Fig. 4 (a) The images of $E_F$ versus ${\Delta }_2$ with (a1) ${E^{\prime }}_2$ blocked, (a2) $E_2$ blocked, (a3) no beam blocked and the powers of pumping fields set as ${I^{\prime }}_2\approx 10I_2$, (a4) no beam blocked and ${I^{\prime }}_2\approx 20I_2$, (a5) no beam blocked and ${I^{\prime }}_2\approx 50I_2$. (b) The 2D plots of $E_F$ around two-photon resonant (TPR) point corresponding to (a1)-(a3). For all the cases, ${\Delta }_1=30GHz$.

Download Full Size | PDF

In Figs. 4(a4) and (a5), we increase the power ${P^{\prime }}_2$ with $E_2$ fixed, and find that the higher total intensity breaks the balance between the cubic and quintic nonlinearities, and the latter one defocuses the beam. This becomes more obvious around TPR because the magnitude of $\Delta n_4^{X2}$ increases much more significantly than that of $\Delta n_2^{X2}$, and therefore the defocusing effect exceeds the focusing effect. The detailed explanations are similar to the ones used for Figs. 3(a1)-(a3) and (c1)-(c3). Figs. 4(b1)-(b3) obtained from Eq. (5), which correspond to Figs. 4(a1)-(a3), visually display that the modulated FWM transform vortex modes into fundamental mode or dipole mode around TPR. In Fig. 4, it is worth noting that the vortex versus ${\Delta }_2$ formed here is significantly different from those exhibited in [5,6,7], i.e., the pattern does not rotate compared to the usual vortex solitons. Such stability is due to the counteraction between nonlinear phase shift ${\varphi }_F$ and the intrinsic phase from nonzero topology charge and azimuthal modulation in Eq. (5).

4. Conclusion

In conclusion, we have experimentally investigated the multi-parameter controllable solitons under competing ${\chi }^{(3)}$ and ${\chi }^{(5)}$ nonlinear susceptibilities enhanced by atomic coherence or EIT in a three-level atomic system. Both in the spatial images and spectral intensity curves of the probe transmission and FWM beams, we have found that the nonlinear propagation processes can be significantly affected by the fifth-order nonlinearity if the pump field intensities are high. In addition, we have demonstrated transformations among fundamental, dipole and AMV solitons by controlling the pump beam geometric configuration and/or competing CQ nonlinearities. The experimental and theoretical results agree with each other very well. Such spatial pattern formation and propagation control can have important applications in all-optical image storage, processing and communication.