1. Introduction

Artificial intelligence (AI) is a technology that enables computers to mimic human behaviors. Based on learning from large amounts of data, these systems can intelligently perform detection, inference, and non-linear modeling tasks across a multitude of disciplines. AI has shown its amazing potential in applications such as voice recognition [1–4], medical diagnosis [5–9], and autonomous vehicles [10,11], and is a driving force in technological development and social progress. Traditional computing architectures of CPU or GPU are currently used for AI training, however, the demand for exponentially growing data processing capabilities and the gradual failure of Moore’s Law have prompted the search for new architectures that enable higher speed processing with lower power consumption.

Application-specific integrated circuits (ASICs), such as Google’s Tensor Processing Units (TPUs), have emerged as the current state-of-the-art technology for AI accelerators. TPUs can efficiently handle matrix-vector multiplication operations, a major component of machine learning algorithms, at faster rates than current GPU technologies. Alternatively, integrated circuits inspired by biological nervous systems, the so-called “neuromorphic architecture “, are also becoming a subject of intense interest. Unlike von Neumann processors, which require the continuous transfer of data between separated processing units and memory via a shared bus, neuromorphic processors of special architectures can process data in memory and work in parallel or mimic the spiking behavior of biological neurons [12–20]. Photonic neuromorphic processors present an interesting opportunity to further expand on the performance improvements offered by TPUs, such systems can provide faster operations with lower energy consumption and latency. The technology appears ideally suited to hardware accelerators for specific tasks that do not require high-resolution weight quantization, such as inference [21,22]. A photonic neuromorphic processor can theoretically achieve 16 peta operations per second (POPS/s) and consume less than 2 W of power. This low-power consumption is due to the near-zero static power consumption of multiplication and accumulation operations in the photonic domain, which helps to alleviate the need for power in the age of information explosion [23]. The state-of-the-art GPU (NVIDIA A100, May 2020) in the same time period achieves only 1.26 POPS/s with a maximum power of 400 W [24]. In a photonic neuromorphic processor, the multiplication is done when the light passes through a weighting element of a few micrometers. Due to the ultra-fast propagation speed of light, photonic neuromorphic processors can have sub-nanosecond latency [15,22,24,25] compared to microseconds for GPUs [26,27]. The low latency makes photonic neuromorphic processors ideal for applications that require real-time control, such as robotics and driverless vehicles.

By leveraging the properties of light, neuromorphic processors based on photonic integrated circuits (PICs) have great potential for high-speed, parallel, low-energy, and low-latency computing. PICs are semiconductor devices that compactly integrate various components with different functions, including couplers, modulators, (de)multiplexers, memories, and photodetectors, on a single chip. Photonic signals can be transmitted between each photonic component at the speed of light, enabling extremely fast data transfer and computing. In addition, PICs have great potential for parallelized operations because the light of different wavelengths and polarizations can propagate independently in the same waveguide. Wavelength-division multiplexing (WDM) [24,28–32], and mode-division multiplexing (MDM) [33] techniques have been used to enhance the computing parallelism of neuromorphic photonic processors. Although significant progress has been made in the development of PICs, it has been noted that no single material platform can provide all the desired functionalities for neuromorphic photonic processors. Therefore, researchers have proposed hybrid PICs to augment or improve the functionality of PICs by combining multiple materials with different properties [34–43].

In this paper, the focus will be on reviewing materials that can be integrated with PICs, as well as recent advances in hybrid PICs for enhancing the functionality of photonic components and mathematical operations for neuromorphic computing. The paper is organized as follows: Section 2 briefly introduces the artificial neural network (ANN) associated with PICs. Section 3 describes materials that can be used for hybrid PICs, including phase change materials (PCMs), III-V semiconductors, graphene, and strong Pockels materials. Section 4 details the phase shifters, modulators, PCM-based memories, neuromorphic computational PIC architectures based on crossbar arrays, ring resonators, Mach-Zehnder interferometers (MZI), and multimode interferometers (MMI), and activation functions based on PICs.

2. Artificial neural networks

An ANN is a general model used in neuromorphic computing. A simple fully connected neural network consists of three layers, including an input layer, a hidden layer, and an output layer, as shown in Fig. 1(a). It is worth noting that various other types of layers, such as multiple hidden layers, convolutional layers, pooling layers, recurrent layers, and normalization layers, can be used and are typically stacked in an ANN. The Input layer contains only initial data and is only used for data transmission in the ANN. Synapses refer to the connection between input neurons to hidden neurons, hidden neurons to hidden neurons, and hidden neurons to output neurons. Synapses are related to weights, which are values that control the strength between two connected neurons. The hidden layer, used for computation, exists between the input and output layers. Finally, the output layer provides predicted results for the initial data input from the input layer.

 

Fig. 1. (a) Schematic of a fully connected neural network and a single neuron. This fully connected neural network includes an input layer, a hidden layer, and an output layer. The nonlinear model of a neuron in the dashed box consists of multiple synapses, a linear combiner, and a nonlinear activation function. (b) Schematic of applying convolutional layers for feature extraction before a fully connected layer. The kernel size for the convolution operation is 3 × 3 with a stride of 1 (no padding). The kernel slides over the input matrix. An element-wise product is performed between each element of the kernel and the element at the corresponding position of the input matrix, and then summed to obtain the output value.

Download Full Size | PDF

The structure of a single neuron is shown in the dashed box in Fig. 1(a). The input to the neuron comes from the input layer or connecting neurons of the previous layer. Each input value X_n is multiplied by the corresponding weight W_n, with a constant bias b that is then applied. A nonlinear activation function is applied to the sum of the weighted inputs and bias. Typical activation functions are ReLU, sigmoid, and tanh [44]. Finally, the results are used in the computation of all neurons in the next layer.

A convolutional neural network (CNN) [44,45] is an ANN specifically designed for image and speech recognition [46–51] that reduces computational complexity. A CNN usually consists of three layers, including a convolutional layer, a pooling layer, and a fully connected layer. The convolutional layer is the main building block of a CNN. In this layer, certain features of input data, such as the edges of objects in an image [29,30], can be extracted by performing a convolution operation between an input matrix and some kernel. Figure 1(b) shows how the convolution operation is performed with an example of a 6 × 6 input matrix and a 3 × 3 kernel. After performing an element-wise product between each element in the same location of the 3 × 3 matrix in the input matrix and the kernel, the result is added to obtain an output value. After the kernel slides over the input matrix and performs the convolution operation, a matrix of output values is obtained. The output matrix can be considered as a feature map with certain information about the input data, depending on the kernel applied. Note that the center of the kernel cannot overlap with the outermost element of the current 6 × 6 matrix, and therefore, the output feature map is composed of a 4 × 4 matrix. However, by adding zero rows or columns to each side of the input matrix (changing from 6 × 6 to 8 × 8), the so-called “zero padding”, the size of the output matrix can be the same as the size of the input matrix [44]. The stride defines the distance between two consecutive kernel locations, with the usual choice being 1. If the stride is greater than 1, the output feature map is downsampled. After the convolutional layer, a pooling layer can be used to reduce the spatial dimension of the feature map. Max pooling is typically used in the pooling layer. The feature map is divided into non-overlapping regions, which are called “pooling regions”. The highest values in the pooling region become the output because they represent the most prominent feature of the region. The size of output feature maps after the pooling operation depends on the size of the pooling region. If a 2 × 2 pooling region is used, the size of the output feature map is reduced by a factor of 2 (e.g., from 6 × 6 to 3 × 3). As the feature map size is reduced, the computational resource requirements of the later layers are also reduced. Subsequently, the feature map processed by the pooling layer can be flattened and imported into the fully connected layer.

To perform convolution operations and mimic artificial neurons in a PIC architecture, a number of basic components are required, including optical modulators (inputs), programmable weighting elements (weights), and nonlinear components (activation functions). If these components can be made from different materials and integrated on the same chip as a hybrid PIC, they can capitalize on their strengths and avoid weaknesses of each material [52]. For example, silicon nitride is CMOS-compatible [53,54] and has a greater transparency range and lower waveguide loss than silicon [55]. However, it is difficult to realize high-speed, energy-efficient modulators using only silicon nitride. This is because silicon nitride is insulating and plasma dispersion effects do not apply to silicon nitride. In addition, silicon nitride has a negligible Pockels effect [55] and a low thermo-optic coefficient of 2.45 × 10⁻⁵ K⁻¹ at 1.55 µm [56]. However, by introducing different materials such as ferroelectric lead zirconate titanate [57], lithium niobate (LiNbO₃) [58,59], and zinc oxide [60] onto the silicon nitride platform, it is possible to realize optical modulators with an excellent performance.

Through advanced design and integration, hybrid PICs can be used to mimic neural networks for computing [32,61,62]. In addition, hybrid PICs can also be used for convolution operations and achieve unparalleled high speed and low energy cost [28,30,63–69]. Related studies are reviewed in Section 4.

3. Materials for enhancing the functionality of photonic integrated circuit

The emergence of the next wave of telecommunications, sensing, and computing devices has encouraged the quest to further advance the PICs. In this endeavor, attention has been directed towards a diverse range of materials with exceptional properties that hold immense potential to transform the capabilities of PICs. This section explores a carefully curated selection of materials that have garnered significant attention for their unique characteristics that facilitate faster, more efficient, and multifunctional photonic devices.

3.1 Phase-change materials

PCMs have emerged as a promising class of materials that exhibit a reversible and controllable change in their physical, optical, and electrical properties upon exposure to external electrical or optical stimulus. The large change in optical properties ($\Delta n\; > 1,\; \Delta \; \kappa \; \approx $ order of magnitude) can be obtained by stimuli such as temperature, applied voltage, and optical excitation [70]. This unique characteristic has sparked considerable interest in the scientific community, as the application of PCMs in PICs can lead to the development of highly efficient and versatile devices for optical communication and information processing. The integration of PCMs in PICs has facilitated the creation of new functionalities, such as reconfigurable PICs, optical switching, and modulation, which have the potential to significantly improve the performance of photonic vices [71,72]. Furthermore, the development of different types of PCMs with distinct thermal, optical, and electronic properties has opened up new possibilities for designing advanced photonic devices with tailored functionalities [73]. The primary attribute of photonic storage materials lies in their inherent ability to manifest two distinct physical and chemical states, representing the binary code of ‘0’ and ‘1’ [74]. PCMs can undergo reversible transformations between a long-range atomic ordered crystalline state and a short-range atomic ordered amorphous state. Transition metal oxides and chalcogen-based alloys are extensively investigated PCMs. While chalcogen-based PCMs demonstrate an amorphous-crystalline phase transformation as shown in Fig. 2(a), transition metal oxides, particularly vanadium oxide, exhibit a crystalline-crystalline transition. However, PCMs suiting for nonvolatile photonic applications also need to fulfill the following criteria: (1) pronounced contrast in optical properties; (2) high-speed phase transition; (3) relatively low phase-transition temperature; (4) long thermal stability of amorphous state, and (5) large number of rewritable cycles [74,75]. With this in mind, based on the illustration in Fig. 2(b) only a very few of PCMs during the past three decades were found to meet the aforementioned demands.

 

Fig. 2. (a) Temperature-induced phase transition from amorphization/crystallization to crystallization/amorphization. Crystallization (also called ‘SET’), denoted by the blue curve, is achieved by heating amorphous PCMs above glass transition temperature (i.e., Tglass), whereas heating crystalline PCMs above melting temperature (Tmelting), followed by a subsequent quench, can induce amorphization (also called ‘RESET), represented by red curve. (b) Ternary Ge-Sb-Te phase diagram showing some popular PCMs with their years of discovery and several milestones in the history of phase-change memories. Reprinted with permission from [74], IEEE Xplore.

Download Full Size | PDF

Chalcogenides, formed by combining chalcogen elements (sulfur, selenium, tellurium, and polonium) with metals, undergo a transition from octahedral to rhombohedral crystalline structures [76]. Among chalcogenide PCMs, Ge₂Sb₂Te₅ (GST) is particularly advantageous due to its versatile composition. GeTe, Sb₂Te, AIST (Ag₅In₅Sb₆₀Te₃₀), and Ge₂Sb₂Se₄Te₁ (GSST) are noteworthy examples of these PCMs [77–79]. These materials exhibit profound resonance bonding in their crystalline state, resulting in distinct optical properties that differentiate them from other materials. By comparing the properties of GST, GSST, and Sb₂S₃, it is possible to ascertain their unique characteristics. Notably, these chalcogenide PCMs share a small electronegativity difference, leading to their covalent bonding, and both of their phases remain stable at room temperature. Despite their small electronegativity difference, these chalcogenide PCMs form stable covalent bonds and are valuable for memory devices. GeTe possesses reversible phase transitions, low-power consumption, and high data storage density. Sb₂Te excels in rapid switching and compatibility with various architectures. GST demonstrates “resonance bonding” and finds applications in optical data storage and switches. GSST offers enhanced stability and faster crystallization, while AIST showcases excellent electrical and thermal properties, facilitating high-speed phase change and reliable data storage [77–79]. These PCMs contribute significantly to optical device advancements, enabling state switching, data storage, and light manipulation [74].

Several transition metal oxides are commonly used as phase change materials in optical devices. One such oxide is vanadium dioxide (VO₂), which displays a unique property known as metal-insulator transition [80,81]. This transition occurs reversibly, causing VO₂ to change from a semiconducting state to a metallic state at a critical temperature. Another well-known phase change transition metal oxide utilized in optical devices is titanium dioxide (TiO₂). TiO₂ undergoes a phase transition called the anatase-rutile transition, where it transitions from the amorphous anatase phase at lower temperatures to the crystalline rutile phase as the temperature increases [80,81].

3.1.1 Material design and microstructure analysis

The design of materials is a multifaceted process, at the heart of which lies the comprehensive understanding of the material’s inherent characteristics. To achieve this understanding, sophisticated analysis methods are often employed, and simulation techniques play a crucial role in enhancing the design process for novel alloy systems. In the case of PCMs, the initial step in the design process involves simulations that generate a multitude of potential compositions exhibiting specific properties, such as thermal and electrical conductivity, band gap, refractive index, lattice constant, and crystal structure within a given temperature range. This step is pivotal in developing materials with tailored properties for diverse applications. Obtaining precise and reliable data on these properties is crucial for accurate simulations. Additionally, selecting an appropriate thermodynamic model capable of representing the intricate phase change behavior exhibited by the material is equally critical. Commonly used models include enthalpy-based formulations, heat capacity-based approaches, and phase fraction models, each tailored to the specific PCM under study. Furthermore, defining appropriate boundary conditions within the simulation framework is vital. Parameters such as temperature, pressure, and initial states must be thoughtfully determined to align with the intended application or experimental context being simulated.

To start by analyzing the basic properties of PCMs, calculation of phase diagrams (CALPHAD) is a widely used thermodynamic modeling approach that predicts the stable phases and compositions of materials at different temperatures and compositions [83,84]. While not specifically a simulation method itself, CALPHAD is often used with other simulation techniques to design and simulate PCMs for different applications by offering valuable insights into the phase stability, phase transformations, and thermodynamic properties of materials [83]. Further calculation of the electronic structure and properties of materials is often carried out using a powerful quantum mechanical approach called the Density Functional Theory (DFT). It is commonly employed to study the optical properties of PCMs, such as band gaps and refractive indices, by simulating the behavior of electrons within the material [85,86]. Additionally, ab-initio molecular dynamics (AIMD) is a highly versatile molecular dynamics simulation software package that allows for the simulation of various materials at the atomic or molecular level [87]. AIMD enables researchers to investigate the behavior and properties of materials under different conditions, such as temperature, pressure, and external forces. In the specific context of PCMs, AIMD provides valuable insights into the mechanisms of phase transitions, such as solidification or melting, intermixing, and the consequent changes in optical properties.

Subsequent to the fabrication of photonic chips incorporating PCMs, it is crucial to undertake a comprehensive analysis of their pristine state using suitable and comprehensive microstructure analysis techniques. The selection of the most suitable method requires a clear discernment between quantitative and qualitative approaches, which contribute to identifying or evaluating the key properties of materials. It is also essential to acknowledge the inherent limitations in the level of analysis achievable by each method. Scanning/transmission electron microscopy (S/TEM) is a widely recognized method that offers numerous possibilities for investigating various microstructural aspects, depending on the material’s functional objectives. Techniques such as high-resolution transmission electron microscopy (HRTEM), geometric phase analysis (GPA), energy dispersive x-ray spectroscopy (EDS), and nanobeam diffraction (NBD) strain/phase mapping provide a comprehensive overview of various features in the material, with atomic-scale resolution [88]. The selection and implementation of each technique heavily depend on the sample’s condition as well as the sample preparation technique. For instance, high-resolution imaging methods, as illustrated in Fig. 3, are suitable for analyzing a crystalline structure which lies in a specific zone axis, while NBD strain mapping goes a step further by enabling the mapping of strain in the amorphous phase [89,90]. To further analyze amorphous structures and identify size or variation in the medium-range order, fluctuation electron microscopy (FEM) in STEM mode has demonstrated its robustness [91]. Electron energy loss spectroscopy (EELS) utilizes the characteristic energy-loss spectrum of electrons that have transmitted through the sample [88]. Since the amount of electron energy loss is directly correlated to the ionization energy, which exhibits well-known characteristic behavior for any given element, valuable information can be obtained regarding elemental composition, chemical bonding, and electronic structure [92].

 

Fig. 3. HAADF-STEM images of (a) GST lattice without planar defects, (b) vacancy layer structure, and (c) antisite defect. The insets in (a)–(b) show simulated HAADF images of the GST lattice without relaxation. The GeSb/V atomic columns in (a)–(c) are located in octahedral sites. (d) Quantitative evaluation of intensity distribution along a mixed Te/GeSbV layer in the (202) orientation (white line in (a)), measured values for peak-to-peak interatomic distances, standard deviation of measured peak intensities of GeSb and Te, as well as peak height ratios (I) of Te:GeSb and corresponding standard deviation are denoted for image profile and simulation results. (e) GPA map of biaxial lattice distortion across defect network. Reproduced from [82], with the permission of AIP publishing.

Download Full Size | PDF

3.2 III-V semiconductors

The escalating demand for data production, transmission, reception, and processing can be met by photonic devices, specifically PICs. Notably, the past decade has witnessed a remarkable prodigy in the miniaturization of photonic components, facilitating the seamless integration of a more significant number of devices while simultaneously mitigating power consumption. Moreover, through deft manipulation of cavity geometry, spectra, and atom orientation, significant advancements have been made modifying spontaneous emission, thereby endowing the light source within PICs with enhanced sophistication and stability. Within this context, the plasmonic nano-laser emerges as an exceptionally promising solution for achieving a nano-scale footprint and generating coherent light. This novel technology leverages a meticulously designed utilizing a waveguide structure (nanowire-SiO₂-Ag), which ingeniously incorporates varying combinations of elements from groups III-V [93,94]. Noteworthy is the fact that the nanowire structure offers exceptional light confinement and boasts an elevated level of reflectivity, courtesy of its remarkable one-dimensional architecture. Crucially, the integration of III-V materials, such as GaAs, GaP, GaSb, InAs, and InP, plays a pivotal role in these cutting-edge integration techniques [95,96]. The foremost methods deployed for this integration encompass direct growth, bonding, and selective-area hetero-epitaxy, each bearing its own distinct advantages and challenges. The growth of III-V compound semiconductors directly on silicon presents challenges including antiphase boundaries (APBs), thermal mismatch, and lattice mismatch. Overcoming these obstacles is imperative for the production of GaAs or InP-based devices on Si. APBs arise when III-V materials are cultivated on elemental semiconductors, resulting in surface irregularities and scattering. Common approaches to address this issue involve employing miscut substrates and subjecting the substrate surface to high-temperature treatment. GaAs stands as the most extensively studied III-V compound on Si, due to its widespread application and comparatively smaller lattice mismatch [94].

Recent advancements in III-V quantum dot (QD) lasers, monolithically grown on Si substrates, have exhibited extremely promising outcomes, surpassing Si-based quantum well (QW) lasers in terms of longevity, output power, and threshold current densities. Furthermore, monolithically integrated lasers have outperformed heterogeneously integrated devices while benefiting from reduced parasitic capacitances and lower packaging costs. Nevertheless, the monolithic integration of QD lasers on silicon-on-insulator (SOI) platforms necessitates the resolution of optical coupling issues with waveguides. The utilization of thick buffer layers in QD lasers grown on Si offcut and V-grooved platforms necessitates distinct strategies for light coupling into Si-based waveguides [93]. Although III-V lasers selectively grown within Si trenches may offer a potential solution, imperfect crystal quality might impede this technique, resulting in polycrystalline III-V crystals during selective growth. Consequently, further research is imperative in this area in the immediate future. The proposal of a novel III/V-on-Si photonic integration platform combining the advantageous features of different methodologies has been successfully demonstrated. Figure 4 illustrates the integration of an InP-based multiple-QW laser epitaxy with a standard vertical p–i–n diode structure on a bonded InP-on-SOI substrate. This fabrication strategy has yielded satisfactory results despite the presence of high p-type doping and fabrication imperfections [97]. The regrown epitaxy displayed a material with high quality with significantly low dislocation density. By employing the bonding plus epitaxy approach, a general method for merging diverse materials onto various substrates has been established, in addition to being a potentially cost-competitive and highly scalable solution with substantial integration proximity and density [93].

 

Fig. 4. (a)-(f) Schematic of the process of III/V-to-Si integration. Reprinted with permission from [97], Springer Nature.

Download Full Size | PDF

Aligned with the emergence of the third-generation neuromorphic architecture, which focuses on integrating time-resolved dynamics for spike processing, there has been a simultaneous rise in interest regarding the utilization of III-V semiconductors [98]. This involves leveraging the capabilities of lasers operating in an excitable regime to explore the cognitive performance of networks. The dynamic nature of such “photonic (laser) neurons” that exhibit spiking behavior at extremely fast timescales, ranging from picoseconds to nanoseconds, allows for significant scalability and low-power consumption, further enhancing their utility in various applications. These excitable semiconductor lasers possess an impressive ability to selectively respond to specific input sequences, highlighting the network’s exceptional adaptability and versatility for future applications in neuromorphic computing. In contrast to ANNs, Spiking Neural Networks utilize sparse and asynchronous binary spikes to encode information [99]. This encoding mechanism closely mimics the behavior of the human brain on artificial neuromorphic hardware, thanks to its robustness against noise and energy efficiency [100].

3.3 Graphene

Graphene comprises a layer of carbon atoms that are organized in a hexagonal lattice resembling a honeycomb. Being one of the pioneering two-dimensional (2D) materials, graphene holds significant potential for enhancing the functionality of photonic components, given its remarkable electronic and optical properties. Specifically, for a single-layer graphene suspended in air, carrier mobility of up to $200,000$. $c{m^2}{V^{ - 1}}{s^{ - 1}}$ can be achieved [101]. The high carrier mobility implies that the graphene-based photonic devices can theoretically operate at $500\; \textrm{GHz}$ [102,103]. In addition, the unique linear band structure of a zero-band gap gives graphene a universal absorption of approximately $2.3\%$. from $0.5$ to $1.2\; \textrm{eV}$ [104], which makes graphene a great candidate for broadband applications. Graphene-based devices have been extensively studied to demonstrate their excellent performance in modulation [102,105–109] and photodetection [103,110–113]. Owing to their great potential to be used in the PICs aimed for neuromorphic computing, graphene-based modulators will be discussed in section 4.1.2.

3.4 Strong Pockels materials

The emergence of silicon photonics has led to the idea of integrating cutting-edge CMOS electronic circuits with the significant computational advantages offered by optical circuits. However, it is crucial that optical materials possess specific characteristics to be considered suitable candidates for hybrid integration. In particular, the materials must exhibit a reasonably large response to electrical fields to drive an electro-optical circuit with the small voltages provided by high-speed electronics. Additionally, the materials must possess sufficient speed to maintain the GHz range electrical bandwidth [114].

The corresponding material parameter that measures the linear response of an optical medium’s refractive index to an applied electric field is known as the Pockels- or electro-optic coefficient. The Pockels effect is a second-order non-linearity that manifests in media lacking inversion symmetry. While there has been significant research in bulk media like potassium dihydrogen phosphate (KDP) or beta barium borate (BBO), for instance, the concept of photonic integration of such media is relatively new. The following sections will introduce Lithium-Niobate (LiNbO₃) and Barium-titanate (BaTiO₃), two of the most promising candidates for hybrid integration of Pockels-materials into neuromorphic computing circuits.

3.4.1 Lithium-niobate (LiNbO₃)

Lithium-Niobate (LiNbO₃, LN) is an artificially crystalline material, that garnered significant attention, offering a considerable breakthrough in optics and telecommunication, when its optical properties were first discovered in the mid of 1960s, at Bell labs by Ashkin et al. and Chen et al. [115,116]. In [117], a significant examination of the material characteristics of LN is documented, wherein it is categorized as a constituent of the 3m point group (Hermann- Mauguin notation), thereby rendering it a trigonal crystal system displaying 3-fold rotational symmetry along the c-axis (commonly designated as the Z-axis) and mirror symmetry along the aforementioned, in 3 planes that are $60^\circ $ apart from one another, as depicted in Fig. 5 [118]. The octahedra spanned by the oxygen atoms are consecutively filled in the sequence lithium, niobium, vacancy, lithium, niobium, and vacancy along the z-axis, which implies z-glide symmetry [119], as shown in Fig. 5. In strong electric fields these domains can be reversed, by switching positions of the lithium atoms with the vacancy, giving rise to periodic poling enabling quasi-phase matching, hence nonlinear frequency conversion [118].

 

Fig. 5. Lattice structure of Lithium Niobate. (a) Difference between tetragonal and cubic crystal structure. The appearance of BTO depends on the temperature of the crystal being above or below the Curie temperature of 120°C. (b) Phase diagram of the BTO unit cell in dependence on its temperature. Taken from [118] with the permission of Annual Review of Materials Research.

Download Full Size | PDF

Additionally, to this, LN’s optical properties make it suitable as a promising candidate for PICs. LN is birefringent, having relatively large refractive indices of ${n_o}\; \sim \; 2.21$ and ${n_e} \sim \; 2.14$. (commonly defined in z-direction) at 1550 nm, enabling fabrication of high-index-contrast waveguides on common substrates such as SiO₂ or sapphire [120]. LN’s broad transparency window, covering the visible, near-infrared, and mid-infrared wavelength ranges from $350\; nm$. to $5\; \mu m$, makes it applicable in a wide range of PICs [120]. Its high Curie temperature of ${\sim} 1210\; ^\circ \textrm{C}$. ensures a stable ferroelectric phase, allowing compatibility with diverse fabrication processes and operating conditions. Its non-centrosymmetric crystal structure and large second-order nonlinear coefficient ${d_{33}} ={-} 27\; pm\; {V^{ - 1}}$ [121] make LN particularly suitable for optical wavelength conversion and photon-pair generation, especially when combined with ferroelectric domain engineering, as described previously. Additionally, cascading second-order optical nonlinear processes can implement effective Kerr nonlinearities with an efficiency much larger than t one available with standard ${\chi ^3}$. materials [122]. Very recently, this has been proposed for power-efficient all-optical switching [123] and nonlinear activation functions [124] which are all essential functionalities for photonic neural networks. More importantly, LN’s exceptional Pockels coefficient ${r_{33}}\; \approx \; 31\; pm\; {V^{ - 1}}$ [125] positions it as the material of choice for fast electro-optic modulators (EOMs), which play a vital role in the core functionality of telecommunication networks. Furthermore, LN possesses a third-order nonlinear coefficient ${n_2} = 1.8\; \times \; {10^{ - 19}}{m^2}\; {W^{ - 1}}$ comparable to other commonly used optical materials, enabling the generation of Kerr combs. These unique material properties of LN have led to its widespread utilization in advanced telecommunication and optoelectronic applications.

Properties of thin film lithium-niobate

Thin Film Lithium-Niobate (TFLN) emerged, as the traditional diffused-waveguides on bulk LN could not overcome their technical limitations, mainly originating from weak mode confinement, and the resulting impossibility to create tight bends as needed e.g., for ring resonators, hence limiting the PICs design space tremendously.

Regrettably, deposition techniques, commonly known from for different varieties of material platforms, including RF-Sputtering [126], chemical vapor deposition [127] or sol-gel [128], were proven to not being capable of epitaxially growing highly crystalline TFLN due to a severe lattice mismatch to the underlying handle wafer.

The availability of TFLN in the commercial market occurred subsequent to the demonstration of wafer bonding of LN to a SiO₂ handle by M. Levy et al. at Columbia University [129]. This was achieved by employing the well-known “Smart-Cut” technology, which had been recently devised [130]. The approach involved subjecting a Czochralski-grown LN single crystal to ion slicing and bonding the subsequent thin LN layer to a SiO₂ handle wafer. The technique ensured a uniform thickness of the LN-on-insulator wafer and maintained the high crystal quality of the Czochralski-grown LN. The dielectric LN thin film fabricated exhibited “as-bulk” behavior, possessing a significant refractive index contrast to SiO₂. This property allowed for strong mode confinement and facilitated compact device designs in TFLN waveguides, thereby opening new possibilities for PIC design [129].

Patterning TFLN poses inherent challenges, requiring laborious post dry-etch processing. After electron-beam patterning, with either commercially available resists (e.g., ArN7520 [131] or hydrogen silsesquioxane (HSQ) [132]) or common hard masks, such as Si [133] or Cr [134], the most favorable process to transfer the desired structures into the TFLN is inductively coupled plasma reactive-ion etching (ICP-RIE) [118]. This highly directional process is a combination of physical etching, namely ion-milling and chemical etching with free radicals and ions. Even though fluorine chemistry-based ICP-RIE is capable of effectively etching TFLN, it forms niobium fluoride alongside lithium fluoride. Especially the latter is non-volatile, having high redeposition rates combined with a vast resistance to further ICP-RIE, hence encapsulating the TFLN [135]. Due to resulting surface impurities and roughness, the waveguides fabricated with ICP-RIE fluorine chemistry have high propagation losses, interdicting the use of such. However, physical etching in ICP-RIE remains possible, utilizing Ar⁺ plasma. Redeposition of sputtered amorphous LN caused by the physical Ar etching process can be removed by standard wafer cleaning chemistry [136] with a negligible impact on waveguide propagation loss [131]. Recent developments in all the aforementioned fabrication steps allow current TFLN waveguides to have propagation losses $< 3\; \textrm{dB}/\textrm{m}$, giving rise to wafer-scale, arbitrary waveguide design.

3.4.2 Barium-titanate (BaTiO₃)

Compared to LN, Barium-Titanate (BaTiO₃, BTO) has an even longer history, starting in 1926, when it was predicted by Goldschmidt et al. as a perovskite-type crystal [137]. However, BTO did not gain much interest in the following years, until in World War II, an enormous demand for high dielectric materials ($\epsilon \; > \; 100$) arose. Consequently, a milestone for the BTO industry was achieved in the USA [138], England [139], Russia [140], and Japan [141], nearly all at the same time, in the mid-1940s, when BTO was grown, being the first-ever ferroelectric metal-oxide. Following discovery of its material properties showed a surprisingly high dielectric constant exceeding a relative permittivity of 1000 [142]. Since then, BTO drew a lot of attention, and its crystal properties were examined in the following years. In 1949, Kay et al. studied the ferroelectric properties of BTO with respect to the crystal temperature and measured a Curie-Temperature of $120^\circ \textrm{C}$., when the structure turns cubic, and the ferroelectricity vanishes [143]. A complete diagram of the crystal phases they discovered is shown in Fig. 6. At room temperature, BTO is non-centrosymmetric tetragonal crystal, as expected from the perovskite structure Goldschmidt predicted [137].

 

Fig. 6. Lattice structure of Barium Titanate. (a) Niobium and Lithium atoms are encapsulated by oxygen octahedra. The octahedra are filled in a repeating sequence of Lithium, Niobium, vacancy in z-direction. Poling allows reversing the domain, thus the direction of the sequence. (b) Mirror symmetries of LN giving the standard definition of x-, y- and z-axis, where y aligns with the mirror plane and z is parallel to them. Reprinted with permission from [144], Springer Nature.

Download Full Size | PDF

Properties of thin film barium-titanate

Thin Film Barium-Titanate (TFBTO) for optical applications only became prominent recently, as epitaxial film growth of high-quality crystals on standard wafers has only become possible with scientific progress in various deposition techniques. The single crystal quality of BTO thin film strongly depends on the lattice mismatch with the carrier material, introducing stress and lattice defects, resulting in poor linear and nonlinear optical properties [144]. With the introduction of lattice matching buffer layers, deposited via atomic layer deposition (ALD), it became possible to overcome those problems and fabricate monocrystalline TFBTO wafers. The current fabrication standard of TFBTO on a SOI handle requires a combination of different deposition techniques, as depicted in Fig. 7. At first, a 4 nm thin strontium titanate (STO) layer is deposited via molecular beam epitaxy (MBE) on top of the SOI handle, to ensure lattice matching and epitaxial growth. Using the same deposition mechanism, the BTO layer is grown on top of the STO. Due to the lower refractive index of BTO compared to silicon, waveguide structuring must happen in the latter. In order to access the silicon layer, the BTO has to be wafer bonded atop of a SiO₂ on Si acceptor wafer. A bonding interface is created by ALD of $5\; \textrm{nm}$. thick Aluminum oxide (ALO) on both, the donor and acceptor wafer. This reduces the lattice mismatch between the BTO and SiO₂ and preserves the BTO’s crystal quality. After wafer bonding, the resulting layer stack from top to bottom is Si, STO, BTO, ALO, SiO₂, Si, giving rise to BTO enhanced Si photonic structures with monocrystalline BTO having “as bulk” properties [145].

 

Fig. 7. Fabrication flow of Barium Titanate on silicon thin film. A thin lattice matching layer of STO is deposited via MBI on a SOI handle wafer. The same mechanism allows the growth of BTO atop of STO, before this donor wafer if bonded to a SiO₂ on Si acceptor wafer, that was prepared with an ALO lattice matching layer. Adapted with permission from [145], Springer Nature.

Download Full Size | PDF

In its tetragonal state, BTO is birefringent, having an ordinary refractive index of ${\approx} \; 2.3$. along the b-axis and an extraordinary index of ${\approx} \; 2.27$ along the a-axis at $1550\; \textrm{nm}$ [145], which yields a high contrast to both, SiO₂ and Si, allowing t mentioned BTO enhanced Si waveguides, but also direct patterned BTO-on-insulator waveguides. Particular interest for optical applications aroused due to its Pockels coefficient ${r_{42}} \approx 1300\; pm\; {V^{ - 1}}$ (${r_{42}} \approx 730\; pm\; {V^{ - 1}}$. in the unclamped case) being nearly two orders of magnitude larger than that of LN [144,146]. Thus, increasing the efficiency of electro-optical devices by lower drive voltage and smaller footprints compared to TFLN PICs.

Optical patterning of BTO enhanced Si is compatible with ICP-RIE silicon etching, resulting in propagation losses around the $4.8\; \textrm{dB}\; \textrm{cm}^{ - 1}$. [147], making it a suitable candidate for large scale PICs.

4. Neuromorphic computing based on photonic iegrated circuits

To achieve PIC-based neuromorphic computation photonic structures with different functions, such as modulators (encoding input vectors), weight memories (multiplication), crossbar arrays (summation), and nonlinear elements (activation functions), need to be integrated into the chip. Section 4.1 describes light intensity modulation devices based on different mechanisms. Section 4.2 details the various types of PCM-based memories and their performance. In addition, three common memory write, and erasure methods are discussed. Section 4.3 shows different PIC architectures based on crossbar arrays, MRRs, MZIs, and MMIs. Devices based on these architectures can be used to weight and sum input vectors. Section 4.4 reviews methods that utilize opto-electronic conversion or photonic nonlinear elements to achieve a nonlinear response. Photonic artificial neurons can be realized when combining the photonic structures and mechanisms reviewed in the four sections above. Section 4.5 presents an example of PIC-based artificial neurons. In Section 4.6, light coupling and post-fabrication trimming are discussed, which are not directly related to mathematical operations but are important for designing and fabricating photonic neuromorphic processors.

4.1 Optical intensity modulation

When using PICs for matrix-vector multiplication, the input vector is represented by input light pulses with different intensities, which are modulated by optical modulators. Later, these light pulses will enter photonic structures for multiplication and addition operations. The optical modulator must take into account parameters, such as modulation speed, modulation depth, optical bandwidth, insertion loss, and energy consumption. The performance of single-element-based optical modulators is usually limited by the nature of the material itself.

We use silicon as an example here. Silicon optical modulators can be achieved based on thermos-optics and plasma dispersion effects [148,149]. Silicon has a large thermo-optic coefficient (e.g., $1.8 \times {10^{ - 4}}{K^{ - 1}}$. at $1550\; \textrm{nm}$. [150]), so the optical response can be effectively modulated by heating silicon structures [151–155]. However, thermos-optic-based silicon modulators are not suitable for neuromorphic computing because the kilohertz modulation speed of the modulators [156,157] does not satisfy the requirements of high-speed computing. Most silicon optical modulators are based on the plasma dispersion effect. Figure 8(a) shows three typical schemes for silicon modulators, namely carrier injection, carrier depletion, and carrier accumulation. By varying the concentration of free carriers, the refractive index and extinction coefficient of silicon can be changed [158–160]. MZI and ring resonators are applied to silicon optical modulators. For MZI-based modulators, as shown in Fig. 8(b), the change in refractive index causes change in the relative phase of two propagating waves inside silicon waveguides, resulting in destructive or constructive interference that generates modulated signals. For ring resonator-based modulators, changes in the refractive index lead to changes in the resonant conditions of resonators, i.e., changes in the resonant wavelength. For a fixed wavelength, the shift of the resonant peak results in a switch between on and off states. The silicon optical modulators can achieve high data rates of up to $100\; \textrm{Gb}/\textrm{s}$ [161,162] or low energy consumption of tens of $\textrm{fJ}/\textrm{bit}$. [162]. However, these modulators have their own drawbacks. The MZI-based modulators are large in size because the phase shifter is typically a few millimeters in length [161,163]. Ring resonator-based modulators, on the other hand, can only operate in a very narrow frequency band and are temperature-sensitive [164].

 

Fig. 8. (a) Three schemes of silicon light modulators based on plasma dispersion effect. Schematic of silicon optical modulators based on (b) MZI and (c) ring resonators.

Download Full Size | PDF

To avoid these drawbacks and further improve the performance of modulators, other materials with excellent properties can be integrated with silicon or silicon nitride structures to form hybrid modulators. In the following sections, recent advances in phase shifters and optical modulators are reviewed.

4.1.1 Reconfigurable phase shifters and modulators based on non-volatile phase-change materials

Reconfigurable ultracompact high-speed photonic devices on-chip such as switches and low-loss modulators are desirable and necessary building blocks to control the light flow within the neuromorphic photonic network to address the growing demand for AI and machine learning. A number of technologies are focused on the implementations of the tuning mechanisms which can be successfully integrated into PICs to fulfill the above-mentioned requirements. Thus, reconfigurability, namely tuning information-carrying optical signals can be obtained via modulation of effective complex refractive index of the propagating signal-optical-mode, which can be achieved via thermo-optic effect [165–167], carrier-injection effect [168–170], electro-optic effect [131,171], through microelectromechanical systems and via integration of PCMs [172].

Importantly, non-volatile switch technology based on reversible PCMs integrated in PICs that maintain the switched phase state without consuming additional energy (zero static power) are revolutionizing the rapidly developed fields of high-performance computing. A distinct feature of PCM is drastic electrical and optical contrast over a wide spectral region [77] between two structural phase states (crystalline and amorphous) which PCM undergoes at corresponding phase transition temperatures and can maintain over a long time (non-volatility). This leads to large refractive index contrast and extinction coefficient difference [72,173] accompanied with fast and reversible switching of the states in the order of sub-nanosecond [174] for a large number of cycles, thus ensuring fast control and tailoring the amplitude and phase of the carried electromagnetic waves, which makes possible broadband and multilevel operations for ultracompact non-volatile PIC applications with ultralow energy consumption and zero static consuming power [30,32,64,175–178].

Transparent and non-transparent PCM

Chalcogenide-based PCMs including the most ubiquitous GST, provide a straightforward solution for exceptional non-volatile reconfigurability [77]. These PCMs can be reproducibly electrically and all-optically switched on a sub-nanosecond timescale [174,179] over 10¹⁵ switching cycles [173,180] leading to a drastic increase in the refractive index ($\Delta n\; \sim \; 2 - 3$) and extinction coefficient ($\Delta k\; \sim \; 1$) [181] upon crystallization in telecommunication wavelength band ($\lambda = 1550\; \textrm{nm}\; \sim \; 0.8\; \textrm{eV}$) and importantly without energy consumption to maintain the phase state [182]. These unique properties of GST make it a promising tuning instrument for integrated on-chip ultrafast optical switches, optical modulators [183–186], photonic multilevel storage [31,187–191], to mimic biological neural synapses on-chip and to build neural networks [30,32,192,193].

Notably, due to high optical absorption, especially in crystalline state GST cell cannot provide phase-only modulations, while in contrast - pioneer-investigated low-loss non-volatile reversibly switched PCMs such as Sb₂S₃, Sb₂Se₃ [194–199], and direct alternative to GST-GSST [200] offer ultra-small optical absorption in both structural phase states at telecommunication wavelength $1550\; \textrm{nm}$ and at the same time high index contrast. These PCMs are emerging for wide-bandgap reconfigurable phase control in programmable hybrid PICs on-chip.

Switching mechanisms PCMs

Phase transitions in PCM can be induced optically as well as electrically by Joule heating the material above the crystallization or melting temperature (amorphization), thus all-optical or all-electrical operations can be realized depending on the circuit functionality requirements and complexity [176]. In contrast to free-space optical switching obtained via utilization of external incident focused laser beam in the far-field [201], evanescent coupling of low-power femtosecond pulses in the near-field is successfully utilized for on-chip switches and modulators providing energy-efficient switching [32,175,190,192,202,203]. The latter type of direct (self-heating) switching enables full-integration PCM cells into PICs on-chip and supports all-optical operation of the circuit, leading to high-speed switching and ensuring scalability.

In order to access optically induced fast and reversible switching behavior of PCM, in on-chip photonic structures, the PCM cell is placed in the near-field of the photonic waveguide, where evanescent interaction between signal-carrying mode and the cell leads to modification of the amplitude and optical phase of the optical mode due to index change of PCM in different structural states, which is the basis of the all-optical operation. In particular, high contrast of refractive index and absorption coefficient of GST cell between two phase states deposited on top of passive waveguide results in pulling the light mode up to the GST cell leading to strong attenuation of the propagating signal which enables to engineer high extinction ratio amplitude switches. Thus, Rios et al. [31] performed gradual switching of GST cell (eight transmission levels) and demonstrated for the first-time all-optical non-volatile multi-level multi-bit memory (three bit) with capability of fast $({\sim} 500\; \textrm{ps}$) and energy-efficient (${\sim} 480\; \textrm{fJ}$) single readout of the storage state, which was obtained via integration GST cells on silicon nitride microrings. Further, M. Stegmaier et al. [185] demonstrated the first fully integrated entirely non-volatile all-optical $1 \times 2$ switch obtaining switching contrast up to ${\approx} \; 5\; \textrm{dB}$ over ${\sim} 1000$ cycles of operations with low switching-energy consumption $({\sim} \textrm{pJ}-{-}\textrm{nJ}$). Increase in switching optical contrast ($on \sim 3.3\; \textrm{dB}$) and decrease in energy consumption $310\; \textrm{pJ}$ ($\textrm{on} \sim 14\%$) in comparison with bare-waveguide design is obtained by integration of GST cell into tailor-made photonic crystal cavity due to more efficient absorption within GST cell as a result of on-resonance enhanced interaction between the PCM cell and the electric field of the resonance mode in the center of the cavity via Purcell effect [204] demonstrated by J. von Keitz et al. [205].

Patterning of PCM cells (GST) in subwavelength-scale nanostructures (500 nm cells) and placing them into electric field maximum of the propagating mode on microring resonator significantly improves switching contrast up to ${\sim} 21\; \textrm{dB}$ at the drop port preventing deformations of the cell, demonstrated by C. Wu et al [206]. Optical switches based on GST integrated in Si platform are well-explored in [181,207,208]. In particular, obtained high extinction ratio ($\textrm{up}\; \textrm{to}\; \sim 33\; \textrm{dB}$) of reversible optical switching via external free-space focused laser beam and quasi-continuous tuning of GST-state was obtained by J. Zheng et al. [41] via integration of GST cell in silicon ridge microring, near-critically coupled when the cell is in the amorphous state. Novel $2 \times 2$ wavelength-selective optical switch based on GST-assisted silicon microring, demonstrated by C. Zhang et al. [209].

Electrical indirect-switching of PCM via external microheaters provides extending switching region (scalable on-chip integration), thus ensuring higher extinction ratio of photonic switches and modulators [184,210,211]. This method can be advantageous to obtain easier access to complex optical circuits, due to the independence of PIC and heating elements, the possible drawbacks are higher power consumption in comparison with all-optical evanescent switching and larger footprint. However, this method is beneficial to utilize in case of transparent PCMs (with weak or non-existent absorption coefficient) [71,195].

In particular, compact and energy-efficient reconfigurable $({\sim} 1000$ phase transition cycles) photonic switch based on GST cell placed on silicon micro-ring resonator triggered by electrical pulse-induced Joule heat from PIN junction (p-type intrinsic and n-type junction) resulting in ${\approx} 14\; \textrm{dB}$ extinction ratio near the resonance wavelength and less switching energy consumption (${\approx} \; 86\; \textrm{nJ}$) for one cycle in comparison with identical GST cell optical switching [212]. Furthermore, various reconfigurable switches and modulators (up to ${\sim} 0.09\pi /\mathrm{\mu}\textrm{m}$) based on large-bandgap low-loss PCM Sb₂Se₃ triggered by PIN silicon heater integrated in MZI and micro-ring resonator (MRR) were demonstrated in [198]. Interestingly, microring switch based on broadband transparent PCM Sb₂S₃ (with negative thermo-optic coefficient) integrated on SOI, makes the hybrid platform less sensitive to thermal fluctuations, obtains extinction ratio over $30\; \textrm{dB}$ [197].

Step-forward was done in the modeling and numerical optimization of the electrically driven phase-change integrated nanophotonic switches with external microheaters [213–215]. Notably, metal microheaters in the vicinity of optical structures insert drastic optical loss, while ITO external heaters cause lower additional optical loss [184,206]. In contrast, emerging low-loss photonics-compatible graphene and nanocrystalline graphene [216] is a reliable material to tune PCM temperature on-chip due to the ultrahigh thermal conductivity of graphene, which can be electrically controlled by CMOS technology. Furthermore, graphene reduces the switching power and increases switching speed in comparison with hybrid devices equipped with ITO and doped silicon microheaters, and it is compatible with both SOI and Si₃N₄ platforms. In particular, C. Rios et al. [217] demonstrated reversible switching and electro-thermal modulation of silicon nitride photonic circuits with embedded PCM cell (low-loss GSST) atop single-layer graphene microheater, which benefits from minimal insertion loss in comparison with absorptive metal microheaters and low switching power consumption $({\approx} 9\; \textrm{mW}$). Z. Fang et al. [218] demonstrated on-chip graphene-assisted reconfigurable and non-volatile photonic waveguide-switch based on integrated GST (endurance over 1500 cycles) and phase-shifter based on low-loss Sb₂Se₃ PCM cell deposited on micro-ring resonator with endurance over 1000 cycles. Notably, novel non-volatile electro-optic devices, which are electrically programmable and optically addressable based on integrated GST cells into silicon nitride circuits are demonstrated by N. Farmakidis et al. [219,220] which hold promise for ultrafast light routing on-chip in applications of hybrid in-memory optoelectronics computing.

4.1.2 Graphene-based modulators on-chip

There are well-established techniques to obtain phase light modulation of the signal-carrying optical mode propagating through PIC via tuning of effective refractive index of the mode, obtained via thermo-optic (reviewed in paragraph 4.1.3), electro-optic (reviewed in paragraph 4.1.4), opto-mechanical effects, free-carrier absorption and dispersion in silicon, as well as to obtain amplitude light modulation - via tuning of absorption coefficient by utilization modulators with integrated phase-change materials (reviewed in Paragraph 4.1.1) and electro-absorptive modulators based on graphene (active medium) [221–224].

Graphene is an appealing material [225,226] due to unique properties in comparison with other semiconductor materials and Si [102,108,227,228], it is successfully integrated in Si and Si₃N₄ [108] PICs as large-bandwidth modulators and broadband photodetectors.

Graphene is a flat monolayer of carbon atoms in a two-dimensional (2D) honeycomb lattice, was introduced by Novoselov et al. [222–224] and has attracted tremendous interest due to its well-controllable electrical and optical properties [229,230]. Besides key properties of graphene such as high-speed operation due to ultra-high carrier mobility [231] at room temperature and compatibility with CMOS technology, high thermal conductivity, uniquely the optical absorption coefficient of graphene can be tuned by shifting the electronic Fermi level from the Dirac point [232] through electric gating, namely absorption can be suppressed due to Pauli blocking mechanism [232], leading to ultra-broadband operation of graphene-based photonic modulators on-chip [233–235]. Furthermore, graphene can be integrated in close vicinity to photonic devices and even on microrings [108] and photonic crystal cavities without degradation of the Quality-factor demonstrated by A.P. Ovvyan et al. [216]. Furthermore, high thermal conductivity leads to energy-efficient utilization of graphene microheaters to build thermo-optic modulators [236,237] as well as to switch integrated phase change material cells described in Paragraph 4.1.1. In general, hybrid graphene modulators are based on gate-tuned interband optical absorption in single [102], double [108,227], and more [238] graphene layers deposited on insulator layer atop PICs on-chip.

The first waveguide-integrated graphene-based electro-absorption modulator was demonstrated by M. Liu et al. in 2011 [102], where evanescent interaction of optical-signal carrying mode propagating through Si waveguide with deposited atop single-layer of graphene with a spacer (insulator) of thin ($7\; \textrm{nm}$) Al₂O₃ layer results in optical signal modulation (depth $0.1\; \textrm{dB}/\mathrm{\mu}\textrm{m}$) over broadband wavelength range (1.3-1.65 µm) with high operation speed $1.2\; \textrm{GHz}$ at $3\; \textrm{dB}$. This was achieved by tuning of the Fermi level of single-layer graphene via applying gate voltage between doped silicon and graphene. The way to increase modulation depth via enhancing optical absorption is the utilization of several layers of active medium, thus M. Liu et al. [227] in 2012 demonstrated the first double-layer graphene modulator on SOI with Al₂O₃ insulator thin ($5\; \textrm{nm}$) layer in between, obtaining almost two times the single layer [102] modulations depth ${\sim} 0.16\; \textrm{dB}/\mathrm{\mu}\textrm{m}$ with the dynamic response up to $1\; \textrm{GHz}$ ($3 - \textrm{dB}$ bandwidth). An increase in the modulation depth up to $10\; \textrm{dB}$ can be obtained via enhancement of light-graphene interaction by gating single layer graphene which is coupled in microring resonator (Si) which was demonstrated by C. Qiu et al. [235] at telecom wavelength $1550\; \textrm{nm}$. Further progress was made on the improvements of graphene-coupled microring modulators, in particular, it is obtained a high modulation depth of $12.5\; \textrm{dB}$, which was obtained by designing a slightly undercoupling condition before tuning the Fermi level of graphene and correspondingly critical-coupling after tuning [239].

Drastic rise of modulator bandwidth up to $30\; \textrm{GHz}$ with ${\sim} \; 15\; \textrm{dB}$ modulation depth was demonstrated by C. T. Phare [108] in 2015 via integration graphene-graphene capacitor with Al₂O₃ ($65\; \textrm{nm}$.) interlayer spacer atop silicon nitride micro-ring, which is critically coupled in case of low loss, while with increase in the loss, the transmission through the bus waveguide rises. Thus, importantly, attenuation of signal-carrying light in the bus waveguide is the result of destructive interference in the coupling region due to tuning of the voltage-controlled absorption of graphene layers on the ring. Notably, a thick Al₂O₃ insulator between graphene layers leads to small capacitance and thus enables high demonstrated bandwidth ($30\; \textrm{GHz}$). Further, Heidari et al. [109] demonstrated the double-layer graphene modulator with improved working 3-dB bandwidth ${\sim} \; 60\; \textrm{GHz}$. Step forward was done also in the development of $\textrm{THz}$. graphene-based electro-absorption modulators [240–242].

Phase-modulation was theoretically proposed via electro-refractive graphene modulators integrated on the doped silicon waveguide in both arms of MZI by C. Xu et al. [243], where the modulation is performed in the region, where the Fermi level of graphene ${\textrm{E}_\textrm{F}}\; > \; 0.4\; \textrm{eV}$, which is low-loss ‘transparent’ for telecommunication working wavelength $1550\; \textrm{nm}$. Further improved and experimentally demonstrated ectro-refractive modulator was shown by V. Sorianello et al. [244], where the authors obtained gigahertz regime of modulation with static modulation depth of $35\; \textrm{dB}$. Notably, H. Shu et al. [245] demonstrated hybrid working regimes of graphene-based silicon Mach-Zehnder modulator which can be switched between electro-absorptive (obtained $25\; \textrm{dB}$. extinction ratio) and electro-refractive operation modes (utilized driving voltages are $1\; \textrm{V}\; -\; 3\; \textrm{V}$ to obtain $0.185\; \textrm{nm}$ spectrum-shift).

4.1.3 Thermo-optic modulators and phase shifters

Thermo-optic modulators are widely utilized due to their efficient phase shift modulation leading to high extinction ratio, broad optical band. wth, and ease-of-fabrication process for massive optical interconnects. Matrixes of arranged thermo-optic phase shifters and modulators were implemented in on-chip programmable nanophotonic processors [246] and optical neural networks [247–250].

Thermo-optic modulators are based on the change of effective refractive index of the signal-carried mode via Joule heating supplied by a resistive heater. Practically, thermo-optic modulators based on silicon are feasible due to rather big thermo-optic coefficient (TOC) $dn/dT = 1.87 \times {10^{ - 4}}\; [{1/K} ]$ [251–255].

Trade-off between modulation speed and power-consumption

Notably, engineering of thermo-optic modulators and phase-shifters requires finding a trade-off between modulation speed and power-consumption for particular applications.

Isolation of the modulator from beneath cladding obtained in freestanding thermo-optic modulators, where the waveguide is suspended, results in the reduction of the heat dissipation though the bottom cladding to the substrate leading tlow power consumption, however not optimized tuning speed and limitations in scalability [254,256,257]. While geometrically optimized design of dense distributed SOI waveguides with TiN microheater integrated in MZI was demonstrated by Chung et al. [258] enabled record-low switching power $P\pi \; \approx \; 2.5\; \textrm{mW}$. over $100\; \textrm{nm}$ optical bandwidth.

Maximizing overlap between the optical mode and the thermal heating profile of the nanophotonic device leads to a decrease in the length of the phase-shifter, resulting in low-power consumption, which was demonstrated by N.C. Harris et al. [166], where the resistance profile was developed by engineering different concentrations of dopant in ridge silicon-on-insulator (SOI) waveguide, obtaining $61.6\; \mathrm{\mu}\textrm{m}$ long phase shifter with $P\pi \; \approx \; 24.7\; \textrm{mW}$ and a $- 3\textrm{dB}$ bandwidth ${\approx} \; 130\; \textrm{kHz}$.

Improvement of power consumption can be obtained via more effective heat absorption by the signal-carries waveguides via their arrangement into spiral pattern in the MZI arms [252,259]. In order to minimize switching power further, A. P. Ovvyan et al. [260] proposed spiral-shaped geometry of the gold microheaters placed directly atop Si₃N₄ rib waveguide arranged in a spiral, where in case of determined optimal spiral-microheater-width ≈ half of the width of the rib waveguide it is obtained experimental minimum switching power $P\pi = 12.2$ mW with tuning efficiency $0.43\; \textrm{nm}/\textrm{mW}$., which compares favorably to published results, considering rather low thermo-optic coefficient of Si₃N₄ ${\sim} \; {10^{ - 5}}\; [{1/\textrm{K}} ]\; $ in the near-infrared region. Furthermore, placing microheaters in both arms of MZI allows doubling the spectral shift while cascading thermo-optic MZI filters lead to increased extinction ratio up to ${\approx} \; 36\; \textrm{dB}$ [260].

Graphene-based thermo-optic modulators

Indeed, thick spacing layer between waveguide and microheater leads to low response speed as well as heating efficiency. Improvement on the modulation bandwidth (switching rate), power consumption, and at the same time maintaining low optical loss is obtained by utilization transparent ITO-based and graphene-based thermo-optic modulators, where the latter have ultra-high thermal-conductivity, which facilitates fast transfer of Joule heat to the nanophotonic circuit as well as provides fast dissipation [261]. Further, moderate transparency (reviewed in Section 4.1.2) of graphene enables to place graphene-microheater directly atop nanophotonic device without cladding layer. In particular, phase shift was converted into amplitude modulation via graphene-based thermo-optic modulators based on MZI [262], microrings [263], resonators [264], and photonic crystal structures [237,265]. Notably, large mode volume of thermo-optic modulators based on microrings leads to low heating efficiency [263,264]. To increase heating efficiency and to obtain a small footprint, graphene nanoheater was integrated on the cavity region of photonic crystal cavity nanobeam (SOI platform), where due to enhanced light-matter interaction in the center of the cavity on the resonance wavelength (around $1550\; \textrm{nm}$) and ultra-small mode volume ($0.145\; \mathrm{\mu }{\textrm{m}^3}$.) it is obtained higher thermo-optic tunning efficiency $1.5\; \textrm{nm}/\textrm{mW}$. accompanied by resonance wavelength shift, where achieved rise time constant is $1.11\; \mathrm{\mu s}$ and fall-time −$1.47\; \mathrm{\mu s}$. [265]. Furthermore, to increase working wavelength range in comparison with cavities, Yan et al. [237] employed photonic-crystal slow-light waveguide with graphene heaters on top leading to improved tuning efficiency $1.04\; \textrm{nm}/\textrm{mW}$. over the whole telecommunication range with rising time of $750\; \textrm{ns}$ and falling time $-{-}\; 525\;\textrm{ ns}$.

The thermo-optic modulators advantageous from well-controlled optical insertion loss, however, they suffer from extremely slow modulation speed and due to large power consumption, the pattern for scaled-up nanophotonic circuits requires well-engineering of heat dissipation and the thermal crosstalk.

Overall, heterogeneous integration of cutting-edge platform to carry the signal with low optical insertion loss in the necessary transparency window with the active platform which enables realization of modulation/phase-shift with required targeting trade-off between power consumption, bandwidth and optical insertion loss fo. t particular application holds promise for a new generation of reconfigurable large-scale ultra-low-loss PICs.

4.1.4 Mach-zehnder interferrometer based electro-optic modulators

The MZI was pioneered by Ludwig Mach and Ludwig Zehnder [266,267] in the year 1881 as a more advanced iteration of the Jamin-Interferometer for the purpose of establishing a correlation between optical phase shifts and optical intensities via interference. The MZI operates by dividing a coherent laser beam into two separate arms, which are then combined once more after traversing a certain path length. Should both arms encounter an unequal phase shift, this will be directly reflected in the intensity fluctuations of the recombined beam, as the light’s phase serves to determine the constructive or destructive interference regime.

Constructing a PIC utilizing MZI geometry necessitates the utilization of two $50/50$ beam splitters, such as MMIs in section 4.3.4 or directional couplers, and ordinary waveguides, all of which are customary components in modern foundries. The selection of phase modulator is of greater significance since it determines the material, modulation speed, and modulation range. Thermal phase shifters and electro-optical phase shifters are two of the most prevalent types of phase modulators, necessitating a material with high thermo-optic and electro-optic response, respectively.

Thermal phase shifters, which have been demonstrated to be successfully integrated into MZIs, are typically resistive wires that are located in close proximity to one arm of the modulator [268]. Their presence affects the effective refractive index of the waveguide through Joule heating. However, the implementation of this mechanism is not without drawbacks. For instance, if the two arms of the MZI are not sufficiently separated, there may be high crosstalk between them, leading to a large device footprint. Additionally, thermal phase shifters are relatively slow and cannot operate in the GHz regime due to long thermal relaxation [269]. Further details about thermo-optical modulation and its drawbacks are given in section 4.1.3.

An alternative method to overcome the aforementioned limitations is through the utilization of electro-optical phase shifters which make use of the second-order nonlinear Pockels effect (refer to Section 3.4). The selection of material for these types of phase shifters is crucial; however, once an appropriate material has been identified, it can offer significant advantages. Strong Pockels materials such as LN (refer to Section 3.4.1) or BTO (refer to Section 3.4.2) exhibit an instantaneous change in the effective refractive index of a waveguide in the presence of an electric field. By exploiting this effect, a phase modulator can be constructed consisting of a traveling wave microstrip with a waveguide in between, as illustrated in Fig. 9(a). When an electric potential is applied to the microstrip, an electric field is generated which instantaneously passes through the waveguide and alters the phase of the transmitted light. This configuration ensures that the electric field is confined on one side of the MZI arm, thus minimizing the likelihood of crosstalk. Furthermore, it can be modulated at common CMOS electron speeds due to the instantaneous material response. Additionally, the Pockels effect is contingent upon the direction of the applied electric field, making it possible to apply a phase change with opposite polarity in the other arm. In a coplanar waveguide configuration as depicted in Fig. 9(b), a shared signal electrode can reduce the drive voltage by a factor of two. Additionally, a $2 \times 2$. MMI can be implemented to give rise to balance detection, that allows encoding of negative weights through modulation [178].

 

Fig. 9. Sketch of electro-optic modulator. (a) Shows the schematic of a phase shifter that consists out of a condensator with a waveguide in between. In (b) a coplanar waveguide is integrated in an MZI to make use of the directionality of the Pockels effect to reduce the half-wave voltage, thus the footprint of the MZI. The two output waveguides on the right of the MZI give rise to balance detection, that allows encoding of negative weights.

Download Full Size | PDF

State-of-the-art TFLN modulators are demonstrated to achieve bandwidth up to $100\; \textrm{GHz}$. [270] while offering extinction rations up to $30\; \textrm{dB}$, when integrated on SOI photonics [271]. The voltage-length product of TFLN EOMs is approximately $2.2\; \textrm{V}\; \textrm{cm}$, resulting in either large drive voltages or large modulator footprints. The comparably much larger electro-optic coefficient of BTO (see Section 3.4) offers great opportunities to counteract this issue. A voltage-length product of only $0.2\; \textrm{V}\; \textrm{cm}$ is demonstrated by F. Eltes et al. [272], reducing the device footprint by one order of magnitude, while maintaining the drive voltages. As in TFLN, the predicted bandwidth for TFBTO modulators exceeds the 100 GHz limit [273].

Table 1 compares several optical shifters and modulators made of different materials. Electro-optic modulators based on the strong Pockels materials and graphene-based modulators can achieve very high bandwidth. Notably, the graphene-based modulator can also achieve both low insertion loss and energy consumption [109]. These hybrid material-based optical modulators offer unlimited possibilities to further increase the computational speed and reduce the energy consumption of photonic neuromorphic processors.

Table 1. Comparison of optical shifters and modulators.

View Table | View all tables in this article

4.2 Memory

Optical memories for storing data are indispensable in a variety of computing tasks. Optical bit-level memories can be classified into volatile and non-volatile types, depending on if the stored data can be maintained without a power supply. Volatile memories can be implemented based on optical bistability, ion excitation, and recirculating loops [278,279]. Non-volatile memories are achieved based on PCMs. Nathan et al. compared the performance of different types of optical memories [280]. Optical memories utilizing PCMs offer a more suitable solution for PICs. This type of memory can achieve a small footprint of less than $1\; \mathrm{\mu }{\textrm{m}^2}$. per cell [31,219], while micro-ring lasers-based memories ($720\; \mathrm{\mu }{\textrm{m}^2}$ [281]) and semiconductor optical amplifier-MZI-based memories ($12\; \textrm{m}{\textrm{m}^2}$. [282]) have much larger sizes. Similar to their electronic counterparts, compact photonic components allow dense integration of PICs to improve performance. Furthermore, PCM-based non-volatile memories are highly energy-efficient because no energy is required to maintain the stored data after programming. Due to its significance in neuromorphic computing, we review here the progress of PCMs-based optical memories.

4.2.1 Non-volatile type

The characteristics of non-volatile memories are determined by the choice of PCMs and photonic structures. Table 2 lists some of the available PCMs [176,283] and their optical properties in different states. Figure of merit (FOM) is the ratio of the change in refractive index Δn to the change in extinction coefficient $\mathrm{\Delta }k$ when the phase is changed (FOM = $\mathrm{\Delta }n/\; \mathrm{\Delta }k)$. Data for scandium-doped antimony telluride (SST) [284], GeSb [285], GeTe [285], In₂Se₃ [286], and AgInSbTe [285] are not included because their information in other papers is plotted as graphical images rather than as original datasets. When the above PCMs are varied between amorphous and crystalline states, their optical properties at $1550\; \textrm{nm}$ change dramatically, thus ensuring significant changes in the light transmission of the memories.

Table 2. Properties of typical PCMs at 1550 nm

View Table | View all tables in this article

Memory based on different photonic structures

A variety of photonic structures, such as waveguides. rg resonators, MMIs, photonic crystal cavities, and metal-dielectric-metal waveguides, can be combined with PCMs to become the non-volatile memories. The simplest configuration is a waveguide with a deposited PCM cell [190,191,293–298]. The electric field of guided modes extends to the outside of the waveguide and interacts with the deposited PCMs, resulting in a change in light transmission. Figure 10(a) shows the schematic of a waveguide-based memory [291]. The material absorption of amorphous GST is low, while that of crystalline GST is high. As a result, the light transmission changes from $98\%$. to $25\%$ due to the phase transition of the $4\; \mathrm{\mu}\textrm{m}$-long GST patch. For waveguide-based memories, the transmission contrast can be improved by using different types of waveguides [191,291,292,299] and tuning the geometries (e.g., cascade PCM nanodisks [300] and multiple PCM wires [24]) and dimensions [181,288,291] of the deposited PCMs.

 

Fig. 10. Schematic of non-volatile memories based on various photonic structures. (a) Waveguide. Reprinted with permission from [291], Optica Publishing Group. (b) Ring resonator. Taken with permission from [187], AIP publishing. (c) MMI. Preprinted with permission from [210], ACS publications. (d) Metamaterial waveguide. Reprinted with permission from [292], Optica Publishing Group. (e) Plasmonic waveguide. Reprinted with permission from [219], American association for the advancement of science. (f) Photonic cavity. Reprinted with permission from [205], ACS publications. (g) Metasurface on waveguide. Reprinted with permission from [33], Springer Nature.

Download Full Size | PDF

By depositing PCMs on ring resonators, greater transmission contrast and lower programming energy can be achieved [41,185,186,208]. Figure 10(b) shows a schematic of a ring resonator-based memory [187]. The material loss variation due to the GST phase transition can strongly affect the round-trip loss insi the ring resonators. In this device, moderate light coupling is sufficient to satisfy the condition of critical coupling when the GST cell is in the amorphous phase. With a fixed gap size between the ring resonator and the waveguide, the increase in the round-trip loss results in an undercoupled ring resonator and therefore an increase in light transmission at the through port. By programming the small GST cell, which is only $0.5\;\mu m$-long, the light transmission can be switched between $0{\%}$. to $90{\%}$. The small volume reduces the energy required for programming, and the phase transition of the GST cell from amorphous to crystalline states requires only $5.4\;\textrm{ pJ}$. of energy.

If the PCM cell is to be electrically programmed, MMIs [186,210,301] can be selected instead of silicon strip waveguides. This is because the highly doped silicon strip waveguide used as a resistive heater is orthogonal to the silicon strip waveguide used to guide light. The crossing strips can introduce significant losses [302]. Figure 10(c) shows an example of an MMI-based memory [210]. Here, a highly doped silicon strip is used as a resistive heater to trigger the phase transition of the GST. Doped silicon attenuates light in the near-infrared range [158–160]. Due to the self-imaging principle, the light is focused on the central part of the MMI, so the size of the highly doped silicon rion is reduced, resulting in a lower insertion loss (less than $1\; \textrm{dB}$. for a $1\; \mathrm{\mu}\textrm{m}$-length GST cell [210]). In addition, the introduction of crossing has a negligible effect on the light transmission of the MMI [303]. The device of the µm-size GST cell achieves a high transmission contrast of more than $10\; \textrm{dB}$., where the energy consumption for reamorphization and crystallization is $10\;\textrm{ nJ}$. and $9\; \textrm{nJ}$, respectively [210].

Subwavelength grating cavities, photonic crystal cavities, and metal-dielectric-metal waveguides can be used to realize memories with lower programming energy consumption. Figures 10(d) and 10(f) show the schematic of n-volatile memories based on subwavelength grating cavities [292] and photonic crystal cavities [205]. At specific wavelengths, light undergoes multiple bounces in the cavity, resulting in a significant enhancement of the electric field. The enhanced electric fld can lead to enhanced light-matter interactions, dramatically increasing the absorption of light by lossy materials [307–312]. Therefore, if PCMs are deposited in the cavities, the light absorption by the PCMs can be significantly enhanced, thereby reducing the energy required for programming. The amorphization of the GST cell in the subwavelength grating cavity requires only $16\; \textrm{pJ}$., while crystallization requires $190\; \textrm{pJ}$ [292]. The GST cell in the photonic crystal cavity requires only $19\%$. of the programming energy to achieve $1\; \textrm{dB}$ transmission contrast compared to the GST on waveguides [205]. The disadvantage of these cavities is that they can only pass light of certain wavelengths, limiting broadband operation. Figure 10(e) shows the metal-dielectric-metal waveguide. For this waveguide, the reduced mode volume results in an enhanced electric field and reduced PCM volume, so less energy is required for programming. The switching energy for this configuration is as low a $16\; \textrm{pJ}$ [219]. However, the metal-dielectric-metal waveguides introduce additional insertion and propagation losses, which should be taken into account during design [313–315].

Meta-waveguides consisting of waveguides and metasurfaces are promising devices currently under development. Metasurfaces are two-dimensional arrays of subwavelength artificial structures capable of controlling light at the nanoscale [316–319]. A metasurface on a waveguide can control the propagation of light through the waveguide [320]. Figure 10(g) shows a schematic of a non-volatile memory based on a programmable meta-waveguide. The crystalline GST array on the waveguide can convert TE₀ mode to TE₁ mode, while the amorphous GST array cannot. The weights here are represented by the waveguide mode contrast between TE₀ and TE₁ modes, which can achieve up to 64 levels of modal contrast [33]. The integration of photonic integrated circuits with metasurfaces brings functional enhancements and numerous opportunities [320–327].

Write and erase of non-volatile memory

There are several programming methods for non-volatile memories, including on-chip optical pulses, on-chip electrical heaters, and free-space optical pulses. Table 3 summarizes the information on programming PCMs using the above methods in previous studies. In this table, the abbreviation GE stands for graphene. Figure 11(a) shows the schematic of on-chip optical programming based on a silicon nitride waveguide [31]. Intense light pulses in the waveguide are used to induce the phase transition in the GST cell. Amorphization requires rapid cooling to preserve the disordered state, so a single pulse is used. Crystallization requires sufficient time to recover the atomic order, so a train of consecutive pulses with decreasing energy is used. The device ($1\; \mathrm{\mu}\textrm{m}$ long GST cell) has a transmission contrast of $0.67\%,$ switching energies as low as $13.4\; \textrm{pJ}$., and speeds approaching $1\; \textrm{GHz}$. [31]. Increasing the switching energy to $596\; \textrm{pJ}$. ($5\; \mathrm{\mu}\textrm{m}$ long GST cell) achieves a greater transmission contrast of $58.2\%$ [31]. The on-chip optical programming works well on a single memory. However, in large-scale optical circuits, it is challenging to provide write and erase pulses for multiple PCM cells [64].

 

Fig. 11. Schematic of the various methods of programming PCM cells. (a) On-chip optical programming. Reprinted with permission from [31], Springer Nature. (b) On-chip electrical programming via an ion-implanted waveguide microheater. Reprinted with permission from [305], Springer Nature. (c) Free space optical programming. Reprinted with permission from [207], Optica Publishing Group. (d) On-chip electrical programming via a graphene microheater. Reprinted with permission from [217], Wiley.

Download Full Size | PDF

Table 3. Programming of PCMs

View Table | View all tables in this article

Electrical programming can be used to solve this problem. Figures 11(b) and 11(d) show two electrical heaters consisting of a highly doped silicon strip [305] and two-dimensional materials such as graphene [217], respectively. For the device using the highly doped silicon strip, crystallization and amorphization require $6.9\;\textrm{ nJ}$ and $8.8\;\textrm{ nJ}$ of energy, respectively. The device achieves low-energy consumption per unitodulation depth ($1.7\;\textrm{ nJ}/\textrm{dB}$ for the crystallization operation) and a high switching contrast of $158.5{\%}$ [305]. The graphene heater requires an energy of $111.8\;\textrm{ nJ}$ for amorphization and $110.4\;\mathrm{\mu}\textrm{J}$ for crystallization [217]. The energy required for electrical programming is greater than that required for optical programming. This is because, in optical programming, waveguides and claddings are usually lossless, and therefore light is only absorbed by PCMs. For electrical programming, the heated area is much larger than the size of PCMs, resulting in wasted energy.

In 2012, researchers investigated writing and erasing memories via free-space optical programming [207]. Figure 11(c) shows the schematic of the device. Light from a laser diode operating at $660\; \textrm{nm}$ was guided by a single-mode fiber and focused through a lens onto the GST. The size of the light spot matches the size of the $35\; \textrm{nm}$-thick GST cell. The GST in this device requires $6.4\; \textrm{nJ}$ ($40\; \textrm{ns}$ and $160\; \textrm{mW}$) and $20\; \textrm{nJ}$ ($400\; \textrm{ns}$ and $50\; \textrm{mW}$) for amorphization and crystallization processes, respectively. In 2013, another study (20 nm-thick GST) achieved lower energy consumption for amorphization and crystallization, $0.9\; \textrm{nJ}$ ($20\; \textrm{ns}$ and $45\; \textrm{mW}$) and $3.6\; \textrm{nJ}$ ($300\; \textrm{ns}$ and $12\; \textrm{mW}$) [208]. This study indicates that the energy consumption of free-space optical programming is comparable to that of on-chip optical programming with optimized PCM thickness and laser pulse width. Despite the relative complexity of the setup, the method of programming non-volatile memories using free-space optical pulse excitations remains promising.

4.2.2 Volatile type

Volatile memories require energy to maintain data storage and have relatively higher power consumption than non-volatile memories. Therefore, volatile memories are usually less frequently used in studies of photonic neuromorphic computing at present. Synapses in the mammalian brain can be categorized into long-term plasticity (LTP) [329] and short-term plasticity (STP) [330]. The LTP refers to permanent changes in neural connections, whereas STP refers to temporary changes in strength between neural connections. Volatile memories have great potential to mimic the STP in synapses [331]. Transition metal oxides, such as VO₂, can be used for volatile memories [172,293,332,333]. The complex refractive index of VO₂ at $1550\;\textrm{ nm}$ changes if VO₂ is transformed from an insulator ($n = 2.88$ and $k = 0.36$) to a conductor ($n = 1.58$ and $k = 2.63$) [334]. Figure 12(a) shows a schematic of a hybrid memory based on VO₂ [331]. The memory is biased with a voltage. If a voltage of 9.35 V is applied, VO₂ undergoes the phase transition. Combining an applied voltage of $8.5\;\textrm{ V}$ with an optical pulse of 150 ms and $490\;\mathrm{\mu}\textrm{W}$ also induces the phase transition. After programming, the device can remain written for at least $200$ hours if the voltage is continuously applied [331]. At a voltage $0.2\;\textrm{ V}$ lower than the transition voltage, a $23.5\;\textrm{ pJ}$ write pulse can induce the phase transition.

 

Fig. 12. (a) A volatile memory based on VO₂. Reprinted with permission from [328], ACS publications. (b) A GST-based volatile memory. Reprinted with permission from [296], Wiley.

Download Full Size | PDF

Volatility is usually determined by the characteristics of the PCMs. However, GST, which is widely used for non-volatile memories, can also be realized as a volatile memory. Figure 12(b) shows the schematic of a volatile memory based on GST [296], which is identical to a non-volatile memory. For non-volatile operation, a low-power readout pulse is used after writing, which does not affect the physical state of the programmed GST cells, as shown by the black curve in Fig. 12(b). As the power of the readout pulses increases, the GST cell is heated above the glass transition temperature, leading to recrystallization and return to the initial state. As a result, the memory based on GST becomes volatile, and its retention time depends on the power of the readout pulses and the state of GST after programming.

4.3 Architectures for mathematical operations

4.3.1 Incoherent crossbar arrays

Incoherent crossbar arrays are photonic processors which overlap optical pulses in time to perform matrix-vector multiplications (MVMs). The information is only encoded in the intensity, but not in the phase [30]. In order to obtain a phase insensitive accumulation, the pulses must be sufficiently detuned in wavelength [29]. The multiplications are directly implemented via attenuation of the pulses. The mapping between the MVMs and the working principle of the crossbar array is sketched in Fig. 13(a). The input vector is directly encoded in the intensity of the optical pulses, each pulse has a different wavelength to avoid interference. Next, the pulses propagate through the crossbar array structure, which equally distributes all input pulses to all output of the crossbar. In order to encode the matrix elements, each coupling structure between every input and every output has a separate variable attenuator. In this way, the coupling of each input to each output can be separately controlled and hence the matrix encoded. Finally, the combined pulse power of the overlapped and weighted input pulses can be measured at the output of the crossbar array, which corresponds to the output vector of the MVM. In addition to deploying high-speed modulators as the variable attenuators, also non-volatile PCM can be deployed for the multiplication [190], which increases the overall energy efficiency. This is especially intriguing for applications with constant matrices, for example, convolutions for edge detection as shown in Fig. 13(b).

 

Fig. 13. (a) The photonic crossbar array (PCA) performs MVMs by encoding the input vector in the intensity of optical pulses and the matrix weights into the transmission properties of the crossbar itself. The output vector is determined by measuring the intensity at the output of the crossbar. Reprinted with permission from [30], Springer Nature. (b) Exemplary convolution processing with a PCA. The various filters are encoded in the columns as depicted in (a) in the pixel values sent as input vectors to the photonic circuit. Reprinted with the permission from [30], Springer Nature. (c) The photonic computation scheme is compatible with WDM, allowing to perform several MVMs in parallel with a single PCA. The required wavelength multiplexers can be monolithically integrated with the PCA. Reprinted with permission from [64], De Gruyter.

Download Full Size | PDF

Apart from the robustness arising from the phase-insensitivity, it also directly enables parallelization via WDM. In this way, the large optical bandwidth easily spanning several THz is divided into several smaller computational channels with a bandwidth comparable to one of the electronic drivers. As the operations performed by the crossbar array are linear, those independent wavelength channels can be multiplexed together, processed by a single crossbar array, and finally demultiplexed again to obtain the individual results. Figure 13(c) shows an integrated photonic circuit consisting of the crossbar array itself together with the multiplexers (MUX) and demultiplexers (DEMUX) attached to the input respectively output. Wavelength multiplexers based on cascaded Bragg gratings ensure a low crosstalk between the computational channels and thus no deterioration of the precision due to the parallelization [64].

Hybrid electro-optic hardware accelerators complement analogue photonic computation units with analogue electronic processes to improve the scalability. Especially, charge accumulation inside a photodetector in combination with temporal encoding of the vector elements greatly improves the directly processable vector sizes [335]. This concept can be directly transferred to incoherent coherent crossbar arrays by modulating the matrix weights together with the input signal, virtually increasing the overall crossbar size [178]. In combination with coherent crossbar arrays based on homodyne detection, hybrid electro-optic systems enable the direct computation of matrix-matrix multiplications [336].

4.3.2 Micro-ring resonators

Mathematical operations can also be performed based on MRRs. Figure 14(a) shows a schematic of an optical processor proposed in 2012, which is the first optical processor implemented in full WDM [337]. Input vectors are represented by different intensities of light emitted by different laser diodes. The light of different wavelengths is multiplexed into one waveguide, which is then split into multiple channels to ensure parallel computing operations. The MRRs that selectively affect the transmitted light are controlled by heaters to perform multiplication operations. The accumulation of the light of different wavelengths is realized by a photodetector array. The device achieves $8 \times {10^7}$ multiplications and accumulations (MACs) per second at a clock frequency of 10 MHz. In addition to the WDM technology, the architecture can also employ mode-division-multiplexing (MDM) technology to further improve performance [338]. Figure 14(b) shows the architecture of the MRR crossbar array. The basic idea of this architecture is similar to the architecture proposed in 2012. Light of specific wavelengths in the horizontal (vertical) waveguides is selectively coupled to the MRRs and then multiplexed in the vertical (horizontal) waveguides [143,149]. The difference is that the MRR crossbar array has two inputs. The two inputs of forward and backward signals can be used for inference and training, respectively.

 

Fig. 14. Schematic of various MRRs-based architectures. (a) First on-chip multiplier based on WDM. Reprinted with permission from [337], Optica Publishing Group. (b) A processing unit based on MRR crossbar array. Reprinted with permission from [339], ACS publications. (c) A processing unit based on broadcast-and-weight. Reprinted with permission from [340], IEEE Xplore. (d) A photonic tensor core based on add-drop MRRs. Reprinted with permission from [341], Under CC BY license from Research Square. (e) A photonic tensor core with PCMs. Reprinted with permission from [177], American association for advancement of science.

Download Full Size | PDF

 

Fig. 15. (a) Example of a Unitary Matrix produced by beam-splitter-MZI configurations. Reprinted with permission from [343], American physical society. (b) Realization of a diagonal matrix achieved through tunable splitters and MZI with thermo-optic phase shifters. Reprinted with permission from [344], Springer Nature. (c) Large-scale realization of a coherent photonic matrix multiplier. Reprinted with permission from [248], Springer Nature.

Download Full Size | PDF

The “broadcast-and-weight” approach proposed in 2014 [340] has been widely used in neuromorphic computing [202,342]. Figure 14(c) shows the schematic of a device based on the “broadcast-and-weight” [144]. The output of each neuron has a unique wavelength, and the output light propagates in a broadcast waveguide. There are two tunable filter banks based on MRRs representing excitatory (positive) weight and inhibitory (negative) weight, respectively. The multi-wavelength input light in the broadcast waveguide is selectively coupled through the MRRs to a balanced photodiode to calculate the weighted sum of the input. The electrical signal obtained from the balanced photodiode is used to modulate the output of the laser. The output light with the unique wavelength is finally coupled to the broadcast waveguide to start the next round of calculations. In 2017, the same research group experimentally investigated the “broadcast-and-weight” approach [250].

Figure 14(d) shows a computational architecture based on add-drop MRRs [341]. The output light of different wavelengths from a laser array is modulated according to input vectors. The modulated light propagating in the same waveguide is demultiplexed by the MRRs. Subsequently, the demultiplexed light passes through an optical attenuator, and a multiplication operation is performed. The multiplexing is performed by the second MRR. The multiplexed light finally arrives at a photodetector, and the summation operation is performed. The advantage of this architecture is that the weighting components can be flexibly selected according to the computational requirements, e.g., PCMs can be used for energy-efficient multiplication operations [24,73] or high-speed Mach-Zehnder modulators (MZMs) can be used to accelerate the training processes. Figure 14(e) shows an example of weighting using PCM cells [177].

4.3.3 Mach-Zehnder Interferometers

Calculation architectures which utilize phase control instead of amplitude modulation have also been demonstrated through the use of integrated MZI structures cascaded together with controllable phase shifters. At its very core, the simple configuration of a beam splitter (typically a directional coupler), phase shifters on one or both arms of the following MZI, and a secondary beam splitter [343] are capable of generating an effective unitary matrix. From a physical perspective, the interference of the propagating optical signal when recombining, after experiencing a phase shift through both splitting and an external applied change, produces a signal with a changed amplitude and phase. When placed in a parallel sense with N inputs, producing an effective input vector, this is analogous in the mathematical sense to rotating each input vector by some phase $\phi $. When placed in a cascade, with this configuration sequentially applied after an input vector, this can be used to apply an N-dimensional Unitary Matrix. Research works have further shown that such a configuration produces an output that is analogous to a linear algebra matrix applied to some arbitrary input, similar to current matrix calculation paradigms like those found in a GPU or TPU [345].

Research works have further extended on this configuration to include a splitter with a tunable splitting ratio. The addition of these components creates a platform for effectively 2x2 optical gates, which when cascaded produce a matrix system with a diagonal structure. This modified architecture has been identified as a route to improving the performance of the original system through increased tenability, providing a means to adapt in real time to varying input, and has been explored in a feed-forward configuration as illustrated in Fig. 15.

To date, this architecture has been employed primarily on a silicon photonic platform. This can likely be attributed to the availability of high-quality, large-scale silicon foundries, the ubiquity of knowledge behind integrated photonic components, and the mechanisms for phase control the platform provides both inherently and which can be applied post-fabrication. Within this architecture, the most common phase shifter employed is that of exploiting the thermo-optic effect in silicon, wherein through the application of heat through electronic heaters the phase of light propagating through the silicon waveguide can be proportionally modified [344]. Compared to electro-absorption and electro-optic effects, this mechanism benefits from a constant absorption vs the modulating signal, whereas in free-carrier dominated effects the absorption of the system changes proportionally to the applied input signal, which hinders large-scale realizations. However, alternative phase shifters and absorbing materials are currently the subject of intense interest for this computational architecture to both boost performance and reduce the effective footprint of these devices. Systems such as Micro-Electro-Mechanical Systems (MEMS) on Silicon have been demonstrated as potential replacements for the current thermo-optic paradigm [346], alongside investigations into PCMs [31], doped photonic crystals [347], or even graphene as an absorptive material [248,348].

Large-scale realizations of this architecture are currently being explored, wherein a 4 × 4 matrix has been successfully demonstrated [248] and larger-scale realizations in the form of commercial prototypes are currently under development.

4.3.4 Multimode interferometer

MMIs are commonly used as combiners and splitters in interferometers such as MZIs [350–353]. A recent study proposed that the MMI can also be used as an optical convolution processing unit [349], as shown in Fig. 16. Input vectors are encoded into four incoherent lights with different wavelengths and fed into the MMI-based processing unit. The processing unit consists of the two MMIs with a transfer matrix of M and a phase shifter array with a transfer matrix of $\Phi $. The phase shifter array is programmed based on the thermo-optic effect to control the intensity distribution in the MMI [354]. It is worth noting that the kernels are correlated to each other. That is, when one kernel changes, the rest of the kernels change with it. If kernel elements need to be programmed individually, the crossbar arrays described in section 4.3.1 can be used. Each output port of the MMI-based processing unit is equal to the weighted sum of the input vectors. One of the four ports can be selected as a ground line and subtracted from the remaining three ports to obtain negative values. Hence, the MMI-based processing unit can process three convolutional computing operations in parallel. This individual processing unit provides a new direction for future research.

 

Fig. 16. (a) Schematic of the optical convolutional processing unit (OCPU) based on MMIs. (b) Schematic of the convolution process for extracting the feature map. The 28 × 28 input image is divided into 27 2 × 28 sub-images. After flattening, the one-dimensional vectors of the sub-images are fed to the OCPU to perform convolutional processing. Reprinted with permission from [349], Springer Nature.

Download Full Size | PDF

Table 4 presents a performance comparison of photonic neuromorphic processors based on the different architectures discussed in section 4.3. All these photonic neuromorphic processors demonstrate their potential in terms of computing performance and low energy consumption. Notably, the 8 × 8 crossbar array achieves a high compute density of $162\; \textrm{TOPS}/\textrm{m}{\textrm{m}^2}$. Due to the high refractive index of Si, which enables a small bending radius, each cell in the Si crossbar array has a small footprint of $127\; \mathrm{\mu}\textrm{m}\; \times \; 127\; \mathrm{\mu}\textrm{m}$ (Si₃N₄: $285\; \mathrm{\mu}\textrm{m}\; \times \; 354\mathrm{\mu}\textrm{m}$). With future scaling, the compute density of the $64\; \times \; 64$ Si crossbar array is expected to reach $880\; \textrm{TOPS}/\textrm{m}{\textrm{m}^2}$ [30]. A comparison between photonic and electronic processors in terms of MVM operations can be found in the following study [14,22,30,349,355,356].

Table 4. Comparison of 5-bit precision photonic processors.

View Table | View all tables in this article

4.4 Activation functions

Extending beyond traditional integrated photonics components, the development of materials and device constructs which demonstrate non-linear optical responses has been the subject of great interest, wherein a non-linear change in the optical signal or material dispersion can be obtained through applied electronic, optical, thermal, mechanical, or chemical stimuli. In the context of neuromorphic photonics, the novelty of these responses is their analogous nature to current mathematical functions used for AI, coined as activation functions owing to their biological inspiration [22]. Full realizations of these activation functions in the photonic regime would be invaluable for pursuing Physical Neural Network systems that leverage inherent bandwidth and speed advantages, while serving as a suitable base for exploration of next generation spiking and reservoir photonic networks, which rely heavily on such highly non-linear responses [362].

To date, an array of photonic activation functions have been demonstrated across a range of PIC material platforms and employing a range of both physical mechanisms and engineered structures to achieve a suitable response. Two broad categories have emerged: activation functions which employ an opto-electronic conversion which is then re-broadcasted using a second photonic element, and functions which apply a non-linearity to the currently propagating optical signal. The first of these generally employs photodetectors or balanced photodetectors following the computational cell, which sums the resulting signals across the entire bandwidth and uses the generated electronic signal to drive a non-linear photonic element, commonly denoted as the “Broadcast and Weight” architecture [344].

Early instances have successfully employed an electrically driven laser to achieve the desired activation function. Through careful selection of an active semiconductor gain medium, the demonstrated laser cavity effectively produces a spiking optical signal when the applied voltage drives the gain of the laser above unity [344]. Moreover, the systems exhibit temporal integration of the input optical signal prior to spiking, which in conjunction with an appropriate photodetector to provide both inhibitory and excitatory signals, induces spiking behavior. Ensuring that the photodetector/absorber voltage inputs can induce lasing, the inherent non-linearity of the laser serves to create a delta-spiking function with a short refractory period. Works have demonstrated this architecture using a VCSEL with a suitable absorber in a 2D semiconductor stack, or through implementation of a full balanced photodiode architecture with a bias-able laser on chip [357,358], as shown in Fig. 17.

 

Fig. 17. (a) Illustration of the Photonic Leaky Integrate and Fire activation function, achieved through an on-chip detection and re-emit schema. Reprinted with permission from [357], IEEE Xplore. (b) Representation of various proposed electro-optic nanostructures and mechanisms. When employed in a broadcast and weight configuration, a range of non-linear photonic absorption responses can be achieved. Reprinted with permission from [361], Optica Publishing Group. (c) Realized Sigmoidal photonic activation function achieved through the use of two bias lasers, a driving laser, and two semiconductor optical amplifiers. Reprinted with permission from [362], Optica Publishing Group. (d) Realization of a photonic sigmoidal activation function achieved through a thermally tuned MRR attached to one arm of an MZI following a Mach–Zehnder coupler (MZC). The inset numbers in order refer to the change in phase produced by the MZC, the MZI, and the wavelength detuning. Reprinted with permission from [363], Optica Publishing Group.

Download Full Size | PDF

Alternatively, the use of an electro-optic MZM, applied to a secondary laser source, has been shown as a route to achieve a photonic activation function. Rather than relying on the inherent non-linearity of the laser, the MZM structure provides more control over of the output signal. With proper design and construction of the modulator, the amplitude of the output laser can be modulated from nearly full transmission to full annihilation, corresponding a sinusoidal voltage versus transmission curve [359]. This has been postulated in theoretical works and demonstrated in an off-chip configuration in conjunction with silicon MRR weight banks to produce a neuromorphic photonic system [251]. Moreover, a distinct advantage presented by this platform is the implementation of electrical non-linearities in the connecting integrated circuit to introduce programmability. As has been previously demonstrated, the use of a balanced photodetector system in conjunction with a transimpedance amplifier (to obtain signals large enough to propagate through a larger circuit), an integrated circuit with built-in non-linearities introduces the ability to incorporate tanh- and ELU activation functions that are typically employed in machine learning with minimum overhead [360].

Systems have also demonstrated employing current optical amplification technologies, namely semiconductor optical amplifiers (SOA), as a means to generate an activation function. Works have shown that differential biasing two opposing SOAs, separated by an effective $\pi $ phase shift, allows an input pulse to achieve gain in either one or two lasers depending on the input optical signal. This produces a resulting sigmoidal activation function that is analogous to current ML non-linearities [362].

The advantages offered by the detection and re-broadcasting platform are self-evident: The use of balanced photodetectors provides a means to integrate a wide bandwidth of photonic signals while providing a large operating point through the use of positive and negative arms of the photodetector. Moreover, intelligent techniques of circuit filtering, amplification, etc. can be utilized to modify the response of the activation function. However, a significant challenge is presented in achieving very high-frequency operation with this architecture. The conversion and re-conversion from optical signal to the electrical signal, adds overhead that slows the max operation speed, specifically attributed to intermediate electronic amplification, sampling, etc. [364]. Alongside this, the necessity of photodetectors requires a PIC platform that can support such devices (such as SOI with Ge, or InP) to remain in one integrated package, or coupling to photodetectors off chip. However, current realizations of this architecture and further development appear very promising as a route to achieve non-linear activation functions for neuromorphic photonic devices.

The second category of photonic activation functions contain devices which have a non-linear element that acts on the propagating photonic signal. This non-linear element is commonly an on-chip component that is tuned through external stimuli prior to or during operation, and which produces a non-linear response either dependent or independent of the power or phase of the propagating signal from the photonic computational cell. In this configuration, the complete summation of all wavelength signals is typically not employed, as there are no photodetectors or systems which can inherently sum all signals, but rather the activation function applies to each propagating signal.

For instance, the well-known SOI platform has been able to exploit electro-absorption to achieve non-linear modulation when Ge is further deposited on top. Through the application of an applied external voltage, the number of free carriers in the p-i-n junction can be drastically modified, thereby increasing the absorption of the system [365]. Typically, this absorption is wavelength-dependent, and will produce a blue-shifted absorption peak with increasing voltage. Alternative materials such as ITO have also emerged as electro-absorption materials to generate photonic activation functions. When deposited as a cladding layer on a typical SOI waveguide with appropriate electrical contacts, the propagating light in the waveguide can be absorbed proportional to the applied voltage. Compared to the silicon platform, the ITO platform presents a highly non-linear absorption that changes faster than the Si platform, while exhibiting improved absorption at lower voltages [366].

Utilizing integrated optical components which can produce a non-linear response, and which can be tuned through external stimuli has also been postulated as a route to achieve photonic activation functions. Employing an SOI platform and a ring resonator added to one arm of an MZM, along with the addition of thermally tuned free carrier dispersion controllers via heaters at segmented regions of the system, a non-linearity can be added with a programmable response can be tuned a-priori. This response exhibits a programmable response that can be tuned from ReLU to Sigmoidal, to even softplus with impressive performance [363].

Various nanomaterials and nanoscale systems have also been proposed and show significant potential. In the all-optical regime, CdSe QDs in a plasmonic hybrid system have been suggested to produce a highly effective non-linearity, while other electro-optic platforms such as graphene, QDs, QWs, and excitons demonstrate performance that should be uniquely suited for this role [361].

Though this second category presents promising results in removing the bottleneck of E-O conversion and electrical elements while remaining suitable for more material platforms, a number of challenges must be addressed. Typically, the optical non-linear elements present a response that is wavelength-dependent, and in many cases a significant deviation in the response is something that must be accounted for depending on the input signal. Moreover, the necessity of a-prior tuning through fabrication means or with stimulus at a slower rate compared to electronic signaling poses a challenge in both the degree which the non-linearity can be programmed and the rate at which it can be changed.

Ultimately, a number of different mechanisms, material platforms, and architectures have to date been demonstrated, and the realization of programmable activation functions for classical and spiking photonic neural networks, either wavelength-dependent or independent, is rapidly approaching and presents an exciting future for neuromorphic photonics.

4.5 Spike generation and artificial neurons

As described in Section 2, the artificial neuron model contains multiple inputs, input weights, weighted summation of inputs, an activation function, and an output. Section 4.3 details the various PIC architectures used to perform convolutional operations and weighted summation of inputs. Section 4.4 provides a comprehensive review of the various methods used to implement the activation function. By combining the architectures used to implement weighted summations with non-linear structures, PICs can successfully simulate artificial neurons. Figures 18(a) and 18(b) show the schematic of a PIC-based artificial neuron and the corresponding functional photonic structures in the PIC [32]. Figures 18(c) and 18(d) show the image of fabricated three neurons and the SEM image of a ring resonator used to realize the activation function [32]. Input pulses of different wavelengths are fed into the PIC. If the PCM cells of the plastic synapses are in an amorphous state, the transmission of the input light is higher, indicating a stronger connection between the neurons. If the PCM cells are in a crystalline state, most of the light in the waveguide is absorbed, which represents a weaker connection between neurons. The input pulses of different wavelengths are coupled to a single waveguide (weighted sum), which then reaches the spiking neuron circuit. If the power of the weighted sum of the input pulses exceeds a threshold, the PCM state on the ring resonators will change, resulting a change in the resonance condition of the ring resonator. The probe pulses previously coupled into the ring resonator will continue to propagate in the original waveguide, so that the probe pulses (spike) can be detected at the output. In this way, the PIC-based artificial neuron is realized.

 

Fig. 18. Schematic of (a) the spiking neural circuit and (b) functional photonic structures. (c) Optical image of three fabricated neurons. (d) SEM image of a fabricated ring resonator used for the activation function. Reprinted with permission from [32], Springer Nature.

Download Full Size | PDF

4.6 others

4.6.1 Couplers

Coupling from optical fibers to on-chip photonic logic systems is one of the key challenges for photonic integration. The methods to solve this challenge can be generally divided into two different approaches [367,368]: In-plane coupling and out-of-plane coupling.

In the case of in-plane coupling, the light emitted by a single-mode fiber is coupled directly into the input facet of the on-chip waveguide. To achieve a high transmission rate, the Gaussian-like beam emitted by the fiber and the fundamental mode of the on-chip waveguide need to be matched. For high-efficiency coupling, optimized fiber geometry [367], on-chip spot size converters [370], or adding polymer, SiO₂, Si₃N₄ structures [371–373] involving additional fabrication steps are exploited. Additionally, in-plane coupling results in high coupling efficiencies over a wide wavelength range [374,375], but strongly restricts the design of the on-chip photonic integrated circuits because all the waveguide inputs need to be routed to the edge.

On the other hand, for out-of-plane coupling, the light beam is incident from a fiber placed vertically on top of the chip. The light is redirected by a suitably designed structure, which modifies the propagation direction of the chip. Therefore, the design constraints induced by the in-plane coupling are circumvented. Also, this offers more space at the edge of the chip for photonic or electronic packaging, i.e., bonding pads at the edge of the chip [376].

For this approach, grating couplers are the most commonly used design approach, which are periodic, planar structures scattering the light out of the plane [367]. A schematic illustration of the design and simulation results showing the light propagation through such a device is depicted in Fig. 19.

 

Fig. 19. (a) Schematic illustration of a cut through a grating coupler along the propagation direction of light in the on-chip waveguide. The mode is enlarged by the taper and afterward redirected and focused by a grating with the grating period Λ and a linear apodization of the filling factor FF = d/Λ. The emitted Gaussian-like beam matches the mode field diameter of the angled single-mode fiber at its focus point. (b) Top view of the same grating coupler. (c) 2D FDTD simulation of the optical power profile resulting from coupling out of a grating coupler. The scattered beam possesses a focus point at around 175 µm and is incident onto the fiber with an angle of 12°. (d) Mode field diameter matching of the SMF-28 fibers at λ = 1550 nm and the focused beam at 175 µm atop the grating coupler. (e) Poynting flux spectra computed in the wavelength range for upward-(blue) and downward-diffracted (orange) light, for light transmitted from 1500 nm to 1600 nm at the end of the grating structure (red), and for the back-reflected light (green). All the power is normalized to the one at the grating’s input. Reprinted with permission from [369], Under CC BY license from Research Square.

Download Full Size | PDF

A great advantage of the grating couplers is that they can be fabricated in the same step as the on-chip waveguides. For standard grating couplers, the directionality of the scattering results in a maximal transmission of $- 3\; \textrm{dB}$ as upward and downward diffraction are completely equivalent [369]. An apodization of the filling factor results in enhanced scattering efficiency and at the same time achieves a focusing effect of the beam at a distance of $100 - 200\; \mathrm{\mu}\textrm{m}$ [377].

To further enhance the transmission characteristics of the devices, distributed Bragg reflectors (DBRs) [379], metal mirrors [380], or dual-etch gratings [381] are used to reduce the downward diffraction. Those devices possess a maximum peak efficiency of $- 0.5\; \textrm{dB}$.

Due to the periodically arranged grating, they show a narrower bandwidth and are sensitive to the polarization of the input light [369]. Bandwidth always comes at the price of transmission. Additionally, the grating-based structures possess a large footprint due to the adiabatic increasing of the mode field diameter.

An approach where the easily addressable devices and the low number of additional fabrication steps of the grating couplers, but also the high transmission over a long wavelength range of in-plane coupling, are combined, is 3D-printed polymer couplers. The polymer couplers can be divided into three different sections. First, an adiabatic 3D polymer positive taper with simultaneously 2D waveguide negative taper efficiently interfaces the 2D on-chip waveguide with the 3D-printed polymer waveguide. Afterward, the mode is widened in a non-adiabatic taper, which works as a transition from a highly confined mode to a freely propagating beam inside the polymer. As a last step, the total reflection plane in combination with a lens redirects the beam out of the chip plane and focuses the diverging beam on the input facet of the single-mode fiber [378]. Artistic illustrations, SEM, as well as optical micrographs and a 2D-FDTD simulation of light propagating through such a device, are depicted in Fig. 20.

 

Fig. 20. (a) Schematic illustration of a 3D total internal reflection coupler attached to an integrated photonic circuit. (b) 2D FDTD simulation of the optical power profile of such a device. (c) SEM-micrograph of a 3D total internal reflection coupler attached to an integrated photonic circuit. (d) Optical micrograph of a photonic integrated chip using 3D polymer couplers. Reprinted with permission from [378], Optica Publishing Group.

Download Full Size | PDF

Using 3D couplers, a coupling loss smaller than $- 1.8\; \textrm{dB}$ in a wavelength range between $1480\; \textrm{nm}$ and $1620\;\textrm{ nm}$ has been achieved [382]. Additionally, in a range between $730\; \textrm{nm}$ and $1700\; \textrm{nm}$ an incoupling loss under $- 8\; \textrm{dB}$ has been reported [382]. 3D couplers show a high $- 1\; \textrm{dB}$ alignment tolerance regarding optical fibers of $5\; \mathrm{\mu}\textrm{m}$. For the fabrication, planar electron-beam lithography and 3D direct laser writing are combined, inducing one additional fabrication step in comparison with standard grating couplers.

In summary, the ideal coupling mechanism is strongly dependent on the wished application. For broad-band low-loss fiber-to-chip coupling, in-plane coupling, or 3D polymer couplers can be used. In-plane coupling has the strong disadvantage of a reduced number of addressable devices. On the other hand, 3D polymer couplers induce an additional fabrication step, which can be bypassed by using grating couplers, which can be written in the same step as the photonic logic. Those grating couplers can be optimized by using backside mirrors for a small wavelength regime, but do not show the broadband low-loss characteristics in comparison with the other coupling approaches.

4.6.2 Trimming of fabricated structures

For structures in photonic neuromorphic processors, accurate fabrication is imperative. This is because the performance of photonic structures, especially cavities such as ring resonators, is very sensitive to variations in the refractive index of the materials used, wafer thicknesses, and the dimensions of the fabricated structures. Hence, after fabrication, it is necessary to fine-tune the properties of the fabricated structures to compensate for deviations to achieve the desired performance.

Several methods are available for fine-tuning the optical properties of fabricated photonic structures. Microheaters based on the thermos-optic effect are a common choice. For a silicon ring resonator, microheaters with a heating power of approximately $10\; \textrm{mW}$ can achieve a peak shift of $5\; \textrm{nm}$ [389]. Liquid crystals can also be used for trimming because their optical properties change with applied electric fields. Figure 21(a) shows the schematic of a photonic structure with a cladding layer of polymerizable liquid crystals [383]. Unlike common liquid crystals, the resonance wavelength shift can be fixed if the polymerizable liquid crystal layer is irradiated with UV light. This means that the applied electric field can be removed after polymerization. A fine-tuning range of less than $0.56\; \textrm{nm}$ can be achieved using the polymerizable liquid crystals [383].

 

Fig. 21. Various post-fabrication trimming methods for PICs. (a) Polymerizable liquid crystals. Reprinted with permission from [383], Optica Publishing Group. (b) Altering the volume of PMMA cladding with UV light. Reprinted with permission from [384], ACS publications. (c) Tuning the refractive index of HSQ cladding by laser annealing. Reprinted with permission from [385], Optica Publishing Group. (d) Under electron beam irradiation, the HSQ cladding provides a greater range of fine-tuning. Reprinted with permission from [386], Optica Publishing Group. (e) Implantation of Germanium enables wafer-level fine-tuning. Reprinted with permission from [387], IEEE Xplore. (f) Wafer-level trimming by etching technology. Reprinted with permission from [388], Optica Publishing Group.

Download Full Size | PDF

Fine-tuning can also be achieved by irradiation of light, electron beams, and ions or other particles. The volume [390–392] or refractive index [385] of cladding layers on the photonic structures can be changed under light irradiation. Figure 21(b) shows the schematic of a cladding layer trimming with UV light [384]. Poly(methyl methacrylate) (PMMA) undergoes chain scission and crosslinking under UV radiation, resulting in changes in the volume of the PMMA cladding. By adjusting the volume of the PMMA cladding, the resonance peaks can be shifted by more than $1\; \textrm{nm}$. In addition to PMMA, HSQ can also be used as a cladding layer. As shown in Fig. 21(c), the refractive index of HSQ increases from approximately $1.39$ to $1.45$ by local laser annealing to $1150^\circ \textrm{C}$ [385]. Theoretically, the maximum peak shift achievable at $1550\; \textrm{nm}$ can be as high as $3.63\; \textrm{nm}$. HSQ can also be tuned by electron beams. At high exposure intensities of electron beams, the refractive index of the HSQ can be further increased to $1.62$ [386] due to the formation of silicon rich SiO₂ [393,394]. In the device shown in Fig. 21(d), a resonance peak shift of up to $3.08\; \textrm{nm}$ was measured [386]. Implantation techniques have great potential to enable wafer-scale trimming. As shown in Fig. 21(e), for a single ring resonator, a small portion of the Si ring resonator was implanted with Ge. The implantation of Ge causes disorder to the Si lattice, and the crystalline Si in the implanted region is transformed into amorphous Si with a different refractive index. Upon annealing, the Si in the implanted region can recrystallize and return to its original state, thus improving the tunability of the device. The peak shift achieved is $8.5\; \textrm{nm}$, which is larger than the free spectral range of the device. Wafer-scale trimming has been achieved with this approach, as shown in Fig. 21(e) [387]. Figure 21(f) shows another wafer-scale trimming technique where peak wavelength tuning is achieved by partially etching a Si₃N₄ layer on the Si structures [388]. All of the above methods have a positive impact on the accurate computation and future mass production of photonic neuromorphic processors.

5. Summary and outlook

As summarized, the prospect of incorporating neuromorphic photonic architectures into current and emerging computation systems is incredibly attractive for extending AI applications into the future. Through leveraging the inherent advantages of parallelism and low latency offered by photonic components, this technology is well poised to provide a platform for low-power, high-speed calculations at a considerable scale. The work herein has presented a comprehensive overview of hybrid PIC technology. The range of photonic platforms currently available has been surveyed and assessed with respect to the strengths of each, including the performance of passive optical components and the availability of active components. This was supplemented with an overview of reconfigurable and functional materials which can be integrated into the respective PIC platforms, their methods of control, and what they offer regarding intensity/phase modulation, memories, or nonlinearities for activation functions. The foundational PIC components that can be combined to realize neuromorphic photonic computing architectures are also presented, including crossbar arrays, MRRs, MZIs, and MMIs, alongside the use cases of each.

As reviewed in this paper, hybrid material platforms have enabled PICs to make great strides in emulating artificial neurons and performing convolution operations. However, there is still room for progress in device integration and performance improvement. Currently, parallel computation in photonic neuromorphic processors relies heavily on WDM technology. In the case of a crossbar array with PCM, for example, a comb source is demultiplexed through array waveguide gratings, and input vectors are encoded into the amplitude of each comb tooth and then sent to the MAC unit. Off-chip devices are typically used in this step. With intensive research on on-chip III-V semiconductor light sources [395], microresonators (microcomb) [396,397], and arrayed waveguide gratings [398,399], the integration of integrated indium phosphide lasers and silicon nitride microresonators on a silicon substrate [400], as well as the integration of silicon nitride microresonators and arrayed waveguide gratings on a chip [401], has been successfully realized. Since the miniaturization of devices helps to reduce costs, improve portability, and develop new applications, integrating these components with photonic structures that perform mathematical operations to miniaturize photonic neuromorphic processors will be a challenging but rewarding endeavor. In addition to WDM, there are more multiplexing techniques in the photonic domain, such as MDM, polarization division multiplexing, space division multiplexing, and time division multiplexing, which provide more options for improving the parallelism of the system [28]. The combination of multiple multiplexing technologies [402] holds great promise for designing new architectures [33] and increasing bandwidth [338]. With the option to multiplex such a large amount of signals over a single bus, the number of calculations that can be achieved may increase by orders of magnitude, providing a means to dramatically increase the efficiency and reduce the power consumption of such system. Finally, systems combining photonic devices from different fields, such as metasurfaces [320] and vertical-cavity surface-emitting lasers (VCSELs), with PICs can enhance the functionality of photonic structures in PICs and contribute to the miniaturization of photonic neuromorphic processors. Photonic structures combined with metasurfaces enable compact mode converters [33,327], mode multiplexers [403], wavelength demultiplexers [404], and polarization beam splitters [405]. These devices are expected to lay the foundation for the realization of miniature PICs based on multiple multiplexing technologies. The 3D vertical integration of VCSELs and PICs [406–409] can provide flexible programming of non-volatile memories without additional signal loss.

Looking ahead, hybrid material platforms, multiple multiplexing techniques, and the introduction of photonic devices from different fields will drive the development of photonic neuromorphic processors based on PICs. As research continues, we expect to realize more advanced photonic neuromorphic processors that will revolutionize the computing paradigm and facilitate the flourishing of AI-based applications.