1. Introduction

The chromatic dispersion of light is fundamental to optical spectroscopy, spectral microscopy, and multiplexed imaging. Chromatic dispersion is most often implemented using either diffraction gratings or prisms, and diffraction gratings are often favored over prisms due to the large number of commercial options that generally prioritize spectral resolving power over signal-to-noise ratio. However, photon-starved applications are emerging in video microscopy and microendoscopy for which the collection efficiency of light is high priority.

The absolute efficiency for many commercial transmission and reflective gratings tends to peak around 70% for the design wavelength. However, for many types of commercial gratings there is a dramatic loss in the absolute efficiency as the incident wavelength strays from the design wavelength by approximately 100-300 nm (sources available online: Thorlabs near-infrared transmission grating GT125-03, near-infrared reflective grating GR25-0608, visible transmission grating GT25-06V, visible reflective grating GR25-0605, and visible reflective holographic grating GR25-18V data sheets) and volume phase holographic transmission gratings which have a peak transmission efficiency of 90% for bandwidth of approximately 100 nm, sufficient for applications such as optical coherence tomography. (sources available online: Thorlabs volume phase holographic transmission gratings GP3510M data sheet) Alternatively, prisms offer a comparable peak transmission efficiency, which is maintained over a far wider wavelength range. Refraction does not exhibit a significant multi-ordered dispersion as is well known for diffraction, which contributes to the enhanced transmission of prisms versus gratings. Unfortunately, single element prisms exhibit a linear dispersion in energy scaling, leading to a compression of the blue-green wavelengths, whereas diffraction creates the familiar linear dispersion in wavelength that better separates commonly used optical signals and fluorophores. Compound prisms can in principle address this however they are highly complex and therefore they are unavailable commercially.

Recognizing the potential of compound prisms and the need to address the design complexity, Hagen and Tkaczyk introduced first principles and a software optimizer to establish the first compound prism designs [1–3]. We expand on these principles laid out by Hagen and Tkaczyk in order to further improve the prism array design process as shown in Code 1 [4], the GitHub repository for which has been provided in Supplement 1. The overall optimization process is restructured around the use of information theory. Rather than using a typical merit function as is seen in many direct optimization processes, we perform a multi-objective optimization of the linear dispersion, transmission, and the physical size of the array. In order to efficiently perform this multi-objective optimization, we employ a genetic evolutionary algorithm. This allows for the user to progressively screen designs for a desirable balance of the optical and geometric design characteristics to best fit their custom application. This requires the simulation of a large number of optimization generations, each with many individual prism designs. In order to facilitate this large scale simulation, Rust and CUDA-based calculations are used to both improve the speed of individual calculations (Rust), as well as to perform a vast number of calculations simultaneously (CUDA). This new approach enables an unprecedented ability to rapidly design numerous prism arrays, requiring only a few milliseconds to design an individual array. In contrast, existing software packages require expertise and an intricate setup of merit functions to optimize a single design.

Here we apply the new designer to design a compound prism for a hyperspectral detector. Multiplexed microscopy is a growing field where higher efficiency dispersion is of use. Multiplexed imaging is important for detecting and distinguishing multiple fluorescent markers. In this example we are attempting to separate multiple fluorescent dyes over 500-820 nm. We simulate the prism’s dispersion profile and the corresponding geometries necessary to achieve this. We compare the performance of multiple-element prism arrangements in terms of their transmission efficiency, dispersion qualities, and geometries. We provide examples of some potential prism models, including the compound prism used in our own hyperspectral microendoscope. This finely tuned prism array exemplifies how a multitude of complex compound prism designs are now achievable, paving the way for the design of compound prisms for a number of applications and varying wavelength ranges in optical spectroscopy and microscopy.

2. Methods

2.1 Vectorized system parameterization

The foundation of the designer software ray tracing process is established through a vectorized parameterization of the optical system (Supplement 1). Ray-surface interactions are determined using the vector form of Snell’s law and Fresnel equations in order to calculate direction and transmission of a refracted ray.

A Monte Carlo simulation is used to propagate rays at a variety of wavelengths and initial positions as determined by the user-defined incident beam. These rays are traced from the beginning of the compound prism to the surface of a user-defined detector where their direction vector is defined as $\hat {s}_a$. The point of the ray-detector intersection for wavelength $\lambda$ and initial position y is $S(\lambda,y)$. Additionally, each ray’s transmission is defined as $\vec {T}(\lambda,y)$, given that it falls within the upper and lower bounds, $ub_d$ and $lb_d$, of a bin on the detector. Finally, we define the position and direction vectors of the detector as $\vec {p}_a$ and $\vec {S}(\lambda,y)$ in order to verify a valid ray-detector intersection (within the acceptance cone angle of an active area of the detector). The full details of the system parameterization and further derivations are included in Supplement 1. With these definitions, we define the probability of a ray intersection at a specific detection location on the detector surface:

2.2 Mutual information enables wide-scope customization over multiple parameters

Mutual information allows us to consolidate the transmission and dispersion parameters and performance (glass types and geometries, dispersion linearity and more) of a prism array into a single quantitative variable. Provided a detection event, mutual information describes how well defined the wavelength is that caused said event.

In order to define the mutual information of a wavelength and it’s corresponding detection event, we start with the conditional probability shown in Eq. (1). The Monte Carlo process estimates the expectation values over the spatial variables:

Similarly, we can determine the conditional wavelength dependence through the estimation of the expectation value over the set of wavelengths. To facilitate multi-objective optimization a quasi-random determination of the initial wavelength is performed using the inverse cumulative distribution function (CDF) of the user-defined wavelength distribution. This estimation gives the probability of a detection event occurring at a specific detector bin $d_e$:

The mutual information can then be formulated from Eq. (3). As developed in information theory, the mutual information of two variables is a probabilistic quantification of the information known about one variable given some set of conditions for the other [5]. The general symmetric form of mutual information, $I(U;V)$, depends on the Shannon or information entropy, $H$, and is given as follows (for generic variables U and V):

In this formulism we can quantify the marginal entropy of a detection event as follows:

This conditional entropy term further quantifies the dispersion regularity of the spectrum. An irregular dispersion can cause detector bins to detect varied spectral signals or limited bands of wavelengths to spread across far too many detectors. Both of these problems cause the associated conditional entropy to increase, therefore reducing the overall mutual information value. From these definitions for the marginal and conditional entropies, the mutual information of a detection event and the ray that caused this detection is obtained:

This mutual information is the key quantity that we aim to maximize with our design optimization. In a more practical sense, maximizing the mutual information in our system is effectively the same as minimizing the ambiguity in a detection event. In summary mutual information is being used to minimize the ambiguity in which wavelength causes a given detection event. This process leads to linearization of the dispersion as well as maintaining high transmission.

2.3 Rapid multi-objective evolutionary optimization

A compound prism array typically consists of 2-5 trapezoidal or triangular prism elements, the desired number of which can be tuned by the user. Here, the final surface of the prism array is made horizontally cylindrical in order to adequately focus the dispersed fluorescence to the detector plane. A schematic drawing of this described typical prism can be seen in Fig. 1.

 

Fig. 1. An exemplary compound prism spectrometer system design resulting from the designer. A few of the physical parameters are noted ($n_{i-1,2,3}$, refractive indices of the individual prisms) with the user determined incoming beam parameters ($\lambda$) and detector geometry and spectral characteristics (D) in green.

Download Full Size | PDF

As depicted in this schematic, there are a large number of degrees of freedom to be considered in designing an optimized prism array (a minimum of 6 with an additional 3 for each added surface). This complexity is further increased with the addition of each successive prism element. Hence, it is impractical to directly and iteratively optimize over these variables when dealing with several hundred prisms per generation (250 prisms, 1000 generations). Instead the optimization objectives have been reduced to 3 quantities that determine the overall quality of the prism array: the mutual information, $I(\Lambda ;D)$, the total track length of the spectrometer, and the spectrometer deviation angle. As previously stated, mutual information characterizes the dispersive quality of the prism array. These 3 quantities form the axes of a Pareto analysis used in determining an array’s overall design quality. The deviation angle and spectrometer size are additionally bounded in order to filter out physically irrelevant designs.

Starting with the first generation of prisms, a Monte Carlo ray propagation is used to optimize individual designs. From the user provided beam parameters, ray wavelengths and initial positions are quasi-randomly selected. These rays are propagated through a prism array to the user determined detector, defined by the number, shape, and size of individual detector bins, total detector size, and relative spectral sensitivity for different wavelengths. This propagation is used to determine the dispersion characterization of an array for its associated detector. Designs are then updated and their surface geometries and materials optimized to maximize the array’s mutual information. This ray propagation followed by array optimization is repeated until the predetermined number of ray propagations are performed. Alternatively, if the standard error of the mean (SEM) for the Monte Carlo integration falls below the chosen error thresh hold ($5*10^{-3}$ for the simulations done here) then the simulation is terminated at that point. The final design is then checked for whether it forms a new Pareto front, a metric for comparing various multi-objective optimization solutions [7]. This check is performed by using the mutual information and total track length of each design as coordinates, which are then evaluated to determine whether those coordinates are at the leading edge of the current Pareto front. If this check succeeds, then the design is archived and the evolutionary optimization process continues with a new generation of prisms. A flowchart of this optimization process is provided in Supplement 1 for further clarity.

The library of genetic evolutionary algorithms available in Pymoo, a python based optimization package, are of particular use in the designer software [8]. We chose an evolutionary algorithm due to the relative speed that they offer as well as the diverse range of solutions that they can produce. The current branch of the designer utilizes the non-dominated sorting genetic algorithm, NSGA-II [9]. This algorithm was chosen for the default setting as it provides an exceptionally fast optimization without sacrificing the integrity of the designer results. Specifically, the time complexity of the NSGA-II algorithm is $O(MN^2)$, where M is the number of objectives and N is the population size. This, in contrast with the $O(MN^3)$ time complexity offered in similar evolutionary optimization algorithms (NSGA), makes it the much faster option for our purposes. However, a user can change the optimization algorithm in order to further tailor the designer to their needs as they see fit.

While the techniques described above for accurately optimizing prism designs are efficient and effective, the ability to simulate hundreds of designs over the course of thousands of generations is facilitated by the Rust programming language. The use of Rust improves the performance of the lower level calculations of ray propagation and the overall determination of the fitness of individual array designs. With these individual calculations performed rapidly, the Monte Carlo ray propagation is further accelerated by through parallelization via CUDA. These two major software design choices allow for the high speed optimization of a large number of compound prism arrays, while maintaining the ease of use provided by Python at the user level.

3. Results

3.1 Designed for linear 32 channel PMT array

The emergence of spectrally resolved microscopy, including hyperspectral, multiplexed, and superplexed imaging modalities for biomedical applications has driven the need to improve imaging spectrometers [10–17]. Many of these techniques become photon-starved as speed and tissue depth increase. We sought to maximize transmission efficiency and dispersion linearity to facilitate multiplexing of fluorescent probes across the VIS-NIR spectrum as an example of an application of our compound prism designer. In this example we optimize a compound prism design for transmitting and linearly dispersing 500-820 nm onto a 32 channel PMT common in confocal microscopy.

Here the goal is to maximize signal efficiency with sufficient spectral resolution to resolve the spectral bandwidth (320 nm) in approximately 10 nm steps. In doing so, the signal detected by individual PMT channels now corresponds to specific wavelength ranges. To accomplish this, we use mutual information to optimize the linearity of the dispersion, the average transmission of the array, and overall spectrometer size.

The results of a simulation of the 32 channel PMT are shown in Fig. 2, filtered down to the 25% highest performance designs. The quantitative metric of dispersion linearity used in Fig. 2(a) is the inverse of the following chi-squared analysis of the detection probabilities at each detector bin.

 

Fig. 2. (a) A comparison of the dispersive linearity, transmission efficiency, and physical length for all of the designs created in a single run of the designer grouped into 1, 2, 3, and 4 array elements. (b) Linearity of four exemplary designs representative of optimized linearity in 1, 2, 3, and 4 element prism arrays. (c) 3 prism designs corresponding to the well optimized extremities of the clusters in (a) composed of 1, 2, and 3 elements.

Download Full Size | PDF

The ideal dispersion ($p(D=d_e)_E$) that each experimental design ($p(D=d_e)_O,i$) is compared to is that in which each of the detector bins exhibit an equal likelihood of a detection event. The inverse of this chi-squared analysis is used as the linearity metric in order to have linearity increase along the positive x-axis. Mutual information is used to filter out poorly optimized designs as any design with a near equal-likelihood detection profile that isn’t indeed linear in wavelength space scores quite poorly in this metric. In the idealized case where a design yields 5 bits mutual information each 1.6 nm wide wavelength range is detected by a single detector channel. A reduction in the mutual information corresponds to these wavelength ranges being dispersed across additional detector channels. Therefore, 3 bits of mutual information is chosen as the cutoff in order to filter out non-linearly dispersed designs.

The plot in Fig. 2(a) depicts a collection of designs as a function of their dispersion linearity, average transmission, and overall spectrometer size. In order to show the effect of increasing number of prism elements this figure groups the designs by color based on the number of elements that make up the array. The most significant improvement in linearity can be seen with the addition of a second prism element. As further elements are added, there is some additional improvement in the dispersion linearity. While one could reasonably expect there to be some loss in transmission with added prism elements, the difference in transmission for those designs located in the "best" region of the plot, is relatively small. This is a result of the more gradual surface transitions (in terms of index difference and incident angle) required to achieve a similar dispersion linearity with increased surface number.

It is important to note that the use of average transmission in Fig. 2(a) refers to the average transmission of all wavelengths from the initial surface of the compound prism to the point of detection at the PMT surface. This definition results in the average transmission being lower than the transmission exhibited solely by the prism itself. This is due to several wavelength bands being lost in the dead regions between each of the detector bins. Fortunately, because of the even spacing of these dead spaces and the narrow wavelength bands lost to them, approximately 3 nm, this does not have a significant impact on the overall transmission of the system. Furthermore, the user can examine exactly which wavelengths are lost to these dead spaces via the direct exporting of specific designs to either ASAP or Zemax OpticStudio, a functionality provided by this designer software.

In calculating the prism array transmission, we assumed a transmission of 99 percent that would be achievable using custom AR coatings for the particular bandwidth modeled here. In addition the simulation includes a penalty for any light rays with an angle of incidence > 60$^{\circ }$, based on specifications for typical AR coatings (sources available online: Thorlabs AB coating and Edmund Optics VIS-NIR coating).

Improvements in dispersion quality with increased prism element number are quite significant. To illustrate this point further, Fig. 2(b) shows a detection-probability violin plot for four designs of varying prism element numbers. These four designs were chosen from the most linear designs corresponding to each number of prism elements. The dashed line has been provided as a visual guide denoting an ideal linear case. While none are perfectly linear, it is clear that the 4 element prism is significantly more linear than the simpler 3, 2, and 1 element prisms, archetypal designs of which can be seen in Fig. 2(c). Simulations for higher numbers of element prisms did not significantly improve linearity. An average improvement of approximately 215% in prism linearity is seen between 1 and 4 element prisms, whereas only an additional 15% improvement is seen from 4 to 5 elements

A direct comparison of the dispersive qualities of the optimal simulated 4 element prism and two commercial gratings, one reflective and one transmission, can be seen in Fig. 3. The simulation of the dispersion of the displayed 4 element prism and the GTI25-03 and GR25-0608 gratings from Thorlabs, was performed in Zemax OpticStudio. Incident beams consisting of the same 500-820 nm wavelength range were set to the blaze angle of each grating. The detector was positioned so as to aim the extremes of the first diffraction order at the outer edges of the detector array. The spot diagram at the detector surface of each of these elements has been overlaid with an outline of the 32 channel PMT array and is shown in Fig. 3(b). The spacing of each successive wavelength is notably more consistent in the prism dispersion, with some slight bunching at the ends of the detector.

 

Fig. 3. (a) The design duration as it depends on the chosen number of prism elements is plotted. The left y-axis shows the required time for each individual prism array while the right shows the total design duration in minutes. (b) A comparison of the dispersion profiles for a selected 4 element prism (P) and comparable reflective (R) and transmission (T) gratings. The dispersion profiles consisting of a uniform distribution of wavelengths are projected onto the theoretical 32 channel PMT detector surface. (c) The selected 4 element compound prism array is shown. From right to left, each prism surface serves to tune the linear chromatic dispersion. The final curved surface focuses the dispersed signal to the detector. (d) A theoretical comparison of the transmission efficiency for the selected 4 element prism and comparable gratings is seen across the chosen wavelength range.

Download Full Size | PDF

3.2 Rapid prism simulation

The compound prism designer software introduces the capability to rapidly prototype compound prisms for the first time. This design speed enables the user to explore a multiple parameter space to optimize a prism design specific to their application. choose a selection of variable prism designs at a rate that far surpasses currently available design methods. This opens the door to progressive analysis where one can vary optimization parameters and priorities that best suit the user needs.

The speed of the design process is determined by a multitude of parameters depending on the specific application. Figure 3(a) shows the runtime of the designer software. These test runs were performed with 1000 generations, each consisting of 250 individual prisms (250,000 total designs). These tests were performed using a Ryzen 9 5900X CPU in conjunction with a GeForce RTX 3090 GPU. The CPU performs final calculations for designs deemed plausible by initial GPU based calculations. This final calculation is done in order to mitigate any inaccuracies that can occur in parallel floating point calculations such as those performed in the initial GPU optimization [18,19]. Because so much of the design process is performed by the GPU, any estimation of relative hardware performance should likely be based on this component.

One variable that obviously impacts the designer run time is the number of elements in the prism array. This increase in prism design time follows an approximately linear behavior with respect to increasing number of prism elements as is depicted in Fig. 3(a). From testing, it was seen that each additional surface adds roughly 0.75 ms to the individual array design time. This in conjunction with the diminishing returns seen in design quality with the addition of array elements beyond 4, and considering fabrication costs, reveals that those prisms with 3 or 4 elements strike the best balance between overall design time and cost with performance.

The high degree of performance exhibited by the compound prism designer is a direct result of specific choices made regarding both the optimization algorithm and method. The use of the evolutionary genetic algorithm, NSGA-II, provides efficient multi-objective optimization. In making the main optimization objective the mutual information of the prism design, the optical system is given a simplified parameterization that can be determined and directly compared between designs. These two aspects of the designer work in conjunction to provide an efficient methodology for performing the optimization. Further improvements to the overall performance of the designer are results of programming decisions that best take advantage of the optimization method. The use of Rust for simple but crucial arithmetic calculations alongside the large scale parallelization of the Monte Carlo ray tracing procedure via CUDA, serves to further improve the designer performance. These cumulative decisions provide the designer software with the speed and reliability as demonstrated here.

Development of the designer software is still ongoing, with new features and options being added regularly. Further work is being done in order to provide designer capabilities to suit a wider variety of optical systems. Additional specified parameters and optical system properties are in the process of being implemented in order to further broaden the use cases of the designer software.

For those working in low-photon regimes, such as hyper-spectral fluorescence imaging, the efficient usage and transmission of available photons is of the utmost importance. In order to achieve this, we propose the use of this custom designed compound prism array for the effective dispersion of fluorescence signals. Rather than settling for an off-the-shelf option that is sub-optimal, this designer software provides users with the ability to curate a suite of designs specifically tailored to best suit their needs.

4. Conclusion

We have developed a high-speed compound prism designer software that provides users the ability to simulate a pool of arrays specific to their use case. This designer expands on pre-existing principles of compound prism design, a methodology for achieving chromatic dispersion profiles unobtainable by a single prism. This software takes advantage of several improvements, including the use of Rust as the primary software language, mutual information as the primary optimization objective, and CUDA parallelization. These design choices provide the basis for the millisecond rate prism simulations exhibited here (Fig. 3(a)). Rapid prism design facilitates the high-volume of array simulations necessary for the effective use of evolutionary multi-objective optimization. The designer capabilities were demonstrated by simulating a variety of compound prisms for a 500-820 nm bandwidth dispersed along a linear 32 channel PMT. The dispersion quality of an optimized prism array was compared to commercial diffraction gratings with comparable design bandwidths (Fig. 3(b)). With a higher average transmission across the chosen bandwidth and a more uniformly linear dispersion profile, we believe that the optimized prism array would serve to directly improve a variety of hyperspectral imaging systems. Furthermore, this test case exemplified the potential of the designer to simulate a multitude of design solutions that provide the user with further agency in the final determination of a preferred compound prism.