1. ROMAN CORONAGRAPH INSTRUMENT

The coronagraph instrument [1] is a technology demonstration that will fly aboard the Nancy Grace Roman Space Telescope (RST). It will prove the capability to design, build, and operate an active optics-enabled coronagraph capable of direct observation of exo-Jupiter-like planets. These planets have very little angular separation to their parent stars, making them prohibitively difficult to image directly with conventional cameras because of glare. A coronagraph is specially engineered to have a region called the dark hole, in which only trace amounts of light from the parent star remain. It is here where exoplanets can be observed directly. High-performance designs, such as those used by the Roman Coronagraph instrument, require the optical wavefront to be held constant at the subnanometer level. While the RST provides an ultrastable observatory, further reduction in disturbance is required to meet the Roman Coronagraph’s performance aspirations.

For this purpose, the Roman Coronagraph instrument team has designed and engineered a low-order wavefront sensing and control (LOWFS) [2] system that rejects dynamic changes in the system’s wavefront error within the span of Noll’s Zernike Z2 to Z11 [3]. Concurrent to LOWFS, higher-order wavefront sensing and control (HOWFS) is used to produce the dark hole. A schematic diagram of the coronagraph instrument’s dual control architecture is shown in Fig. 1. The instrument has a fast steering mirror used to actuate tip-tilt, a focus control mirror, and a pair of deformable mirrors (DMs) that control both the phase and amplitude of the wavefront for higher-order correction.

 

Fig. 1. Schematic representation of the coronagraph instrument and its control systems. The Hz indications on each control path represent the temporal sample rate, not the control bandwidth. FSM, fast steering mirror; FCM, focus control mirror; ACS, attitude control system; TCA, tertiary collimator assembly; PMA, precision alignment mechanism; S/C, spacecraft; and PZT, piezoelectric transducer. Graphic courtesy of Mark Colavita.

Download Full Size | PDF

 

Fig. 2. Example input-output pair for astigmatism, comparing conventional and co-designed masks. (${{l}} - {{r}}$): input phase, phase of focal plane mask, and output intensity. Top: conventional PCWFS. Bottom: LOWFS for the Roman Coronagraph instrument. The lateral scale of the two center panes differ by approximately ${{10\times}}$; the conventional mask’s “pimple” is approximately $1\lambda /D$ in diameter, while the detailed area of the lower mask has a diameter of about $5.5\;\lambda /D$. Most of this area is design content for the coronagraph, with the central maroon disk being the feature that supports LOWFS.

Download Full Size | PDF

LOWFS uses a small 50x50 region of interest on the low-order wavefront sensing camera (LOCAM), which is read out at 1 kHz. A field-progammable gate array (FPGA) processes each of these images to produce an estimate of Z2 to Z11, as well as a small number of other dynamics that will be described later in this paper. The estimates of tip-tilt are used for the line-of-sight control loop with the fast steering mirror, which is also implemented on an FPGA. The remaining coefficients are averaged to 0.1 Hz and used by the instrument flight software (iFSW) to implement the Z4 and Z5 to Z11 control loops that operate the focus control mirror and first deformable mirror, respectively. The line-of-sight control bandwidth is 20 Hz, while all other control loops have a bandwidth of 1.6 mHz. DM1 is shared between LOWFS and HOWFS; a DM combiner merges inputs from the two wavefront controllers.

LOWFS uses a phase contrast wavefront sensor (PCWFS), which is also called a Zernike wavefront sensor (ZWFS). The principle of phase contrast is a linear conversion of an otherwise unobservable optical phase into intensity by placing a small phase-advancing or phase-delaying mask at the focus of a beam. This technique was first discovered by Zernike [4] as an enhancement to the knife-edge test to evaluate the figure error of mirrors. Less than a year later, Burch and Stratton published an experimental work with phase contrast masks, including using them to measure the wavefront error of a 36-inch telescope at the Cambridge Solar Physics Observatory [5]. This rarely cited paper also describes in great detail a fabrication process to create phase contrast masks and offers a qualitative interpretation of the interferograms produced using them.

To reduce noncommon path errors, the phase contrast masks for the coronagraph instrument are integrated as design features of the coronagraphic focal plane masks, which are complex dielectric patterns simultaneously optimized for both the coronagraph and LOWFS. In a typical phase contrast wavefront sensor, the change in intensity on the camera closely resembles the morphology of the change in phase. This does not happen when additional features are introduced to the mask to improve the performance of the coronagraph. Figure 2 shows a simple example of input astigmatism and the resulting change in the image on the camera at the output for a conventional phase contrast wavefront sensor as well as the coronagraph instrument’s co-designed masks.

When using these specialized masks, the morphological mismatch occurs for any decomposition of the input phase, not only for shapes that are compactly represented as polynomials. Of additional interest is the scatter of light outside the support of the pupil when using the hybrid masks that simultaneously support the coronagraph and LOWFS. Regardless of this fact, the system still behaves in a linear manner operated differentially over a small dynamic range of about ${\pm}\;\lambda /10$ RMS of wavefront error.

2. MODAL PHASE CONTRAST WAVEFRONT SENSING

The majority of groups operating phase contrast wavefront sensors use zonal wavefront estimation schemes, in which a separate phase estimate is produced for each pixel in the input image [6–9]. This process is straightforward, given a classical phase contrast mask. Fits to this pixel-wise data may or may not be performed to produce modal wavefront estimates, in terms of Zernike or other polynomials. Due to the aforementioned co-designed nature of Roman’s focal plane masks, a zonal estimation algorithm would be highly nontrivial for LOWFS. We use an empirical approach that is based on direct in-situ measurement of the change in the wavefront caused by the motion of each active mirror and an assumption that the system remains locally linear where the initial wavefront error is calibrated. Without a loss of generality, the optics of the wavefront sensor maps the input plane phases to the output plane intensities via an optical propagation operator $\mathfrak{L}$, which describes the path of light through the system as

Implementation of Eq. (6) in a real-time control system requires one elementwise difference of two vectors $\Delta I = I - {I_0}$ and the matrix-vector product ${\bf{P}}(\Delta I)$. It is more efficient to pre-compute the matrix-vector product ${\bf{P}}{I_0}$ and subtract that from the vector that results from ${\bf{P}}I$ in real time:

Equation (8) requires only tens of microseconds, even on a laptop processor, and is well-suited to use within a real-time control system. In treating the sensing problem this way, we have made no more assumptions than there is a linear relationship between $\Delta \phi$ and $\Delta I$ within a small range of $\Delta \phi$; i.e., that the system is locally linear.

One advantage of this modal approach is that physical changes of the system that at least approximately linearly cause changes in $I$ can be included as new columns of ${\bf{A}}$. Doing so improves the robustness of the sensor to these changes. To great effect, LOWFS uses four of its spare modes for precisely this issue, carrying modes for translation of the image on the detector; as a result, the “pupil shear” as well as the detector bias drift and overall flux level change.

In this section, we have described the flight wavefront sensing algorithm for LOWFS. The remainder of this paper will focus on integrated modeling of LOWFS, and specific innovations that have been made to model this high-speed system rapidly, without compromising model accuracy.

3. INTEGRATED MODEL OF LOWFS

The core task in modeling LOWFS is a high-fidelity implementation of the $\mathfrak{L}$ operator of Eq. (1). It has become common practice to use custom diffraction propagation software to model optical systems. Since the 1990s, tools such as Tiny Tim [11] have been developed and used to model space telescopes. John Krist, who developed Tiny Tim also developed PROPER [12] to model coronagraphs. Similarly, Perrin et al. developed POPPY [13], which is widely used for similar purposes. A recent summary is available in [14]. These programs have many features, the most prominent of which is the ability to simulate the wave propagation of light through an optical system. Both use Gaussian pilot beams to automatically select the propagation routine to move between any two planes and have the user enter a prescription for the optical system. The CGISim model released to the public by the coronagraph instrument team is written with PROPER. CGISim is a so-called full model and has a long runtime, taking several seconds to for each wavelength propagated. LOWFS operates at 1 kHz and shows the need for faster modeling capability or order to complete analysis in a timely manner.

Table 1. Comparison of the Time Required to Run the PROPER Compact Model and the LOWFSSim Model Based on Prysm for a Single Wavelength and Polarization^a

View Table | View all tables in this article

 

Fig. 3. Left: CPU and GPU floating point throughput. Right: Same, normalized by the performance figures in 2011. The performance figures used in these plots are gathered from [19–24].

Download Full Size | PDF

To accomplish this, we have created a multiplane diffraction model of the system using prysm [15], the lead author's library for physical optics. While prysm was originally developed for image chain modeling, it was significantly performance-optimized from 2017–2018 to do large-scale statistical studies of a new phase retrieval method discussed in [16]. A number of these optimizations are discussed further in Section 4. Modeling the HLC configuration of the coronagraph instrument, our model is more than ${{750\times}}$ faster than the equivalent PROPER model, as shown in Table 1. For the SPC [17] configuration, the prysm model is more than ${{1600\times}}$ faster.

Our LOWFS simulator (LOWFSSim) model begins at the fast steering mirror. In this plane or its equivalent pupil conjugate, we wish to sense the dynamics of the low-order aberrations with LOWFS. The obscuration of the Roman pupil is applied here and the beam is then propagated through the DMs to the shaped pupil mask plane. After this, it is focused on the focal plane mask where the complex reflectivity of the occulter, including all thin film effects, is applied to the beam. The light is then reimaged to a pupil, and optionally defocused via free-space propagation if the camera’s along-optical-axis misalignment term is nonzero. If that distance is zero, the final free-space propagation is skipped. This is similar to the compact model of the coronagraphic side of the Roman Coronagraph constructed using PROPER, if that model were to use the complex reflection of the focal plane mask and terminated at the Lyot stop. We have modified the Phase ${\text{C}}$ Compact PROPER model to do so to compare computationally equivalent models. The runtime of this modified model and various configurations of the prysm-based model are shown in Table 1.

Phase C Compact models constructed using POPPY and HCIpy were not available for comparison. We note that Milani et al. [18] compared PROPER and POPPY in the context of full models. Examining their Table 3, comparing the fastest POPPY results to the fastest PROPER results, POPPY is up to ${{3}.{85\times}}$ faster than PROPER. However, in the worst case, POPPY is ${{2}.{35\times}}$ slower than PROPER. On average, POPPY is 12% faster than PROPER. The reader should note that full models and compact models are not directly comparable in terms of runtime, but they are composed of the same elementary operations: free-space propagation, multiplication by complex screens, and so on. A hypothetical full model constructed with prysm would therefore be expected to be ${{1000\times}}$ faster than POPPY, to an order of magnitude.

We have found that using 1024 or 512 sample wide arrays while holding the sampling pitch constant negligibly affects the results modeling LOWFS. The optimizations of the individual routines within prysm accelerate the computation by about ${{30\times}}$, as shown in Table 1, Row 2. An additional ${{15\times}}$ results from the use of a GPU, a change enabled by three lines of configuration at the top of a given user script. GPU computing brings an additional order of magnitude improvement today, but the rate of growth in GPU performance exceeds the rate of growth of CPU performance, as shown in Fig. 3. Future gains made by adopting GPU computing will be larger than those shown today. The cores of CPUs have not gotten significantly faster, about 59% over the last 10 years. In the same period, GPU performance has advanced by more than ${{17\times}}$, which is 50% more growth than has been exhibited by CPUs, even considering multicore performance growth. The proliferation of robust, easy-to-use GPU computing libraries has also made it relatively trivial to use all available GPU resources, provided array sizes are large enough to make this meaningful. The same is not true on CPUs, where many operations such as elementwise addition, subtraction, multiplication, and division of arrays use only a single core, limiting scaling on multicore systems. Task distribution libraries do not spread these operations across multiple cores.

4. HIGH-PERFORMANCE SOFTWARE ARCHITECTURE

In this section, we describe and show through profiling how the architecture of prysm lends itself to high-speed modeling. Readers interested in the source code may consult the GitHub repository for prysm [15] or LOWFSSim [25]. The only significant distinction between a model constructed with prysm instead of PROPER or POPPY is that the latter two libraries have automatic propagator selection as a core function, based on the propagation of Gaussian pilot beams. With prysm, a user is responsible for choosing an appropriate method for each propagation. This increase in required user domain knowledge then allows the user eliminate unnecessary computations that would be effectively canceled later in the model.

A. Full Support for GPU Computing

Over the years, several papers have discussed the use of GPUs in various areas of optics, such as in the computation of polynomials [26] or to perform Fourier computations in diffraction models [27]. The latter reference reports only modest improvements to the total diffraction model runtime with GPUs used to perform FFTs and FFTshifts. While polynomial calculations and FFTs do form a significant fraction of the runtime in a diffraction model, additional overhead is created if the data must be exchanged between the CPU and GPU to offload these computations. To support fully GPU-accelerated computing free of host-device data transfer overheads, prysm contains a small, 10 line of code interposer that holds a reference to either NumPy [28] or any library with the same API. The interposer is imported as though it were NumPy and forwards any lookup to the underlying library. We favor CuPy [29] over alternatives such as PyTorch [30] or JAX [31] for its exceptional compatibility with the NumPy API; fewer than 1% of the lines of code in both prysm and LOWFSSim have needed modification to support it.

The net effect of this interposer is that when configured to use the GPU, all arrays in prysm originate on the GPU, all operations occur on the GPU, and all variables in the user’s code live on the GPU. Thus, there are no transfers of large arrays between the CPU and GPU. This is in contrast to the work in [27], where only a subset of operations occur on the GPU, and the programming model is that all user variables exist on the CPU. Thus, to perform an FFT, elementwise multiply, and iFFT, the work of [27] performs two transfers from the CPU to the GPU, prior to each FFT or iFFT, and two transfers from the GPU to the CPU after each FFT or iFFT. The elementwise product occurs on the CPU. The GPU’s memory bandwidth is 10 to 50 times higher than the PCI-e connection between the CPU and GPU, making these wasteful transfers an extreme bottleneck. The key to achieve substantial acceleration is to have the significant majority of all computations on the GPU and minimize the number of transfers to or from it.

Non-datacenter GPUs typically run much slower with double precision floating point numbers compared to 32-bit computation, with slowdowns of 32 and ${{64\times}}$ common when 64 bit floating point values are used [32]. To enhance GPU support, prysm contains a global configuration object that controls the precision used for computations. By setting it to 32-bit, models are able to run at reduced numerical precision in favor of a significant increase in speed. Because prysm uses a system of preferred units (i.e., those that keep most quantities relatively near unity), there typically is a negligible impact to the accuracy of the outputs because no quantity is particularly near the resolution or dynamic range restrictions imposed by the number format. In the Roman LOWFS model, the peak difference between the synthesized image at the camera in 32-bit or 64-bit is less than 0.1%. Use of 32-bit numbers is only significantly beneficial on a GPU; on a CPU, there is less than a ${{2\times}}$ speedup. Double precision computation should be used on a CPU, unless a memory limitation is present. In 2008, Shimobaba et al. also reported a nearly ${{50\times}}$ acceleration [33], although they wrote their program directly in CUDA, NVIDIA’s proprietary language for GPU computing.

B. Accelerated Algorithms

In Table 1, we show a nearly ${{30\times}}$ increase in speed between the PROPER compact model and the equivalent prysm model on a CPU. Inspection of the source code reveals that the sequence of operations in the two models is nearly identical at a high level. We have used VizTracer [34] to profile and analyze the runtime of each model. The top three contributors to the PROPER model and their equivalent operations in the prysm model are shown in Table 2. In this table, the array sizes used for both models use ${{1024 \times 1024}}$ arrays to make an apples-to-apples comparison.

Table 2. Comparison of the Total Runtime Over the Entire Model of a Top-Level Operation Using Both PROPER and prysm in Equivalent Models with the Same Array Sizes on the Same Machine

View Table | View all tables in this article

Prysm is able to, and in this case does, use Intel oneAPI Math Kernel Library (MKL) FFTs, as PROPER does within propagation routines. PROPER still uses Scipy FFTs within the prop_dm routine, although the array sizes there are smaller. Prysm permits MKL to autonomously select the number of threads used for FFTs, while PROPER restricts that value. Additionally, prysm uses Intel’s own mkl_fft package, while PROPER uses a custom wrapper. We also analyzed one level deeper than the top-level routines. The top three functions at this level are enumerated, alongside their fraction of total runtime, in Table 3. Matrix DFTs computed with prysm have the number of threads used determined by appropriate environment variables, while PROPER hard-codes the use of a single thread. Table 3 uses 48 threads for prysm’s matrix DFTs.

Table 3. Calculation Time for the Top Three Lower Level Contributors to Each Model on a Per-Call Basis, as Well as the Fraction of Total Model Runtime Represented

View Table | View all tables in this article

No quadratic phase screens are created during repeat runs of the prysm model, and they are therefore excluded in this case because we run thousands of model cases per session. The majority of the prysm model’s runtime is spent on combination of dynamic amplitudes and phases into complex wavefunctions and padding arrays, operations that are not multithreaded. The PROPER model generates the DM OPD map from the its actuator commands while the prysm model loads the PROPER-generated map from disk as part of its initialization. The current prerelease of prysm includes DM modeling routines that are equivalent to those in PROPER and run ${{2\! -\! 3\times}}$ faster.

C. Detector Modeling

The Roman Coronagraph has also released the emccd_detect library for electron multiplying CCD (EMCCD) image simulation [35]. This library is considerably faster than real-time when modeling the exoplanetary camera, requiring about 100 milliseconds to simulate an image. For our use case simulating the LOWFS system at 1 kHz, it is still 100x slower than real time.

A separate EMCCD model was developed for LOWFSSim based on Hirsch et al. [36] by adapting prysm’s general detector model routines. This method allows the EM gain register to be simulated with a single draw of random samples from a gamma distribution, which is 484x faster than the implementation of the methods by Basden et al. in [37] in emccd_detect. This code can draw ${{N}}$ frames of random samples for the same expectation value simultaneously, which expands the unit of work from ${{50 \times 50}}$ to $50 \times 50 \times 10,\!000$, a meaningful array size for the GPU to increase the speed.

The combination of the fundamentally simpler and faster EM register simulation and multiframe parallelization yields a $20,\!690\times$ increase in speed when simulating 10,000 frames for the same expectation value, as is done for the LOWFS dynamical modeling of Z4-Z11. It takes our model only 58 milliseconds to simulate 10,000 frames. This speed has proven to be instrumental in accurately modeling the calibration sequence for LOWFS, which requires several hundred thousand frames for any single pass through the calibration sequence. It would not be possible to survey a significant tract of the calibration parameter space in finite time without resorting to more crude approximations without these accelerations. The simultaneous draw of 10 seconds’ worth of frames is only used for lower bandwidth Z4 to Z11 modeling; when Z2 and Z3 are required in a dynamical model, each frame is drawn from a stochastic output of the optical model at 1 kHz.

5. MOTIVATING EXAMPLES

In this section, we will showcase a few analysis problems that emphasize the need for exascale modeling throughput in the form of several examples borrowed from our work on LOWFS for Roman. The content shown is far from exhaustive; in the last year, the LOWFSSim model has been executed more than 2 billion times, and more than 10 petabytes of ephemeral data has been computed in doing an engineering analysis of LOWFS.

A. Extended Dynamic Range LOWFS Sensing Errors

Historical models of LOWFS have analyzed the system using a cross of points about the origin to evaluate the wavefront sensor’s performance in Z2 and Z3. Tip and Tilt span a uniquely large dynamic range that is hundreds of nanometers as opposed to less than 1 nm for the focus through spherical modes Z4 to Z11. Such comparatively large inputs are far beyond the linear regime of the wavefront sensor and require careful study to ensure that the control system can reliably acquire and maintain lock. The response of the sensor is shown in Fig. 4 in the Z2 and Z3 channels over a range of ${{\pm 150}}$ milliarcseconds on the sky, about ${\pm}3/4\lambda$ RMS. Producing Fig. 4 entails running LOWFSSim for ${{301 \times 301}}$ points, each with nine wavelengths to produce realistic polychromatic imagery and wavefront estimates. This is a total of 815,409 executions of the model. Running sequentially on GPU LOWFSSim produces these results in 20 min. These graphics are used to perform a cursory analysis of the likelihood of line-of-sight (LOS) loop closure success.

 

Fig. 4. Zernike Z2, Z3 estimates in milliarcseconds on the sky (mas ots) as a function in the input coordinate for LOWFS in the HLC-NFOV configuration. While the wavefront sensor is linear and responds in an independent way to the orthogonal inputs near the origin, it does not when far from the origin, including large regions where the sign of the estimate is wrong.

Download Full Size | PDF

Figure 4 concerns the ideal case with no misalignments. A detailed study of the topic requires simulation of cases where the center is not actually Z2 = Z3 = 0, as when LOWFS is calibrated about a nonzero operating point, cases where the image on LOCAM has sheared with respect to when the calibration was done, and study for each CGI configuration and each candidate HLC occulter before acceptance as a flight mask. Exploring each of these dimensions of the problem required several hundred different versions of Fig. 4 to be generated and the underlying data analyzed totaled more than 100,000,000 model executions.

B. LOS Capture Monte Carlo Studies

The previous section discusses qualitative studies of the likelihood of LOS loop closure success based purely on the sensor’s response. A more rigorous study requires integration of the control algorithms and models. To this end, we created a server program that provides an interface to LOWFSSim over the network to the pointing acquisition and control element (PACE) team’s control analysis simulation testbed (CAST), which uses MATLAB. The PACE team then integrated the LOWFSSim model as the sensor in the CAST model for tip-tilt control modeling. CAST is used by the PACE team to evaluate the total pointing error performance. It contains models of the spacecraft flexible dynamics, PACE controllers (piezoelectric transducer (PZT) local loop controller and fast steering mirror (FSM) feedback controller), the spacecraft attitude control system (ACS) controller, and the reaction wheel disturbance models.

The spacecraft dynamics model is an integrated finite element model (FEM) of CGI and the spacecraft bus, including articulating reaction wheels and fast steering mirror, as well as reaction wheel isolators and telescope isolator models. Outputs include spacecraft attitude and angular rate, reaction wheel motion, and LOS angles using the linear optical model provided by the Goddard Space Flight Center. The full FEM contains more than 10,000 flexible modes including a number of high-frequency residual vector modes. A model reduction is performed that results in 600 modes below 1850 Hz. Control forces are applied to the FSM PZTs to actuate the FSM, and reaction wheel joint torques are applied to actuate the reaction wheels. In addition to the desired ACS control torques, the reaction wheel joint torques also include disturbance models of reaction wheel bearing friction implemented as a LuGre [38] model with stiction, as well as motor cogging and torque ripple. Reaction wheel exported forces and torques for the remaining five degrees of freedom (three forces and two nonjoint torques) are modeled as eccentricity-based harmonic disturbances with 73 tones that scale with the square of the reaction wheel speeds. A model uncertainty factor is applied individually to each tone for each wheel according to the tonal frequency.

CAST was used to simulate LOWFS acquisition with a variety of LOWFS configurations (HLC, WFOV, and SPEC, calibrated both at Z2 = Z3 = 0 and a number of off-center positions) with a randomized initial LOS error up to 80 milliarcseconds (radial). In total, more than 7000 acquisition simulations were performed. When the LOWFS sensor is calibrated at true zero, 100% of simulations converged successfully within 10 s of starting the FSM feedback. As the distance from the origin where calibration is performed increases, the percentage of random cases that converge decreases. In some cases, the FSM remained within the region of the local minimum for some amount of time (up to 9 s in some cases) before being knocked out by noise/disturbances and ultimately converging, as shown in Fig. 5. When LOWFS is calibrated inside a circle of radius 140% of the final requirement, 93% of simulations converged within 10 s of starting FSM feedback. The nulls that can trap the system appear white in Fig. 4.

 

Fig. 5. Zernike Z2 (blue) and Z3 (red) position as a function of time for CAST Monte Carlo trials when LOWFS is calibrated without the focal plane mask being exactly centered, to a degree that satisfies requirements. Each line is a separate trial, of which there are 525 for this particular choice of Z2 and Z3 misalignments. The red and blue lines do not converge to precisely zero because the absolute Z2, Z3 position is shown and a particular offset is assumed as the position where the differential LOWFS sensor is calibrated.

Download Full Size | PDF

C. LOS Capture with Extremely Low Flux

The Roman Coronagraph’s project science team requested an answer to the question, “Does LOWFS capture work on a Vmag 12 star?” Such a dim star produces only 0.035 photons per pixel per frame for LOWFS, which has a fixed 1 kHz sample rate. While the EMCCD camera is able to use EM gain to raise the corresponding output signal level, the SNR is still exceptionally low. While we have integrated LOWFSSim into CAST, the CAST dynamics simulation is exceptionally detailed and runs substantially slower than LOWFSSim. The 7,000 acquisition simulations done from within CAST took multiple months of computation to complete, and an answer was required quickly. We created an implementation of a controller with the same bandwidth and gain and phase margin as the flight control algorithm for use directly with LOWFSSim, as well as simplified ACS and RWA models in a precomputed disturbance time series. A total of 559 starting points were chosen using the sampling technique described in [39]. Each starting point was used as an initial condition for the system’s state, and the discrete-time model run for 10 real seconds, or 10,000 timesteps per case. Seven pairs of visual magnitude and LOCAM EM gain were simulated, for a total of more than 2.5 billion model evaluations over the course of about one month of computer time. Graphical results were generated, showing the starting points as well as the trajectory of each trial. A sample of the starting points and graphics for a ${{V}} = {{10}}$ star at an EM gain of 150 are shown in Fig. 6. Due to the strongly nonlinear response of the sensor over such a large dynamic range, the system follows nontrivial paths and finds its way into three “rivers” that lead to the origin. Despite this, 100% of trials do converge.

 

Fig. 6. Top: total of 559 starting points spanning 30 to 80 milliarcseconds on the sky radial. Bottom: trajectory of each run of the model.

Download Full Size | PDF

As the star’s brightness is lowered, the right-hand graph of Fig. 6 becomes increasingly non-smooth due to the increased noise of the wavefront sensor. At magnitude 12, additional EM gain is required, but even at a gain of 4,000 there is still a significant percentage of cases that fail. Based on this analysis, we determined that the control bandwidth and sample rate would have to be reduced for reliable line-of-sight capture at magnitude 12. However, even at a modest EM gain of 150, capture is reliable and as dim as magnitude 11.

D. Observing Scenario 11 LOWFS Dynamics

Observing Scenario datasets are a flagship modeling product generated by the Roman Coronagraph team and provided to the community to scientific proposals and other work. Previously, Erkin Sidick’s LOWFS model was used for OS9 [40] at a temporal resolution that produced 2,127 timesteps of modeling work. For OS11 shown in Fig. 7, there are 15,750 timesteps at 10 second temporal resolution; for each timestep, the LOWFSSim model is run with nine wavelengths and unpolarized light. This combinations of inputs results in 141,750 runs of the diffraction model, and the synthesis of 157,500,000 unique frames. The complete runtime for all diffraction, control, and detector effects using LOWFSSim is about 21 min, dominated by the time taken to synthesize all of the EMCCD frames. Running the model with the EMCCD component disabled resulted in a runtime of about 2 min and 50 s.

 

Fig. 7. Red: dynamic disturbances to the coronagraph produced by STOP modeling. Blue: results after LOWFS correction produced using LOWFSSim. LOWFS effectively eliminates large changes in Z4, Z6, Z7, and Z8 but injects noise into the system for the quieter Z5, Z9, Z10, and Z11 inputs. The Roman Coronagraph may enable and disable any combination of control loops and will open the loops on modes that LOWFS is not improving if OS11 predictions hold on-orbit.

Download Full Size | PDF

The STOP predictions for OS11 are extremely benign, with no Zernike term varying by more than about 50 picometers over the duration of an observation (red line, Fig. 7). The LOWFS sensor operates in an extremely low flux condition; observing the target star has on the order of just 10 photons per pixel. This leads to high noise at the picometer level, and in these conditions LOWFS is noise limited. The impact of having on average a lower bias but higher short-term variance is discussed in [41].

In running OS11, we discovered a mistake in our operational baseline to capture the reference frames that set the zero point of the sensor. We integrated for the same length of time on both $\zeta$ Pup and 47 UMa, which have visual magnitudes of 2.25 and 5. This resulted in a larger contribution to the standard error of the mean from 47 UMa, which is dimmer than $\zeta$ Pup. The standard error of the mean (SE) is

As the target star is dimmer, there is more shot noise and ${{\text{SE}}_{{\text{target}}}} \gt {{\text{SE}}_{{\text{ref}}}}$ if the same value of $N$ is used for each. For example, if $\sigma = 2\,\,\text{nm}$ and $N = 10000$, SE = 20 pm. We corrected this error by increasing the length of time allocated to reference frame collection on the latter star, which has reduced the bias between $\zeta$ Pup and 47 UMa with LOWFS. The OS11 LOWFS dynamics were re-run after making this discovery. While the statistics are simple and easily understood, they were not well appreciated until OS11 was run using LOWFSSim with the integrated detector model; that is, the high-speed integrated modeling capability uncovered gaps in our error budgeting that were later corrected, and would have been missed otherwise.

6. SUMMARY, CONCLUSION, AND FUTURE WORK

In this paper, we have demonstrated the capability to model ultra-precision $\lambda /10,\!000$ class wavefront sensing and control systems far faster than previously published works. We have shown several examples of engineering questions and modeling deliverables that show the need for this capability, including a number of problems that require correction that were not known prior to the use of LOWFSSim. These issues include, but are not limited to, significant coupling between parameters that were previously assumed to be separable, gaps in error budgets, and vulnerability of co-designed masks to wavefront control properties outside the set of optimization objectives. In total, the analyses shown in this paper alone required several billion evaluations of the LOWFSSim model. Modeling tools that are significantly slower would make this work computationally infeasible.

Next-generation space-based coronagraphs have more challenging requirements than the Roman Coronagraph and will require extensive engineering to achieve their scientific goals. Given the need for the exascale modeling demonstrated for the Roman Coronagraph, we believe a similar or even enhanced capability will be required to enable future mission success. The tools we developed can modeling larger systems with segmented apertures faster than real-time, and are well suited to handle future challenging modeling tasks.