Open Access
Add to BibSonomy
Abstract
The frequency recognition algorithm for multiple exposures (FRAME) is a progressive single-shot high-speed videography technique that employs the spatial-frequency multiplexing concept to provide high temporal and spatial resolution. However, the inherent crosstalk from the zero-frequency component to the carrier-frequency component leads to resolution degradation and artifacts. To improve recovered frames’ quality, we propose a FRAME reconstruction method using guided filters for a removal of the zero-frequency component, which can minimize the artifacts while enhance spatial resolution. A total variation (TV) denoising operation is involved to remove artifacts further to achieve optimized performances. Simulations and experiments were conducted to demonstrate the robust and efficient post-processing capability of the proposed method. With a two-frame experimental system, the results of a USAF 1951 resolution target reveal a 1.8-fold improvement in spatial resolution from 16 lp/mm to 28.5 lp/mm. For complex dynamic scenarios, the wide field of high-speed fuel spray was shot and the proposed method can resolve two droplets with a 30 μm distance which outperforms the traditional method.
© 2023 Optica Publishing Group under the terms of the Optica Open Access Publishing Agreement
1. Introduction
The relentless pursuit of scientific understanding has led to advancements in the microscopic and ultrafast domains, such as molecular motion [1–3], photosynthesis [4–6], organic crystallization [7–9], inertial confinement fusion (ICF) ignition [10–12], plasma discharge [13,14], and shock wave diagnoses [15]. Despite these advances, the challenge of high-resolution imaging of ultrafast events still remains [16–18]. For the frequency recognition algorithm for multiple exposures (FRAME) technique, a single-shot ultrafast imaging method presented by A. Ehn et. al. [19], spatial-frequency multiplexing is achieved via sequentially modulated pulse illumination, multiple exposures, and temporal frame reconstruction in the Fourier domain. During the reconstruction stage, the spatial lock-in algorithm was introduced to extract dynamic scenes delivered by carrier-frequency components [20]. Although reconstructed images have a lower spatial resolution than corresponding ground-truth images, the sparsity of natural images in the spatial-frequency domain ensures the preservation of most information [21,22]. However, non-negligible crosstalk and distortion from adjacent carrier- and zero-frequency components can occur when the number of frames increases, especially for frames around the zero frequency [23]. Despite attempts to improve reconstruction performance by limiting the number of frames or removing frames near the zero frequency, the intense zero-frequency component may still pose a challenge when the modulation of the illumination pattern is low [24].
In this work, we identify the crosstalk from the zero-frequency component as a main factor contributing to the deterioration of FRAME reconstruction quality. The intense zero-frequency component causes reconstruction artifacts especially for a low illumination pattern modulation due to imperfect imaging and environmental noise. Potential solutions are employing high frequency illumination patterns and high-resolution imaging system to move the carrier-frequency components far away from the zero-frequency and suppress the zero-frequency component. These strategies, however, have limited effect and increase the difficulty of system construction. Instead of system optimization, we proposed a data pre-processing method to effectively eliminate the zero-frequency component for reconstruction quality enhancement. Prior to the spatial lock-in reconstruction algorithm [22], guided filtering [25] is adopted to separate the zero-frequency component image from the superimposed image, which can be removed during reconstruction. This allows for the expansion of frame extraction filters without introducing artifacts, leading to significant enhancement of spatial resolution. Simulations and experiments were conducted, revealing that the proposed method can deliver at least 1.8-fold improvement in spatial resolution.
2. Methods
2.1 Principle of FRAME
The data acquisition and reconstruction processes of the FRAME methodology are presented in Fig. 1 [19]. In the data acquisition phase, sequential frames of a dynamic scenario are encoded by a sequence of sinusoidally modulated illumination pulses as shown in Fig. 1(a). The time-resolved frames are superimposed on a detector in a single exposure as shown in Fig. 1(b). The detector ’s output image containing $N$ frames can be expressed as
where $I_n$ is the ground truth of the $n^{th}$ frame of the dynamic scenario and $P_n$ is the $n^{th}$ illumination pulse intensity pattern which can be expressed as where $M$ is the modulation, $u_n$ and $v_n$ are spatial frequencies along the $x$ and $y$ axes, respectively. Via Fourier transformation ($FT$), $P_n$ in the Fourier domain is given by
Fig. 1. Workflow of FRAME. (a) Frames modulated with different structural illumination patterns. Superimposed image provided by an imaging sensor in a single-shot in (b) the spatial domain ($I$) and (c) the Fourier domain ($F_I$). (d) Reconstructed frames. $FT$ and $IFT$ represent the Fourier transformation and the inverse Fourier transformation. $O_{EF}$ represents the frame extraction operation.
In the reconstruction phase, frame extraction filters are applied to $F_I (u,v)$ for the $n^{th}$ frame extraction denoted as an operator of $O_{EF,n}$, shown in Fig. 1(c). Subsequently, the extracted frames are restored into the spatial domain via inverse Fourier transformations ($IFT$) as shown in Fig. 1(d). The reconstructed $n^{th}$ frame can be expressed as
2.2 Crosstalk from the zero-frequency component
For the traditional method shown in Fig. 1, reconstruction quality is significantly influenced by the crosstalk from the zero-frequency component, which locates at the center in the Fourier domain as shown in Fig. 1(c). The choice of the frame extraction bandwidth is crucial to compromise the loss of high-frequency detail with the artifacts introduced from the zero-frequency component crosstalk, whose energy is at least $2N$-fold of an individual carrier-frequency component ($M$ = 1) according to Eqs. (2) and (3).
To demonstrate the impact of the zero-frequency component crosstalk with different pattern modulations in FRAME reconstruction, simulations were conducted with $M$ = [0.2, 1] as shown in Fig. 2. To ensure resolvable fringe imaging with a camera, the bound of the valid Fourier map for frames arrangement is limited in a diamond with a diagonal of $R_{max}$= 1/2 $R_c$, where $R_c$ is the cut-off frequency of the detector, which has been clarified in Ref. [24], as shown in Figs. 2(d) and (h). Then, the patterns with arbitrary phases can be resolved with a reasonable modulation degree to ensure robust coding of the objects’ information. The extraction filter radius is set to $R$ = 1/2 $R_{carrier}$, where $R_{carrier}$ represents the carrier frequency. As expected, the reconstruction images are clearer with $M$ = 1 ((Figs. 2(i) and (j)) than $M$ = 0.2 (Figs. 2(e) and (f)), but still noisy due to the irremovable zero-frequency component crosstalk.
Fig. 2. Reconstruction of FRAME with different modulations. (a) – (b) Ground truth of two sequential frames. Superimposed images in the spatial and Fourier domain and the recovered frames with (c) – (f) $M$ = 0.2, and (g) – (j) $M$ = 1.
2.3 Removal of the zero-frequency component with guided filters
In this paper, guided filters are involved in the reconstruction process to isolate and remove the zero-frequency component. The workflow of the FRAME reconstruction methodology with guided filters is shown in Fig. 3. Prior to extracting frames from the Fourier domain using the spatial lock-in algorithm [22], the zero-frequency component acquired by guided filtering (Figs. 3(b) and (e)) is subtracted from the initial FRAME image (Figs. 3(a) and (d)). Without the zero-frequency component, the image (Figs. 3(c) and (f)) is then processed with the frame extraction filters to restore individual frames (Figs. 3(g) and (h)). For the $n^{th}$ frame, the workflow can be expressed as
where $G$ denotes the guided filtering output.Fig. 3. Workflow of the FRAME reconstruction methodology with guided filters. The superimposed image, image filtered by guided filters (GF), processed image by subtracting the filtered image from the superimposed image in (a) - (c) the spatial domain and (d) - (f) the Fourier domain. (g) – (h) Restored frames.
In the guided filtering process, the superimposed image $I$ from the detector is regarded as both the filtering input and guidance image [25]. The filtered output $G_i$ (the value at Pixel $i$ in $G$) is a linear transform of the pixel $i$ ($I_i$) in a window $w_k$ ($N_w$ pixels) centered at the pixel $k$.
where $(a_k, b_k)$ are constant linear coefficients in $w_k$, which are determined by minimizing the difference between $G$ and $I$ for denoising. The cost function in the window $w_k$ is expressed as where $\epsilon$ is a regularization parameter preventing $a_k$ from being too large. The solution is given by linear regression where $\mu _k$ and ${\sigma _k}^2$ are the mean and variance of $I$ in $w_k$ and $N_w$ is the pixel amount in $w_k$.Because a pixel $i$ is involved in all the windows $w_k$ that contain $i$, after computing $(a_k,b_k)$ for all patches $w_k$ in the image, we average the output by
where ${\bar {a}}_i=\sum _{k\in w_i} a_k$ and ${\bar {b}}_i=\sum _{k\in w_i} b_k$. The averaging strategy of overlapping windows leads to the denoising of original images. More details of the guided filtering can be found in [25]. In the context of FRAME, the sinusoidal textures of the detector image can be regarded as noise to be flattened for guided filters, while edges of the object can maintain sharp, producing the zero-frequency component separately for further processing.3. Simulations
To investigate the performance of the proposed method, simulations were conducted with a two-frame configuration. Figure 4(a) shows the ground truth of Frames 1 and 2 with a size of 512$\times$512 pixels and a pixel size of 5 $\mu$m. Thus, the cut-off frequency is $R_c$ = 100 lp/mm and $R_{max}$ = 50 lp/mm. Figures 4(b) and (c) show the superimposed image of modulated Frames 1 and 2 with $R_{carrier}$ = 30 lp/mm, $M$ = 0.5, and modulated angles of −30$^{\circ }$ and 60$^{\circ }$ in the spatial domain $I$ and the Fourier domain $F_I$, respectively. Figures 4(d) and (e) show the Fourier map generated by subtracting the zero-component $F_G$ from $F_I$ and the ratio image of $(F_I-F_G)/ F_I$. The images are normalized into the range from 0 to 1.
Fig. 4. Simulations with a two-frame configuration. (a) Ground truth of Frames 1 and 2. (b) Superimposed image of modulated Frames 1 and 2 in (b) the spatial domain $I$ and (c) the Fourier domain $F_I$. (d) Fourier map ($F_I$ – $F_G$) used for reconstruction by removing the zero-frequency component $F_G$ generated with guided filters from $F_I$. (e) Ratio image of $(F_I$ – $F_G)/ F_I$.
The traditional method extracts individual frames from $F_I$ directly, which inevitably introduces crosstalk from the zero-frequency component especially with large frame extraction filters. The proposed method using $F_I-F_G$ for reconstruction can effectively reduce the influence of the zero-frequency component crosstalk. The ratio image $(F_I-F_G)/ F_I$ implies that the zero-frequency component is removed while the carrier-frequency components are maintained even they are mixed.
By employing frame extraction filters with radii $R$ = [1/3, 2/3, 1] $R_{carrier}$, the reconstructed frames with the traditional and proposed methods are shown in Figs. 5(a) – (c) and Figs. 5(d) – (f), respectively. The assessment values of mean square error (MSE), peak signal-to-noise ratio (PSNR), structure similarity index measure (SSIM) indexes [26] are labeled on each frame. The intensity profiles of the labeled lines in the frames are shown in Figs. 5(g) – (i) with the ground truth (black), the traditional method results (dash), and the proposed method results (solid).
Fig. 5. Reconstruction performance comparison between the (a) – (c) traditional and (d) – (f) proposed methods with various frame extraction filter radii $R$ = [1/3, 2/3, 1] $R_{carrier}$. The assessment values of MSE, PSNR, SSIM indexes are labeled on each frame. (g) – (i) Intensity profiles of the labeled lines in the recovered images including the ground truth (black), traditional method results (dash), and proposed method results (solid).
For small frame extraction filters with $R$ = 1/3 $R_{carrier}$, Figs. 5(a) and (d), the traditional and proposed methods have the similar performances with smoothed edges (Fig. 5(g)), due to both the high-frequency information of the carrier-frequency components and the crosstalk of the zero-frequency component has been blocked. With a larger $R$ = 2/3 $R_{carrier}$, Figs. 5(b) and (e), more artifacts are introduced in the results of the traditional method with larger MSEs and lower PSNRs than the proposed method, while both the edges are getting sharper (Fig. 5(h)). When $R$ is getting to $R_{carrier}$, Figs. 5(c) and (f), the traditional method results are heavily distorted due to more zero-frequency component crosstalk are included, while the proposed method can remain high fidelity and provide edges with further elevated sharpness (Fig. 5(i)).
To investigate the scenarios with different modulations ($M$ = 0.2, 0.5, 1), simulations were conducted as shown in Fig. 6. The results show that high modulations lead to high reconstruction quality for both the traditional and proposed methods. The proposed method can still extract the underground truth under a weak modulation ($M$ = 0.2), while the traditional method delivers worse results due to the relatively stronger crosstalk from the zero-frequency component. To approach experimental conditions, $M$ = 0.5 was used for other simulations.
Fig. 6. Reconstruction performance comparison between the (a) – (c) traditional and (d) – (f) proposed methods with various pattern modulations $M$ = [0.2, 0.5, 1] $R_{carrier}$. Bar plots of (g) SSIM indexes, (h) PSNRs, (i) MSEs.
Figure 7 presents the influence of guided filters on reconstructed performance of the proposed method with different number of pixels $N_w$ in a filtering window and the degree of smoothing $\epsilon$. The frame extraction filter radius is $R$ = $R_{carrier}$. With small $\epsilon$ = $0.2^2$, artifacts from the other frame are significant; and with the increase of $\epsilon$, the background is cleaner. In addition, the larger the size of the filter window, the more the artifacts. This is because the averaging effect of filter increases when the window size increases. Generally, the pixels only show the strong correlation locally. In this way, a large window introduces the excessive averaging for the edges of patterns. In the following simulations, $\epsilon$ = $1^2$ and $N_w$ = $3^2$ are adopted.
Fig. 7. Influence of the parameters of guided filters, number of pixels $N_w$ in a filtering window and the degree of smoothing $\epsilon$, on the reconstructed frames using the proposed method. Recovered frames with (a) – (c) $N_w$ = $3^2$ and (a) – (c) $N_w$ = $15^2$ for $\epsilon$ = [0.2$^2$, 0.5$^2$, 1$^2$], respectively.
To improve reconstruction performance further, the total variation (TV) denoising method using the TV-L1 model optimized with a primal-dual algorithm [27] is used to remove artifacts of the recovered frames. The results of the traditional and proposed methods without and with TV under the extraction filter radius of $R_{carrier}$ are shown in Figs. 8(a) –(d). TV denoising can effectively smooth the noises and remain sharp edges. However, the results of traditional method with TV lost detailed information labeled with green circles and suffers shape distortion (Fig. 8(b)); whereas the proposed method recovers the frames with high fidelity (Fig. 8(d)). The assessment values of Frames 1 and 2 for the four cases are shown in Figs. 8(e) – (g), revealing that the proposed method with TV has the most outstanding performance.
Fig. 8. Reconstruction results of the traditional method (a) without TV (Case 1) and (b) with TV (Case 2), and the proposed method (c) without TV (Case 3) and (d) with TV (Case 4). (e) SSIM indexes, (f) PSNRs, and (g) MSEs of Frames 1 and 2 for Cases 1 – 4.
For a more complex scene with the ground truths from the UKBench image dataset (Fig. 9(a)), two frames are extracted from the superimposed image (Fig. 9(b)) for four cases (Figs. 9(c) – (f)) and the reconstruction quality improvement is obvious from Case 1 (the traditional method) to Case 4 (the proposed method with TV). The results of Case 4 (Fig. 9(f)) approach to the ground truth by remaining the structure and shape edges. The SSIM indexes, PSNRs, and MSEs improve significantly from Case 1 to Case 4 as shown in Figs. 9(g) – (i).
Fig. 9. (a) Ground truth of a more complex scene from the UKBench image dataset. (b) Superimposed image used for reconstruction. Reconstruction results with the traditional method (c) without TV (Case 1) and (d) with TV (Case 2), and the proposed method (e) without TV (Case 3) and (f) with TV (Case 4). (g) SSIM indexes, (h) PSNRs, and (i) MSEs of Frames 1 and 2 for Cases 1 – 4.
For more frames, the distances between individual frames in the Fourier domain decrease, leading to more crosstalk from the adjacent frames and limiting the size of the frame extraction filter. Figure 10 and Fig. 11 show the performance of the proposed method for multiple frame cases with four and eight frames, respectively. In Fig. 10, $R_{carrier}$ = [$\sqrt {2}/2$, $\sqrt {2}/2$, 1, 1]$R_{max}$ for Frames 1 - 4 and the frame extraction filters’ radii $R = \sqrt {2}R_{max}/2$. In Fig. 11, $R_{carrier}$ = [5/8, 5/8, $\sqrt {2}/2$, $\sqrt {2}/2$, 5/8, 5/8, 1, 1]$R_{max}$ for Frames 1 - 8 and the frame extraction filters’ radii $R$ = $\sqrt {2}R_{max}/2$. When the size of the frame extraction filter is larger than the distance between the adjacent frames, here is a tip for minimizing the influence of the adjacent frames by setting the value surrounding $R_{carrier}$ of the adjacent frames to zero to avoid periodic sinusoidal patterns’ interference in the reconstructed frames. It shows that the proposed method can still remove the zero-frequency component and outperforms the traditional method; however, the reconstruction quality is irretrievably decrease due to the more intense inter-frame crosstalk.
Fig. 10. (a) Ground truth of four frames from the UKBench image dataset. (b) Superimposed image of the four frames used for reconstruction in the spatial domain and the Fourier domain where corresponding frame numbers are labeled. Reconstruction results with the (c) traditional and (d) proposed methods without and with TV. (e) Bar plots of SSIM indexes, PSNRs, and MSEs.
Fig. 11. (a) Ground truth of eight frames from the UKBench image dataset. (b) Superimposed image of the eight frames used for reconstruction in the spatial domain and the Fourier domain where corresponding frame numbers are labeled. Reconstruction results with the (c) traditional and (d) proposed methods without and with TV. (e) Bar plots of SSIM indexes, PSNRs, and MSEs.
4. Experiments
To demonstrate the efficacy of the proposed method in enhancing reconstruction performance experimentally, single-shot experiments shooting temporal frames of a 1951 USAF resolution target and a high-speed fuel spray were conducted. The experimental setup is shown in Fig. 12(a). Pulsed LEDs (Cree XHP35) were used to illuminate Ronchi gratings (40 lp/mm, Thorlabs R1L3S13N) through diffusers and achromatic lenses ($f$ = 50 mm, DaHeng GCL-010630) to generate sinusoidally modulated illumination patterns. Through a beam splitter and a tube lens ($f$ = 50 mm, Thorlabs TTL200), the illumination pulses are imaged on a dynamic target. We defocused the tube lens slightly to block high order information of Ronchi gratings. The sinusoidally modulated temporal frames of the dynamic target are superimposed on a CMOS camera (DaHeng ME2P-2621-15U3M/C, 5120$\times$5120 pixels, 2.5 $\mu$m$\times$2.5 $\mu$m pixel size) through a telecentric lens with a magnification of 1$\times$. The frame rate and exposure time were set by adjusting the trigger delays and pulse widths of the two illumination channels, as shown in Fig. 12(b).
Fig. 12. (a) Schematic diagram of the adopted FRAME experimental setup. (b) Illumination pulses’ patterns and delays.
4.1 Resolution enhancement
The resolution enhancement was quantified with a moving 1951 USAF resolution target. The two frames of target were shot separately. In the first shot, we put the target in the upright position and illuminated it by the first LED. In the second shot, we rotated the target and illuminated it by the second LED. In the process of calculation, we added the two pictures and reconstructed it using our method. The performances of the reconstructed images obtained with the traditional and proposed methods combined with TV were compared. Superimposed image on the CMOS camera in a single exposure is shown in Figs. 13(a) and (b) in the spatial and Fourier domain, which are normalized. With the proposed method, the Fourier map demonstrates an effective removal of the zero-frequency component, shown in Fig. 13(c). The traditional and proposed methods with frame extraction frequencies $R$ = [20, 40] lp/mm are shown in Figs. 13(d) – (g), where the zoomed-in area in the green boxes contains the minimum resolved line pairs labeled by the yellow boxes. For the case of $R$ = 20 lp/mm, the traditional and proposed methods show similar performance with a resolution of 16 lp/mm (the $1^{st}$ line pair in the $4^{th}$ group), whose intensity profiles of the vertical (blue) and horizontal (red) line pairs for Frames 1 and 2 are shown in Figs. 13(h) and (i). For the case of $R$ = 40 lp/mm, the proposed method not only enhances the contrast of the resolvable line pairs in Figs. 13(h) and (f) with $R$ = 20 lp/mm but also increases the spatial resolution to 28.5 lp/mm (the $6^{th}$ line pair in the $4^{th}$ group) for both Frames 1 and 2 whose intensity profiles are shown in Figs. 13(j) and (k); whereas the images restored by the traditional method are drowned in artifacts and failed to be resolved. It is worth noting that the difference of resolution in vertical and horizontal directions is caused by the crosstalk from the other frame.
Fig. 13. The original output superimposed image consists of two resolution target images in the (a) spatial and (b) Fourier domain. (c) The Fourier map removing the zero-frequency component generated with guided filters. The restored images using (d) – (e) the traditional and (f) – (g) the proposed methods. The intensity profile of the minimum resolved line pairs for Frames 1 and 2 with (h) - (i) $R$ = 20 lp/mm and (j) - (k) $R$ = 40 lp/mm for the proposed method.
4.2 High-speed spray experiments
For a complex dynamic scenario, the transient states of the high-speed spray were photographed. Pulses with a width of 10 $\mu$s and an interval of 100 $\mu$s were employed to illuminate the spray. The superimposed images in the spatial and Fourier domain are shown in Figs. 14(a) – (b), and the Fourier map removing the zero-frequency component by guided filters is shown in Fig. 14(c). Two restored frames with $R$ = [20, 40] lp/mm for the traditional and the proposed strategies combined with TV are shown in top left and bottom right in Figs. 14(d) – (g), respectively. When $R$ = 20 lp/mm, the traditional and proposed methods have the similar performance, shown in Figs. 14(d) and (e); when $R$ = 40 lp/mm, the proposed method provides an outstanding performance and the traditional method generates gross artifacts, shown in Figs. 14(f) and (g).
Fig. 14. Superimposed image consists of two high-speed spray images in the (a) spatial and (b) Fourier domain. (c) Fourier map removing the zero-frequency component with guided filters. (d) – (g) Restored frames employing the traditional (top left) and proposed (bottom right) methods with $R$ = [20, 40] lp/mm. (h) Area recovered by the traditional method with $R$ = 20 lp/mm (bottom) and the proposed method with $R$ = 40 lp/mm (top), which is labeled by a black box in (e). (i1) – (i3) Two droplets labeled by a yellow box in (d) for Cases 1-3. (j1) – (j3) Intensity profiles along the dash lines in (i1) – (i3).
The proposed method can outperform the traditional one in details. For Frame 1, the details for Cases 1-3 located in the yellow box in Fig. 14(d) are shown in Figs. 14(i1) – (i3) and the corresponding intensity profiles along the dash lines are shown in Figs. 14(j1) – (j3). Two droplets with a distance of 30 $\mu$m are recovered by the proposed method with $R$ = 40 lp/mm (Case 1, Figs. 14(i1) and (j1)). However, for the traditional method, this detail is blurry with $R$ = 20 lp/mm (Case 2, Figs. 14(i2) and (j2)) and drowned in noise with $R$ = 40 lp/mm (Case 3, Figs. 14(i3) and (j3)). For Frame 2, the quality enhancement is shown in Fig. 14(h), which is the zoomed-in area labeled in Fig. 14(e). The top half of Fig. 14(h) recovered by the proposed method with $R$ = 40 lp/mm shows sharper edges and smaller droplets; in contrast, the bottom part recovered by the traditional method with $R$ = 20 lp/mm is smoother leading to blurred edges and droplets.
5. Conclusion
We presented a guided filtering method to eliminate the zero-frequency crosstalk in FRAME, enabling resolution enhancement for transient frame reconstruction. TV-L1 method is involved to further denoise the recovered frames, providing a robust reconstruction, as we show in Code 1 [28]. Two-frame configuration simulations were conducted to demonstrate the performance of the proposed method, in which the super-parameters $\epsilon = 1^2$ and $N_w = 3^2$ are recommended for guided filter optimization. A LED-based two-frame validation setup was applied to shoot the USAF 1951 test target and the dynamic of the high-speed spray. The proposed method provides a 1.8-fold enhancement in reconstruction resolution from 16 lp/mm to 28.5 lp/mm for the USAF 1951 test target, and more detail recovery for the spray scenario, resolving two droplets spacing 30 $\mu$m. The methodology can be further applied to ultrafast FRAME systems, where an ultrafast laser system with optical delaying is used, with a temporal resolution on the order of picoseconds or femtoseconds to achieve enhanced and robust frame reconstruction, revealing dynamic scenes with high fidelity and promoting the understanding of ultrafast phenomena.
Funding
National Natural Science Foundation of China (12204529); Scientific Instrument Developing Project of the Chinese Academy of Sciences (GJJSTD20220006).
Disclosures
The authors declare no conflicts of interest.
Data availability
Data underlying the results presented in this paper are not publicly available at this time but may be obtained from the authors upon reasonable request.
References
1. K. S. Zinchenko, F. Ardana-Lamas, I. Seidu, et al., “Sub-7-femtosecond conical-intersection dynamics probed at the carbon k-edge,” Science 371(6528), 489–494 (2021). [CrossRef]
2. T. L. Cocker, D. Peller, P. Yu, et al., “Tracking the ultrafast motion of a single molecule by femtosecond orbital imaging,” Nature 539(7628), 263–267 (2016). [CrossRef]
3. J. Clark, L. Beitra, G. Xiong, et al., “Ultrafast three-dimensional imaging of lattice dynamics in individual gold nanocrystals,” Science 341(6141), 56–59 (2013). [CrossRef]
4. R. Dods, P. Båth, D. Morozov, et al., “Ultrafast structural changes within a photosynthetic reaction centre,” Nature 589(7841), 310–314 (2021). [CrossRef]
5. B. Kriete, J. Lüttig, T. Kunsel, et al., “Interplay between structural hierarchy and exciton diffusion in artificial light harvesting,” Nat. Commun. 10(1), 4615 (2019). [CrossRef]
6. K. E. Sheetz and J. Squier, “Ultrafast optics: Imaging and manipulating biological systems,” J. Appl. Phys. 105(5), 051101 (2009). [CrossRef]
7. S. Chen, Z. Li, Y. Qiao, et al., “Solution-processed organic semiconductor crystals for field-effect transistors: from crystallization mechanism towards morphology control,” J. Mater. Chem. C 9(4), 1126–1149 (2021). [CrossRef]
8. W. Chyan and R. T. Raines, “Enzyme-activated fluorogenic probes for live-cell and in vivo imaging,” ACS Chem. Biol. 13(7), 1810–1823 (2018). [CrossRef]
9. N. Pal, M. Wu, and H. P. Lu, “Probing conformational dynamics of an enzymatic active site by an in situ single fluorogenic probe under piconewton force manipulation,” Proc. Natl. Acad. Sci. 113(52), 15006–15011 (2016). [CrossRef]
10. A. Oudin, A. Debayle, C. Ruyer, et al., “Reduction of cross-beam energy transfer by a speckle pattern,” Phys. Rev. Lett. 127(26), 265001 (2021). [CrossRef]
11. D. Casey, O. Landen, E. Hartouni, et al., “Three dimensional low-mode areal-density non-uniformities in indirect-drive implosions at the national ignition facility,” Phys. Plasmas 28(4), 042708 (2021). [CrossRef]
12. J. Zhou, P. Gan, H. Zhao, et al., “Numerical analysis of temperature characterization of air gap discharge in solid insulation under power frequency voltage,” Phys. Plasmas 29(3), 033509 (2022). [CrossRef]
13. Z. Wu, G. Sun, Z. Liu, et al., “Characteristics of plasma properties in double discharge ablative pulsed plasma thrusters,” Phys. Plasmas 24(11), 113521 (2017). [CrossRef]
14. V. Prokof’ev and V. Smolyakov, “Ignition and unstable regimes of gasless combustion of a disk-shaped sample,” Combust., Explos. Shock Waves 53(1), 36–40 (2017). [CrossRef]
15. M. Y. Timokhin, M. Tikhonov, I. Mursenkova, et al., “Shock-wave thickness influence to the light diffraction on a plane shock wave,” Phys. Fluids 32(11), 116103 (2020). [CrossRef]
16. V. Tiwari, M. Sutton, and S. McNeill, “Assessment of high speed imaging systems for 2d and 3d deformation measurements: methodology development and validation,” Exp. Mech. 47(4), 561–579 (2007). [CrossRef]
17. V. Lebedev, A. Demchenko, V. Krutikov, et al., “Miniaturization of high-speed streak cameras for fast running processes recording,” in The 31st International Congress on High-speed Imaging and Photonics, (2016), pp. 271–276.
18. L. Gao, J. Liang, C. Li, et al., “Single-shot compressed ultrafast photography at one hundred billion frames per second,” Nature 516(7529), 74–77 (2014). [CrossRef]
19. A. Ehn, J. Bood, Z. Li, et al., “Frame: femtosecond videography for atomic and molecular dynamics,” Light: Sci. Appl. 6(9), e17045 (2017). [CrossRef]
20. E. Kristensson, Z. Li, E. Berrocal, et al., “Instantaneous 3d imaging of flame species using coded laser illumination,” Proc. Combust. Inst. 36(3), 4585–4591 (2017). [CrossRef]
21. Z. Li, J. Borggren, E. Berrocal, et al., “Simultaneous multispectral imaging of flame species using frequency recognition algorithm for multiple exposures (frame),” Combust. Flame 192, 160–169 (2018). [CrossRef]
22. V. Kornienko, E. Kristensson, A. Ehn, et al., “Beyond mhz image recordings using leds and the frame concept,” Sci. Rep. 10(1), 16650 (2020). [CrossRef]
23. S. Ek, V. Kornienko, and E. Kristensson, “Long sequence single-exposure videography using spatially modulated illumination,” Sci. Rep. 10(1), 18920 (2020). [CrossRef]
24. H. Li, Y. Li, X. Wang, et al., “Investigation of single-shot high-speed photography based on spatial frequency multiplexing,” J. Opt. Soc. Am. A 40(3), 521–529 (2023). [CrossRef]
25. K. He, J. Sun, and X. Tang, “Guided image filtering,” IEEE Trans. Pattern Anal. Mach. Intell. 35(6), 1397–1409 (2013). [CrossRef]
26. Z. Wang, A. C. Bovik, H. R. Sheikh, et al., “Image quality assessment: from error visibility to structural similarity,” IEEE Trans. on Image Process. 13(4), 600–612 (2004). [CrossRef]
27. A. Chambolle and T. Pock, “A first-order primal-dual algorithm for convex problems with applications to imaging,” J. Math. Imaging Vis. 40(1), 120–145 (2011). [CrossRef]
28. H. Li and Y. Li, “Code for the frame method using the guided filtering for enhancing spatial resolution,” figshare (2023), https://doi.org/10.6084/m9.figshare.24079395.
