Open Access
Add to BibSonomy
Abstract
The generalized contrast-to-noise ratio (gCNR) is a relatively new image quality metric designed to assess the probability of lesion detectability in ultrasound images. Although gCNR was initially demonstrated with ultrasound images, the metric is theoretically applicable to multiple types of medical images. In this paper, the applicability of gCNR to photoacoustic images is investigated. The gCNR was computed for both simulated and experimental photoacoustic images generated by amplitude-based (i.e., delay-and-sum) and coherence-based (i.e., short-lag spatial coherence) beamformers. These gCNR measurements were compared to three more traditional image quality metrics (i.e., contrast, contrast-to-noise ratio, and signal-to-noise ratio) applied to the same datasets. An increase in qualitative target visibility generally corresponded with increased gCNR. In addition, gCNR magnitude was more directly related to the separability of photoacoustic signals from their background, which degraded with the presence of limited bandwidth artifacts and increased levels of channel noise. At high gCNR values (i.e., 0.95-1), contrast, contrast-to-noise ratio, and signal-to-noise ratio varied by up to 23.7-56.2 dB, 2.0-3.4, and 26.5-7.6×1020, respectively, for simulated, experimental phantom, and in vivo data. Therefore, these traditional metrics can experience large variations when a target is fully detectable, and additional increases in these values would have no impact on photoacoustic target detectability. In addition, gCNR is robust to changes in traditional metrics introduced by applying a minimum threshold to image amplitudes. In tandem with other photoacoustic image quality metrics and with a defined range of 0 to 1, gCNR has promising potential to provide additional insight, particularly when designing new beamformers and image formation techniques and when reporting quantitative performance without an opportunity to qualitatively assess corresponding images (e.g., in text-only abstracts).
© 2020 Optical Society of America under the terms of the OSA Open Access Publishing Agreement
1. Introduction
Multiple image quality metrics are currently available to assess target detectability in photoacoustic images. Common metrics include contrast, contrast-to-noise ratio (CNR), and signal-to-noise ratio (SNR). One shortcoming of these traditional metrics is that adjustments to the dynamic range of images can arbitrarily raise contrast and CNR measurements, giving a false impression of improved image quality without improving target detectability [1]. In addition, because contrast, CNR, and SNR measurements lack an upper bound, these metrics can have high values that further complicate interpretations and comparisons. From the perspective of target detection in a photoacoustic image, if the photoacoustic target is clearly distinguishable from the surrounding tissue, there is minimal clinical rationale to developing advanced beamforming methods that further increase contrast, CNR, and SNR. From a probability perspective, contrast and SNR do not provide information about target separability. While CNR was originally intended to provide this information [2,3], modern beamformers have introduced a shift from this original intent, producing CNR magnitudes that are not intuitive for comparisons across beamforming methods and do not directly correspond to target detectability.
The generalized contrast-to-noise ratio (gCNR) is a relatively new metric for target detectability that is expected to be robust to the known shortcomings of commonly used image quality metrics. This metric was initially introduced for and applied to ultrasound images to determine a separability rate (e.g., a gCNR of 0.9 indicates that, at best, 90% of pixels can be separated) [4]. Because gCNR is fundamentally a measurement of the separability between two regions in an image, it is applicable to other medical images, such as photoacoustic images.
Photoacoustic imaging requires the emission of non-ionizing electromagnetic radiation from a light source and subsequent optical absorption by a target of interest, which causes thermal expansion and the generation of acoustic pressure that can be detected with a conventional ultrasound transducer [5–7]. Generally, when a region of tissue is illuminated with sufficient fluence, a photoacoustic signal is generated and the background region surrounding the signal in photoacoustic images is typically associated with noise. Therefore, gCNR has the potential to be more relevant than existing photoacoustic image quality metrics when quantifying how well a target can be distinguished from its background. This potential is applicable to a wide range of photoacoustic imaging applications, including detection of hemoglobin, lipids, and dyes [7,8], simultaneous visualization of biological photoacoustic targets and nearby needle tips or surgical tool tips [9–11], identification of atherosclerotic plaque [12], and visualization of tumor vascularization [12–14].
We consider that photoacoustic target detection can be framed as a two-class classification problem where pixels in the image are categorized as either inside or outside the target region of interest (ROI). From an image quality perspective, in an ideal scenario, all photoacoustic target pixels would be assigned to the target and all background pixels (i.e., regions outside the photoacoustic target) would be assigned to the background, which corresponds to a gCNR of 1. Therefore, a gCNR <1 is considered to be less than ideal, and a gCNR of 0 is considered to be indicative of poor image quality. Expanding on the example of a gCNR of 0.9, this indicates that 90% of the pixels in the image can be unambiguously assigned to either the target or background based on their amplitudes. The remaining 10% of pixels can be assigned to either the target or the background region. If we assume that the target and background ROI sizes are the same, this would lead to a minimum error of 50% when assigning these remaining pixels, resulting in an overall misclassification error of at least 5%. Therefore, in addition to being robust to the previously mentioned shortcomings of traditional image quality metrics, gCNR provides a more intuitive measurement of the Bayes risk associated with the proposed classification task.
The motivation of this paper is two-fold. First, we investigate the suitability of gCNR to assess photoacoustic image quality by comparing against and demonstrating similarity with three traditional image quality metrics (i.e., contrast, CNR, and SNR), as confirmation that gCNR is generally consistent with qualitative observations of photoacoustic images. Second, after establishing gCNR as a suitable photoacoustic image quality metric, we explore the ability of gCNR to overcome limitations of more traditional image quality metrics, with particular attention to common mechanisms for photoacoustic signal degradation or enhancement, such as channel noise, clutter, limited transducer bandwidth, and the common post-processing step of applying a minimum threshold to photoacoustic images. The primary contributions of this paper include an evaluation of gCNR applied to simulated and experimental photoacoustic images created with amplitude- and coherence-based beamformers, a comparison of this application with three traditional image quality metrics, and an assessment of gCNR performance in the presence of the common mechanisms for photoacoustic signal degradation or enhancement described above.
The remainder of this paper is organized as follows. Section 2 presents a formulation of gCNR that is specific to photoacoustic signal generation after wavelength selection to identify singular photoacoustic targets (which often appear as strong photoacoustic signals surrounded by background noise). Section 3 describes our simulation parameters, phantom and in vivo experimental setups, and evaluation methods. Section 4 presents results of the simulation studies and validation with experiments. Section 5 discusses implications of our results and some possible future directions. Finally, Section 6 presents our concluding remarks.
2. Generalized contrast-to-noise ratio of singular photoacoustic targets
The gCNR metric is calculated from the overlap of the probability density functions (PDFs) of ROIs inside a target and background, as shown in Fig. 1. While it would be desirable for these two regions to be distinctly separated from each other, there are often cases where some overlap occurs, which contributes to complications surrounding image interpretation and overall image quality. In particular, the red shaded region in Fig. 1 represents the probability of not detecting target pixels, and the blue shaded region represents the probability of falsely identifying background regions as target.
Fig. 1. Probability density functions (PDFs) for regions inside and outside a photoacoustic target (i.e., $p_i(x)$ and $p_o(x)$, respectively), where $\varepsilon _0$ is the optimal threshold for minimizing error, represented by the shaded regions.
Note that the PDF of the target ROI in Fig. 1 is generally expected to lie to the right of the PDF of the background ROI due to the expectation that singular photoacoustic targets (e.g., surgical tool tips, blood vessels, contrast agents) are typically brighter than the surrounding background. In ultrasound imaging, the relative locations of these PDFs are interchangeable, depending on the presence of hyperechoic, hypoechoic, or anechoic targets. This subtle difference, caused by differences in the mechanisms of ultrasound and photoacoustic image generation, requires a rearrangement of the presented formulation for ultrasound images (i.e., Eq. 14 in Ref. [4]). This rearrangement results in the following equation, which describes the total overlap area under two PDFs from regions inside and outside of a photoacoustic target:
where $p_i(x)$ and $p_o(x)$ are the PDFs for regions inside and outside of a target, respectively, $OVL$ is the total overlap area of the two PDFs derived from photoacoustic images, and $\varepsilon _0$ is the optimal threshold which minimizes the probability of misidentifying background as target and target as background.The overlap area is then related to gCNR using the equation:
Like other traditional metrics, gCNR is dependent on ROI selection. Therefore, it is important to carefully select ROIs to represent the target and background pixels to ensure that the obtained values are representative of the intended probability measurement.In a practical setting, the overlap is difficult to compute using Eq. (1) due to a lack of knowledge of the underlying distributions $p_i(x)$ and $p_o(x)$. An equivalent histogram-based expression for the gCNR was derived with $N$ bins centered at $\{x_0, x_1,\ldots , x_{N-1}\}$. The associated histograms, $h_i$ and $h_o$, respectively, were computed for the target and background ROIs, respectively. Equation (2) was then discretized to obtain the following equation:
where $k$ is the index of the bin. This histogram-based method is a simpler implementation than the theoretical representations presented in Fig. 1 and Eq. (1), as Eq. (3) does not require estimation of an optimal threshold (i.e., $\varepsilon _0$).3. Methods
3.1 Simulation methods
Photoacoustic channel data were simulated using the k-Wave software package [15]. A transducer was defined with 0.3 mm pitch, 0.06 mm kerf, and 128 elements. Unless otherwise stated, the transducer had a center frequency of 5 MHz and 2-8 MHz bandwidth. A 6 mm-, 8 mm-, or 10 mm-diameter circular target containing randomly distributed optical absorbers with a spatial density of 299 absorbers/mm$^2$ was placed at the center of a $23.1 \textrm { mm}\times 38.4\textrm { mm}$ phantom. The spatial density of these absorbers was chosen based on the concentration of red blood cells in human blood [16]. The size of each absorber was limited to one grid point, which had an area of 5.8 $\times$ 10$^{-3}$ mm$^2$, approximately equal to the cross-sectional area of a red blood cell [17].
To explore the impact of channel noise (which is common in most experimental photoacoustic imaging setups), simulated images with channel SNR values ranging from −40 dB to 40 dB were analyzed. Channel SNR was determined by first segmenting the channel data in time to isolate the target signal. Gaussian-distributed, bandpass-filtered noise (within the transducer bandwidth of 2-8 MHz) was scaled to achieve the desired channel SNR and added to the channel data using methods available in the k-Wave ‘addNoise.m’ function. A negative channel SNR indicates that the ratio between the mean of the signal and the standard deviation of the noise is less than 1. Note that no thresholding was applied to the channel data.
To explore the impact of limited transducer bandwidth, simulated channel data from the 6 mm-diameter target with −10 dB channel SNR were filtered to bandwidths ranging 1-6 MHz within the range of the entire transducer bandwidth (i.e., 2-8 MHz).
3.2 Experimental phantom
An experimental phantom was created from Tygon tubing with internal diameters of 6 mm, 8 mm, and 10 mm, each suspended in a water-filled container. The tubing was first filled with water to determine the photoacoustic signal caused by the tubing alone. The water in the tubing was then replaced with 0.067% India ink solution. Each tube diameter was individually imaged. A total of ten photoacoustic images were acquired per tube diameter.
The photoacoustic imaging equipment consisted of an Alpinion ECUBE 12R ultrasound system (Seoul, Korea) with a 128-element Alpinion L3-8 linear transducer. The ultrasound system was connected to a Phocus Mobile laser (OPOTEK, Carlsbad, CA) with laser light delivered through a 5 mm-diameter fiber bundle. A fixed laser wavelength of 760 nm was selected to visualize the targets. The laser operated at a frequency of 10 Hz, with a pulse width of 10 ns and an energy output of 80 mJ from the tip of the fiber bundle.
3.3 In vivo experiment
In vivo data from a Sus domesticus porcine model were acquired during the experiments described in our previous publication [18], using the photoacoustic imaging equipment described in Section 3.2. An open laparotomy was performed to expose the liver. A laser wavelength of 750 nm and an output energy from the fiber bundle of 53 mJ were used to visualize a hepatic vein, which is shown in the ultrasound image of Fig. 2. This study was approved by the Johns Hopkins University Animal Care and Use Committee.
3.4 Beamforming methods
Photoacoustic images were created using standard delay-and-sum (DAS) beamforming, as well as short-lag spatial coherence (SLSC) imaging [19–21]. The process implemented to create SLSC images is described by the following equations:
3.5 Image quality metrics
Traditional image quality metrics (e.g., contrast, CNR, and SNR) were measured from the image data obtained after beamforming and normalization (and thresholding for SLSC), using the following equations:
where $\mu _i$ and $\mu _o$ are the mean of signal amplitudes within circular regions of interest (ROIs) inside and outside of a photoacoustic target, respectively, $\sigma _i$ and $\sigma _o$ are the standard deviations of signal amplitudes within circular ROIs inside and outside the photoacoustic target, respectively.For the simulated datasets, the circular ROIs were 6 mm in diameter, and the ROI outside of the photoacoustic target (i.e., the background ROI) was shifted 15 mm to the right of the ROI within the target. For the experimental phantom datasets, the circular ROIs were 80% of the tubing diameter, and the background ROI was shifted 12 mm to the right of the ROI within the target. For the in vivo datasets, the ROIs were 7.5 mm in diameter, and the background ROI was shifted by 18 mm to the left of the ROI within the target. In addition to these traditional metrics, gCNR was measured from the same image data, using Eq. (3) and the same ROIs described above. A total of $N=256$ bins were used to create the histograms for gCNR measurement.
4. Results
4.1 Application to simulated DAS images
Figure 3 shows simulated DAS photoacoustic images of a 6 mm-diameter target with channel SNR varied from −20 dB to 0 dB. The target and background ROIs are shown in red and blue, respectively, and the corresponding gCNR values are reported at the bottom of each image. The minimum channel SNR that enabled us to detect a target displayed with −30 dB dynamic range was −12 dB. Therefore, −12 dB channel SNR is considered to be the qualitatively determined limit for 6 mm-diameter target detectability within these parameters of the simulated dataset.
Fig. 3. Simulated DAS photoacoustic images with varying target diameters and channel SNRs. The corresponding gCNR value is overlaid on each image. The target ROI is outlined in red, and the background ROI is outlined in blue.
Figure 4(a) shows gCNR, contrast, CNR, and SNR as channel SNR varied from −40 dB to 40 dB for the simulated DAS 6 mm-, 8 mm-, and 10 mm-diameter targets. At −12 dB channel SNR (i.e., the qualitatively determined detectability limit), the gCNR values ranged from 0.35-0.46 for DAS images of the 6 mm-diameter target. At higher channel SNR values, gCNR, contrast, CNR, and SNR decreased with increased target size, consistent with qualitative observations of Fig. 3.
Fig. 4. (a) Mean gCNR, contrast, CNR, and SNR with standard deviation error bars are shown as functions of channel SNR for 6 mm-, 8 mm-, and 10 mm-diameter simulated targets. Image quality metrics were calculated using DAS images. (b) The same image quality metrics shown for 6 mm-diameter simulated targets using SLSC images with varying short-lag values.
In addition to channel noise, another common cause of photoacoustic image quality degradation is limited receiver bandwidth. To determine the ability of gCNR to detect changes in bandwidth, Fig. 5 shows the simulated DAS photoacoustic images of a 6 mm-diameter target with varying limited detection bandwidth. In addition to showing the DAS image when the channel data was filtered with the full bandwidth of the transducer (i.e., 2-8 MHz), Fig. 5 also shows images when the channel data was bandpass filtered with a bandwidth 1 MHz, with either an upper cutoff frequency of 8 MHz or a lower cutoff frequency of 2 MHz. The displayed images are shown with a channel SNR of −10 dB, and the corresponding gCNR values are reported at the bottom of each image. The target was not visible with the 2 MHz cutoff frequency, which is quantified by the gCNR of 0.1. The target was qualitatively similar to the full bandwidth image when the cutoff frequency was 8 MHz, which is quantified by the minimal gCNR decrease when compared to the full bandwidth image.
Fig. 5. Simulated DAS photoacoustic images of a 6 mm-diameter target with varying receiver bandwidths and −10 dB channel SNR. The corresponding gCNR value is overlaid on each image.
Figure 6 shows contrast, CNR, SNR, and gCNR for the same target shown in Fig. 5, as a function of bandwidth when the bandpass filters shared either an upper cutoff frequency of 8 MHz or a lower cutoff frequency of 2 MHz. Similar to the qualitative observations shown in Fig. 5, an upper cutoff frequency of 8 MHz resulted in minimal changes to the image quality metrics. However, a lower cutoff frequency of 2 MHz resulted in image quality improvements as bandwidth increased, which was reflected in the higher gCNR values, as well as higher contrast, CNR, and SNR. Therefore, the gCNR metric is able to represent image quality degradations due to a limited bandwidth, which provides additional confidence that the gCNR is representative of photoacoustic image quality.
Fig. 6. Mean (a) contrast, (b) CNR, (c) SNR, and (d) gCNR with standard deviation error bars shown as a function of limited receiver bandwidth for the 6 mm-diameter simulated DAS targets with −10 dB channel SNR. Bandpass filters shared either an upper cutoff frequency of 8 MHz or a lower cutoff frequency of 2 MHz.
4.2 Application to simulated SLSC images
Figure 7 shows simulated SLSC photoacoustic images of a 6 mm-diameter target with 0 dB and −12 dB channel SNR (top and bottom of Fig. 7, respectively), displayed with $M=1$, $5$, and $10$, from left to right, respectively. The corresponding gCNR values are reported at the bottom of each image. Qualitative observations and the reported quantitative values at the bottom of each image in Fig. 7 demonstrate that target detection decreases as the short-lag valued $M$ increases.
Fig. 7. Simulated SLSC photoacoustic images with a target diameter of 6 mm and varying channel SNR (i.e., SNR$_{\textrm {ch}}$). Short-lag values of M = 1, M = 5, and M = 10 are displayed. The corresponding gCNR value is reported on each image.
Figure 4(b) shows the gCNR, contrast, CNR, and SNR measured from SLSC images of the simulated 6 mm-diameter target as functions of channel SNR. The short-lag value ranged from $M$=2 to $M$=10. Lower short-lag values consistently resulted in higher gCNR values and generally resulted in higher contrast, CNR, and SNR values. Although image SNR is generally expected to have a sigmoidal relationship with channel SNR that asymptotes to a constant image SNR at higher channel SNR values, an unexpected decrease in image SNR was observed at the higher channel SNR values (e.g., 20-40 dB) of the lower lag values (e.g., M = 2) in Fig. 4(b). This decrease was caused by an increase in the standard deviation of the amplitudes within the background ROI of SLSC images. Note that gCNR is more robust to this decrease.
In addition to $M$, another parameter known to influence the quality of SLSC images is the kernel size. Figure 8 shows simulated SLSC photoacoustic images of a 6 mm-diameter target with kernel size varied from 3$\lambda$ to 0.5$\lambda$, from left to right, respectively. The corresponding gCNR value is reported at the bottom of each image. Qualitatively, decreasing kernel size resulted in less visible targets, which is consistent with the gCNR values reported at the bottom of each image.
Fig. 8. Simulated SLSC images with $M=5$ are shown for the 6 mm-diameter target with varying kernel sizes. The channel SNR of the displayed simulated data is −12 dB.
Table 1 summarizes the corresponding contrast, CNR, and SNR for the simulated SLSC images with kernel sizes of 3$\lambda$, 2$\lambda$, $1\lambda$, and 0.5$\lambda$, a short-lag value of $M$=5, and a channel SNR of −12 dB. Similar to the qualitative observations and quantitative results reported in Fig. 8, decreasing the kernel size resulted in decreased gCNR. Contrast, CNR, and SNR, similarly decreased with kernel size, which confirms the degradation in image quality quantified with gCNR.
Table 1. Image quality metrics for simulated SLSC images of the 6 mm-diameter target with varying kernel size. Mean $\pm$ standard deviation are shown for each metric. The channel SNR was −12 dB. Data shown are for $M$=5.
4.3 Experimental validation
Figure 9 shows experimental DAS and SLSC photoacoustic images. Experimental tubing boundaries are shown in red, and images of the photoacoustic target are shown with short-lag values ranging from $M$=5 to $M$=20. The in vivo experimental SLSC results are shown for short-lag values ranging from $M$=1 to $M$=10. These $M$ values were empirically chosen to display the coherence information while maintaining target visibility, with consideration that the plastic tubing used in the experimental setup caused additional coherent reverberations that complicated target visibility at lower $M$ values. These choices do not affect our interpretation of relative differences in image quality. Qualitatively, both the experimental phantom and in vivo targets were more difficult to detect at higher $M$ values when compared to visualization at lower $M$ values, which is consistent with the observations reported in Section 4.2. There were additional coherent reverberations caused by the photoacoustic response from the laser interacting with the water medium surrounding the tubing. Due to these coherent signals in the axial direction, the measured gCNR with this experimental setup is primarily applicable to distinguishing the target from background in the lateral direction.
Fig. 9. DAS and SLSC images created from experimental phantom and in vivo data. The 6 mm-diameter target is displayed for the phantom experiment, and the tubing boundaries are outlined in red. SLSC images are displayed with $M$=5, $M$=10, and $M$=20 for phantom data and $M$=1, $M$=5, and $M$=10 for in vivo experiments.
Table 2 reports gCNR, contrast, CNR, and SNR for experimental phantom and in vivo SLSC images. Experimental phantom DAS image quality metrics were calculated for the 6 mm, 8 mm, and 10 mm targets, and the SLSC image quality metrics were calculated for the 6 mm target with $M$=5, $M$=10, and $M$=20. The in vivo SLSC image quality metrics were calculated with $M$=1, $M$=5, and $M$=10. Similar to the simulation results reported in Section 4.2, the experimental phantom and in vivo SLSC images resulted in gCNR, contrast, CNR, and SNR values that decreased with increased $M$. The reported SNR values for the experimental phantom SLSC data have a higher order of magnitude (i.e., 10$^{19}$) when compared to the other datasets due to the small standard deviation of the background ROI (i.e., 10$^{-20}$) caused by the thresholding applied to the negative values prior to image display. When SNR was instead calculated before thresholding and normalizing the image, the SNR ranged from 29.24 $\pm$ 1.76 to 20.00 $\pm$ 1.83 for $M$=5 through $M$=20, respectively.
Table 2. Mean $\pm$ standard deviation of image quality metrics of ten experimental phantom DAS and SLSC images, compared the image quality metrics of the in vivo result shown in Fig. 9.
Although we typically threshold SLSC images to remove negative values prior to image display (considering that the images in this paper are displayed on a log scale, the log of a negative number is undefined, and maintaining these negative values would otherwise bias contrast to lower values), Fig. 10 shows a more detailed analysis of the impact of thresholding on the traditional image quality metrics and on gCNR. Thresholding was applied to experimental phantom SLSC images created with $M$=5, 10, or 20. The minimum threshold values ranged from −0.2 to 0 (which is equivalent to altering the dynamic range when images are displayed on a linear scale). Although contrast, CNR, and SNR vary as a function of the chosen threshold, the gCNR metric remains relatively constant, regardless of the chosen threshold.
Fig. 10. (a) Contrast, (b) CNR, (c) SNR, and (d) gCNR as functions of the minimum selected threshold value of an experimental phantom SLSC image.
To investigate possible trends for a wider range of short-lag values, Fig. 11 displays gCNR as a function of $M$ for simulated, experimental phantom, and in vivo data. Considering that we observed a linear relationship when plotting these results, we performed linear regression on these datasets. The linear fit indicates that gCNR has a reasonably linear relationship with respect to $M$ for simulated data (R$^2$ = 0.88) and in vivo data (R$^2$ = 0.96). The corresponding R$^2$ value was 0.26 for the experimental phantom data, likely due to confounding coherent effects from the tubing and surrounding water medium as observed in Fig. 9. The linearity observed in Fig. 11 was not apparent when performing linear regression of contrast, CNR, and SNR as functions of $M$, resulting in R$^2$ values of 0.56, 0.84, and 0.73, respectively, for the simulated data, 0.14, 0.84, and 0.03, respectively, for the experimental phantom data, and 0.81, 0.73, and 0.65, respectively, for the in vivo data.
Fig. 11. Demonstration of the linearity of gCNR as a function of short-lag value, $M$, measured from SLSC images created with simulated data (−12 dB channel SNR, 0.85$\lambda$ kernel length, 6 mm-diameter target), experimental phantom data (6 mm-diameter target), and in vivo data. The points represent measurements and the line shows the linear regression result. The left and bottom axes correspond to simulated and in vivo results, while the right and top axes correspond to experimental phantom results.
As demonstrated in Fig. 4(a), simulated images were unable to achieve a gCNR of 1 with DAS beamforming. In order to evaluate the variability of the traditional image quality metrics when gCNR $\geq$ 0.95, Fig. 12 shows contrast, CNR, and SNR as functions of the gCNR measured in the SLSC images reported throughout this manuscript. There is no obvious correlation between gCNR and the three more traditional image quality metrics when gCNR $\geq$ 0.95. The low R$^2$ values of 0.01-0.26 for best-fit regression lines of the combined datasets confirm this observation. In addition, at these high gCNR values, contrast, CNR, and SNR varied by up to 23.7 dB, 2.0, and 26.5, respectively, for simulated data, 56.2 dB, 3.4, and 7.6$\times 10^{20}$, respectively, for experimental phantom data, and 43.3 dB, 3.4, and 517.3, respectively, for in vivo data. Therefore, these traditional metrics can experience large variations when a target is fully detectable (i.e., gCNR = 1), and additional increases in these values would have no impact on photoacoustic target detectability, which is an important consideration for future beamformer development.
Fig. 12. Combined contrast, CNR, and SNR of the simulation, experimental phantom, and in vivo SLSC images reported throughout this manuscript with gCNR $\geq$ 0.95. Contrast and SNR scale adjustments and breaks were implemented and the experimental phantom results with SNR $>$ 2$\times$10$^3$ were omitted to simplify plot readability. Note the large variation in contrast, CNR, and SNR when gCNR $\geq$ 0.95.
5. Discussion
This paper highlights the use of gCNR as a new metric to quantitatively assess the detectability of a target in photoacoustic images. This observation is supported by simulated, experimental phantom, and in vivo photoacoustic data. Overall, photoacoustic images with qualitatively visible targets corresponded with high gCNR values, indicating a high probability of target detection. There are four key benefits to using gCNR to assess photoacoustic image quality. First, because the magnitude of gCNR is limited to values between 0 and 1, gCNR has the ability to standardize our expectations of the quality of a photoacoustic image. Second, the magnitude of gCNR directly corresponds to the separability of the target from the background, whereas the magnitude of contrast, CNR, and SNR can be improved without improving overall target detectability (see Fig. 12). Although Fig. 4 demonstrates that these metrics experience an improvement as channel SNR increases, the improvement does not occur at the same rate nor with the same relative increases in magnitude, which further highlights the advantage of relating image quality to a probabilistic principle such as gCNR, which is applicable across a wide range of modern beamformers (unlike CNR). Third, changes in gCNR consider image degradation caused by common photoacoustic artifacts, such as artifacts associated with limited bandwidth (Figs. 5–6). Fourth, gCNR is not sensitive to common photoacoustic image post-processing methods (e.g., thresholding as shown in Fig. 10). These benefits represent four important details to consider when designing and assessing new photoacoustic beamformers.
The effectiveness of gCNR was demonstrated using both DAS and SLSC beamformers. Focusing on the benefits obtained with the application of gCNR to SLSC images, when adjusting SLSC beamforming parameters, gCNR values were consistent with our expectations. Specifically, higher $M$ values are associated with increased lateral resolution [21–23], and these higher $M$ values therefore increase the variance of the measured coherence estimate, resulting in an increase in the variance of both the target and the background. We expected this increased resolution and variance to reduce gCNR values, which is supported by the qualitative and quantitative observations reported in Sections 4.2–4.3. Similarly, decreasing kernel size is expected to increase axial resolution and variance, resulting in less qualitatively detectable images. Therefore, we similarly expected lower gCNR values with smaller kernel sizes, which is supported by the qualitative and quantitative observations reported in Section 4.2. Lower $M$ values and larger correlation kernel sizes generally produce smoother images with poorer spatial resolution, and this smoothing is thought to be responsible for more easily identifiable targets and the higher gCNR values of these SLSC images.
Although decreasing $M$ increased the value of the three more traditional metrics, gCNR has a more linear relationship as a function of $M$, as illustrated in Fig. 11. This linearity was observed for multiple values of channel SNR (see Fig. 8). This linearity points to yet another potential benefit of gCNR for standardization of image quality interpretation, as it can potentially be used to both predict target visibility given a set of input parameters, as well as predict beamforming parameters (e.g., $M$ values) needed to increase target detectability. Future studies will explore the use of this linear relationship for target visibility predictions. Similar to the benefits obtained with the application of gCNR to SLSC images, other advanced, nonlinear beamformers have the potential to benefit as well.
One limitation of the discrete gCNR implementation is sensitivity to the number of selected bins (i.e. $N$ in Eq. (3)). In addition, gCNR is not recommended as a standalone metric to determine photoacoustic image quality. For example, previous work indicates that increasing the $M$ value in SLSC images leads to an improvement in resolution [21,24]. When applying this principle to the experimental phantom data in Fig. 9, we observe an improvement in the resolution of the SLSC image as $M$ increases from 5 to 20. However, Table 2 reports a reduction in gCNR as $M$ increases. Therefore, while gCNR offers multiple advantages over traditional metrics such as contrast, CNR, and SNR, other criteria such as resolution must also be considered when determining the overall quality of photoacoustic images. Nonetheless, in tandem with other photoacoustic image quality metrics, gCNR has promising potential to provide additional insight, particularly when designing new beamformers and image formation techniques or when reporting quantitative performance values without the opportunity to qualitatively assess corresponding images, such as in text-only abstracts.
6. Conclusion
We propose gCNR as an improvement over the more traditional metrics of contrast, CNR, and SNR when investigating photoacoustic target visibility and detectability. This improvement was confirmed with simulated, phantom, and in vivo data and with two beamformers. When comparing across beamformers, gCNR was successfully demonstrated as a method to assess the meaningfulness of large or improved contrast, CNR, or SNR with regard to target detectability. We also demonstrated that gCNR is not directly correlated with any of these three traditional metrics in most cases. With a defined range of 0 to 1, gCNR has the potential to be more meaningful than traditional metrics, particularly in cases where it is necessary to report quantitative performance values without the opportunity to qualitatively assess corresponding images (e.g., in text-only abstracts). The detailed assessment of the applicability of gCNR to photoacoustic images provided in this paper newly exposes the photoacoustic community to an additional tool to quantify image quality.
Funding
National Science Foundation (DGE1746891, ECCS-1751522); National Institute of General Medical Sciences (2T32GM007057-44).
Disclosures
The authors declare no conflicts of interest.
References
1. O. M. H. Rindal, A. Austeng, A. Fatemi, and A. Rodriguez-Molares, “The effect of dynamic range alterations in the estimation of contrast,” IEEE Trans. Ultrason. Ferroelect. Freq. Contr. 66(7), 1198–1208 (2019). [CrossRef]
2. M. Patterson and F. Foster, “The improvement and quantitative assessment of B-mode images produced by an annular array/cone hybrid,” Ultrason. Imaging 5(3), 195–213 (1983). [CrossRef]
3. S. W. Smith, R. F. Wagner, J. M. Sandrik, and H. Lopez, “Low contrast detectability and contrast/detail analysis in medical ultrasound,” IEEE Trans. Sonics Ultrason. 30(3), 164–173 (1983). [CrossRef]
4. A. Rodriguez-Molares, O. M. H. Rindal, J. D’hooge, S.-E. Måsøy, A. Austeng, M. A. L. Bell, and H. Torp, “The generalized contrast-to-noise ratio: a formal definition for lesion detectability,” IEEE Trans. Ultrason. Ferroelect. Freq. Contr. 67(4), 745–759 (2020). [CrossRef]
5. R. Bouchard, O. Sahin, and S. Emelianov, “Ultrasound-guided photoacoustic imaging: Current state and future development,” IEEE Trans. Ultrason. Ferroelect. Freq. Contr. 61(3), 450–466 (2014). [CrossRef]
6. M. Xu and L. V. Wang, “Photoacoustic imaging in biomedicine,” Rev. Sci. Instrum. 77(4), 041101 (2006). [CrossRef]
7. P. Beard, “Biomedical photoacoustic imaging,” Interface Focus 1(4), 602–631 (2011). [CrossRef]
8. C. Kim, K. H. Song, F. Gao, and L. V. Wang, “Sentinel lymph nodes and lymphatic vessels: noninvasive dual-modality in vivo mapping by using indocyanine green in rats–volumetric spectroscopic photoacoustic imaging and planar fluorescence imaging,” Radiology 255(2), 442–450 (2010). [CrossRef]
9. B. Eddins and M. A. L. Bell, “Design of a multifiber light delivery system for photoacoustic-guided surgery,” J. Biomed. Opt. 22(4), 041011 (2017). [CrossRef]
10. D. Piras, C. Grijsen, P. Schutte, W. Steenbergen, and S. Manohar, “Photoacoustic needle: minimally invasive guidance to biopsy,” J. Biomed. Opt. 18(7), 070502 (2013). [CrossRef]
11. W. Xia, M. Kuniyil Ajith Singh, E. Maneas, N. Sato, Y. Shigeta, T. Agano, S. Ourselin, S. J West, and A. E Desjardins, “Handheld real-time led-based photoacoustic and ultrasound imaging system for accurate visualization of clinical metal needles and superficial vasculature to guide minimally invasive procedures,” Sensors 18(5), 1394 (2018). [CrossRef]
12. J. L. Su, B. Wang, K. E. Wilson, C. L. Bayer, Y.-S. Chen, S. Kim, K. A. Homan, and S. Y. Emelianov, “Advances in clinical and biomedical applications of photoacoustic imaging,” Expert Opin. Med. Diagn. 4(6), 497–510 (2010). [CrossRef]
13. G. F. Lungu, M. L. Li, X. Xie, L. V. Wang, and G. Stoica, “In vivo imaging and characterization of hypoxia-induced neovascularization and tumor invasion,” Int. J. Oncol. 30(1), 45–54 (2007). [CrossRef]
14. Y. Lao, D. Xing, S. Yang, and L. Xiang, “Noninvasive photoacoustic imaging of the developing vasculature during early tumor growth,” Phys. Med. Biol. 53(15), 4203–4212 (2008). [CrossRef]
15. B. E. Treeby and B. T. Cox, “k-wave: Matlab toolbox for the simulation and reconstruction of photoacoustic wave fields,” J. Biomed. Opt. 15(2), 021314 (2010). [CrossRef]
16. L. R. Dixon, “The complete blood count: physiologic basis and clinical usage,” J. Perinat. Neonat. Nur. 11(3), 1–18 (1997). [CrossRef]
17. M. Turgeon, Clinical Hematology: Theory and Procedure (Lippincott Williams & Wilkinson, 2004).
18. K. M. Kempski, A. Wiacek, M. Graham, E. Gonzalez, B. Goodson, D. Allman, J. Palmer, H. Hou, S. Beck, J. He, and M. A. L. Bell, “In vivo photoacoustic imaging of major blood vessels in the pancreas and liver during surgery,” J. Biomed. Opt. 24(12), 1 (2019). [CrossRef]
19. M. A. Lediju, G. E. Trahey, B. C. Byram, and J. J. Dahl, “Short-lag spatial coherence of backscattered echoes: imaging characteristics,” IEEE Trans. Ultrason. Ferroelect. Freq. Contr. 58(7), 1377–1388 (2011). [CrossRef]
20. B. Pourebrahimi, S. Yoon, D. Dopsa, and M. C. Kolios, “Improving the quality of photoacoustic images using the short-lag spatial coherence imaging technique,” in Photons Plus Ultrasound: Imaging and Sensing 2013, vol. 8581 (International Society for Optics and Photonics, 2013), p. 85813Y.
21. M. A. L. Bell, N. Kuo, D. Y. Song, and E. M. Boctor, “Short-lag spatial coherence beamforming of photoacoustic images for enhanced visualization of prostate brachytherapy seeds,” Biomed. Opt. Express 4(10), 1964–1977 (2013). [CrossRef]
22. M. T. Graham and M. A. L. Bell, “Development and validation of a short-lag spatial coherence theory for photoacoustic imaging,” in Photons Plus Ultrasound: Imaging and Sensing 2018, vol. 10494 (International Society for Optics and Photonics, 2018), p. 104945K.
23. M. A. L. Bell, N. P. Kuo, D. Y. Song, J. U. Kang, and E. M. Boctor, “In vivo visualization of prostate brachytherapy seeds with photoacoustic imaging,” J. Biomed. Opt. 19(12), 126011 (2014). [CrossRef]
24. M. A. L. Bell, J. J. Dahl, and G. E. Trahey, “Resolution and brightness characteristics of short-lag spatial coherence (SLSC) images,” IEEE Trans. Ultrason. Ferroelect. Freq. Contr. 62(7), 1265–1276 (2015). [CrossRef]
