1. Introduction

Laser speckle contrast imaging (LSCI) [1] provides label-free, high resolution and real-time imaging of blood flow and tissue perfusion [2]. The recent theoretical and hardware developments [3–5] lead more potentials in the clinical applications, e.g. fundus imaging [6], neurosurgery monitoring [7] and endoscopic imaging [8]. As a purely 2D imaging technique, traditional LSCI cannot resolve the blood flow in different depths. For fundus imaging and neurosurgery monitoring, the imaging field of view (FOV) is clearly accessible and stably fixed, thus direct translation of laboratory LSCI methods fulfills the clinical requirements. The depth-resolved physiological and pathological information is critical for clinical diagnostics and therapy assistance. LSCI has been combined with a confocal line-scan strategy to produce depth-resolved absorption information, augmenting relative blood flow map [9,10]. However, there are more challenges in developing the LSCI-based endoscope due to the complex scattering and inevitable tissue motions in abdominal environments and/or inside the cavities.

The current clinical laparoscopic system uses the ICG-based fluorescent method to visualize the large vasculature and evaluate the tissue perfusion [11]. Although the fluorescent-based method has worked well in abdominal surgeries, the ICG injection increases the complexity of the operation and thus is not possible for continuously imaging [12]. When the ICG is used as the indicator for tumor boundaries and sentinel lymph nodes [13], it cannot be re-injected for vessel visualization and tissue perfusion evaluation during the surgery. Furthermore, the estimate of flow using ICG may be distorted by the concentration differences of the contrast agent in the blood plasma rather than a change in flow (i.e. in ischemic areas) [14]. Recently, imaging photoplethysmography (iPPG) method has been applied in gastrointestinal surgery for label-free monitoring [15,16]. iPPG uses the incoherent green light illumination and estimates the tissue blood volume intraoperatively [17,18]. However, iPPG interferes with continuous white-light imaging in laparoscopic surgery. Therefore, LSCI provides new modality for real-time and label-free intraoperative visualizing the vasculature and evaluating the tissue perfusion in laparoscopic surgery.

LSCI has been applied in open surgical settings, e.g. brain surgery [19], abdominal surgery [20,21]. Recently, Ambrus et al. performed the clinical evaluations of LSCI for gastric microcirculation during esophagectomy [22]. All these studies validate the clinical effectiveness of LSCI. There are also previous studies attempting to merge the LSCI into endoscopic or laparoscope systems. The LSCI-enabled endoscopic systems were firstly applied for animal studies, e.g. the blood flow in rabbit knee [23] and rodent retina [24]. Bray et al. utilized the standard endoscope with an additional laser source and contrast analysis to visualize the blood flow in human knee surgery [8]. Because the standard endoscope has a single RGB camera, light sources were manually switched to obtain the white light video or LSCI contrast images. Heeman et al. performed a two-center clinical study using a standard laparoscope showing that LSCI was capable of detecting ischemic areas on the large intestine during gastrointestinal surgery [25]. This single-camera limitation has been overcome by Zheng et al. utilizing a dual-camera system (one RGB camera and one NIR camera) in the laparoscope [26]. Both the color monitoring video and the LSCI video can be displayed simultaneously during the surgery. However, the dual-camera design made the hand-held part inconvenient for surgeon operations.

Traditional LSCI utilizes the contrast parameter to analyze the flow motion-induced blur in the dynamically back-scattered speckle images. For random medium, we recently developed a random matrix-based strategy to separate the single and multiple scattering components in the dynamically back-scattered speckle images [27]. This method has been validated and applied in the LSCI for in-vivo animal cerebral blood flow (CBF) imaging in the laboratory environment [28].

In this study, we develop and validate the random matrix-based LSCI method (rmLSCI) for clinical laparoscopic surgery. To optimize the imaging ability, we further design a single-camera laparoscope system with integrated dual CMOS sensors. The new system provides simultaneous outputs of rmLSCI contrast images (single and multiple scattering) and white light video. The blood flow and vasculature structure in both supra-diaphragm and infra-diaphragm can be separated simultaneously in the pre-clinical swine’s laparoscopic surgery experiment. We also demonstrate the benefits of quasi-3D ability in assistance of occlusion operation. The proposed method provides a powerful tool in laparoscopic surgery and other clinical applications.

2. Theory and laparoscopy system

2.1 Random matrix-based reconstruction theory

The coherent light produces the speckle phenomenon when transmitting through the random medium due to the interference among different propagating paths. The changes in speckle intensities contain information on spatiotemporal correlations in dynamically backscattered speckle images. The rmLSCI utilizes the random matrix description of dynamic speckle images under wide-field coherent illumination. The Brownian motions inside the random medium dynamically perturb the trajectories of coherent wave propagation and thus provide a unique separation mechanism based on the eigenvalue density distribution of the hybrid Wishart RM. The full theoretical development and validation can be found in [27]. Here we summarize the main construction strategy and separation procedure used in this study.

For each $T$ recorded speckle images $\left \{I_{i} \mid i=1 \cdots T\right \}$, we firstly reshape the $I_{i}$ to a column vector $h_{i}$ with entries $h_{i}[n], n=1 \cdots N, N=N_{1} \times N_{2}$ as the $\mathrm {i}^{\text {th }}$ column and construct the $N \times T$ temporal intensity random matrix $R$. The $R$ matrix contains both the single and multiple scattering components: $R=R_{S}+R_{M}$. We further centralize the $R$ by $\hat {R}=R-\bar {R}$ where each entry in $\bar {R}$ is the mean value of the same row. We have $\hat {R}=$ $\hat {R}_{S}+\hat {R}_{M}$. The subscripts $S$ and $M$ denote the single and multiple scattering components hereafter, respectively.

For the Wishart random matrix $E=\hat {R} \hat {R}^{\prime }$, the eigenvalues $\{s(i)\}$ with $i=1 \cdots N$ and $s(1) \geq s(2) \geq \cdots \geq s(N)$ have probability densities as $\rho (s) \triangleq \frac {1}{N} \sum _{i=1}^{N} \delta \left (s-s(i)\right )$. The entries in $\hat {R}_{M}$ come from the independent and identically distributed random variables following the Gaussian distribution $\left (\sim \mathbb {N}\left (0, \sigma _{M}^{2}\right )\right )$ and thus the eigenvalue density of $E_{M}$ is fully described by the Marčenko-Pastur law with $T \geq N$ and $N,T \rightarrow \infty$ [29]:

Although the column-wise strategy is applied in the construction of $R$, other strategies (e.g. row-wise) output the same eigenvalues of $E$ due to the orthogonality of raw permuting operations [30]. When the eigenvalues density of $E_{M}$ has finite $4^{\text {th }}$ moments and $T \rightarrow \infty$, we have the following convergence about the maximum and minimum eigenvalues $s_{M}(1){\longrightarrow } s_{M+}$ [31] and $s_{M}(N){\longrightarrow } s_{M-}$ [32]. In practice, the eigenvalues of the $E$ can be modeled as a biased version from that of $E_{M}$ due to the single scattering components. However, the low-rank characteristic of the single scattering component bounded its bias effects in the larger eigenvalues of $E$. The eigenvalues of $E_{M}$ and $E$ converge to each other as the eigenvalue decreased to the minimum one. Thus we can use the minimal eigenvalue of $E$, i.e. $s(N)$, to estimate the $\sigma _{M}^{2}$ [33]:

Applying the trace relation $\operatorname {tr}\left (\tilde {R} \tilde {R}^{\prime }\right )=\operatorname {tr}\left (\tilde {R}_{S} \tilde {R}_{S}^{\prime }\right )+2 \operatorname {tr}\left (\tilde {R}_{S} \tilde {R}_{M}^{\prime }\right )+\operatorname {tr}\left (\tilde {R}_{M} \tilde {R}_{M}^{\prime }\right )$ and the sampling variance $\tilde {\sigma }^{2}=\operatorname {tr}\left (\tilde {R} \tilde {R}^{\prime }\right ) / N T, \tilde {\sigma }_{S}^{2}=$ $\operatorname {tr}\left (\tilde {R}_{S} \tilde {R}_{S}^{\prime }\right ) / N T, \tilde {\sigma }_{M}^{2}=\operatorname {tr}\left (\tilde {R}_{M} \tilde {R}_{M}^{\prime }\right ) / N T$, we have $\tilde {\sigma }_{M}^{2} \rightarrow \sigma _{M}^{2}$ and $\tilde {R}_{M}[n,t] \sim N\left (0, \sigma _{M}^{2}\right )$. Then, the commutative part $\operatorname {tr}\left (\tilde {R}_{S} \tilde {R}_{M}^{\prime }\right ) / M T \sim \mathbb {N} \left (0, \tilde {\sigma }^{2} / N T\right ) \approx 0$. Finally, we have $\tilde {\sigma }_{S}^{2} \approx \tilde {\sigma }^{2}-\tilde {\sigma }_{M}^{2}$ as an estimation of $\sigma _{S}$. After obtaining the $\sigma _{S}$, the corresponding mean intensity $\mu _{S}$ is obtained since exponential distribution has $\mu _{S}=\sigma _{S}$ [27]. Finally, the mean intensity $\mu _{M}=\mu -\mu _{S}$.

In LSCI, the coherent scattering from red blood cells (RBCs) forms the speckle patterns. The motion of RBCs is the major source leading the speckle decorrelation (blurring effect) [34]. In practice, we calculate the contrast values $K=\sigma / \mu$ [1] to quantify the blurring effect of ordered flow. The relative blood flow velocity $v$ can thus be estimated based on the well-established auto-correlation model $K^2=\beta [exp(-2x)-1+2x]/(2x^2)$ [35] with $x=T_{exp}/\tau _c \propto v$, the exposure time $T_{exp}$ and the coherence factor $\beta$. The in-vivo validation of LSCI-based blood flow imaging has been performed using sidestream dark field flowmetry [36]. This is also adapted in rmLSCI for multiple scattering contrast image calculation $K_{M}=\sigma _{M} / \mu _{M}$. For the single scattering component, there is no need for normalization by the estimated $\mu _{S}$. Both the electric field Monte Carlo simulation [28] and in-vitro experiment (Fig. 2(b)) confirmed that relative blood flow velocity $v$ can be directly calculated from $\sigma _S$ based on the above auto-correlation model. Thus, we define $K_S \triangleq \sigma _S$ as the blood flow indicator in the following analysis.

2.2 Laparoscopy system

The prototype laparoscopy system (Fig. 1) is composed of a rigid laparoscope, a single camera with dual CMOS sensors, light sources, an image acquisition and processing unit, and two medical display monitors. The rigid laparoscope (HOPKINS rod lens telescope HD3, Henke-Sass Wolf GmbH., Germany) has a $10 mm$ diameter and is attached to an objective lens (OL) adaptor ($14 mm$ focal length). The OL adaptor provides manual focus adjustment under different working distances. In the custom-designed dual-CMOS camera, a longpass dichroic filter ($730nm$) spectrally separates light by transmitting the infrared band ($750 - 1800 nm$) and reflecting the visible band ($400 - 690 nm$). The reflected light is captured by a color CMOS sensor (IMX290LQR-C, Sony, Tokyo, Japan) with $1920\times 1080$ pixels and $60 fps$. The transmitted light is further filtered by a band-pass filter ($830 nm \pm 0.4 nm$, OD5) and recorded by the monochromatic camera sensor (IMX290LLR-C, Sony, Tokyo, Japan) with the same pixel resolution and frame rate. To minimize the input power of the near-infrared laser, this monochromatic sensor is selected due to its high quantum efficiency ($\sim 60\%$) near $830 nm$. The total weight of the camera and laparoscope is less than $500 g$.

 

Fig. 1. The Laparoscopy system developed in this study. The system has five major subsystems: rigid laparoscope, dual-CMOS camera, light sources, image acquisition and processing unit, and medical display monitors.

Download Full Size | PDF

The camera is controlled by the image acquisition and processing unit through a GIGE connection. The color video is captured and processed by the camera control unit and projected on the medical display monitor using an HDMI connection. The raw data from the monochromatic sensor are transferred to a GPU board (Xviaver-AGX, Nvidia Inc., USA) for parallel and real-time calculation of the single and multiple scattering contrast images based on rmLSCI method. The calculated contrast images are converted to blood flow index (BFI) values and visualized on the medical display monitor with pseudo-color mapping. A press button on the handle is used to switch the single scattering and multiple scattering contrast images on the monitor. In this study, the jet mapping adapted from the traditional LSCI is applied for animal CBF imaging. To further improve the blood flow and tissue perfusion visualization, parula mapping is used in laparoscopic surgery based on the surgeon’s suggestion.

The standard and commercial light sources, i.e. Xenon 300 (with visible light band-pass filter) and Laser Diode ($650mW$, $830nm$, LD830-SE650, Thorlabs Inc., USA), are used in the laparoscopy system. The two light sources are coupled into the integrated light guide (Henke-Sass Wolf GmbH., Germany) through a ’Y’ type fiber bundle. The resulting maximum illumination powers of visible light and laser light are $5000 lux$ and $200 mW$ respectively in CW modes.

3. Experiments and data processing

3.1 In-vitro blood flow phantom experiments

We further used the in-vitro flow phantom experiment to test the flow velocity estimation accuracy and linearity using the contrast values obtained from the separated single and multiple scattering components. The intralipid (IL) (Kabivitrum Inc., USA) solution was used as the Mie scattering random medium. The mean diameter of lipid droplets was $0.7 \mu \mathrm {m}$, and its reduced scattering coefficient $\mu '_{\mathrm {s}}$ at $830 \mathrm {~nm}$ was approximately $1.91 \mathrm {~mm}^{-1}$ at 2% concentration. The mean free path (MFP) was $0.2 \mathrm {~mm}$ at 2% concentration. We used a motorized pump syringe to control the flow of IL inside a polyethylene tube (PE-50, outer diameter: $0.97 \mathrm {~mm}$; inner diameter: $0.58 \mathrm {~mm}$ ) with different velocity $(2 \mathrm {~mm} / \mathrm {s} \sim 10 \mathrm {~mm} / \mathrm {s})$.

The laparoscope was focused at the surface of the phantom but with different working distances ($1cm$, $5cm$, $8cm$). For each working distance, the backscattered speckle images were captured by the IR channel and further processed to obtain the single and multiple scattering contrast images respectively. The optimized window sizes ($3\times 3$ pixels in spatial and $20$ frames in temporal) are applied in rmLSCI processing. The average velocities covering the center area of the tube are estimated from the contrast values and compared with the true velocities.

3.2 In-vivo rat’s cerebral blood flow imaging

All animal experimental procedures were performed using protocols approved by the Animal Care and Use Committee of Shanghai Jiao Tong University. Since the rat’s CBF is one of the most successful imaging applications of LSCI, we first tested our prototype laparoscopy system in imaging rat’s CBF. It is also as a test for open surgery applications. The adult Sprague Dawley rat $(\sim 250 \mathrm {~g}$, female) was inhaled anesthetic with Isoflurane ($4\%$ for induction and $2\%$ for maintenance) and fixed in a stereotactic frame (David Kopf Instruments, Tujunga, CA, USA). A homeothermic blanket system was used to maintain the rectal temperature of rats at $37^{\circ } \mathrm {C}$. After a midline incision on the scalp, using a high-speed dental drill (Fine Science Tools Inc. North Vancouver, Canada) to thin a $5 \mathrm {~mm} \times 5 \mathrm {~mm}$ area centered at $3.5 \mathrm {~mm}$ lateral to and $3 \mathrm {~mm}$ posterior to the Bregma.

The laparoscopic system is used to acquire the backscattered speckle images at different working distances ($1cm$, $5cm$, $8cm$). For each working distance, a total of 30 speckle images were recorded. The contrast images of single and multiple scattering components were calculated respectively. The traditional tLASCA algorithm was also applied to the recorded speckle images to obtain the hybrid contrast image. To compare with the standard laboratory LSCI setup, the same rat was imaged by a 12-bit cooled monochrome CCD camera (acA1300-60gmNIR, Basler AG, Germany) with a $60 \mathrm {~mm}$ f/2.8 macro lens (Nikon Inc., Melville, NY, USA). The standard LSCI system illuminates the imaging area using the same $830nm$ laser light source (beam was expanded by a diffuser) and records the laser speckle images ($1280 \times 1024$ pixels) in $60 \mathrm {fps}$.

3.3 Pre-clinical swine’s laparoscopic surgery

To demonstrate the quasi-3D blood flow imaging ability using our laparoscopic system and rmLSCI method, we performed the pre-clinical swine laparoscopic surgery experiment. The standard operations in laparoscopic pancreaticoduodenectomy surgery were used in the experiment. Both white light video and rmLSCI contrast images are recorded at key steps. The key steps of the surgery were summarized as follows:

(1). Exploration of the abdominal cavity: the diaphragm and inferior phrenic vessels, hepatoduodenal ligament (hilum), and mesentery were exposed.

(2). Harmonic scalpel was used to perform the Kocher maneuver to expose the pancreatic head and duodenum from their posterior wall, underlying were the vena cava and right kidney, and the treitz ligament was dissected.

(3). Dissected the pancreatic neck at the level of the superior mesenteric vein (SMV), the branches of the uncinate process were dissected from the SMV and superior mesenteric artery (SMA), the lymph nodes and adipose tissue around the SMV and SMA were dissected and the vessels were clearly exposed.

(4). The upper part of the pancreatic head was separated, and the lymph nodes and adipose tissue around the common hepatic artery, the gastroduodenal artery (GDA), and the proper hepatic artery were carefully dissected. The portal vein and the bile duct were also skeletonized, the gall bladder was resected and the small branches in the hilum were dissected and ligated, including the gall bladder artery, cystic duct, right gastric artery, gastric coronary vein, etc. The proximal part of the jejunum and distal part of the stomach was dissected by surgical staplers.

(5). After the process described above, the lymph nodes and adipose tissue around the vessels were dissected and the main blood flow of these tissues still remained. We ligated the GDA and occluded the portal vein, and then observe the hepatic tissue, mesentery, jejunum, and stomach.

4. Results

4.1 Validation using in-vitro blood flow phantom

Our laparoscope system provides the illumination of white light and laser light simultaneously. The returned white light and laser light are gathered by the same endoscopic optics and finally separated by the dichroic beamsplitter into the RGB video stream and LSCI contrast video stream. Using the same light-collecting optics may introduce non-linearity and estimation errors. The corresponding non-linearity and estimation errors can be different in single and multiple scattering components or under varying working distances.

To validate the estimation linearity and accuracy for blood flow imaging, we performed the in-vitro flow phantom experiment (Fig. 2(a)). Fig. 2(b) shows the relation between the estimated flow velocity (single or multiple scattering components) and the true flow velocity. The current system works well for a working distance of less than $10cm$. For the $5cm$ working distance (the circles and disks in Fig. 2(b)), both the single and multiple scattering components have sufficiently high linearities (adjusted $R^2=0.976$ and $R^2=0.993$) and low estimation errors ($RMSE=0.458$ and $RMSE=0.292$). Shorter working distance, e.g. $1cm$ (the dots and stars in Fig. 2(b)), slightly degenerate the flow estimation linearity and accuracy in both single (adjusted $R^2=0.958$, $RMSE=0.71$) and multiple (adjusted $R^2=0.988$, $RMSE=0.362$) scattering components. Longer working distance, e.g. $8cm$, reduces the flow estimation linearity in single (adjusted $R^2=0.963$, $RMSE=0.726$) and multiple (adjusted $R^2=0.989$, $RMSE=0.346$) scattering components due to the spatial averaging effect in the single pixel area. For a working distance longer than $20cm$, the averaging effect in a single pixel entirely destroys the speckle intensity fluctuations and thus violates the working principle of LSCI.

 

Fig. 2. The in-vitro flow phantom imaging experiment (a) and the velocity estimated using the rmLSCI method under different working distances ($1cm$, $5cm$ and $8cm$).

Download Full Size | PDF

4.2 Evaluation in in-vivo rat’s cerebral blood flow imaging experiment

We further evaluate the imaging performance of our laparoscope system and compared it with the standard laboratory LSCI system under the open surgery settings. We use the rat’s CBF with thinned skull preparation as the imaging subject. Fig. 3(a) and (b) show the focused white light image and contrast image (Jet pseudo-color mapping is applied) with working distance $5cm$. The resulting spatial resolution is too low to resolve the fine vasculature in the rat’s brain. A shorter working distance, e.g. $1cm$, improves the spatial resolution and thus reveals the vasculature details (Fig. 3(c)). In LSCI contrast images, the tissue area (thinned skull) always demonstrates a higher perfusion index than the skull area. Meanwhile, the standard laboratory LSCI system also outputs a good contrast image with vasculature details in high spatial resolution (Fig. 3(d)).

 

Fig. 3. The white light image (a) and enlarged pseudo-color contrast image (b) simultaneously obtained by the laparoscope system in imaging the in-vivo rat’s cerebral blood flow with $5cm$ working distance. (c) A shorter working distance ($1cm$) outputs a contrast image with high-resolution vasculature details. (d) The contrast image was obtained using a standard laboratory LSCI system. The contrast images presented in this figure are directly calculated using the traditional tLASCA method.

Download Full Size | PDF

Although the two systems reveal the same large vasculature structures, e.g. black arrows in Fig. 3(c) and (d), some small vasculature details are lost (black circles in Fig. 3(c) and (d)). This is due to the restricted light collection optics in the laparoscope compared with the macro-lens used in the standard laboratory LSCI system. Furthermore, there are inevitably strong specular points or areas in the hybrid contrast image from the laparoscope system (white circle in Fig. 3(c)). The standard laboratory LSCI system usually has no interruptions by these artifacts (white circle in Fig. 3(d)). These strong specular artifacts may significantly limit the applicability of LSCI in the surgery procedure.

The proposed rmLSCI separates the single and the multiple scattering contrast images corresponding to the superficial (Fig. 4(a)) and deeper (Fig. 4(d)) layers of the imaging areas. Although the single scattering contrast images still suffer from the specular artifacts, the multiple scattering contrast images are immune to the specular artifacts. Furthermore, a single scattering contrast image provides better visualization of vasculature details. This is because the majority of the vasculatures in the imaging area are located in the superficial layer. Compared with longer working distance (Fig. 4(b, c and e, f)), shorter working distance always produces better visualization of vasculatures (Fig. 4(a $\sim$ d)).

 

Fig. 4. The single scattering and multiple scattering contrast images obtained in the in-vivo rat’s CBF imaging using the rmLSCI method. (a,b,c) show the single scattering contrast images under $1cm$, $5cm$ and $8cm$ working distances. (d,e,f) show the corresponding multiple scattering contrast images.

Download Full Size | PDF

4.3 Applications in pre-clinical swine’s laparoscopic surgery

To demonstrate the quasi-3D imaging ability, we perform the pre-clinical swine’s laparoscopic surgery using our laparoscope system (Fig. 5(a)) and rmLSCI method. The dual modalities, i.e. white light and contrast videos, display in real-time during the whole surgery procedure. In this pre-clinical swine’s laparoscopic surgery experiment, the region of the right inferior phrenic vein can be clearly displayed in white light imaging (Fig. 5(b)). The rmLSCI method statistically separates the single and multiple scattering components and outputs the corresponding contrast images (Fig. 5(c) and (d)). Here parula mapping is used to facilitate the blood flow and tissue perfusion visualization during surgical operations. Based on the anatomy in (b), Fig. 5(c) demonstrates the superficial vessels and blood flow of the diaphragm. While Fig. 5(d) shows the deep vessels and blood flow in the infra-diaphragm region.

 

Fig. 5. (a) Pre-clinical swine’s laparoscopic surgery experiment. Our laparoscopic imaging system provides both the white light and rmLSCI in real-time. (b) White light imaging of the right inferior phrenic vein. (c) The supra-diaphragm tissue and vessels are extracted in the single scattering contrast image. (d) The infra-diaphragm tissue and vessels are demonstrated in the multiple scattering contrast image. Please note that parula mapping is used to facilitate blood flow and tissue perfusion visualization during surgical operations.

Download Full Size | PDF

Single scattering contrast image reveals the detailed vasculatures and corresponding blood flow in the diaphragm (white arrows in Fig. 5(c)). The right inferior phrenic vein under the diaphragm has been successfully isolated in the multiple scattering contrast image which demonstrates high blood flow velocity (white arrow in Fig. 5(d)). The small vasculatures under the diaphragm are also identified in the multiple scattering contrast image. The white circle areas in Fig. 5(c) and (d) demonstrate significant differences in the tissue perfusion in the diaphragm and under the diaphragm.

As a real-time imaging method, LSCI provides intraoperative monitoring and evaluation tool for blood vessel occlusion and tissue anastomosis. Jejunum mesentery was selected as the target region for testing. In normal physiological conditions without any surgical intervention (Fig. 6(a)), the vasculatures in superficial and deep mesenteric tissue were clearly demonstrated in the single scattering and multiple scattering contrast images (Fig. 6(c) and (e)). The tissue perfusion in deep mesenteric tissue was relatively homogeneous while superficial mesenteric tissue showed higher perfusion in the distal areas (white circles in Fig. 6(c)).

 

Fig. 6. Jejunum mesentery imaging during the occlusion operation in the pre-clinical swine’s laparoscopic surgery experiment. (a) and (b) are contrast images, single scattering contrast images, and multiple scattering contrast images before the occlusion. (c) and (d) are corresponding single scattering contrast reconstructed by the rmLSCI method. (e) and (f) are the corresponding multiple scattering contrast images, respectively.

Download Full Size | PDF

After blocking the jejunum mesenteric vessels with titanium clips for 30 seconds (Fig. 6(b)), the ischemic jejunum mesentery did not show obvious color changes in white light image (Fig. 6(b)). For the left side, the corresponding single scattering (Fig. 6(d)) and multiple scattering (Fig. 6(f)) contrast images revealed a significant decrease in both the superficial and deep mesenteric tissue. The blood flow velocity was calculated based on the auto-correlation model and normalized by the highest value in the imaging area before the occlusion. This normalization outputs the relative blood flow index which facilitates the statistical analysis. Fig. 7(a) presents the statistical analysis of relative blood flow changes in the randomly selected regions of interest (ROIs) covering the left and right sides after the occlusions. For the left side, the superficial (single scattering component) blood flow decreased significantly ($89.4\% \pm 9.7\%$ v.s. $16.6\% \pm 3.5\%$ in $Mean \pm SD$, Student’s t-test, $p<0.001$). The deep tissue perfusion (multiple scattering component) also showed a statistically significant decrease ($82.5\% \pm 8.6\%$ v.s. $24.4\% \pm 4.8\%$ in $Mean \pm SD$, Student’s t-test, $p<0.001$) due to the blood supply occlusion. Both contrast images and statistical analysis confirmed that the blood supply of the left side tissue was successfully blocked.

 

Fig. 7. Statistical analysis of relative blood flow changes before and after blocking the jejunum mesenteric vessels. (a) The box plots of relative blood flow index in randomly selected regions of interest (ROIs) covering the left side before (n=10) and after (n=10) the occlusion. (b) The corresponding box plots of relative blood flow index in randomly selected ROIs covering the right side. Student’s t-test, $^{**}p<0.001$.

Download Full Size | PDF

The operation of right side occlusion was intended to be incomplete blocking the blood supply. The remaining tissue perfusion in the right side of jejunum mesentery was revealed in both of the single scattering and multiple scattering contrast images (white circle areas in Fig. 6(d) and (f)). Compared to the superficial mesenteric tissue, there was more residual blood flow and tissue perfusion in deep mesenteric tissue (Fig. 6(f)). The statistical analysis of relative blood flow changes (Fig. 7(b)) demonstrated no statistically significant decrease in deep mesenteric tissue ($84.3\% \pm 9.9\%$ v.s. $78.0\% \pm 4.5\%$ in $Mean \pm SD$, Student’s t-test, $p>0.05$). For superficial tissue, although there was a statistically significant blood flow decrease ($83.8\% \pm 10.1\%$ v.s. $57.8\% \pm 9.1\%$ in $Mean \pm SD$, Student’s t-test, $p<0.001$), the remaining perfusion was still higher than the left side (Student’s t-test, $p<0.001$).

5. Discussions

5.1 Advantages and benefits of current study

The current laparoscope system integrates the two image sensors inside a single box. This design is convenient for the surgeon to hold the laparoscope and facilitates clinical laparoscopic surgeries. Another advantage of this study is the application of the rmLSCI method which provides both superficial and deeper tissue contrast images. More benefits are revealed in the multiple scattering contrast images, including the immunity to specular artifacts and stable visualization of deeper larger vessels. In the surgeons’ opinion, they need the multiple scattering contrast video as a higher priority as the intraoperative assistance. This study also established the standard validation procedures, including in-vitro flow phantom, in-vivo small animal, and pre-clinical big animal experiments, for future development of rmLSCI-enabled endoscope and laparoscope systems.

In endoscopic surgery, like laparoscopic surgery, thoracoscopic surgery, and hysteroscopic surgery, white light imaging is still the cornerstone of the image displaying, however, the operators sometimes cannot identify specific anatomical layers of vessels and blood flow by white light imaging, especially when the anatomic layers have no enough transparency. These limitations of white light imaging may increase the surgical difficulties and the risk of intraoperative intra-operative hemorrhage. Applying the rmLSCI method, we have achieved blood flow imaging of both the superficial and submembrane areas of the membrane. The operator can observe the specific layers of the vascular tissue, this can improve the accuracy and security of surgeries. Besides the inferior phrenic vein and jejunum mesenteric vessels imaging in this study, the rmLSCI method may also be applied in a variety of surgeries, such as imaging of segmental arteries and veins imaging in thoracic surgeries, imaging of the hepatic venous system and portal venous system in hepatic surgeries, evaluation of perfusion in deep tissue.

5.2 Limitations of current study

Motion artifacts always challenge the blood flow measurement in LSCI. For laparoscopic surgery, the tissue peristalsis and respiration are relatively slow motion which is resisted by the short exposure time and fast frame rate, e.g. 5ms exposure and 60 $fps$ in this study. The capillary wall motions don’t bias the tissue perfusion estimation in LSCI [37]. The pulsatile motion of the large artery produces the intra-frame motion artifacts and the inter-frame displacements of the vessel wall. A high-speed camera (e.g. fps > 1000) can significantly shorten the acquisition time and thus reduce the pulsatile effect of the vessel wall [38]. The video stitching techniques also support the removal of the inter-frame artifacts. Another benefit of a high-speed camera is that the pulsatile waveform can also be extracted based on the contrast analysis [39]. Meanwhile, other techniques, e.g. the Doppler ultrasonography [40], also provide more convenient solutions in artery flow measurement. As previous studies [25,26], the motion artifact due to the surgeon’s operating motion is still the major limitation in the current study. To produce Fig. 5 and Fig. 6, the surgeon needed to hold the laparoscope stably. This limitation can be solved by using a high-speed camera and an effective video stitching algorithm. Meanwhile, the computational load will be significantly increased and the current GPU-based design may not be capable. New FPGA and ASIC solutions should be developed for this purpose.

Because LSCI produces estimations of relative blood flow, blood leakage during the surgery always disables the continuous blood flow and tissue perfusion monitoring in the current laparoscope system. However, in practice, the relative changes in blood flow and tissue perfusion during a short time window are sufficiently useful for most intraoperative applications.

The current rmLSCI supports the separation of single and multiple scattering components. The penetration depth of either component can be characterized by the transport mean free path $l_t$. The single scattering component contains information on superficial tissue with thickness $<l_t$ [27]. For brain tissue ($l_t \approx 100 \mu m$) [41], $95\%$ reflectively detected signals come from the top $700 \mu m$ ($\approx 7l_t$) [42]. The penetration depth of multiple scattering component is thus about $1l_t \sim 7l_t$. A new strategy for further decomposition of multiple scattering component will improve the depth-resolved ability in deep tissue.

5.3 Future developments

The current laparoscope system provides white light video and LSCI contrast images simultaneously. It is not compatible with ICG fluorescent imaging. Actually, ICG imaging is the unique tool and clinically accepted criteria for tumor boundaries and sentinel lymph nodes. It will provide more intraoperative assistance if the ICG imaging modality is combined. New designs of the detection module, e.g. using triple sensors, together with hybrid light sources are required for future multi-modality laparoscope systems.

Some of the current laparoscope systems have dual-camera for binocular 3D visualization. The rmLSCI method can also be combined with such systems using the switch light sources strategy. This dual-camera setup facilitates the random matrix construction in rmLSCI and allows a shorter time window. This can speed up the contrast video frame rate and partially overcome the limitations of the current system.

6. Conclusion

In conclusion, we proposed a rmLSCI method for quasi-3D imaging of the blood flow and tissue perfusion for laparoscopic surgery. The rmLSCI-enabled laparoscopic system was also developed and validated in pre-clinical swine’s laparoscopic surgery experiment. The current rmLSCI-enabled laparoscopic system is ready for the next stage of clinical trials. The quasi-3D ability of the rmLSCI method shows more potential in other clinical applications, e.g. gastroscopy, colonoscopy, and surgical microscope. Random matrix description provides a powerful tool for decoding the information in the dynamic speckle phenomenons in the eigenvalue space. The applications of random matrix description in other coherent domain optical imaging will enable new tools for clinical applications.