Open Access
Add to BibSonomy
Abstract
An optical/nuclear hybrid surgical technique using ICG-99mTc-nanocolloid can improve lesion detectability by detecting both fluorescence and gamma signals. However, a hybrid multimodal laparoscope that can obtain both NIR and gamma images is not available yet. In this work, we present a proof-of-concept study of a prototype multimodal laparoscope that can provide simultaneous NIR/gamma/visible imaging using wavelength division multiplexing. The performances of optical and gamma imaging were evaluated using a USAF 1951 negative resolution target and 99mTc-filled tumor-like sources, respectively. Simultaneous NIR/gamma/visible images of two Eppendorf tubes containing a mixture of 99mTc-ICG are presented.
© 2018 Optical Society of America under the terms of the OSA Open Access Publishing Agreement
1. Introduction
Near-infrared (NIR) fluorescence-guided laparoscopic surgery [1] that uses indocyanine green (ICG) dye has been widely used for the localization of the sentinel lymph node (SLN) of prostate cancer patients [2]. The characteristics of NIR such as biological tissue-transparent window (700–900 nm) [3, 4] and less auto-fluorescence [5] make NIR fluorescence-guided cancer surgery more attractive to surgeons. NIR fluorescence-guided cancer surgery has become an established surgical technique due to its high resolution and real-time imaging capabilities. Nevertheless, NIR fluorescence imaging is still suffering from the low tissue penetration depth of less than 10 mm [1, 6], which may lead to a false-negative result [6].
On the other hand, radio guided surgery has been playing an important role for the identification of the SLN in clinical practice since early 1990s [7]. In particular, 99mTc-nanocolloid, which accumulates into the SLN, has been widely used either for the preoperative or intraoperative localization of the SLN for prostate or breast cancer patients. Unlike NIR imaging, radio guided surgery using 99mTc-based radiopharmaceuticals could identify the SLN located in deep tissues. However, the detection of the SLN close to the injection site (typically tumor) is hardly possible due to the poor spatial resolution and the high background gamma photons originating from the injection site of the radiopharmaceuticals.
Recently, a hybrid surgical technique combining radio guided and NIR fluorescence guided surgery that uses a hybrid optical/nuclide radiopharmaceutical tracer ICG-99mTc-nanocolloid has been developed to overcome the depth limit of NIR by using radio guided surgery [2, 8]. The hybrid surgical technique can improve the SLN detection rate as compared to either NIR fluorescence-guided surgery or radio guided surgery [2, 9]. Lately, a hybrid opto/nuclear probe system has been developed and evaluated in the clinical environment for SLN biopsy [10]. Although the hybrid opto/nuclear probe system can detect gamma and NIR signals simultaneously, imaging is not possible with the hybrid probe system. To take the full advantages of the hybrid surgical technique, a hybrid optical/nuclear laparoscopic imaging system is required. Nevertheless, a hybrid optical/nuclear laparoscopic imaging system has not been developed yet. In the previous study, we integrated the NIR, gamma, and visible imaging systems into a single endoscopic imaging system [11]. However, the three different images (NIR, gamma, and visible) should be obtained in a sequential manner due to the lack of a beam splitter module.
In this study, we demonstrate the proof-of-concept of a prototype multimodal laparoscope for simultaneous NIR/gamma/visible imaging of 99mTc-ICG optical/nuclear hybrid tracer by using a wavelength division multiplexing method. The performances of the prototype laparoscope were evaluated for each imaging modality i.e., NIR, gamma, and visible imaging. The first simultaneous NIR/gamma/visible fusion image was obtained using the proposed multimodal laparoscope.
2. Methods
2.1 Prototype multimodal laparoscope
The proposed multimodal laparoscope consists of a laparoscope, a beam splitter module, and an illumination system as shown in Fig. 1 and Fig. 2. The beam splitter module was used for the wavelength division multiplexing of three different images (gamma, NIR, visible images).
Fig. 1 Schematic diagram of the prototype multimodal laparoscope system using wavelength division multiplexing method.
Fig. 2 Photograph of the prototype multimodal laparoscope system using wavelength division multiplexing method.
2.1.1 Laparoscope
The laparoscope was specially designed for the integration of distal-end assemblies, imaging fiber bundle (IG-163-36, format size = 8 × 10 mm2, length = 915 mm, Schott, USA) [12], and illumination fiber bundle (#42347, Edmund Optics, USA) as shown in Fig. 3. The distal-end consists of a tungsten pinhole collimator, a focusing lens (f = 7.5 mm, #83-567, Edmund Optics, USA), a 2-mm-thick long-pass filter (525 nm~, OD > 4.0, #84-738, Edmund Optics, USA), a polished GSO crystal (9 × 11 × 2 mm3, OXIDE, Japan), and the imaging fiber bundle as shown in Fig. 3(a) and 3(b). The long-pass filter was attached to the front surface of the GSO crystal using optical grease (BC-630, Saint-Gobain, France) with a refractive index of 1.465 and the long-pass filter acts as a reflector for the scintillation light with a wavelength less than 500 nm while it passes only the light with a wavelength greater than 500 nm (visible and NIR emission light). The GSO crystal with a refractive index of 1.85 was optically coupled to the entrance surface of the optical fiber bundle using optical grease. The assemblies were placed inside an aluminum pipe (inner diameter = 16 mm, outer diameter = 17 mm, length = 320 mm). The aluminum pipe and the illumination fiber bundle were placed inside a stainless-steel pipe (inner diameter = 19 mm, outer diameter = 20 mm, length = 300 mm) as shown in Fig. 3(a). As a result, the laparoscope is 340 mm in length and 20 mm in diameter as shown in Fig. 3(a). The material of the outer pipe is stainless steel to prevent corrosion during laparoscopic surgery. Figure 3(b) describes the cross-sectional view of the distal-end part. The focusing lens can be fit into the tungsten pinhole collimator as shown in Fig. 3(b). The tungsten pinhole collimator can be inserted into the laparoscope and can be fixed using an O-ring as shown in Fig. 3(b) and 3(c). The assemblies of the distal-end part are shown in Fig. 3(d). The three different images of NIR, gamma, and visible were formed on the entrance surface of the imaging fiber bundle at the distal-end and then transferred to the proximal end of the laparoscope located in the beam splitter module.
Fig. 3 Laparoscope design: (a) cross section of the laparoscope, (b) distal-end cross section, (c) isotropic cross sectional view of the distal-end, and (d) photograph of the distal-end assemblies.
2.1.2 Beam splitter module and CCD cameras
The beam splitter module consists of two dichroic mirrors, aspheric lenses (focal length = 30 mm), band-pass filters and aluminum jigs [13]. The optical paths of the scintillation light, visible light, and NIR light were optimized using the ZEMAX simulation software as shown in Fig. 4(a). Custom-made lens adapters were used to adjust the focus of the NIR, gamma, and visible images. The NIR, visible, and gamma induced GSO scintillation photons were focused at the distal-end of the imaging fiber bundle and transferred to the proximal end of the imaging fiber bundle. Subsequently, these photons were focused by an aspheric achromatic doublet lens (f = 30 mm, Edmund Optics, USA), and then transferred to the first dichroic long-pass mirror (525 nm~, 525dcxru, Chroma, USA) as shown in Fig. 4(a). A custom-made jig was used for an accurate alignment of the two dichroic mirrors as shown in Fig. 4(b).
Fig. 4 Beam splitter module design: (a) ZEMAX simulation of the beam splitter, (b) photograph of the beam splitter module with three CCD cameras.
For gamma imaging, only GSO scintillation photons were reflected by the first dichroic long-pass mirror and then passed through a band-pass filter (HQ450/100 nm, Chroma, USA). Subsequently, the scintillation photons were focused by a lens and detected on the back-illuminated cooled CCD camera (Andor, iKon-M 934, USA), which has a quantum efficiency of 60% at 400 nm as shown in Fig. 5(b). The temperature and pixel binning of the gamma CCD were set to −80 °C and 16 × 16, respectively.
Fig. 5 (a) Transmission spectra of the optical filters and dichroic mirrors for the multiplexed NIR, gamma, visible imaging, (b) quantum efficiencies of the gamma CCD (Andor, iKon-M) and NIR CCD (FLI, MLx285) cameras, respectively.
For visible imaging, the visible photons passed through the first dichroic long-pass mirror were reflected at the second dichroic long-pass mirror (685 nm~, #67-085, Edmund Optics, USA) as shown in Fig. 4(a). Subsequently, the visible photons were focused by two aspheric lenses (One lens has a focal length of 30 mm and the other lens has a focal length of 50 mm) and detected by a color CCD (Basler, ac640-90uc, USA).
For NIR imaging, the NIR emission photons passed through the second dichroic long-pass mirror were filtered by a band-pass filter with a wavelength range of 814~851 nm (832/37 nm, #84-107, Edmund Optics, USA). Subsequently, the NIR emission photons were focused by a lens and then detected by a monochrome cooled CCD camera (FLI, MLx285, USA). The temperature and pixel binning of the NIR CCD were set to −30 °C and 4 × 4, respectively.
As a result, the information of NIR, gamma, and visible images could be multiplexed without a light interference each other as shown in Fig. 5(a). The spectral ranges of gamma, visible, and NIR imaging were 400~500 nm, 525~685 nm, and 814~851 nm, respectively.
2.1.3 Illumination system
To illuminate the imaging object with white light and NIR excitation light simultaneously, a custom-made illumination system was developed as shown in Fig. 6. The white and NIR excitation light could illuminate the object simultaneously by using a custom-made source combiner as shown in Fig. 6(a) and 6(b). The paths of white light and NIR at the source combiner were optimized by the ZEMAX optical simulation software as shown in Fig. 6(c). The illumination system consists of a halogen lamp (OSL2, Thorlabs, USA), an NIR light-emitting diode (730L4, 730/37 nm, Thorlabs, USA), a dichroic mirror (700dcxr, Chroma, USA), two long-pass filters (525 nm~, optical density > 4.0, Edmund Optics, USA), a band-pass filter (723~758 nm, HQ740/35 nm, Chroma, USA), and two illumination fiber bundles as shown in Fig. 6(c).
Fig. 6 Illumination system: (a) white light illumination, (b) NIR excitation light illumination, and (c) design of the illumination source combiner.
The white light emitted from the halogen lamp was transferred to the source combiner via an optical fiber bundle (D = 6.35 mm, L = 305 mm, Edmund Optics, USA). Subsequently, the white light was passed through an aspheric lens and the two long-pass filters (525 nm~) and reflected by the dichroic long-pass mirror (~700 nm). The reflected white light was focused by an aspheric lens (f = 30 mm, Edmund Optics, USA) into the entrance surface of the flexible illumination fiber bundle (OSL2FB, Thorlab, USA), which is 6.35 mm in diameter and 900 mm in length. The two long-pass filters (525 nm~) placed in front of the white light fiber bundle prevent the light interference between the scintillation (400~500 nm) and white light (525~700 nm), while the dichroic long-pass mirror (700 nm~) prevents the light interference between the white light (525–700 nm) and NIR emission light (723~758 nm).
The NIR excitation light emitted from the NIR LED was focused by a plastic aspheric lens (f = 17.5 mm, f-number = 0.7, #66006, Edmund Optics, USA) and then passed through a band-pass filter (723~758 nm). Subsequently, the NIR excitation light was passed through the dichroic long-pass mirror (700 nm~) and focused onto the illumination fiber bundle.
2.1.4 CCD image acquisition and processing
To obtain simultaneous NIR/gamma/visible images from the three multiple CCD cameras, a custom-written CCD image acquisition program was developed using MATLAB. The CCD temperature and CCD image acquisition parameters (exposure time, binning, readout speed) could be controlled using the software. The three CCD cameras were interfaced with a personal computer (PC) via universal serial bus (USB) cables.
The dark images of NIR, gamma, and visible images were subtracted, respectively. For gamma image, a 3 × 3 median filter was used to remove speckle noise. The visible image obtained by the Basler color CCD (matrix size = 658 × 492) was cropped with a matrix size of 400 × 400. The NIR and gamma images were resampled with the same matrix size (400 × 400) and co-registered with the color visible image. For the visibility, a green and purple pseudo-colored NIR and gamma images were overlaid on top of the visible color image, respectively.
Since the wavelength range from 400 nm to 525 nm was allocated for gamma imaging (Fig. 5), this spectral range is missing in the visible image. The wavelength range of visible image is from 525 nm to 675 nm and this resulted in color distortion due to the lack of the blue component (400~525 nm) in the RGB color image. Therefore, the white balance process provided by the Pylon CCD control software was applied to the visible image to correct the color distortion. To correct the non-uniformity of the pixel intensity across the FOV, a flat-field correction was applied to the visible color image as well as the NIR image [14].
2.2 Performance evaluation of the multimodal laparoscope
2.2.1 Performance evaluation of visible and NIR imaging
The performance of visible and NIR imaging was evaluated a US Air Force (USAF) 1951 negative resolution target (2” × 2”, #38256, Edmund Optics, USA). To obtain uniform illumination, a white paper was attached at the rear side of the target as a light diffuser. Then a white LED or NIR LED was illuminated from the rear side of the target. The USAF 1951 negative resolution target was placed 20 mm away from the tip of the laparoscope and visible color and NIR images were obtained for 100 ms, respectively [15]. The contrast transfer functions (CTFs) of visible and NIR images were calculated from the line profiles of the vertical and horizontal bar patterns by using the following equation [13, 16].
where and are the maximum and minimum pixel intensities of the line profile and is the spatial frequency of the line profile (line pairs/mm) [16].2.2.2 Performance evaluation of gamma imaging
Since the pinhole diameter is the most important factor that determines the gamma image quality, the gamma image was obtained with different tungsten pinhole diameters (0.5, 1.0, 2.0, and 4.0 mm) [11]. A custom-made cylindrical source container 2 mm in depth and 5 mm in diameter (a volume of 40 μL) was filled with different 99mTc activities of 0.6, 1.2, and 2.3 MBq, respectively. The center-to-center distance of each source container was 10 mm and the three sources were placed 30 mm away from the laparoscope tip.
To investigate the effect of CCD acquisition time on the gamma image quality, an Eppendorf (EP) tube containing 99mTc of 2.85 MBq (volume = 50 μL) was placed 45 mm away from the laparoscope tip and then gamma images were obtained with different CCD acquisition times (5, 10, 20, 30, and 40 s) using a tungsten pinhole aperture size of 2 mm. The contrast-to-noise ratio (CNR) of the gamma image was calculated using the following equations.
where andare the average pixel intensities (net counts/pixel) of the signal and background regions, respectively. represents the fractional standard deviation of background counts due to random statistical variation.2.3 Simultaneous NIR/gamma/visible imaging of 99mTc-ICG hybrid tracer
To obtain simultaneous NIR/gamma/visible fusion images, two EP tubes containing a mixture of 99mTc and ICG dye were placed 40 mm away from the laparoscope tip and then NIR, visible, and gamma images were acquired simultaneously. The 99mTc activity of each EP tube was 1.5 MBq. Each EP tube contained ICG with a volume of 0.1 mL (concentration = 1 mg/mL) [17]. The two EP tubes and the laparoscope head were placed inside a dark box to mimic an abdomen cavity [11]. The visible and NIR images were obtained for 100 ms, while the gamma image was obtained for 30 s.
2.4 Comparison of penetration depth between NIR and gamma
In order to investigate the tissue penetration depths of NIR and gamma imaging, a custom-made gelatin phantom was used. The custom-made tissue equivalent gelatin phantom was made up of 10% gelatin from porcine skin (CAS number 9000-70-8, Sigma Aldrich, USA), 99% distilled water, and 1%-2% intralipidTM (20%). The percent unit here represents the weight per volume (w/v). An EP tube containing a mixture of 99mTc and ICG was used as an imaging object. The EP tube contained 99mTc of 1.1 MBq and ICG of 0.15 mL, was placed 40 mm away from the laparoscope tip. The NIR and gamma images were acquired for 100 ms and 30 sec, respectively with various phantom thicknesses (0, 5, 10, 15, and 20 mm).
3. Results
3.1 Performance of the multimodal laparoscope
3.1.1 Performance of visible and NIR imaging
The visible color and NIR images of the USAF 1951 negative resolution target were obtained at a working distance of 20 mm for 100 ms as shown in Fig. 7. The vertical and horizontal bar patterns of group number 0 and 1 could be clearly resolved either by the visible color image or the NIR reflectance image as shown in Fig. 7(a) and 7(d). The inner patterns of the USAF target were enlarged as shown in Fig. 7(b) and 7(e). The line profile of the fifth element in group number 2 was obtained as shown in Fig. 7(c). In the visible image, the three horizontal white bars could be resolved clearly in the line profile that corresponds to the spatial resolution of 6.35 lp/mm, or ~80 μm as shown in Fig. 7(c). In the NIR image, the three horizontal white bars of the 5th element in group number 2 could be resolved clearly as shown in Fig. 7(f). The bar patterns of the 6th element in group number 6 could be resolved with the NIR image. The contrast transfer functions of the visible and NIR images were evaluated from the USAF 1951 negative resolution target as shown in Fig. 8. The NIR image showed The contrast of NIR image was 1.6 times higher than that of the visible image due to the relatively narrow wavelength range of the NIR image (814~851 nm) and more accurate optical alignment as compared that of the visible image (525~685 nm). The angular field of view and working distance of the multimodal laparoscope were 47.2 deg, and 20~70 mm, respectively [18].
Fig. 7 Visible and NIR images of a USAF 1951 negative resolution target taken at a working distance of 20 mm: (a) visible image, (b) enlarged inner pattern image, (c) line profile of the group number 2, element 5, (d) NIR image, (e) enlarged inner pattern image, and (f) line profile of the group number 2, element 5 corresponding to the spatial resolution of 6.35 lp/mm.
Fig. 8 The contrast transfer functions of (a) visible and (b) NIR images of the USAF 1951 negative target.
3.1.2 Performance of gamma imaging
The gamma images of three 99mTc radioactive sources were obtained with different pinhole sizes (0.5, 1, 2, 3, and 4 mm) for 2 min as shown in Fig. 9(a). The pinhole-to-source distance was 30 mm. The three adjacent sources, each of which is 5 mm in diameter and 2 mm in depth (left = 0.6 MBq, center = 1.2 MBq, and right = 2.3 MBq) with a center-to-center distance of 10 mm, could be resolved clearly with a pinhole diameter of 0.5 mm. However, the two 99mTc sources of the center (1.2 MBq) and right (2.3 MBq) cannot be distinguished with the pinhole diameter of 2 mm as shown in Fig. 9(a) and 9(b).
Fig. 9 (a) Gamma image obtained for 2 min with different pinhole sizes (0.5, 1, 2, 3,and 4 mm), and (b) line profile across the three 99mTc sources (0.6, 1.2, and 2.3 MBq).
The gamma images of an EP tube containing a 99mTc source (2.85 MBq) with a volume of 50 μL were obtained with different CCD acquisition times (5, 10, 20, 30, and 40 s) as shown in Fig. 10(a). A tungsten pinhole of diameter 2 mm was used, and the pinhole-to-source distance was set to 45 mm. The signal intensity was calculated from the region of interest (ROI) encircled by a dotted line while the background intensity was calculated from the rectangle drawn at the right side of the gamma image (acquisition time = 40 sec) as shown in Fig. 10(a). To obtain the net counts/pixels, the total pixel intensity of the ROI was divided by the number of pixels. The total pixel intensity of the signal region linearly increases with acquisition time as shown in Fig. 10(b). The contrast-to-noise ratio (CNR) also increases with CCD acquisition time as shown in Fig. 10(c). The CNR fluctuation at the CCD acquisition time of 10 s was attributed to the slightly decreased background intensity.
Fig. 10 Gamma image quality evaluation with different acquisition times: (a) gamma images of an EP-tube containing a 99mTc of 2.85 MBq (volume = 50 μL) with different acquisition times (5, 10, 20, 30, and 40 sec), using pinhole diameter of 2 mm, (b) total pixel intensities of signal and background regions, and (c) CNR and net count/pixel values of the signal and background regions.
3.2 Simultaneous NIR/gamma/visible imaging of a 99mTc-ICG hybrid tracer
Simultaneous NIR/gamma/visible images of two EP tubes containing a mixture of 99mTc and ICG fluorophore were acquired using the prototype multimodal laparoscope as shown in Fig. 10. With the NIR image, the two EP tubes (15 mm apart) could be identified clearly as shown in Fig. 11(b). The pseudo green-colored NIR image shows an excellent spatial correlation with the visible color image as shown in Fig. 11(c). The two EP tubes also could be identified with the gamma image obtained for 30 s as shown in Fig. 11(e). The gamma image shows a good spatial correlation with the visible color image as shown in Fig. 11(f). The net counts/pixel values of the two EP tubes in the gamma image were 6.82 (left) and 6.84 (right), respectively.
Fig. 11 Simultaneous NIR/gamma/visible images: (a) visible image of two EP-tube containing a mixture of 99mTc and ICG dye, (b) NIR image, (c) visible/NIR fusion image, (d) photograph of the imaging setup, (e) gamma image obtained for 30 sec, and (f) visible/gamma/fusion image.
3.3 Comparison of the penetration depth between NIR and gamma imaging
The NIR image quality was strongly affected either by the phantom thickness and intralipid concentration as shown in Fig. 12(a). The EP tube containing 99mTc-ICG mixture could not be identified with the gelatin phantom thickness of 10 mm. In contrast, the gamma image quality was slightly affected by only the gelatin phantom thickness and the EP tube could be identified even with the gelatin phantom thickness of 20 mm as shown in Fig. 12(b). The NIR and gamma signal intensities were calculated from the ROI enclose by the white dotted rectangle as shown in Fig. 12(a) and 12(b). The NIR noise level was calculated from the ROI indicated by a white solid rectangle as shown in Fig. 12(a). The relative NIR signal intensities were 100, 60, 19, 15, and 13%, for the gelatin phantom (Intralipid 1%) thicknesses of 0, 5, 10, 15, and 20 mm, respectively. The relative NIR signal intensities were 100, 39, 12, 11, and 11% for the gelatin phantom (Intralipid 2%) thicknesses of 0, 5, 10, 15, and 20 mm, respectively. The relative gamma signal intensities were 100, 95, 89, 87, and 82%, for the gelatin phantom (Intralipid 1%) thickness of 0, 5, 10, 15, and 20 mm, respectively. The relative gamma signal intensities were 100, 96, 91, 85, and 84%, for the gelatin phantom (Intralipid 2%) thicknesses of 0, 5, 10, 15, and 20 mm, respectively. The NIR signal intensity was drastically decreased to the noise level as the gelatin phantom thickness was increased over 10 mm. However, the gamma signal intensity was decreased to 80% even with the gelatin phantom thickness of 20 mm as shown in Fig. 13.
Fig. 12 Comparison of penetration depth between NIR and gamma images with a custom-made gelatin phantom: (a) NIR images of an EP tube containing a mixture of 99mTc (1.1 MBq) and ICG (0.15 mL) obtained for 100 ms, (b) gamma images of the identical EP tube obtained for 30 s.
Fig. 13 Comparison of penetration depth between NIR and gamma images with a custom-made gelatin phantom. The NIR noise level was calculated from the ROI enclosed by a white solid rectangle.
4. Discussion
Near-infrared fluorescence-guided laparoscopic cancer surgery is a promising surgical technique that can visualize the cancer or SLN in real-time during surgery [1]. Due to the relatively longer biological tissue penetration depth and less auto-fluorescence, NIR fluorescence-guided surgery has been widely used in clinical practice. However, NIR fluorescence-guided SLN mapping is still suffering from the limited tissue penetration depth [1]. Recent studies have demonstrated that the hybrid surgical technique combining radio-guided surgery and NIR fluorescence-guided surgery could improve the detection accuracy of SLN located in deep tissues [2, 8, 10]. Nevertheless, a hybrid laparoscope for performing simultaneous NIR/gamma imaging has not been developed .
The proposed prototype multimodal laparoscope could provide NIR, gamma, and visible images simultaneously. The three different imaging modalities of NIR, gamma, and visible imaging could be integrated into a single laparoscope using a specially designed distal-end part [11] and a beam splitter module as shown in Fig. 3, and Fig. 4. The NIR, gamma, and visible images formed at the distal-end of the laparoscope could be multiplexed using a beam splitter module. The white and NIR excitation light could be illuminated simultaneously into the imaging object using a custom-made illumination system as shown in Fig. 6. The interference between white light illumination and scintillation light which carries the gamma image information is minimized by using long-pass filters (525 nm~) at the distal-end part and the illumination source combiner, respectively as shown in Fig. 3(b) and Fig. 6(c). The spatial resolution of visible and NIR imaging are 6.35 lp/mm and 7.13 lp/mm, respectively. The contrast transfer function of NIR image was 60% higher than that of the visible image as shown in Fig. 8. The spatial resolution of the NIR image is better than that of the visible image because of the more accurate optical alignment between the distal-end of the optical fiber bundle and the CCD sensor. Unlike the NIR light, the visible light undergoes dichroic reflection at 90° resulting in a slightly distorted optical alignment. Unlike the contrast transfer function in horizontal direction, the contrast transfer function of the vertical direction was deteriorated as the spatial frequency was increased over 2.5 lp/mm for both visible and NIR images as shown in Fig. 8. In future study, the discrepancy of the contrast transfer function between the horizontal and vertical directions should be minimized by optimizing the optical system. The gamma image quality was strongly dependent on the aperture size of the tungsten pinhole collimator as shown in Fig. 10. The tungsten pinhole diameter of less than 2 mm should be used to secure the gamma spatial resolution of less than 10 mm. The gamma signal intensity and CNR show a linear relationship with the CCD acquisition time ranging from 5 s to 40 s as shown in Fig. 10. To obtain a CNR greater than 3, the gamma imaging time should be longer than 20~30 s. The simultaneous NIR/gamma/visible fusion images of the optical/nuclear hybrid tracer (99mTc-ICG) could be obtained with the proposed prototype multimodal laparoscope as shown in Fig. 11. The visible and NIR images show a good spatial correlation with each other as shown in Fig. 11(c). The gamma image shows a good agreement with the NIR and visible image as shown in Fig. 11(f). The gamma imaging time of 30 s is still too long to be used in clinical practice. The long gamma imaging time is mainly attributed to the use of the optical fiber bundle and the low quantum efficiency (60–80%) of the gamma CCD (Andor, iKon-M 934) in the wavelength range from 400~450 nm as shown in Fig. 5(b). The gamma imaging time could be further reduced by using a back-illuminated UV-enhanced CCD having a quantum efficiency of 90% at a wavelength of 400 nm. The band-pass filter used for the gamma imaging has a transmission efficiency of 65~80% at a wavelength range of 400~500 nm. The use of the band-pass optical filter with a higher transmission efficiency (95%) could minimize the gamma imaging time. The gamma imaging which takes 30 s could complement the NIR image in terms of penetration depth as shown in Fig. 12 and Fig. 13. Unlike the NIR imaging which is limited by the tissue penetration depth of less than 10 mm, the gamma imaging could identify the 99mTc-ICG mixture covered by a 20 mm thick gelatin phantom. In future, the gamma imaging time of 30 s will be further minimized for the clinical application.
The concept of simultaneous NIR/gamma/visible imaging using a wavelength-based multiplexing method was demonstrated successfully. The proposed multimodal imaging method could give an insight for the further development of a clinical hybrid laparoscopic imaging system which could be used for the hybrid optical/nuclear tracer-based surgery [8].
5. Conclusion
We developed a novel prototype multimodal laparoscope system for simultaneous NIR/gamma/visible imaging by employing wavelength division multiplexing method. The prototype multimodal laparoscope has the potential to take advantages of the deep penetration capability of gamma photons (140 keV), and the high resolution of NIR/visible imaging. In future, the clinical efficacy of the multimodal laparoscope for SLN mapping will be evaluated using preclinical trials.
Funding
Korea Evaluation Institute of Industrial Technology funded by the Ministry of Trade, Industry and Energy of the Korean Government (2015-4-10051988); National Research Foundation (NRF) of Korea of the Ministry of Science, ICT & Future Planning, Nuclear R&D Program (NRF-2016M2A2A4A03913619); National Research Foundation (NRF) of Korea of the Ministry of Science, ICT & Future Planning, Nuclear R&D Program (NRF-2017M2A2A4A01071175).
References and links
1. A. L. Vahrmeijer, M. Hutteman, J. R. van der Vorst, C. J. van de Velde, and J. V. Frangioni, “Image-guided cancer surgery using near-infrared fluorescence,” Nat. Rev. Clin. Oncol. 10(9), 507–518 (2013). [CrossRef] [PubMed]
2. H. G. van der Poel, T. Buckle, O. R. Brouwer, R. A. Valdés Olmos, and F. W. van Leeuwen, “Intraoperative laparoscopic fluorescence guidance to the sentinel lymph node in prostate cancer patients: clinical proof of concept of an integrated functional imaging approach using a multimodal tracer,” Eur. Urol. 60(4), 826–833 (2011). [CrossRef] [PubMed]
3. A. N. Bashkatov, E. A. Genina, V. I. Kochubey, and V. V. Tuchin, “Optical properties of human skin, subcutaneous and mucous tissues in the wavelength range from 400 to 2000 nm,” J. Phys. D Appl. Phys. 38(15), 2543–2555 (2005). [CrossRef]
4. R. Weissleder, “A clearer vision for in vivo imaging,” Nat. Biotechnol. 19(4), 316–317 (2001). [CrossRef] [PubMed]
5. J. V. Frangioni, “In vivo near-infrared fluorescence imaging,” Curr. Opin. Chem. Biol. 7(5), 626–634 (2003). [CrossRef] [PubMed]
6. C. Chi, Y. Du, J. Ye, D. Kou, J. Qiu, J. Wang, J. Tian, and X. Chen, “Intraoperative imaging-guided cancer surgery: from current fluorescence molecular imaging methods to future multi-modality imaging technology,” Theranostics 4(11), 1072–1084 (2014). [CrossRef] [PubMed]
7. S. P. Povoski, R. L. Neff, C. M. Mojzisik, D. M. O’Malley, G. H. Hinkle, N. C. Hall, D. A. Murrey Jr, M. V. Knopp, and E. W. Martin Jr., “A comprehensive overview of radioguided surgery using gamma detection probe technology,” World J. Surg. Oncol. 7(1), 11 (2009). [CrossRef] [PubMed]
8. J. P. Gambini and T. P. Quinn, “Hybrid tracers and devices for intraoperative imaging: the future for radio guided surgery?” Clin. Transl. Imaging 4(5), 343–351 (2016). [CrossRef]
9. G. H. KleinJan, D. Hellingman, N. S. van den Berg, M. N. van Oosterom, K. Hendricksen, S. Horenblas, R. A. Valdes Olmos, and F. W. van Leeuwen, “Hybrid Surgical Guidance: Does Hardware Integration of gamma- and Fluorescence Imaging Modalities Make Sense?” J. Nucl. Med. 58(4), 646–650 (2017). [CrossRef] [PubMed]
10. N. S. van den Berg, H. Simon, G. H. Kleinjan, T. Engelen, A. Bunschoten, M. M. Welling, B. M. Tijink, S. Horenblas, J. Chambron, and F. W. B. van Leeuwen, “First-in-human evaluation of a hybrid modality that allows combined radio- and (near-infrared) fluorescence tracing during surgery,” Eur. J. Nucl. Med. Mol. Imaging 42(11), 1639–1647 (2015). [CrossRef] [PubMed]
11. H. G. Kang, H. Y. Lee, K. M. Kim, S. H. Song, G. C. Hong, and S. J. Hong, “A feasibility study of an integrated NIR/gamma/visible imaging system for endoscopic sentinel lymph node mapping,” Med. Phys. 44(1), 227–239 (2017). [CrossRef] [PubMed]
12. S. A. Burgess, D. Ratner, B. R. Chen, and E. M. C. Hillman, “Fiber-optic and articulating arm implementations of laminar optical tomography for clinical applications,” Biomed. Opt. Express 1(3), 780–790 (2010). [CrossRef] [PubMed]
13. G. Themelis, J. S. Yoo, K. S. Soh, R. Schulz, and V. Ntziachristos, “Real-time intraoperative fluorescence imaging system using light-absorption correction,” J. Biomed. Opt. 14(6), 064012 (2009). [CrossRef] [PubMed]
14. M. Model, “Intensity Calibration and Flat-Field Correction for Fluorescence Microscopes,” Curr. Protoc. Cytom. 68, 1–10 (2014). [PubMed]
15. J. Glatz, J. Varga, P. B. Garcia-Allende, M. Koch, F. R. Greten, and V. Ntziachristos, “Concurrent video-rate color and near-infrared fluorescence laparoscopy,” J. Biomed. Opt. 18(10), 101302 (2013). [CrossRef] [PubMed]
16. H. G. Kang, S. H. Song, Y. B. Han, K. M. Kim, and S. J. Hong, “Lens implementation on the GATE Monte Carlo toolkit for optical imaging simulation,” J. Biomed. Opt. 23(2), 1–13 (2018). [CrossRef] [PubMed]
17. C. Chi, J. Ye, H. Ding, D. He, W. Huang, G. J. Zhang, and J. Tian, “Use of indocyanine green for detecting the sentinel lymph node in breast cancer patients: from preclinical evaluation to clinical validation,” PLoS One 8(12), e83927 (2013). [CrossRef] [PubMed]
18. Q. Wang, A. Khanicheh, D. Leiner, D. Shafer, and J. Zobel, “Endoscope field of view measurement,” Biomed. Opt. Express 8(3), 1441–1454 (2017). [CrossRef] [PubMed]
