Open Access
Add to BibSonomy
Abstract
Polarization imaging can reveal orthogonal information with respect to color about the structural composition of biological tissue, and with the advance of superior polarimeters its use for biomedical applications has proliferated in the last decade. Polarimetry can be used in pre-clinical and clinical settings for the early detection of cancerous tissue. Polarization-based endoscopy with the complementary near-infrared fluorescence imaging modality improves the early diagnosis of flat cancerous lesions in colorectal tumor models. With the development of new polarization sensors the need to use standard laboratory optics to create custom imaging systems increases. These additional optics can behave as polarization filters effectively degrading and modifying the original tissue’s polarization signatures leading to erroneous judgments. Here, we present a framework to characterize the spectral and polarization properties of rigid endoscopes for polarization-based endoscopic imaging. We describe and evaluate two calibration schemes based on Mueller calculus to reconstruct the original polarization information. Optical limitations of the endoscopes and minimum polarimeter requirements are discussed that may be of interest to other researchers working with custom polarization-based imaging systems.
© 2017 Optical Society of America
1. Introduction
The polarization state of light has been demonstrated to carry paramount information about the intrinsic properties of objects, tissues, and media with which light interacts [1]. The polarization properties of light are essentially a fingerprint of its past optical interactions. These properties can reveal the structural composition and shape of the object that reflected or refracted the light beam, and the composition of the medium through which the light propagated. Polarization signatures hold orthogonal information with respect to the other fundamental properties of light, i.e. intensity and wavelength. Although the unaided human eye lacks the ability to differentiate polarization states, a vast range of polarization-sensitive imagers or polarimeters have been developed by combining polarization filters with polarization-blind photodetectors. To realize these sensors, polarization filters are modulated either in time, light amplitude, or focal plane [2–4].
Polarization imaging sensors have enabled a wide range of applications, such as fingerprint detection, shape reconstruction, underwater target recognition, contrast enhancement in hazy conditions, material detection, and many others [5–11]. In the biomedical arena, polarization imaging has been used to discriminate cancerous vs. healthy cells and colorectal cancer detection, to identify stress in ligaments and tendons, to extend the capability of optical coherence tomography, and more [12–17]. For example, due to the differences in nuclei size in healthy vs. cancerous tissue, circularly polarized light is utilized for non-invasive diagnosis of lung cancerous tissue [18].
Minimally invasive surgeries have the benefit of faster recovery, less post-operative pain and discomfort, shorter hospital stay, and lower morbidity compared to regular surgeries. Incorporating spectral and polarization imaging in endoscopes can greatly enhance the surgical outcome. Charanya et al. reported an endoscopic imaging technique which combines three imaging modalities for the detection of flat lesions in colitis-associated cancer [19]. Colitis-associated cancer arises from premalignant flat lesions in the colon. These flat lesions, which blend seamlessly with healthy tissue, are often misdiagnosed during colonoscopy utilizing intensity and color-based endoscope systems. To mitigate these false negatives, a color-polarization-fluorescence endoscope system was reported, which improved the overall sensitivity and specificity in identifying and differentiating pre-cancerous and cancerous from healthy tissue. Figure 1 shows a gray scale image (left) and a polarization image (right) in a linear false-color map for an in vivo polarization endoscopy of an adenomatous tumor in a mouse. Red areas indicate a higher degree of polarization while dark blue areas indicate a lower degree of polarization. A higher polarization is associated with healthy tissue while lower polarization signatures indicate possible tumor locations. Polarization-based endoscopic imaging has also enabled biomechanics studies by evaluating stress in tendons and ligaments under circular polarized light [13], multispectral studies by detecting micrometer sized particles using elastic light scattering spectroscopy [20], and more. Analogously, near-infrared fluorescence image guided surgery for colonoscopy with tissue biomarkers has demonstrated diagnostic yield superior to that of white-light colonoscopy [21–23].
Fig. 1 Gray scale image (left) and a polarization image (right) in a linear false-color map for an in vivo polarization endoscopy of an adenomatous tumor in a mouse. Red areas indicate a higher degree of polarization while dark blue areas indicate a lower degree of polarization. A higher polarization is associated with healthy tissue while lower polarization signatures indicate possible tumor locations.
As more biomedical applications emerge which utilize the polarization imaging modality, the need increases for compact, flexible, and versatile polarimeters, i.e. polarization cameras that are ready to be integrated into a custom optical setup similarly to other industrial-grade scientific instruments. In response to this demand, both research groups and industry have developed adaptable and precise polarimeters with standards similar to those of the traditional color sensor industry. These off-the-shelf polarization-sensitive camera systems offer the ability to couple with existing laboratory optics, such as camera lenses or endoscopes, to create tailored imaging setups for specific applications. Nevertheless, this versatility can jeopardize the integrity of the polarization states of interest due to the use of non-inert polarization optics that behave as polarization filters. Optical elements within endoscopes can modify or degrade the original polarization signatures, and even create unreadable or unrecoverable polarization states for linear polarimeters. It is crucial to perform optical evaluation of each additional optical component used in the polarization imaging modality and extract its Mueller matrix, which models the polarization behavior of an individual element. Although some work has been reported in the literature to characterize rigid endoscopes, little effort has been made to outline and evaluate calibration schemes that thoroughly correct for the induced polarization effects caused by the endoscopes’ optical elements [24, 25]. Comprehensive calibration schemes are needed and their performance has to be properly evaluated to assess their polarization reconstruction capabilities. As a result of this shortcoming, modified endoscopes have been utilized to mitigate the polarization effects of their optics instead of employing polarization calibration methods [20]. These custom endoscopes are still far from behaving as ideal inert polarization elements and sacrifice the ability of choosing off-the-shelf commercial and clinically approved endoscopes.
In this paper, a framework is presented to evaluate, characterize, and calibrate polarization and spectral properties of rigid endoscopes, which are widely used in endoscopic pituitary surgeries, laparoscopic surgeries, orthopedic surgeries, endoscopic biopsies, brain tumor resections, and more [26–30]. We describe two polarization calibration schemes based on Mueller matrix theory and machine learning algorithms and provide experimental data demonstrating the benefits of these methods in polarization imaging. In addition, we provide spectral response, optical resolution, depth of field (DOF), and modulation transfer function (MTF) for two commercially available rigid endoscopes: (1) 5 mm x 30cm (WA50372B, Olympus Germany) and (2) 10 mm x 33 cm (WA53000A, Olympus Germany). These rigid endoscopes include an input light port which directs the illumination to the imaged object through optical fibers and a series of relay lenses which transmit the image captured by the objective lens at the distal tip to the eyepiece at the proximal end. While the endoscopes are intended for use primarily in reflection-based imaging, the calibration methods presented here work in either transmission- or reflection-based imaging since no assumptions about the input light are made. These endoscopes are the bedrock for many small and medium animal studies and are used in minimally invasive surgeries in human patients. Hence, correct polarization signatures and in-depth understanding of the optical performance of these endoscopes will help scientists and physicians to choose the endoscope for their applications. Furthermore, the polarization calibration schemes presented in this paper can be applied to any endoscope or other instrument utilized for polarization imaging, which will enable correct collection and interpretation of polarization information.
2. Experimental setup and theory
2.1 Polarization theory overview
Light as an electromagnetic wave can be described with three fundamental properties: (1) intensity, (2) wavelength, and (3) polarization. Intensity and wavelength can be interpreted as brightness and color respectively, while polarization describes the phase between the orthogonal components of the electric field as light propagates through time and space. A random phase indicates that the light is completely unpolarized, while a phase ranging from -π/2 to + π/2 indicates that the light is elliptically polarized with the special cases of being circularly polarized at the extremes and linearly polarized at a null phase. Polarization refers to the aggregated photon result, with partial polarization being the most common case in nature. The Stokes vector is commonly used to describe the polarization state of the light wave and it is composed of four parameters: (1) S0 describes the light intensity, (2) S1 describes the portion of the light that is either horizontally or vertically polarized, (3) S2 describes the portion of the light that is diagonally polarized, and (4) S3 describes the portion of the light that is circularly polarized. The Poincaré sphere serves as a visualization tool for the Stokes vector in a three-dimensional Cartesian coordinate system. The x, y, and z axes are represented by the S1, S2, and S3 parameters respectively. Any possible polarization state can be placed as a data point inside or on the surface of the Poincaré sphere with center at the origin and radius S0. Data points on the sphere’s surface are fully polarized, while data points inside the sphere are partially polarized.
Since silicon-based photodetectors cannot measure the polarization properties of light directly, multiple light measurements employing a set of polarization filters are needed. This set commonly consists of linear polarization filters and retarders which are used to modulate the light intensity of the measured light as a function of the incoming polarization state. Equation (1) shows the relationship between an incoming Stokes vector and the measured intensity after the light has gone through a retarder with retardance ϕ and a linear polarizer with a rotation angle of θ, in that order. As shown in Eq. (1), four independent measurements are enough to solve for the four Stokes parameters; however, an over-constrained system is usually preferred to reduce the measurement noise.
Other figures of merit are also used to describe particular characteristics of a polarization state. The degree of polarization (DoP) is the distance, on the Poincaré sphere’s coordinate system, from the data point to the origin, normalized with respect to S0. Similarly, the degree of circular polarization (DoCP) is the distance from the data point to the S1S2 plane normalized by S0, and the degree of linear polarization (DoLP) is the magnitude of the data point’s projection to the S1S2 plane normalized by S0. Lastly, the angle of polarization (AoP) is half the data point’s azimuth angle [1].
2.2 Polarization characterization
The optical setup, shown in Fig. 2, was used to create a collection of different incoming polarization states which are measured before and after going through the endoscopes. The optical setup comprises three parts: (a) the Stokes vector generator, (b) the endoscope under evaluation, and (c) the polarization analyzer or polarimeter. The Stokes vector generator was built with (1) a DPSS Green (532 nm) laser (GL532T3-200, Shanghai Laser & Optics Century), (2) an integrating sphere (819D-SF-4, Newport), (3) an adjustable iris (SM2D25, Thorlabs), (4) an aspheric collimating lens (ACL7560, Thorlabs), (5) a 532 nm laser clean-up filter (LL01-532-12.5, Semrock), (6) a linear polarizer (20LP-VIS-B, Newport), (7) a zero-order quarter wave retarder at 532 nm (20RP34-532, Newport), and (8) two nano-rotator stages (NR360S, Newport), one for the linear polarization filter and one for the quarter wave retarder. The analyzer was built with (1) a quarter wave retarder, (2) a linear polarizer, (3) two rotation stages, and (4) a calibrated photodiode (S130C, Thorlabs) driven by (5) a power meter (PM100D, Thorlabs), with elements (1) through (3) being identical to those used to generate the Stokes vectors. In the analyzer setup, the retarder and polarizer can switch places to effectively create zero retardance while keeping the same transmission ratio.
2.3 Polarization Calibration
The Mueller matrix is a powerful mathematical tool that describes any optical component that may reflect, refract, or scatter the light and thereby change the initial polarization state of incoherent light. By the same principle, it is possible to reconstruct an unknown input Stokes vector if the optical component’s Mueller matrix and the output Stokes vector are known. This relation is shown in Eq. (2), where the 4-by-4 matrix M is known as the Mueller matrix.
Two learning algorithms are presented in this paper that enable correct estimation of the Mueller matrix for two rigid endoscopes widely used in clinical and pre-clinical settings.
2.3.1 Modeling the rigid endoscopes as double rotated retarders
Mueller matrices for ideal polarization components, e.g. linear polarizers, retarders, rotators, or diattenuators, are well documented in the literature [1]. For more complex optical elements with multiple polarization effects, which can be difficult to interpret from a single matrix, it is possible to decompose their non-degenerate Mueller matrix into a set of simpler Mueller matrices describing single optical effects. These effects can be decomposed into depolarization, retardance, and diattenuation [31]. We hypothesize that the birefringent crystals at the input and output endoscope windows of a rigid endoscope can be modeled as two rotated retarders. This hypothesis is based on the fact that the DoP calculated on the output data for both endoscopes is virtually one, meaning no depolarization occurs in the endoscope, and that the endoscopes’ optics contain minimal diattenuation effects. It is possible to replace the endoscope crystals with fused silica or other non-birefringent materials to create a customized endoscope; however, the intention of this method is to calibrate off-the-shelf unmodified commercial endoscopes that can be easily implemented in clinical trials and require no further approval by a regulatory body, e.g., the U.S. Food and Drug Administration (FDA). The Mueller matrix of a rotated retarder is presented in Eq. (3), where θ represents the retarder’s rotation angle and ϕ represents the retardance or phase shift.
The training algorithm consists of finding the four angle terms that minimize the error between the training Stokes vector sets Sout and S*out. Equations (4) and (5) define S*out and the error between two sets of Stokes vectors, respectively.
To find the parameters that minimize this error function, the algorithm first does a four-dimensional coarse grid search with the objective of finding a good pair of initial guesses. After acquiring the initial search points, the Broyden-Fletcher-Goldfarb-Shanno (BFGS) Quasi-Newton algorithm [32], an off-the-shelf optimization algorithm, was used. The grid searches revealed that there is only one local minimum for each endoscope on the non-redundant range of the angle parameters. Hence, more sophisticated optimization algorithms are not needed. Once the four angle parameters—and thus the endoscope’s Mueller matrix—are found, its pseudo-inverse can be calculated and used to compute the input Stokes vectors.
2.3.2 Linear regression with cross validation
Linear regression algorithms are typically used to extract more robust and unconstrained models constructed from a data set. This approach has the benefit of being immune to physical assumptions about the data acquired. However, linear regression can suffer from overfitting, which happens when errors from the measurements, such as noise, are incorporated in the model instead of the linear relationship between the variables. K-fold cross validation was used to test for overfitting, with the standard k = 10. The data was randomly divided into 10 subsets of equal size. The algorithm consists of 10 iterations, where for each iteration the model (Eq. (6)) is trained using the merged data from the other 9 subsets and tested (Eq. (7)) on the left-out subset. For each iteration a different subset is left out such that each data point is used 9 times for training and once for testing. For each iteration, the out-of-sample error is computed between S*in_test and Sin_test (Eq. (5)). The mean and the standard deviation of these 10 errors are used to evaluate the performance of the algorithm. For data outside the initial calibration, an averaged M*, computed from the 10 training iterations, is used.
Since the individual values in the 4-by-4 Mueller matrix M* can be any real number, it is important to check that the transmittance is greater than zero and less than one, i.e. that the optical element does not amplify light or produce negative intensities. The Mueller matrix should also not overpolarize the light, i.e. produce Stokes vectors with DoP > 1. These two cases may happen if the model tries to solve for data that has been acquired incorrectly or is significantly noisy.
2.4 Optical characterization
A thorough optical characterization of optical elements in a customized optical setup is critical, particularly when the optical elements are intended for use outside of their conventional use, e.g. near-infrared fluorescence or polarization imaging, since commercially available optical specifications may not give enough information to assess whether a given element is adequate for the system. We performed a series of optical tests on both endoscopes to completely evaluate the endoscopes’ optical performance. The optical transmission as a function of wavelength, the DOF number, the MTF curve, and the optical resolution were measured. These metrics delineate the characteristics of the sensor behind the endoscope as well as appropriate experimental conditions.
The optical setup shown in Fig. 3 was utilized to measure the spectral response. The setup consisted of (1) a monochromator (Acton SP2150, Princeton Instruments), (2) an integrating sphere (819D-SF-4, Newport), (3) an adjustable iris (SM2D25, Thorlabs), (4) an aspheric collimating lens (ACL7560, Thorlabs), and (5) a calibrated photodiode (S130C, Thorlabs) driven by (6) a power meter (PM100D, Thorlabs). The optical power was measured from 400 nm to 1000 nm with and without the endoscope of interest on the optical path to calculate a transmission ratio.
To measure the DOF numbers, the MTF curves, and the optical resolutions for the two endoscopes, a DOF target (DOF 5-15, Edmund Optics), an ISO 12233 resolution chart (1X-13A/ISO, Edmund Optics), and a 1951United States Air Force (USAF) resolution chart (USAF Resolution Target, Edmund Optics) were used, respectively. For all three experiments the endoscopes were directly coupled to a digital camera (B1923, Imperx) containing a CCD sensor (KAI-02170, ON Semiconductor). The sensor has an effective resolution of 1920-by-1080 pixels, with a 7.4 μm pixel pitch. For the DOF measurements the endoscope imaging axis was placed at a 45° angle with respect to the target’s plane. The endoscope was focused on top of the target such that the focused portion represented the uppermost portion of the field depth. No additional iris elements were added to the optical path to keep the endoscopes’ DOFs unmodified. For the MTF measurements, different sections of the ISO 12233 resolution chart were used as knife-edge targets to compute individual MTF curves, and their response was averaged and reported. For the resolution measurements, high contrast images were taken of the 1951 USAF resolution chart and the standard resolution formula was used to calculate the number of line pairs per mm, lp/mm = 2group + (element-1)/6.
3. Results and discussion
3.1 Polarization
Permutations of different rotation angles for the linear polarizer and retarder were used to generate a Stokes vector cloud that served as the input polarization states for the endoscopes. A representation of these 1,800 input data points on the Poincaré sphere is shown in Fig. 4(a). Figures 4(b) and 4(c) show the output Stokes vectors after passing through the 5 mm and 10 mm endoscopes, respectively. The data in Fig. 4 is clustered into 10 groups of color-coded data points. This is done solely with the purpose of visually identifying the polarization migration of the initial input Stokes vectors (Fig. 4(a)) caused by the rigid endoscopes to the final output Stokes vectors (Figs. 4(b) and 4(c)); i.e., this color coding indicates corresponding pairs of input and output Stokes vectors. From Fig. 4 the significant change of polarization is evident, e.g. the dark blue cluster has a relatively low and constant DoCP for all its members at the input; however the outputs for both endoscopes show highly variable DoCP that approaches one on some data points. This effect is more pronounced in the 10 mm endoscope. The cluster arrangement suggests that the data has suffered a 2 DOF rotation around the Poincaré sphere. To further understand the consequences of not using a polarization calibration scheme to counteract this data rotation, the output figures of merit have been plotted as a function of the input figures of merit. Since this data is multi-dimensional, we have set some of the input parameters to a fixed value while sweeping one of the input variables. This is analogous to plotting data points from the Poincaré sphere that lie on a particular vector. Figure 5 shows the output AoP as a function of the input AoP with a constant input DoLP of one for both endoscopes. The AoPs have not only suffered a significant shift but lost their linearity. Furthermore, for the 5 mm endoscope the relationship is monotonically decreasing. A misreading on the AoP can lead to misalignments of the optical components and improper tissue pairing for different DoLPs. Figure 6 shows the output DoLP as a function of the input DoLP with a constant input DoP and AoP of one and zero, respectively, for both endoscopes. For the 10 mm endoscope the relationship is monotonically decreasing. Also, the high output DoLP at low input DoLPs, for both endoscopes, suggests a birefringent effect. Figure 7 shows the output DoLP and the output DoCP as a function of the input AoP with a constant input DoLP of one. Figure 7 confirms the retardance behavior of the endoscopes. This output response seems to be periodic, where the DoLP decreases below 40% and 20% and the DoCP increases from a null value to above 85% and 95% for the 5 mm and 10 mm endoscope, respectively. The birefringent effect is more pronounced on the 10 mm than the 5 mm endoscope, due to the larger optics used, causing the light to travel farther in a birefringent medium. These graphs show the alarming change of the input polarization state caused by the endoscopes. The polarimeter readings will fall outside the acceptable tolerance for most of the polarimeter’s readable polarization domain. It is imperative to characterize and apply a calibration scheme to these optics if they are intended for polarimetry in biomedical applications.
Fig. 4 Data plotted on the Poincaré sphere. Input Stokes vector data (a) and output Stokes vector data for the 5 mm (b) and 10 mm (c) endoscopes. Three-dimensional representation (left) and two-dimensional profiles (right). The data is clustered into 10 groups of color-coded data points with the purpose of visually identifying the polarization migration of the initial input Stokes vectors (a) caused by the rigid endoscopes to the final output Stokes vectors (b and c); i.e., this color coding indicates corresponding pairs of input and output Stokes vectors.
Fig. 6 Output DoLP vs. input DoLP, with DoLP = 0 & AoP = 0, for the 5 mm (left) and 10 mm endoscope (right).
The calibration results for the double retarder model are shown in Table 1 and Fig. 8 for both endoscopes. Table 1 shows the angle parameters estimated by the double rotated retarder model for the two rotated retarders, the obtained Mueller matrices characterizing the endoscopes obtained by applying the angle parameters in the model, and the error (Eq. (5)) and standard deviation between the input and the reconstructed input Stokes vectors using the obtained Mueller matrix. Figure 8 shows the input Stokes vector data and the reconstructed input Stokes vector data on the Poincaré sphere as well as the error histograms. For both endoscopes the mean error is less than 4%, meaning that the error on the DoCP, DoLP, and DoP has been reduced to less than 4% and the standard deviation is significantly lower, indicating that there are few outliers that fall outside of the double retarder model. The calibration results for the linear regression model are analogously shown in Table 2 and Fig. 9. Table 2 shows the average Mueller matrices characterizing the endoscopes and average cross validation error (Eq. (5)) between the input and the reconstructed input Stokes vectors in the test data sets obtained by the 10 iterations of the linear regression model as well as the standard deviation in the cross validation errors. For the linear regression model, the Poincaré sphere and the error histograms show the reconstructed data and errors on test data, from a single iteration out of 10, of the cross-validation algorithm. For both endoscopes the cross-validation error is less than 3% with very low standard deviation. These errors may be due to variation of the field of view caused by the optics’ non-uniform spatial response and could be minimized by using a spatially discretized calibration scheme. These results indicate that the linear regression method will out-perform the double rotated retarder model in additional test data.
Table 1. Double rotated retarder model calibration results.
Fig. 8 Input Stokes vector data (red cubes) and reconstructed input Stokes vector data (green spheres) on the Poincaré sphere (top) with error histograms (bottom) for the 5 mm (left) and the 10 mm (right) endoscopes using the double rotated retarder model.
Table 2. Linear regression model calibration results.
Fig. 9 Input Stokes vector data (red cubes) and reconstructed input Stokes vector data (green spheres) on the Poincaré sphere (top) for one cross-validation iteration with error histograms (bottom) for the 5 mm (left) and the 10 mm (right) endoscope using the linear regression model.
It is interesting that both models output very similar Mueller matrices for both endoscopes. This similarity is preserved on each of the 16 matrices’ elements. Note that no physical assumptions were made on the linear regression model. Elements on the first row and column, with the exception of (1,1), are virtually set to zero, indicating non-significant diattenuation effects. As well, the other 9 elements have similar values that vary by a few percentage points between models. This suggests that the hypothesis of modeling the endoscopes as retarders is accurate. However, the linear regression algorithm has lower error than the double rotated retarder model. Considering that the linear regression used a cross-validation approach to minimize overfitting and the low standard deviation over the out-of-sample errors, it is reasonable to hypothesize that the linear regression algorithm is also modeling other smaller retarder effects caused by the rest of the inner endoscope optics, e.g. small lenses and coatings. The double retarder model can be expanded to include additional DOFs, e.g. diattenuation or extra retardation effects, by adding Mueller matrices to the model; however, a reasonable assumption about the endoscope’s polarization optics needs to be made, while the linear regression model yields the advantage of treating the endoscope as a black box with unknown individual polarization elements, but this powerful scheme can suffer from overfitting. It is important to notice as well that both models agree that the last column and last row of the Mueller matrices contain non-zero elements; i.e., energy has been exchanged between the linearly and circularly polarized components. Therefore, to reconstruct the initial input Stokes vector, it is necessary to measure all four parameters of the output Stokes vector, even if the input S3 parameter is not of interest or is known to be null. In other words, a full polarimeter [33–35] is needed for polarimetry with endoscopes that have retardance effects, whereas most real-time polarimeters in the literature capture only the first three Stokes parameters [19].
3.2 Optical characterization
Figure 10 shows the spectral responses of the two endoscopes, where P and Pe indicate the optical power generated by the photon flux at the input and output windows, respectively. The 5 mm endoscope has a lower transmittance than the 10 mm endoscope due to its smaller aperture. The maximum transmittances are ~6% and ~16% for the 5 mm and 10 mm endoscopes, respectively. These low maximum transmission ratios are to be expected from optical systems with low optical apertures and long optical trains. Both endoscopes reach maximum transmittance around 660 nm which may be optimal for tissue hues; however, after 670 nm both transmittances decay rapidly— to ~2% and ~6%, respectively, at 800 nm. If the endoscopes are intended for use in a complementary imaging modality, such as near-infrared fluorescence imaging, the imaging sensor would need to have an increased fluorophore sensitivity to compensate for this low transmission ratio. The near-infrared fluorescence imaging setting is already limited by the preference of low dye concentrations, low fluorophore quantum yields, and low sensor quantum efficiencies in the near-infrared spectrum. Endoscopes with a more generous transmittance curve would be preferred for this imaging modality.
Fig. 10 Spectral response as a function of wavelength for the 5 mm (left) and 10 mm (right) endoscope.
Figure 11 shows the images acquired from the DOF target for both endoscopes. The 5 mm endoscope resolves 5 lp/mm at ~9 mm while the 10 mm endoscope can only resolve 5 lp/mm at ~4 mm. The DOF gain is attributed again to the smaller aperture on the 5 mm endoscope. This transmittance versus DOF trade-off is a typical dilemma in optical setup designs, especially when real-time frame rates are desired. Figures 12(a) and 12(b) show the sections of the ISO 12233 resolution chart that were used as knife-edge targets to compute the MTF curves for the 5 mm and 10 mm endoscope, respectively. Figure 12(c) shows the MTF plots for both endoscopes. The normalized spatial frequency is calculated as the spatial frequency divided by the cutoff or Nyquist frequency which is half-cycle per pixel pitch. The 5 mm endoscope has an MTF of 44.6% at a normalized spatial frequency of 0.1 while the 10 mm endoscope has an MTF of 69.5% at the same normalized spatial frequency. This difference is again attributed to the higher aperture on the 10 mm endoscope in comparison to the 5 mm endoscope. As expected, the endoscopes’ MTF curves are lower than commercial lenses, but such inferior performance is expected from this complex and compact set of optics. Figure 13 shows the images acquired from the 1951 USAF resolution chart for the 5 mm (a) and 10 mm (b) endoscopes. The 5 mm endoscope can resolve the 2nd element in group 3, giving 8.97 lp/mm or 55.68 µm/l. This is equivalent to an imaging pixel with a 22.51 µm pitch. For the 10 mm endoscope, the smallest resolved pattern is the 5th element in group 3, giving 12.69 lp/mm or 39.37 µm/l. This is equivalent to an imaging pixel with an 18.51 µm pitch. Both of these results are far from optimal, i.e. the sensor’s pixel pitch of 7.4 µm. However, knowing this information can influence the optical design in a positive way. It is possible to use sensors with higher pixel pitch (and hence higher quantum efficiency), leading to better performance in low light conditions which translates to higher polarization and fluorophore sensitivity, without compromising the overall system’s optical resolution. The 10 mm endoscope has much higher transmittance, a better MTF curve, and higher optical resolution, but it is outperformed on the DOF measurement. These comparisons should be kept among endoscopes built similarly but with different apertures. However, if the experimental setup permits the utilization of an endoscope with a bigger aperture, the DOF can be improved by adding a variable iris on the optical path, at the expense of accuracy in the other metrics.
Fig. 12 Sample images used to calculate the MTF curve for the 5 mm (a) and 10 mm (b) endoscope and MTF curve plots for the 5 mm and 10 mm endoscope (c).
Fig. 13 1951 USAF resolution chart images for the 5 mm (a) and 10 mm (b) endoscope and magnified versions (right).
4. Conclusions
Advances in nanotechnology have led to the proliferation of a vast range of polarization-sensitive imagers or polarimeters suitable for emerging biomedical applications. These polarimeters can be used in conjunction with rigid endoscopes or other optics to create a tailored imaging system for specific applications. A trimodal endoscopic imaging technique has been proven to increase the overall sensitivity and specificity of identifying cancerous tissue in the diagnosis of colitis-associated cancer. However, little effort has been made to evaluate and characterize the optical and polarization properties of rigid endoscopes or additional optics that couple with these polarimeters. Previous works do not delineate comprehensive calibration methods; consequently the errors have not been qualitatively documented. Here, we present a framework to analyze the endoscopes’ optical limitations and influence over polarized light. Two calibration schemes are shown to reconstruct the original polarization state of interest before light goes through the endoscope. These calibration methods can easily be expanded to flexible endoscopes given that polarization-maintaining optical fibers are used and the rotation of the distal tip coordinate system with respect to the proximal tip coordinate system is known.
5. Data availability
All data supporting the findings of this study is uploaded on the Dryad website for general access.
Funding
National Science Foundation (NSF) (1636028, 1603933).
References and links
1. D. H. Goldstein, Polarized light (CRC press, 2016).
2. J. S. Tyo, D. L. Goldstein, D. B. Chenault, and J. A. Shaw, “Review of passive imaging polarimetry for remote sensing applications,” Appl. Opt. 45(22), 5453–5469 (2006). [CrossRef] [PubMed]
3. M. Garcia, S. Gao, C. Edmiston, T. York, and V. Gruev, “A 1300× 800, 700 mW, 30 fps spectral polarization imager,” in IEEE International Symposium on Circuits and Systems (ISCAS), (IEEE, 2015), 1106–1109. [CrossRef]
4. V. Gruev, R. Perkins, and T. York, “CCD polarization imaging sensor with aluminum nanowire optical filters,” Opt. Express 18(18), 19087–19094 (2010). [CrossRef] [PubMed]
5. S.-S. Lin, K. M. Yemelyanov, E. N. Pugh Jr, and N. Engheta, “Polarization-based and specular-reflection-based noncontact latent fingerprint imaging and lifting,” J. Opt. Soc. Am. A 23(9), 2137–2153 (2006). [CrossRef] [PubMed]
6. C. P. Huynh, A. Robles-Kelly, and E. Hancock, “Shape and refractive index recovery from single-view polarisation images,” in IEEE Conference on Computer Vision and Pattern Recognition (CVPR), (IEEE, 2010), 1229–1236. [CrossRef]
7. N. M. Garcia, I. de Erausquin, C. Edmiston, and V. Gruev, “Surface normal reconstruction using circularly polarized light,” Opt. Express 23(11), 14391–14406 (2015). [CrossRef] [PubMed]
8. S. Johnsen, Y. L. Gagnon, N. J. Marshall, T. W. Cronin, V. Gruev, and S. Powell, “Polarization vision seldom increases the sighting distance of silvery fish,” Curr. Biol. 26(16), R752–R754 (2016). [CrossRef] [PubMed]
9. S. Shwartz, E. Namer, and Y. Y. Schechner, “Blind haze separation,” in Computer Vision and Pattern Recognition, 2006 IEEE Computer Society Conference on, (IEEE, 2006), 1984–1991. [CrossRef]
10. J. S. Tyo, M. P. Rowe, E. N. Pugh Jr, and N. Engheta, “Target detection in optically scattering media by polarization-difference imaging,” Appl. Opt. 35(11), 1855–1870 (1996). [CrossRef] [PubMed]
11. S. Tominaga and A. Kimachi, “Polarization imaging for material classification,” Opt. Eng. 47(12), 123201 (2008). [CrossRef]
12. E. Salomatina-Motts, V. Neel, and A. Yaroslavskaya, “Multimodal polarization system for imaging skin cancer,” Opt. Spectrosc. 107(6), 884–890 (2009). [CrossRef]
13. T. York, S. B. Powell, S. Gao, L. Kahan, T. Charanya, D. Saha, N. W. Roberts, T. W. Cronin, J. Marshall, S. Achilefu, S. P. Lake, B. Raman, and V. Gruev, “Bioinspired polarization imaging sensors: from circuits and optics to signal processing algorithms and biomedical applications,” Proc IEEE Inst Electr Electron Eng 102(10), 1450–1469 (2014). [CrossRef] [PubMed]
14. P. Doradla, K. Alavi, C. Joseph, and R. Giles, “Single-channel prototype terahertz endoscopic system,” J. Biomed. Opt. 19(8), 080501 (2014). [CrossRef] [PubMed]
15. P. Doradla, K. Alavi, C. S. Joseph, and R. H. Giles, “Development of terahertz endoscopic system for cancer detection,” in SPIE OPTO (International Society for Optics and Photonics, 2016), paper 97470F.
16. J. F. de Boer and T. E. Milner, “Review of polarization sensitive optical coherence tomography and Stokes vector determination,” J. Biomed. Opt. 7(3), 359–371 (2002). [CrossRef] [PubMed]
17. M. Pierce, M. Shishkov, B. Park, N. Nassif, B. Bouma, G. Tearney, and J. de Boer, “Effects of sample arm motion in endoscopic polarization-sensitive optical coherence tomography,” Opt. Express 13(15), 5739–5749 (2005). [CrossRef] [PubMed]
18. B. Kunnen, C. Macdonald, A. Doronin, S. Jacques, M. Eccles, and I. Meglinski, “Application of circularly polarized light for non-invasive diagnosis of cancerous tissues and turbid tissue-like scattering media,” J. Biophotonics 8(4), 317–323 (2015). [CrossRef] [PubMed]
19. T. Charanya, T. York, S. Bloch, G. Sudlow, K. Liang, M. Garcia, W. J. Akers, D. Rubin, V. Gruev, and S. Achilefu, “Trimodal color-fluorescence-polarization endoscopy aided by a tumor selective molecular probe accurately detects flat lesions in colitis-associated cancer,” J. Biomed. Opt. 19(12), 126002 (2014). [CrossRef] [PubMed]
20. J. Qi, C. Barrière, T. C. Wood, and D. S. Elson, “Polarized multispectral imaging in a rigid endoscope based on elastic light scattering spectroscopy,” Biomed. Opt. Express 3(9), 2087–2099 (2012). [CrossRef] [PubMed]
21. J. Glatz, J. Varga, P. B. Garcia-Allende, M. Koch, F. R. Greten, and V. Ntziachristos, “Concurrent video-rate color and near-infrared fluorescence laparoscopy,” J. Biomed. Opt. 18(10), 101302 (2013). [CrossRef] [PubMed]
22. P. B. Garcia-Allende, J. Glatz, M. Koch, J. J. Tjalma, E. Hartmans, A. G. Terwisscha van Scheltinga, P. Symvoulidis, G. M. van Dam, W. B. Nagengast, and V. Ntziachristos, “Towards clinically translatable NIR fluorescence molecular guidance for colonoscopy,” Biomed. Opt. Express 5(1), 78–92 (2013). [CrossRef] [PubMed]
23. E. de Boer, N. J. Harlaar, A. Taruttis, W. B. Nagengast, E. L. Rosenthal, V. Ntziachristos, and G. M. van Dam, “Optical innovations in surgery,” Br. J. Surg. 102(2), e56–e72 (2015). [CrossRef] [PubMed]
24. T. C. Wood and D. S. Elson, “Polarization response measurement and simulation of rigid endoscopes,” Biomed. Opt. Express 1(2), 463–470 (2010). [CrossRef] [PubMed]
25. M. Garcia and V. Gruev, “Optical characterization and polarization calibration for rigid endoscopes,” in Proc. of SPIE (SPIE, 2017), 1004008.
26. P. Cappabianca, L. M. Cavallo, O. de Divitiis, D. Solari, F. Esposito, and A. Colao, “Endoscopic pituitary surgery,” Pituitary 11(4), 385–390 (2008). [CrossRef] [PubMed]
27. J. D. Raman, J. A. Cadeddu, P. Rao, and A. Rane, “Single-incision laparoscopic surgery: initial urological experience and comparison with natural-orifice transluminal endoscopic surgery,” BJU Int. 101(12), 1493–1496 (2008). [CrossRef] [PubMed]
28. C. W. Cha, “Surgical instrument for orthopedic surgery,” (Google Patents, 2012).
29. T. Tanei, N. Nakahara, S. Takebayashi, M. Hirano, T. Nagatani, T. Nishihata, and T. Wakabayashi, “Endoscopic biopsy for lesions located in the parenchyma of the brain: preoperative planning based on stereotactic methods. Technical note,” Neurol. Med. Chir. (Tokyo) 52(8), 617–621 (2012). [CrossRef] [PubMed]
30. Y. Akiyama, M. Wanibuchi, T. Mikami, Y. Horita, K. Komatsu, K. Suzuki, S. Otaki, and N. Mikuni, “Rigid endoscopic resection of deep-seated or intraventricular brain tumors,” Neurol. Res. 37(3), 278–282 (2015). [CrossRef] [PubMed]
31. F. L. Roy-Bréhonnet, B. L. Jeune, P. Eliès, J. Cariou, and J. Lotrian, “Optical media and target characterization by Mueller matrix decomposition,” J. Phys. D Appl. Phys. 29(1), 34–38 (1996). [CrossRef]
32. D. F. Shanno, “Conditioning of quasi-Newton methods for function minimization,” Math. Comput. 24(111), 647–656 (1970). [CrossRef]
33. W.-L. Hsu, G. Myhre, K. Balakrishnan, N. Brock, M. Ibn-Elhaj, and S. Pau, “Full-Stokes imaging polarimeter using an array of elliptical polarizer,” Opt. Express 22(3), 3063–3074 (2014). [CrossRef] [PubMed]
34. W.-L. Hsu, J. Davis, K. Balakrishnan, M. Ibn-Elhaj, S. Kroto, N. Brock, and S. Pau, “Polarization microscope using a near infrared full-Stokes imaging polarimeter,” Opt. Express 23(4), 4357–4368 (2015). [CrossRef] [PubMed]
35. X. Tu, L. Jiang, M. Ibn-Elhaj, and S. Pau, “Design, fabrication and testing of achromatic elliptical polarizer,” Opt. Express 25(9), 10355–10367 (2017). [CrossRef] [PubMed]
