Open Access
Add to BibSonomy
Abstract
We propose a snapshot hyperspectral imaging system and methods for skin morphological feature analysis and real-time monitoring of skin activities. The analysis method includes a strategy using weighted subtractions between sub-channel images to extract absorption information due to specific chromophores within skin tissue, for example hemoglobin and melanin. Based on morphological analysis results, we carry out real-time monitoring of the skin features to verify the ability of this method to provide temporal responses of the skin tissue activities, which is experimentally shown to be useful in the measurement of heartrate, monitoring of the tissue recovery after a body exercise, and studying of the tissue response due to a vascular occlusion. Compared to conventional multispectral imaging system, the proposed system improves the device simplicity and is immune to motion artifacts. Coupled with the extraction algorithms, the hyperspectral imaging promises a robust skin assessment tool with abilities for qualitative visualization and potentially quantitative analysis of skin features, useful in the applications of cosmetics and clinical dermatology.
© 2019 Optical Society of America under the terms of the OSA Open Access Publishing Agreement
1. Introduction
Being the most superficial and largest organ in the human body, the skin is closely related to individual physical health. Its features, either morphological or functional, may be used as biomarkers for the diagnosis of, for example, dermatosis [1], diabetes [2], and even cardiovascular function [3]. In addition, the appearance of skin can directly impact personal social and emotional wellbeing [4]. The anomalous appearance of chromophore contents, including hemoglobin and melanin, has been often used as the target in the analysis of the skin features [5]. Currently, in clinical diagnosis, dermatologists mainly recognize and assess skin features with the naked eyes [6]. However, the naked-eye assessment highly depends on the color perception and sensitivity of human eyes, which may be confounded by both the discernments among the assessors and various complexion basements of patients [7]. Notwithstanding the concern on this vulnerability, objective analysis is sometimes required in clinical assessments like quantitative diagnosis and prognosis [8].
In order to provide an objective and stable skin assessment, researchers have attempted the use of optical imaging techniques [9,10]. Among them, multispectral imaging has been increasingly considered as one of the most feasible and appealing strategies [11,12]. Due to spectral absorption characteristics of various chromophores within skin tissue, selective spectral bands can be used to contrast them, for example hemoglobin and melanin [13,14]. Currently, there are a number of dedicated instrumentations that were developed. For example, Diebele et al used solid-state liquid crystal to realize spectral imaging in the clinical evaluation of melanomas and common nevi [15]. Spigulis et al applied color LED to provide illuminations for smartphone-based skin assessment [16]. Common to their system setup includes a camera and light sources in multiple spectra bands which are offered by laser light sources or selected by optical filters [17]. In practical applications, the skin is consecutively illuminated with multispectral light sources, and the reflections are recorded by the camera independently [18,19]. In this mode, additional devices are needed to provide light illumination at different wavebands, which increases the complexity of imaging system. More importantly, the inevitable tissue motion and device movements would introduce motion artifacts that often contaminate the multispectral imaging results, impacting the analysis accuracy and stability [20]. Alternatively, an optical tunable-filter can be employed to provide the needed wavelength bands to illuminate the skin. However, wavelength-selection by the optical filter still requires tens or even hundreds of milliseconds, often not sufficient to eliminate the motion artifacts, thus, complicating the data processing of multiband spectra to achieve accurate qualitative and quantitative measures. Therefore, there is a demand for the imaging system that can image fast enough, so that the motion artifacts can be tolerated.
In this study, we propose a snapshot hyperspectral imaging system for skin feature morphological analyses and temporal quantitative monitoring. This system uses white light sources to provide illumination and collects the skin reflection images with a snapshot hyperspectral camera. The camera is assembled from a fast CMOS sensor and a Fabry-Perrot interference filter array, giving 16 specific sensitive bands covering a spectral range from 470 nm to 630 nm. Without using multiple light sources or tunable filters, the proposed system is simple and easy to operate. Besides, the snapshot imaging mode completely avoids motion artifacts between individual hyperspectral imaging frames, allowing more accurate estimation of desired chromophores. By applying weighted subtractions to the sub-channel images, the chromophore absorption maps of the skin features can be extracted to realize their visualization and further potential quantitative analysis.
2. Materials and methods
2.1 Experimental setup
A schematic setup of the snapshot hyperspectral imaging system is shown in Fig. 1(a), where it employs a white light source (U-LH100-3, Olympus Corporation, Japan) for illumination and a snapshot camera (MQ022HG-IM-SM4X4-VIS, XIMEA, Germany) for capturing the skin reflection images. A 35 mm VIS-NIR prime lens (67-716, Edmund Optics) was mounted in front of the camera to relay the reflection from the sample onto the sensor array at the camera. Polarizer and analyzer were respectively placed before the light source and camera lens to control the polarization states of the lights in both the illumination and detection paths.
Fig. 1. (a) Schematic of the hyperspectral imaging system that consists of a light source and a 16-channel hyperspectral camera, with wavelength bands at each channel shown in the top left. (b) The manufacturer’s data of wavelength-dependent sensitivity for 16 channels in the hyperspectral camera. (c) Waveband-based multispectral images (right 16 columns, representing 16 sub-spectral channels) of self-printed color chart as shown in the left most column, captured by the hyperspectral camera. (d) Spectral power distribution of white light source that is used in this study. (e) Absorption spectra of oxyhemoglobin (oxyHb), deoxyhemoglobin (deoxyHb) and melanin.
The hyperspectral camera houses a CMOS sensor with 2048×1088 pixels, which is covered by a Fabry-Perrot interference filter array that separates the sensor array into 512×272 super-pixels, each containing a 4×4 pixel-matrix that are sensitive to 16 wavelength bands, respectively. The wavelength-dependent sensitivity of each wavelength band is shown in Fig. 1(b), which provides the base for more precise multispectral imaging than conventional RGB mode cameras. Figure 1(c) shows the waveband-based 16-channel images (right columns labeled as 1, 2, …, 16) of a self-printed color chart with 10 color blocks (left column), captured by this hyperspectral camera, where it can be seen that the same color block shows different reflectance at each individual wavebands. The maximum frame rate that the camera can operate is up to 170 frames per second, suitable for high-speed imaging and real-time monitoring. During imaging, the skin surface was continuously and uniformly illuminated by the white light from a halogen lamp. The spectral power distribution of the light source is shown in Fig. 1(d). A LabVIEW control program was developed to control the system and data collection from the camera with a 10-bit digital depth.
During image acquisition, we placed a polarizer at the output of light source and an analyzer on the camera. The polarizer and analyzer were orthogonally oriented to eliminate the specular reflection at the surface of skin so that the reflection signal from within the tissue sample can be maximized. At the camera, the aperture of prime lens was set to be f2 and the exposure time of every snapshot was set to be 100 ms. Depending on the skin color, the illumination power was manually adjusted so that the images captured by the camera were with adequate and consistent brightness. In real-time monitoring mode, the data was sequentially collected, and the framerate of the camera was set to be 10 fps.
2.2 Image acquisitions and post-processing
After raw data acquisition, the image was divided into 16 sub-channel images according to the designated wavelength bands. According to the optical property of the skin, a series of processing steps between different sub-channel images were conducted to spatially extract the absorption information. We assumed that melanin and hemoglobin are the dominant chromophores in the skin. A sub-channel mixing strategy was employed to separate the absorption information from each other. Here, we show a scheme of weighted subtraction among the selected sub-channels to demonstrate the proof concept. In this scheme in order to extract the blood absorption information, several red-light-sensitive sub-channels, including bands 8 (615 nm), 9 (625 nm), 10 (603 nm), 11 (595 nm), were selected and subtracted one by one from green-light-sensitive sub-channels, including bands 6 (556 nm), 7 (544 nm), 12 (529 nm), 13 (543 nm). As shown in Fig. 1(e), the hemoglobin shows absorption efficiency that changes differently with melanin from the green to red bands, the weighted subtraction can minimize or eliminate the melanocytic nevus absorption to show the blood absorption [21]. The weighted subtraction is expressed as:
In the real-time monitoring experiments, we extracted blood absorption information from snapshot sequences and conducted quantitative measurements to analyze the skin hemodynamics during heartbeat cycle, exercise recovery and vascular occlusion. Meanwhile, in the vascular occlusion monitoring, besides the blood absorption, we also estimated oxygen saturation $Sa{O_2}$ with three-band analysis model. First, the hyperspectral reflectance R was calculated as:
where $R({x,y,i} )$ is the reflectance in band $i$ at point $({x,y} )$. $I({x,y,i} )$ is the reflection intensity in band i at point $({x,y} )$ of the image. ${I_B}({x,y,i} )$ is the effect caused by ambient light, which was measured with the light source switched off. ${I_0}(i )$ is the reference intensity which was measured from the polymer white diffuser standard (SphereOptics GmbH, 95% Reflectance) under white light illumination. Then, the reflectance was converted to absorbance $A$ following Eq. (2). The attenuation of the incident light within skin tissue is approximated as:3. Results
A representative set of sub-channel images of skin captured by the hyperspectral camera under white light exposure is shown in Fig. 2. All the bands were recorded simultaneously, which makes the data acquisition inherently immune to inter-spectral motion artifacts. Based on these sub-channel images, weighted subtractions can be directly applied to extract the blood and melanocytic nevus concentration information for morphological analysis. Furthermore, based on extracted blood absorption information, we conducted real-time monitoring of the variations in skin under various conditions, including heartrate cycle, the exercise recovery and the vascular occlusion.
Fig. 2. Sixteen subchannel images captured by the hyperspectral camera from a volunteer skin under white light illumination.
3.1 Morphological analysis
There are several common types of skin features that can be analyzed from the hyperspectral images, including melanocytic nevus, pimples, nonmalignant red spots, etc. Some of them show abnormally high absorptions due to melanocytic nevus, while others can be characterized by strong hemoglobin absorption. Therefore, in spatial morphological analysis, we conducted absorption mapping for these two chromophores to recognize skin features.
3.1.1 Blood absorption
As a dominant chromophore in the blood, hemoglobin is associated with blood located in dermis and hypodermis, causing the redness appearance of the skin depending on its concentration. Since we need to capture the reflection images from the skin tissue beyond epidermis, green and red lights with deeper penetration are suitable for the hemoglobin analysis. To extract and enhance the contrast of hemoglobin information, the absorption caused by other chromophores, mainly melanin, should be minimized or eliminated. From green-wavelength band around 550 nm to red wavelength band over 600 nm, the absorption efficiencies of melanin and hemoglobin vary in different trends (Fig. 1(d)). Therefore, with a suitable ratio, the weighted subtractions between green and red sub-channels are applied to extract the hemoglobin absorption information from the melanin absorption. The subtraction ratios in experiments were calculated with Eq. (1) and calibrated according to the different band sensitivities of the camera. The blood absorption extraction results are shown in Fig. 3. Three cases of initial images (band 10 extracted from the hyperspectral raw images) under white light illumination are shown in Fig. 3(a). In Fig. 3(a)–1, both melanocytic nevi and pimples exist on the skin. In Fig. 3(a)–2, there is a light red spot on the cheek and a nevus on the right corner of the mouth. There are only pimples on the skin in Fig. 3(a)–3. The corresponding results of extracted blood absorption information from weighted subtractions are shown in Fig. 3(b). It can be observed that melanocytic nevi and some other features with large amounts of melanin, including hairs, eyebrows, eyelashes, and beards, are all eliminated in the extracted hemoglobin absorption information maps. The red spots represent the positions where there is high hemoglobin content and their locations match well with corresponding features in raw data, which can be verified in overlaid images in Fig. 3(c). To fully utilize the multiple sub-channels, we applied summation operation among the blood absorption information maps from three sub-channels combinations to realize contrast enhancements. Three cases of weighted subtraction results from combination 2(band11(529 nm)/10(603 nm)) (Fig. 3(d)) and combination 3(band 6 (556 nm)/8 (615 nm)) (Fig. 3(e)) are added together with combination 1 (band 6(556 nm)/9(625 nm)) in Fig. 3(b), the results are shown in Fig. 3(f).
Fig. 3. Extraction of blood absorption information content from snapshot hyperspectral image. (a) The raw images directly exported from the spectral band 10 in the camera, from three volunteers (marked as 1, 2, and 3, respectively). (b) Corresponding images showing blood absorption information content derived from the weighted subtraction between band 6 (556 nm) and 9 (625 nm), (c) Overlaid images of raw images and corresponding blood absorption images. (d) Corresponding images derived from weighted subtractions between band11(529 nm) and 10(603 nm). (e) Corresponding images derived from weighted subtractions between band 6 (556 nm) and 8 (615 nm). (f) Contrast enhancements of the blood absorption information content by summing the images showing in (b), (d) and (e). 1,2,3 represent three different cases in the blood extraction. Red arrow: pimples or light red spots, black arrow: melanocytic nevi.
3.1.2 Melanocytic nevus absorption
Melanocytic nevi are caused by the aggregation of melanocytes, which may present in various layers of skin. Herein, we use subtractions between blue and green channels to contrast the melanocytic nevus within dermis. In our method, bands 6 (556 nm),7 (544 nm),12 (529 nm) and 13 (543 nm) were selected and alternately subtracted from bands 0 (482 nm), 1 (494 nm), 2 (472 nm) and 3 (465 nm) to show the melanocytic nevus absorption. The subtraction ratios were manually adjusted based on the normalization of band sensitivities and absorption coefficients to exclude the interference of hemoglobin. As shown in Fig. 4, we conducted melanocytic nevus absorption information extraction on three cases and two of them are the left and right faces from the volunteer. In these cases, pimples, red spots and melanocytic nevi mixed together in the field of view (Fig. 4(a)). Through the weighted subtraction between blue and green bands, the melanocytic nevi stand out from other features, which are represented in Fig. 4(b). It is observed that pimples and light red spots are eliminated from the melanocytic nevus absorption images. The locations of extracted spots and nevi are in accordance with raw data, which can be checked in the overlaid images in Fig. 4(c). Meanwhile, we also conducted summations to melanocytic nevus absorptions from different sub-channel combinations to enhance the contrast and the results are shown in Fig. 4(d-f).
Fig. 4. Extraction of melanocytic nevus absorption information content from snapshot hyperspectral image. (a) The raw images directly exported from the spectral band 10 in the camera, from three volunteers (marked as 1, 2, and 3, respectively). (b) Corresponding images showing extracted melanocytic nevus absorption information content derived from weighted subtraction between bands 0(482 nm) and 12(543 nm). (c) Overlaid images of raw images and corresponding melanocytic nevus absorption images. (d) Corresponding images derived from weighted subtractions between bands 0(482 nm) and 11(529 nm). (e) Corresponding images derived from weighted subtractions between bands 0(482 nm) and 7(544 nm). (f) Contrast enhancements of the melanocytic nevus absorption information content by summing the images showing in (b), (d) and (e). 1,2,3 represent three different cases in the melanocytic nevus extraction. The insert in each panel is zoomed view of the regions as marked to show more details. Red arrow: pimples or light red spots, black arrow: melanocytic nevi.
3.2 Real-time monitoring
Besides the capability of extracting spatial morphological features, the hyperspectral imaging system would be also amendable to monitor the skin activities in real time due to its snapshot nature of image capturing, thanks to the hyperspectral camera. Below we demonstrate, by examples, the temporal analysis in heartrate measurements, exercise recovery, and vascular occlusion. The derived information due to blood absorption was used as a metric in the visualization and assessment of temporal responses of skin tissue.
3.2.1 Heartrate measurement
We operated the imaging system in video mode and collected a series of images during a time period for temporal analysis. The framerate was at 10 fps and the imaging duration lasted for 60s. Afterwards, the summation of the extracted information content due to blood absorption within the cheek region (square-box marked in Fig. 5(a)) was calculated and used as the blood absorption index for every frame along the time axis. Then, we Fourier-transformed the temporal profile of the blood indices over 60 s duration to show the frequency variation within the profile. As shown in Fig. 5(b), a frequency peak at around 1.19 Hz was reliably detected, which corresponded well to the heartrate. To verify the result, we carried out reference measurements with a pulse sensor and data-logger device (PowerLab 4/30, ADInstruments) during hyperspectral imaging and the result is shown in Fig. 5(c). It is clear that the heartrate frequency, indicated by the first dominant frequency peak, matches very well with the result from our hyperspectral imaging system.
Fig. 5. Heartrate measurement with snapshot-hyperspectral-camera-based multispectral imaging system. (a) The overlaid face skin image of raw image with the derived map of hemoglobin absorption information (coded according to the color bar shown in the right). The red dotted box area on the cheek is the target area for extracting heartrate from blood absorption information content derived from the hyperspectral imaging. (b) The frequency spectrum of temporal profile of blood absorption information content summed within the red-box region in (a). (c) The heartrate reference from the PowerLab pulse sensor.
3.2.2 Exercise recovery
By lengthening the time period of imaging, real-time monitoring of the skin performance in some daily activities, like exercises and fitness, may be realized. Herein, the recovery process of volunteer’s face-skin performance after he jumps up and down for ∼100 times had been monitored and analyzed. We selected the left facial skin as the imaging site and conducted a video recording, which lasted for 6 minutes. The blood absorption information content was extracted for every frame. As shown in Fig. 6(a), the blood absorption information maps and corresponding overlaid images with the raw photograph at 0,3,6 minutes during the exercise recovery process are presented, where it is observed that the information about blood absorption declines as the increase of recovery time. The real-time temporal profile of absorption information summed within the marked region of the cheek area is shown in Fig. 6(b). Obviously, the signal strength shows a rapid decrease within the 6-minute period. Furthermore, from 2 minutes to 6 minutes, we separated the recovery temporal profile into 5 segments and each one lasted for 1 minute. Within each segment, Fourier transform was performed, upon which to extract the heartrates (Fig. 6(c)). The heartrates are 1.56 Hz, 1.53 Hz, 1.51 Hz, 1.49 Hz and 1.46 Hz for 5 segments respectively. We can see that the heartrate slow-down from 2 minutes to 6 minutes, and then gradually into a resting state value, which is in accordance with our common sense.
Fig. 6. Exercise recovery monitoring with the snapshot-hyperspectral-camera-based multispectral imaging system. (a) The hemoglobin absorption information map (top row, color-coded according to color bar shown in the right) and overlaid images (bottom row) of the left facial skin at different time points of the recovery. (b) Real-time response curve of blood absorption information content during exercise recovery. The red dotted box area on the left is the targeted region for analysis. (c) The heartrate measurements of 5 recovery segments at 2, 3, 4, 5 and 6 minutes, respectively.
3.2.3 Vascular occlusion
The vascular occlusion would lead to the blood perfusion variations that can be detected from the blood absorption information content extracted during hyperspectral imaging. Here, we demonstrate its feasibility by monitoring subcutaneous vascular occlusion with the skin tissue. We selected fingers as the test site for the vascular occlusion study because occlusion can be easily realized by applying outside pressures and different fingers can act as control groups to each other. In the experiments, we used the thin rubber ring to provide outside pressures to apply vascular occlusions on the skin. In Fig. 7(a), the rubber ring worked on the mid-finger and forefinger respectively to provide effective pressure to occlude the skin perfusion. The rubber ring was banded on the mid-finger at 60s and removed at 150s. After recovering for around 120s, the banding was applied on the forefinger at 280s and relieved at 360s. From the blood absorption images at 120s and 300s, it is observed that the pressure applied on the skin caused rapid reductions of blood perfusion. After the relief of the banding, the skin blood perfusion showed a short-time over-rebounding and then recovered to the similar levels with other control group fingers, as shown in the images at 160s, 240s, 390s, and 480s. To clearly present the quantitative variation during pressure-induced perfusion occlusion, we conducted quantitative measurements to the fingers. We selected five positions (marked as A, B, C, D, E, shown in top left image of Fig. 7a) on the dorsal skin of the finger and summed the blood information content within the marked tissue region. The temporal variation curves are shown in Fig. 7(b). Position B (mid-finger) and C (forefinger) represent the experiment groups and others are control groups. The blood absorption information content as derived from hyperspectral imaging is apparently declined at the mid-finger and the forefinger under the applied pressure. The recovery processes also match with the images in Fig. 7(a). In contrast, they are generally stable during the time period.
Fig. 7. Vascular occlusion monitoring with the snapshot-hyperspectral-camera-based multispectral imaging system. (a) Finger dorsal skin blood absorption images at different time points of vascular occlusion under the outside pressure. (b) Real-time quantitative blood absorption response curves of fingers under the vascular occlusion and the contrast groups. (c) Finger dorsal $Sa{O_2}$ images at different time points of vascular occlusion under the outside pressure. (d) Real-time quantitative $Sa{O_2}$ response curves of fingers under the vascular occlusion and the contrast groups.
The $Sa{O_2}$ images (Fig. 7(c)) and quantitative curves (Fig. 7(d)) during the pressure-induced vascular occlusion show similar variation with the derived blood absorption information as shown in Fig. 7(a) and Fig. 7(b). The $Sa{O_2}$ intensities showed sharp decrease after rubber ring banding at 60s on the mid-finger and 280s on the forefinger. Before removing the rubber ring, the $Sa{O_2}$ intensities stayed in relatively low levels. After removing the rubber ring at 150s on the mid-finger and 360s on the forefinger, the $Sa{O_2}$ intensities showed quick over-rebounding in a short time and then slowly regressed to normal levels.
4. Discussions
The feature analysis is essential in the skin assessment for cosmetology and clinical diagnosis. Traditionally in clinical dermatology, dermatologists rely on naked eyes to recognize and evaluate the skin features, which lacks both accuracy and objectivity. Recently, multispectral imaging starts to draw attention for the skin feature investigations, however, the requirement on multicolor illumination not only increases the device complexity but also introduces motion artifacts into the image acquisition. To mitigate these limitations, we have developed a snapshot hyperspectral imaging system to spatially analyze the skin features and achieve real-time monitoring to their temporal variation.
The snapshot hyperspectral imaging system consisted of a snapshot hyperspectral camera and regular white illumination. In the data acquisition, multispectral images were extracted from the common frame of the reflection snapshot, which avoids the motion artifacts as often the case in traditional systems. By using a strategy of mixing selected sub-channels in the post-processing, chromophore contents of skin features can be spatially analyzed. In this study, we used subtraction operation between selected channels to demonstrate the usefulness of this strategy. It is clear that other approaches, either co-linear or non-linear mixing, may be used to further optimize the operation to maximize the outcomes.
Hemoglobin and melanin are the primary light absorbers in the skin [23–26]. The oxygenated and de-oxygenated hemoglobin exhibit distinct absorption curves across the visible wavelength window. The oxygen and hemoglobin contents within blood are often used as a biomarker for cosmetic and clinical evaluations of the skin tissue. When there are vascular malformation, dysplasia and angiorrhexis in the skin tissue, the blood absorption information derived from the hyperspectral imaging, as a surrogate of hemoglobin content, deviates from the normal state [27]. Therefore, the blood absorption information may be used as an index for both normal skin inflammations and some skin diseases, including cancers [28,29]. Melanin contents in the skin are often used in some age-related or light-induced skin damage studies [30]. Meanwhile, the malignant development of melanin also happens in melanoma, one of the most dangerous types of skin cancer [31]. In our study, the strategy of weighted subtractions was applied between selected color bands to extract the hemoglobin and melanocytic nevus absorption information contents. By subtracting red bands from green bands, the melanocytic nevus absorption was shown to be minimized in the derivation of the blood absorption content. Also, the melanocytic nevus content could be derived by weighted subtraction between blue bands and green bands. Meanwhile, the contrast of chromophore absorptions was enhanced by applying summation processing among selected sub-channel combinations. With above processing steps, we have demonstrated that the skin features can be described with light absorptions caused by chromophores, making themselves easily distinguishable from each other, which would be useful for quantitative monitoring if needed.
Benefiting from the high speed of the camera, our imaging system can work in video mode to monitor the real-time response of the skin features under various situations. For demonstration purposes, we designed monitoring experiments following the morphological analysis, upon which to provide real-time responses of the skin activities. Through the feature analysis, we have shown that the snapshot hyperspectral imaging system is capable of monitoring the skin variations, which might exist in daily life, exercise, fitness, and clinics.
Heartrate is a widely used health indicator to many body factors, including the aging diseases, emotions, body positions et al [32]. There are mainly two categories of heart rate monitors in medical devices. One of them is the electrocardiography sensor, and the other one is based on optical sensing, which is becoming more and more popular in clinics, fitness and daily monitoring nowadays for its simplicity and conveniences [33,34]. Optical heartrate sensors work based on the blood perfusion variation in the cardiac expansion and contraction cycle. Our proposed system has been shown to be capable of detecting temporal variation of blood volume by providing the derived information maps due to the skin blood absorption. We conducted temporal analysis to extract the blood absorption information from the facial skin of a volunteer. The frequency analysis of the temporal profiles clearly shows the heartrate that matched well with that measured by a standard pulse meter, indicating that our imaging system is amendable to accurately detect the subtle variation within the skin tissue caused by the heartbeat.
Fitness and exercise monitoring are increasingly becoming popular in modern daily life, drawing the interests from research scientists and commercial vendors. In our study, we used the snapshot hyperspectral imaging system to present the temporal quantitative response of the cheek blood contents after a simple exercise, and investigated the change in heartrate during the exercise recovery. In this trial, the snapshot hyperspectral imaging system was shown to capture both the acute decrease of blood information content and minute slowdown of heartrates. Compared with current exercise recovery tracking approaches like Fitbit accessories [35], our method can achieve non-contact monitoring and real-time visualization for skin features.
Vascular occlusion is the blockage of the blood vessel caused by many reasons like endogenous abnormality in the blood and vessel, or alien forces [36,37]. Vascular occlusion can lead to tissue ischemia or even tissue necrosis [38]. It can also be utilized as a treatment strategy in some diseases like reducing pressure in embolization and restricting tumor development [39]. In dermatology, it can act as the recovery index of the skin planting implants [40]. The finger dorsal skin absorption under vascular occlusions caused by outside pressures was studied in our study. The results clearly demonstrated the variations of the skin blood information content and $Sa{O_2}$ under outside pressures, which shows a useful capability of the snapshot hyperspectral imaging system in vascular occlusion monitoring. Our method provides a more compact and cost-efficient imaging system than current strategies, e.g., ultrasound and magnetic resonance imaging, for vascular occlusion monitoring [41,42]. Though some multispectral imaging systems have also been applied, like we stated above, our system would still be superior in terms of the system compactness and immunity to motion artifacts [43].
The snapshot hyperspectral imaging remains a multispectral imaging strategy but works in a simplified device and artifacts-resistant data acquisition mode. Without motion artifacts in multicolor images, the strategy of channel mixing via various operations can be directly applied without a need of complicated image registration. This improvement is essential for the accuracy of morphological analysis and the real-time monitoring capability, especially for quick and subtle variations within the skin. In this study, we analyzed the normal skin features like pimples and melanocytic nevi to demonstrate the proof concept, however further systematic investigations are needed to demonstrate its full potential in clinical settings where rich morphological details are required.
5. Conclusion
We have developed a snapshot hyperspectral imaging system and the corresponding post-processing methods for the skin feature analysis. The spatial morphological analysis demonstrated that the skin features can be depicted and distinguished with the chromophores-based light absorption. We have also shown that the system is useful in the measurement of heartrate, monitoring of exercise recovery, and investigation of the skin responses due to vascular occlusion. Meanwhile, compared with current multispectral imaging methods, our system is improved in device simplicity and is immune to motion artifacts. Based on these advantages, it should be reasonable to expect wider applications of the snapshot hyperspectral imaging system in daily life monitoring and clinical diagnosis applications.
Funding
Washington Research Foundation.
Acknowledgments
Research supported in part by Washington Research Foundation. The funding organizations had no role in the design or conduct of this research.
Disclosures
Dr. Wang discloses intellectual property owned by the Oregon Health and Science University and the University of Washington. Dr. Wang also receives research support from Colgate Palmolive Company, Carl Zeiss Meditec Inc, Tasso Inc, Moptim Inc, and Facebook technologies LLC. He is a consultant to Carl Zeiss Meditec, and Insight Photonic Solutions.
The remaining authors have no disclosures.
References
1. B. Kim, S. H. Lee, C. J. Yoon, Y. S. Gho, G. O. Ahn, and K. H. Kim, “In vivo visualization of skin inflammation by optical coherence tomography and two-photon microscopy,” Biomed. Opt. Express 6(7), 2512–2521 (2015). [CrossRef]
2. R. P. van Waateringe, B. T. Fokkens, S. N. Slagter, M. M. van der Klauw, J. V. van Vliet-Ostaptchouk, R. Graaff, A. D. Paterson, A. J. Smit, H. L. Lutgers, and B. H. R. Wolffenbuttel, “Skin autofluorescence predicts incident type 2 diabetes, cardiovascular disease and mortality in the general population,” Diabetologia 62(2), 269–280 (2019). [CrossRef]
3. F. Nemoto, H. Suzuki, J. Asai, K. Asahi, T. Katoh, T. Watanabe, Y. Tani, Y. Hayashi, Y. Kusano, K. Tanaka, and T. Miyata, “Skin autofluorescence is associated with renal function and cardiovascular diseases in pre-dialysis chronic kidney disease patients,” Nephrol., Dial., Transplant. 26(1), 214–220 (2011). [CrossRef]
4. S. C. Kellett and D. J. Gawkrodger, “The psychological and emotional impact of acne and the effect of treatment with isotretinoin,” Br. J. Dermatol. 140(2), 273–282 (1999). [CrossRef]
5. F. Vasefi, N. MacKinnon, R. B. Saager, A. J. Durkin, R. Chave, E. H. Lindsley, and D. L. Farkas, “Polarization-Sensitive Hyperspectral Imaging in vivo: A Multimode Dermoscope for Skin Analysis,” Sci. Rep. 4(1), 4924 (2015). [CrossRef]
6. A. Nkengne, J. Robic, P. Seroul, S. Gueheunneux, M. Jomier, and K. Vie, “SpectraCam®: A new polarized hyperspectral imaging system for repeatable and reproducible in vivo skin quantification of melanin, total hemoglobin, and oxygen saturation,” Skin Res. Technol. 24(1), 99–107 (2018). [CrossRef]
7. C. F. Rosen, S. L. Jacques, M. E. Stuart, and R. W. Gange, “Immediate pigment darkening: visual and reflectance spectrophotometric analysis of action spectrum,” Photochem. Photobiol. 51(5), 583–588 (1990). [CrossRef]
8. F. E. Robles, J. W. Wilson, and W. S. Warren, “Quantifying melanin spatial distribution using pump-probe microscopy and a 2-D morphological autocorrelation transformation for melanoma diagnosis,” J. Biomed. Opt. 18(12), 120502 (2013). [CrossRef]
9. A. J. Deegan and R. K. Wang, “Microvascular imaging of the skin,” Phys. Med. Biol. 64(7), 07TR01 (2019). [CrossRef]
10. P. Ghassemi, T. E. Travis, L. T. Moffatt, J. W. Shupp, and J. C. Ramella-Roman, “A polarized multispectral imaging system for quantitative assessment of hypertrophic scars,” Biomed. Opt. Express 5(10), 3337–3354 (2014). [CrossRef]
11. A. Basiri, M. Nabili, S. Mathews, A. Libin, S. Groah, H. J. Noordmans, and J. C. Ramella-Roman, “Use of a multi-spectral camera in the characterization of skin wounds,” Opt. Express 18(4), 3244–3257 (2010). [CrossRef]
12. S. Kim, D. Cho, J. Kim, M. Kim, S. Youn, J. E. Jang, M. Je, D. H. Lee, B. Lee, D. L. Farkas, and J. Y. Hwang, “Smartphone-based multispectral imaging: system development and potential for mobile skin diagnosis,” Biomed. Opt. Express 7(12), 5294–5307 (2016). [CrossRef]
13. J. M. Kainerstorfer, M. Ehler, F. Amyot, M. Hassan, S. G. Demos, V. V. Chernomordik, C. K. Hitzenberger, A. H. Gandjbakhche, and J. D. Riley, “Principal component model of multispectral data for near real-time skin chromophore mapping,” J. Biomed. Opt. 15(14), 046007 (2010). [CrossRef]
14. I. Kuzmina, I. Diebele, D. Jakovels, J. Spigulis, L. Valeine, J. Kapostinsh, and A. Berzina, “Towards noncontact skin melanoma selection by multispectral imaging analysis,” J. Biomed. Opt. 16(6), 060502 (2011). [CrossRef]
15. I. Diebele, I. Kuzmina, A. Lihachev, J. Kapostinsh, A. Derjabo, L. Valeine, and J. Spigulis, “Clinical evaluation of melanomas and common nevi by spectral imaging,” Biomed. Opt. Express 3(3), 467–472 (2012). [CrossRef]
16. J. Spigulis, “Multispectral, fluorescent and photoplethysmographic imaging for remote skin assessment,” Sensors 17(5), 1165 (2017). [CrossRef]
17. Y. Hirohara, T. Yamaguchi, H. Aoki, Y. Takahashi, N. Nakazawa, T. Mihashi, S. Sato, T. Morimoto, and T. Fujikado, “Development of fundus camera for spectral imaging using liquid crystal tunable filter,” Invest. Ophthalmol. Visual Sci. 45(13), 2418 (2004).
18. D. Kapsokalyvas, N. Bruscino, D. Alfieri, V. de Giorgi, G. Cannarozzo, R. Cicchi, D. Massi, N. Pimpinelli, and F. S. Pavone, “Spectral morphological analysis of skin lesions with a polarization multispectral dermoscope,” Opt. Express 21(4), 4826–4840 (2013). [CrossRef]
19. J. Spigulis, I. Oshina, A. Berzina, and A. Bykov, “Smartphone snapshot mapping of skin chromophores under triple-wavelength laser illumination,” J. Biomed. Opt. 22(9), 091508 (2017). [CrossRef]
20. L. Gao, R. T. Smith, and T. S. Tkaczyk, “Snapshot hyperspectral retinal camera with the Image Mapping Spectrometer (IMS),” Biomed. Opt. Express 3(1), 48–54 (2012). [CrossRef]
21. W. F. Cheong, S. A. Prahl, and A. J. Welch, “A review of the optical properties of biological tissues,” IEEE J. Quantum Electron. 26(12), 2166–2185 (1990). [CrossRef]
22. N. T. Clancy, S. Arya, D. Stoyanov, M. Singh, G. B. Hanna, and D. S. Elson, “Intraoperative measurement of bowel oxygen saturation using a multispectral imaging laparoscope,” Biomed. Opt. Express 6(10), 4179–4190 (2015). [CrossRef]
23. D. Jakovels and J. Spigulis, “2-D mapping of skin chromophores in the spectral range 500–700 nm,” J. Biophotonics 3(3), 125–129 (2010). [CrossRef]
24. X. Chen, W. Lin, C. Wang, S. Chen, J. Sheng, B. Zeng, and M. Xu, “In vivo real-time imaging of cutaneous hemoglobin concentration, oxygen saturation, scattering properties, melanin content, and epidermal thickness with visible spatially modulated light,” Biomed. Opt. Express 8(12), 5468–5482 (2017). [CrossRef]
25. G. Zonios, J. Bykowski, and N. Kollias, “Skin Melanin, Hemoglobin, and Light Scattering Properties can be Quantitatively Assessed In Vivo Using Diffuse Reflectance Spectroscopy,” J. Invest. Dermatol. 117(6), 1452–1457 (2001). [CrossRef]
26. T. Sheng-Hao, B. Paulo, D. Anthony, and K. Nikiforos, “Chromophore concentrations, absorption and scattering properties of human skin in-vivo,” Opt. Express 17(17), 14599–14617 (2009). [CrossRef]
27. M. Fujiwara, S. Shun, P. K. W. Abeygunawardhana, S. Suzuki, A. Nishiyama, K. Wada, and I. Ishimaru, “Spectroscopic imaging of blood vessels only near the skin surface for non-invasive blood glucose measurement,” in (Optical Society of America, 953714.
28. G. N. Stamatas and N. Kollias, “In vivo documentation of cutaneous inflammation using spectral imaging,” J. Biomed. Opt. 12(5), 051603 (2007). [CrossRef]
29. A. N. Yaroslavsky, V. Neel, and R. R. Anderson, “Demarcation of nonmelanoma skin cancer margins in thick excisions using multispectral polarized light imaging,” J. Invest. Dermatol. 121(2), 259–266 (2003). [CrossRef]
30. P. U. Dugel and C. N. Zimmer, “Imaging of melanin disruption in age-related macular degeneration using multispectral imaging,” Ophthalmic Surg Lasers Imaging Retina 47(2), 134–141 (2016). [CrossRef]
31. T. E. Matthews, J. W. Wilson, S. Degan, M. J. Simpson, J. Y. Jin, J. Y. Zhang, and W. S. Warren, “In vivo and ex vivo epi-mode pump-probe imaging of melanin and microvasculature,” Biomed. Opt. Express 2(6), 1576–1583 (2011). [CrossRef]
32. J. Achten and A. E. Jeukendrup, “Heart rate monitoring,” Sports Med. 33(7), 517–538 (2003). [CrossRef]
33. M. P. Barbosa, N. T. da Silva, F. M. de Azevedo, C. M. Pastre, and L. C. Vanderlei, “Comparison of Polar RS 800G heart rate monitor with P olar S810 and electrocardiogram to obtain the series of RR intervals and analysis of heart rate variability at rest,” Clin. Physiol. Funct. Imaging 36(2), 112–117 (2016). [CrossRef]
34. S. Benedetto, C. Caldato, E. Bazzan, D. C. Greenwood, V. Pensabene, and P. Actis, “Assessment of the Fitbit Charge 2 for monitoring heart rate,” PLoS One 13(2), e0192691 (2018). [CrossRef]
35. K. M. Diaz, D. J. Krupka, M. J. Chang, J. Peacock, Y. Ma, J. Goldsmith, J. E. Schwartz, and K. W. Davidson, “Fitbit: An accurate and reliable device for wireless physical activity tracking,” Int. J. Cardiol. 185(1), 138–140 (2015). [CrossRef]
36. J. Rehak and M. Rehak, “Branch retinal vein occlusion: pathogenesis, visual prognosis, and treatment modalities,” Curr. Eye Res. 33(2), 111–131 (2008). [CrossRef]
37. T. C. Pearson, “Hemorheologic considerations in the pathogenesis of vascular occlusive events in polycythemia vera,” in (Thieme Medical Publishers, Inc., 1997), pp. 433–439.
38. R.-L. Zhang, M. Chopp, H. Chen, and J. H. Garcia, “Temporal profile of ischemic tissue damage, neutrophil response, and vascular plugging following permanent and transient (2H) middle cerebral artery occlusion in the rat,” J. Neurol. Sci. 125(1), 3–10 (1994). [CrossRef]
39. J. E. Jackson, A. O. Mansfield, and D. J. Allison, “Treatment of high-flow vascular malformations by venous embolization aided by flow occlusion techniques,” Cardiovasc. Intervent. Radiol. 19(5), 323–328 (1996). [CrossRef]
40. P. N. Manson, K. K. Narayan, M. J. Im, G. B. Bulkley, and J. E. Hoopes, “Improved survival in free skin flap transfers in rats,” Surgery 99(2), 211–215 (1986).
41. P. A. Magnin, E. I. McNamara, R. W. Bowden, and R. J. Solomon, “Apparatus and methods for low-cost intravascular ultrasound imaging and for crossing severe vascular occlusions,” (Google Patents, 2016).
42. R. Hata, G. Mies, C. Wiessner, K. Fritze, D. Hesselbarth, G. Brinker, and K.-A. Hossmann, “A reproducible model of middle cerebral artery occlusion in mice: hemodynamic, biochemical, and magnetic resonance imaging,” J. Cereb. Blood Flow Metab. 18(4), 367–375 (1998). [CrossRef]
43. C. R. Tracy, J. D. Terrell, R. P. Francis, E. F. Wehner, J. Smith, M. Litorja, D. L. Hawkins, M. S. Pearle, J. A. Cadeddu, and K. J. Zuzak, “First Prize: Characterization of Renal Ischemia Using DLP® Hyperspectral Imaging: A Pilot Study Comparing Artery-Only Occlusion Versus Artery and Vein Occlusion,” Journal of Endourology 24(3), 321–325 (2010). [CrossRef]
