Open Access
Add to BibSonomy
Abstract
The green channel is usually selected as the optimal channel for vital signs monitoring in image photoplethysmography (IPPG) technology. However, some controversies arising from the different penetrability of skin tissue in visible light remain unresolved, i.e., making the optical and physiological information carried by the IPPG signals of the RGB channels inconsistent. This study clarifies that the optimal channels for different diseases are different when IPPG technology is used for disease classification. We further verified this conclusion in the classification model of heart disease and diabetes mellitus based on the random forest classification algorithm. The experimental results indicate that the green channel has a considerably excellent performance in classifying heart disease patients and the healthy with an average Accuracy value of 88.43% and an average F1score value of 93.72%. The optimal channel for classifying diabetes mellitus patients and the healthy is the red channel with an average Accuracy value of 82.12% and the average F1score value of 89.31%. Due to the limited penetration depth of the blue channel into the skin tissue, the blue channel is not as effective as the green and red channels as a disease classification channel. This investigation is of great significance to the development of IPPG technology and its application in disease classification.
© 2022 Optica Publishing Group under the terms of the Optica Open Access Publishing Agreement
1. Introduction
Image photoplethysmography (IPPG) technology enables contactless monitoring of human cardiac activities such as heart rate and blood oxygen saturation by measuring the pulse signal, i.e., the subtle color variations on the human skin tissue induced by the blood volume change through a regular RGB color camera [1]. IPPG technology is developed on the basis of traditional contact mode photoplethysmography (PPG) technology. Due to the advantages of non-contact and non-invasive monitoring, IPPG technology has been further developed to monitor human physiological and psychological states in various circumstances, even in an intensive care unit [2–6].
IPPG technology is based on the fact that blood circulation changes the number of chromophores in tissues over time, such as hemoglobin, which causes fluctuations in the light action mode of the entire spectrum. An image sensor can be used to capture the changes that the fluctuations of reflectance spectrum of the blood circulation, i.e., minute skin tones changes caused by the changes in the number of chromophores. IPPG technology is initially mainly used for monitoring heart rate. Although the IPPG signals can be extracted in the RGB channels [7,8], the green channel (G channel) possesses the best response in the image sensor, has a higher penetration depth than the blue channel (B channel), and is less affected by motion artifacts compared with the red channel (R channel) [9]. Therefore, the IPPG signals of the G channel have a high signal-to-noise ratio, which makes it the optimal channel for heart rate monitoring in the laboratory and daily life [10]. With the further development of IPPG technology, the G channel IPPG signal has even been applied in the diagnosis of complex physiological characteristics and the classification of diseases. Zhang et al. [11] extracted the IPPG signals from the videos collected by the smartphone camera. They used some characteristics of the G channel IPPG signal, including the dominant peaks, the second peaks, and the dicrotic notches to categorize 80 subjects into three blood glucose levels by implementing machine learning algorithms. Djeldjli et al. [12] also chose the G channel to evaluate waveform features of IPPG signals related to arterial stiffness and blood pressure. For the monitoring of simple physiological characteristics using the pulsation amplitudes of the IPPG signals, such as heart rate, the G channel is undoubtedly the best. However, the optical information of the G channel is relatively scarce due to the limitation of penetration capability. The transmission depth of green light is less than 1 mm, green light can only reach the dermis layer filled with capillaries. The red light (600nm–750 nm) and near-infrared light (850nm–1000 nm) mainly penetrate the deep arteries and arterioles in the dermis and subcutaneous tissue. In PPG technology, the PPG signals of the red light and near-infrared light carry more deep physiological information, thereby being used to analyze the blood lacking oxygen degree and dicrotic notches of PPG waveforms [13,14]. Age, cardiovascular disease, endothelial dysfunction, neuropathy, and diabetes-related microvascular arteriosclerosis also affect the characteristic waveforms of PPG signals [15–19]. Compared with the PPG contact mode, the signal strength of the IPPG technology in the measurement pulse is weaker due to the shorter skin investigation depth [20]. Moreover, the origin of the IPPG signal in the RGB channels has also been controversial [8,21–23]. The absorption and scattering of the interaction between the light and skin tissue exhibit dynamic changes in multi-layer skin tissue, which complicates the physiological information carried by the IPPG signals in the RGB channels. How to effectively use the IPPG signals of the RGB channels to obtain more physiological characteristic information, and how to determine the best IPPG signal channel to realize the diagnosis of a certain disease, which are of great significance for the monitoring of complex vital signs based on IPPG technology. To our best of knowledge, the rational selection of RGB channels for disease classification based on IPPG technology has not been well studied.
The purpose of our current research is to determine which channels of IPPG signals are most suitable for diagnosing people with complicated cardiovascular diseases. In this paper, we obtain the video data of subjects’ fingers involving healthy individuals and the patients with heart disease and diabetes mellitus by IPPG technology, then extract the raw IPPG signals in the RGB channels. Combining the pathological mechanism and optical information of IPPG signals in the RGB channels, we conduct the experiments on the heart disease model and diabetes mellitus model to further verify which channels of IPPG signals are the most valuable for their respective classification models. The raw IPPG signals in the RGB channels are processed slightly and then input to the random forest (RF) classification algorithm. According to the classification performance of the IPPG signals in the RGB channels, the appropriate channels as disease diagnosis channels for the heart disease classification model and diabetes mellitus classification model are determined, respectively.
2. Methods
Part of the light emitted by the light source penetrates into the skin tissue, which interacts with the multi-layer of skin tissue mainly including scattering and absorption, and the other part of the light is specularly reflected on the skin tissue surface. Since the vibration generated by the heartbeat will be transmitted to the peripheral capillaries and skin tissue along with the vascular tissue, the weak vibration will cause changes in the light absorption of the skin tissue. The imaging equipment records the weak changes of light signals during the systole and diastole of the heart over time in the form of an image. Finally, we extract IPPG signals from the video stream through video analysis and image processing technology.
Accordingly, the IPPG signal may be modeled as a weighted average of the contributions from skin tissue [24]:
Equation (1) shows a discrete simplification of the IPPG signal. In reality, the IPPG signal is an integral over-weighted contribution from multi-layer skin tissue. Therefore, the IPPG signal is calculated by the total diffusely reflected light of the skin tissue reflection model. Further, the IPPG relative amplitude in the operating wavelength λ in the diastolic and systolic of the heart is defined as the following formula [25].
where Rdd and Rds denote the total diffuse reflectance spectra of the multi-layer skin tissue corresponding to the wavelength λ in the diastolic and systolic of the heart, respectively. According to Eq. (2), the curve of IPPG signal relative amplitude versus wavelength λ is shown in Fig. 1.
Fig. 1. The curve of IPPG signal relative amplitude versus wavelength λ. The relative amplitude values of IPPG signal with characteristic wavelengths in blue light (460 nm), green light (580 nm), and red light (660 nm) are marked with corresponding colored dashed circles.
The curve shows that the IPPG signal relative amplitude of green light is larger than that of red light and blue light theoretically. This is determined by the absorption coefficient of the blood chromophores, the spectral response of the image sensor, and the penetration depth of the green light into the tissue. The chromophores absorb blue and green light the most, followed by red light. In addition, blue (460 nm) and green light (580 nm) penetrate the skin tissue up to a depth of 0.7 mm and 0.9 mm, respectively, whereas red (660 nm) light goes deeper to 1.8 mm [21,26,27]. Ordinary image sensors also have the highest response at the green light, followed by blue and red light. Therefore, the green light IPPG signal can more carry the pulsation information of the heart. IPPG signal relative amplitude of red light is much smaller (at least five-fold) than that of green light and blue light at some wavelengths.
However, although the green channel is theoretically the most suitable diagnostic channel for basic vital signs monitoring, it is questioned as a diagnostic channel for complex cardiovascular diseases due to the lack of optical information in deep tissues. Clarifying the specific manifestations in the RGB channels of IPPG signals from different pathological diseases, which can accurately classify patients with specific diseases and healthy individuals through a certain channel of IPPG signals.
3. Experiments
3.1 Experimental setup and data acquisition
Figure 2 shows the reflection model of the IPPG imaging device we implemented, which mainly consists of a visible light source, a polarizer, an analyzer, and an image sensor. Visible light (380nm–800 nm) from a halogen light source passes through a polarizer and is focused on a finger. Subsequently, the diffuse reflected light from the finger passes through an analyzer and is received by an image sensor connected to the computer. The image sensor is used to collect the finger video data, which has included a 1/1.2” CCD sensor (FLIR co. BFLY-U3-23S6C-C) and a lens (ZLKC.VM0812MP). The capture frame rate and pixel resolution of the CCD are set to 30 frames/s and 1920 pixels × 1000 pixels, respectively. The polarizer and the analyzer are used to eliminate specularly reflected light without the optical information from the finger. The distance between fingers and CCD shall be kept at 2 cm. All the subjects are asked to avoid strenuous exercise and breathe normally in a supine position on a comfortable bed for at least 10 minutes to ensure cardiovascular stability. The index finger pulp is located as the interest of region (ROI) to record video data. During the video data collection, the ROI of each subject is recorded with a duration of 30s. Finally, the IPPG signals from these video data are extracted.
Fig. 2. The skin tissue reflection model of IPPG technology. The light from a halogen light source passes through a polarizer and is focused on the index finger. Subsequently, the diffuse reflected light from the index finger passes through an analyzer and is received by a CCD connected to the computer.
The research protocol was approved by the Medical Ethics Committee of Medical and Laboratory Animal with approval code 2021-004. We obtained 136 video data from subjects including 98 healthy individuals and 38 patients (heart disease and diabetes mellitus). The healthy subjects are with an average age of 33.9, and the healthy states are verified via hospital examinations. The diseased subjects with heart disease and diabetes mellitus are from the Cardiology Department and Diabetes Center in the Taiyuan Central Hospital with an average age of 40.2 and 37.7, respectively. 14 heart disease patients with an average duration of 4.0 years, and 24 diabetes mellitus patients with an average duration of 4.8 years. The detailed information on these subjects is shown in Table 1.
Table 1. The detailed information of the subjects from the healthy, the patients with heart disease and diabetes mellitus
3.2 RGB channel characteristics of the IPPG signals from the different types of subjects
The IPPG signal is the most direct manifestation of human life activities, and its waveform and amplitude changes are important bases for assessing the cardiovascular physiological and pathological conditions of the human body. Different diseases have different mechanisms of action on vascular dynamics, which makes IPPG waveforms have obvious characteristic differences. The characteristics of IPPG signals from healthy subjects, heart disease subjects, and diabetes mellitus subjects are discussed in detail as follows.
Figure 3 shows the IPPG waveform in the RGB channels from a healthy subject. An IPPG waveform is composed of a quasi-periodic AC-component and a static DC-component. The AC-component indicates the synchronous changes in the blood volume of the heart with each heartbeat, while the DC-component is related to the absorption of light by non-pulsating parts, containing valuable information about respiration, venous flow, sympathetic nervous system activities and thermoregulation. A complete IPPG waveform contains morphologic and temporal information, specifically showing a dominant peak (systolic peak), a second peak (diastolic peak), and an obvious dicrotic notch. These characteristic parameters of a complete IPPG waveform are depicted in Fig. 3(a). In fact, the IPPG signals we extracted show high quality in the RGB channels. Thus, the detail-reserved characteristics of second peaks and dicrotic notches in an IPPG waveform are actually different in the RGB channels. We circle part of the differences in a 30s IPPG signal from a healthy subject with a black dashed original circle for display (See Fig. 3(b)).
Fig. 3. Characteristics of the IPPG signals. (a) A complete IPPG waveform in the R channel; (b) IPPG waveforms in the RGB channels.
Even within the same subject, the waveforms of the IPPG signals of each channel possess a certain difference. However, common information is still available. The information includes physiological and non-physiological information. The main characteristics of IPPG signals from the same type of subjects are consistent, including the dominant peak, the second peak, the dicrotic notch, and so on. In different types of subjects (healthy and diseased), these characteristic peaks can then be used to classify the disease. Studies have shown that these characteristics can be used for the classification of blood glucose and the related diagnosis of cardiovascular disease. Besides, the effects of non-physiological information such as motion artifacts on the RGB channels are also consistent. For the interference of non-physiological information, researchers have also developed algorithms to remove motion artifacts such as Color-Distortion Filtering (CDF) [28] and Chrominance (CHROM) [29]. Therefore, whether it is a physiological signal or non-physiological information, there is common information in the IPPG signal of each channel. Figure 3 also illustrates that the difference of amplitude between the IPPG signal of the R channel and the other two channels is not as obvious as the theoretical analysis (See Fig. 1). This is due to the fact that IPPG signal is more complex in human tissue. So far, the origin of the IPPG signal is still controversial. Skin tissue such as fingers, arms, neck, etc., have different IPPG signal strengths. In addition, melanin in the skin absorbs some of the diffuse light, resulting in IPPG signals with poor signal-to-noise ratios for darker skin tones. Different wavelengths of visible light also have different penetration depths for skin tissue. Red light has the longest wavelength and penetrates deeper into the tissue, but is less absorbed by blood. Therefore, in the relative amplitude of the IPPG signal actually obtained by the human tissue, the amplitude difference between the R channel and the other two channels becomes smaller.
Further, we applied a bandpass filter [0.4HZ−10HZ] [30] and detrending techniques to slightly pre-process the IPPG signals to avoid over-filtering the detail-reserved characteristics in the PPG waveforms [31]. The classification and diagnosis of diseases based on IPPG signals are mainly based on the characteristics of the AC-component. Therefore, the detrending techniques convert the amplitude of the IPPG signal into a relative amplitude. The amplitude differences (DC-component) of the IPPG signal are removed and normalized to within a certain range. The relative amplitude values of the IPPG signal in the RGB channels are shown in Fig. 4. In each complete IPPG waveform, the relative amplitude value of the G channel is significantly higher than that of the other two channels. However, the second peaks and dicrotic notches of the R channel are more obvious. These phenomena are found in the IPPG signals of all healthy subjects, which indicates that the physiological information and tissue optical information of the IPPG signals in the RGB channel are different. The healthy subjects are 24–45 years old in our experiment, the influence of age and other diseases may be ignored. During the diastole, the dominant peak of the IPPG signal occurs. The differences of the second peaks and dicrotic notches of IPPG signals mainly come from the interact mode of light and skin tissue, including absorption and scattering.
Fig. 4. The IPPG signal in the RGB channels is filtered and removed from the trend. The waveforms in the small box pointed by the arrow are the local features of the amplified IPPG signal.
A total of 14 heart disease subjects’ IPPG waveforms are collected in the experiment. The waveforms of the 11 subjects are mostly two peaks and a few three peaks, and the peak values of the two peaks are far apart. The remaining 3 subjects have more single peaks and a few two peaks. Figure 5 shows a 10s IPPG signal in the RGB channels from a typical heart disease subject. These incomplete IPPG waveforms are characteristics of atrial fibrillation (AF), which is mainly caused by poor heartbeat [32]. AF is the most common sustained cardiac arrhythmia. AF is characterized by rapid and irregular heartbeat frequency, and loss of effective contractile function of the atrium. Therefore, for AF-type heart disease patients, the influence of the heartbeat on the IPPG waveforms basically maintains the same trend and detailed information in the RGB channels. The peak and amplitude values of the three channels almost coincide. However, some waveforms of the R channel still show inconsistent characteristics compared with the G channel and the B channel, especially in dicrotic notches. The part details-reserved characteristics in the IPPG waveform are circled with a black dashed original circle.
Fig. 5. The IPPG waveforms in the RGB channels extracted from the heart disease subject. The part details-reserved characteristics in the IPPG waveforms are circled with a black dashed original circle.
Figure 6 shows the IPPG signal in the RGB channels from a diabetes mellitus subject. This 10s IPPG signal illustrates complex characteristics of multiple peaks. The IPPG waveforms have significant differences in the RGB channels including the amplitude values and the peaks. In terms of amplitude values, the G channel is the highest, followed by the R channel and B channel. These peaks have a certain difference, which is clearly distinguished into peaks. Part of the details differential information is circled with a black dashed original circle for display. We intuitively observe that the IPPG signals of the RGB channels present differential waveform characteristics. However, these specific characteristics can hardly be generalized. This phenomenon is caused by the long-term effect of hyperglycemia on the blood vessels. Hyperglycemia can damage nerve fibers and reduce blood supply, leading to tissue hypoxia and altered metabolism. The changes in the human skin tissue of different individuals are affected by the disease duration time and disease degree. The specific effects of hyperglycemia on each layer of skin tissue are markedly inconsistent. Therefore, the waveforms of IPPG signals are irregular and are characterized by clutter, with multiple peaks in the RGB channels in diabetes mellitus patients. Most subjects have suffered from diabetes mellitus for at least 4 years in our experiment. The effects of hyperglycemia on multi-layer skin tissue of individuals are also significantly different mainly due to the individual physique differences, the duration of the diabetes mellitus, the degree of the diabetes mellitus, and personal blood glucose endurance, and other reasons. Hence, the IPPG waveforms in the RGB channels are obviously different in each diabetes mellitus patient.
Fig. 6. The IPPG waveforms in the RGB channels extracted from a diabetes mellitus subject. Multiple peaks are shown in the RGB channels. Part of the details differential information is circled with a black dashed original circle.
According to the above analysis, we know that the IPPG signal waveforms are inconsistent in the RGB channels of healthy individuals. The inconsistency of the waveforms in the RGB channels is also observed on the IPPG signals of heart disease patients and diabetes mellitus patients. The IPPG signals vary greatly between heart disease patients and diabetes mellitus patients. Besides, the IPPG signals reflect the pathological characteristics of the above two diseases. Therefore, to achieve accurate diseases classification based on IPPG technology, a channel containing rich optical information and pathological characteristics of the IPPG signals is required. A classification algorithm is adopted to establish classification models to select the optimal IPPG signal channel for heart disease patients and diabetes mellitus patients, respectively.
4. Analysis and discussion
4.1 Random forest classification algorithm
The random forest (RF) classification algorithm is an ensemble learning algorithm, which can cope with complicated interaction structures as well as highly correlated variables [33]. An RF classifier is a collection of decision tree predictors. Each decision tree grows depending on the bootstrap and growth phases. The parameter of each node in a decision tree is derived from the number of randomly selected features. This kind of randomly selected features has good scalability when facing many unknown features of each feature vector and also helps to reduce the interdependence between feature attributes. The classification error rate of the RF classifier depends on the correlation between any two decision trees and the classification strength of each decision tree. The Out-of-Bag error rate is a reliable estimate of the classification error rate. Therefore, when using the RF classification algorithm, cross-validation or a separate test set is usually not required.
RF algorithm exhibits an excellent performance in settings where the number of variables is much larger than the number of observations. Therefore, RF algorithm possesses the advantage of using the complete IPPG signal to classify different types of diseases, instead of a specific set of suspected IPPG features. The RF classification algorithm is chosen to verify our proposed idea.
4.2 Establishment of the RF classification model
Combined with the manifestations of IPPG signal differences caused by the different types of diseases, we set up RGB channels disease classification models for heart disease and diabetes mellitus. Since the number of IPPG signals obtained is limited, especially the diseased subjects, we divide the IPPG signals to expand the data amount. The 30s IPPG signal of each subject is divided into a 5s segment, and each IPPG signal is expanded 6 times. When expanding the IPPG signals, the expanded signals contain at least one dominant peak of the waveform to eliminate the invalid signals. Given the classification analysis of these two diseases should ideally leverage the relatively abundant IPPG signal characteristics, the IPPG signals we input to the RF classification models are slightly filtered (See section 3.2). Finally, we get 588 IPPG signals from healthy subjects, 84 IPPG signals from heart disease subjects, and 144 IPPG signals from diabetes mellitus subjects, respectively.
The RF classification model is based on the random forest package of MATLAB. The flowchart of the disease classification models in the RGB channels based on the RF algorithm is shown in Fig. 7. The healthy subjects and heart disease patients are marked as the H group, with a total of 672 IPPG signals. The healthy subjects and diabetes mellitus subjects are marked as the D group, with the 732 IPPG signals. In the H group, the label of healthy subjects is 0, and the label of subjects with heart disease is 1. Similarly, in the D group, the labels of healthy subjects are still marked as 0, and the labels of diabetes mellitus subjects are marked as 2. For the H group, 600 IPPG signals are selected as the training set and 72 IPPG signals are selected as the test set in each channel. As for the D group, 650 IPPG signals are selected as the training set and 82 IPPG signals are selected as the test set in each channel. The RF classification models of the H group and the D group are trained in the RGB channels, respectively. Finally, the classification models of the RGB channels of the two diseases are established. The health status of the subjects in the test set of each channel is judged by the classification model. The healthy subjects and the diseased subjects in the training set and test set are freely and randomly for each training and test. When the RF classification model completes the classification, record the classification results. The three classification results are used to verify the reliability of the classification model. According to multiple evaluation indicators, the average value of the three classification results of each channel in the H group and D group is evaluated.
Fig. 7. The flowchart of the disease classification models in the RGB channels based on the RF algorithm. The RF classification models of the H group and the D group are trained in the R-G-B channel, respectively. The training set of each channel is input into the random forest classifier to train the classification model. Then, the health status of the subjects in the test set of each channel is judged by the classification model.
4.3 Classification evaluation indices
To evaluate the classification results in the RGB channels of the H group and D group, evaluation indices are adopted including Accuracy, Precision, Recall, and F1score [34]. Accuracy is used to assess the predictive capability of classification models, which is the proportion of the healthy subjects and patients correctly classified by the model. Precision denotes the proportion of predicted healthy subjects that are correctly real healthy subjects. Recall denotes the proportion of the healthy people who are predicted correctly. F1score is the balance index of Precision and Recall. F1score can comprehensively evaluate the classification performance in the H group classification model and D group classification model. Table 2 summarizes these equations, where TP (true positive) and TN (true negative) are the numbers of the healthy and the diseased subjects. FP (false positive) and FN (false negative) are the numbers of the healthy and the diseased subjects incorrectly classified.
Table 2. Classification evaluation indices
The classification performance in the RGB channels of the H group and D group are shown in Table 3. In the H group, the classification performance of the G channel is up to 88.43% in terms of the Accuracy value, whereas the R channel is 85.65%. The G channel has the highest Recall with a value of 94.08% compared with other channels. However, the Precision value of the G channel is lower than that of the B channel, which is 93.58%. In the G channel, the comprehensive index F1score shows the best classification performance with a value of 93.72%. The R channel and B channel have F1score values of 91.95% and 90.34%, respectively. In the D group, the classification result of the R channel reaches 82.12% in terms of the Accuracy value. The Precision value is 89.52% and the Recall value is 89.51% in the R channel, respectively. The G channel also shows good classification results, with Precision and Recall values of 86.38% and 89.42%, respectively. These indices in the R channel are higher than those of the G channel and B channel. Although the classification performance of the B channel is the worst in the H group and D group classification model, the classification results of the B channel in the H group are still higher than that of the D group.
Table 3. Multiple evaluation indices of the RGB channels of the H group and D group classification models performance
Figure 8 highlights the three classification performances in the RGB channels of the H group and D group in terms of the F1score values. The G channel is the most reasonable channel to classify healthy individuals and heart disease patients with an average F1score value of 93.72%. The R channel and B channel also indicate excellent classification performance with F1score average values of 91.95% and 90.34%, respectively. Correspondingly, the R channel has the highest classification result in judging whether the subject has diabetes mellitus, in terms of F1score value of 89.31%, followed by the G channel and B channel with the F1score average values of 87.84% and 84.77%, respectively. The G channel and R channel could be used as effective channels of IPPG signals to classify different types of diseases. The classification performance of the B channel is the worst regardless of in the H group, or the D group.
Fig. 8. The three classification performances in the RGB channels including the H group and D group in terms of the F1score values. The F1score values are illustrated on the R-G-B channel order. Each channel is represented by a corresponding color, and the channel is divided into H group and D group. Each group contains the F1score values of three classification results, represented by horizontal lines, and the average of three F1score values, represented by a circle. When the F1score values are the same, the horizontal lines will overlap.
The differential IPPG waveforms in the RGB channels are a comprehensive manifestation that combines the optical information of the multi-layer skin tissue and the pathological mechanism of the disease. For heart disease patients, the characteristics of the IPPG waveforms caused by the impaired heartbeat are all similar in the RGB channels. Although the amplitudes of the IPPG signals are inconsistent in the RGB channels, the influence of the amplitudes can be ignored due to the IPPG signal pre-processing. The G channel is the best channel for obtaining IPPG signals that we have analyzed before. Undoubtedly, the G channel has the best classification performance for judging whether the subject has heart disease. In fact, we extract high-quality IPPG signals in the RGB channels from all subjects. Although the B channel only reaches the surface layer of the skin tissue, the IPPG signals of the B channel also carry the same pathological features from an impaired heartbeat as the other two channels. Thus, the B channel achieves a good classification performance. Our experimental results also prove this related research that the IPPG signal of the RGB channels carries certain physiological information [35].
For diabetes mellitus subjects, the specific effects of hyperglycemia on each layer of skin tissue are significantly incoherent. The characteristics of IPPG waveforms are messy, with multiple peaks in the RGB channels. We observe that the IPPG waveforms in the R channel have the more obvious dicrotic notches and second peaks compared with other channels. However, we can hardly summarize the characteristics of IPPG waveforms in the RGB channels for each diabetes mellitus patient. The G channel still performs well in determining whether the subject is a diabetes mellitus patient. However, the R channel has a more satisfactory classification performance in distinguishing diabetes mellitus subjects from healthy subjects compared with the G channel. Although the signal-to-noise ratio of the IPPG signal in the R channel is lower than that of the G channel, the IPPG signals of the R channel carry the most abundant physiological and pathological information of the skin tissue due to the deepest penetration depth of the skin tissue. Therefore, in the diabetes mellitus classification model, classification results fully prove the superiority of the R channel as a disease diagnosis channel. The superficial skin tissue is the least affected by hyperglycemia, so the classification performance of the IPPG signal in the B channel is not ideal.
According to the characteristics of different disease types, selecting the appropriate IPPG signal channel as the disease classification channel can improve the classification accuracy of subjects. Our experimental results verify the great importance of combining pathological mechanisms and tissue optical properties to select RGB channel IPPG signals to classify diseases. Indeed, the data amount of IPPG signals we obtained is limited, especially the IPPG signals from heart disease and diabetes mellitus patients. However, the raw IPPG signals we obtained are from subjects in the same environment. And the processing methods of IPPG signals are also consistent. Therefore, the data amount of IPPG signals maybe affect the classification accuracy of the two classification models but does not interfere with the channel selection. Moreover, the difference in skin tones is also a great challenge that affects the signal-to-noise ratio of IPPG signals. Since the subjects in our experiment are all Asian, the skin tones of the subjects (healthy people and patients with disease) are a relatively symmetrical yellow color. In this investigation, the difference in skin tones is ignored. The degree of the disease, duration of the disease, and division of different types of diseases will affect the classification results. In the future, we will expand the data amount of IPPG signals involving the different types of diseases, and minutely classify the differences of the pathology to get more accurate diseases classification performance. Our classification results clearly demonstrate the best classification channel based on IPPG technology is different for different types of diseases. For the classification of cardiovascular disease such as heart disease, the G channel indicates an extremely excellent classification performance. For diabetes-related microvascular diseases that have long-term effects on the epidermis and dermal microvasculature of the skin tissue, the IPPG signal of the R channel can provide strong support for the disease classification.
5. Conclusion
This investigation indicated that IPPG technology had different optimal channels for the classification of different diseases. We found that the characteristics of the IPPG signals were distinct in the RGB channels including healthy individuals, heart disease patients, and diabetes mellitus patients. Besides, the IPPG signals varying greatly between heart disease and diabetes mellitus presented their pathological mechanism. Combining the pathological mechanism and optical information carried by the IPPG signals in the RGB channels, the heart disease classification model and diabetes mellitus classification model are established based on the RF algorithm. The experimental results are verified on the IPPG signals, which include 588 IPPG signals from the healthy subjects, 84 IPPG signals from heart disease patients, and 144 IPPG signals from diabetes mellitus patients. In the heart disease classification model and diabetes mellitus classification model, the best classification performances are the G channel with an average F1score value of 93.72% and the R channel with an average F1score value of 89.31%, respectively. Our investigation has a strong potential for clinical applications and disease classification of the IPPG technology.
Funding
National Natural Science Foundation of China (No. 61705010, No. 11774031, No. 61935001).
Disclosures
The authors declare no conflicts of interest.
Data availability
Data underlying the results presented in this paper are not publicly available at this time but may be obtained from the authors upon reasonable request.
Supplemental document
See Supplement 1 for supporting content.
References
1. W. Wang, A. C. den Brinker, S. Stuijk, and G. de Haan, “Amplitude-selective filtering for remote-PPG,” Biomed. Opt. Express 8(3), 1965–1980 (2017). [CrossRef]
2. F. P. Wieringa, F. Mastik, and A. Steen, “Contactless multiple wavelength photoplethysmographic imaging: a first step toward ‘spo2 camera’ technology,” Ann. Biomed. Eng. 33(8), 1034–1041 (2005). [CrossRef]
3. Y. Sun and N. Thakor, “Photoplethysmography revisited: from contact to noncontact, from point to imaging,” IEEE Trans. Biomed. Eng. 63(3), 463–477 (2016). [CrossRef]
4. M. Z. Poh, D. J. McDuff, and R. W. Picard, “Advancements in noncontact multiparameter physiological measurements using a webcam,” IEEE Trans. Biomed. Eng. 58(1), 7–11 (2011). [CrossRef]
5. L. Tarassenko, M. Villarroel, A. Guazzi, J. Jorge, D. A. Clifton, and C. Pugh, “Non-contact video-based vital sign monitoring using ambient light and auto-regressive models,” Physiol. Meas. 35(5), 807–831 (2014). [CrossRef]
6. S. Rasche, A. Trumpp, T. Waldow, F. Gaetjen, K. Plötze, D. Wedekind, M. Schmidt, H. Malberg, K. Matschke, and S. Zaunseder, “Camera-based photoplethysmography in critical care patients,” Clin. Hemorheol. Microcirc. 64(1), 77–90 (2016). [CrossRef]
7. A. A. Kamshilin, E. Nippolainen, I. S. Sidorov, P. V. Vasilev, N. P. Erofeev, N. P. Podolian, and R. V. Romashko, “A new look at the essence of the imaging photoplethysmography,” Sci. Rep. 5(1), 10494–9 (2015). [CrossRef]
8. A. A. Kamshilin and N. B. Margaryants, “Origin of photoplethysmographic waveform at green light,” Phys. Procedia 86, 72–80 (2017). [CrossRef]
9. Y. Maeda, M. Sekine, and T. Tamura, “Relationship between measurement site and motion artifacts in wearable reflected photoplethysmography,” J. Med. Syst. 35(5), 969–976 (2011). [CrossRef]
10. K. Matsumura and T. Yamakoshi, “iPhysioMeter: A new approach for measuring heart rate and normalized pulse volume using only a smartphone,” Behav Res 45(4), 1272–1278 (2013). [CrossRef]
11. G. Zhang, Z. Mei, Y. Zhang, X. Ma, B. Lo, D. Chen, and Y. Zhang, “A noninvasive blood glucose monitoring system based on smartphone PPG signal processing and machine learning,” IEEE Trans. Ind. Inf. 16(11), 7209–7218 (2020). [CrossRef]
12. D. Djeldjli, F. Bousefsaf, C. Maaoui, F. B. Reguig, and A. Pruski, “Remote estimation of pulse wave features related to arterial stiffness and blood pressure using a camera,” Biomedical Signal Processing and Control 64, 102242 (2021). [CrossRef]
13. M. Nitzan, A. Romem, and R. Koppel, “Pulse oximetry: fundamentals and technology update,” MDER 7, 231–239 (2014). [CrossRef]
14. P. Shi, S. Hu, Y. Zhu, J. Zheng, Y. Qiu, and P. Y. S. Cheang, “Insight into the dicrotic notch in photoplethysmographic pulses from the finger tip of young adults,” Journal of medical engineering and technology 33(8), 628–633 (2009). [CrossRef]
15. X. He, R. A. Goubran, and X. P. Liu, “Secondary peak detection of PPG signal for continuous cuffless arterial blood pressure measurement,” IEEE Trans. Instrum. Meas. 63(6), 1431–1439 (2014). [CrossRef]
16. W. Liu, X. Fang, Q. Chen, Y. Li, and T. Li, “Reliability analysis of an integrated device of ECG, PPG and pressure pulse wave for cardiovascular disease,” Microelectron. Reliab. 87, 183–187 (2018). [CrossRef]
17. C. M. Lilia, O. S. Gerson, M. O. Alvaro, O. T. Arturo, D. C. Keirn, D. R. Fern, and M. M. Raul, “Endothelial dysfunction evaluated using photoplethysmography in patients with type 2 diabetes,” J Cardiovasc Dis Diagn 03(05), 219 (2015). [CrossRef]
18. K. Pilt, K. Meigas, R. Ferenets, K. Temitski, and M. Viigimaa, “Photoplethysmographic signal waveform index for detection of increased arterial stiffness,” Physiol. Meas. 35(10), 2027–2036 (2014). [CrossRef]
19. M. Schönauer, A. Thomas, S. Morbach, J. Niebauer, U. Schönauer, and H. Thiele, “Cardiac autonomic diabetic neuropathy,” Diabetes Vasc. Dis. Res. 5(4), 336–344 (2008). [CrossRef]
20. W. Verkruysse, M. Bartula, E. Bresch, M. Rocque, M. Meftah, and I. Kirenko, “Calibration of contactless pulse oximetry,” Anesthesia Analgesia 124(1), 136–145 (2017). [CrossRef]
21. M. V. Volkov, N. B. Margaryants, A. V. Potemkin, M. A. Volynsky, I. P. Gurov, O. V. Mamontov, and A. A. Kamshilin, “Video capillaroscopy clarifies mechanism of the photoplethysmographic waveform appearance,” Sci. Rep. 7(1), 13298–8 (2017). [CrossRef]
22. I. S. Sidorov, R. V. Romashko, V. T. Koval, R. Giniatullin, and A. A. Kamshilin, “Origin of infrared light modulation in reflectance-mode photoplethysmography,” PLoS One 11(10), e0165413 (2016). [CrossRef]
23. A. A. Kamshilin, O. V. Mamontov, V. T. Koval, G. A. Zayats, and R. V. Romashko, “Influence of a skin status on the light interaction with dermis,” Biomed. Opt. Express 6(11), 4326–4334 (2015). [CrossRef]
24. A. V. Moço, S. Stuijk, and G. de Haan, “Skin inhomogeneity as a source of error in remote PPG-imaging,” Biomed. Opt. Express 7(11), 4718–4733 (2016). [CrossRef]
25. A. V. Moço, S. Stuijk, and G. de Haan, “New insights into the origin of remote PPG signals in visible light and infrared,” Sci. Rep. 8(1), 8501 (2018). [CrossRef]
26. A. N. Bashkatov, E. A. Genina, V. I. Kochubey, and V. V. Tuchin, “Optical properties of human skin, subcutaneous and mucous tissues in the wavelength range from 400 to 2000nm,” J. Phys. D: Appl. Phys. 38(15), 2543–2555 (2005). [CrossRef]
27. W. Wang, A. C. Den Brinker, and G. De Haan, “Single-element remote-ppg,” IEEE Trans. Biomed. Eng. 66(7), 2032–2043 (2019). [CrossRef]
28. W. Wang, A.C. Den Brinker, S Stuijk, and G. De Haan, “Color-distortion filtering for remote photoplethysmography,” 2017 12th IEEE International Conference on Automatic Face & Gesture Recognition (FG 2017)IEEE, 71–78 (2017).
29. G. De Haan and V. Jeanne, “Robust pulse rate from chrominance-based rPPG,” IEEE Trans. Biomed. Eng. 60(10), 2878–2886 (2013). [CrossRef]
30. R. Avram, J. E. Olgin, P. Kuhar, J. W. Hughes, G. M. Marcus, M. J. Pletcher, K. Aschbacher, and G. H. Tison, “A digital biomarker of diabetes from smartphone-based vascular signals,” Nat. Med. 26(10), 1576–1582 (2020). [CrossRef]
31. D. Wang, X. Yang, X. Liu, J. Jing, and S. Fang, “Detail-preserving pulse wave extraction from facial videos using consumer-level camera,” Biomed. Opt. Express 11(4), 1876–1891 (2020). [CrossRef]
32. C. Yang, C. Veiga, J. J. Rodríguez-Andina, J. Fariña, A. Iñiguez, and S. Yin, “Using PPG signals and wearable devices for atrial fibrillation screening,” IEEE Trans. Ind. Electron. 66(11), 8832–8842 (2019). [CrossRef]
33. L. Breiman, “Random Forests,” Machine Learning 45(1), 5–32 (2001). [CrossRef]
34. L. F. S. Siqueira, R. F. Araújo Júnior, A. A. de Araújo, C. L. M. Morais, and K. M. G. Lima, “LDA vs. QDA for FT-MIR prostate cancer tissue classification,” Chemometr. Intell. Lab. Syst. 162, 123–129 (2017). [CrossRef]
35. W. Verkruysse, L. O. Svaasand, and J. S. Nelson, “Remote plethysmographic imaging using ambient light,” Opt. Express 16(26), 21434–21445 (2008). [CrossRef]
