1. Introduction

In the human body, carbon dioxide is a byproduct of metabolism. The venous bloodstream transports CO₂ to the lungs where it is ultimately removed from the body through exhalation. CO₂ is involved in regulation of blood pH, respiratory drive, and affinity of hemoglobin for oxygen (O₂) [1]. High level of the blood carbon dioxide partial pressure, known as hypercapnia, can cause headaches, dizziness, and fatigue, as well as serious complications, such as seizures or loss of consciousness. Hypercapnia may develop as a complication of chronic lung diseases, interstitial lung disease, and cystic fibrosis, as well as under some neurological and muscle diseases [2].

Concentration of CO₂ in natural ambient air is about 0.04% or 400 ppm [3]. With each breath, humans convert oxygen into carbon dioxide. CO₂ concentration in the exhaled human breath is about 38,000 ppm [4]. CO₂ blood concentration is measured in mill moles per liter (mmol/L). Normal values in adult’s exhalation are 23 to 29 mmol/L [5].

CO₂ concentration in human exhalation and in the blood have a strong relationship. Elevated partial pressure of carbon dioxide (PCO₂) in the blood and reduced alveolar ventilation, resulting from respiratory failure, are primarily affecting the respiratory pump driving the ventilation. Several methods for PCO₂ estimation, such as arterial blood gas analysis (ABG), capillary blood gas analysis (CBG) [6], venous blood gas analysis (VBG) [7], capnography, and transcutaneous PCO₂ measurements [8], are currently in use [9]. The existing clinical methods for PCO₂ estimation require physical contact with a patient. After the COVID-19 outbreak causing hypercapnia, a non-contact application for measuring PCO₂ could be helpful [10].

Optical sensors became widespread in biomedical diagnostics. Compared to other types of sensors, numerous advantages of optical sensors have been commonly recognized. The main advantage of optical sensors is their non-invasive nature. Some sensors are also noncontact and thus are safer and more convenient.

Numerous optical sensors for biomedical applications are based on the analyses of secondary speckle patterns, a well-known phenomenon in coherent imaging. It occurs when coherent light source illuminates a rough surface. As a result, random reflections generated from the illuminated area interfere and produce a speckle pattern. The Speckle-based remote sensing approach has been investigated for some time and used in various biomedical applications such as evaluation of blood pulse pressure [11], the temporal signature of the heartbeats and breath [12], blood oxygen saturation [13], and even alcohol concentration in the bloodstream [14] and more.

Instability in partial pressure of blood carbon dioxide causes human body to activate stabilization of partial blood CO₂ pressure. Among the possible stabilization body activities are an undesirable heartbeat filling, increase in breath and heart rate. Such body reactions could affect the skin vibration, which is possible to detect by optical speckle-based sensors.

This paper proposes a novel laser-based method for remote estimation of partial blood carbon dioxide pressure. The method is based on tracking secondary speckles patterns reflected from the human skin illuminated by a laser beam. Speckle pattern analyses combined with machine learning could create a classifier for remote carbon dioxide estimation in the human blood.

2. Background

2.1 Carbon dioxide in bloodstream

Under normal metabolic process, the human organs are supplied by oxygen through the arterial blood flow while carbon dioxide is removed by the venous blood flow. Respiration and blood systems are involved in gas exchange in the human body.

CO₂ is a product of cellular metabolism generated during the Krebs cycle, which takes place in the mitochondria of a cell [15]. CO₂ enters the bloodstream by diffusion, where it eventually reaches the pulmonary capillaries (convection). It diffuses through the alveolar membrane into the alveoli, and then is exalted via the airways. The driving force for diffusion is the partial CO₂ pressure difference between the alveoli (about 40 mm Hg) and blood in the pulmonary capillary (about 46 mm Hg).

Although the difference between the pressures is rather small, CO₂ exchange occurs due to its excellent blood solubility, which is higher than that of oxygen (O₂). Blood transports carbon dioxide in the three ways. It is partially dissolved within the blood plasma, also bound to hemoglobin, or converted into bicarbonate [16]. The condition of carbon dioxide imbalance in the blood is abnormal and, in most cases, it points to the metabolism process failure. The respiratory control system constitutes a negative-feedback system, with the arterial PCO₂ (PaCO₂), pH and arterial PO₂ (PaO₂) the controlled variables. To act as a negative-feedback system, the respiratory controller receives information concerning the level of the controlled variables from the body sensors. These sensors, or chemoreceptors, are located within the systemic arterial system and within the brain itself. Both the peripheral and central chemoreceptors respond to the variations in the partial pressure of the blood carbon dioxide.

For example, when the human body is in a “Hypercapnia” condition (high PCO₂ level in blood), to fix the instability, the body is changing the breath and heart rates. In addition, the blood pH decreases, and in a response, the kidneys work to balance it. The pH drop causes a decrease in heart muscle function and heart rhythm interference called “premature heartbeats”, a situation where the initiative of heartbeat comes from “Purkinje fibers” instead of “sinoatrial nodes” [17]. Such variation in the body condition cause a change of its vibration. Our goal is to find relation between the body response to the variation in blood CO₂ and to estimate its concentration by menace of laser based speckle pattern analyses.

2.2 Blood CO₂ estimation by speckle pattern analyses

Speckle patterns are self-interfering random images produced by self-interference of coherent wave fronts of different phases and amplitudes reflected from a rough surface [18]. When added together on the detector plane the wave fronts provide a random intensity interference pattern known as secondary speckle pattern. The working principle underlying our suggested technology proposes not to focus the camera on the object, but on the far field such that the object itself is defocused. In this case, the object vibrations cause a lateral shift in the speckles pattern [19]. Following the defocusing, the speckle pattern is mainly affected by the tilt angle of the vibration and the transversal and axial shifts can be neglected. The tilt movement is the main component, and it can be evaluated using spatial pattern correlation [19]. The tilt movement could be expressed as follows [19]:

We assume that elevated blood CO₂ affects micro-vibration of the skin and could be detected by speckle pattern analyses. The coherent light spot penetrating inside skin dermis interacts with the vein blood hemoglobin-containing CO₂. Therefore, the light reflected from the red blood cells creates a superposition with the light reflected from the skin surface affecting the speckle pattern including its intensity and contrast, as seen in Fig. 1.

 

Fig. 1. Coherent light skin interaction scheme.

Download Full Size | PDF

We assume that Deep Learning (DL) of the recorded speckle patterns containing the overall hidden information could allow remote evaluation of the blood CO₂ concentration.

3. Motivation and experimenting

Data for the speckle-based blood CO₂ classification collected from six healthy male subjects, aged 20 to 60, in a controlled optical laboratory environment. Each participant seated during the recording. His arm we tied to a support avoiding voluntary movements. Green laser beam (wavelength 532 nm) illuminated the participant’s wrist. The laser was positioned 20 cm above the participant’s arm. The camera recorder (Basler acA1440-220um USB 3.0) fixed 50 cm from the participant’s arm. A mask connected to Capnograph (Medtronic PM35) for PetCO₂ recording covered the subject’s nose and mouth. The end-tidal pressure of expired CO₂ (PetCO₂) is the partial pressure of CO₂ at the end of exhalation. It represents the alveolar CO₂ concentration, which, in turn, allows the estimation of arterial PCO₂ [20,21]. The capnograph may reduce the need for invasive monitoring and/or repeated arterial blood gas analyses [22]. The capnograph PCO₂ measurement accuracy tolerance is ±2 mmHg under regular conditions (0-38mmHg) and reaches ±2.5mmHg for high PCO₂ values (39-150mmHg).

A defocused camera recorded the speckle patterns reflected from the wrist. Each video contained ten second recording under 100 frames per second (fps) and 128 × 128 pixels resolution. Two recording scenarios for each subject were tested: normal and stimulated high PetCO₂ level. The normal PetCO₂ level recording conducted when a subject was normally breathing. High PetCO₂ level was stimulated by a subject breathing into a plastic bag, increasing CO₂ concentration of the inhaled air. Capnograph receiving the subject’s exhalation air through a nose and mouth mask, measured the reference PCO₂ level. The speckle pattern recordings taken from a subjects wrist, where the heartbeat could be detected. Figure 2 and Fig. 3 show the experimental setup scheme and the actual installation with a subject participating in the data acquisition.

 

Fig. 2. Experimental setups scheme.

Download Full Size | PDF

 

Fig. 3. Experimental setup for recording of speckles patterns reflected from a subject’s wrist.

Download Full Size | PDF

However, PetCO₂ measurements are not as accurate as arterial blood gas (ABG), because there is a difference of approximately 2–5 mm Hg between arterial PCO₂ and PetCO₂ (PaCO₂–PetCO₂ difference) due to the alveolar dead space (lungs size) [9]. Therefore, the normal PetCO₂ values are specific for each tested subject. We determined the high PetCO₂ level for each subject, which is at least by 5mmHg higher than normal PetCO₂. During each video session repeated twelve times, we also recorded the readings of the capnograph, See Table 1.

Table 1. PetCO₂ min and max values per subject in two different CO₂ conditions.

View Table | View all tables in this article

4. Method

The goals are to determine the most informative feature representation from the speckle patterns that could be used to classify between high and normal CO₂ level, as well as to find a model architecture that could deal with those features in a way that would yield high accuracy.

4.1 Data exploration and pre-processing

Preprocessing and data exploration are important components of a classifier, especially based on the machine learning. In this research, the input data contained speckle pattern video recordings. The labeled videos separated randomly into three sets – train, validation, and test sets. Each ten second recording contained 1000 frames with 0.01s resolution.

A skin-reflected speckle pattern are random images containing information related to the skin vibration. The skin vibration is extracted as a position shift between two adjacent speckle pattern frames, using cross correlation algorithm, tracking movement of several speckles acting together as a sub-image. The 2-D correlation function calculated for each pair of corresponding sub-images allows to obtain the respective displacement derived from the maximum correlation position, producing a displacement graph [19].

For each video containing thousand frames, we obtained the skin displacement graph, (Fig. 4 (a)). As a result, two vectors called “pos” (position shift) are created. These vectors describe the movement of the illuminated tissue in the X and Y directions. Figure 4 (b) shows that the heartbeat rhythm can be identified from the “Pos” signal.

 

Fig. 4. (a) 2D correlation map of two consecutive speckle frames; (b) Speckle pattern correlation graph reflecting the nano-vibrations occurring in the wrist while showing displacement along axis x.

Download Full Size | PDF

To explore the difference between the “pos” signals of two classes, the two signals (x and y) of each video merged into one signal. For every sample, the maximum absolute magnitude between the “pos” signals was extracted. As a result, we obtained a signal reflecting maximum displacement without reference to the axes. An example of the combined “pos” signals for every class is presented in Fig. 5(a).

 

Fig. 5. Normal and high blood CO₂ level 2D-correlation pattern difference: (a) combined “pos” signal in time domain; (b) max 2D-correlation point distribution; (c) Fast Furrier Transfer (FFT) graph of the combined signal.

Download Full Size | PDF

At first glance, Fig. 5(a) shows that for a high CO₂ level, the displacement value is increased compared to the normal CO₂ level. We used a confidence ellipse for visual examination of the “pos” signals, see Fig. 5(b). The confidence ellipses present distribution of the max correlation point (X, Y) at each class. The confidence ellipses are different for high and normal CO₂ levels. The ellipse for high CO₂ is considerably larger than the confidence ellipse of normal CO₂ levels.

The Fast Fourier Transform (FFT) contains the frequencies making up the “pos” signals. Figure 5(c), clearly shows that the classes are distinguishable. The FFT plot is visually distinguishable pointing out on a possibility to classify the two blood CO₂ classes by deep learning (DL).

4.2 Algorithm

Several combinations of parameter extraction and model selection were tried, and three leading hypotheses are presented in the following sections. At first, a naïve method was tested, to use a stream of speckles pattern images with a well-known video classification model such as 3D-Conv, ConvLSTM, and ResNet-50 CNN (pre-trained model). All of these models gave poor performance.

The next parameter extraction hypothesis was to present the correlation pattern between followed speckles as a picture, based on a wavelet transform. This feature extraction uses a known image classification model such as VGG16, Res-Net50, and more. These produced unsatisfactory accuracy score.

The final hypothesis focused on the correlation of signals extracted from the speckle pattern videos. Two types of models were tested: LSTM model and CNN model. The last achieved great performance in comparison to the LSTM model.

4.2.1 CNN model

The main reason to apply DL is the complexity and unpredictably of human body reactions. Our preliminary study found no evidence of blood PCO₂ variation based on visual or specific body events. DL is a good classifier able to learn and distinguish between patterns, especially when those patterns contain visually hidden features. This architecture based on a convolution neural network (CNN).

CNNs have applications in image and video recognition, image classification, image segmentation, medical image analysis [23], natural language processing, brain-computer interfaces [24], and financial time series [25]. For this research, a one-dimensional CNN model we used to classify speckle patterns of different labels.

The model input contains parts of the “pos” signal, a time series signal that presents the illuminated skin movements and intensity variations. Each “pos” signal, based on thousand frames data, sub-divided into blocks of 128 samples with a shift of one sample between them. In that way, one video presents 873 vectors of 128 lengths for each axis (x and y).

Our model comprises three identical Conv1D blocks, a pooling layer, and a dense layer, as shown in Fig. 6(a). Conv1D block is the main component of our model, and it contains three steps: convolution layer, batch normalization process, and “ReLU” activation function (see Fig. 6(b)). Each step maintains a specific task:

 

Fig. 6. Classifier architecture: (a) Our deep learning model; (b) Conv1D block with conv1D layer, batch normalization and activation function.

Download Full Size | PDF

A Global Average Pooling layer follows the three Conv1D blocks, which is a pooling operation that designed to replace fully connected layers in a classical CNNs [29]. Instead of adding fully connected layers on top of the feature maps, we used the average of each feature map. Furthermore, global average pooling sums out the spatial information, thus it is more robust to spatial translations of the input.

The final layer in our model is the dense layer, which is a fully connected layer followed by the Softmax activation function [30] that divides the Global Average Pooling layer into two units (x_i where i = 0,1) based on the classification task. Unit at the index “c” represents the probability of the last layer being classified at class “c” i.e., the output units in index 0 and 1 are the probability of being in a normal or high CO₂ class respectively (see Eq. (5)).

Our model use “Sparse Categorical Cross entropy” loss function metric for classification of the categorical data. Cross entropy [31] is a method from information theory. It measures the certainty level to be in a particular state. It is based on the state probability distribution. The certainty level is used as the loss value to the model’s accuracy. The Sparse Categorical Cross entropy is partly different from the basic categorical cross entropy and the calculation of the loss depend only on the ground truth probability. It allows to compute loss function logarithm once per instance and to omit the summation, reducing duration of the data processing and memory use when dealing with a large number of categories. The sparse categorical cross entropy loss function is shown in Eq. (6). The parameter “output_c” is the resulting output of the final layer at index “c”, which presents the model probability score to be at class “c”.

4.2.2 Final classification layer

We should determine whether a complete “pos” signal label is 0 or 1 based on the CNN model classification of each block. For that purpose, we selected the complete video classification according to the percentage of blocks classified as high CO₂ levels (see Fig. 7). When it reaches this threshold level, the complete video classification is considered as high CO₂, otherwise, as normal CO₂. Since the threshold values depend on several model parameters, such as block length, number of blocks in a single “pos” signal, etc., the threshold values require calibration.

 

Fig. 7. Complete algorithm: CNN model and final classification layer.

Download Full Size | PDF

5. Experimental results

The DL model input dataset collected from six healthy participants. We recorded 24 videos, 12 under normal blood CO₂ and 12 under stimulated high CO₂ concentrations for each subject. Each video we pre-processed in two signals (two vectors 1000 × 1 dimension each). The “pos” signals list for each participant split into three sets (train, validation, and test) with a 60%, 20%, and 20% subdivision, respectively.

The model benchmarks are presented in Fig. 8, Fig. 9 and Table 2. The model achieved accuracy score of almost 90% on the train set. The validation set accuracy exceeds 80% with 100 epochs. The model accuracy on the test set is 78%, as seen in Table 2.

 

Fig. 8. Model training and validation graphs.

Download Full Size | PDF

 

Fig. 9. Model CO₂ level predictions for two tested subjects.

Download Full Size | PDF

Table 2. Model results on test set

View Table | View all tables in this article

Despite satisfactory results from individual video parts classification, a combined classification could yield better results, in case all video parts classifications were gathered and maximum classification between all video parts selected. Figure 9 shows the classification of eight videos for two subjects with two videos for normal and elevated CO₂ conditions (2 × 2x2). Video recordings on the right picture representing high CO₂, while video recordings on the left made under normal CO₂. In that way, even for a video having a higher confusion in their video parts classification, eventually, the total classification pointed on the correct classification of the original video. It demonstrates the power of combining multiple sub-video classifications over classifying them separately. Using the final classification layer, we determined the complete video classification based on all block predictions and the high CO₂ threshold level parameter.

The confusion matrices, reflecting the CO₂ classification presented in Fig. 10. It depends on the high CO₂ threshold level used at the final classification task. For the left confusion matrix (Fig. 10(a)), we used the naive approach, assuming that most of the sub-video classification determines the full video classification (threshold = 50%). In that way, the algorithm accuracy is 89% while the algorithm FN (false negative) score is 21%. Therefore, our model tends to select the negative class (regular CO₂ level) over the positive class. A situation like this is not ideal when dealing with medical applications, which require strict determination to avoid missing a dangerous health condition like high blood carbon dioxide levels. Medical applications often require FN to be as low as possible. Consequently, we determined that for a 43% high CO₂ threshold level (Fig. 10(b)), the algorithm FN (false negative) score drops to 7%, and the algorithm accuracy reaches 96.4%.

 

Fig. 10. Confusion matrix on the test set with different high CO₂ level.

Download Full Size | PDF

6. Discussion and conclusion

We presented a novel method for remote estimation of partial blood CO₂ pressure. Our method for remote blood CO₂ estimation propose to use speckle based optical sensor, which commonly used in biomedical research, combined with the developed DL algorithm for blood CO₂ pressure classification.

The system consists of a laser-illuminating subject’s wrist, a defocused digital camera capturing speckle patterns, and a computer equipped with MATLAB and ML algorithms. The algorithm based on a CNN model and final classification layer.

Twelve testing trials were conducted for each condition (high and normal CO₂ level) for each of the six participants. The collected data set we used for the DL algorithm training and testing. The algorithm achieved 96% accuracy for 10 seconds video recording, those by using a calibrated threshold value in the algorithm’s final classification layer.

According to the experimental results, the proposed system could monitor partial blood CO₂ pressure effectively. It could be helpful for the preliminary and remote Covid diagnostics. It is the first time a non-contact measurement of blood CO₂ pressure evaluated, providing a powerful inexpensive diagnostic tool. Our method allows simple blood CO₂ evaluation, preventing unnecessary invasive and expensive test.

Further model prediction improvement requires involvement males and females, of different ages and races, healthy and having reduced blood CO₂. The representative group-based model should be used to calibrate the high CO₂ threshold level parameter. In addition, transfer learning is a powerful tool that should be considered to use for improving the model score. There is also a need for future research to assess the use of other laser wavelengths, such as NIR, on the ability to distinguish between blood CO₂ concentrations at high and normal levels.

The algorithm’s high accuracy demonstrates that the speckles pattern correlation peak position shift is proportional to the difference in blood CO₂ pressure between high and normal conditions. Further research to develop a regression model to predict the partial blood CO₂ pressure, should be considered instead of binary classification. In Ref. [32], we describe the code that was used in our research. Customized training can be run using individually collected data and with the correct configuration and hyper-parameters.