New neural-network-based method to infer total ozone column amounts and cloud effects from multi-channel, moderate bandwidth filter instruments

Lingling Fan; Wei Li; Arne Dahlback; Jakob J. Stamnes; Snorre Stamnes; Knut Stamnes

doi:10.1364/OE.22.019595

1. Introduction

Although UV radiation received by the Earth occupies only 7% of the total solar energy it plays a critical role in the biosphere. UV radiation is divided into three parts: UVA (315–400 nm), UVB (280–315 nm) and UVC (200–280 nm). UVC radiation is absent at the surface of the Earth because absorption by diatomic oxygen and ozone prevents it from penetrating the atmosphere. UVB radiation is significantly attenuated before reaching the surface of the Earth by the TOC in the atmosphere, while UVA radiation is little affected by the ozone layer. Since UVB radiation is harmful to human beings, animals and plants, the TOC is a very important quantity to be monitored. Although ozone constitutes only a tiny fraction by mass (∼ 0.0000007%) of the Earth’s atmosphere, it is very important for life, because it shields the biosphere against harmful UV radiation. The ozone layer, which contains about 90% percent of total amount of ozone in the atmosphere, lies in the stratosphere between about 15 and 35 km altitude. The TOC varies with location and season [1].

Several types of instruments are available for measuring UV irradiance. They include spectroradiometers, multi-channel, narrow bandwidth filter instruments, multi-channel, moderate bandwidth filter instruments, and broadband filter instruments. The NILU-UV instrument, used in this study, belongs to the multi-channel, moderate bandwidth filter instrument class. It measures UV irradiance in five UV channels and one visible channel, which can be used to infer TOC values and UV dose rates.

Many studies have been conducted based on multi-channel, moderate bandwidth filter instruments, including the ground-based UV (GUV) instrument (manufactured by Biospherical Instruments, USA) and the Norwegian Institute for Air Research UV (NILU-UV) instrument (manufactured by Innovation NILU, Norway) [2–6]. A recent study compared NILU-UV measurements of solar UV radiation at four sites on the Tibetan plateau at altitudes ranging from 2995 to 4510 m with very high UV exposure. In that study good agreement was found between TOC values derived from NILU-UV measurements and inferred from the Ozone Monitoring Instrument (OMI) deployed on NASA’s Aura satellite, the difference between average TOC values being less than 2.5% at all sites [7].

The concept of an artificial neural network is a new interdisciplinary subject, and its development is pursued in several fields including neural science, cognitive science, information science, computer science, and artificial intelligence. It reflects many basic functions of the human brain and has nonlinear self-adaptive, self-organizing and self-learning features [8]. In recent years, neural networks have been successfully used in many scientific fields such as pattern recognition [9], function approximation [10], noise deduction [11], 3-D image recognition [12], and forecasting of consumer behavior [13]. In 2005 Blackwell [14] proposed using a neural network method to retrieve atmospheric temperature and moisture, while Maitha et al. [15] presented a successful application of neural networks for predicting global solar radiation in Al Ain City UAE. Neural networks has also been used to retrieve TOC from space borne instruments. Møller et al. (2000) [16] provided an approach based on a neural network to retrieve TOC from GOME data, while Diego et al. (2012) [17] reported on the use of a neural network for the creation of long-term climate data records from satellite data.

In this paper, a new method based on radial basis function neural network (RBF-NN) training is successfully employed to analyze data from multi-channel, moderate bandwidth filter instruments, such as the NILU-UV instrument. Due to the fast learning rate and rapid speed of convergence of the RBF-NN, we find it to be very convenient and useful for retrieval of TOC and cloud optical depth (COD) from UV irradiance data collected by ground-based instruments such as the NILU-UV device. We use a radiative transfer model (RTM) to compute irradiances in the NILU-UV channels (output parameters) as a function of three input parameters consisting of (i) the solar zenith angle, (ii) the TOC, and (iii) the COD. The resulting model data set is used in the RBF-NN to create a relationship between the input and output parameters in terms of a set of coefficients. To retrieve the desired atmospheric parameters (TOC and COD), we apply these coefficients along with data from a NILU-UV instrument.

The paper is organized as follows. In section 2, we introduce the NILU-UV and OMI instruments. In Section 3 we discuss the radiative transfer model used in this work. We describe the RBF-NN methodology and the traditional LUT method in Section 4. In Section 5, we discuss the main results derived by using the RBF-NN approach and provide a comparison of TOC values derived by applying the RBF-NN and LUT methods to NILU-UV data, and corresponding TOC values inferred from the OMI deployed in space. In the LUT approach a “radiation modification factor” (RMF) is used to account for environmental factors including cloud cover and surface reflection, while a COD is used as a proxy for such effects in the RBF-NN method. A comparison of RMF and COD values is also presented in Section 5. A conclusion is provided in Section 6.

2. Instruments

2.1. The NILU-UV instrument

The NILU-UV device measures irradiances in one visible (400–700 nm) and five UV channels with center wavelengths at 305, 312, 320, 340, and 380 nm, each with a 10 nm spectral width, full-width-at half-maximum (FWHM). These irradiances can be used to infer TOC and RMF values as well as UV dose rates [18–20]. The NILU-UV instrument is temperature-stabilized, suitable for deployment in harsh environments, and has a built-in data logger to record data. It relies on moderate bandwidth filters to record data in the five UV channels. The sensitivity of these filters is not stable, but tend to drift with time. To correct for that drift it is necessary to do relative calibrations at regular time intervals, typically twice per month.

A method for absolute calibration of NILU-UV type instruments was developed by Dahlback [18] and was further discussed by Høiskar et al. [20]. If a lamp were to be used to illuminate a NILU-UV instrument hemispherically, then the photodetector inside the NILU-UV instrument would measure a voltage given by

V_{i} = \int_{0}^{\infty} k_{i} R_{i}^{'} (λ) F (λ) d λ

where k_i is a constant and R′_i(λ) is the relative spectral response function of channel labeled i of the NILU-UV instrument and F(λ) is the irradiance at wavelength λ.

In order to make the absolute calibration accurate, the Sun is used as the light source, and a spectroradiometer is used to measure the spectral irradiance F(λ). At the same time and the same place a NILU-UV instrument is used to measure the voltage V_i, so that we can calculate the constant k_i from

k_{i} = \frac{V_{i}}{\int_{0}^{\infty} R_{i}^{'} (λ) F (λ) d λ} .

Once the k_i is determined, the voltage V_i that the instrument would measure when it is exposed to any spectral irradiance F(λ) (measured or calculated) can be determined using Eq. (1). Absolute calibration of the NILU-UV instrument used in this study was done on June 3, 2010 in Oslo, Norway against a Bentham BM150BC double grating spectroradiometer, which is the reference instrument for the Norwegian UV monitoring network (Aalerud and Johnsen, 2006 [21]). The NILU-UV instrument was optically characterized including measurements of both the relative and the spectral response function R′_i(λ) and the cosine response at the optical laboratory of the Norwegian Radiation Protection Authority (NRPA) in 2005.

The calibration uncertainty of multi-channel instruments given by Aalerud and Johnsen (2006) [21] is about ± 6%. According to the report by Corder et al. (2008) [22], the overall uncertainty of spectroradiometers for the absolute calibration lies between ± 7%, and based on uncertainties due to factors directly related to multi-channel instruments, the overall uncertainty of the NILU-UV instrument was estimated to be within ± 8% [7].

The NILU-UV measures the hemispherical irradiance on a flat surface, and its angular response deviates from perfect cosine response as discussed by Høiskar et al. [20]. However, Norsang et al. [7] pointed out that the largest deviation of the NILU-UV instrument from perfect cosine response is less than 3%. We ignored this small deviation from a perfect cosine response in this study because it is expected to lead to small errors.

Three such instruments were deployed on the roof of the Burchard building on the campus of Stevens Institute of Technology in Hoboken, New Jersey in August 2009. It has been demonstrated that these three instruments provided essentially identical TOC and RMF values [23]. In this study, we use one of them (NILU-UV 29) to demonstrate the merits of the new method.

2.2. Ozone monitoring instrument

The Ozone Monitoring Instrument (OMI) is a satellite instrument deployed on NASA’s AURA satellite. AURA was launched in July 2004 and its main purpose is to study the Earth’s atmosphere. AURA’s swath is 2600 km and the nadir viewing foot print is 13×24 km². The OMI measures the TOC in the atmosphere as well as aerosol loading and UV radiation. Detailed information about the OMI is available at http://www.nasa.gov/mission-pages/aura/spacecraft/omi.html.

The OMI algorithm currently used for retrieval of the TOC in the atmosphere is version 8.5, which employs backscattered radiances in only two channels. For solar zenith angles smaller than θ₀ = 80°, the two channels are centered at 317.5 nm and 331.2 nm. At the shorter 317.5 nm wavelength, the radiation is quite sensitive to the TOC, while the radiation at the 331.2 nm wavelength is much less sensitive. For θ₀ values larger than 80°, channels centered at 331.2 nm and 360 nm are used. For all values of θ₀ the shorter wavelength is used to estimate TOC values, and the longer wavelength is used to estimate an effective cloud fraction based on the Mixed Lambert Equivalent Reflectivity (MLER) model [24]. The effective cloud fraction is used to estimate the impact of clouds on the backscattered radiation in order to improve the accuracy of the inferred TOC value in the presence of clouds. Detailed information about the algorithm is available at the NASA website: http://eospso.gsfc.nasa.gov as well as in a paper by Barthia et al. [25]. The uncertainty of the OMI algorithm, also reported by Bhartia et al. [25], is ± 2%, and the uncertainty in TOC values retrieved by the OMI algorithm is less than ± 5 DU [26].

3. Atmospheric radiative transfer model

When solar radiation passes through the ozone layer of the atmosphere, a portion of the UV radiation will be absorbed by ozone, while the portion that penetrates the ozone layer will be multiply scattered or absorbed by air molecules, aerosols, and cloud particles [27]. Therefore, a RTM is needed to quantify how the UV radiation is affected by ozone, other molecules and particles in the atmosphere, and to compute the fraction of the incoming solar radiation that reaches the Earth’s surface. In this study, we used the DISORT (discrete ordinate) RTM developed by Stamnes and co-workers [28, 29], with corrections for Earth curvature effects [30] to simulate the radiation measured by the NILU-UV instruments. In the DISORT RTM, we used 16 streams and the US standard atmosphere [31]. Further, we used an extra-terrestrial solar spectrum covering the range from 200 nm to 800 nm with a resolution of 1 nm, according to Woods et al. [32] for the 200 nm to 420 nm spectral range, and according to the spectrum adopted in the Moderate resolution atmospheric transmission model (Modtran) [Berk et al. [33]] for the 420 nm to 800 nm spectral range.

4. Methodology

4.1. LUT methodology

4.1.1. Total ozone column (TOC) amount

The TOC is determined from the ratio N of irradiances in two different UV channels with spectral responses R_i(λ) and R_j(λ), one of which is quite sensitive to the TOC, while the other is significantly less sensitive, i.e.

N (θ_{0}, TOC) = \frac{\sum_{λ = 0}^{\infty} R_{i} (λ) F (λ, θ_{0}, TOC)}{\sum_{λ = 0}^{\infty} R_{j} (λ) F (λ, θ_{0}, TOC)}

where θ₀ is the solar zenith angle and F(λ, θ₀, TOC) is the spectral irradiance. We used the ratio involving irradiances from channel 3 and channel 1 (centered at 305 nm). In order to derive the TOC, we used the RTM to create a LUT of irradiances as a function of θ₀ and TOC, as described by Dahlback (1996) [18]. In the RTM the TOMS version 7 pressure, ozone and temperature profiles [34] were used. Since cloud cover and surface albedo do not affect the ratio of irradiances in the two channels used for TOC retrieval too much (for a COD of 100 and a surface albedo of 0.8 the error is less than 20 DU for TOC retrievals) the LUT was created for a cloud-free sky and a totally absorbing surface. By comparing the measured N value with the computed N value stored in the LUT, one can infer the TOC from NILU-UV data.

4.1.2. Cloud optical depth and radiation modification factor

Clouds, aerosols, and surface reflection have a significant impact on incoming solar radiation through reflection, absorption and multiple scattering, which affect the radiation reaching the Earth’s surface both in the UVA and UVB bands [35]. The attenuation of solar radiation by clouds depends weakly on wavelength, while the wavelength dependence of radiation absorption by ozone is very strong. Although the derivation of the TOC relies on the ratio of the measured irradiances in two channels, which tend to cancel out the cloud effect, the accuracy of the derived TOC deteriorates for optically thick clouds. Dahlback [18] showed that for a stratified cloud layer with optical depth (COD) of 100 located between 2 km and 4 km (zero surface albedo), the error in the inferred TOC is less than 2 DU compared with the TOC obtained in the absences of clouds. But the error will increase if the surface albedo increases. Thus, for a cloud with COD of 100 located between 2 km and 4 km and a surface albedo of 0.8, the error is larger, but still less than 20 DU.

The radiation modification factor (RMF) is a simple way to quantify the combined effect of clouds, aerosols, and surface reflection on measured UV irradiances. The RMF is defined as the ratio of the measured irradiance and the irradiance computed by the RTM under cloud-free sky conditions for a ground surface at sea level that is assumed to be totally absorbing (black) [20], i.e.

RMF = \frac{F^{m} (θ_{0})}{F^{c} (θ_{0})} \times 100

where θ₀ is the solar zenith angle, F^m(θ₀) is the measured irradiance and F^c(θ₀) the computed irradiance (using the RTM) under cloud-free sky conditions [29]. To determine the RMF we used channel 4 (centered at 340 nm) of the NILU-UV instrument.

4.2. Radial basis function neural network methodology

4.2.1. Radial basis function neural network

The RBF-NN was proposed by Broomhead et al. [36] in 1988. The main feature of a radial basis function is that the response depends on the distance from its center. The RBF-NN employs radial basis functions (RBFs) as the activation functions, and the smaller the distance from its center, the larger the activation. The center and the width of the RBF are the basic parameters of the neural network [37]. In most cases, a RBF is taken to be a Gaussian function. A RBF-NN consists of three strata: an input layer, hidden layers, and an output layer [38], and it has two hidden layers: a RBF layer and a linear layer. The output after the RBF layer is commonly written as

O_{1 i} = exp [- {((x_{1 i} - w_{1 i}) b_{1})}^{2}]

where x_1i is the input, w_1i is the weight, b₁ is called the bias, and O_1i is the output of the RBF layer. After the RBF layer, there is a linear layer given by

O_{2 i} = w_{L i} x_{2 i} + b_{2 i}

where w_Li is a weight and b_2i is the bias of the second layer. The final output of the whole RBF-NN is given by O_2i. We need to combine the two hidden layers. If there are N_in input parameters, the equation for the first hidden layer becomes

O_{1 i} = exp [- {(b_{1} \sum_{k = 1}^{N_{in}} (w_{1 k} - x_{1 k}))}^{2}] .

The input to the linear layer is the output of the RBF layer, the same as Eq. (7), which upon substitution in Eq. (6) yields

O_{2 i} = w_{L i} exp [- {(b_{1} \sum_{k = 1}^{N_{in}} (w_{1 k} - x_{1 k}))}^{2}] + b_{2 i} .

If there are a total of N neurons, the ith output of the RBF-NN becomes:

O_{2 i} = \sum_{j = 1}^{N} w_{L i j} exp [- {(b_{1} \sum_{k = 1}^{N_{in}} (w_{1 j k} - x_{1 k}))}^{2}] + b_{2 i} .

To simplify the notation, we set a_ij = w_Lij, b = b₁, d_i = b_2i, c_jk = w_1jk, R_k = x_1k, and set the ith output of the whole RBF-NN to p_i = O_2i, so that the complete RBF-NN function turns into:

p_{i} = \sum_{j = 1}^{N} a_{i j} exp [- {(b \sum_{k = 1}^{N_{in}} (c_{j k} - R_{k}))}^{2}] + d_{i}

where N is the total number of neurons and N_in is the number of input parameters. The purpose of the training of the RBF-NN is to determine the coefficients a_ij, b, c_jk, d_i appearing in Eq. (10). In our study, the input parameters R_k are the irradiances in three of the UV channels (305, 320, and 340 nm) plus the solar zenith angle, and the output parameters p_i are the TOC and COD values.

4.2.2. Design of the training data

Getting adequate data for the neural network training to calculate the coefficients is very important in order to get accurate final results from Eq. (10). Hence, in order to apply the RBF-NN, we need a set of model data for the training. We used the RTM described in Section 3 to generate the model data. The three input parameters are: θ₀ (20–70°), TOC (200–500 DU), and COD (0–150). A total of 20,000 different combinations of the three input parameters were generated by randomly sampling each of them within the ranges indicated above, and then we used these 20,000 combinations of the three input parameters in the RTM to simulate the irradiances in the three UV bands centered at 305 nm (channel 1), 320 nm (channel 3), and 340 nm (channel 4) to construct a dataset consisting of 20,000 synthetic “measurements” covering the range of the input parameters densely enough to ensure adequate accuracy.

The RMF is a simple way to quantify the effects of clouds, aerosols and surface reflectance on the UV irradiances measured by an instrument such as the NILU-UV. To simulate the impact of clouds on UV irradiances in the RTM we used the COD as a proxy for describing the combined effect of clouds, aerosols, and surface reflectance. The COD is a measure of the attenuation due to absorption and scattering of sunlight by cloud particles (water droplets or ice crystals). For simplicity we assumed that the cloud consisted of water droplets in this study, and cloud inherent optical properties were calculated using the parameterization developed by Hu and Stamnes (1993) [39]. Figure 1 shows the relation between the RMF [Eq. (4)] and the COD obtained from the generated model data.

Fig. 1 Relation between RMF and COD based on simulated data obtained from the RTM.

Download Full Size | PDF

4.2.3. Use of synthetic data to train the neural network

We used our model data to train the RBF-NN. For the RBF-NN, the spread, proportional to b⁻¹ in Eq. (10) represents the width of the RBFs. If the weighted input is taken to lie in the range between − spread and + spread, the result of the RBF will be distributed in the FWHM. The range of the spread is taken to lie between 0 and 100. If the spread is large enough, the transfer function of the neural network will be smooth. But if the spread is too large, then a lot of neurons will be needed to fit a rapidly changing function. If the spread is too small, the neural network may not converge. We have to experiment to find the best value of the spread. We found a spread value of 1 to be a good choice in our case. The target parameters are the TOC and the COD. The following steps were repeated until the number of neurons of the network reached an optimum value or the mean squared error (MSE) reached the goal of the MSE: (i) simulate the network based on weights, (ii) calculate the MSE and compare with the MSE’s goal, (iii) add a neuron, and (iv) adjust the weights. After training using our model data and the RBF-NN, we got a set of trained coefficients: a_ij, b, c_jk, d_i which we used in Eq. (10) with O_i being either COD or TOC to infer the two retrieval parameters.

To verify the correctness of our trained coefficients, we used the 20,000 synthetic irradiances of the three channels as input to Eq. (10) with the trained coefficients a_ij, b, c_jk, d_i to infer TOC and COD by the RBF-NN method. The correlation between the inferred TOC values and the input values was 0.997. Thus, we can conclude that the trained coefficients are accurate. The correlation used here is defined as [40]

correl (x, y) \equiv \frac{E [(x - E [x]) (y - E [y])]}{σ_{x} σ_{y}}

where E is the expectation value operator, and σ_x and σ_y are the standard deviations. The correlation is a measure of the degree of validity of a linear relationship between two variables.

4.2.4. Retrieval of COD and TOC

Applying Eq. (10) with the trained coefficients a_ij, b, c_jk, and d_i to the NILU-UV measurements adjusted by the instrument response function and instrument drift factors [23], we inferred the TOC and COD values. We used the solar zenith angle θ₀, the ratio of irradiances of channel 1 and channel 3, and the irradiance of channel 4 from the NILU-UV 29 instrument as input to Eq. (10), from which the TOC and COD values were derived. The TOC values derived from RBF-NN, LUT, and OMI were generally in good agreement, and the COD value derived from the RBF-NN and the RMF values derived from the LUT fitted the relation presented in Fig. 1 as will be discussed in more detail below.

5. Comparisons

We used the RBF-NN method to infer TOC and COD values from data obtained by the NILU-UV 29 instrument from 08/05/2010 to 03/01/2013. Figure 2 shows the TOC and COD values derived by the RBF-NN method. We note that there is good agreement between the TOC derived from OMI and NILU-UV. The maximum value of the difference between TOC inferred from OMI and from NILU-UV is 13.45 DU, while the standard deviation of differences is 2.83 DU, and the median of the difference is only 0.178 DU. Additionally we found that the minimum TOC values occur around December every year.

Fig. 2 Daily averaged TOC and COD values derived by the RBF-NN method using data from the NILU-UV 29 instrument from 08/05/2010 to 03/01/2013. TOC values derived from the OMI are also included in the upper panel for comparison.

Download Full Size | PDF

5.1. Comparisons of COD and TOC values derived by the RBF-NN and LUT methods

In this section, we compare COD and TOC values derived by using the RBF-NN method with RMF and TOC values derived from the LUT method based on measurements recorded by the NILU-UV 29 instrument every day (minute-by-minute) from 08/05/2010 to 03/01/2013. Figure 3(a) shows correlations between TOC derived from the RBF-NN and LUT methods in 2012. The corresponding figures for the other years (not shown here) are similar to Fig. 3(a). The correlations through 2010 to 2013 vary little with time and are larger than 0.99, indicating a good match, as expected because the physics is the same in both methods. Figure 3(b) shows the relation between COD and RMF which is the similar to the model data (Fig. 1) except for some points in red color above the curve, which are caused by incorrect RMF values occurring for heavy cloud cover, broken clouds, or snow on the ground. According to the definition of the RMF [Eq. (4)], its maximum value should be 100 for a cloud-free sky. Hence, we may consider RMF values larger than 100 to be “incorrect”. When the cloud coverage is within 5–7oktas, the sky is said to have broken cloud cover. Heavy cloud cover means that the cloud coverage is 8 oktas which is complete overcast. An abrupt and sharp increase in the surface albedo usually indicates snow on the ground. Broken cloud cover and/or snow on the ground will enhance the downwelling irradiance above the cloud-free sky case and lead to RMF values that are larger than 100. On days with incorrect RMF values, the TOC values in red color derived by the LUT are different from those derived by the RBF-NN method. The main difference is that the RMF value is derived directly from one channel (340 nm), and is expected to be unreliable under broken cloud conditions. The COD is based on a simultaneous retrieval of COD and TOC, and is therefore expected to give a more reliable representation of the combined effect of cloud/aerosol/surface conditions. COD values derived by the RBF-NN method are more reliable than the RMF values, which are incorrect for heavy cloud cover and broken cloud situations or snow-covered ground. TOC values derived by the RBF-NN method are more accurate for all weather conditions, because in the RBF-NN method, the cloud effect is accounted for in the TOC retrieval, while in the LUT method it is ignored.

Fig. 3 (a) Correlations between TOC values derived by the RBF-NN and LUT methods in 2012. Red points represent TOC values on days with RMF values in excess of 100, and blue points represent TOC values on days with RMF values smaller than 100. (b) Relation between COD values derived using the RBF-NN method and RMF values derived using the LUT method in 2012. Red points represent RMF and COD values on days with RMF values in excess of 100, while blue points indicate RMF and COD values on days with RMF values smaller than 100.

Download Full Size | PDF

Table 1 shows the mean relative differences and the standard deviations of the relative differences between TOC values derived by the RBF-NN and LUT methods for the 4 years. Evidently, the mean relative difference does not vary too much with time. It is smaller than 1.5%, and the standard deviation is smaller than 0.95. Here the relative difference is defined as [41]

rel diff (x, x_{ref}) \equiv \frac{x_{ref} - x}{x_{ref}} \times 100

where x_ref is a TOC reference value derived by RBF-NN method. One interesting finding is that by using the RBF-NN method, we retrieved more valid results than by using the RBF-NN method. In 2010, data collection started on August 5, and the RBF-NN method provided valid retrievals for 148 days, while the LUT method provided valid retrievals for 143 days. By “valid retrievals”, we mean that the daily average of the retrieved value lies in a meaningful range (200–500 DU). In 2011, data were collected for the entire year, and the RBF-NN method provided valid retrievals for 353 days, while the LUT method provided valid retrievals for only 331 days. In 2012, the RBF-NN method provided valid retrievals for 355 valid days, and the LUT for 329 days. In 2013, only two months of data (January and February) were analyzed, and the RBF-NN method provided valid retrievals for 57 days, and the LUT method for 54 days. In total, the RBF-NN method provided valid retrievals for 914 days and the LUT method for only 857 days.

Table 1. Comparisons of TOC values derived by the RBF-NN and LUT methods.

View Table | View all tables in this article

Clouds affect the accuracy of the inferred TOC values. Figure 4 shows the ratio between the TOC value derived from the RBF-NN method and that inferred from the LUT method versus the COD. A ratio smaller (larger) than 1 means the TOC value inferred from the LUT method is overestimated (underestimated) compared with that derived from the RBF-NN method.

Fig. 4 The impact of environmental effects modeled as a cloud optical depth (COD) and a corresponding RMF on TOC values derived from RBF-NN and LUT for four different years (2010–2013).

Download Full Size | PDF

Color is used to indicate the number of TOC values for a specific value of the ratio in a COD interval of 0.005. From blue to red, the number of TOC values indicated by color ranges from 10⁰ to 10⁴. 80% of the TOC values correspond to COD values less than 10. When the COD value increases, the ratio decreases. Figure 4 shows that in 2011 and 2012, for COD values less than 5 (relatively clear sky), the ratio lies between 1.02 and 0.98. When the COD increases to 150, the ratio decreases to 0.9. In 2010 (08/05–12/31) and 2013 (01/01–03/01), when data for only a fraction of the whole year were available, the COD value was less than 5, and the ratio was close to 1.

We can conclude that TOC values derived by the RBF-NN method are in agreement with those inferred from the LUT method. The relation between COD and RMF values agrees with that shown in Fig. 1 except for some “incorrect” RMF values larger than 100 due to heavy cloud cover, broken clouds or snow-covered ground.

5.2. Comparisons of TOC from the NILU-UV with RBF-NN method and OMI

While OMI only provides a daily averaged value of the TOC, the NILU-UV instrument records data every minute. In order to compare TOC values from OMI with corresponding values obtained from NILU-UV measurements, we calculated the daily averaged TOC values by applying both the RBF-NN and LUT methods to the NILU-UV data. Our results show that RBF-NN-derived TOC values have a better agreement with OMI TOC values than LUT-derived TOC values. The relative differences are smaller and the correlations higher between TOC values retrieved from the RBF-NN method and OMI than those between TOC values inferred from the LUT method and OMI. In this case we used the OMI-inferred TOC value as a reference for calculating the relative differences. The mean relative differences and standard deviations for the TOC values derived from OMI and from RBF-NN and LUT methods applied to NILU-UV data for the period 2010–2013 are given in Table 2. The mean relative differences and standard deviations for OMI versus RBF-NN are smaller than for OMI versus LUT. The mean relative difference for OMI versus RBF-NN is less than or close to that for OMI versus LUT every year and the standard deviation for OMI versus RBF-NN is significantly smaller than that for OMI versus LUT.

Table 2. Mean relative differences and standard deviations between TOC values derived from OMI and from NILU-UV measurements using the RBF-NN and LUT methods.

View Table | View all tables in this article

Table 3(a) shows correlations between TOC values derived from OMI data and from NILU-UV measurements using the RBF-NN and LUT methods. RBF-NN-derived values match OMI values better than LUT-derived values. The correlation for RBF-NN-derived values is about 0.03 larger for every year except for 2013. In 2013, only 60 days for January and February were analyzed with fewer overcast days than in summer. Clouds affect the agreement between TOC values derived from OMI and those obtained by applying the RBF-NN method to NILU-UV data. Table 3(b) shows correlations between these two sets of TOC values for different COD values. For COD values smaller than 5 (relatively clear sky condition), the correlation is 0.9812. For COD values between 5 and 10, the correlation is 0.9688, and for COD values between 10 and 20, the correlation is 0.9570. For COD values larger than 20 (heavily overcast sky condition), the correlation drops to 0.9469. Thus, the smaller the COD value, the higher the correlation, and the better the agreement between TOC values derived from the OMI and those obtained by applying the RBF-NN method to NILU-UV data.

Table 3. (a) Correlations between TOC values derived from OMI and NILU-UV data using the RBF-NN and LUT methods. (b) Correlations between TOC values derived from OMI and RBF-NN for different COD values.

View Table | View all tables in this article

Clouds appear to have a smaller effect on the correlation between TOC values derived from OMI and those obtained by applying the RBF-NN method to NILU-UV data than they have on the correlation between TOC values derived from OMI and those obtained by applying the LUT method to NILU-UV data. Figure 5 shows how the COD value impacts the agreement between TOC values derived from OMI and by applying the RBF-NN method to NILU-UV data (left panel), and also the agreement between TOC values derived from OMI and by applying the LUT method to NILU-UV data (right panel). The left panel shows the distribution of the ratio of the TOC value from RBF-NN to that from OMI for different COD values in the period 2010–2013, while the right panel shows the distribution of the ratio of the TOC value from LUT to that from OMI for different RMF values in the same period. Colors are used to indicate the number of a specific value of the ratio in a certain COD or RMF interval. For OMI versus RBF-NN, 80% of the data correspond to COD values less than 10, while for OMI versus LUT, 80% of the data correspond to RMF values larger than 80. The larger the COD or the smaller the RMF, the more the ratio deviates from 1.0. When the ratio is larger (smaller) than 1.0, the TOC from RBF-NN or LUT is larger (smaller) than the TOC from OMI. Thus, TOC from RBF-NN or LUT is overestimated compared with TOC from OMI on overcast days. There are much more values of the ratio close to 1 in the RBF-NN versus OMI panel than in the LUT versus OMI panel. Also, the ratio interval is larger in the LUT versus OMI panel than in the RBF-NN versus OMI panel.

Fig. 5 Left: COD impact on the ratio of the TOC values derived from the RBF-NN method and OMI (2010–2013). Right: RMF impact on the TOC values derived from the LUT method and OMI (2010–2013).

Download Full Size | PDF

Although the portion of the TOC due to tropospheric ozone below the level of typical clouds in the atmosphere is expected to be quite small, the presence of clouds presents a challenge for satellite observations of the TOC. As discussed by Anton and Loyola (2011) [42] TOC values obtained from satellite observations for cloudy conditions show a significant underestimation that tends to decrease with increasing solar zenith angle. Compared with RBF-NN retrievals, TOC values obtained from OMI are underestimated under cloudy conditions. The heavier the cloud cover, the larger the underestimation. Thus, when the COD is larger than 20, the mean relative difference is (−2.17 ± 3.74)%; when the COD is between 20 and 10, the mean relative difference is (−0.75 ± 3.26)%; when the COD between is 10 and 5, the mean relative difference is (−0.19 ± 2.45)%; when the COD is smaller than 5, the mean relative difference is (0.75 ± 2.10)%.

In summary, TOC values from RBF-NN and OMI are in better agreement than those from LUT and OMI. Environmental factors including clouds, aerosols, and surface reflectance, represented in this study by a COD or RMF value as a proxy, have a bigger negative impact on the accuracy of TOC values derived by applying the LUT method rather than the RBF-NN method to NILU-UV data. The reason is that the “cloud effect” is explicitly accounted for in the inference of TOC by the RBF-NN method while it is not taken into account in the LUT method.

6. Conclusions

A method for simultaneous derivation of total ozone column (TOC) amount and cloud optical depth (COD) from moderate bandwidth, multi-channel instruments has been developed. This new method relies on using a radial basis function neural network (RBF-NN) to derive TOC and COD values directly from the measured irradiances in three UV channels. The RBF-NN is trained by using a radiative transfer model to simulate the measured UV irradiances as a function of solar zenith angle, TOC value and COD value. Applying the method to three years of data recorded by a NILU-UV irradiance meter using channels at 305, 320, and 340 nm, we conclude that the RBF-NN method generally yields results in close agreement with those derived from the traditional lookup table (LUT) method.

However, the RBF-NN method is less influenced by environmental factors including clouds, aerosols, and surface reflectance, represented in this study as a “cloud effect”, than the LUT method, and TOC values derived from the RBF-NN method are in better agreement with corresponding results derived from OMI. Thus, compared to the LUT method, the RBF-NN method yields an increase of 0.03 in the correlation with OMI results. Furthermore, the RBF-NN method retrieves more valid results than the LUT method. One plausible reason is that both the RBF-NN and the OMI methods take “cloud effects” into account, while the LUT method ignores “cloud effects” in the TOC retrieval. In essence, by performing a simultaneous retrieval of TOC and COD values from the NILU-UV measurements, the RBF-NN method leads to improved accuracy compared to the LUT method.

Compared with RBF-NN retrievals, TOC values obtained from OMI are underestimated under cloudy conditions, and the heavier the cloud cover, the larger the underestimation. These results agree with conclusions reached by Anton and Loyola (2011) [42] that TOC values obtained from satellite observations for cloudy conditions show a significant underestimation compared to ground-based observations obtained by direct sun observations with Brewer spectrometers.

References and links

1. B. Lapeta, I. Dyras, and Z. Ustrnu, “Homogenization of the total ozone amount series derived from NOAA/TOVS data,” in Proceedings of International TOVS Study Conference, 4–10, Maratea, Italy (2006), pp. 599–605.

2. A. Dahlback, H. A. Eide, B. A. K. Høiskar, R. O. Olsen, F. J. Schmidlin, S. C. Tsay, and K. Stamnes, “Comparison of data for ozone amounts and ultraviolet doses obtained from simultaneous measurements with various standard ultraviolet instruments,” Opt. Eng. 44, 041010 (2005). [CrossRef]

3. A. Dahlback, N. Gelsor, J. J. Stamnes, and Y. Gjessing, “UV measurements in the 3000–5000 m altitude region in Tibet,” J. Geophys. Res. 112, D09308 (2007).

4. A. Kazantzidis, A. F. Bais, M. M. Zempila, C. Meleti, K. Eleftheratos, and C. S. Zerefos, “Evaluation of ozone column measurements over Greece with NILU-UV multi-channel radiometers,” Int. J. Remote Sens. 30(15–16), 4273–4281 (2009). [CrossRef]

5. G. Norsang, L. Kocbach, W. Tsoja, J. J. Stamnes, A. Dahlbach, and P. Nema, “Ground-based measurements and modelling of solar UV-B radiation in Lhasa, Tibet,” Atmos. Environ. 43, 1498–1502 (2009). [CrossRef]

6. G. Norsang, L. Kocbach, J. J. Stamnes, W. Tsoja, and N. Pingcuo, “Spatial distribution and temporal variation of solar UV radiation over the Tibetan Plateau,” Appl. Phys. Res. 3, 37–46 (2011). [CrossRef]

7. G. Norsang, Y. Chen, N. Pingcuo, A Dahlback, Ø. Frette, B. Kjeldstad, B. Hamre, K. Stamnes, and J. J. Stamnes, “Comparison of ground-based measurements of solar UV radiation at four sites on the Tibetan Plateau,” Appl. Opt., in press (2014). [CrossRef] [PubMed]

8. M. H. Beale, M. T. Hagan, and B. Demuth, Neural Network Toolbox User’s Guide (The MathWorks. Inc, 2013).

9. C. M. Bishop, Neural Networks for Pattern Recognition (Clarendon, 1995).

10. S. Whiteson and P. Stone, “Evolutionary function approximation for reinforcement learning,” J. Mach. Learn. Res. 7, 877–917 (2006).

11. S. Tamura and A. Waibel, “Noise reduction using connectionist models,” in ICASSP (1988), pp. 553–556.

12. J. Matej and R. M. Lewitt, “Practical consideration of 3-D image reconstruction using spherically symmetric volume elements,” IEEE Trans. Med. Imag. 15, 68–78 (1996). [CrossRef]

13. Z. Caiqing, Q. Ruonan, and Q. Zhiwen, “Comparing BP and RBF neural network for forecasting the resident consumer level by MATLAB,” in International Conference on Computer and Electrical Engineering (2008), pp. 169–172.

14. W. J. Blackwell, “A neural-network technique for the retrieval of atmospheric temperature and moisture profiles from high spectral resolution sounding data,” IEEE Trans. Geosci. Remote Sens. 43, 2535–2546 (2005). [CrossRef]

15. H. Maitha, A. Shamisi, A. H. Assi, and H. A. N. Hejase, “Engineering education and research using MATLAB,” Using MATLAB to Develop Artificial Neural Network Models for Predicting Global Solar Radiation in Al Ain City, UAE (CC BY 3.0US, 2011).

16. M. D. Müller, A. K. Kaifel, and M. Weber, “Ozone from GOME data using neural network technique,” in Proc. Looking Down to Earth in the New Millenium Symp. (2000).

17. D. G. Loyola and M. Coldewey-Egbers, “Multi-sensor data merging with stacked neural networks for the creation of satellite long-term climate data records,” EURASIP J. Adv. Sig. Pr. 1, 1–10 (2012).

18. A. Dahlback, “Measurements of biologically effective UV doses, total ozone abundances, and cloud effects with multichannel, moderate bandwidth filter instrument,” Appl. Opt. 35, 6514–6521 (1996). [CrossRef] [PubMed]

19. K. Stamnes, J. Slusser, and M. Bowen, “Derivation of total ozone abundance and cloud effects from spectral irradiance measurements,” Appl. Opt. 30, 4418–4426 (1991). [CrossRef] [PubMed]

20. B. A. K. Høiskar, R. Haugen, T. Danielsen, A. Kylling, K. Edvardsen, A. Dahlback, B. Johnsen, M. Blumthaler, and J. Schreder, “Multichannel moderate-bandwidth filter instrument for measurement of the ozone-column amount, cloud transmittance, and ultraviolet dose rates,” Appl. Opt. 42, 3472–3479 (2003). [CrossRef] [PubMed]

21. T. N. Aalerud and B. Johnsen, “The Norwegian UV Monitoring Network,” Norwegian Radiation Protection Authority, Østerås, Norway (2006).

22. R. R. Cordero, G. Seckmeyer, D. Pissulla, L. DaSilva, and F. Labbe, “Uncertainty evaluation of spectral UV irradiance measurements,” Meas. Sci. Technol. 19, 045104 (2008). [CrossRef]

23. L. Fan, W. Li, A. Dahlback, J. J. Stamnes, S. Stamnes, and K. Stamnes, “Comparisons of three NILU-UV instruments deployed at the same site in the New York area,” Appl. Opt. 53(17), 3598–3606 (2014). [CrossRef] [PubMed]

24. Z. Ahmad, P. K. Bhartia, and N. Krotkov, “Spectral properties of backscattered UV radiation in cloudy atmospheres,” J. Geophys. Res. 109, D01201 (2003).

25. P. K. Bhartia and C. W. Wellemeyer, “OMI TOMS-V8 Total O3 Algorithm, Algorithm Theoretical Baseline Document:OMI Ozone Products,” vol. II, ATBDOMI-02, version 2.0 (2002).

26. R. V. Martin, B. Sauvage, I. Folkins, C. E. Sioris, C. Boone, P. Bernath, and J. Ziemke, “Space-based constraints on the production of nitric oxide by lightning,” J. Geophys. Res. 112, D09309 (2007).

27. G. E. Thomas and K. Stamnes, Radiative Transfer in the Atmosphere and Ocean (Cambridge University, 1999). [CrossRef]

28. K. Stamnes, S. C. Tsay, W. J. Wiscombe, and K. Jayaweera, “Numerically stable algorithm for discrete-ordinate-method radiative transfer in multiple scattering and emitting layered media,” Appl. Opt. 27, 2502–2509 (1988). [CrossRef] [PubMed]

29. K. Stamnes, “The theory of multiple scattering of radiation in plane parallel atmospheres,” Rev. Geophys. 24, 299–310 (1986). [CrossRef]

30. A. Dahlback and K. Stamnes, “A new spherical model for computing the radiation field available for photolysis and heating at twilight,” Planet. Space Sci. 39, 671–683 (1991). [CrossRef]

31. U. S. Air Force, Gov. Print. Off., “U. S. Standard Atmosphere,” (NASA, 1976), available at http://modelweb.gsfc.nasa.gov.

32. T. N. Woods, D. K. Prinz, G. J. Rottmann, J. London, P. C. Crane, R. P. Cebula, E. Hilsenrath, G. E. Brueckner, M. D. Andrews, O. R. White, M. E. VanHoosier, L. E. Floyd, L. C. Herring, B. G. Knapp, C. K. Pankrantz, and P. A. Reiser, “Validation of the UARS solar ultraviolet irradiances: Comparison with the Atlas 1 and 2 measurements,” J. Geophys. Res. 101, 9541–9569 (1996). [CrossRef]

33. A. Berk, G. P. Anderson, P. K. Acharya, L. S. Bernstein, L. Muratov, J. Lee, M. Fox, S. M. Adler-Golden, J. H. Chetwynd, Ml, and Hoke, and others, “MODTRAN 5: a reformulated atmospheric band model with auxiliary species and practical multiple scattering options,” in Fourth International Asia-Pacific Environmental Remote Sensing Symposium 2004: Remote Sensing of the Atmosphere, Ocean, Environment, and Space, International Society for Optics and Photonics (2005), pp. 88–95.

34. R. D. McPeters, P. K. Bhartia, A. J. Krueger, J. R. Herman, B. M. Schlesinger, C. G. Wellemeyer, C. J. Seftor, G. Jaross, S. L. Taylor, T. Swissler, O. Torres, G. Labow, W. Byerly, and R. P. Cebula, “Nimbus-7 Total Ozone Mapping Spectrometer (TOMS) Data Products User’s Guide,” NASA Reference Publication(1996).

35. C. R. Roy, H. P. Gies, D. J. Lugg, S. Toomey, and D. W. Tomlinson, “The effect of clouds on enhancing UVB irradiance at the Earth’s surface: A one year study,” Geophys. Res. Lett. 27, 3337–3340 (1998).

36. D. S. Broomhead and D. Lowe, “Radial basis functions, multi-variable functional interpolation and adaptive networks,” in Royal Signals and Radar Establishment, Malvern, United Kingdom, No. RSRE-MEMO-4148 (1988).

37. M. J. L. Orr, “Introduction to radial basis function networks,” Technical report, University of Edinburgh (1996).

38. A. G. Bors, “Introduction of the radial basis function (RBF) networks,” OSEE 1, DSP Algorithms: Multimedia, 1–7 (2001).

39. Y. X. Hu and K. Stamnes, “An accurate parameterization of the radiative properties of water clouds suitable for use in climate models,” J. Clim. 6, 728–742 (1993). [CrossRef]

40. E. W. Weisstein, “Statistical Correlation,” available at http://mathworld.wolfram.com.

41. J. Bennett and W. Briggs, Using and Understanding Mathematics: A Quantitative Reasoning Approach (Pearson, 2008).

42. M. Antón and D. Loyola, “Influence of cloud properties on satellite total ozone observations,” J. Geophys. Res. 116, D03208 (2011).

Year	Mean relative difference %	STD of relative difference
2010	1.00	0.84
2011	0.87	0.94
2012	0.90	0.93
2013	1.46	0.52
Total	0.95	0.99

Year	Mean relative differences %		STD
Year	OMI vs RBF-NN	OMI vs LUT	OMI vs RBF-NN	OMI vs LUT
2010	0.71	0.55	2.68	3.54
2011	−0.36	−1.34	2.87	4.30
2012	−0.37	−1.66	2.51	3.98
2013	0.55	0.39	3.29	3.51
Total	−0.11	−1.07	2.82	4.11

(a)
Year	Correlations
Year	OMI vs NN	OMI vs LUT
2010	0.9509	0.9153
2011	0.9673	0.9313
2012	0.9674	0.9313
2013	0.9801	0.9712
Total	0.9678	0.9347

(b)
Cloud Optical Depth	Correlations
Cloud Optical Depth	OMI vs NN
COD < 5	0.9812
5 < COD < 10	0.9688
10 < COD < 20	0.9570
20 < COD	0.9469

Year	Mean relative difference %	STD of relative difference
2010	1.00	0.84
2011	0.87	0.94
2012	0.90	0.93
2013	1.46	0.52
Total	0.95	0.99

New neural-network-based method to infer total ozone column amounts and cloud effects from multi-channel, moderate bandwidth filter instruments

Abstract

1. Introduction

2. Instruments

2.1. The NILU-UV instrument

2.2. Ozone monitoring instrument

3. Atmospheric radiative transfer model

4. Methodology

4.1. LUT methodology

4.1.1. Total ozone column (TOC) amount

4.1.2. Cloud optical depth and radiation modification factor

4.2. Radial basis function neural network methodology

4.2.1. Radial basis function neural network

4.2.2. Design of the training data

4.2.3. Use of synthetic data to train the neural network

4.2.4. Retrieval of COD and TOC

5. Comparisons

5.1. Comparisons of COD and TOC values derived by the RBF-NN and LUT methods

5.2. Comparisons of TOC from the NILU-UV with RBF-NN method and OMI

6. Conclusions

References and links

Cited By

Figures (5)

Tables (3)

Equations (12)

Optics Express