1. Introduction

Insects are of major concern for human society. Vector-borne diseases account for 17% of all infectious diseases, leading to more than 700,000 deaths annually [1]. Mosquitoes are one of the deadliest disease vectors in the world. In 2015, malaria alone caused 438,000 deaths worldwide. Other mosquito-borne diseases include dengue, Zika, yellow fever viruses and more [2]. Besides, agricultural pests are a major source of crop loss every year. According to the Food and Agriculture Organization of the United Nations (FAO), during plagues of desert locusts (Schistocerca gregaria), 20 percent of the Earth’s land and more than 65 of the world’s poorest countries have been affected, and the livelihood of 10% of the world population could potentially be damaged [3]. In addition, pesticides applied to kill those pests can put not only insect predators in danger but also the farmers who work in the fields [4]. Meanwhile, insects can also play a role as pollen vectors. More than 75% of the world’s food crops and 35% of the world’s crop production depend on the pollination by animals; bees, moths, flies, wasps, beetles, and butterflies make up most pollinating species [5]. Finally, with a massive diversity and many individuals, insects constitute the main part of the atmospheric fauna. A new study on insect behavior could provide a better understanding of the atmospheric biomass.

Methods like mark-recapture and insect-trapping have been commonly employed to study insects. These methods provide a straightforward approach to collect entomological information with the limitations being labor-intensive, poor temporal resolution, and difficult to conduct without disturbing insect behavior [6,7]. Insect radar is a technique that emerged after WW II and developed rapidly with the advance of electronic technologies. Such technologies have shown that they can reveal the migration behavior of insects at high altitudes on a routine basis [8,9]. Recent work showed a promising approach in interpreting the heading direction and ascent and descent rates of high-flying insects [10,11]. When it comes to ground-dwelling insects in the field, strong reflection from terrain turns out to be a strong interference. Harmonic radar techniques, proposed in 1967 and first demonstrated in 1986, have been dominant in the study of insect flying movements at low altitudes [12,13]. The limitation of harmonic radar, though, is that it is only applicable to large insect species, e.g., honeybees, grasshoppers, butterflies, etc. Other techniques, like digital image processing, are also applied. However, to take sharp images, they mostly work at short distances. Also, in many cases, they work as a complementary method to the conventional insect trapping schemes [6,14,15]. There will be a long way to go before these apply to real remote sensing fieldwork.

The employment of laser radar (lidar) in insect research can be traced back to the early 2000s when insects were trained as biosensors to help locate explosives and land mines [16,17,18]. Soon, the focus of research shifted to insects themselves. First efforts were feasibility studies. After exploring the possibility of detecting fluorescence from birds and dye powder on insects, the proper spectrum region for detection and the possible set-up for remote sensing measurement, researchers at Lund University proposed a new type of laser radar by using the century-old Scheimpflug principle, known as Scheimpflug lidar or S-Lidar [19–23]. The technique, using a CW laser and a linear array detector attached to a receiving telescope in a Scheimpflug arrangement, turned out to be a particularly promising approach. With this set-up, considerable general entomological information can be collected [24,25,26]. Recently, work has been directed toward more specific biomass information, e.g., 3-D distribution of insects and more importantly, insect identification [27]. It has been shown that a combination of spectral reflectance and temporal wing-beat information provides a robust means to identify insects [28–30]. However, unlike molecules which are commonly detected by differential absorption lidar, optical properties of insects, e.g., optical cross-section, are highly anisotropic [31]. Without a proper understanding of the direction of flight, we may not be able to achieve high accuracy in insect identification. The possibility of using harmonic frequencies to tell heading direction has been discussed [32], however, as research has demonstrated, harmonics may potentially reveal the more detailed condition of insects, e.g., load-carrying (such as blood in mosquitoes) [33]. It would be reasonable to get heading direction of flying insects in the field of view (FOV) of the whole system with direct measurement other than interpreting from harmonic frequencies. Another benefit of getting a flight velocity of insects is that it could help to understand insect population dynamics and develop forecasting systems to the alarm of pest or disease vector invasion [7]. We present our progress on 1-D insect flight velocity measurement with a CW near-IR Scheimpflug lidar system. Due to the uncertainty caused by practical issues like varied refractive index along our detection path under unevenly cooled or heated atmospheric condition, an in-door calibration would be less helpful in our case than the common particle image velocimetry techniques. In this paper, we will mostly rely on the self-consistency of the phenomenon we observed throughout the field experiment for validation of our work.

2. Scheimpflug lidar for velocity measurement

The idea of using Scheimpflug lidar for insect detection can be traced back to 2013 [23]. With an infinite focal depth and high sensitivity, this technique offers an excellent solution for insect remote sensing at low altitude. Figure 1 shows a scheme of distance measurement by employing the Scheimpflug principle. The principle states that the entire object plane can be imaged in focus if it intersects with the LP and IP (See definitions of abbreviations in figure caption.) at the same point, the Scheimpflug intersection in Fig. 1. When the system is in the Scheimpflug condition, objects, i.e., insects, in the entire OP will be imaged in focus with a large aperture thus ensuring a high photon collection efficiency of the backscattered light from the object. In a real set-up, the laser beam is sent out along OP, such that the scattered light from any insects passing through the FOV will be collected by the detector efficiently. The distance between flying insects in FOV and the Scheimpflug lidar system (${Z_x}$) can be calculated from the pixel number ${P_x}$ of the image formed on the linear CMOS placed on the IP once we know the value of ${u_0}$ or $u_0^{\prime}$ using the following equations, based upon geometrical consideration in Fig. 1, with f as the focal length of the receiver as shown in Fig. 2 and fixing the camera angle θ and the distance from the lens to the CMOS center $u_0^{\prime}$:

 

Fig. 1. Scheme of the Scheimpflug principle and the estimation of the flight speed perpendicular to the optical axis of the receiver telescope. $\varphi$: Angle between the plane of focus and optical axis of the lens; $\mathrm{\gamma}$: Swing angle for pixel ${\textrm{P}_\textrm{x}}$; L: Distance between Scheimpflug intersect and the optical axis of the receiver; $\textrm{u}_0^{\prime}$: Distance between the lens plane and the intersect of linear CMOS and the optical axis; ${\textrm{u}_0}$: Corresponding object distance to $\textrm{u}_0^{\prime}$; $\textrm{u}_\textrm{x}^{\prime}$: Distance from pixel ${\textrm{P}_\textrm{x}}$ to the lens plane; ${\textrm{u}_\textrm{x}}$: Corresponding object distance to $\textrm{u}_\textrm{x}^{\prime}$; The abbreviation LP stands for lens plane, IP is the image plane and OP is the object plane. ${\textrm{P}_\textrm{x}}$ is the pixel number, an integer between 1 and 2048. ${\textrm{P}_{1024}}{\; }$indicates the midpoint of the detector. FOV perpendicular to OP is the shaded area in the figure while the orange solid lines represent the boundary of the laser beam.

Download Full Size | PDF

 

Fig. 2. Set-up of the Scheimpflug Lidar system at Texas A&M. One 808 nm diode laser is employed for active monitoring. The transmitter and receiver unit are tilted to a small angle according to the Scheimpflug principle. The linear CMOS is mounted at 34° to the optical axis of the receiver.

Download Full Size | PDF

We should note that the scheme shown in Fig. 1 is exaggerated and angle $\mathrm{\gamma}$ in our set-up is less than $0.83^\circ $ at all distances. For simplicity, the divisor $\textrm{cos}\; {\varphi }$ can be treated as a constant equal to 1, consequently, we have ${Z_x} \approx {u_x}$. With this approximation, we can treat the optical axes of the transmitter and the receiver as parallel. This will help calculate the velocity component perpendicular to the probing path. Without interference from an artificial light source, insects’ flight direction can be regarded as randomly distributed within the whole 360 degrees. Therefore the probability of our assumption being wrong is less than 0.5%. We can estimate the failure rate from the portion of extraordinarily low velocities we get through this method, i.e., velocities of less than 1.6 cm/s in our case.

Considering the geometric property of the Scheimpflug lidar system, the dimension of the field of view for each pixel (fov) in the direction perpendicular to the probing path, determined by pixel pitch (14 $\mathrm{\mu}\textrm{m}$), varies from hundreds of microns to millimeters as distance increases, far smaller than the dimension of FOV that is determined by sensor length (28.7 mm) and beamwidth (on the order of centimeters). Namely, whenever an insect passes through the FOV, its trajectory has a considerable projection perpendicular to the probing direction and covers several fovs. If we look into the pixels whose fov are covered by a single insect transit, we should be capable of describing the average velocity component of the insect normal to the probing path as it passes through the FOV. The velocity component can be described as below:

3. Experimental set-up

The set-up of the new TAMU Scheimpflug Lidar system is shown in Fig. 2. In a field monitoring project on flying insects, one 808 nm diode laser from CNI (MLD-808-5000) working with output power at 2.6 W was employed. A piece of micro cylindrical lens was mounted on the laser chip to better collimate the fast axis. To remove the constraint on the highest detectable wingbeat frequency, as mentioned in [26], we sacrificed the de-polarized channel and obtain only the co-polarized one. To optimize the signal intensity, a λ/2 wave plate was placed in front of the diode laser for fine-tuning. The linearly polarized beam was collimated and sent out through the transmitter, a Galilean refractor (SKY-WATCHER STARTRAVEL 120 AZ3) with a focal length at 600 mm and a diameter at 120 mm. Scattered light of the transmitted laser beam by aerosol or aerial fauna was collected by the receiver, a Newtonian telescope (Sky-Watcher Quattro 300P) with a focal length of 1200 mm and primary aperture of 305 mm. The transmitter and receiver were tilted by a small angle according to the Scheimpflug configuration. The linear CMOS camera (Model S11639-01 from HAMAMATSU) was mounted at $34^\circ $ to the optical axis to the receiver. An optical, long-pass filter (FILTER LONG 50 DIA 780NM) and an optical band-pass filter (FILTER BP 808NM x 10NM OD4 50MM) were employed to suppress background radiation. A linear polarizer (Ø2” Linear Polarizer with N-BK7 Protective Windows B Coating) was used to selectively filter the polarized radiation. The system worked at a sampling rate of 4000 Hz on Sept. 05, 2019. The rate was reduced to 3600 Hz for higher robustness on Sept. 06, 07 and 08, 2019, with a time sequence of background (laser off) to co-polarization (laser on) and back to background (laser off) again. The background intensity was subtracted from the co-polarization channel by linear averaging of two adjacent background intensities. The trigger signal was generated by the linear camera and amplified by a compact laser driver unit. A 2-D camera together with a refractive telescope (Astro-Tech 80 mm EDT) was used to present the IR image of the detecting area in real time for safety concern and convenience of alignment. In the past, three telescopes were mounted in either vertical direction or an arbitrary inclined angle [24,34]. As has been shown in [35], in certain weather conditions, the vertical velocity component of insects is rather limited. A vertical or close-to vertical set-up does not generally offer much information on insects’ flight. In related work, attempts have been made to estimate raindrop velocity with the vertical set-up, which may further help to estimate raindrop size distribution along the probing path [36]. In our experiment, three telescopes were designed and mounted in a horizontal plane, as shown in Fig. 3(c). In this way, we could measure insect movements in the horizontal plane perpendicular to the detection path.

 

Fig. 3. Overview of the experimental site on the Salter Research Farm (N31.03, E-96.77), Robertson County, Texas. The probing path and the location of the weather station are illustrated. The location of the Salter Research Farm relative to College Station is presented in (a). The image of the laser beam on the termination is shown in (b) and the photograph of the Scheimpflug system, installed under a canopy is shown in (c). The whole probing path was 1.6 m above the ground and ranged from around 56 m away from the lidar system until the end of the probing path, 500 m away.

Download Full Size | PDF

Our field experiment was conducted at the Salter Research Farm during the nights of Sept. 05 to Sept. 08, 2019. Common plants on the farm include Coastal bermuda (Cynodon dactylon), Broomweed (Gutierrezia sarothrae), Bitter sneezeweed (Helenium amarum), Wooly croton (Croton capitatus) and Silverleaf nightshade (Solanum elaeagnifolium). Although night-flying insects at the study site were not obtained, certain species like Mosquitoes (Aedes sp., Culex sp.), Gnats (Hippelates sp.), and Midges (Chironomus sp.) are known to be common to the area. The system was mounted in the deeper part of a canopy to avoid direct and scattered solar radiation at dusk and dawn. Three side walls of the canopy were closed throughout our field experiment to shield our system from the turbulence of wind. Our detection path was chosen to pass by a small pond in the hope of encountering more insects, thus ensuring a good amount of useful data. A weather station (Ambient Weather WS-2902A) was mounted around 20 meters away from our system by the side of the probing volume to record weather conditions like temperature, humidity, and wind speed.

During the experiment, the sun rose at around 7:08 am and set at around 7:43 pm. Wind speed was less than 4 m/s and in most cases less than 2 m/s throughout the measurement. The local outdoor humidity varied from less than 40% before sunset to close to 100% before sunrise. The fast change in humidity was a consequence of rapid temperature variation at dusk and dawn hours. Unevenly cooled or heated atmosphere would lead to air convection and the varied refractive index along our detection path that we believe caused variability in some of our observations. One apparent evidence was the fluctuating image in the surveillance camera soon after sunrise. Many insects are largely reliant on atmospheric currents for movement. Therefore, they may gain a considerably large velocity component in the vertical direction in our case [37].

4. Experimental results and discussion

To focus on those signals with a clear modulation, we started the data analysis by getting the frequency information first. A range-time map of five-second raw data collected on the night of Sept. 7, 2019 is shown in Fig. 4(a). To mitigate the strong reflection signal from the termination, the map is shown on a log scale so that the weak insect signals can be distinguished from the termination signal and background. In the inset, the insect passing through the field of view led to an oscillatory signal with a total duration of about 35 ms. The distance between the observed insect and our Scheimpflug lidar system was calculated from the median pixel number covered by the insect signal. For the signal in the inset, the median pixel number is 784, representing a distance at 86 m. The intensity of the insect signal is shown in the time domain in Fig. 4(b) and the corresponding power spectrum is evaluated by using the fast Fourier transform (FFT). It shows a 362 Hz wingbeat frequency and its harmonics up to the second order. The similar signal processing to calculate wingbeat frequency has been employed in [24,26,28].

 

Fig. 4. A Range-Time map of five-second raw data is shown in (a) and the inset shows the intensity information and modulation pattern of an insect-like signal. The modulation intensity of the insect signal is shown in the time domain in (b), the corresponding power spectrum (c) is evaluated by using the fast Fourier transform (FFT) showing a 362 Hz wingbeat frequency and its second harmonic.

Download Full Size | PDF

The location of the insect at each moment in the FOV was represented by the pixel with the highest intensity, shown as white dots in Fig. 5. The trajectory of the insect passing through the FOV was described by the linear fitting. The deviation of the fitting is described with an R-square of 0.9895. The result of the linear fitting is shown as the red solid line with a slope (k in Eq. (7)) of 0.6554 (fov/ms) with 95% confidence bounds (0.6381, 0.6727) indicating a flight speed component perpendicular to the optical axis of the receiver telescope at 0.363 m/s (with 95% confidence bounds 0.353 m/s to 0.372 m/s). The pixel number of the white dots increases as the insect passes through the FOV (k is larger than zero), which indicates insect flew toward the right. In the discussion below, only the signals with a clear frequency component (screening condition 1) and with an R-square larger than 0.9 (screening condition 2) were included. In the early inspection of entomological lidar, we have pointed out that the observations of insects in the field of view are rare events [22]. The chance of more than one insect in the FOV is low. Also, the behavior of insect is not coherent, such that the chance of two insects entered the FOV and overlap, with similar wingbeat frequency and same direction would be extremely low. With the two screening conditions, we can probably avoid the interference of other insects in the path. Meanwhile, insect signals whose trajectory can not be well described by a simple linear fitting have been screened out.

 

Fig. 5. Insect signal in the inset in Fig. 4 with trajectory linear fitting.

Download Full Size | PDF

The method we used to select the insect-like signals is like what has been employed previously [26]. This time, with a modified screening condition that has less bias on different kinds of signals, we achieved an overall consistency at 92% with 200 randomly selected signals as we checked manually later. The amount of recorded insect-like signals and the wind speed data throughout the whole measurement are shown in Fig. 6.

 

Fig. 6. The amount of recorded insect-like signals and the wind speed data throughout the whole measurement during Sept. 05, 06, 07, and 08 nights are presented.

Download Full Size | PDF

Two major peaks of insect activity can be observed at dusk and dawn hours, which agrees with findings in previous work [24]. The insect activity, as a general trend, increased as the wind decreased. This finding shows that insects on the farm were highly sensitive to the wind conditions and the flight velocity of insects was expected to be comparable to wind speed, otherwise the insect activity would be unaffected by the wind conditions. Since few insect signals were observed when the wind speed was higher than 2 m/s, we believe that 2 m/s would be a good reference to be set as a threshold for the upper boundary of the flight speed for most of the insects on the farm. Considering that most of the insects’ signals were found in the range of around 100 m, we achieved the reference value of extraordinarily low velocities of 1.6 cm/s, as mentioned in Section 2.

As shown in Fig. 7, a general trend of the wingbeat frequency distribution can be observed to increase in both dusk and dawn hours. On the night of Sept. 05, most of the recorded insect signals showed a wingbeat frequency of less than 200 Hz. Gradually, a cluster of signals with a wingbeat frequency near 400 Hz emerged at dusk hours. The change in the morning “rush hour” was more obvious as the cluster of the wingbeat frequencies shifted from ∼200 Hz to ∼500 Hz. Such a shift in wingbeat frequency cannot be explained by different load-carrying conditions or interference by wind and probably indicates the presence of new insect species. As illustrated in Fig. 6, it was mostly windless during dusk hours on the nights of Sept. 05, 06, and 07. We conclude that a change in the species of the insects active in the region of measurement can be expected. This phenomenon can be well-explained from the directional flight information shown below.

 

Fig. 7. General biomass information from data collected on the nights of Sept. 05, 06, 07 and 08 show the wingbeat frequency distribution at different times with time interval at 10 min and frequency interval at 50 Hz.

Download Full Size | PDF

 

Fig. 8. Distribution of flight speed component normal to the optical axis of the receiver telescope as estimated from data recorded during each night. The positive value indicates the insect flew toward the right, while the negative value shows the flight direction toward the left.

Download Full Size | PDF

We can see in Fig. 8 that most of the transient events indicate a speed component of less than 2 m/s, which is a consistent trend as presented in Fig. 6, showing that insect activity is sensitive to the wind. During the whole study, the total insect flight toward the right outnumbered that toward the left, indicating a net directional motion of the insects. This phenomenon gets more pronounced on Sept. 07, providing a good self-consistency of the frequency shift shown in Fig. 7. To further remove the uncertainty caused by wind conditions, we plotted the flight velocity information from 20:15 to 20:55 on the first three nights of the experiment when there was little to no wind. The same directional motion trend can still be observed. It indicates that this behavior, the net directional motion, may be an intrinsic and natural behavior of insects. As an examination of our assumption on the failure rate in section 2, we checked the number of insect signals with an extraordinarily low velocity on each night. None out of 6618 signals (0%) had a velocity component less than 1.6 cm/s on the night of Sept. 05, and only 1 out of 6811 signals (0.015%) on Sept. 06 and zero out of 14898 signals (0%) on Sept. 07 had such component. This perfectly matches the percentage we proposed (less than 0.5%).

In conclusion, we have developed a novel scheme to measure the insect velocity component in a certain direction. During the measurement, we observed a gradual arrival of individual insects with higher wingbeat frequencies, which, we believe, indicates the presence of new insect species. The 1-D study of flight velocity reveals a net directional movement of insects, providing supportive evidence of different species and behavior. The velocity distribution confirms our assumption of the overall velocity of insects and in turn demonstrates the validity of our method. A more accurate 2-D velocity or even 3-D velocity measurement requires a high range resolution along detection paths compared to the beam width. Such resolution may be achieved by time-of-flight measurements utilizing ultrashort pulses. The uneven intensity distribution in the laser beam results in extra intensity modulation of signals and may lead to difficulties in the calibration of optical cross-section; this is an issue that needs to be investigated in the future. Analysis of air turbulence would also be a problem to be addressed.