1. INTRODUCTION

Light detection and ranging (LiDAR) systems are becoming crucial components for autonomous robots, vehicles, drones, and precision distance measurement tools in mining and surveying fields [1–3]. Among the various LiDAR methods, the time of flight (ToF) method is attracting much attention due to its simplicity, reliability, and cost-effectiveness [3–5]. The combination of a flash laser source and a ToF camera enables one to realize non-mechanical 3D solid-state LiDAR sensing [4–11]. Flash laser sources consist of vertical cavity surface emitting laser (VCSEL) arrays combined with external optical elements, which impose limitations on the minimum size of the system. In addition, this type of LiDAR system cannot accurately measure the distance of poorly reflective objects, such as black metallic vehicles, in the field of view (FoV) for the following reason: Under normal operation, the intensity of light reflected from poorly reflective objects is too weak to be detected by the camera; thus, to generate a detectable signal from which the distance of such objects can be measured, the irradiation power of the flash laser source should be increased. However, doing so causes the signals from highly reflective objects in the same FoV to saturate and, therefore, encumber accurate measurements.

Here, we demonstrate a new type of LiDAR that addresses the above issue by combining the flash laser source with a beam-scanning laser source for spot illumination. For both flash and beam-scanning laser sources, we employ external-optical-elements-free on-chip dually modulated photonic crystal surface-emitting lasers (DM-PCSELs) [12–16]. In DM-PCSELs, a singularity point of the photonic crystal known as the ${{M}}$ point, which lies outside of the light-cone, is used for laser oscillation, and the light is diffracted into the light-cone in desired directions by introducing a modulation to the positions and sizes of the photonic crystal lattice points simultaneously [15]. For the beam-scanning laser source, an array of DM-PCSELs is used, which is capable of scanning the beam over a desired range of emission angles. For on-chip flash illumination, we broaden the beam by suitably designing the modulation of DM-PCSELs [16]. To implement this in the LiDAR system, we also develop a driving circuit for each of the beam-scanning and flash-illumination DM-PCSELs. Electronics, including the driving circuit, a ToF camera module and control units, are miniaturized to the extent that the LiDAR system fits in the palm of the hand. The developed LiDAR system enables us to measure poorly reflective objects by selectively illuminating such objects with sufficient power by the beam-scanning light source. Furthermore, we develop and implement a program to have the LiDAR system automatically recognize the poorly reflective objects in the FoV and measure their distances by selective illumination. The outline of our paper is as follows. In Section 2, we briefly present our newly proposed LiDAR system. In Section 3, we discuss the developed LiDAR system, including its DM-PCSEL-based flash and beam-scanning laser sources. In Section 4, we describe an evaluation of the LiDAR system. A summary is provided in Section 5.

2. PROPOSAL OF THE NEW LIDAR SYSTEM

Conventional 3D ToF flash LiDAR systems consist of a flash laser source, a ToF camera with a lens system, and control units. As mentioned above, such flash LiDAR systems cannot accurately measure the distance of poorly reflective objects, such as black metallic vehicles, in the FoV since the intensity of light reflected from these objects becomes inherently weak due to the broad-area flash illumination [Fig. 1(a)]. To rectify this issue, we utilize a solid-state two-dimensional beam-scanning DM-PCSEL source to selectively illuminate the poorly reflective objects in the FoV with sufficient power and, thereby, enable the measurement of their distance as illustrated in Fig. 1(b). Note that, to recognize the existence of poorly reflective objects in the FoV, we can apply pattern recognition to a two-dimensional image (with no distance information) that is simultaneously measured by the ToF camera. Introducing such pattern recognition in the LiDAR system would enable us to realize automated distance measurements of the poorly reflective objects. This system has another advantage: if the recognized object is not living matter, illumination can be done at a high peak power since eye-safety requirements in this scenario can be neglected; furthermore, even in the case of living matter, a high-peak-power beam can be illuminated by recognizing and preventing beam exposure on the eyes.

 

Fig. 1. (a) Conventional flash LiDAR system consisting of a flash laser source and a ToF camera. This system struggles to measure poorly reflective objects in the field of view due the low reflected light intensity from these objects. (b) Proposed new LiDAR system combining DM-PCSEL-based flash and beam-scanning laser sources. A two-dimensional beam-scanning DM-PCSEL array is used to selectively illuminate and measure the distances of poorly reflective objects.

Download Full Size | PDF

 

Fig. 2. (a) Photograph of the developed 3D ToF-LiDAR system implementing DM-PCSEL-based flash and beam-scanning laser sources. A business card is placed in front of the system for perspective. (b) Driving circuits for the DM-PCSEL-based flash and beam-scanning laser sources. Schematic diagrams illustrating the circuit for each laser source are also shown. (c) Photo of a ${{320}} \times {{240}}$ ToF camera module and a FPGA control module incorporated into the compact LiDAR system. (d) Photograph of the LiDAR system with its assembled ToF camera module, FPGA module, and developed driving circuit with laser sources.

Download Full Size | PDF

3. DEVELOPED LIDAR SYSTEM

A photograph of the developed LiDAR system, consisting of DM-PCSEL-based flash and beam-scanning laser sources, is shown in Fig. 2(a). The two DM-PCSEL sources are visible in the upper half of the unit while the lens to collect reflected light is visible in the bottom half. For perspective, a business card is placed in front of the system, indicating that the footprint of the system is smaller than that of a business card. The volume of the LiDAR system, including its outer cover, is ${\sim}{{360}}\;{\rm{cm}}^3$, which allows the system to fit in the palm of the hand. To operate the DM-PCSEL sources in the LiDAR system, a suitable laser driving circuit compatible with the ToF camera is essential. A photograph of the driving circuit board separated from the LiDAR system [Fig. 2(a)] is shown in Fig. 2(b). Schematic diagrams illustrating the circuits used to drive the flash and beam-scanning laser sources are also shown in the same figure. To operate the flash laser source, two driving integrated circuits (ICs) are connected to the cathode of the flash-type DM-PCSEL, whose anode is connected to a power supply; electrode configurations of the DM-PCSELs will be explained later [Fig. 3(a)]. The driving IC has six channels, each of which is able to inject a maximum current of 3 A; thus, implementing two driver ICs enables us to inject a maximum current of 36 A to drive the flash-type DM-PCSEL, which has a threshold current of ${\sim}{1.2}\;{\rm{A}}$. The injection current is adjustable by controlling the gate voltages of the driving ICs via a digital-to-analog converter (DAC). A control signal is used to manipulate the pulse width, repetition rate, and pulse timing of the laser source. The pulse timing is synchronized with the ToF camera, which is essential to determine the distance of objects. Next, the driving circuit to operate the beam-scanning laser source, consisting of a DM-PCSEL array, is contained on the right side of the circuit board shown in Fig. 2(b). This array has a ${{10}} \times {{10}}$ matrix of cathode (${{n}}$) and anode (${{p}}$) electrodes as will be explained later [Fig. 4(a)]. Each cathode line of the DM-PCSEL array is connected to two channels of a driving IC, and each anode line is connected to a semiconductor switch. To selectively inject current to a specific array element, we control the gate voltage of the relevant driver-IC channel and turn on the relevant switch according to the control signals. Since a maximum current of 3 A can be injected by a single channel as mentioned above, each element, which has a threshold current of ${\sim}{{100}}\;{\rm{mA}}$, can be operated at a maximum injection current of 6 A. The injection current is controlled by the DAC connected to the driver as explained above. The circuits are designed to drive the laser with a rise–fall time of ${\sim}{{2}}\;{\rm{ns}}$.

 

Fig. 3. (a) Microscope image of the backside electrodes of the DM-PCSEL flash laser source. (b) Far-field patterns of ${{30}}^\circ \times {{30}}^\circ$-wide illumination. (c) Light output characteristic of the device.

Download Full Size | PDF

For LiDAR, we use a CMOS-type ToF camera [7,8]. In each of the camera’s ${{320}} \times {{240}}$ pixels, capacitors are attached to the active region of a Si photosensor via gates. Carriers generated in the active region are transferred to the capacitors by opening the relevant intervening gates. The turn on/off times of the gates are synchronized with the laser pulse rise/fall times for determining the distance of objects by collecting the carriers generated by reflected light. Note that the maximum measurable distance is determined by the pulse width. Details of the operating principle of the ToF camera and its specifications are discussed in Supplement 1, Section A. A photograph of the camera module and the field programmable gate array (FPGA) module, which is used for controlling the driving circuits boards and signal processing, is shown in Fig. 2(c). In addition, a photograph of the compact LiDAR system with its assembled camera module, FPGA module, and driving circuit boards with the laser sources is shown in Fig. 2(d).

Next, we explain the on-chip DM-PCSELs developed for the LiDAR system. For on-chip flash illumination, we broaden the beam emitted from a single resonant area (of 100 µm diameter) by suitably designing the modulation of DM-PCSELs and combining several such DM-PCSELs to obtain illumination over a FoV of desired width [16]. The air hole positions and sizes of the photonic crystals are modulated with amplitudes of $0.08a$ and $0.03a^{2}$, respectively (where $a$ is the lattice constant). These parameters were chosen to achieve sufficient diffraction of laser light oscillating in band edge modes into the desired directions [16]. Here, we broaden the divergence angle of a single beam to 8°, and we combine ${{10}} \times {{10}}$ of such beams to illuminate a FoV of ${{30}}^\circ \times {{30}}^\circ$, while properly overlapping the beams in order to create spatially uniform illumination. The lasers are fabricated on a semi-insulating (SI)-GaAs substrate, with both $n$-electrodes and $p$-electrodes fabricated on its backside. Details of the design and fabrication method of flash-type DM-PCSELs are discussed in Supplement 1, Section B.

The ${{10}} \times {{10}}$ electrode matrix on the backside of the DM-PCSEL-based flash laser source, utilized to inject current uniformly throughout its active region, is shown in Fig. 3(a). Along each line of this matrix, pairs of adjacent elements are connected in parallel, as shown in the magnified inset, and five of these pairs are connected in series. Ten of such lines are then connected in parallel. (For further details see Supplement 1, Section B.) In Fig. 3(a), the anode and cathode of the lasers appear on the left and right ends of the matrix, respectively. The far-field pattern emitted from the top of the device (from a SI-GaAs surface with an anti-reflection coating) is shown in Fig. 3(b). On-chip wide illumination over a ${{30}}^\circ \times {{30}}^\circ$ FoV is confirmed. We note that only one of two identical far-field patterns emitted from the DM-PCSEL is shown in the figure. Next, the light-output characteristic of the flash laser source is shown in Fig. 3(c). The threshold current and slope efficiency are found to be ${\sim}{1.2}\;{\rm{A}}$ and 1 W/A, respectively. The emission wavelength of the device is measured to be ${\sim}{{940}}\;{\rm{nm}}$.

For the beam-scanning laser source, we use a DM-PCSEL array. We designed an array of DM-PCSEL elements in a ${{10}} \times {{10}}$ matrix configuration, which can scan the beam two-dimensionally over an angular range of ${{30}}^\circ \times {{30}}^\circ$. As with the flash laser source, the air hole positions and sizes of the photonic crystals are modulated with amplitudes of ${0.08}a$ and ${0.03}{a^2}$, respectively, and the lasers are fabricated on a SI-GaAs substrate, with both $n$-electrode and $p$-electrodes fabricated on its backside. We connect the cathode ($n$)-electrode and anode ($p$)-electrodes in each line as shown in Fig. 4(a). We are able to operate specific elements of the DM-PCSEL array at the intersections of these line electrodes by selectively injecting current into the lines. Each element emits a laser beam with a divergence angle of 1° in a direction differing from the others by 3°, such that all elements cover a FoV of ${{30}}^\circ \times {{30}}^\circ$. Details of the design, fabrication method, and beam-divergence evaluations are discussed in Supplement 1, Section C. A few spot illumination patterns are shown in Fig. 4(b). Visualization 1 shows a video of beam-scanning operation. The light-output characteristic of a single element of the beam-scanning laser source is shown in Fig. 4(c). The threshold current and slope efficiency are found to be 100 mA and 0.4 W/A, respectively. We can increase the slope efficiency by introducing backreflectors to the laser. We will discuss such improvements elsewhere in the near future. The emission wavelength of the device is measured to be ${\sim}{{940}}\;{\rm{nm}}$.

 

Fig. 4. (a) Microscope image of the backside electrodes of the DM-PCSEL-based beam-scanning laser source array. (b) Far-field pattern of spot illumination. Visualization 1 shows a video of beam-scanning operation. (c) Light-output characteristic of a single element of the device.

Download Full Size | PDF

4. EVALUATION OF THE NEW LIDAR SYSTEM

To experimentally evaluate our developed LiDAR system, we placed a highly reflective diffuse white object and two poorly reflective diffuse black objects in the FoV of the system as shown by the camera image of Fig. 5(a). Reflectivities of the white object and the black objects were measured to be ${\sim}{{98}}\%$ and ${\sim}{{2}}\%$, respectively (details of these measurements are discussed in Supplement 1, Section D). The objects were placed at distances of ${\sim}{1.5}\;{\rm{m}}$ from the LiDAR system. In the ranging measurements, the lasers were driven with a pulse width of 24 ns and a repetition period of 2.7 µs. We operated the flash and beam-scanning laser sources with peak powers of ${\sim}{1.6}\;{\rm{W}}$ and ${\sim}{{100}}\;{\rm{mW}}$, respectively; the corresponding power per square degree from the flash and beam-scanning laser sources was estimated to be ${\sim}{1.8}\;{\rm{mW/degre}}{{\rm{e}}^2}$ and ${\sim}{{127}}\;{\rm{mW/degre}}{{\rm{e}}^2}$, respectively. Therefore, the illumination power density of the beam-scanning laser source was ${\sim}{{70}}$ times larger than that of the flash laser source. This reflects the capability of the beam-scanning laser source for measuring the poorly reflective black objects, which have 49 times lower reflectivity (${\sim}{{2}}\%$) than the highly reflective white object (${\sim}{{98}}\%$). The frame rate of the imaging system was set to 10 fps, and the number of signal integrations in a single frame was set to ${\sim}{1.85} \times {{1}}{{{0}}^4}$. First, we measured the distance of the objects with only flash illumination. The results of this measurement are shown in Fig. 5(b). Evidently, the distance of the white object was successfully measured, whereas the distances of black objects were not measured. However, we can recognize the presence of the black objects in the camera image, which was taken by our ToF camera system, shown in Fig. 5(a), and then selectively illuminate the two black objects simultaneously by the DM-PCSEL array. Upon doing so, the distances of the black objects were clearly measured from their spot-sized illuminations by the DM-PCSEL, as shown in Fig. 5(c), and from these measurements we could infer the distances of the black objects in their entirety.

 

Fig. 5. (a) Camera image (taken by the ToF camera) of targets used to evaluate the LiDAR system, consisting of a highly reflective white object and two poorly reflective black objects. (b) Distance measurements obtained by the LiDAR system using only the flash illumination (ToF image). The distances of the black objects cannot be measured. (c) Distance measurements achieved by the LiDAR system with flash illumination and beam-scanning illumination by the DM-PCSEL array (ToF image).

Download Full Size | PDF

We also developed a program to recognize the black objects from the camera images by pattern recognition and then automatically selectively illuminate these objects, even during their movement. In this experiment, we placed a white object (at rest) in the FoV, then we moved a black object into the FoV by hand. As seen in the camera image (left image of Fig. 6, taken by the ToF camera), the position of the black object (in the two-dimensional plane) can be recognized by the difference in contrast between the object and the background. Upon the detection of this contrast, pattern recognition of the black object was carried out by using the FPGA in our LiDAR system. First, in one frame under only flash illumination, the position of the black object was recognized, and then, in the next frame, the beam-scanning laser source was turned on to selectively illuminate this black object to measure its distance in real time. In Fig. 6, we show some snapshots showing the fully automated distance measurements of the black object moving in the FoV. Visualization 2 shows a video of these 3D ToF measurements. Note that, as mentioned above, if the recognized object is living matter, beam-scanning at a high peak power can be done everywhere except on the eyes, as eye-safety requirements in these regions can be neglected.

 

Fig. 6. Automatic distance measurement of a black object by detecting its position by the camera image (only a few snapshots are shown). Visualization 2 shows a video of these measurements.

Download Full Size | PDF

In addition, we performed long-distance ranging by using the LiDAR system while including a poorly reflective object in the FoV. Here, we employed a new type of flash DM-PCSEL, which is capable of much more uniform flash illumination from a single element, owing to the introduction of a modulation method based on an inverse Fourier transform. Details are provided in Supplement 1, Section E. A long-distance measurement using our LiDAR system with this new flash DM-PCSEL at a flash illumination power of a few watts is shown in Fig. 7.

 

Fig. 7. (a) Camera image (not taken by the ToF camera) showing a long-distance measurement. (b) Distance image with both flash and spot illumination. The distance of the black object can be measured by spot illumination.

Download Full Size | PDF

5. SUMMARY

We have developed a new type of non-mechanical 3D-LiDAR system that incorporates flash and beam-scanning DM-PCSEL sources. With this system, we experimentally demonstrated the measurement of poorly reflective objects in the FoV by selectively illuminating these objects with sufficient power by the beam-scanning laser. In addition, we have developed a program for automatically recognizing, illuminating, and measuring the distances of such objects even during their movement. We have also performed long-distance ranging by using the LiDAR system while including a poorly reflective object in the FoV.