1. Introduction

During the last few decades, research in the field of robotics has advanced considerably, but there still exist very few sighted robots which are able to behave appropriately, regardless of changes in the illuminance (see Fig. 5 in [1], for instance), such as those which occur outdoors. One of the reasons for this lack is that it is difficult to design pixels that combine high sensitivity with a wide luminosity range.

 

Fig. 5 (a) Simulated example of the adaptive pixel’s response (blue line) to a step change in the light intensity (light to dark gray stripe). The red and magenta lines stand for the photodiode current and the average current, respectively. (b) Theoretical S-shaped curve based on equation (4), giving the peak values of the pixel’s response V_i to step changes in the photodiode current from I_ph0 to I_phi.

Download Full Size | PDF

A large variety of Wide-Dynamic-Range (WDR) CMOS image sensors has been proposed throughout the years [2], trying to widen the operating luminosity range as much as the visible spectrum while keeping sensitivity to small changes for every average luminosity in the operating range. Although the WDR image sensors capture images in a luminosity range of up to 7 decades, they provide different contrast sensitivities at different average luminosities. Vision applications such as event-based and bio-medical applications often require high and constant sensitivity in a large luminosity range, in order to detect small temporal and/or spatial changes in the intensity in several lighting conditions [3–6]. One possible solution to this problem can be found by looking at the auto-adaptive response of human and animal photoreceptors.

In their physiological studies on fish, Naka and Rushton established for the first time that a vertebrate retina obeys a process of adaptation whereby each photoreceptor’s response is normalized by a representative value of the average local luminosity [7], in line with the Michaelis-Menten equation [8]. In the light of these and subsequent findings [7, 9–12], many efforts have been made to mimic the Outer Plexiform Layer (OPL) circuitry in silicon retinas [3, 13–19], or to implement the model in software for image processing [20–23]. In the latter case, the normalization giving the adaptation is implemented numerically after digitalization which gives rise to noise amplification, especially in dark scenes. Consequently, considerable interest has focused during the last twenty years on developing a silicon retina giving an OPL-like response over the entire visible spectrum, also thanks to the latest advances in retinal implant systems [24, 25].

The first example of an auto-adaptive silicon retina was presented in [13], where a logarithmic photoreceptor was used to handle transient changes in light in a 1-decade range, while light adaptation within a 1-decade range was obtained by “subtracting” a local spatio-temporal average. This circuit improved the contrast resolution of equally illuminated areas in comparison with standard logarithmic photoreceptor retinas, but there was no improvement in the low signal-to-noise ratio inherent to the logarithmic amplification. In [14], a modified version of this chip was compared with the OPL response described in Necturus [26] (see Section 2), showing light adaptation in a 5-decade range but sensitivity to luminous changes within a range of only 0.5 decades.

To overcome these limitations, a more biologically inspired solution was subsequently developed in a study by [15], which consisted in modulating the synaptic strengths locally to control the sensitivity and including cone-to-cone gap junctions to attenuate the noise. Although the sensitivity was improved in this way from 0.5 to 2 decades, the adaptation to light was not satisfactory because of the circuit deviations resulting from the increasing inter-receptor coupling strength. A good compromise between contrast sensitivity and light adaptation was reached in [16], which gave light adaptation in a 6-decade range and sensitivity in a 1-decade range. However, the steady-state response of this pixel was found to increase with the light intensity (i.e., the photodiode current) and the transient response was not always monotonic when large lighting variations occurred (see Fig. 2.13 in [27]). The Delbrück adaptive pixel was also found in studies on optic flow measurements to have little practical use in situations where changes in the light greater than 1 decade are liable to occur (see Figs. 7(b),7(d),7(j) and 7(l) from [28]). In [29], the Gamma correction method presented in [21] for local tone-mapping purposes was improved by digitally normalizing the pixel output directly in VLSI in line with the Michaelis-Menten law: only a few preliminary results on light adaptation and contrast sensitivity were presented, however, in that study.

 

Fig. 7 (a) Pictures and (b) exploded view of the Lighting Box composed of a PCB with a red LED (λ ≈ 618nm) and an optical filter support. The direct control of the LED current makes the illuminance vary in a 3-decade range. Additional optical filters (neutral density filters) were used to drastically increase the mean luminosity range from 3 to 7 decades.

Download Full Size | PDF

Other solutions not involving the use of auto-adaptive elements have been suggested. In [3], a subretinal stimulator was endowed with light adaptation by shifting the dynamic input range via an externally generated signal. The results presented showed that photoreceptor adaptation was achieved in a 7-decade range. In [19], a silicon retina was provided with a wide dynamic operating range and a high contrast sensitivity by applying a spatial and temporal filtering process based on resistive networks. The results obtained showed that the pixels could deal with 4 decades of luminosity changes, but their contrast responses depended on the external voltage controlling a reset transistor.

As far as we know, no artificial retinas have ever been endowed up to now with pixels with the following features at once: (i) auto-adaptation to the mean local luminosity over a range as wide as the visible spectrum; (ii) constant sensitivity to luminous changes, i.e. contrasts, at any average luminosity in the operating range; (iii) reliable response even in the presence of sudden large changes in the luminosity (i.e., without causing circuit saturations or deviations).

In this paper, it is proposed to present: (i) a novel analog silicon retina featuring auto-adaptive pixels that obey the Michaelis-Menten law faithfully in a 7-decade range without causing any saturation of the VLSI transistors, while keeping an effective resolution of the integrated analog-to-digital conversion of about 9 bits; (ii) a method of characterizing silicon retinas, which can be used to accurately assess the pixels’ response to transient luminous changes within a ±3-decade range and to changes in the steady-state intensity within a 7-decade range. We have called this novel pixel the M²APix, which stands for the Michaelis-Menten Auto-adaptive Pixel.

The present artificial retina consists of a 2 × 2mm CMOS circuit comprising four lines of six auto-adaptive pixels, and a digital interface giving a fast serial read-out of up to 1MHz connecting the retina directly to an external microprocessor or microcontroller. The adaptation time constant of the M²APix can be changed by means of an external capacitor, providing additional flexibility to eventually meet the application’s requirements in term of preferred bandwidth.

The biological background to this study is presented in Section 2. The chip implementation is presented in Section 3, and a detailed description of our auto-adaptive pixel is provided in Section 4. The method of characterization used is presented in Section 5, and the results obtained using this method are presented and discussed in Section 6. A comparison between the M²APix and the Delbrück pixel present on the same silicon retina is proposed in Section 7. Some conclusions are reached in the last section.

2. Biological background

Light adaptation of the photoreceptors present in human and animal retinas has been extensively studied in a large number of species since the early 50’s, using both intra and extracellular methods [7, 9, 11, 30–34]. In all these studies, the relationship between light stimuli and photoreceptor responses has been documented, both in the dark and with background illumination, via an adaptation process described by the so-called Michaelis-Menten equation [8]:

The first micro-electrode recordings of rod and cone responses were obtained on saltwater fish (Gerridae) by Svaetichin in 1953 [35]. In his pioneering study, Svaetichin discovered the S-potentials, as they were subsequently called by Oikawa et al. in [36], which stands for “slow potentials”, referring to the slow adaptation process which occurs in the photoreceptor potentials when they are exposed to flash lights against a steady background.

However, the first mathematical description of the cone response given by equation (1) was provided by Naka and Rushton in the case of the freshwater fish (Cyprinidae) [7]. The equation (1) where n = 1 is therefore also known as the Naka-Rushton law. The same model was subsequently validated and applied to turtles’ cones by Baylor et al. [30] and to monkeys’ cones by Boynton and Whitten [31], who introduced the exponent n < 1 for the first time. Many studies were then carried out on vertebrates and invertebrates, all confirming the equation in (1) with various values of n and sometimes with different interpretations of the adaptation parameter σ (in the salamander [9], gecko [32], frog [33], locust, fly and dragonfly [34], for instance, and in the human fovea [11]).

Figure 1 shows the responses of dark- and light-adapted red cone photoreceptors recorded intracellularly in the retina of the turtle (Pseduemys Scripta Elegans) by Normann and Perlman [10]. As can be seen from this figure, the function V (I) defined in (1) gives rise in the Log(I) domain to curves with a fairly smooth “S” shape (continuous curves), where the slope of the “S” is given by the value of n (n = 1 in that case) and the lateral shift by the value of σ.

Based on the S-shaped curves shown in Fig. 1, two main features of the light-adaptation behavior can be described by the incident-light model [37]:

 

Fig. 1 S-shaped curves corresponding to dark- and light-adapted response curves recorded in a red cone of the turtle. The peak of either the incremental or decremental response measured from the dark-adapted potential recorded before the background onset (dashed line) is plotted as a function of the log of the test pulse intensity which elicited each response. The steady hyperpolarization produced by each background lighting condition is given by the intersection between the intensity-response curve and the small horizontal line. The continuous curves were drawn from a single template which describes the function $V=V_m\frac{I}{I+\sigma }$. Adapted from [10].

Download Full Size | PDF

3. Chip implementation

In this section, we present our 2-D photosensor array, featuring a silicon retina composed of 24 auto-adaptive pixels of two different kinds. A picture of the chip package, with a zoom on the retina, is presented in Fig. 2.

 

Fig. 2 (a) The silicon retina in its 9 × 9mm package; (b) Magnified view of the silicon retina composed of 12 Michaelis-Menten pixels presented in this study, and 12 additional Delbrück pixels; (c) Magnified view of 3 Michaelis-Menten pixels giving the photodiode’s dimensions and the inter-receptor distance.

Download Full Size | PDF

The retina consists of a 2 × 2mm CMOS circuit designed using the 350nm XFAB standard CMOS process. To facilitate the integration of the chip into custom-made Printed Circuit Boards (PCBs), the circuit was encapsulated in a standard 9 × 9mm (LCC24) package with 24 pins. Four rows of auto-adaptive pixels with a 254 μm diameter N-well/P-substrate photodiode were implemented in the PCB, as shown in Fig. 2(a). The photodiode is mostly sensitive to red light (λ ≈ 650nm) and its sensitivity S_ph is equal to $1.1\times {10}^{-8}\hspace{0.17em}\frac{{\mathit{Am}}^2}{W}$. The retina is completely self-biased and can be read out by means of digital serial read-out architecture implemented directly on the chip. The main functional blocks implemented on the chip can therefore be summarized as follows:

The photodiodes were aligned on two horizontally staggered rows so that the hexagons fit together like a puzzle, recalling the shape and the arrangement of insects’ hexagonal ommatidia (Fig. 2). This pattern of alignment of the photodiodes, which is particularly suitable for detecting luminous contrasts in the main direction, also makes it possible to sense light variations in any other direction.

The analog signals originating from each pixel were digitized on-chip so that they could be directly processed by an external microprocessor or microcontroller, saving the power consumption and the computational resources of the latter for further data processing. Since the pixel type is selected by a digital input, only one set of pixels can be converted at a time. The twelve M²APix outputs are low-pass filtered with a cut-off frequency of 300Hz before being digitally converted, giving a minimum sampling frequency of 600Hz in order to prevent the occurrence of aliasing. An integrated DC reference voltage was connected to the ADC as an additional pixel output for testing and calibration purposes. To improve the LSB voltage, the dynamic input range of the ADC can be reduced by using an external voltage source. A synchronous direct-connection protocol similar to that used in the artificial compound eye CurvACE [38] was adopted, as it provides a very simple, robust solution.

A functional diagram of the interface protocol is given in Fig. 3. For the serial communications, a similar internal state machine working with a maximum frequency of 1MHz to that adopted in the CurvACE sensor [38] was used to transfer the data to an external device.

 

Fig. 3 Block diagram of the retina’s serial synchronous read-out interface (see [39]).

Download Full Size | PDF

4. M²APix: Michaelis-Menten auto-adaptive pixel

In this section, we present our novel auto-adaptive pixel implementing the Michaelis-Menten model in analog VLSI, as described in Section 2. The theoretical basis of the analog circuit is first presented, and an example of the auto-adaptive pixel’s response is then given to illustrate the behavior of the M²APix and show how the pixel is related to S-shaped curves such as those presented in Fig. 1.

4.1. Circuit description

The block scheme depicted in Fig. 4(a) gives an overview of the Michaelis-Menten pixel implementation. All the blocks in the area delimited by the dashed lines belong to a single pixel.

 

Fig. 4 (a) Block diagram of the M²APix: Michaelis-Menten Auto-adaptive Pixel. The blocks in the dashed-line area, which are replicated 12 times, belong to a single pixel, whereas the two blocks outside the dashed-line area are common to all twelve pixels. (b) Hardware implementation in VLSI of an elementary auto-adaptive pixel (photodiode and current normalizer), the output signal of which is noted I_outi. A switch S can be used to select either I_0i as the mean current I_meani provided by the built-in averaging circuit, or an external current I_exti provided by an external circuit. (c) Hardware implementation in VLSI of the filtering and averaging circuit computing the mean current of the 12 mirrored photodiode currents (I′_phi) produced by the 12 normalizer circuits.

Download Full Size | PDF

These blocks are therefore replicated twelve times, whereas the two blocks outside the dashed-line area are common to all twelve pixels.

To implement the Michaelis-Menten function in (1), we adopted the current normalizer model presented in [40] (Chapter 6, pp. 148–150), with an arbitrary number of current inputs, and patented in [41] in the context of photosensing. In the present application, two current inputs are required: one for the photodiode current I_phi and one for the current I_0i corresponding to the average illuminance. The functional scheme of the current normalizer is presented in Fig. 4(b). A switch S can be used to select either I_0i as the mean current I_meani provided by the builtin averaging circuit or an external current I_exti provided by an external circuit. In what follows, we take I_0i = I_meani.

The scheme adopted here improves the functioning of the current normalizer by adjusting the V_ref voltage to a different value from that of V_dd (3.3V). The auto-adaptive pixels we designed work efficiently in a wide range of luminosities corresponding to a photodiode current ranging from about 20 pA to 20 μA. The V_ref voltage optimizes the functioning of the system at low currents by preventing the current source M_b transistor from saturating.

Operating principle of the normalizer

If the transistors M₁,..., M₄ have the same dimensions and are working in their sub-threshold region, the current output I_outi can be expressed as follows:

To obtain the same auto-adaptation to light as that which occurs in animals’ eyes, I_mean has to be a representative value of the background luminosity perceived by the artificial retina (see Section 1 and 2). Accordingly, the computation of I_mean must reflect only the low-frequency changes in the light perceived by all the photoreceptors in the retina. The current I_mean is therefore the average value of copies of all 12 photodiode currents (I′_phi) filtered with a first order low-pass filter. As shown in Fig. 4(c), the low-pass filter is provided by a gm-C structure using an operational transconductance amplifier (OTA) G_m and an external capacitor C_m. In particular, if the external capacity is set at 100nF, as done in our tests, the cut-off frequency will be 150mHz.

As shown in Fig. 4(a), the output current of the normalizer is converted into a voltage via a high gain transimpedance amplifier (TIA). This high-factor current-to-voltage conversion R_f is obtained using a low-transconductance OTA in the feedback loop of an operational amplifier stage. The output voltage V_out can therefore be expressed as follows:

To prevent the occurrence of aliasing due to the sampling frequency of the digital conversion, a first-order low-pass filter with a gm-C structure is added to each pixel. A cut-off frequency of 300Hz is achieved by means of a low-transconductance OTA and an internal capacitance.

Lastly, a voltage follower helps the pixel to drive the input sampling capacitance of the ADC.

The output pixel voltage can then be written as follows:

4.2. M²APix response

The fact that the term I_mean in equation (4) corresponds to the average luminosity constitutes a key point in the adaptive behavior of the pixel.

Let us assume that in the absence of any optical lenses placed on the retina, all the pixels are exposed to the same light intensity. If no changes or only very slow changes in the luminosity occur, all the photodiode currents and their average will be identical. Therefore by substituting I_mean = I_ph into (4), the steady-state value of the pixel’s output can be obtained:

This feature is the main difference with respect to the biological findings, which on the contrary, show the existence of a logarithmic increase in the steady-state response with respect to the luminosity, giving rise to the so called “response compression” (see Section 2). In fact, while the contrast sensitivity of the OPL varies, depending on the average luminosity (see for instance the slope around the small horizontal line of the full-triangle curve in comparison with the empty-triangle one in Fig. 1), our silicon retina shows the same contrast sensitivity whatever the average luminosity. In other words, an object showing a contrast of 10% will generate the same signal amplitude at the output of the M²APix under both low and high luminosity levels.

In order to explain this behavior more clearly and show how it is related to S-shaped curves such as those presented in Fig. 1, an example of the pixel’s response was plotted as shown in Fig. 5(a).

At the time t₀, all the photodiode currents and the pixel’s output are stabilized at I_ph0 and V_out0, respectively. Then, after the time T_s, a step change in the luminosity occurs in front of the silicon retina, making all the photodiode currents “rapidly” change from I_ph0 to I_phi (red line), while the average current changes much more slowly (magenta line). During the stabilization of the average current, the pixel’s output first increases from V_out0 to the peak value V_outi, and then decreases until it reaches its stable baseline value V_out0 (blue line) again. We take rise time (T_r) to denote the time taken by V_out to go from V_out0 to 90% of (V_outi − V_out0) and fall time (T_f) the time taken by the signal to go from V_outi to 90% of (V_out0 − V_outi). It can be shown that T_r and T_f mainly depend on the photodiode and transistors time-constant and the averaging block time-constant ( ${\tau }_m=\frac{C_m}{G_m}$), respectively.

As T_r ≪ τ_m, it can be assumed that when V_out = V_outi, I_mean does not change (I_mean ≈ I_ph0) independently from I_phi. Therefore, the points defining the S-shaped curve correspond to the peak values V_outi reached at all the step values I_phi after the same initial value I_ph0 (Fig. 5(b)). It is worth noting that when the value of I_phi reaches up to ±1 decades of I_ph0, the pixel shows a logarithmic sensitivity to changes in the lighting conditions (see the sloping part of the curve around the operating point), while the sensitivity decreases drastically in response to greater changes in the light.

Thanks to the intrinsic properties of the normalization, the peak values depend only on the ratio $\frac{I_{{\mathit{ph}}_i}}{I_{{\mathit{ph}}_0}}$, resulting in a horizontal shift of the S-shaped curve depending on I_phi in the same way as the curves in Fig. 1.

In addition, as I_ph is linearly proportional to the luminous intensity, we can assume the presence of a luminous contrast defined by the Michelson formula: $c_i=\frac{I_{{\mathit{ph}}_i}-I_{{\mathit{ph}}_0}}{I_{{\mathit{ph}}_i}+I_{{\mathit{ph}}_0}}$. Substituting the inverse of this formula into equation (4), i.e. $I_{{\mathit{ph}}_i}=\frac{1+c_i}{1-c_i}{I}_{{\mathit{ph}}_0}$, we obtain:

The pixel can therefore be said to give a linear contrast response, and the contrast resolution is given by the coefficient of c_i in equation (6).

Lastly, an AC noise simulation was performed with a white noise at the input (corresponding to the shot noise of the photodiode), to obtain the Root Mean Square (RMS) of the output noise and consequently the minimum detectable contrast for different values of the average luminosity, i.e. the DC photodiode current. The RMS values are given by integrating the output noise in [10⁻⁵, 10⁸]Hz. The minimum detectable contrast can be defined as the contrast that gives rise to a transient response of the output signal ±6-fold the RMS noise.

Figure 6 shows the RMS of the simulated output noise (blue) and the minimum detectable contrast (red) with respect to the photodiode current. We can notice that the noise is decreasing with the DC photodiode current. At very low background luminosity, the noise is dominated by the input white noise of the photodiode which is amplified by the gain of the circuit. At higher luminosity, this gain is smaller, so the simulated input noise becomes negligible and the output noise is nearly equal to the transistors’ noise. In any case, the RMS values obtained are always very low compared to the output variations (V_outi − V_out0) in the transient response, as shown in the simulated response in Fig. 5. Consequently, the minimum detectable contrast is very low over the entire operating range of the average luminosity, varying from ±1.1% at 1Lux to ±0.4% at 10⁵ Lux.

 

Fig. 6 Root Mean Square (RMS) of the simulated output noise (blue) and corresponding minimum detectable contrast (red) with respect to the photodiode current. The RMS values are given by integrating in [10⁻⁵, 10⁸]Hz the output noise obtained with an AC noise simulation which takes into account all the transistor noises and a white noise for the input photodiode. The minimum detectable contrast can be defined as the contrast that gives rise to a transient response of the output signal ±6-fold the RMS noise.

Download Full Size | PDF

5. Method of characterization

In the studies presented in Section 1, the light adaptation and contrast sensitivity of silicon retinas were often tested by applying a series of lighting steps (AC light) in addition to various background lights (DC light). As a result, the pixels’ responses have often been described in terms of the stimulus intensity, and direct comparisons can therefore be made with the biological findings (see Section 2). However, the method used to characterize pixels’ responses is sometimes not clear or has not even been described at all, which makes comparisons with other results very difficult.

A standard method is presented here for accurately characterizing pixels’ responses to luminous changes of up to ±3 decades in a 7-decade mean luminosity range by implementing a single light source, which has been called the Lighting Box (Fig. 7).

The Lighting Box consists of a 50 × 25 × 25mm 3-D printed box with a 10 × 10mm aperture. The Lighting Box also includes a Printed Circuit Board (PCB) which accurately controls the light intensity of a red Light Emitting Diode (LED) (TLWR7600, Vishay Semiconductors) by means of a specific digital current driver (ADN8810, Analog Devices). The PCB with the LED and the PCB supporting the retina to be characterized are fixed to each side of the box so that both the LED and the silicon retina can fit into the aperture, facing each other inside the box (see Fig. 7(b)). The box also contains an optical filter support, which can be inserted between the LED and the retina in order to attenuate the LED’s intensity and thus characterize the pixel’s response in the illuminance range of interest.

The following main tools were used for this purpose:

 

Fig. 8 Block diagram of the hardware setup and communication flow involved in the pixel characterization procedure.

Download Full Size | PDF

Characterization procedure

The 12 pixels’ output signals were acquired while they were being exposed to step changes in the luminous intensity, as described in Section 4.2. During the overall acquisition process, the FPGA acts as the master component handling and synchronizing the communications with the Lighting Box in order to drive the intensity of the LED, and with the Explorer 16 in order to acquire data from the chip. Based on Fig. 5 and Fig. 8, the i-th step in the procedure can be described as follows:

As we were interested in characterizing our auto-adaptive pixel in 7 decades of photodiode current (see Section 4.1) using a LED covering three decades, four neutral optical filters (NG filters, Schott) were used: the 1 and 2mm NG3 type for dealing with 1- and 2-decade attenuation, respectively, and the 2 and 3mm NG9 type for dealing with 3- and 4-decade attenuation, respectively. To check the full set of S-shaped curves within ±3 decades about I_ph0 (see Fig. 5(b), for example), a complete pixel characterization was carried out by merging the data obtained with the various optical filters. In particular, as the latter give contiguous 1-decade attenuation, S-shaped curves were obtained by merging and averaging the peak values V_outi acquired at the same initial irradiance with several filters. For instance, the S-shaped curve centered in $0.1\hspace{0.17em}\frac{W}{m^2}$ (Fig. 5(b)) was obtained by merging the peak values obtained with I_LED0 = 0.1mA without any filter, I_LED0 = 1mA with a 1-decade attenuation filter, I_LED0 = 10mA with a 2-decade attenuation filter and I_LED0 = 100mA with a 3-decade attenuation filter.

6. M²APix characterization results

In this section, it is proposed to present and discuss the experimental data obtained when our auto-adaptive silicon retina was exposed to changes in the light, as described in Section 5.

To characterize the pixel’s response in the light-adapted condition, the LED light changes were triggered when the pixel’s output signal reached its steady-state value. In other words, by referring to the steps in the characterization procedure presented in Section 5, the waiting time T_wait in Step 3 was equal to 5T_f, where T_f denotes the fall time, as described in Section 4.2.

6.1. M²APix transient response

As mentioned in Section 4.2, we were interested in measuring the peak values V_outi of the twelve pixels with each pair of photodiode currents I_ph0, I_phi corresponding to a step change in the light intensity (Fig. 5). Pixel output signals were recorded with various pairs of LED irradiance values I_LED0, I_LEDi in the [10⁻⁵, 10²] $\frac{W}{m^2}$ range at a sampling frequency f_s = 2kHz, in order to accurately determine the instant at which the peak value occurred, using the method described in Section 5. The set of all the mean values obtained when I_LEDi = I_LED0 corresponds to the steady-state response of the pixel and constitutes an important auto-adaptive characteristic of the M²APix, which will be discussed below.

Some examples of the pixel’s response with $I_{{\mathit{LED}}_0}=1\hspace{0.17em}\frac{W}{m^2}$ are presented in Fig. 9.

 

Fig. 9 (a) Example of M²APix responses to step changes in the LED irradiance (I_LEDi), starting with the same initial irradiance ( $I_{{\mathit{LED}}_0}=1\hspace{0.17em}\frac{W}{m^2}$). (b) Zoom of the temporal pixel responses shown in Fig. 9(a) ranging between 0 and 50ms. To be able to distinguish more clearly between the steady-state responses and the transient responses, the step changes were delayed by 10ms (T_s) after the acquisition procedure had started. The black circles amount to 90% of the peak values V_outi and give qualitative information about the rise time (T_r). The contrast values are given by the Michelson formula $c_i=\frac{I_{{\mathit{LED}}_i}-I_{{\mathit{LED}}_0}}{I_{{\mathit{LED}}_i}+I_{{\mathit{LED}}_0}}$.

Download Full Size | PDF

Figure 9 shows the auto-adaptation of the pixel’s time response. According to the simulated example presented in Section 4.2, the pixel’s output rapidly increases from the steady value V_out0 to a peak value V_outi and then returns slowly to the steady value regardless of the contrast, which was obtained here by making step changes of I_LEDi. However, as shown in Fig. 9(b), the rise time (T_r), namely the time required to reach 90% of the peak value (black circles), is nearly constant with positive contrasts (I_LEDi > I_LED0), but depends on the contrast with negative ones (I_LEDi < I_LED0).

Figure 10 shows the mean rise time over the 12 pixels plotted with respect to the contrast. Each point corresponds to the time (T_r) required to reach 90% of the peak value V_outi, as depicted in Fig. 9(b), for every step change (I_LED0, I_LEDi).

 

Fig. 10 Average rise time with respect to luminous contrast. Each point corresponds to the time (T_r) required to reach 90% of the peak value V_outi, as depicted in Fig. 9(b). Each color refers to a different initial irradiance value I_LED0: red about $0.001\hspace{0.17em}\frac{W}{m^2}$, pink about $0.01\hspace{0.17em}\frac{W}{m^2}$, dark blue about $0.1\hspace{0.17em}\frac{W}{m^2}$, light blue about $1\hspace{0.17em}\frac{W}{m^2}$, cyan about $10\hspace{0.17em}\frac{W}{m^2}$, green about $100\hspace{0.17em}\frac{W}{m^2}$.

Download Full Size | PDF

By looking at Fig. 10, we can notice that the rise time depends on both the contrast and the average luminosity. This coupling between contrast and rise time is directly due to the current-mode functioning of the system. A high negative contrast corresponds to a normalizer’s output current tending to zero, as we can see by considering I_phi ≪ I_meani in equation (2). The time constants of the output transistors will therefore be higher and the output signal will be slower in this case. Furthermore, the time constant of the photodiode and the transistors increases for low current, i.e. low average luminosity, explaining the higher rise times for the pink and red data points. It is worth noting that the mean rise time for positive contrasts at medium/high average luminosity (blue and green data points) is about 1ms because of the anti-aliasing low-pass filter (F_c = 300Hz, see Section 4.1).

The rise time (T_r) and the fall time (T_f) depends on the bandwidth of the output signal that is determined mainly by the photodiode and transistors time constant and the current-averaging block time constant (τ_m). Such a bandwidth can be modified by modifying the external capacitor C_m, since ${\tau }_m=\frac{C_m}{G_m}$ (see Section 4.1). As the fall time T_f determines the time the pixel’s output takes to reach the maximum of its contrast sensitivity, i.e. the sensitivity at steady-state, it might be useful to set T_f as small as possible. If we consider an irradiance higher than $0.1\hspace{0.17em}\frac{W}{m^2}$ (about 100 Lux), the fall time T_f can be reasonably reduced up to 0.1s by changing the value of the external capacitor C_m to 5nF, because the rise time T_r is lower than 0.01s for any contrast in this irradiance range. In fact, T_f should be about one decade greater than T_r to guarantee the correct functioning of the pixel in this luminosity range.

6.2. M²APix S-shaped and steady-state response

The peak values obtained in response to step changes (I_LEDi) starting with several initial irradiances (I_LED0) are presented in Fig. 11, with respect to (a) the LED irradiance and (b) the Michelson contrast c_i, defined as follows $c_i=\frac{I_{{\mathit{LED}}_i}-I_{{\mathit{LED}}_0}}{I_{{\mathit{LED}}_i}+I_{{\mathit{LED}}_0}}$.

 

Fig. 11 (a) S-shaped curves and steady-state responses of the 12 pixels to LED irradiance changes in a 7-decade range. Each color refers to a different initial irradiance value I_LED0 (red about $0.001\hspace{0.17em}\frac{W}{m^2}$, pink about $0.01\frac{W}{m^2}$, dark blue about $0.1\frac{W}{m^2}$, light blue about $1\hspace{0.17em}\frac{W}{m^2}$, cyan about $10\hspace{0.17em}\frac{W}{m^2}$, green about $100\hspace{0.17em}\frac{W}{m^2}$, same color and marker as Fig. 10), and the data points correspond to the average peak value V_outi reached by the 12 pixels in response to a step change in the irradiance I_LEDi, as shown in Fig. 5. The steady-state response (black points) was obtained with I_LEDi = I_LED0 at several values of I_LED0, whereas the initial values of the irradiance I_LED0 are indicated by large black circles. The shaded areas were obtained by plotting the minimum to maximum values of the mean pixel output voltages. The average dispersion of each curve (σ_mean) ranged from 37.2mV in the case of the green one, to 85.4mV, in that of the red one. (b) Average peak response of the 12 pixels versus the contrast. The various curves correspond to the S-shaped curves in Fig. 11(a) (same colors and markers). The contrast is given by the Michelson formula: $c_i=\frac{I_{{\mathit{LED}}_i}-I_{{\mathit{LED}}_0}}{I_{{\mathit{LED}}_i}+I_{{\mathit{LED}}_0}}$.

Download Full Size | PDF

By comparing the S-shaped curves in Fig. 11(a) with those in Fig. 1, it can be seen in the first place that our silicon retina shows in qualitative terms the same adaptation process as that observed in the OPL. In particular, the pixels auto-adapt to the average light in a 7-decade range while keeping a sensitivity of nearly $600\hspace{0.17em}\frac{mV}{\mathit{Log}(I)}$ in a range of about 2 decades (corresponding to the linear part of the curves on the log scale). Each S-shaped curve obtained was well defined within a range of 6 decades, which means that the circuit did not deviate, but remained consistent with the model, even when the light changed suddenly by anything up to ±3 decades, which would correspond, for instance, to shifting from a very dark overcast sky to direct sunlight (see the lower half of the dark-blue curve). In addition, in line with equation (5), the steady-state response (black points) was almost constant throughout the 7 decades, giving the same contrast sensitivity whatever the average luminosity. Lastly, upon setting the high reference of the ADC at 2.4V (see Section 3 for further details about the integrated ADC), we obtained a LSB voltage of $V_{\mathit{LSB}}=\frac{2.4}{2^{10}}=2.35\hspace{0.17em}mV$, which corresponds to half of the voltage we would have obtained by setting the high reference of the ADC at V_dd. As the pixel output signal was limited to [1, 2.4]V, we obtained a effective resolution of the analog-to-digital conversion of $\frac{1.4V}{2.35\hspace{0.17em}mV}=596=2^{9.22}$, i.e. about 9 bits.

The fact that the left part of the pink and red S-shaped curves is lower than that of the other ones was due to the dark current of the photodiodes. The photodiode current can be divided into 2 components: the background current I_lbg, which depends on the irradiance, and the dark current I_dark, which has a low constant value that does not depend on the irradiance. Therefore, as long as I_lbg ≫ I_dark, we can assume that I_ph ≈ I_lgb and equation (4) still holds if we substitute I_LED into I_ph, whereas when I_lbg ≪ I_dark, we have I_ph ≈ I_dark, and V_out takes a constant value regardless of I_LED.

In addition, the dispersion of the S-shaped curves from one pixel to another can be seen in the shaded areas in Fig. 11(a). It is worth noting that this dispersion seems to be higher in the case of positive contrasts. This pattern is mainly due to the generation of the current I_b and not to the pixels themselves. As I_b is entirely conveyed to the normalizer’s output current I_out in the case of positive contrasts, its dispersion is directly transmitted as well. This behavior can be improved by improving the dispersion of the I_b generation cell.

In Fig. 11(b), the data points in Fig. 11(a) have been plotted versus the contrast. It can be seen from this figure that the pixel’s response decreased almost linearly with the luminous contrast regardless of the average luminosity, as predicted by equation (6). The non-linearity observed with highly negative contrasts was due to the non-linearity of the VLSI current-to-voltage converter. The contrast resolution K_c can be defined as the coefficient of c_i in (6) divided by 100, i.e. $K_c=-\frac{R_f{I}_b}{2\times 100}=-4.4\frac{mV}{\%}$. As the noise level is about 5mV (i.e. 2×LSB), contrasts as low as 2% can be detected. In addition, as the LSB voltage V_LSB is about 2.35mV, it is worth noting that a 1% contrast gave rise to a 2-bit change.

6.3. Faithfulness of the M²APix characterization to the Michaelis-Menten model

The absolute value of the errors observed between the peak values V_outi in Fig. 11(a) and the theoretical values $V_{{\mathit{out}}_i}^{*}$, based on the model for the circuit defined in Section 4.1, is presented in Fig. 12. As post-layout simulations of the circuit showed that the current-to-voltage converter gave a non-linear response at low current values, the theoretical values $V_{{\mathit{out}}_i}^{*}$ were computed by applying a look-up table of the current-to-voltage converter to equation (2), with I_ph = I_LED instead of using equation (4).

 

Fig. 12 Absolute value of the error between the M²APix responses measured and those predicted by the model with respect to the LED irradiance. As post-layout simulations of the circuit showed that the current-to-voltage converter gave a non-linear response at low current values, the theoretical values $V_{{\mathit{out}}_i}^{*}$ were computed by applying a look-up table of the current-to-voltage converter to (2), with I_ph = I_LED instead of using equation (4). (Same color and marker as previous figures)

Download Full Size | PDF

It can be noted that on each S-shaped curve (each of which is presented in a different color), the difference between the pixel output signals and the model outputs ranges from 0.1% to 8% of the whole output range, showing a good match with the Michaelis-Menten function. Possibly due to the existence of a mismatch between the actual response of the current-to-voltage converter and the simulated post-layout response, the error decreased almost monotonically with the irradiance. In addition, the existence of a greater error in the case of the red curve ( $I_{{\mathit{LED}}_0}=0.001\hspace{0.17em}\frac{W}{m^2}$) was due to the presence of the photodiode dark current (I_dark), as explained above.

7. Comparison between M²APix and Delbrück pixels

In this section, it is proposed to present a quantitative comparison between the M²APix and the Delbrück pixel [16] implemented in the same silicon retina (see Fig. 2) when tested under the same conditions.

Table 1 lists all the values discussed in Section 6 for the M²APix and the corresponding values obtained for the Delbrück pixel.

Table 1. Specifications of the M²APix and Delbrück pixels. The main advantage is that M²APix responds monotonically over very wide illuminance range without any loss of sensitivity and contrast resolution.

View Table

The Delbrück pixel reproduced non-monotonic responses with respect to the LED intensity for medium-high luminosity, showing unreliable responses for changes higher than ±1 decade at $I_{{\mathit{LED}}_0}>1\hspace{0.17em}\frac{W}{m^2}$. Furthermore, the steady-state response increased with the luminosity, resulting in a higher DC output variation and a lower effective output range. Finally, the adaptation time constant of the M²APix, which determines the bandwidth of the output signal, can be potentially modified externally.

To show how the two types of pixel respond to small contrasts while they are still adapting to the average luminosity, we applied the procedure presented in Section 5, but contrary to the static case (see Section 6), they were exposed to repeated sequences of 0.5s-long changes in the LED irradiance (corresponding to contrasts ranging from −10 to 10%) while the average irradiance was increased by 1 or 2 decades every 5s.

Figure 13 shows the time responses of one M²APix and one Delbrück pixel when exposed to two different step sequences. Both types of pixel responded quickly to small changes (small contrasts) while adapting to the average luminosity, regardless of the average luminosity and the changes in the luminosity (amounting to 1 or 2 decades in this example). However, the two pixels’ responses presented some remarkable differences:

 

Fig. 13 (a) Two sequences of small contrasts in a 2-decade irradiance range. Time responses of (b) M²APix and (c) Delbrück pixel [16], implemented in the same silicon retina (see Fig. 2), when exposed to the irradiance sequences in Fig. 13(a). The sequences were obtained by repeating: ±2% and ±4% contrasts (blue line), ±6% and ±12% contrasts (red line). The steps in the sequence were triggered every 0.5 s and the changes in the average irradiance every 5s. For the sake of clarity, the timing of the sequence has been slightly shifted. (The spurious peaks, such as that which occurred at 10.5 s, may have been due to some error in the data transmission)

Download Full Size | PDF

It is worth noting that the M²APix responded appropriately to a 6% contrast just after a 1-decade change (see the red line in the right-hand zoomed part of Fig. 13(b)), which still corresponds to a good contrast sensitivity under light-adapting conditions. Moreover, such contrast sensitivity can be improved by decreasing the time constant of the current averaging block by reducing the value of the external capacitor C_m (see Section 4.1 for details). In this way, the fall time T_f would be lower and the pixel’s sensitivity would increase faster (see Section 4.2 for details).

8. Conclusion

The results presented here show that our Michaelis-Menten analog auto-adaptive pixel (called M²APix) can adapt automatically to any irradiance from 10⁻⁵ to ${10}^2\hspace{0.17em}\frac{W}{m^2}$, which would correspond, if the LED light was green, to an illuminance of about 7 × 10⁻³ and 7 × 10⁴ Lux respectively, that is from half moon on a clear night to nearly direct sunlight. At the same time, the results obtained showed that the circuit does not deviate from the model, even when the light suddenly changes by up to ±3 decades, which would correspond, for instance, to shifting from a very dark overcast sky to direct sunlight.

In short, our auto-adaptive pixel can be said to have the following noteworthy features:

Some further improvements could be made in a future version of the M²APix in the conversion stage, by subtracting 1V from the pixel output signal before the conversion or increasing the resistance R_f of the current-to-voltage converter in order to eventually increase the conversion range to [0, 2.4]V. This would increase the contrast resolution nearly two-fold, giving a 4-bit change in response to a 1% contrast, corresponding to a minimum detectable contrast of 1% in the light-adapted condition and 3% in the light-adapting condition.

Since the M²APix makes a satisfying compromise between a high sensitivity and a wide dynamic range, it should provide a useful tool for motion detection and optical flow processing in a very large range of lighting conditions, from half-moonlight to full daylight. Thus, our auto-adaptive silicon retina, or retinas composed of larger arrays of M²APix, could be employed in several fields, from event-based applications to bio-robotics and bio-medical applications.

In the future, we plan to test our silicon retina outdoors with a suitable optical lens both at night and in the daytime, by using it to measure the optic flow, for instance. In particular, we are willing to mount one or more M²APix-based sensors onboard mobile and aerial robots (as in [42]) as aids to navigation under various luminosity conditions, as well as on vehicles in the case of automotive applications.