Expand this Topic clickable element to expand a topic
Skip to content
Optica Publishing Group

Experimental validation of self-focusing image formation for retinal projection display

Open Access Open Access

Abstract

The augmented reality (AR) industry requires both aesthetic designs and high performances of AR devices. This complex dilemma challenges R&D groups from all over the world to improve existing systems or propose new, breakthrough designs. The unconventional concept of direct retinal projection display may be one. It is based on see-through holographic retinal projection, with the image being formed via the so-called self-focusing effect. In this paper, we describe an experimental validation of this self-focusing effect and introduce a possible approach of self-focusing performance evaluation. Experimental image formation capability is demonstrated and compared with simulation results. Main present limitations of the concept are discussed, such as pixel addressing design and image resolution/sharpness conflict.

© 2019 Optical Society of America under the terms of the OSA Open Access Publishing Agreement

1. Introduction

Augmented reality (AR) comes smoothly but steadily into our daily life. There are a lot of applications for AR [1,2], in very diverse domains such as medicine and surgery, education construction safety, or maintenance, and numerous others [3–8]. Many scientific groups are in the process of creating their own smart glasses devices [9,10]. In this field, the need to overlay digital information to the surrounding real world usually requires the use of micro-displays. These devices, in turn, require complex optical systems to provide a comfortable vision. Furthermore, the formation of the virtual image of the screen by the device is accompanied by constraints related to the management of pupils that limit the viewing angle and the positioning latitude of the eye while leading to bulky systems.

Bernard Kress and Thad Starner [11] described a wide range of different types of head-mounted displays (HMD). Most of them use conventional approaches for image formation based on the use of a micro-display and an optical system. The CEA proposed in 2014 an original concept of image projection that combines holography, integrated photonics and free space optics [12]. The concept aims at projecting an image into the observer's eye, onto the retina, without any lens. Convergence is provided by the eye through a phenomenon that we describe as self-focusing. This unconventional approach was explained recently for a concept of retinal projection display [13].

The concept is based on the design of an emissive screen close to the eye, where the emission points are duplicated so as to form the illusion of a plane wavefront. This is the case of Fig. 1(a). Thanks to the self-focusing effect, the multiple beams emitted from the display focalize onto the retina without any lens between the emission surface and the eye. The resulting focalized spot corresponds to one Emissive Point Distribution (EPD). An image, composed of several spots, requires a respective number of EPDs, which means that the surface of the display is covered by various EPDs to be able to form an image. Every EPD, in turn, consists in emissive points that are activated simultaneously and coherently for a given EPD, and designed to generate a directive spherical wave front with main wave vector ks.

 figure: Fig. 1

Fig. 1 Self-focusing image formation: (a) concept of near to eye display where a distribution of spherical wavefronts is emitted and produces the illusion of a plane wavefront that the eye lens focuses on the retina; (b) our validation setup: a laser beam passes through a transmission device consisting of a distribution of pinholes in an opaque metallic layer.

Download Full Size | PDF

As the wavefronts emitted from every point are phase adjusted for a given EPD and angularly orientated, they produce a resulting planar wavefront of wave vector kp. Through multiple interferences the spherical wavefronts focalize the beams onto the retina due to self-focusing effect through the eye pupil.

Self-focusing effect has already been demonstrated experimentally for data storage applications [14]. The use of a distribution of small apertures that approximates the behavior of a large, continuous aperture has also been demonstrated experimentally for astronomical applications [15–17]. However, the unconventional aspect of this approach for display applications opens questions about the perception of the so-formed images on the retina. Theoretical part was discussed previously [13], and we now experimentally investigate the capability of an unconventional image formation directly onto the retina using a device that simulates the self-focusing display concept as shown in Fig. 1(b). A planar wavefront with a given angular direction of wave vector kp passes through a transmission device consisting of a distribution of apertures in an opaque metallic layer. The device simulates a distribution of emitters that generate coherent, phase-adjusted spherical wavefronts ks with main wave vector direction kp.

The pinhole distribution represents the EPD and the device is placed in front of an optical system that mimics the eye.

In this article, we present first experimental results of unconventional image formation for the concept of retinal projection display developed by our laboratory. Section 2 presents the choice of the EPD and the mathematical development used to evaluate the randomness of the distributions. The third section describes the experimental validation of the self-focusing effect. Various EPDs are evaluated and a double Gaussian model is introduced to describe the signal formed by a central peak and a surrounding noise. This model is used in the last section to theoretically evaluate the behavior of the concept as an image-forming device. Simulations are then compared to experimental measurements of a self-focused image obtained from an optical system that mimics the eye.

2. Emissive points distribution for self-focusing

The EPD design plays a dominant role for self-focusing, therefore the best distribution to implement must be found. We introduce some possible solutions and discuss their advantages and drawbacks. As previously shown [13], the best theoretical EPD for image formation is a random distribution. However, such a solution represents a technological challenge for the device manufacturing and the EPD addressing.

On the opposite, the easiest solution for the device manufacturing and addressing is a periodic EPD, but this solution is not acceptable from the image formation point of view. Hence, a compromise is to be found, such as a quasi-random distribution.

We introduce the Emissive Unit Cell (EUC) to refer to the elementary building block of the display. EUC is a region with sizes Λ1 and Λ2, composed by one emissive point from each EPD. As described in [13], each emissive point corresponds to the intersection of a waveguide and an electrode. To provide the interference phenomenon, each EPD is addressed by a single laser. Image formation is then conceived in a scanning mode with a frequency that depends on the number of lasers used to project the image. It comes that each EUC will be addressed by several lasers nlaser and electrodes nelect that fixes the number of the pixels N of the projected image:

N=nelectnlaser

The size of the EUC is then given by the distances between rows of waveguides dg and between columns of electrodes de :

Λ1=nelectdeΛ2=nlaserdg

A simple way to solve the addressing problem is to create a 2D grid of periodically-distributed, straight electrodes. The coordinates of the EPD xi,j and yi,j are then given by the equation:

xi,j=Λ1iyi,j=Λ2j

Another solution for the EPD is to define a quasi-random distribution inside the EUC:

xi,j=Λ1(i+rnd)yi,j=Λ2(j+rnd)

Where rnd is a random number, with rnd ∈ [0:1].

This solution is theoretically interesting but is hard to implement in practice because of an intersection distribution between a waveguide and an electrode trajectory function. To generalize the addressing problem, we define these two trajectory functions as fe(x,y) and fg(x,y). Solving the EPD design problem then requires solving the intersection equation between a distribution of waveguide and electrode function.

With respect to Eqs. (3) and (4), this can be done in two ways. First by keeping a periodic distribution:

(xi,j,yi,j)=fe(x+Λ1i,y)fg(x,y+Λ2j)

Secondly by introducing a random access addressing:

(xi,j,yi,j)=fe(x+Λ1(i+rnd),y)fg(x,y+Λ2(j+rnd))

To facilitate the determination of the intersections between fe and fg we choose to describe these functions as the trajectory of a moving particle. The location r(x,y) of the particle is given by the time varying relation:

f:r(t+δt)=r(t)+V(t)δt

The velocity function V(t) that fixes the trajectory is defined by a time varying amplitude and direction:

V(t)=[v0(t)cos(α(t))v0(t)sin(α(t))

In this work, we choose a simple sinusoidal model with the following definition:

v0(t)=v0=const.α(t)=α0cos(2πt0t)

Starting from these equations, we calculate the intersections of the functions fe and fg according to the two cases described in Eq. (5) (Cross Sinusoidal distribution CS) and (6) (Cross Random Sinusoidal distribution CRS). The resulting EPDs are given in Figs. 2(e) and 2(g). It consists on a set of NEPD points. The cases of periodic and quasi-random distributions are described in Figs. 2(a) and 2(c) respectively.

 figure: Fig. 2

Fig. 2 Calculated distributions and their histograms of the closest distance between the nearest pinholes: (a) and (b) periodic distribution; (c) and (d) quasi-random distribution; (e) and (f) CS distribution; (g) and (h) CRS distribution.

Download Full Size | PDF

The physical process involved in self-focusing is based on a multiple interference phenomenon. It can be described as a superposition of series of Young’s fringes figure, with a period and an orientation given by the amplitude and orientation of the vector up,q that connects two interfering emissive points p and q from a given EPD.

The efficiency of the self-focusing effect is related to the number and to the diversity of this vector distribution. To evaluate this diversity, we introduce the histogram of the minimal distance distribution.

We first calculate the distribution of the minimal distance s from a point to its neighbors:

sp=min(|up,q|)qEPD

We consider the case of a square EUC: Λ1 = Λ2 = Λ. The histogram is calculated with a number Nh of bins calculated between 0 and Λ:

hq=p=1NEPD[0<(spq2ΛNh)2ΛNh]

The histograms of the various EPDs are presented Fig. 2, right part. In the case of the periodic distribution, the histogram is a Dirac located at the period of the distribution, see Fig. 2(b). The histogram is a large distribution for the quasi-random EPD as shown in Fig. 2(d). The choice of a sinusoidal design for the waveguides and electrodes allows to enlarge the histogram function particularly in the case of the random access, case of Fig. 2(h) with respect to the periodic one, as depicted in Fig. 2(f).

This first mathematical approach to the design of the EPD shows some orientations that will be followed in next investigations to improve the histogram distribution in order to achieve better self-focusing performances.

To validate the link between the EPD histogram and the self-focusing behavior, the various EPDs considered in this study have been manufactured in an opaque layer, and experimentally evaluated using an optical set-up that transposes the self-focusing behavior from an emissive to a transmissive configuration.

3. Experimental validation of self-focusing effect

3.1. Set-up description

The optical set-up consists in a 513 nm laser diode, emitting through an optical fiber. The beam impacts an opaque sample made out of a thin glass, coated with an opaque, black chrome layer. A distribution of apertures of various sizes and distribution functions has been etched through the opaque layer with a maskless lithographic process Fig. 3(a) [18]. In order to simulate a phase-adjusted emission through the EPD, the fiber laser is located at a large distance from the sample. The beam incident on the EPD sample can then be considered as representing a planar wave, as shown in Fig. 3(b).

 figure: Fig. 3

Fig. 3 Experimental set-up for self-focusing effect validation: (a) pictures of a sinusoidal periodic apertures distribution mask, (b) set-up description.

Download Full Size | PDF

Beyond the aperture distribution sample, we evaluate the self-focusing effect with an imaging system that mimics the eye. We use as optical system an objective with a focal length f0 = 51 mm and a high resolution 6.6 MegaPixels monochrome CMOS sensor with a 3.5 µm pixel size. The distance between the aperture distribution sample and the optical system is about 20 mm.

3.2. Experimental results

The first results given in Figs. 4(a) and 4(b) present the case of a single aperture. In this case self-focusing does not occur, and one observes the blurred image of the aperture. Figure 4(a) shows the intensity of the beam measured on the CMOS sensor in angular coordinates. Figure 4(b) depicts the cross section of the signal.

 figure: Fig. 4

Fig. 4 Diffraction patterns produced by various EPD, on the left the image on the image sensor (in green false color), on the right the intensity cross section compared with Gaussian function: (a) and (b) one pinhole; (c) and (d) periodic distribution; (e) and (f) quasi-random distribution; (g) and (h) CS distribution; (i) and j) CRS distribution. Figure (f), (h) and (j) are plotted in logarithmic intensity scale. Inset in Fig. (f) is a detail of the spel theory/measurement comparison in linear scale near 0 arcmin.

Download Full Size | PDF

We use Gaussian beam formalism as described in [13]. The waist w0 of the beam emitted from the aperture is taken as the radius of the aperture. The angular radius Δθ (HWHM) of the blurred signal is then given by:

Δθ=λπw0

With a 50 µm diameter aperture, an angular radius of 22 arcmin is calculated. The value is consistent with the value of the waist deduced from the Gaussian model of Fig. 4(b) (waist 30 arcmin).

Figures 4(c) and 4(d) describe the image resulting from a periodic aperture distribution. The period Λ of the 50 µm diameter apertures distribution is 400 µm. The periodic aperture distribution generates diffraction orders with an angular period given by λ/Λ. It corresponds to an angular period of 4.4 arcmin, consistent with our measurements.

Figures 4(e) and 4(f) describe the case of a quasi-random distribution calculated on a 400 µm periodic EUC. These results confirm the self-focusing behavior. One observes a central peak, that we name spel (for elementary spot), with an angular radius δθ (first zero of the Airy function) given by the entrance pupil diameter Dp:

δθ=1.22λDp

The aperture of the optical system is given by the choice of the f#. In order to image a spel over several pixels, while still keeping a sufficiently high signal level, we choose an aperture of f/15 that corresponds to Dp = 3.4 mm. The calculated angular radius of 0.6 arcmin is consistent with the measurements (the inset of Fig. 4(f) shows the cross section pixel signal of the spel and a Gaussian model with waist 0.45 arcmin).

The signal outside of the spel is distributed as speckle noise.

Figures 4(g) and 4(h) show the result for the CS distribution. The resonant contribution is strongly reduced, compared to Fig. 4(d), but speckle noise still presents remaining peaks, compared to Fig. 4(f).

Finally, the choice of a CRS distribution allows for reducing the contribution of the unwanted peak, as shown on Figs. 4(i) and 4(j).

3.3. Analysis

The experimental results confirm that the self-focusing effect implemented with a random EPD allows for isolating a spel on the retina. This behavior is accomplished at the price of a speckle noise surrounding the spel. The characteristics of this noise contribution are related to the EPD:

  • • the size of the emissive points fixes the size of the noise envelop,
  • • the type of the EPD reduces the number of unwanted ghost spel in the noise,
  • • the number of points in the EPD fixes the signal to noise ratio between the spel and the noise.

To evaluate the impact of the noise on the imaging performance of the self-focusing effect, we introduce a double Gaussian model to describe the spel and the noise. Figure 5 shows a cross section of the spel for the quasi-random EPD case.

 figure: Fig. 5

Fig. 5 Double-Gaussian model for the spel. (a), (b) and (c) spel intensity distributions for three LD power (d) cross intensity function of the concatenated images (blue dotted curve) and Gaussian model for the peak and the noise (orange curves).

Download Full Size | PDF

One can separate the signal in two contributions:

  • • the spel is described by a first intensity Gaussian function with a waist w1,
  • • the noise is described by a second intensity Gaussian function with a waist w2.

Both intensity functions are described by the Gaussian beam formula:

Ii(r)=I0ie2(rwi)2i=1,2

I0i being the maximum intensity of the signal that fixes the total energy of the signal:

Ei=2πI0iwi2i=1,2

The size of the spel w1 is given by the total pupil aperture:

w10.7δθf0=0.86λf0Dp

The size of the noise contribution w2 is given by the pinhole aperture diameter dp as for the case of Eq. (12):

w2=2λf0πdp

The terms E1 and E2 represent the energy contains in the spel and in the noise, respectively. We define the parameter γ as the ratio in dB between the energy of the spel and the energy of the self-focused signal:

γ=10log(E1E1+E2)=10log(I01w12I01w12+I02w22)

Figure 5 shows the experimental characterization of the spel in the case of a random EPD with the aperture diameter dp = 75 µm. To fit both Gaussian models, three intensities of the LD are used to overcome the limited CMOS sensor dynamic: Fig. 5(a) to Fig. 5(c).

The cross intensities functions are given in spatial coordinates on the CMOS sensor. The double Gaussian parameters are: I01 = 1; w1 = 6.5 µm; I02 = 0.065; w2 = 200 µm and are consistent with the theoretical value given by Eqs. (16) and (17): w1 = 6.7 µm and w2 = 224 µm. The Gaussian model parameters give a noise ratio γ = 18.

The double Gaussian model has the great interest to give an analytical expression for the spel that can be used to evaluate the image forming process involved in self focusing as a function of γ.

4. Self-focusing image formation

4.1. Imaging process simulation

Simulation of imaging process in conventional imaging is based on the convolution of an image by the Points Spread Function (PSF) of the optical system. This process related to linear systems traduces the continuity of the image in the object and image plane of the optical system. Here, one has to consider a process with no image in the object plane of the optical system, and an image representation that is not continuous in the image plane. The simulation can still be made on the basis of the convolution with the optical response of the system, but we must consider here the image as a series of Dirac that localize the spels on the retina.

Considering an image formed onto the retina by a succession of spels with positions on the retina given by the vector ru,v and an intensity given by the matrix Mu,v, one describes the spel intensity distribution with the function g(r). The image Iret on the retina is given by the sum of the spel contributions:

Iret(r)=u,vMu,vg(rru,v)

We must here underline the specificity of the imaging process in self-focusing. In conventional optics the image on the retina is expressed by:

I˜ret(r)=u,vMu,vrectW(sru,v)PSF(rs)dxsdys

The function rectW represents the geometric image of the square pixel on the retina (with size and pitch W) and traduces the continuity of the image as a succession of adjacent squares with size given by the display characteristics and the magnification of the optical system.

According to Eq. (20), if we consider a micro display with resolution Nl x Nc pixels, an optical system gives the same result I˜ret(r) for an image of size Nl /2 x Nc/2 or for the same image oversampled by a factor 2.

In the case of self-focusing imaging, the oversampling by a factor 2 leads to an increase of a factor 4 on the number of spels and modifies the resulting image. This effect is described in Fig. 6. The image to be displayed is the letter T and is shown in Fig. 6(a) as given in a conventional display with adjacent pixel. In self-focusing each pixel can be represented by one spel as in Fig. 6(b), by four spels as in Fig. 6(c) or more. Increasing the number of spel is supposed to increase the image quality. However, as the total number of emissive points is fixed for a given display, increasing the number of spel reduces the number of emissive points allocated to each spel. This reduction of the EPD size degrades each individual spel, and by this way is supposed to degrade the overall image quality.

 figure: Fig. 6

Fig. 6 (a) Image of the letter “T” coded with 5x5 pixels, (b) self-focusing imaging with one spel per pixel and (c) self-focusing imaging with four spels per pixel.

Download Full Size | PDF

A compromise must be found between the number of spel and the efficiency of the spel in terms of image quality, so-called resolution/sharpness conflict. The goal of this paper being to present first experimental validations of self-focusing imaging, this complex aspect of self-focusing will not be more discussed here.

4.2. Experimental set-up

The set-up used to evaluate self-focusing imaging is described in Fig. 7. As we don’t have yet retinal display devices, the principle of the validation is to pass through a distribution of apertures as in the case of Fig. 3. The objective is not to generate a single planar wave through our EPD but a series of planar wave that project an image onto the retina. We use a micro-display with a pixel size δ and a collimating optics with focal fc as wavefront generator.

 figure: Fig. 7

Fig. 7 Optical set-up used to evaluate self-focusing imaging.

Download Full Size | PDF

Each pixel u,v of the display generates a wavefront of vector ku,v with the angular coordinates:

ku,v=[sin(ρu,v)cos(θu,v)sin(ρu,v)sin(θu,v)cos(ρu,v)

With the angular relations given in the paraxial approximation:

ρu,v=δfcu2+v2θu,v=cos1(uu2+v2)

The wavefronts are transmitted to the imaging system through the apertures holes of the EPD and form the spels of the image at coordinates ru,v on the CMOS sensor that mimics the retina.

ru,v=[f0ρu,vcos(θu,v)f0ρu,vsin(θu,v)

The microdisplay used in this work is depicted on Fig. 8. As the aperture distribution used in this work has a very low transmission factor due to the small size of the pinholes, a specific high power monochromatic microdisplay developed in our laboratory has been used. This binary microdisplay forms a static image of the word “LETI” on a 13 x 5 pixels resolution, and a single, isolated pixel is also activated. The size of the pixel is 8 x 8 µm2 with a pixel pitch of 10 µm. Typical brightness for this display is about 3000 Cd/m2 [19].

 figure: Fig. 8

Fig. 8 MicroLED display used to generate the image wavefront. On the left the display size compared with 1 eurocent; on the right the magnification of the display with an isolated activated pixel as well as the “LETI” acronym.

Download Full Size | PDF

For these measurements, the resolution of the sensor is increased and a new CMOS sensor is used with a smaller 1.67 µm pixel size.

4.3. Experimental results

We show on Fig. 9 the results of the image forming process on the various EPDs considered in this study. Figures are given in gray level and angular coordinates. To give a better visual rendering the gray scale of the figure is reversed. Figure 9(a) shows the image of the micro display without any aperture. It represents the perfect image only limited by the aberration of the optical system. Due to a too low energy level, the experimental evaluation of the imaging process through a single aperture was not possible.

 figure: Fig. 9

Fig. 9 Results on the experiment (a) diffraction limited, (b) periodic distribution, (c) quasi-random EPD and (d) CRS EPD.

Download Full Size | PDF

Figure 9(b) shows the imaging through a periodic aperture distribution (dp = 75 µm, Λ = 400 µm). We observe a duplication of the elementary image as expected from the theory. Figures 9(c) and 9(d) show the result of imaging through the quasi-random and CRS EPDs.

4.4. Simulation results

We use as image a 13 x 5 pixels sampling of the acronym “LETI” representative of the micro-display of Fig. 8. We choose a 130 x 50 pixels resolution for the whole image, with an emissive-pixel periodicity of the display of 10 pixels, and each emissive zone being itself sampled with 8 x 8 pixels.

Simulations are made on the basis of the double Gaussian model. The results based on the parameters given from Fig. 5 give low correspondence with our experimental measurements and correcting factors must be introduced in the value of w1, w2 and γ as given by Eqs. (16)– (18). Both values of the waist must be increased by a factor 1.5 and a noise ratio of 15 dB is used instead of 18 dB.

Figure 10(a) shows simulation for a conventional imaging behavior. We use the double Gaussian model with a ratio factor γ ~0 dB.

 figure: Fig. 10

Fig. 10 Results on the imaging simulation process (a) diffraction limited, (b) periodic distribution, (c) quasi-random EPD, (d) CRS EPD and (e) comparison of the middle cross section of the measured and simulated images in the quasi-random EPD.

Download Full Size | PDF

We simulate the case of a periodic EPD with a function that describes the spel distribution as shown on Fig. 4(c). The following definition is used for g (with the corrected waist values):

gper(r)=G0e2r2w22k,le2rrk,l2w12

First term G0 is used to normalize the total energy. Second term is the envelope of the apodization function. Third term is the grid of resonances due to the periodic distribution, vector rk,l giving the coordinates of the resonant peaks:

rk,l=[kλf0Λlλf0Λ

Figure 10(b) gives the result of the simulation. One can see that the repetition of the resonant peaks prevents the recognition of the word LETI, as observed in the measurements and as predicted by the theory.

The quasi-random distribution is simulated with the double Gaussian formalism:

gqr(r)=I1(r)+I2(r)

The simulation result given in Fig. 10(c) is used as a comparison between theory and experiment. Figure 10(e) compares the cross section of the experimental and simulated image made on the middle of the image in the quasi-random case. This case is used as a reference as it corresponds to the optimal imaging result for the EPD design. Correcting factors are determined to give the best fit between the two curves. Note that the Cross Sinusoidal and Cross Random Sinusoidal distributions (CS and CRS) distributions are easier to fabricate but don’t correspond to an optimal distribution realistic with our manufacturing capabilities. Future work will focus on determining such distributions with better self-focusing performances than CS and CRS. Note also that the brightness of the image is not strongly modified by the kind of EPD used, as long as the main characteristics of the EPD (size and density of the apertures) remain the same.

We can now recognize in Fig. 10(c) the word LETI surrounded by a strong noise contribution that degrades the contrast.

The CRS case is described by a function that mixes the formalism of Eqs. (24) and (26).

gCRS(r)=I1(r)+I2(r)+H0e2r2w32q,pe2rrq,p2w12

The first two terms of the equation correspond to the spel and the noise, respectively, as in Eq. (24). The last term corresponds to the ghost peaks that appear at coordinate rq,p with an angular extension w3 given in the experimental results of Fig. 4(i).

Figure 10(d) gives the result of image simulation with the CRS distribution. The image of the word LETI can be recognized but some ghost images appear at the ghost peak location as in the measurements.

4.5 Analysis

The experimental results confirm the ability to produce an image through the self-focusing effect. The model of a double Gaussian intensity distribution is efficient to describe the imaging process on the retina. It allows for fast simulations as summations of the various spels that form the image.

The main drawback of the device will concern the contrast of the retinal image and the presence of ghost images produced by limited random distributions. The EPDs that have been considered throughout this paper are related to simple sinusoidal models, and more complex configurations are currently under investigations to improve the image quality.

The best fit between simulation and measurements in Fig. 10 requires introducing some correcting factors. This correction can be explained by the difference in spectral characteristics between the laser diode used in Fig. 5 (FWHM ~1 nm) and the pixel of the microLED display used in Fig. 9 (FWHM ~30 nm) [20].

The EPD parameters have been chosen in order to simplify the visual analysis as shown on Figs. 9 and 10. Future work will focus on the parameters corresponding to our device with an emissive aperture diameter smaller than 10 µm and an entrance pupil diameter of about 6 mm. The three orders of magnitude difference between these two values make visual analysis difficult if one wants to evaluate on the same graphic the effect of the spel and of the whole surrounding noise as in Fig. 10(e).

For this analysis it is also of primary importance to consider the eye model, particularly in terms of extended sensing dynamic and non-uniform resolution. Specific diffusion behavior of the holographic part of the device also has to be considered for a rigorous analysis of the system performance [21].

It is also worth mentioning the influence of the source bandwidth. In particular, the increase of the spectral bandwidth expands the speckle figure around the central peak and tends to increase the ratio between the energy of the spel and the energy of the noise. This aspect still requires an in-depth analysis, however, effects of wavelength and bandwidth have been described experimentally in [20], to which the interested reader is referred.

Then, the display concept we propose works with a set of different Emissive Point Distributions (EPDs) each associated with an image pixel. To be fully representative of the concept, each pixel of the word “LETI” should have been associated with a specific EPD. This configuration is not possible in the frame of this first experiment, as all the display pixels highlight a transmission aperture distribution. We are currently investigating use of a holographic printer, to evaluate a configuration with different EPDs, for a closer-to-real-case application. Nevertheless, even if the aperture distribution is here still common to all the pixels, it anyway well describes the effect of image formation through self-focusing effect, to demonstrate its practical interest.

5. Conclusion

After the introduction of a new concept of retinal projector [13], we studied an implementation of the imaging process through the self-focusing effect. This first experimental demonstration allows introducing first elements of design for an Emissive Point Distribution consistent with the requirement of the addressing process through the intersection of sinusoidal waveguides and electrodes. First results are positive, but show that more complex addressing design has to be found in view of improving the visual rendering of the retinal projection.

Main limitations of the concept have been highlighted. The noise generated by the sampling of the planar wave by a series of spherical waves is a first limitation, inherent to the concept. Its reduction requires a compromise between the size and the number of the EPD, that is, between the resolution and the sharpness of the image (resolution/sharpness conflict). It is well described by a simple double Gaussian model that allows for fixing a signal to noise design parameter γ.

Another limitation is the presence of ghost images, directly related to the choice of the EPD and to its actual degree of randomness.

The link between γ, the EPD definition, the sampling and the angular size of the images is difficult to establish and will require pursuing research with the introduction of a more detailed eye model, in order to finely tune the image formation parameters, which are specific to this new image projection modality.

Finally, a natural extension of this work would be to study application for colors display. Here, monochrome demonstration of the self-focusing effect is provided, but the use of multiple wavelengths is possible, using same EPD for all the pixels of the image. It could allow for evaluating the spectral behavior of the self-focusing effect in a red-green-blue (RGB) configuration. However, in the case of the originally proposed retinal display concept [12,13], each EPD is to be associated with a specific hologram that fixes the emissive angle. As holograms are selective in wavelength, one would then have to use three separate holographic layers in order to encode three EPD color families for full RGB display.

Acknowledgments

The authors would like to thank both anonymous reviewers for their valuable suggestions to improve the manuscript.

References

1. B. Kress, E. Saeedi, and V. Brac-de-la-Perriere, “The segmentation of the HMD market: optics for smart glasses, smart eyewear, AR and VR headsets,” Proc. SPIE 9202, 92020D (2014).

2. M. Billinghurst, A. Clark, and G. Lee, “A Survey of Augmented Reality,” Found. Trends Human–Comp. Inter. 8, 73–272 (2015).

3. A. Meola, F. Cutolo, M. Carbone, F. Cagnazzo, M. Ferrari, and V. Ferrari, “Augmented reality in neurosurgery: a systematic review,” Neurosurg. Rev. 40(4), 537–548 (2017). [CrossRef]   [PubMed]  

4. J. W. Yoon, R. E. Chen, E. J. Kim, O. O. Akinduro, P. Kerezoudis, P. K. Han, P. Si, W. D. Freeman, R. J. Diaz, R. J. Komotar, S. M. Pirris, B. L. Brown, M. Bydon, M. Y. Wang, R. E. Wharen Jr., and A. Quinones-Hinojosa, “Augmented reality for the surgeon: Systematic review,” Int. J. Med. Robot. 14(4), e1914 (2018). [CrossRef]   [PubMed]  

5. J. Bacca, S. Baldiris, R. Fabregat, S. Graf, and Kinshuk, “Augmented Reality Trends in Education: A Systematic Review of Research and Applications,” J. Educ. Technol. Soc. 17(4), 133–149 (2014).

6. P. Chen, X. Liu, W. Cheng, and R. Huang, “A review of using Augmented Reality in Education from 2011 to 2016,” in Innovations in Smart Learning. Lecture Notes in Educational Technology, E. Popescu, Kinshuk, M. K. Khribi, R. Huang, M. Jemni, N.-S. Chen, and D. G. Sampson (eds). (Springer, 2017).

7. R. Palmarini, J. A. Erkoyuncu, R. Roy, and H. Torabmostaedi, “A systematic review of augmented reality applications in maintenance,” Robot. Comput.-Integr. Manuf. 49, 215–228 (2018). [CrossRef]  

8. X. Li, W. Yi, H.-L. Chi, X. Wang, and A. P. C. Chan, “A critical review of virtual and augmented reality (VR/AR) applications in construction safety,” Autom. Construct. 86, 150–162 (2018). [CrossRef]  

9. K. Akşit, W. Lopes, J. Kim, P. Shirley, and D. Luebke, “Near-eye varifocal augmented reality display using see-through screens,” ACM Trans. Graph. 36(6), 189 (2017). [CrossRef]  

10. A. Maimone, A. Georgiou, and J. S. Kollin, “Holographic near-eye displays for virtual and augmented reality,” ACM Trans. Graph. 36(4), 85 (2017). [CrossRef]  

11. B. Kress and T. Starner, “A review of head-mounted displays (HMD) technologies and applications for consumer electronics,” Proc. SPIE 8720, 87200A (2013). [CrossRef]  

12. C. Martinez, “Image projection Device,” US patent 2015/0370073 A1 (2015).

13. C. Martinez, V. Krotov, B. Meynard, and D. Fowler, “See-through holographic retinal projection display concept,” Optica 5(10), 1200–1209 (2018). [CrossRef]  

14. S. S. Hong, B. K. P. Horn, D. M. Freeman, and M. S. Mermelstein, “Lensless focusing with subwavelength resolution by direct synthesis of the angular spectrum,” Appl. Phys. Lett. 88(26), 261107 (2006). [CrossRef]  

15. P. S. Salvaggio, J. R. Schott, and D. M. McKeown, “Laboratory validation of a sparse aperture image quality model,” Proc. SPIE 9617, 961708 (2015).

16. A. Jiang, S. Wang, Z. Dong, J. Xue, J. Wang, and Y. Dai, “Wide-band white light sparse-aperture Fizeau imaging interferometer testbed for a distributed small-satellites constellation,” Appl. Opt. 57(11), 2736–2746 (2018). [CrossRef]   [PubMed]  

17. F. Eisenhauer, G. Perrin, W. Brandner, C. Straubmeier, K. Perraut, A. Amorim, M. Schöller, S. Gillessen, P. Kervella, M. Benisty, C. Araujo-Hauck, L. Jocou, J. Lima, G. Jakob, M. Haug, Y. Clénet, T. Henning, A. Eckart, J.-P. Berger, P. Garcia, R. Abuter, S. Kellner, T. Paumard, S. Hippler, S. Fischer, T. Moulin, J. Villate, G. Avila, A. Gräter, S. Lacour, A. Huber, M. Wiest, A. Nolot, P. Carvas, R. Dorn, O. Pfuhl, E. Gendron, S. Kendrew, S. Yazici, S. Anton, Y. Jung, M. Thiel, É. Choquet, R. Klein, P. Teixeira, P. Gitton, D. Moch, F. Vincent, N. Kudryavtseva, S. Ströbele, E. Sturm, P. Fédou, R. Lenzen, P. Jolley, C. Kister, V. Lapeyrère, V. Naranjo, C. Lucuix, R. Hofmann, F. Chapron, U. Neumann, L. Mehrgan, O. Hans, G. Rousset, J. Ramos, M. Suarez, R. Lederer, J.-M. Reess, R.-R. Rohloff, P. Haguenauer, H. Bartko, A. Sevin, K. Wagner, J.-L. Lizon, S. Rabien, C. Collin, G. Finger, R. Davies, D. Rouan, M. Wittkowski, K. Dodds-Eden, D. Ziegler, F. Cassaing, H. Bonnet, M. Casali, R. Genzel, and P. Lena, “GRAVITY: Observing the Universe in Motion,” Messenger (Los Angel.) 143, 16–24 (2011).

18. V. Krotov, C. Martinez, and O. Haeberlé, “Multiple beam diffractive setup for intraocular accommodation evaluation,” in Imaging and Applied Optics 2016 (2016), paper JT3A.32.

19. F. Templier, L. Dupré, B. Dupont, A. Daami, B. Aventurier, F. Henry, D. Sarrasin, S. R. Berger, F. Olivier, and L. Mathieu, “High-resolution active-matrix 10-μm pixel-pitch GaN LED microdisplays for augmented reality applications,” Proc. SPIE 10556, 105560I (2018).

20. C. Martinez, V. Krotov, and O. Haeberlé, “Experimental evaluation of self-focusing image formation in unconventional near-eye display,” Proc.SPIE 10676, 106760N (2018).

21. C. Martinez, B. Meynard, and Y. Lee, “Multiplexed pixelated hologram recording process for retinal projection device,” Proc. SPIE 10944, 109440N (2019).

Cited By

Optica participates in Crossref's Cited-By Linking service. Citing articles from Optica Publishing Group journals and other participating publishers are listed here.

Alert me when this article is cited.


Figures (10)

Fig. 1
Fig. 1 Self-focusing image formation: (a) concept of near to eye display where a distribution of spherical wavefronts is emitted and produces the illusion of a plane wavefront that the eye lens focuses on the retina; (b) our validation setup: a laser beam passes through a transmission device consisting of a distribution of pinholes in an opaque metallic layer.
Fig. 2
Fig. 2 Calculated distributions and their histograms of the closest distance between the nearest pinholes: (a) and (b) periodic distribution; (c) and (d) quasi-random distribution; (e) and (f) CS distribution; (g) and (h) CRS distribution.
Fig. 3
Fig. 3 Experimental set-up for self-focusing effect validation: (a) pictures of a sinusoidal periodic apertures distribution mask, (b) set-up description.
Fig. 4
Fig. 4 Diffraction patterns produced by various EPD, on the left the image on the image sensor (in green false color), on the right the intensity cross section compared with Gaussian function: (a) and (b) one pinhole; (c) and (d) periodic distribution; (e) and (f) quasi-random distribution; (g) and (h) CS distribution; (i) and j) CRS distribution. Figure (f), (h) and (j) are plotted in logarithmic intensity scale. Inset in Fig. (f) is a detail of the spel theory/measurement comparison in linear scale near 0 arcmin.
Fig. 5
Fig. 5 Double-Gaussian model for the spel. (a), (b) and (c) spel intensity distributions for three LD power (d) cross intensity function of the concatenated images (blue dotted curve) and Gaussian model for the peak and the noise (orange curves).
Fig. 6
Fig. 6 (a) Image of the letter “T” coded with 5x5 pixels, (b) self-focusing imaging with one spel per pixel and (c) self-focusing imaging with four spels per pixel.
Fig. 7
Fig. 7 Optical set-up used to evaluate self-focusing imaging.
Fig. 8
Fig. 8 MicroLED display used to generate the image wavefront. On the left the display size compared with 1 eurocent; on the right the magnification of the display with an isolated activated pixel as well as the “LETI” acronym.
Fig. 9
Fig. 9 Results on the experiment (a) diffraction limited, (b) periodic distribution, (c) quasi-random EPD and (d) CRS EPD.
Fig. 10
Fig. 10 Results on the imaging simulation process (a) diffraction limited, (b) periodic distribution, (c) quasi-random EPD, (d) CRS EPD and (e) comparison of the middle cross section of the measured and simulated images in the quasi-random EPD.

Equations (27)

Equations on this page are rendered with MathJax. Learn more.

N= n elect n laser
Λ 1 = n elect d e Λ 2 = n laser d g
x i,j = Λ 1 i y i,j = Λ 2 j
x i,j = Λ 1 ( i+rnd ) y i,j = Λ 2 ( j+rnd )
( x i,j , y i,j )= f e ( x+ Λ 1 i,y ) f g ( x,y+ Λ 2 j )
( x i,j , y i,j )= f e ( x+ Λ 1 ( i+rnd ),y ) f g ( x,y+ Λ 2 ( j+rnd ) )
f: r ( t+δt )= r ( t )+ V ( t )δt
V ( t )= [ v 0 ( t )cos( α( t ) ) v 0 ( t )sin( α( t ) )
v 0 ( t )= v 0 =const. α( t )= α 0 cos( 2π t 0 t )
s p =min ( | u p,q | ) qEPD
h q = p=1 N EPD [ 0<( s p q 2Λ N h ) 2Λ N h ]
Δθ= λ π w 0
δθ=1.22 λ D p
I i ( r )= I 0i e 2 ( r w i ) 2 i=1,2
E i =2π I 0i w i 2 i=1,2
w 1 0.7δθ f 0 =0.86 λ f 0 D p
w 2 =2 λ f 0 π d p
γ=10log( E 1 E 1 + E 2 )=10log( I 01 w 1 2 I 01 w 1 2 + I 02 w 2 2 )
I ret ( r )= u,v M u,v g( r r u,v )
I ˜ ret ( r )= u,v M u,v rec t W ( s r u,v ) PSF( r s )d x s d y s
k u,v = [ sin( ρ u,v )cos( θ u,v ) sin( ρ u,v )sin( θ u,v ) cos( ρ u,v )
ρ u,v = δ f c u 2 + v 2 θ u,v = cos 1 ( u u 2 + v 2 )
r u,v = [ f 0 ρ u,v cos( θ u,v ) f 0 ρ u,v sin( θ u,v )
g per ( r )= G 0 e 2 r 2 w 2 2 k,l e 2 r r k,l 2 w 1 2
r k,l = [ k λ f 0 Λ l λ f 0 Λ
g qr ( r )= I 1 ( r )+ I 2 ( r )
g CRS ( r )= I 1 ( r )+ I 2 ( r )+ H 0 e 2 r 2 w 3 2 q,p e 2 r r q,p 2 w 1 2
Select as filters


Select Topics Cancel
© Copyright 2024 | Optica Publishing Group. All rights reserved, including rights for text and data mining and training of artificial technologies or similar technologies.