1. Introduction

Computer generated hologram (CGH) techniques synthesize complex field of three-dimensional (3D) scene [1]. The 3D scene is represented in various forms and the complex field for the scene is synthesized by calculating and accumulating the complex field of individual primitives of the scene. The possible forms representing the 3D scene for the CGH synthesis include point cloud [2–4], multiple layers [5], polygon meshes [6–8], and light field [9–10]. The light field is a collection of numerous views of the scene which are captured at slightly different directions. As the light field can be readily rendered for a virtual scene using computer graphics or directly captured for a real scene using light field camera [11–12], the light field could be universal representation of the virtual and real scene for the CGH synthesis. The detailed scene appearance like view dependent occlusion and material reflection property is already included in the light field, which alleviates complicated calculation in the hologram synthesis step for its realization [13–14].

One limitation of the light field representation is that it does not provide explicit depth information of individual 3D points or surfaces of the scene. Instead, the depth information is only implicitly included as a form of slight difference, or disparity, between the views. Conventional light field based CGH techniques which are usually called holographic stereogram divide the hologram into many non-overlapping segments called hogels in spatial or Fourier domain [9,15]. They then calculate the complex field in each segment exclusively from the corresponding single view in the light field data. Each hologram segment simply reconstructs the corresponding view without proper depth reconstruction nor global phase relationship across multiple hologram segments. Therefore, those conventional light field based CGHs only reconstruct light rays with random phase relationship without reconstructing ideal continuous parabolic wavefront for individual 3D object point. Although advanced techniques which match the phase relationship between the non-overlapping hogels [16], encode a 3D view with the proper depth distribution to each hogel [17–19], or overlap the hogels to overcome the spatio-angular resolution tradeoff [14] have been reported, all of them require the explicit depth map information in addition to the original light field data.

Recently, a novel non-hogel-based CGH technique has been proposed [20]. Instead of segmenting the hologram and assigning the corresponding view, this technique globally processes the light field data, or processes all the views collectively to synthesize the entire hologram without requiring the depth map. The depth information implicitly encoded in the difference between the views is automatically reflected in the synthesized hologram, creating ideal continuous parabolic wavefront for each object point. The continuous wavefront realizes high reconstruction resolution only limited by the numerical aperture (NA) of the hologram. The carrier wave, or equivalently the phase distribution on the surface of the reconstructed 3D objects can also be controlled arbitrarily, making this technique versatile for various applications. It was shown that this technique can be considered as the application of the complex field recovery technique from its Wigner distribution function (WDF) to the light field data [20].

Despite the benefits of the non-hogel-based technique, its high computational load is problematic. Unlike the conventional methods where each pixel in the hologram spatial domain or Fourier domain is only affected by the single corresponding view, the non-hogel-based technique calculates each hologram pixel value from all light field data, i.e. all image pixels in all views within the NA range. Therefore, the computational time and the memory requirement are much higher than conventional methods, which prevent generation of high pixel resolution holograms. Although an enhanced implementation using two-dimensional (2D) fast Fourier transform (FFT) over projection angle axes has been reported [20], significant further improvement is required.

In this paper, we propose an efficient calculation scheme for the non-hogel-based CGH technique. The proposed technique considers that the spatial resolution of the 3D scene can be lower than the NA limit of the hologram, which is reasonable in practical applications where holograms are used to show 3D images to users. For a light field data prepared at the spatial sampling rate corresponding to the spatial resolution of the 3D scene, the proposed technique performs 2D FFT over its projection angle axes. Then the individual 2D angular frequency slice is interpolated to the hologram resolution, multiplied with a complex weighting function, and finally accumulated in the hologram plane. This angular frequency slice processing is independent for each slice, enabling concurrent parallel processing for multiple slices. The hologram tiling scheme is also proposed to make the process scalable to arbitrary resolution, not being limited to physical memory of the calculation system. Using the angular-frequency-slice-based processing and the hologram tiling, the 25K×25 K pixel resolution hologram has been synthesized successfully based on the non-hogel-based CGH technique. In the followings, we explain the principle of the proposed method after brief review on the non-hogel-based CGH. Then we present the verification results of the speed enhancement and the artefact-free hologram synthesis by the proposed angular-frequency-slice-based processing and the hologram tiling. Finally, two high pixel resolution holograms synthesized using the proposed method are presented with demonstrations of the extended depth range and the realistic reproduction of the parallax.

2. Principle of the proposed technique

2.1 Brief review of non-hogel-based CGH

The non-hogel-based CGH technique synthesizes complex field from the light field data of the 3D scene. The light field data is a spatio-angular distribution of the light ray radiance. In a plane, the light field can be represented by L(t_x, t_y, u, v) where (t_x, t_y) is the spatial position of the light ray and (u, v) represents its angular direction. In this paper, we represent the angular direction of the light ray as the corresponding spatial frequency (u, v)=(sinθ_x/λ, sinθ_y/λ) where λ is the wavelength and (θ_x, θ_y) is the ray angle with respect to the z axis. The light field data can also be considered as the collection of the orthographic view images I^ortho_uo,vo(t_x,t_y) = L(t_x, t_y, u_o, v_o) at different observation directions (u_o, v_o) or the collection of the perspective view images I^persp_txo,tyo(u,v) = L(t_xo, t_yo, u, v) at different observation positions (t_xo, t_yo).

From the given light field data L(t_x, t_y, u, v), the non-hogel-based CGH technique synthesizes the complex field by [20]

Although the non-hogel-based CGH technique can generate coherent complex field with arbitrary carrier waves, its implementation requires large computational load. Given the target hologram resolution N_x×N_y, the light field L(t_x, t_y, u, v) should be prepared with the same spatial resolution N_tx×N_ty=N_x×N_y. The prepared light field L(t_x, t_y, u, v) is first 2D Fourier transformed along the (u, v) axes at each (t_x, t_y) position to obtain $\tilde{L}({t_x},{t_y},{\tau _x},{\tau _y})$ which requires N_tx×N_ty=N_x×N_y times fast Fourier transform (FFT) operations of N_u×N_v size. The calculated $\tilde{L}({t_x},{t_y},{\tau _x},{\tau _y})$ is then processed according to Eq. (1) which requires extractions of different slice of $\tilde{L}({t_x},{t_y},{\tau _x},{\tau _y})$ for the corresponding (x_c, y_c) and their accumulations. Since the (x_c, y_c) is the spatial position in the hologram plane defined on the same sampling grid as the hologram itself, the slice extraction and the addition should be performed N_x×N_y times. Therefore, the overall computation of the non-hogel-based CGH technique requires N_x×N_y times FFTs and N_x×N_y times slice extractions and additions, which is huge and non-tractable especially when the hologram resolution N_x×N_y is large.

2.2 Angular-frequency-slice-based processing

Figure 1 is the conceptual diagram of the proposed angular-frequency-slice-based processing. In the proposed method, the light field L(t_x, t_y, u, v) is prepared with a spatial resolution N_tx×N_ty which is sufficient for the 3D scene bandwidth but can be smaller than the hologram resolution N_x×N_y. The light field is then 2D Fourier transformed along the (u, v) axes at each (t_x, t_y) position to obtain $\tilde{L}({t_x},{t_y},{\tau _x},{\tau _y})$ by N_tx×N_ty times FFT operations. Next, at each angular frequency (τ_x, τ_y), the 2D slice ${\tilde{I}_{{\tau _x},{\tau _y}}}({{t_x},{t_y}} )= \tilde{L}({t_x},{t_y},{\tau _x},{\tau _y})$ of the resolution N_tx×N_ty is interpolated to the hologram resolution N_x×N_y, multiplied with the complex weight function W(x_c, y_c), and finally accumulated on the hologram plane. For the multiplication and the accumulation for each angular frequency (τ_x, τ_y), the integration variables x_c and y_c in Eq. (1) are modified to τ_x=x-x_c, τ_y=y-y_c, giving

 

Fig. 1. Angular frequency slice processing.

Download Full Size | PDF

Note that once $\tilde{L}({t_x},{t_y},{\tau _x},{\tau _y})$ is obtained by taking N_tx×N_ty times 2D FFTs over (u, v) axes to the given light field L(t_x, t_y, u, v), all following procedures including the spatial interpolation to N_x×N_y resolution, multiplication with the complex weight function W, and the accumulation in the hologram plane are performed for each angular frequency slice ${\tilde{I}_{{\tau _x},{\tau _y}}}({{t_x},{t_y}} )= \tilde{L}({t_x},{t_y},{\tau _x},{\tau _y})$, independently. Therefore, the proposed technique allows the parallel processing, which could contribute to the calculation time reduction.

Also note that the proposed technique performs the spatial interpolation along (t_x, t_y) axes not for the light field L(t_x, t_y, u, v) itself but for the $\tilde{L}({t_x},{t_y},{\tau _x},{\tau _y})$ after the 2D FFT operations over (u, v) axes. In the original technique, the light field L(t_x, t_y, u, v) should be prepared in the same spatial resolution as the final hologram. If the initial light field is given with a low resolution N_tx×N_ty (< N_x×N_y), it should be first interpolated to the hologram resolution N_x×N_y, then it is 2D Fourier transformed, which requires N_x×N_y times 2D FFT operations. However, in the proposed method, the required number of the 2D FFT operations is kept low, i.e. N_tx×N_ty, by performing the interpolation over (t_x, t_y) axes after the 2D Fourier transform over (u, v) axes. The equivalence between the spatial interpolation over (t_x, t_y) axes before and after the 2D Fourier transform over (u, v) axes can be shown easily. Representing the spatial interpolation as the linear weighted addition of the neighborhood, the spatially interpolated value of the light field at (t_xi, t_yi) can be written by

Finally, it is noted that, for a given light field data, the synthesized hologram and its reconstruction are dependent on the spatial interpolation method used in the proposed angular-frequency-slice-based processing technique. The maximum effective spatial resolution of the reconstruction is, however, still given by the spatial resolution of the original light field before the spatial interpolation. An extensive analysis of the effect of the interpolation method on the wavefront reconstructed from the synthesized hologram and the search of the optimum interpolation method are left for the further research. All demonstrations presented in this paper were obtained using a simple two-dimensional linear interpolation method in our implementation.

2.3 Hologram tiling

The proposed-angular-frequency-slice-based processing technique also allows the tiled processing of the hologram. Figure 2 shows the area in the hologram plane which is affected by a single tile of the light field data. Suppose that the light field tile L(t_x, t_y, u, v) is given for a spatial range (t_x,min, t_y,min) < (t_x, t_y) < (t_x,max, t_y,max) with a sampling interval Δt_x, Δt_y and for a directional range (u_min, v_min) < (u, v) < (u_max, v_max) with a sampling interval Δu and Δv. Then its Fourier transform over u, v axes $\tilde{L}({t_x},{t_y},{\tau _x},{\tau _y})$ has an angular frequency range (-1/(2Δu), -1/(2Δv)) < (τ_x, τ_y) < (1/(2Δu), 1/(2Δv)). Equation (4) indicates that this light field data contributes to the hologram H(x, y) in a limited spatial range (t_x,min- 1/(4Δu), t_y,min- 1/(4Δv)) < (x, y) < (t_x,max+ 1/(4Δu), t_y,max+ 1/(4Δv)) in the hologram plane, as illustrated in Fig. 2.

 

Fig. 2. Spatial extent of the hologram corresponding to a given light field.

Download Full Size | PDF

Since the spatial range (x,y) of the hologram affected by a light field data is limited around the spatial range (t_x, t_y) of the light field, the tiled processing is possible. Figure 3 show the concept. The whole spatial range (t_x, t_y) is divided into non-overlapping spatial tiles in the preparation of the light field data. In the hologram synthesis, each spatial tile of the light field data is processed to produce the corresponding hologram tile around its spatial range. The produced hologram tiles are accumulated with an overlapping area, populating the whole hologram area as shown in Fig. 3. The spatial range of each light field tile can be determined considering the computational capacity of the processing unit. Since the processing of each light field tile is independent, the proposed tiled processing enables the synthesis of hologram with arbitrary pixel resolution over the limit of a single processing unit.

 

Fig. 3. Hologram tiling process.

Download Full Size | PDF

2.4 Light field sampling and the depth range of the hologram

One advantage of the non-hogel-based CGH technique over the conventional hogel based techniques is that the 3D scene reconstructed by the non-hogel-based CGH has higher spatial resolution only limited by the NA of the hologram. This high resolution is enabled by the feature that the non-hogel-based CGH is free from the tradeoff between the spatial resolution and the angular resolution. In this paper, however, it is considered that the 3D scene can have lower spatial resolution, i.e. lower than the NA limit of the hologram but still plausible to the human visual system. In this case, the tradeoff-free feature of the non-hogel-based CGH enables more angular samplings of the light field data at the same spatial resolution, producing longer depth range of the reconstruction than the conventional hogel based techniques.

Suppose that a hologram H(x,y) has N_x×N_y pixel resolution with a pixel pitch Δx and Δy. In the proposed technique, the light field L(t_x,t_y,u,v) is assumed to be spatially sampled with the sampling interval Δt_x=M_xΔx and Δt_y=M_yΔy which corresponds to the spatial bandwidth of the 3D scene B_x=1/Δt_x, and B_y=1/Δt_y. The angular range of the light field is determined to fill the whole viewing angle or diffraction range of the hologram. With the hologram pixel pitch Δx and Δy, the viewing angle of the hologram reconstruction is |θ_x|<sin^-1(λ/2Δx) and |θ_y|<sin^-1(λ/2Δy), which gives the angular range of the light field |u|=|sinθ_x/λ|<1/(2Δx) and |v|=|sinθ_y/λ|<1/(2Δy). If the number of the light field samplings in the angular u, v directions is given by N_u and N_v, then the angular sampling pitch is determined to be

In the conventional hogel based techniques, the N_u×N_v hologram pixels consist of a single hogel. Since each hogel acts as a single 3D pixel of the reconstruction, the physical hogel size N_uΔx × N_vΔy should be the same as the spatial sampling interval of the light field Δt_x=M_xΔx and Δt_y=M_yΔy. Therefore, in the conventional hogel based techniques, the number of the angular sampling N_u and N_v is limited to be N_u=M_x and N_v=M_y. From Eq. (9), the corresponding maximum depth of the conventional hogel based techniques is given by

Note that increasing N_u and N_v requires rendering more orthographic view images with finer angular sampling pitch Δu and Δv, which increases computational load and time in the preparation of the light field data. Once the light field data is prepared with N_u and N_v larger than M_x and M_y, however, the non-hogel-based CGH utilizes all data, increasing the depth range, while the conventional hogel based CGH techniques should down-sample it to M_x and M_y to fit the hogel size, limiting the depth range.

Also note that even though Eq. (9) deduced from the light field sampling theory shows linear increase of the maximum depth proportional to the number of the angular samples N_u and N_v, there is an upper limit of the depth range imposed by the NA of the hologram itself. For a spatial transverse spot size Δt_x=M_xΔx and Δt_y=M_yΔy, the required NA is about λ/(2Δt_x)=λ/(2M_xΔx) or λ/(2Δt_y)=λ/(2M_yΔy) [21]. With the physical size of the hologram N_xΔx × N_yΔy, the maximum depth corresponding to this NA is |z|≤N_xΔx/(2NA), |z|≤N_yΔy/(2NA), or |z|≤min(N_xM_x²Δx²/λ, N_yM_y²Δy²/λ). Comparison with Eq. (9) shows that the upper limit imposed by the hologram is larger than the light field sampling limit of Eq. (9) by the ratio of N_x/N_u or N_y/N_v, which is huge in most cases. Therefore, in most practical cases, the depth range of the reconstruction increases linearly with the number of the angular samplings N_u and N_v as given by Eq. (9).

3. CGH synthesis and verification

The verification results of the speed enhancement using the proposed angular-frequency-slice-based processing and the high resolution hologram synthesis using the proposed hologram tiling are presented in this section. For the speed enhancement verification, a light field data which consists of N_u×N_v=40 × 40 orthographic views with N_tx×N_ty=300 × 200 pixel resolution was prepared as shown in Fig. 4(a). Holograms were synthesized from this light field using 5 different values of M_x=M_y=M=1, 2, 4, 8, and 16, which give the final hologram pixel resolution of N_x×N_y=MN_tx×MN_ty=300 × 200, 600 × 400, 1200 × 800, 2400 × 1600, and 4800 × 3200, respectively. In this speed verification, the hologram tiling was not applied, and each hologram was synthesized as a whole. The conventional processing scheme of the non-hogel-based CGH following Eq. (1) and the proposed angular-frequency-slice-based processing scheme were implemented using MATLAB environment running on a PC having Intel i7-8700 K CPU, 32GB memory, and GeForce GTX 1080 TI GPU.

 

Fig. 4. Light field and synthesized holograms with different interpolation factor M. (a) Light field data used for the hologram synthesis, (b) synthesized holograms (top two rows) and their numerical reconstructions (bottom two rows).

Download Full Size | PDF

In the conventional processing scheme, the light field L(t_x, t_y, u, v) of the N_tx×N_ty×N_u×N_v = 300 × 200 × 40 × 40 data size was first spatially interpolated at each (u, v) to MN_tx×MN_ty×N_u×N_v data size, then Fourier transformed along the u, v axes at each interpolated (t_x, t_y) to obtain $\tilde{L}({t_x},{t_y},{\tau _x},{\tau _y})$ of MN_tx×MN_ty×N_u×N_v data size, and its various slices were finally extracted and accumulated following Eq. (1) with the complex weights W. In the proposed angular-frequency-slice-based processing scheme, the light field L(t_x, t_y, u, v) of the N_tx×N_ty×N_u×N_v = 300 × 200 × 40 × 40 data size was first Fourier transformed along the u, v axes at each (t_x, t_y) to obtain $\tilde{L}({t_x},{t_y},{\tau _x},{\tau _y})$ of the same N_tx×N_ty×N_u×N_v size, then each angular frequency slice ${\tilde{I}_{{\tau _x},{\tau _y}}}({{t_x},{t_y}} )= \tilde{L}({t_x},{t_y},{\tau _x},{\tau _y})$ of the N_tx×N_ty resolution was interpolated to MN_tx×MN_ty resolution, and accumulated on the hologram plane with the complex weight distribution W following Eq. (4) and Fig. 1. In both conventional and proposed processing implementations, a random complex field of MN_tx×MN_ty resolution was used as the complex weight distribution W(x_c, y_c). Figure 4(b) shows the synthesized holograms and their numerical reconstructions. Note that the apparent quality enhancement observed in the numerical reconstructions of the higher pixel resolution holograms, i.e. larger M, is not due to the inherent resolution enhancement of the light field data but due to the smaller speckle grain size relative to the reconstruction window of MN_tx×MN_ty.

Figure 5 and Table 1 show the speed measurement result. Note that the ‘LF load’ in Table 1 only means the time to read the prepared N_u×N_v=40 × 40 orthographic view image files of the N_tx×N_ty=300 × 200 pixel resolution, not including their rendering time. Also, in our implementation of the conventional processing scheme, each orthographic view image of the N_tx×N_ty=300 × 200 pixel resolution is spatially interpolated to MN_tx×MN_ty=(300×M)×(200×M) right after read from the file to populate the corresponding slice of the interpolated light field data array. Thus the first step of the conventional processing scheme denoted by ‘LF load & interpolation’ in Table 1 means the time to create the interpolated light field data of MN_tx×MN_ty×N_u×N_v=(300×M)×(200×M) × 40 × 40 size in memory from the N_u×N_v image files of the N_tx×N_ty=300 × 200 pixel resolution in the PC disk. To the contrary, the first step of the proposed angular-frequency-slice-based processing scheme denoted by ‘LF load’ is just to read N_u×N_v image files of the N_tx×N_ty=300 × 200 pixel resolution from the PC disk without interpolation to load the light field data of N_tx×N_ty×N_u×N_v=300 × 200 × 40 × 40 size in memory. The interpolation is conducted in the last step after its Fourier transform. The last step of the proposed angular frequency slice processing scheme denoted by ‘Interpolation & weighted superposition’ in Table 1 includes the time to interpolate every angular frequency slice ${\tilde{I}_{{\tau _x},{\tau _y}}}({{t_x},{t_y}} )= \tilde{L}({t_x},{t_y},{\tau _x},{\tau _y})$ of the Fourier transformed light field data, multiply it with the shifted complex weight function W(t_x-τ_x/2, t_y-τ_y/2), and accumulate in the hologram plane with the corresponding shift following Eq. (4) to complete the final hologram H(x,y) as illustrated in Fig. 1.

 

Fig. 5. Measured computation time for three implementations of the non-hogel-based CGH technique, i.e. conventional direct implementation of Eq. (1) using CPU, proposed implementation based on the angular frequency slice processing using CPU, and the proposed implementation based on the same algorithm using GPU.

Download Full Size | PDF

Table 1. Computation time comparison.

View Table

In the conventional processing scheme, the spatial interpolation is first applied to the whole light field data, increasing the data size. Then the Fourier transform and the weighted accumulation are performed on the interpolated light field data of the large size. As shown in Table 1, this makes the processing time of all steps including the light field interpolation, the Fourier transform, and the weighted accumulation increases with the interpolation ratio M. Also, the memory requirement increases with the interpolation ratio M because the whole interpolated light field data is maintained in the memory. In our MATLAB implementation which uses 16 bytes for each complex number, the minimum memory requirement is about MN_tx×MN_ty×N_u×N_v×16=(300×M)×(200×M) × 40 × 40 × 16 bytes to store the complex Fourier transformed and interpolated light field data $\tilde{L}({t_x},{t_y},{\tau _x},{\tau _y})$ in the memory, which measures 1.43GB, 5.72GB, 22.89GB, 91.55GB, and 366.21GB for M=1, 2, 4, 8, and 16, respectively. With the 32GB physical memory of the PC used in the implementation, the maximum interpolation factor M was limited to 4.

In the proposed angular-frequency-slice-based processing, however, the original light field data before the spatial interpolation is maintained in the memory. The spatial interpolation and the weighted accumulation are performed not for the whole light field but only for a selected angular frequency slice at a time. The processed angular frequency slice can then be removed in the memory. This feature not only enhances the processing speed at the same CPU environment as detailed in Table 1, but also lowers the memory requirement significantly. In our MATLAB implementation, the minimum memory requirement is maintained to be about N_tx×N_ty×N_u×N_v×16 = 300 × 200 × 40 × 40 × 16 = 1.43GB to store the complex Fourier transformed light field data in the memory regardless of the interpolation factors M=1, 2, 4, 8, and 16. Moreover, the proposed angular-frequency-slice-based processing enables independent and concurrent processing of each angular frequency slice, making it suitable for the GPU processing. Table 1 shows significant speed enhancement of the proposed processing scheme with the GPU implementation.

Figures 6 and 7 show the verification results of the proposed hologram tiling for the scalable processing. For the verification, three light field data were used including N_tx×N_ty×N_u×N_v = 100 × 100 × 64 × 64 resolution light field of a single pixel point object as shown in Fig. 6(a), the same resolution light field of a 8 × 8 pixel size square object as shown in Fig. 6(b), and N_tx×N_ty×N_u×N_v = 300 × 200 × 40 × 40 resolution light field of two socket object shown in Fig. 6(c). Each light field data was spatially divided into 2 × 2 tiles, i.e. L_m,n(t_x, t_y, u, v) with m, n = 1, 2, and the corresponding hologram tiles H_m,n(x,y) were synthesized and combined as explained in section 2.3. In the hologram synthesis of all light field data, the spatial interpolation of the light field was not applied, giving M_x=M_y=M=1 or Δx=Δy=Δt_x=Δt_y=4µm. As the complex weight distribution W(x_c, y_c), a constant distribution W(x_c, y_c) = 1 for the light field of Figs. 6(a) and 6(b), and a parabolic phase distribution W(x_c, y_c) = exp[j{π/(λf)}(x_c²+y_c²)] with λ=532 nm and f=3 cm for the light field of Fig. 6(c) were used instead of the random complex field in order to verify the phase consistency more easily.

 

Fig. 6. Light field data used for the verification of the proposed hologram tiling algorithm. (a) Single pixel point object, (b) 8 × 8 pixel width square object, and (c) two sockets at different depths.

Download Full Size | PDF

 

Fig. 7. Hologram tiles generated from corresponding light field tiles (left), the composite hologram generated by combining the hologram tiles (center), and the original hologram generated from whole light field without tiling process (right). Holograms are generated from (a) light field of single pixel point object in Fig. 6(a), 6(b) light field of 8 × 8 pixel square object in Fig. 6(b), and 6(c) light field of two socket object in Fig. 6(c).

Download Full Size | PDF

Figure 7 shows the synthesized holograms. In Figs. 7(a)–7(c), the left column shows the amplitude and phase of individual hologram tiles synthesized from the corresponding light field tiles shown in Fig. 6. The center column shows the hologram combined from those hologram tiles and the right column is the hologram directly synthesized from the whole light field without the tiled processing. Note that the hologram tiles in the left column are not simply stitched side by side but are combined with overlap area as explained in section 2.3. From Figs. 2 and 3, the overlap area is given by ±1/4Δu and ±1/4Δv around the original light field area, which corresponds to ±N_u/4 and ±N_v/4 hologram pixels by Eq. (7). In Fig. 7, the original light field area and the overlap area are indicated by a green rectangle and orange dashed lines in the left column. From the comparison between the holograms synthesized by the tiling process shown in the center column and the holograms synthesized directly from the whole light field data shown in the right column, it can be verified that the proposed hologram tiling process generates the original hologram without any error, successfully.

Using the proposed angular-frequency-slice-based processing and the hologram tiling, we demonstrate the synthesis of large pixel resolution holograms. Figure 8 shows the light field data used for a 25K×25 K pixel resolution hologram synthesis. In this hologram synthesis and the numerical reconstructions, not only the large pixel resolution hologram synthesis but also the enhanced depth range of the non-hogel-based CGH explained in section 2.4 is also demonstrated. The light field has N_tx×N_ty×N_u×N_v = 1563 × 1563 × 64 × 64 data size with a spatial sampling pitch of 64µm, i.e. Δt_x=Δt_y=64µm. In the hologram synthesis, the spatial interpolation by a factor of M=16 was performed using the proposed angular-frequency-slice-based processing scheme, resulting in (1563 × 16)×(1563 × 16) = 25K×25 K hologram pixel resolution with the hologram pixel pitch Δx=Δy=4µm (=Δt_x/M=Δt_y/M). The angular range of the light field was given by the viewing angle of the hologram, i.e. |θ_x|=|θ_y|=λ/2Δx=λ/2Δy=3.81° at λ=532 nm or |u|=|v|=1/2Δx=1/2Δy=1.25 × 10⁵m^-1. With N_u=N_v=64, the angular sampling pitch of the light field was determined to be Δθ_x=Δθ_y=0.12° or Δu=Δv=3.9 × 10³m^-1. The full light field data of the N_tx×N_ty×N_u×N_v = 1563 × 1563 × 64 × 64 data size was divided into 9 × 9 tiles of the N_tx×N_ty×N_u×N_v = 180 × 180 × 64 × 64 data size (note that the N_tx of the light field tiles in the rightmost column and the N_ty of the light field tiles in the bottom row are smaller to be 1563-180 × 8 = 123, not 180), and the final hologram was obtained by combining the 9 × 9 hologram tiles of (180×M)×(180×M) = 2880 × 2880 resolution synthesized from the corresponding light field tiles with M=16. A random complex field was used as the complex weight distribution W(x_c, y_c).

 

Fig. 8. Light field used for the 25K×25 K resolution hologram and its extended depth range verification. (a) Single orthographic view in the light field, (b) collection of 64 × 64 orthographic views.

Download Full Size | PDF

For the depth range comparison, a holographic stereogram was also synthesized from the same light field. The full spatial resolution of the light field N_tx×N_ty=1563 × 1563 was still used without change in the holographic stereogram synthesis. The number of the angular samplings N_u×N_v, however, was reduced from 64 × 64 to 16 × 16 by down-sampling, in order to make hogels of 16 × 16 size, giving the holographic stereogram of the same pixel resolution (1563 × 16)×(1563 × 16) = 25K×25 K as the hologram synthesized by the non-hogel-based CGH. In the synthesis of the holographic stereogram, 16 × 16 pixel resolution perspective view at each spatial position (t_xo, t_yo), i.e. I^persp_txo,tyo(u,v)=L(t_xo,t_yo,u,v) was multiplied with a random phase distribution of the same resolution and Fourier transformed to generate the corresponding hogel.

With these parameters of the non-hogel-based CGH and the holographic stereogram, the maximum depths that can be reconstructed without aliasing are given by Eqs. (9) and (10) to be |z|=30.8 mm for the non-hogel-based CGH and |z|=7.7 mm for the holographic stereogram. Note that the non-hogel-based CGH is expected to have 4 times larger depth range than the holographic stereogram.

For the clear demonstration of the enhanced depth range, the 3D scene that consists of 16 identical resolution target images located at different depths as shown in Fig. 8(a) was used in the light field generation. The depths of the 16 resolution targets are distributed from -4z_o to +4z_o spanning the depth range of the non-hogel-based CGH where z_o is the maximum depth of the holographic stereogram, i.e. z_o=7.7 mm. Figure 8(b) shows some orthographic views in the whole light field data.

Figure 9 shows the amplitudes of the synthesized holographic stereogram and the non-hogel-based hologram. Figure 10 and Visualizations 1 and 2 shows their numerical reconstructions at various depths. For the numerical reconstructions, the synthesized 25K×25 K resolution holograms are divided into overlapping tiles and the band-limited angular spectrum based numerical propagation algorithm [22] was applied to each tile. The tile boundaries for the numerical reconstructions were selected not to coincide with the tile boundaries used in the hologram synthesis. In Fig. 10, the focused reconstructions of the corresponding resolution target image at depths z=1.0z_o, 2.5z_o, and 4.0z_o are magnified to compare the resolution. From the comparison between Figs. 10(a) and 10(b), it is observed that the reconstructions of the non-hogel-based CGH maintain the original spatial resolution of the light field for the entire depth range |z|≤4z_o=30.8 mm, while the holographic stereogram preserves the original resolution only within |z|≤z_o=7.7 mm as expected. Therefore, the extended depth range advantage of the non-hogel-based CGH explained in section 2.4 can be confirmed.

 

Fig. 9. Amplitude of generated holograms from the light field in Fig. 8. (a) Holographic stereogram, (b) hologram synthesize by the non-hogel-based CGH.

Download Full Size | PDF

 

Fig. 10. Numerical reconstructions for different depths of the holograms in Fig. 9. (a) Holographic stereogram (Visualization 1), (b) hologram synthesize by the non-hogel-based CGH (Visualization 2).

Download Full Size | PDF

Figure 11 shows another light field data for the large pixel resolution hologram synthesis. A chessboard scene spanning the depth range -4z_o≤z≤4z_o, or -30.8mm ≤ z≤30.8 mm was rendered at various orthographic projection angles using a ray tracing software POV-RAY, giving a light field data of N_tx×N_ty×N_u×N_v = 1563 × 1081 × 64 × 64 data size with Δt_x=Δt_y=64µm spatial sampling pitch. The non-hogel-based hologram was synthesized from this light field using the spatial interpolation M=16, giving the hologram of (N_tx×M)×(N_ty×M) = 25K×17 K resolution with Δx=Δy=4µm (=Δt_x/M=Δt_y/M) pixel pitch. The angular range and the angular sampling pitch of the light field data were determined by the hologram viewing angle and the number of the orthographic views to be |u|=|v|=1/2Δx=1/2Δy=1.25 × 10⁵m^-1, and Δu=Δv=2|u|/N_u=2|v|/N_v=3.9 × 10³m^-1, respectively. In the hologram synthesis, this light field was spatially divided into 9 × 7 tiles of N_tx×N_ty×N_u×N_v = 180 × 180 × 64 × 64 data size and processed independently. A random complex distribution was used as the complex weight W(x_c, y_c) in the hologram synthesis. Figure 12 is the amplitude of the synthesized hologram.

 

Fig. 11. Light field used for 25K×17 K resolution hologram synthesis. (a) Single orthographic view in the light field, (b) collection of 64 × 64 orthographic views.

Download Full Size | PDF

 

Fig. 12. Amplitude of generated holograms from the light field in Fig. 11.

Download Full Size | PDF

Figure 13 and Visualization 3 show the numerical reconstructions at various depths of the synthesized hologram. From Fig. 13, it can be confirmed that the numerical reconstructions of the hologram synthesized by the proposed technique shows high resolution reconstructions as high as the spatial resolution of the original light field within the extended depth range -4.0z_o≤z≤4.0z_o with correct focusing effects. It is also interesting to observe in the magnified red rectangle portion that, the chess pawn (green arrow) and its reflection (yellow arrow) from the chess board are focused at different depths following their physical depths. Moreover, the chess board itself (blue arrow) and the reflection of the chess pawn (yellow arrow) are focused independently even though they are super-imposed in the same spatial position in the view. Note that this case cannot be properly dealt with previously reported techniques which enhance the reconstruction resolution by matching the phase between the hogels using the depth map, because the scene depth cannot be determined uniquely in this super-imposed area. The non-hogel-based CGH of this paper generates coherent parabolic wavefront only from the light field without requiring the depth map, which enables high resolution reconstruction with individual focusing even in this super-imposed area.

 

Fig. 13. Numerical reconstructions for different depths of the hologram in Fig. 12. (Visualization 3)

Download Full Size | PDF

Finally, several perspective views were extracted from the synthesized hologram to verify the parallax reconstruction. Figure 14 shows the geometry used in the perspective view extraction. The hologram has 25K×17 K resolution with Δx=Δy=4µm pixel pitch, which measures 10cm×6.9 cm physical size. The diffraction angle is |θ_x|=|θ_y|=λ/2Δx=λ/2Δy=3.81°. The perspective views were extracted by scanning a virtual camera within the diffraction angle. More specifically, the virtual camera of a 2 cm diameter aperture was placed at 150 cm distance from the hologram as shown in Fig. 14. The transverse position of the camera was scanned from -5 cm to +5 cm both in horizontal and vertical directions. The camera lens focal length and the lens to image plane distance were adjusted to focus on the hologram plane. For the perspective view extraction at the given camera location, the hologram field is numerically propagated to the camera lens plane, clipped by the lens aperture, multiplied by the lens function, and again numerically propagated to the image plane.

 

Fig. 14. Geometry used in the perspective view extraction from the hologram in Fig. 12.

Download Full Size | PDF

The extracted perspective views are shown in Fig. 15 and Visualization 4. Unlike the numerical reconstructions shown in Fig. 13 which use the full hologram NA, the 2 cm aperture of the virtual camera located at 150 cm distance significantly limits the NA, increasing the depth of focus of the extracted views. The enlarged speckle grains shown in Fig. 15 are also due to the limited NA of the virtual camera. In spite of the enlarged speckle grain size, Fig. 15 and Visualization 4 clearly demonstrate the reproduction of the horizontal and vertical parallax of the 3D scene as expected.

 

Fig. 15. Perspective views extracted from the hologram in Fig. 12. (Visualization 4)

Download Full Size | PDF

Figure 16 is the magnified portion of the extracted perspective views. It is observed from Fig. 16 and Visualization 4 that the relative positions of the chess pawn and its reflection with respect to the chess board change in opposite directions, which again verifies the correct parallax reproduction even in the case where the usual depth-map-based techniques are not available as the depth map is not uniquely determined.

 

Fig. 16. Magnified portion in the perspective views extracted from the hologram in Fig. 12.

Download Full Size | PDF

4. Conclusion

In this paper, we proposed an efficient calculation scheme for the non-hogel-based CGH technique which synthesizes the hologram from light field data without requiring depth map information. Unlike usual holographic stereograms which generate each hogel only from corresponding local view, the non-hogel-based CGH generates the hologram by globally processing the light field data, generating continuous parabolic wavefront for each object point with desired initial phase distribution determined by the carrier wave used in the calculation. The global processing, however, increases the computational load of the non-hogel-based CGH significantly, prohibiting the synthesis of high-resolution holograms. The angular-frequency-slice-based processing scheme proposed in this paper enables independent and concurrent processing of each angular frequency slice of the light field, making it suitable for the GPU implementation with reduced memory requirement. The tiled processing scheme was also proposed, enabling the synthesis of arbitrarily large resolution holograms without limited by the physical memory of the processing device. Using these two techniques, the synthesis of 25K×25 K resolution hologram from the light field data having 64 × 64 orthographic views of 1563 × 1563 resolution was successfully demonstrated. The enhanced depth range and the realistic depth and parallax reproduction of the object reflection in the hologram reconstruction were also demonstrated with the synthesized high pixel resolution hologram.