Open Access
Add to BibSonomy
Abstract
A novel technique is proposed to process the occlusion of a background hologram when synthesizing a front scene hologram from its light field. Unlike conventional techniques which process the occlusion in the light field domain after converting the background hologram to its light field, the proposed technique directly processes the occlusion between different domains, i.e., the background hologram and foreground light field. The key idea is to consider the background hologram as a carrier wave illuminating the front scene when synthesizing the front scene hologram from its light field. The proposed technique is not only computationally efficient as it does not require conversion between the light field and hologram domains but also accurate because all angular information of the background hologram and foreground light field is naturally considered in the occlusion processing. The proposed technique was verified by numerical synthesis and reconstruction.
© 2022 Optica Publishing Group under the terms of the Optica Open Access Publishing Agreement
1. Introduction
Coherent light from objects is represented by a complex optical field with an amplitude and phase distribution. Computer generated hologram (CGH) techniques digitally synthesize the complex optical field for a three-dimensional (3D) scene represented in various forms [1]. A light field, or also called a ray space, is a spatio-angular distribution of the light rays. CGH synthesis from a light field has been an active research topic. Various techniques have been proposed including the traditional hogel based holographic stereogram [2–5], and ray-wavefront conversion [6–7]. More recently, novel techniques which do not rely on the traditional hogel-based tiling of the light field have been reported including the overlapped hogel based technique [8], the hogel-free direct phase-only hologram synthesis technique [9], the non-hogel-based complex field synthesis technique [10], and the time-multiplexed neural holography technique [11]. These recent techniques utilize spatially overlapped hogel accumulation [8,10], iterative optimization [9], or deep learning [11] to obtain high quality 3D hologram from its light field, overcoming the limitation imposed by the spatio-angular resolution tradeoff of the traditional hogel based approaches [8–11], and achieving speckle-suppressed reconstruction with shallow depth of focus [11].
Occlusion between 3D objects is an important feature making the reconstructed scene realistic. Unlike the usual two-dimensional (2D) image rendering in which one needs to consider only a single viewpoint, the CGH should reproduce the view-dependent continuous change of the occlusion within its viewing angle, making its realization not trivial.
Several techniques have been proposed to implement the occlusion for point cloud based CGHs [12], layer based CGHs [13–15], and triangular mesh based CGHs [16–18]. In Ref. [12], spherical waves from individual object points are calculated in multiple wavefront recording planes (WRP) using occlusion masks around the object points. The wavefront in the WRPs is then numerically propagated towards the hologram plane and a hologram is finally obtained. In the layer-based method case, the occlusion is processed sequentially at each image layer [13–14]. More advanced techniques including mesh mask [15–16] or mesh aperture [17–18] have also been proposed to handle the complicated shape and reduce the computation time. Recently, an angular spectrum convolution technique has been proposed [18] which achieves exact occlusion by considering the wavefront from rear objects as a carrier wave for the front objects.
Unlike the point cloud, layer, and mesh-based techniques, the light field based CGHs do not usually require special techniques for implementing the occlusion. The occlusion is already included in the light field data, i.e., in the array of views constituting the light field, and the CGH synthesized from the light field naturally exhibits the occlusion effects between the objects in the scene. This is one of the advantages of light field based CGH techniques.
The problem tackled in this paper is the occlusion of the 3D scene outside of the light field data. Suppose that a hologram of a background 3D scene is given, and we want to synthesize a composite hologram that includes the foreground light field and the background 3D scene as shown in Fig. 1. Note that, in this paper, a hologram refers to the complex optical field represented by amplitude and phase distribution in a plane. The background hologram is the complex optical field of 3D scene which will be used as a background in the final composite scene. The background hologram can be a CGH synthesized numerically, or a real object digital hologram captured optically. The light field refers to a spatio-angular ray distribution which is represented by an array of angular views of the 3D scene. Individual view in the light field is amplitude data without phase information. The composite hologram refers to the complex optical field, i.e., amplitude and phase distribution, of the mixed 3D scene which has the objects given by their light field in the foreground and the objects given by their hologram in the background. For the background and the foreground 3D scene, it is assumed in this paper that their depth maps are not available.
Fig. 1. Composite hologram synthesis from background hologram and foreground light field with occlusion processing
In the composite hologram, the foreground 3D scene represented in the light field format needs to occlude the background 3D scene given by its hologram. If both the foreground and background scene are given with their depth maps, then the problem can be easily addressed. However, the foreground light field and the background hologram without their depth maps make the problem not trivial. This situation can happen frequently in the editing and authoring of hologram contents for various applications. A possible approach is to convert the background hologram to the light field using the wavefront-ray conversion [6] or spatial frequency filtering [19] and process the occlusion between the foreground and background light field. By synthesizing a new hologram with the occlusion-processed light field, the desired composite hologram can be generated as shown in Fig. 2(a). However, this approach requires many computations for the bi-directional conversions between the hologram and the light field. The resolution loss accompanied by the light field conversion from the hologram is another problem.
Fig. 2. Possible approaches in synthesizing a composite hologram with occlusion processing between the foreground light field and background hologram. (a) Conventional technique that transforms the background hologram to a light field, processes the occlusion in the light field domain, and synthesizes the hologram from the light field. (b) Proposed technique that directly processes the occlusion without a hologram-to-light field conversion.
In this paper, we propose a novel method for processing occlusion between the background hologram and foreground light field data. The proposed method processes the occlusion without requiring the conversion between the hologram and the light field domains. The key idea is to consider the background hologram as a carrier wave illuminating the foreground objects when applying the occlusion to the background hologram. By using the background hologram as a carrier wave for the binarized foreground light field, the proposed method creates a suitable complex field mask for the background hologram. The occluded background hologram is then simply added to the foreground hologram synthesized from the light field, completing the composite hologram as illustrated in Fig. 2(b). The proposed method is computationally efficient and maintains the full resolution of the background hologram because it does not require light field conversion of the background hologram. The carrier wave approach also ensures the exact and continuous view-dependent occlusion between the background and foreground objects within the viewing angle of the composite hologram. The proposed method is verified by numerical hologram synthesis and reconstruction for objects with continuous depth. A hologram from the JPEG Pleno database [20] was also adopted as a background hologram to prove the independency of the proposed method on the type of background hologram.
2. Brief review of the non-hogel-based CGH method
The proposed occlusion processing is based on the recently reported non-hogel-based CGH technique [10]. Unlike conventional hogel-based techniques, the non-hogel-based technique processes all views globally, generating a continuous spherical wavefront for an individual 3D object point without requiring a depth map for the scene [10]. Complete control of the carrier wave or the phase distribution on the surface of a 3D object is another unique feature of the non-hogel-based CGH technique. As will be explained in later sections, this feature is extensively used in the accurate and efficient processing of the occlusion proposed in this paper. In the initial proposal of the non-hogel-based CGH, a huge computational load was problematic [10]. However, a more efficient calculation method was later proposed, reducing the computation time significantly [21,22].
The non-hogel-based CGH technique synthesizes the complex field of a 3D scene using its light field data. The light field data can be expressed by L(tx, ty, θx, θy), where (tx, ty) is the spatial position of the ray, and (θx, θy) is the angular direction in radians. The angular direction of the rays (θx, θy) can be represented using the corresponding spatial frequency (u, v) = (sinθx/λ, sinθy/λ), where λ is the wavelength. In this paper, we represent the light field using L(tx, ty, u, v). Hologram synthesis from the light field L(tx, ty, u, v) by the non-hogel-based CGH has two steps [10]. The first step is the 2D Fourier transform of the light field L(tx, ty, u, v) along the (u, v) axes, giving
Light field data is an array of the amplitude views which do not contain phase information. For the hologram synthesis, most previous techniques assign the phase based on the depth map directly on the object surface or equivalently on each view of the light field [8,9]. To the contrary, the non-hogel-based technique [10] used in this paper assigns the object phase implicitly. Instead of assigning the phase directly on the object surface, the technique defines a carrier wave W(x,y) in the hologram plane. As explained in [10], the carrier wave W(x,y) acts as a light illuminating the object, determining the phase distribution on its surface without requiring the depth map. As will be explained in the next section, this carrier wave W(x,y) has a vital role in the occlusion proessing proposed in this paper.
3. Proposed occlusion-processing technique
The proposed technique synthesizes the composite hologram of the background hologram and the foreground light field using the non-hogel-based CGH method. Using the carrier-wave-control feature of the non-hogel-based CGH, the proposed technique finds the background optical field that needs to be masked by considering the original background hologram as a carrier wave of the foreground light field.
Figure 3 shows the proposed technique. Suppose that the hologram of the background 3D scene HB(x, y) and the light field of the foreground 3D scene LF(tx,ty,u,v) are given. It is assumed that both are defined in a common plane. The proposed technique synthesizes the final composite hologram Hc(x,y) by
Fig. 3. Procedure of the proposed method. (a) Given data, i.e., background hologram and foreground light field. (b) Procedure for the composite hologram synthesis.
In Eq. (3), HB(x,y) is the background hologram which is already given. HF(x,y) is the foreground hologram calculated from the foreground light field LF(tx,ty,u,v) using the non-hogel-based CGH method, i.e.
The key contribution of this paper is the method for calculating HM(x,y) in Eq. (3), i.e., the part of the background optical field that needs to be occluded by the foreground light field. First, the proposed method prepares the binary light field mask LFM(tx,ty,u,v) which is given by
Note that the proposed technique does not require any depth information of the 3D objects included in the background hologram or in the foreground light field. By simply using the background hologram as the carrier wave of the binarized foreground light field, the occlusion between the foreground and background objects are processed naturally. Additionally, note that this carrier wave treatment ensures the exact view dependent occlusion within the viewing angle of the hologram as will be demonstrated in a later section. The proposed technique does not require a conversion from the hologram to light field, making the whole process free from any resolution loss. Finally, the only added procedure for the occlusion processing is the calculation of HM(x,y), and thus the proposed technique is computationally effective.
4. Verification of the proposed technique
The proposed method was verified using several light field and hologram data which include holograms we synthesized and ones in the JPEG Pleno database [20]. Figure 4(a) shows an example of the background hologram HB and the foreground light field LF. In this example, a 3D scene containing 4 dice is initially prepared using the 3D rendering software Blender. Three dice of blue, green, and yellow color are at the same depth plane z=-0.675 mm, constituting the background scene. The red die at z=-0.425 mm is considered the foreground scene in this example. We used the Blender software to render the light field of the background 3 dice and the foreground red die separately. The background light field was additionally processed to synthesize the background hologram HB whose amplitude and phase are shown in Fig. 4(a). The synthesized background hologram HB has a 2.4K×2.4 K resolution and 0.93 µm pixel pitch. The foreground light field LF in Fig. 4(a) consists of 64 × 64 orthographic views, each of which also has a 600 × 600 resolution and 3.74 µm pixel pitch. Figure 4(b) shows another light field rendered for the entire 3D scene containing 4 dice, which is used to synthesize the ground truth composite hologram. The background hologram HB and the foreground light field LF in Fig. 4(a) are applied to the proposed method to synthesize a composite hologram with occlusion processing. Note that the spatial resolution of the light field was interpolated from 600 × 600 with 3.74 µm pixel pitch to 2.4K×2.4 K with 0.93 µm pixel pitch during the hologram synthesis to match with the spatial resolution of the background hologram. Figure 4(c) shows the intermediate hologram HB-HM before the foreground hologram is added and the final composite hologram HC=HB-HM+HF of the proposed method. Figure 4(d) is the ground truth composite hologram which is directly synthesized from the composite light field in Fig. 4(b). The intermediate, final composite, and ground truth holograms have the same 2.4K×2.4 K resolution and 0.93 µm pixel pitch.
Fig. 4. Given data and synthesized hologram in the case of discrete depth objects. (a) Background hologram and foreground light field for the proposed method. (b) Light field of the composite scene which is used for the ground truth hologram synthesis. (c) Synthesized hologram by the proposed method, i.e., intermediate hologram (left), and final composite hologram (right). (d) Ground truth hologram.
Figure 5 shows the amplitude of the numerical reconstructions of the synthesized holograms for various depths of z, i.e., |Propz(H)|. Figure 5(a) shows the result of the intermediate hologram HB-HM, i.e., the occluded background hologram, calculated by the proposed method. As shown in Fig. 5(a), at a background scene distance z=-0.675 mm, the yellow, green, and blue dice in the background are clearly focused. The area corresponding to the foreground red die is masked and appears blurred. In contrast, at a foreground object distance z=-0.425 mm, the masked foreground red die area becomes focused while the background objects are blurred. This clearly demonstrates that the HB-HM calculated by the proposed method successfully represents the background scene occluded by the foreground objects at the corresponding depth. Figure 5(b) shows the result of the final composite hologram HC=HB-HM+HF from the proposed method. As expected, the entire composite scene with the background and foreground objects is reconstructed with the corresponding depths, and the occlusion of the background 3 dice by the foreground red die is realized successfully. Figure 5(c) shows the ground truth. Figure 5(d) shows the result when the background hologram and the hologram synthesized with the foreground light field are simply combined without the proposed method. As expected, it reveals the problem that the background objects are not occluded by the foreground object. Figure 5(e) shows a pixel-by-pixel difference between the proposed method, ground truth, and the simple addition along a cross section indicated by the white dotted lines in Fig. 5(b)-(d). Figure 5(e) reveals that the proposed method produces exact result identical to the ground truth while the simple addition result deviates largely in the overlapping area.
Fig. 5. Comparison of the proposed method, ground truth, and conventional method using objects with discrete depths. Numerical reconstructions were performed from z=-0.675 mm to z=-0.425 mm for the (a) intermediate and (b) final composite hologram of the proposed method, (c) ground truth hologram, and (d) the simple addition of the background and foreground holograms without the proposed method. (e) The difference between (b), (c), and (d) along a cross section indicated by the white dotted line. The difference was calculated after converting color reconstructions shown in (b), (c), and (d) to gray images.
Second, the proposed method was verified using a 3D scene with a continuous depth distribution. The background hologram and the foreground light field in Fig. 6(a) were used as the input data. The background objects are two balls within -0.438mm ≤ z≤-0.344 mm, and the foreground object is a basket having a continuously curved shape within -0.263mm ≤ z≤-0.081 mm. The background hologram has a 12K×8 K resolution and a 0.47 µm pixel pitch. The light field has a 1563 × 1042 resolution with 64 × 64 orthographic view arrays, and a 3.74 µm pixel pitch. Figure 6(b) shows the light field of the composite scene used for the ground truth hologram synthesis. The synthesized hologram by the proposed method and the ground truth hologram are shown in Figs. 6(c) and 6(d), respectively. During the hologram synthesis, the light field was spatially interpolated to have 12K×8 K resolution and a 0.47 µm pixel pitch for the resolution match with the background hologram. The background, intermediate, final composite, and the ground truth holograms have the same 12K×8 K resolution and 0.47 µm pixel pitch.
Fig. 6. Given data and synthesized hologram in the case of continuous depth objects. (a) Background hologram and foreground light field for the proposed method. (b) Light field of the composite scene used for the ground truth hologram synthesis. (c) Synthesized hologram by the proposed method, i.e., intermediate hologram (left), and final composite hologram (right). (d) Ground truth hologram.
Figure 7 shows the numerical axial reconstructions of the holograms synthesized by the proposed method in Fig. 6(c). It can be seen from Figs. 7(a) and 7(b) that the background objects and the foreground objects are clearly focused at their respective depths. Figure 7(a) additionally indicates that the mask is well formed at the same depth as the foreground basket as expected. Comparison between Figs. 7(b) and 7(c) reveals the composite hologram synthesized by the proposed technique is well matched to the ground truth hologram. Figure 7(d) shows the quantitative comparison between the composite hologram (Fig. 7(b)) with the proposed method and the ground truth (Fig. 7(c)). The peak signal-to-noise ratios (PSNRs) are 43, 43.8, and 42.7 dB for depths z=-0.250 mm, z=-0.188 mm, and z=-0.125 mm, respectively, and the structural similarity index maps (SSIM) are 0.997, 0.997, and 0.995, which demonstrate that the proposed method provides accurate results.
Fig. 7. Comparison of the proposed method and ground truth with objects of continuous depth. Numerical reconstructions were performed from z=-0.250 mm to z=-0.125 mm for the (a) intermediate and (b) composite hologram by the proposed method, and (c) the ground truth hologram. (d) PSNR and SSIM calculated by comparing (b) and (c).
Figure 8 shows angular reconstructions of the proposed hologram in Fig. 6(c). In Fig. 8, various directional views are extracted from the synthesized holograms by applying a band-pass filtering to the hologram [19]. Figure 8(a) show a clear horizontal and vertical parallax between the mask and the background objects (highlighted by the red dashed box), and it matches exactly to the one between the foreground object and the background objects in Fig. 8(b). Therefore, it is confirmed that the proposed technique produces the exact occlusion not only in a specific viewpoint but also within the entire viewing angle or diffraction angle of the hologram which is |θx|≤31.28°, |θy|≤31.28° in this example. See Visualization 1 for video demonstration of the motion parallax.
Fig. 8. Directional views extracted from the (a) intermediate and (b) composite hologram of the proposed method. (Visualization 1)
Finally, we verified the generality of the proposed technique. In the previous verifications shown in Figs. 4–8, to obtain the ground-truth data, the background objects were initially created in the light field format like the foreground objects and their holograms, i.e., the background holograms were prepared by applying the non-hogel-based CGH technique to the background light field. The proposed technique, however, can be applied to any background holograms regardless of the numerical hologram synthesis algorithms or the optical capturing methods. To verify the independence of the proposed method to the type of the background hologram, we picked an arbitrary hologram from the JPEG Pleno holographic dataset [20] and used it as the background hologram. The selected hologram is ‘Dices8k4k’ which has a 7680 × 4320 resolution and a 4.8 µm pixel pitch with the wavelengths λR=640 nm, λG=532 nm, and λB=473 nm [23]. In the original ‘Dices8k4k’ hologram, 3 dice objects are distributed at a depth range from -19 to 0 mm. Before synthesizing the composite hologram, we shifted the depth range to -40.7mm < z<-21 mm by numerically propagating the hologram by 21 mm using the angular spectrum method. We also zero-padded the background ‘Dices8k4k’ hologram to 12K×8 K resolution before processing.
For the foreground, the basket light field was utilized. It consists of 64 × 64 orthographic view arrays with a 1563 × 1042 resolution. The original orthographic views in the light field were rendered with a 3.74 µm pixel pitch for the basket object with a curved shape spanning -2.1mm < z<-0.65 mm. During the hologram synthesis, the light field was spatially expanded by 8 times from the original 1563 × 1042 resolution with 3.74 µm pixel pitch to 12K×8 K resolution with the same 3.74 µm pixel pitch to cover the spatial size of the background hologram. This spatial expansion of the light field leads to the 3D expansion of the foreground basket object, resulting in -(2.1 × 8)mm < z<-(0.65 × 8)mm or -16.8mm < z<-5.2 mm depth range [19]. To match the pixel pitch of the orthographic view to the background hologram, we then simply considered the pixel pitch of the orthographic view as 4.8 µm, which scales again the basket object both in the transverse and longitudinal directions by M=4.8 µm /3.74µm=1.284 [19]. Thus, the final depth range of the basket object in this verification is M×(-16.8 mm)<z < M×(-5.2 mm) or -21.57mm < z<-6.68 mm. Figures 9(a) and 9(b) show the background hologram and foreground light field along with the 3D objects in the composite scene. The final composite hologram synthesized by the proposed method has 12K×8 K resolution with 4.8 µm pixel pitch.
Fig. 9. Verification of the proposed method using a hologram from the JPEG Pleno hologram dataset. (a) Background and foreground objects. (b) Given data, i.e., background hologram ‘Dices8k4k’ from the JPEG Pleno dataset (left) and foreground light field (right). (c) Numerical reconstructions of the foreground hologram (top row), the intermediate masked hologram (middle row), and the final composite hologram by the proposed method (bottom row). (Visualization 2)
The verification results are shown in Figs. 9(c) and 10. Figure 9(c) and Visualization 2 show the numerical reconstructions at various depths of the foreground hologram HF, intermediate hologram HB-HM, and composite hologram HC=HB-HM+HF synthesized by the proposed method. Figure 10 and Visualization 3 show the various directional views extracted from the intermediate hologram HB-HM and the composite hologram HC=HB-HM+HF within their diffraction angle (|θx|≤2.91°, |θy|≤2.91°). The results shown in Figs. 9(c) and 10 demonstrate that the foreground basket with a continuously curved depth successfully occludes the background dice objects contained in the ‘Dices8k4k’ hologram from the JPEG Pleno dataset, showing depth-wise refocusing and a clear parallax without any artefacts. Therefore, it is verified that the proposed method is general and can be applied to a wide range of already-existing holograms without imposing any special requirements for its application.
Fig. 10. Directional views extracted from the (a) intermediate and (b) composite hologram of the proposed method. (Visualization 3)
5. Discussion
The proposed method calculates the occlusion between the background hologram and a foreground light field directly without conversion of the domain. This feature can be utilized in several applications. One of the target applications of the proposed method is a hologram editing tool. The hologram editing tool needs to accept various data format for individual 3D objects and produce the hologram of the composite scene. Although a homogeneous format of data would be beneficial in many cases, there are situations in which the homogeneous format is not available.
An example is the combination of the optically captured hologram of real objects and the digital light field of computer graphics objects. Suppose we want to synthesize a new hologram by augmenting a digital 3D object in front of the existing background hologram which was captured optically. The optically captured hologram has a complex field of the object wave but usually lacks the native 3D model data like the point cloud of the object. Extracting the 3D model data from the background hologram and combining it with the foreground digital object is not accurate. Extracting the light field from the background hologram is also usually accompanied by resolution loss and not efficient. The proposed method provides a direct way to synthesize the hologram of the composite scene by simply using the background hologram as a carrier wave of the foreground light field. No domain conversion between the light field and hologram and no requirement of additional 3D model data of the background hologram makes the proposed method useful and versatile as one of the building blocks of a hologram editing tool. We think that this feature of the proposed method enables the tool to accept a wide range of existing holograms which have been captured optically.
Another potential use scenario of the proposed method is the synthesis of holographic video of a dynamic 3D scene. If the 3D scene consists of a static background and dynamic foreground, the proposed method enables the synthesis of the composite scene by only updating the foreground without re-rendering the light field of the entire scene considering the occlusion and synthesizing the hologram again.
In spite of its useful features, the current implementation of the proposed method has several limitations. One of the current limitations is the computational speed. The key idea of the proposed method is to use the background hologram as a carrier wave of the foreground light field for the synthesis of the complex field mask HM(x,y). This key idea enables the occlusion without converting the background hologram to the light field, making it computationally more efficient than other possible approach which processes the occlusion after converting the background hologram to the light field. However, for the carrier wave control during the light field CGH synthesis, the proposed method uses the non-hogel-based CGH technique [10]. The non-hogel-based CGH technique requires numerical weighted integration of the light field by Eqs. (1) and (2) which originate from the inverse of the Wigner distribution function [10]. This physics based numerical integration is exact but requires enhancement in terms of the computational speed. Recently, deep neural network based CGH algorithms has demonstrated significant progress in the computational efficiency by replacing the physics-based propagation and optimization to network-based propagation and inference [10,11,24]. We believe that by applying the proposed key idea, i.e., using the background hologram as a carrier wave of the foreground light field, to the deep neural network based light field CGH algorithms, the computational speed of the proposed method could be enhanced further.
Another limitation of the proposed method is that the objects given by the hologram need to be behind the objects given by the light field. As the proposed method uses the background hologram as a carrier wave when synthesizing the mask hologram of the foreground light field, the foreground occluding objects should be given by the light field format. Thus the occlusion between the background light field and the foreground hologram cannot be processed by the proposed method. This type of occlusion would require depth map of the foreground hologram. Similarly, if multiple holograms with different depths are used as background, the proposed method only calculates the individual occlusion between the foreground light field and each of the background holograms. The mutual occlusion between the multiple background holograms cannot be considered. Note, however, that the occlusion between multiple foreground light field with non-overlapping depth range and a background hologram can be processed by the proposed method. By sequentially applying the proposed method from the rear-most light field, the proposed method can obtain the composite hologram of the entire scene with proper occlusion. Further extension overcoming current limitations of the proposed method is a topic of further research.
6. Conclusion
In this paper, we proposed a method to process the occlusion when synthesizing a hologram of a composite scene from a foreground light field and background hologram. The main idea is to use the background hologram as a carrier wave illuminating the foreground objects when synthesizing the occlusion mask. The proposed method firsts binarizes the foreground light field and synthesizes its hologram with the background hologram as a carrier wave. The synthesized hologram is used as the occlusion mask and subtracted from the background hologram. The foreground hologram calculated separately from the original light field with an arbitrary carrier wave is then added, completing the composite hologram with the occlusion between the foreground objects in the input light field and the background objects in the input hologram. The proposed method is computationally efficient because it does not need to convert the background hologram to the light field for the occlusion processing. The proposed method does not require any prior depth map information of the foreground and background objects, which makes the proposed method widely applicable. Considering the background hologram as a carrier wave of the foreground light field also enables accurate occlusion processing, exhibiting the correct parallax between the occlusion mask and the background objects within the diffraction angle of the hologram. The proposed method was successfully verified using various 3D scenes including objects with discrete or continuous depths. Verification results using a hologram from JPEG Pleno dataset were also presented, demonstrating the generality of the proposed method.
Funding
Institute of Information & Communications Technology Planning & Evaluation (IITP) (2021-0-00091).
Acknowledgments
This work was supported by Institute of Information & communications Technology Planning & Evaluation (IITP) grant funded by the Korea government (MSIT) (No. 2021-0-00091, Development of real-time high-speed renderer technology for ultra-realistic hologram generation)
Disclosures
The authors declare no conflicts of interest.
Data availability
Data underlying the results presented in this paper are not publicly available at this time but may be obtained from the authors upon reasonable request.
References
1. J.-H. Park, “Recent progresses in computer generated holography for three-dimensional scene,” J. Inform. Display 18(1), 1–12 (2017). [CrossRef]
2. T. Yatagai, “Stereoscopic approach to 3-D display using computer-generated holograms,” Appl. Opt. 15(11), 2722–2729 (1976). [CrossRef]
3. M. Yamaguchi, H. Hoshino, T. Honda, and N. Ohyama, “Phase-added stereogram: calculation of hologram using computer graphics technique,” Proc. SPIE 1914, 25–31 (1993). [CrossRef]
4. P. W. McOwan, W. J. Hossack, and R. E. Burge, “Three-dimensional stereoscopic display using ray traced computer generated holograms,” Opt. Commun. 82(1-2), 6–11 (1991). [CrossRef]
5. K. Wakunami and M. Yamaguchi, “Calculation for computer generated hologram using ray-sampling plane,” Opt. Express 19(10), 9086–9101 (2011). [CrossRef]
6. K. Wakunami, H. Yamashita, and M. Yamaguchi, “Occlusion culling for computer generated hologram based on ray-wavefront conversion,” Opt. Express 21(19), 21811–21822 (2013). [CrossRef]
7. S. Igarashi, T. Nakamura, K. Matsushima, and M. Yamaguchi, “Efficient tiled calculation of over-10-gigapixel holograms using ray-wavefront conversion,” Opt. Express 26(8), 10773–10786 (2018). [CrossRef]
8. N. Padmanaban, Y. Peng, and G. Wetzstein, “Holographic near-eye displays based on overlap-add stereograms,” ACM Trans. Graph. 38(6), 1–13 (2019). [CrossRef]
9. P. Chakravarthula, E. Tseng, H. Fuchs, and F. Heide, “Hogel-free Holography,” ACM Trans. Graph. (2022 accepted for published).
10. J.-H. Park and M. Askari, “Non-hogel-based computer generated hologram from light field using complex field recovery technique from Wigner distribution function,” Opt. Express 27(3), 2562–2574 (2019). [CrossRef]
11. S. Choi, M. Gopakumar, Y. Peng, J. Kim, M. O’Toole, and G. Wetzstein, “Time-multiplexed Neural Holography: A Flexible Framework for Holographic Near-eye Displays with Fast Heavily-quantized Spatial Light Modulators,” ACM Trans. Graph. 32(4), 1–8 (2013). [CrossRef]
12. A. Symeonidou, D. Blinder, A. Munteanu, and P. Schelkens, “Computer-generated holograms by multiple wavefront recording plane method with occlusion culling,” Opt. Express 23(17), 22149–22161 (2015). [CrossRef]
13. T. Shimobaba, N. Masuda, and T. Ito, “Simple and fast calclulation algorithm for computer-generated hologram with wavefront recording plane,” Opt. Lett. 34(20), 3133–3135 (2009). [CrossRef]
14. K. Matsushima, “Exact hidden-surface removal in digitally synthetic full-parallax holograms,” Proc. SPIE 5742, 25–32 (2005). [CrossRef]
15. A. Kondoh and K. Matsushima, “Hidden surface removal in full-parallax CGHs by silhouette approximation,” Syst. Comp. Jpn. 38(6), 53–61 (2007). [CrossRef]
16. K. Matsushima and S. Nakahara, “Extremely high-definition full-parallax computer-generated hologram created by the polygon-based method,” Appl. Opt. 48(34), H54–H63 (2009). [CrossRef]
17. K. Matsushima, M. Nakamura, and S. Nakahara, “Silhouette method for hidden surface removal in computer holography and its acceleration using the switch-back technique,” Opt. Express 22(20), 24450–24465 (2014). [CrossRef]
18. M. Askari, S.-B. Kim, K.-S. Shin, S.-B. Ko, S.-H. Kim, D.-Y. Park, Y.-G. Ju, and J.-H. Park, “Occlusion handling using angular spectrum convolution in fully analytical mesh based computer generated hologram,” Opt. Express 25(21), 25867–25878 (2017). [CrossRef]
19. D.-Y. Park and J.-H. Park, “Generation of distortion-free scaled hologram using light field data conversion,” Opt. Express 29(1), 487–508 (2021). [CrossRef]
20. “JPEG Pleno Dataset”, http://plenodb.jpeg.org.
21. J.-H. Park, “Efficient calculation scheme for high pixel resolution non-hogel-based computer generated hologram from light field,” Opt. Express 28(5), 6663–6683 (2020). [CrossRef]
22. D. Min, K. Min, and J.-H. Park, “Full-color non-hogel-based computer-generated hologram from light field without color aberration,” Curr. Opt. Photon. 5, 409–420 (2021).
23. A. Gilles, P. Gioia, R. Cozot, and L. Morin, “Hybrid approach for fast occlusion processing in computer-generated hologram calculation,” Appl. Opt. 55(20), 5459–5470 (2016). [CrossRef]
24. L. Shi, B. Li, C. Kim, P. Kellnhofer, and W. Matusik, “Towards real-time photorealistic 3D holography with deep neural networks,” Nature 591(7849), 234–239 (2021). [CrossRef]
