Open Access
Add to BibSonomyAbstract
A new robust MPEG-based novel look-up table (MPEG-NLUT) is proposed for accelerated computation of video holograms of fast-moving three-dimensional (3-D) objects in space. Here, the input 3-D video frames are sequentially grouped into sets of four, in which the first and remaining three frames in each set become the reference (RF) and general frames (GFs). Then, the frame images are divided into blocks, from which motion vectors are estimated between the RF and each of the GFs, and with these estimated motion vectors, object motions in all blocks are compensated. Subsequently, only the difference images between the motion-compensated RF and each of the GFs are applied to the NLUT for CGH calculation based on its unique property of shift-invariance. Experiments with three types of test 3-D video scenarios confirm that the average number of calculated object points and the average calculation time of the proposed method, have found to be reduced down to 27.34%, 55.46%, 45.70% and 19.88%, 44.98%, 30.72%, respectively compared to those of the conventional NLUT, temporal redundancy-based NLUT (TR-NLUT) and motion compensation-based NLUT (MC-NLUT) methods.
© 2014 Optical Society of America
1. Introduction
Thus far, a number of approaches for generation of the computer-generated holograms (CGHs) of three-dimensional (3-D) objects have been proposed [1–7]. Recently, a novel look-up table (NLUT) method to highly enhance the computational speed as well as to dramatically reduce the total number of pre-calculated interference patterns required for the generation of computer-generated holograms (CGHs) of three-dimensional (3-D) objects was proposed [7]. In this method, a 3-D object is approximated as a set of discretely sliced image planes having different depth, and only the fringe patterns of the center-located object points on each image plane, which are called two-dimensional principal fringe patterns (2-D PFPs), are pre-calculated and stored. Therefore, the memory size of the NLUT can be reduced down to the order of gigabytes (GB) from the order of terabytes (TB) of the conventional look-up-table (LUT) for the moderate 3-D object [7].
However, for the NLUT to be useful in the future holographic 3-D information terminals, its memory capacity and computation time must be further reduced down to the order of megabytes (MB) and milliseconds per frame, respectively. Hence, various algorithmic and hardware approaches have been suggested to solve the aforementioned challenging issues of the NLUT [8–17].
For reducing the memory capacity, a one-dimensional NLUT (1-D NLUT) in which GB-sized memory size of the conventional NLUT could be decreased down to the order of MB, was proposed by employing a new concept of 1-D sub-PFPs decomposed from the conventional 2-D PFPs [8]. Thus far, this 1-D NLUT might be the most effective approach for generation of CGH patterns for the future lifelike 3-D mobile terminals in terms of the memory capacity.
In addition, in order to accelerate the computational speed, several hardware and software methods have been also proposed [10–17]. In the hardware approach, the original NLUT or its modified versions were attempted to be implemented on field-programmable-gate-arrays (FPGAs) or graphic-processing-units (GPUs) [16,17]. On the other hand, in the software approach, the NLUT employed various image compression algorithms for removal of spatially or temporally redundant object data by taking into account of the shift-invariance property of the NLUT [10–15].
Among them, a temporal redundancy-based NLUT (TR-NLUT) was proposed [10], in which temporally-redundant object data between consecutive video frames of 3-D moving objects are removed with the differential pulse-code modulation (DPCM) algorithm, then only the difference images are applied to the NLUT for CGH calculation. This way, the TR-NLUT can reduce the number of object points to be calculated for CGH generation and obtain a more enhanced computational speed compared to the NLUT.
This method, however, has a critical drawback of its compression efficiency being highly dependent on the temporal redundancy of the 3-D moving objects [10,15]. That is, as the 3-D objects move faster, the image differences between two consecutive video frames get larger, which results in a sharp decrease in the compression efficiency of the TR-NLUT. To be more precise, this TR-NLUT cannot acquire any improvements in computational speed in cases where there are more than 50% image differences between two consecutive video frames.
On the other hand, a motion-compensation concept has been widely employed in compression of 2-D video images [17–20]. In this approach, video data to be processed can be significantly reduced by compensating the object’s motions with an amount of estimated motion-vectors between two consecutive video frames. Moreover, as mentioned above, the NLUT has an inherent shift-invariance property, which means the PFPs for the object points on each depth plane are always same regardless of the location shift of the object points [7,16]. Therefore, by applying this motion-compensation concept to the NLUT based on its shift invariance property, a massive removal of redundant object data between consecutive video frames may be expected.
For this, a motion-compensated NLUT (MC-NLUT) method was proposed [16], in which motion vectors of the 3-D moving objects are extracted between two consecutive video frames and with these, object’s motions at each frame are compensated. Then, only the difference image between the motion-compensated object image of the previous frame and the object image of the current frame is applied to the NLUT for CGH calculation.
However, this MC-NLUT also has a few drawbacks in practical application. For motion compensation, first of all, this method requires a segmentation process of the input 3-D moving objects [17]. Moreover, since this method only extracts the absolute displaced distances of 3-D moving objects between two consecutive frames without a sophisticated motion-estimation process, the accuracy of extracted motion vectors becomes very low, and it gets worse as the 3-D objects move faster. In particular, in cases where a 3-D object with several parts moving in different directions may have multiple moving vectors, each part of this object must be compensated separately with their estimated motion vectors. However, the MC-NLUT method uses only one motion vector for each segmented object, so that the object’ motions cannot be precisely compensated.
Accordingly, in this paper, as an alternative to the conventional TR-NLUT and MC-NLUT methods, a new type of the NLUT employing the MPEG-based motion estimation and compensation scheme, which is called MPEG-based NLUT (MPEG-NLUT), is proposed for rapid generation of video holograms of fast-moving 3-D objects in space.
The MPEG is the most efficient compression algorithm for video images because of its excellent ability to exploit high temporal correlation between successive video frames. That is, the MPEG operates the block-based motion estimation and compensation process based on a strict mathematical model hence, as many redundant object data between consecutive video frames as possible can be removed, which causes the computational speed of the MPEG-NLUT to be highly enhanced.
Furthermore, since the compression performance of the MPEG-NLUT may not depend on the temporal redundancy of the 3-D moving objects, it can be applied to video compression of fast-moving 3-D objects. In other words, contrary to the conventional TR-NLUT and MC-NLUT methods, the MPEG-NLUT method shows an excellent compression performance even for cases where there are more than 50% image differences between consecutive video frames. In addition, since the MPEG stream contains the motion vectors and residual information of 3-D moving objects, the proposed method can directly generate the CGHs from the MPEG stream without a decoding process at the recipients.
To confirm the feasibility of the proposed MPEG-NLUT method, experiments with three types of test 3-D video scenarios are performed and the results are compared to those of the conventional NLUT, TR-NLUT and MC-NLUT methods in terms of the average number of calculated object points and the average calculation time.
2. Shift-invariance property of the NLUT method
Figure 1 shows a geometric structure to compute the Fresnel fringe pattern of a volumetric 3-D object. Here, the location coordinate of the p-th object point is specified by (xp, yp, zp), and each object point is assumed to have an associated real-value magnitude and phase of αp, φp, respectively. The hologram pattern to be calculated is also assumed to be positioned on the depth plane of z = 0 [7].
Basically, a 3-D object can be treated as a set of image planes discretely sliced along the z-direction, and each image plane having a specific depth is approximated as a collection of self-luminous object points of light. In the NLUT method, only the 2-D PFPs representing the fringe patterns of the object points located on the centers of each depth plane are pre-calculated and stored [7]. Then, the fringe patterns of other object points located on each depth plane can be obtained just by shifting these pre-calculated PFPs according to the displaced coordinate values from the center to the corresponding object points without further calculation processes.
Here, we can define the unity-magnitude PFP for the object point (xp, yp, zp) positioned on the center of a depth plane of zp, T(x, y; zp) as Eq. (1).
In which the wave number k is defined as k = 2π/λ, and λ and θR represent the free-space wavelength of the light and the incident angle of the reference beam, respectively. Moreover, the oblique distance rp between the p-th object point of (xp, yp, zp) and the point on the hologram plane of (x, y, 0) is given by Eq. (2).
Then, the fringe patterns for other object points on the depth plane of zp can be obtained by simply shifting the corresponding PFP of Eq. (1). These shifted versions of PFPs are added together to get the CGH pattern for the depth plane of zp. This process is performed for all depth planes to get the final CGH pattern of a 3-D object.
Therefore, in the NLUT method, the CGH pattern for an object I(x, y) can be expressed in terms of the shifted versions of pre-calculated PFPs of Eq. (1) as shown in Eq. (3).
Where N denotes the number of object points. Equation (3) reveals that the NLUT method may enable obtaining the CGH pattern of a 3-D object just by combination of operations such as shifting and adding the PFPs on each depth plane of the 3-D object. That is, on each depth plane, the corresponding PFPs are shifted according to the dislocated values of the object points from the center points, and then these shifted versions of PFPs for all object points located on each depth plane are added up to obtain the final CGH pattern of a 3-D object, which may confirm the unique shift-invariance property of the NLUT.
Figure 2 shows a schematic to show the shift-invariance property of the NLUT. As seen at the left side of Fig. 2, a cubic block is assumed to be located at A(0, 0, -z1) at time t1 and moved to B(x1, y1, -z1) at time t2. Then, the calculated CGH pattern for the cubic block of A, which is called CGHA here, and the reconstructed cubic image from this CGHA at the location of A'(0, 0, z1) are shown at the center and right sides of Fig. 2, respectively.
Moreover, the CGH pattern for the cubic block moved to the location of B, which is called CGHB here, can be obtained by simply shifting the hologram pattern of CGHA to the location of (x1, y1) without any other calculation processes in the NLUT method. At the same time, from this hologram pattern of CGHB, which is regarded as a shifted version of CGHA, the corresponding cubic image can be reconstructed at the shifted location of B'(x1, y1, z1).
Accordingly, in the NLUT method, the CGH pattern for the moved object can be generated only with the moving distances of the object without any additional calculations by taking advantage of the shift-invariance property of the NLUT. That is, if the motion vectors of the 3-D moving objects between two successive video frames of t1 and t2 are effectively extracted, the CGH pattern for the video frame of t2 can be generated just by shifting the pre-calculated CGH pattern for the video frame of t1 according to the extracted motion vectors.
Here, this shift-invariance property of the NLUT is directly matched to the concept of motion estimation and compensation which has been widely employed in compression of 2-D video data. Therefore, by applying this motion estimation and compensation concept to the NLUT, 3-D video data to be calculated for CGH generation is expected to be greatly reduced, which may result in a significant increase of the computational speed of the NLUT.
3. Limitations of the conventional TR-NLT and MC-NLUT
Figure 3 shows an example for relative comparison of the compression performance among the conventional NLUT, TR-NLUT, MC-NLUT and the proposed MPEG-NLUT methods. Here, Figs. 3(a) and 3(b) show the reference image (previous frame) and the input image (current frame). Figures 3(c) and 3(d) also show the object-based motion vector and block-based motion vectors extracted between the reference and input images of Figs. 3(a) and 3(b), respectively. The object points of the input image and the difference image between the reference and input images are also displayed in Figs. 3(e) and 3(f), respectively. In addition, Figs. 3(g) and 3(h) show the object points of the difference images obtained based on the object-based motion vector and the block-based motion vectors of Figs. 3(c) and 3(d), respectively.
Fig. 3 (a) Reference image, (b) Input image, (c) Object-based motion vector, (d) Block-based motion vectors (e) Object points of the input image, (f)-(h) Object points of the difference images extracted between the reference and input images without a motion vector, with the object-based motion vector and the block-based motion vectors, respectively.
Now, for the TR-NLUT, temporally-redundant data between two consecutive 3-D video frames are removed and only the difference image is applied to the NLUT for CGH calculation. Therefore, as seen in Fig. 3(f), the number of object points to be calculated has been much reduced compared to that of the input image shown in Fig. 3(e). However, as mentioned above, the compression efficiency of the TR-NLUT depends on the 3-D object’s motions because only the absolute differences between consecutive video frames are extracted. Hence, the faster the 3-D objects move, the larger the differences between two video frames become, which causes a sharp drop in the compression efficiency of the TR-NLUT.
For the MC-NLUT, motion vectors of the moving object are extracted between two consecutive video images of Figs. 3(a) and 3(b) as shown in Fig. 3(c), and with them the object’s motion on the reference image is compensated. Then, only the CGH pattern for the difference image between the motion-compensated reference image and the input image are computed. As seen in Fig. 3(g), in the MC-NLUT, the number of object points to be calculated has been much more reduced compared to the TR-NLUT of Fig. 3(f) because it employs a motion compensation process even though this extraction process of the difference image between two consecutive frames seems to be the simplest motion-estimation method.
In this MC-NLUT, however, only the displaced distances of the objects between two consecutive video frames are extracted without an accurate motion-estimation process, thus the degree of accuracy of the extracted motion vectors must be much lowered. Moreover, in case of a 3-D object with several parts moving in different direction or speed, some parts of the object may move forward in fast motion while others may move upward in slow motion at the same time. In other words, this object may have multiple motion vectors, which means that the motions of each moving part of the object must be simultaneously compensated with separately estimated motion vectors for effective removal of the redundant object data.
However, as mentioned above, only one motion vector for one object is extracted in the MC-NLUT, therefore the object’s motions cannot be accurately compensated. Furthermore, as the objects move faster, the accuracy of the extracted motion vectors declines. This may cause an increase in the differences between consecutive video frames just like in the TR-NLUT.
Figure 3(d), on the other hand, shows the motion vectors extracted from each block of the 3-D moving object by using the MPEG-based motion estimation and compensation scheme. Figure 3(d) shows the car body moving to the right side while four wheels rotate, the motion vectors corresponding to the car body may have the same moving directions, whereas the motion vectors corresponding to the car wheels have many different moving directions. In the proposed MPEG-NLUT, different motion vectors existed on the moving object can be simultaneously estimated and compensated, so that an accurate compensation of the object’s motion can be achieved on the block basis. Figure 3(h) shows the object points of the difference image extracted between the motion-compensated reference and input images with the MPEG. Figures 3(e)–3(h) finally confirm that the proposed MPEG-NLUT shows the smallest number of object points to be calculated for CGH generation compared to those of the conventional NLUT, TR-NLUT and MC-NLUT methods.
4. Proposed MPEG-based NLUT method
Figure 4 shows an overall block-diagram of the proposed MPEG-based NLUT (MPEG-NLUT) method for accelerated computation of video holograms of fast-moving 3-D objects in space, which is largely composed of six processes.
First, input 3-D video frames are grouped into a set of GOP (group of pictures) in sequence, in which each GOP consists of one RF (reference frame) and several GFs (general frames). Then, all video frames in each GOP are divided into 2-D arrays of image blocks. Second, the CGH pattern of the RF in a GOP is generated by calculating the block-CGHs (B-CGHs) for each block of the RF by using the TR-NLUT and adding them altogether. Third, the motion vectors (MVs) are estimated from the blocks between the RF and each of the GFs in a GOP. Fourth, all object motions on each block of the GFs are compensated just by shifting the image blocks and their corresponding B-CGH patterns according to the estimated MVs, from which motion-compensated RFs (MC-RFs) for each of the GFs are obtained. Fifth, the difference images are extracted from the blocks between the MC-RF and each of the GFs in a GOP. Finally only the B-CGHs for these difference images are calculated and added to those of each block of the MC-RFs to generate the CGH for the GFs.
4.1 Grouping and blocking of input 3-D video frames
In order to apply the MPEG-based motion estimation and compensation process to the sequence of 3-D video frames, a grouping and blocking process of the input video frames is needed. That is, input 3-D video frames with both of intensity and depth data are sequentially grouped into a series of GOPs. Here, a GOP is assumed to be composed of four video frames including one RF and three GFs, in which each video frame consists of intensity and depth images having the resolution of 512 × 512 pixels.
As an example, Fig. 5 shows the video structure for 10 video frames to be grouped into three GOPs of GOP1, GOP2 and GOP3. As mentioned above, the first frame and the remaining three in each GOP are designated as the RF and the GFs, respectively. Here, the number of GFs can be chosen depending on the object’s motions in the input video frames. This means that for slow-moving objects, the similarity of the objects between two adjacent video frames increases, thus the number of video frames composed of a GOP can be increased, whereas for fast-moving objects the corresponding number of video frames can be decreased.
Following the grouping process, all video frames in each GOP are divided into 32 × 32 square blocks for the subsequent block-based motion estimation and compensation process, in which each block has a resolution of 16 × 16 pixels.
4.2 Calculation of B-CGHs for each block of the RF
The CGH pattern of the RF in a GOP can be generated by computing the B-CGHs for each block of the RF using the TR-NLUT, and adding them altogether. With this CGH pattern of the RF, then, the motion-compensated versions of the remaining three GFs in a GOP can be calculated.
Figure 6 shows the RF in a GOP which is divided into M × N image blocks and the calculated B-CGHs for each image block of the RF with the TR-NLUT.
Fig. 6 A reference frame divided into M × N blocks and the calculated B-CGHs for each block with the TR-NLUT.
Here, the IB(m, n) represents the B-CGH of the (m, n)th block of the RF, in which m (1≤m≤M) and n (1≤n≤M) are the integers, then the CGH pattern of the RF in a GOP, which is denoted here as IR, can be simply generated as a sum of the B-CGHs calculated for all blocks of the RF as follows.
Now, for the feasibility test of the proposed method as well as for the comparative performance analysis of the proposed method with those of the conventional NLUT, TR-NLUT and MC-NLUT methods, three types of test video scenarios composed of 150 video frames with fast-moving 3-D objects, which are called here ‘Case I’, ‘Case II’ and ‘Case III’, are computationally generated by using the 3DS MAX as a form of sequential images under the condition that the image differences between two consecutive video frames are to be more than 50% in all video frames.
For the first test scenario of ‘Case I’ shown in Fig. 7(a), only one ‘Car’ object moves fast along the pathway from the left to the right at the distance of 630mm. Hence, motion vectors with one direction exist for all frames of the ‘Case I’. Instead, for the second test scenario of ‘Case II’ shown in Fig. 7(b), two ‘Car’ objects move fast in the opposite directions at the distances of 630mm and 700mm without any overlapped regions. Therefore, motion vectors with different directions exist for all frames of the ‘Case II’.
Fig. 7 Intensity and depth images of the 1st frame for each test 3-D video of (a) Case I (Media 1), (b) Case II (Media 2) and (c) Case III (Media 3).
Furthermore, for the third test scenario of ‘Case III’ shown in Fig. 7(c), two objects move fast in the opposite directions with some overlapping regions between them. For this case, there should be some difficulties in segmentation of those objects into individuals since two ‘Car’ objects are overlapped on the same blocks, which means the MC-NLUT may not be appropriate for this test scenario of ‘Case III’.
4.3 Block-based motion estimation
For the inter-frame coding, a process of motion estimation and compensation is the powerful means for eliminating the temporal redundancy due to high correlation between consecutive frames [18,19]. Figure 8 shows a motion-estimation process for one block between the RF and one of the GFs in a GOP. Video frames are divided into 2-D arrays of blocks, and motion vectors (MVs) for them are extracted by using the block-based motion estimation technique.
Here, in case the block-based motion estimation scheme is applied to the 3-D video composed of intensity and depth images, the MVs for each block of the RF are extracted only from the intensity image because the corresponding blocks of the depth image have the same MVs as mentioned above.
As seen in Fig. 8, Am,n(x1, y1), the (m, n)th image block of the RF centering on the location coordinates of (x1, y1), is assumed to move to the position of Bm,n(x2, y2) in the GF with the moving amounts of dx and dy along the x and y directions, respectively. The shifted location coordinates of Bm,n(x2, y2) in the GF can be, then, expressed as follows
, where dx and dy are designated as the MVs of the block in the x and y direction, respectively, and Δt denotes the time interval between two consecutive video frames. That is, the motion-compensated image block can be obtained by the combined use of the block of the RF and the estimated MVs.
Thus far, many motion-estimation algorithms have been suggested, which includes the ES (exhaustive search), TSS (three-step search), SETSS (simple and efficient TSS), 4SS (four-step search), DS (diamond search) and ARPS (adaptive rood pattern search), etc [15,18,20]. Among them, the ES method is employed here because of its high matching accuracy based on the full-search compared to other methods [20]. In this method, to perform the block-matching process for one block of Am,n(x1, y1) on the RF, a searching area given by the square with a side length of SL around the corresponding block of A'm,n(x1, y1) (the same position with the block of Am,n(x1, y1) on the RF) on the GF, is determined in advance as shown in Fig. 8. Here, R denotes the search parameter. Actually, the faster the object moves, the larger the search parameter of SL becomes.
Here, the block-matching process is performed between the block of Am,n(x1, y1) on the RF and all possible blocks with the same size of Am,n(x1, y1) within the searching region in the GF. The matching degree of one block with another is based on the value of a cost function, and the block showing the least value of the cost function is the one that matches the closest to the block of the GF.
There are many types of cost functions. Among them, the MAD (mean absolute difference) given by Eq. (6) is used as the cost function in this paper because it has been known as one of the most popular and less computationally intensive cost functions [20].
Where SL is the side length of the block, Cij and Rij represent the pixels being compared in the blocks of the GF and the reference block of the RF, respectively.
After finding the best-matching block in the GF, the displacements between the positions of the reference block of the RF and those of the matching block in the GF are computed as output MVs. As you can see in Fig. 8, the center point of the reference block of A and the best-matching block of B are given by (x1, y1) and (x2, y2), which means the reference block of the RF are moved from the location of (x1, y1) to the location of (x2, y2) in the GF. Accordingly, the motion vector of the block A can be computed by using Eq. (7).
That is, the block of A is shifted by dx and dy along the x and y direction, respectively between the RF and GF. Here, if the motion vectors of dx or dy are given by the negative numbers, they are then oriented to the left or to the upper direction along the x and y axis.
In this way, the best-matching blocks on the GF for every block of the RF can be found. Figure 9 shows an example of 9 moved blocks on the RF which are represented by A, B, C,…, I. The MVs for these blocks of the RF can be computed and stored as the motion-vector map for the following steps of the block-based motion estimation and compensation.
Fig. 9 An example of the block-based motion estimation: (a) An optical-flow map showing the MVs of 9 blocks of the RF, which are represented by the arrows, (b) A motion-vector map showing the corresponding motion-vector values given by the number of displaced pixels along the x and y directions.
Figure 10 shows the extracted motion-vector map by using the MPEG-based motion estimation scheme for each of the test video scenarios. Here, the magnitudes and directions of the MVs are represented by the arrows colored with red. In the experiment, each video frame composed of 512 × 512 pixels is divided by 32 × 32 blocks and the number of video frames for a GOP is set to be four, which means the interval between the RFs becomes four.
Fig. 10 Motion vectors of the (a) 1st frame of the ‘Case I’, (b) 2nd frame of the ‘Case I’ (Media 4), (c) 71st frame of the ‘Case I’, (d) 1st frame of the ‘Case II’, (e) 2nd frame of the ‘Case II’ (Media 5), (f) 71st frame of the ‘Case II’, (g) 1st frame of the ‘Case III’, (h) 2nd frame of the ‘Case III’ (Media 6), (i) 71st frame of the ‘Case III’.
Figures 10(a)–10(c), 10(d)–10(f) and 10(g)–10(i) show the 1st, 2nd and 71th frames with the estimated MVs of the moving objects for each case of the ‘Case I’, ‘Case II’ and ‘Case III’. As seen in Figs. 10(a), 10(d) and 10(g), there are no MVs because they are the 1st frames, which is the RFs in each GOP. However, Fig. 10(b) and 10(c), which are the 2nd and 71th frames of the ‘Case I’, show many MVs. In the 2nd frame of Fig. 10(b), most MVs for each block of the moving ‘Car’ object have the same right directions with the moving car, which means that the MVs are closely related to the ‘Car’ object heading to the right direction just at the starting stage of moving. However, for the 71th frame of Fig. 10(c), MVs with different directions exist. As seen in Fig. 10(c), even though most MVs are given by v1(2,0), which signifies that those blocks only move along the x direction with a shift amount of two pixels, two other MVs of v2(−7,-7) and v3(−4,1) also happen to be generated due to the prediction error.
For both of the ‘Case II’ and ‘Case III’, two ‘Car’ objects move fast in the opposite directions, so that MVs with different directions exist at the 2nd frames of Figs. 10(e) and 10(h). That is, most MVs of two cars are given by (4,0) and (−4,0). Moreover, as show in Fig. 10(f), at the 71th frame of the ‘Case II’, most MVs are given by v1(−4,0) for the ‘Car’ moving from the right to the left, which means the block only moves to -x direction with a shift amount of four pixels. Likewise, most MVs for the ‘Car’ moving from the left to the right are given by v2(4,0).
For the ‘Case III’ with mutually overlapped regions from the 30th to 112th video frames between two moving ’Car’ objects, the MVs of one object could be affected by the other object. As seen in Fig. 10(i), two MVs of v1(−4,0) and v2(4,0) exist, which means two cars move from the right to the left and vice versa, respectively. However, due to the prediction error, three different MVs of v3(−1,3), v4(−3,1) and v5(3,3) also exist because two cars are overlapped on the same blocks.
4.4 Block-based motion compensation
Figure 11 shows a motion compensation process, in which the MC-RF can be obtained with estimated MVs of Fig. 9. Here, A, B, …, I and A', B', …, I' represent the blocks of the RF and their compensated versions, respectively. Every block of the RF is shifted to the new positions according to the estimated amounts of pixels in both x and y directions.
Fig. 11 An example of the block-based motion compensation: (a) Motion compensation with the estimated motion vectors, (b) Motion-compensated version of the RF.
For example, as seen in Fig. 9(b), the motion vector of the block of A is given by (3,0) for the x and y directions. Therefore, the block of A', which represents the compensated version of A, can be obtained with the motion vector for the block of A.
Similarly, other blocks can be also compensated with the corresponding MVs. Figure 11(b) shows nine motion-compensated blocks of the RF.
However, during the pixel-shifting process of the blocks of the RF, several blocks happen to be overlapped, which may cause some blocks to be occluded by other blocks just like the blocks of ‘B’ and ‘D’, and ‘C’ and ‘E’ as shown in Fig. 11(b). In these cases, the pixel values of the occluded regions of X and Y are determined by those of the blocks arriving there lastly. That is, if two or more blocks move to the same area in the MC-RF, the lastly entering block may cover all of the previous blocks.
Figure 12 shows the motion-compensated intensity images of the RF with the estimated MVs for each case of the ‘Case I’, ‘Case II’ and ‘Case III’. As you can see in Fig. 12, the 2nd frame image is well compensated with the estimated MVs for each block. For the 32nd and 71st frame images, small blank regions caused by the different MVs of each block may exist in the ‘Car’ object. Therefore, an additional compensation process is needed for the blank region. Especially, for the 71st frame of the ‘Case III’, the big blank regions exist for the overlapped region of two cars due to the inaccurate MVs which are resulted from the coexistence of different object motions in the same block as shown in the red square box of Fig. 12(c).
Fig. 12 Motion-compensated object images of the 2nd, 32nd and 71st frames for each test video of (a) Case I (Media 7), (b) Case II (Media 8), and (c) Case III (Media 9).
Figure 13 shows the difference images between the MC-RF and GF images for each of the test video scenarios. As seen in the 32nd and 71st frames of Figs. 13(b) and 13(c), some line or square region may occur in the difference images between the MC-RF and the GF because of the blank region of the MC-RF. However, image differences between the frames have been well reduced by using the proposed method as seen in Fig. 13.
Fig. 13 Difference images of the 2nd, 32nd and 71st frames for each test video of (a) Case I (Media 10), (b) Case II (Media 11), and (c) Case III (Media 12).
4.5 CGHs generation for the GFs and reconstruction of 3-D objects
4.5.1 Overall procedure for generation of CGHs for the GFs
Figure 14 shows an overall flowchart to generate the CGHs for each of the GFs, which is largely composed of four processes: 1) Calculation of the instantaneous CGH pattern for the MC-RF, 2) Calculation of the final CGH pattern for the MC-RF by compensating the hologram patterns for the overlapped regions between the shifted blocks, 3) Generation of the CGH pattern for the GF by combined use of the hologram pattern for the difference image between the MC-RF and GF, and the final CGH pattern for the MC-RF.
4.5.2 Generation of the CGH pattern for the MC-RF
As shown in Fig. 14, the instantaneous CGH pattern for the RF can be generated just by shifting the B-CGHs for each block of the RF according to the estimated MVs. For instance, Figs. 9(a) and 9(b) show 9 blocks of the RF and their estimated MVs, respectively, and Fig. 15 shows a shifting process of the B-CGHs of the RF with these estimated MVs is shown.
Fig. 15 A shifting process of the B-CGHs of the RF in Fig. 9: (a), (d) and (g) represent the B-CGHs for each block of A, B and I, (b), (e) and (h) show their shifted versions with the corresponding MVs, (c), (f) and (i) show their shifted versions compensated with the hologram patterns for the blank areas, and (j) CGH for the MC-RF obtained by adding all shifted B-CGHs.
That is, for generation of the instantaneous CGH pattern for the MC-RF, the B-CGHs for each block of the RF require two-step processes. In the first step, the B-CGHs for each block of the RF are shifted according to their estimated MVs. Then, in the second step, the hologram patterns for the blank areas occurred due to the shifting process are calculated and with these the shifted B-CGHs are compensated. After the shifting and compensation operation of the B-CGHs for each block of the RF, all shifted and compensated versions of the B-CGHs are summed up together to obtain the instantaneous CGH pattern for the whole MC-RF as shown in Eq. (9).
In which IS(x, y), IRm(x, y) and IRBm(x, y) represent the instantaneous CGH obtained for the MC-RF, the shifted versions of the B-CGHs for each block of the RF with the corresponding MVs, and the hologram patterns for the blank areas of the mth block, respectively. M denotes the total number of blocks, and dx and dy represent the MVs along the x and y directions.
4.5.3 Compensation of hologram patterns for the overlapped regions
The instantaneous CGH pattern of IS(x, y) given by Eq. (9) cannot represent the final CGH version calculated for the MC-RF because some overlapped regions among the shifted blocks happen to occur during the motion-compensation process. In these regions overlapped with several shifted blocks, the same number of different pixel values as of the overlapped blocks exists together, therefore the pixel values for these overlapped regions are to be determined with those of the lastly-coming blocks.
For instance, Fig. 11(b) shows two overlapped areas between the motion-compensated blocks of Fig. 11(a). ‘X’ and ‘Y’ in Fig. 11(b) represent the intersected areas between two blocks of ‘B’ and ‘D’, and ‘C’ and ‘E’, respectively. Here, the final pixel values for the overlapped areas of ‘X’ and ‘Y’ are set to be those of the lastly-coming blocks of ‘D’ and ‘E’, respectively. However, this process may also result in some object points coming from two blocks of ‘B’ and ‘C’ to their corresponding overlapped areas to be removed.
Therefore, the final CGH for the MC-RF, IC (x, y) can be obtained by subtracting the hologram patterns for these removed object points located on each overlapped area from the instantaneous CGH pattern calculated for the MC-RF of Eq. (9), which is given by Eq. (10) as follows.
In which IRO represents the CGH pattern for the overlapped areas and it is generated with the NLUT.
4.5.4 Generation of CGH patterns for the GFs
Right after the final CGH pattern for the MC-RF of Eq. (10) is obtained, the hologram pattern for the difference image between the motion-compensated RF and the GF shown in Fig. 13 is calculated, and with which the CGH pattern for the MC-RF is finally compensated. That is, the hologram pattern for the object points in the difference image between the MC-RF and the GF is calculated, which is called ID(x, y) as shown in Fig. 14. Then, the CGH pattern for the GF, I(x, y) can be obtained just by adding the hologram pattern for the difference image between the MC-RF and the GF of ID(x, y) to the final CGH pattern for the MC-RF of IC(x, y) as shown in Eq. (11)
Here in this paper, the CGH patterns with the resolution of 1,400 × 1,400 pixels, in which each pixel size of the hologram pattern is given by 10μm × 10μm, are generated by using the intensity and depth data of the test 3-D videos of Fig. 7. Moreover, the viewing-distance and the discretization step in the horizontal and vertical directions are set to be 600mm and 30μm, respectively.
Accordingly, to fully display the hologram fringe patterns, the PFP must be shifted by 1,536 pixels (512 × 3 pixels = 1,536 pixels) horizontally and vertically [7]. Thus, the total resolution of the PFP becomes 2,936 (1,400 + 1,536) × 2,936 (1,400 + 1,536) pixels.
4.5.5 Reconstruction of 3-D objects from the CGHs
Figure 16 shows 3-D ‘Car’ object images reconstructed at the 1st, 31st, 72th and 150th frames from the corresponding CGH patterns generated with the proposed method,. As seen in the Media 13 of Fig. 16(a), a reconstructed 3-D ‘Car’ object moves fast from the left to the right direction. Here, the upper-side images of Fig. 16(a) represent the ‘Car’ objects reconstructed at the distance of 630mm, in which all object images are well focused because the ‘Car’ object was originally located at the distance of 630mm. On the other hand, the lower-side images of Fig. 16(a) show the ‘Car’ objects reconstructed at the distance of 700mm, hence they appear to be out of focused and blurred. In addition, in the Media 13 of Fig. 16(a), a perspective variation of the reconstructed ‘Car’ object is also seen because the capturing camera was located on the center location of the input plane in the 3DS Max. That is, some front and back parts of the car are seen at the 1st and 150th frames, respectively.
Fig. 16 Reconstructed 3-D ‘Car’ object images at the distances of 630mm and 700mm for each test video of (a) ‘Case I’ (Media 13), (b) ‘Case II’ (Media 14) and (c) ‘Case III’ (Media 15).
For the ‘Case II’, two ‘Car’ objects with different depths move fast in the opposite directions without any overlapped regions as seen in the Media 14 of Fig. 16(b). The upper-side images of Fig. 16(b) shows two ‘Car’ objects reconstructed at the distance of 630mm, in which one ‘Car’ object moving from the left to the right, which is called here ‘Car-1’, is well focused in all frames because the ‘Car-1’ object was originally located at the distance of 630mm. On the other hand, the other ‘Car’ object moving from the right to the left, which is called here ‘Car-2’, gets blurred because the ‘Car-2’ object was originally located at the distance of 700mm. Hence, in the lower-side of Fig. 16(b), two ‘Car’ objects reconstructed at the distance of 700mm are shown, in which the ‘Car-1’ object is blurred, whereas the ‘Car-2’ object is well focused in all frames. Moreover, perspective variations of the ‘Car’ objects are also seen just like the ‘Case I’.
For the ‘Case III’, two ‘Car’ objects with different depths move fast in the opposite directions with some overlap regions between two cars. As seen in Media 15 of Fig. 16(c), the reconstructed object images from the 1st to the 35th frame and from the 112th to 150th frame look almost as same as those of the ‘Case II’ because two ‘Car’ objects have no overlapping regions. However, the reconstruct images from the 36th to the 113th frame are affected by blurred object images of other ‘Car’ objects because they are overlapped in the same area. That is, for the lower-side of the 72nd frame of Fig. 16(c), some rear part of ‘Car-2’ is well reconstructed at the distance of 700mm however, the reconstructed images of the front part are affected by the blurred images of the ‘Car-2’ object.
5. Performance analysis of the proposed method
Figures 17(a)–17(c) and 17(d)–17(f) show the computed comparison results of the number of calculated object points and the calculation time for one-object point of the conventional NLUT, TR-NLUT, MC-NLUT and proposed MPEG-NLUT methods, respectively for each video scenario of ‘Case I’, ‘Case II’ and ‘Case III’. Table 1 also summarizes the detailed comparison results.
Fig. 17 Comparison results: (a)-(c) Numbers of calculated object points, (d)-(f) Calculation times for one-object point in the conventional NLUT, TR-NLUT, MC-NLUT and proposed methods for each test video of ‘Case I’, ‘Case II’ and ‘Case III’.
Table 1. Average numbers of calculated object points and average calculation times for one-object point in the conventional NLUT, TR-NLUT, MC-NLUT and proposed methods for each test video of ‘Case I’, ‘Case II’ and ‘Case III’
For the ‘Case I’ scenario, in which a ‘Car’ object moves fast along the pathway from the left to the right, the differences in the object points between the previous and current frames were set to be larger than 50% of the object points of the current frames, so that the numbers of calculated object points for these difference images get much larger than those of the current frames. In other words, the numbers of object points to be calculated in the TR-NLUT and MC-NLUT must be increased much more than that of the original NLUT in cases where there are more than 50% image differences between consecutive video frames.
As seen in Fig. 17(a) and Table 1, the average number of calculated object points of the original NLUT is calculated to be 7,462, whereas those of the TR-NLUT and MC-NLUT are estimated to be 12,009 and 9,896, respectively, which means that the average numbers of calculated object points of the TR-NLUT and MC-NLUT are rather increased by 60.94% and 32.63%, respectively compared to the original NLUT instead of being decreased.
Here, the MC-NLUT appears to have less number of calculated object points than the TR-NLUT. This result comes from the fact that some redundant object data between consecutive video frames could be additionally removed in the MC-NLUT with a motion-compensation process contrary to the TR-NLUT in which the absolute image differences between consecutive video frames are just extracted.
On the other hand, in the proposed MPEG-NLUT method, the test video frames of ‘Case I’ are grouped into a set of GOPs, in which each GOP is composed of one RF and three GFs. Here, the CGHs for the RFs in each GOP are calculated with the TR-NLUT. That is, the B-CGHs for each block of the RF of a GOP are computed by combined use of the corresponding B-CGHs for those of the previous GOP and the TR-NLUT. However, the CGHs for three other GFs are generated by combined use of the CGH for the MC-RF and the CGHs for the difference images between the MC-RF and each of the GFs. Therefore, in the proposed method, the numbers of calculated object points for three other GFs could be greatly reduced because the object’s motions between the RF and each of the GFs are precisely compensated based on the block-based motion estimation and compensation process, which signify that the difference images between the MC-RF and each of the GFs can be kept to be very small.
For the ‘Case I’, the average number of calculated object points of the proposed method is estimated to be 5,262, which the smallest number compared to those of the conventional methods mentioned above. That is, the average number of calculated object points of the proposed method has been reduced by 29.48%, 56.19% and 46.83%, respectively compared to each of the conventional NLUT, TR-NLUT and MC-NLUT methods.
Furthermore, the average calculation times are estimated to be 4.717ms, 6.868ms, 5.382ms and 3.749ms, respectively for each of the conventional NLUT, TR-NLUT, MC-NLUT and proposed MPEG-NLUT methods as seen in Fig. 17(d) and Table 1. In other words, the average calculation time of the proposed method has been reduced by 20.51%, 45.41% and 30.34%, respectively compared to each of the conventional NLUT, TR-NLUT and MC-NLUT methods.
For the ‘Case II’, since two ‘Car’ objects move fast in the opposite directions without any overlapped regions, the number of calculated object points and the image differences between two consecutive video frames become almost doubled compared to those of the ‘Case I’ as shown in Fig. 17(b), which means the image differences between two consecutive video frames become much bigger than 50% of the object points of the current frame.
As mentioned above, two ‘Car’ objects move fast in the opposite directions, so that the motion vectors with different directions exist in the ‘Case II’. However, the motion vectors of one ‘Car’ object may not be affected by those of the other because there is no overlapped region between them. Therefore, for the ‘Case II’, the total number of calculated object points is almost doubled compared to the ‘Case I’ as shown in Figs. 17(a) and 17(b) and Table 1.
Here, for the ‘Case II’, the average number of calculated object points of the proposed method is calculated to be 10,355 whereas those of the conventional NLUT, TR-NLUT and MC-NLUT methods are estimated to be 14,217, 23,248 and 18,550, respectively, which means the average numbers of calculated object points of the proposed method has been reduced by 27.17%, 55.46% and 44.18%, respectively compared to each of the NLUT, TR-NLUT and MC-NLUT methods.
Moreover, the average calculation times are also estimated to be 4.766ms, 7.021ms, 5.497ms, 3.786ms, respectively for each of the conventional NLUT, TR-NLUT, MC-NLUT and proposed MPEG-NLUT methods as shown in Fig. 17(e) and Table 1. That is, the average calculation time of the proposed method has been reduced down by 20.55%, 46.07% and 31.12%, respectively compared to each of the conventional NLUT, TR-NLUT and MC-NLUT methods.
On the other hand, for the ‘Case III’, two ‘Car’ objects move fast in the opposite directions with some overlapped regions. The only difference between the ‘Case II’ and ‘Case III’ is whether the overlapping regions between two objects exist or not. In the ‘Case III’, overlapped regions between two objects may occur from the 30th to the 112th frames. Therefore, the motion vectors of one ‘Car’ object may be affected by those of the other one in the overlapped blocks.
As seen in Fig. 17(c) and Table 1, the numbers of calculated object points start to reduce from the 30th to 75th frames in all methods because overlapped regions between two ‘Car’ objects get increased, whereas they are increasing again from the 75th to 112th frames since the overlapped regions get decreased.
Now, for the ‘Case III’, the average number of calculated object points of the proposed method is calculated to be 10,293 whereas those of the NLUT, TR-NLUT and MC-NLUT are estimated to be 13,982, 22,912 and 19,273, respectively, which means that the average numbers of calculated object points of the proposed method has been reduced by 26.38%, 55.07% and 46.59%, respectively compared to each of the NLUT, TR-NLUT and MC-NLUT method.
In addition, the average calculation times are estimated to be 4.697ms, 6.760ms, 5.519ms, 3.826ms, respectively for each of the conventional NLUT, TR-NLUT, MC-NLUT and proposed MPEG-NLUT methods as shown in Fig. 17(f) and Table 1. That is, the average calculation time of the proposed method has been reduced by 18.56%, 43.41% and 30.68%, respectively compared to each of the conventional NLUT, TR-NLUT and MC-NLUT methods.
In summary, the overall results of Figs. 17(a)–17(c) and Table 1 show that the average numbers of calculated object points of the TR-NLUT and MC-NLUT methods have been increased by 60.94%, 63.52%, 63.87%, and 32.63%, 30.48%, 37.84%, whereas those of the proposed method have been decreased by 29.48%, 27.17%, 26.38%, respectively compared to the original NLUT method for each scenario of ‘Case I’, ‘Case II’ and ‘Case III’.
Figures 17(a)–17(c) and Table 1 also reveals that the number of calculated object points and the calculation time of the proposed method for three test video scenarios have been reduced to 8,637 and 3.787ms, respectively on the average, which means that the proposed method has obtained 27.34%, 55.46%, and 45.70% reduction in the average number of calculated object points and 19.88%, 44.98% and 30.72% reduction in the average calculation time compared to those of the conventional NLUT, TR-NLUT and MC-NLUT methods.
Table 1 shows the computed results on the detailed processing times for motion estimation, compensation and CGH calculation for each of the ‘Case I’, ‘Case II’ and ‘Case III’ in the proposed method. That is, motion estimation, motion compensation and CGH calculation times have been found to be 0.219ms, 0.092ms, 3.438ms and 0.118ms, 0.094ms, 3.575ms and 0.118ms, 0.093ms, 3.614ms, respectively for each of the ‘Case I’, ‘Case II’ and ‘Case III’, which means that portions of the preprocessing times for motion estimation and compensation in the total calculation time have been found to be minimal values of 8.30%, 5.60% and 5.52%, respectively for each of ‘Case I’, ‘Case II’ and ‘Case III’.
As seen in Table 1, there might be a tradeoff between the preprocessing time and the reduction rate of the number of calculated object points in the proposed method. That is, if we employ a performance-enhanced motion estimation and compensation algorithm, the corresponding object points to be calculated can be effectively reduced whereas the preprocessing time for motion estimation and compensation gets increased due to its accurate processing. This means a careful selection of the motion estimation and compensation algorithm would be required because the total calculation time must include the preprocessing time.
Finally, aforementioned good experimental results mentioned above confirm that in the proposed method, the number of calculated object points and the resultant calculation time required for generation of video holograms of fast-moving 3-D objects have been significantly reduced compared to those of the conventional NLUT, TR-NLUT and MC-NLUT methods. In particular, the conventional TR-NLUT and MC-NLUT have been analyzed to get no improvement in their computational speed for the case of the 3-D objects moving fast and making more than 50 image differences between two consecutive video frames.
In addition, real-time generation of video holograms of fast-moving 3-D objects with this proposed MPEG-NLUT method on graphic-processing-units (GPUs) may be expected, because the recent results of the GPU-based NLUT system have shown 300-fold enhancement of the computational speed compared to that of the conventional central-processing-unit (CPU)-based approach [16]. This means the calculation time for one video frame can be reduced down to the order of milliseconds.
6. Conclusions
In this paper, a new MPEG-NLUT has been proposed for rapid generation of video holograms of fast-moving 3-D objects by using a block-based motion estimation and compensation scheme. With this method, image differences between the motion-compensated RF and each of the GFs could be greatly reduced, and this has resulted in the computational speed of the proposed method to be greatly enhanced. Experimental results confirm that the average number of calculated object points and the average calculation time of the proposed method have found to be decreased down to 27.34%, 55.46%, 45.70% and 19.88%, 44.98%, 30.72%, respectively compared to those of the conventional NLUT, TR-NLUT and MC-NLUT methods.
Acknowledgments
This work was supported by the National Research Foundation of Korea (NRF) grant funded by the Korea government (MSIP) (No. 2013-067321). The present Research has been partially conducted by the Research Grant of Kwangwoon University in 2014.
References and links
1. C. J. Kuo and M. H. Tsai, Three-Dimensional Holographic Imaging (John Wiley, 2002).
2. T.-C. Poon, Digital Holography and Three-Dimensional Display (Springer, 2007).
3. M. Lucente, “Interactive computation of holograms using a look-up table,” J. Electron. Imaging 2(1), 28–34 (1993). [CrossRef]
4. J. Weng, T. Shimobaba, N. Okada, H. Nakayama, M. Oikawa, N. Masuda, and T. Ito, “Generation of real-time large computer generated hologram using wavefront recording method,” Opt. Express 20(4), 4018–4023 (2012). [CrossRef] [PubMed]
5. J. Jia, Y. Wang, J. Liu, X. Li, Y. Pan, Z. Sun, B. Zhang, Q. Zhao, and W. Jiang, “Reducing the memory usage for effective computer-generated hologram calculation using compressed look-up table in full-color holographic display,” Appl. Opt. 52(7), 1404–1412 (2013). [CrossRef] [PubMed]
6. K. Hosoyachi, K. Yamaguchi, T. Ichikawa, and Y. Sakamoto, “Precalculation method using spherical basic object light for computer-generated hologram,” Appl. Opt. 52(1), A33–A44 (2013). [CrossRef] [PubMed]
7. S.-C. Kim and E.-S. Kim, “Effective generation of digital holograms of three-dimensional objects using a novel look-up table method,” Appl. Opt. 47, D55–D62 (2008). [CrossRef] [PubMed]
8. S. C. Kim, J. M. Kim, and E.-S. Kim, “Effective memory reduction of the novel look-up table with one-dimensional sub-principle fringe patterns in computer-generated holograms,” Opt. Express 20(11), 12021–12034 (2012). [CrossRef] [PubMed]
9. S.-C. Kim, J.-H. Kim, and E.-S. Kim, “Effective reduction of the novel look-up table memory size based on a relationship between the pixel pitch and reconstruction distance of a computer-generated hologram,” Appl. Opt. 50(19), 3375–3382 (2011). [CrossRef] [PubMed]
10. S.-C. Kim, J.-H. Yoon, and E.-S. Kim, “Fast generation of three-dimensional video holograms by combined use of data compression and lookup table techniques,” Appl. Opt. 47, 5986–5995 (2009). [CrossRef] [PubMed]
11. S.-C. Kim, W.-Y. Choe, and E.-S. Kim, “Accelerated computation of hologram patterns by use of interline redundancy of 3-D object images,” Opt. Eng. 50(9), 091305 (2011). [CrossRef]
12. S.-C. Kim, K.-D. Na, and E.-S. Kim, “Accelerated computation of computer-generated holograms of a 3-D object with N×N-point principle fringe patterns in the novel look-up table method,” Opt. Lasers Eng. 51(3), 185–196 (2013). [CrossRef]
13. D.-W. Kwon, S.-C. Kim, and E.-S. Kim, “Memory size reduction of the novel look-up-table method using symmetry of Fresnel zone plate,” Proc. SPIE 7957, 79571B (2011). [CrossRef]
14. Z. Yang, Q. Fan, Y. Zhang, J. Liu, and J. Zhou, “A new method for producing computer generated holograms,” J. Opt. 14(9), 095702 (2012). [CrossRef]
15. S.-C. Kim, X.-B. Dong, M.-W. Kwon, and E.-S. Kim, “Fast generation of video holograms of three-dimensional moving objects using a motion compensation-based novel look-up table,” Opt. Express 21(9), 11568–11584 (2013). [CrossRef] [PubMed]
16. M.-W. Kwon, S.-C. Kim, and E.-S. Kim, “Graphics processing unit-based implementation of a one-dimensional novel-look-up-table for real-time computation of Fresnel hologram patterns of three-dimensional objects,” Opt. Eng. 53(3), 035103 (2014). [CrossRef]
17. D.-W. Kwon, S.-C. Kim, and E.-S. Kim, “Hardware implementation of N-LUT method using Field Programmable Gate Array technology,” Proc. SPIE 7957, 79571C (2011). [CrossRef]
18. Q. Zhang and K. N. Ngan, “Segmentation and tracking multiple objects under occlusion from multiview video,” IEEE Trans. Image Process. 20(11), 3308–3313 (2011). [CrossRef] [PubMed]
19. D. Le Gall, “MPEG: a video compression standard for multimedia applications,” Commun. ACM 34(4), 46–58 (1991). [CrossRef]
20. A. Barjatya, “Block matching algorithms for motion estimation,” in Technical Report, Utah State University (2004).
