1. Introduction

The imaging tracking technology is increasingly applied for target tracking and recognition in many emerging fields, such as advanced robotics, autonomous navigation, security surveillance and augmented reality [1,2]. Smart and compact tracking model with fast and flexible tracking strategy can play a decisive role in improving system performance. Therefore, many efforts are engaged in accomplishing rapid-response and strong-robustness imaging tracking function during the system design. At present, the multi-axis turntable has been widely used for boresight adjustment in imaging tracking applications, in which a typical case is the Pan-Tilt-Zoom (PTZ) camera [3–6]. However, the imaging tracking performance of multi-axis turntable is limited by its dynamic lag, vibration sensitivity and error accumulation. It has also been reported that the tracking component can be directly driven by an ultrasonic motor to achieve fast response and high precision [7,8]. But the coupling effect between the rotor and the stator of the ultrasonic motor will make the system design difficult to some extent.

In addition to translating or rotating the camera body as described above, an alternative approach to imaging boresight adjustment is to add an auxiliary optical device in front of the camera such that the boresight can be changed through refraction or reflection while keeping the camera stationary. Hu et al. [9] proposed an ultrafast mirror-drive vision system featured by time division and multithreading techniques to acquire multi-perspective information as two or more cameras. But such a reflective optical system is limited in that it is usually large in volume and relatively sensitive to the undesired deviation of camera boresight. Compared with reflective components, the refractive prism has the inherent advantages of compact structure, small moment of inertia and vibration insensitivity [10], which can offer great potential for boresight adjustment in imaging tracking applications. It has been demonstrated in most previous studies that prisms can be utilized to steer the camera boresight and enlarge the imaging field of view (FOV). Lavigne et al. [11] developed a step-stare imaging system based on rotation double prisms, in which the boresight could be continuously altered through the coaxial rotation of two prisms. Wang et al. [12] proposed a Risley-prism-based imaging system for super-resolution imaging and FOV extension. Tao et al. [13] established an active imaging system using Risley prisms and a scanning mirror, which can provide a flexible view for microassembly operation. These previous studies are mostly devoted to the application of prism-based imaging systems [10–15], but few are focused on the problems associated with boresight control strategy in the dynamic imaging tracking field, especially the nonlinear relationship between prism rotation and boresight adjustment.

We have ever proposed a closed-loop visual tracking system using Risley prisms, in which a radial-circumferential decoupling control strategy was demonstrated [16]. Regardless of the initial orientations of two prisms, the method can achieve boresight adjustment with satisfactory precision but at the time-consuming cost, which is not always allowed in dynamic applications. The combination of two pairs of Risley prisms has also been explored to enlarge the tracking FOV [17] or achieve high-precision tracking by a coarse-fine coupling method [18]. However, the control strategy of two cascaded prism pairs becomes more complicated and may sacrifice the real-time tracking performance of the system, which limits its applicability to the dynamic imaging tracking field. Another related work is on the iterative refinement method (IRM) for finding the inverse solutions of Risley prisms given the desired beam pointing direction, which takes several milliseconds to calculate the prism orientations with the accuracy better than 0.1 µm [19]. Since the IRM has never been applied for Risley prisms in imaging tracking, more efforts need be concentrated on the modelling of Risley prism-based imaging system and the formulation of fast boresight control strategy to track a moving target in real time.

Based on the previous research, this paper further developed an effective imaging tracking model from a target to a rotation Risley prism pair embedded with a camera. Combined with the inverse ray tracing method and iterative refinement method, a novel boresight adjustment strategy is proposed to accomplish direct and fast control of Risley prisms for tracking a near-distance target, especially for most industrial applications within tracking ranges from a few meters to tens of meters. Moreover, the influence of system parameters on the dynamic characteristics of the tracking system, as well as the imaging tracking accuracy, are investigated to reveal the internal mechanisms for Risley-prism-based imaging tracking prototype.

This paper is organized as follows. In Section 2, the rigorous theoretical model of the Risley-prism-based imaging tracking is established, followed by the systematic analysis on the influence of camera and prism parameters on the model. In Section 3, the boresight adjustment process for fast locating a target is investigated, including the demonstration of the boresight adjustment principle and the discussion on the dynamic characteristics of the system. In Section 4, the imaging tracking experiments are carried out to validate the feasibility of the tracking model. Conclusions are finally drawn in Section 5.

2. Theoretical model

As illustrated in Fig. 1, the imaging tracking system consists of a CCD camera and a rotation double-prism system. Through independent rotation of two Risley prisms, the camera boresight can be altered to a specific cone to change the FOV. The parameters of two prisms, such as refractive index, wedge angle, thinnest-end thickness and clear aperture, are properly matched to ensure the camera imaging FOV at the maximum deviation angle of camera boresight.

 

Fig. 1. Schematic and coordinate diagram of the imaging tracking system using Risley prisms.

Download Full Size | PDF

The Cartesian coordinate system O_CX_CY_CZ_C and O_PX_PY_PZ_P are established as shown in Fig. 1, where the coordinate origins O_C and O_P are located at the optical center of the camera and the center of the plane side of the first prism, respectively. The Z_C-axis and Z_P-axis are oriented along the optical axis of the camera. The two Risley prisms are sequentially named as prism Π₁ and prism Π₂, each of which can rotate independently around Z_P-axis with the rotation angles defined as θ_r1 and θ_r2, and the angular velocities defined as ω_r1 and ω_r2. The pitch angle and the azimuth angle of the adjusted boresight are defined as ρ and φ, respectively.

The beam propagation model of the rotation double prism system is established to illustrate how it is used in imaging tracking. On this basis, the inverse ray tracing model is developed to obtain the light path from the imaging feedback of a target, which is essential for demonstrating boresight adjustment principle. Moreover, the influence of camera and prism parameters on the results of the inverse ray tracing process is analyzed in this section.

2.1 Modelling of the beam propagation in rotation double-prism system

Figure 2 indicates the propagation of the refracted beam in the system. Each prism is designated with thinnest-end thickness d₀, wedge angle α, refraction index n and clear aperture D_p. D₁ is the distance between the plane surfaces of two prisms, while D₂ is the distance between the target and surface 4. Without specific explanation, the parameters of the system are given as d₀ = 5 mm, α = 10°, D_p = 80 mm, n = 1.517 and D₁ = 67 mm in the following analysis in this paper.

 

Fig. 2. Schematic diagram of a beam passing through the rotation double-prism system.

Download Full Size | PDF

The normal vectors at each surface are respectively defined as N₁, N₂, N₃ and N₄ [20],

Based on the coordinates of the incident point M₁ (x_m1, y_m1, z_m1), the intersection point of the refracted beam with respect to surface 2∼5 are represented as M_i (x_mi, y_mi, z_mi), i = 2∼5 [20].

2.2 Modelling of the image-based inverse ray tracing process

Figure 3 intuitively illustrates the image-based inverse ray tracing process according to a usual pinhole camera model, where the center of camera FOV and the spatial position of the target are represented by points C and M₅, respectively. The imaging point of the target locates in point m in the image coordinate system oxyz. In this paper, the target distance in the range of few meters to tens of meters is far larger the focus length of tens of millimeters, and the actual image point of the target on the imaging plane is close to the optical center through continuous boresight adjustment. Therefore, the ideal pinhole model is used to establish the image-based inverse ray tracing model [21]. For the convenience of research, the rays are traced inversely from the imaging point m to the actual target position M₅. The incident beam vector from the target is represented by vector A₄ while the emergent beam vector is represented by vector A₀, which is also the incident beam vector of the camera. The prisms’ orientations are respectively given as θ_r10 and θ_r20 at this moment.

 

Fig. 3. Schematic diagram illustrating the inverse ray tracing process.

Download Full Size | PDF

Assuming that the coordinate of imaging point b is (x_m, y_m) and the focal length of camera is f, the vector A₀ can be derived from the following formula,

By substituting the current orientations of two prisms into Eqs. (2) and (3), the vector A₄ (x_a4, y_a4, z_a4) and the intersection point M₄ (x_m4, y_m4, z_m4) on surface 4 can be derived.

2.3 Analysis of the influence of system parameters on inverse ray tracing results

With the introduction of rotation double prisms into the camera model, the coupling effect between the camera and the prism pair must be clarified for the further optimization of the system. Two important parameters, including the focal length of the camera and the wedge angle of the prism, are taken as typical examples to reveal the influence of system parameters on the inverse ray tracing results. Without loss of generality, this method can be applied for the analysis of other related parameters as well.

In the coordinate system O_PX_PY_PZ_P, the unit vector of the emergent beam (from surface 4) and the exiting point derived by ray tracing method are given as A₄ (x_a4, y_a4, z_a4) and M₄ (x_m4, y_m4, z_m4), respectively. Since z_a4 is determined from x_a4 and y_a4 with the length constraint of A₄, and z_m4 is a constant value for a given system. Hence, only the influences of system parameters on (x_a4, y_a4) and (x_m4, y_m4) need to be taken into account. These influences are evaluated by the partial derivatives of x_a4, y_a4, x_m4 and y_m4 with respect to f, expressed as η_fax, η_fay, η_fmx and η_fmy. More comprehensive evaluation of these influences can be given by η_fa and η_fm as follows

 

Fig. 4. The influence of camera focal length f on the inverse ray tracing results. (a) The influence on the unit emergent beam vector, including η_fax, η_fay and η_fa. (b) The influence on the exiting point position, including η_fmx, η_fmy and η_fm.

Download Full Size | PDF

Similarly, assuming that f = 16 mm and other parameters remain unchanged, the attention is turned to the influences of prism wedge angle α on the inverse ray tracing results. The partial derivatives of x_a4, y_a4, x_m4 and y_m4 with respect to α are represented as η_αax, η_αay, η_αmx and η_αmy, and the comprehensive influences are respectively represented as η_αa and η_αm, as follows

 

Fig. 5. The influence of prism wedge angle α on the inverse ray tracing results. (a) The influence on the unit emergent beam vector, including η_αax, η_αay and η_αa. (b) The influence on the exiting point position, including η_αmx, η_αmy and η_αm.

Download Full Size | PDF

Note that the above analysis is performed under the condition that two prisms rotate to the initial orientations, namely θ_r10 = θ_r20 = 135°, and the analysis results may become different for other prism orientations. In order to further explore whether the prisms’ orientations will affect the partial derivatives η_fax, η_fay, η_fmx, η_fmy, η_αax, η_αay, η_αmx and η_αmy, the initial orientations of two prisms are taken as independent variables for quantitative analysis. Figure 6 depicts the partial derivatives η_fax, η_fay, η_fmx, η_fmy, η_αax, η_αay, η_αmx and η_αmy as functions of the prisms’ initial orientations with fixed camera focal length f = 16 mm and wedge angle of prisms α = 10°. The conclusion can be drawn that the inverse ray tracing results are not only affected by the structural parameters such as focal length and wedge angle, but also dependent on the prisms’ orientations. Therefore, the orientations of two prisms must be taken into account to analyze the sensitivity of the system to various error sources. With the change of prisms’ orientations, the angle between the principal section of each prism and the X_P-axis changes, which causes different trends between the X-component and Y-component of the above partial derivatives. When the orientations of two prisms are set as (θ_r10, θ_r20) = (90°, 90°), the principal sections of two prisms are parallel to the Y_P-axis. In this situation, the partial derivative η_αay reaches its maximum value while η_αax reaches its minimum value. Besides, because the exiting point position is related both to the direction of the incident beam and the position of the incident point, the system parameters have different influences on the emergent beam vector and the exiting point position even if the initial orientations of two prisms are not changed.

 

Fig. 6. The influence of prisms’ initial orientations on the inverse ray tracing results, including the different partial derivatives as functions of the prisms’ orientations: (a) η_fax, (b) η_fay, (c) η_fmx, (d) η_fmy, (e) η_αax, (f) η_αay, (g) η_αmx and (h) η_αmy.

Download Full Size | PDF

3. Camera boresight adjustment process

3.1 Boresight adjustment principle

In the imaging tracking applications, the emergent beam vector can be derived using the aforementioned inverse ray tracing method. Commonly, for a given emergent beam vector, the two-step method is effective to obtain the required rotation angles of two prisms [22]. But in the near-distance situation, there will be inevitable systematic errors introduced by using the two-step method, in which the variation of exiting point is neglected.

It is assumed that the exiting point locates at M₄ (x_m4, y_m4, z_m4), and the distance between M₄ and the center of surface 4 is defined as ${d_m} = \sqrt {x_{m4}^2 + y_{m4}^2} .$ The pitch angle ρ of the emergent beam is decomposed into X-component and Y-component, namely ρ_x and ρ_y. Figure 7 illustrates the relation between the distance d_m and the structural parameters of two prisms. It can be seen from Fig. 7(a) that the distance d_m depends on the pitch angle ρ and has nothing to do with the azimuth angle φ. Since d_m increases with the increment of ρ, the boresight adjustment deviation induced by the two-step method becomes more significant for a larger pitch angle ρ. Using the quantitative method, it can be seen from Fig. 7(b) that compare with the increase of pitch angle ρ, d_m undergoes a considerable increase with the increase of distance D₁. When D₁ is set as the initial value D₁ = 67 mm, the maximum and minimum values of d_m are respectively expressed as d_m,max and d_m,min, as depicted in Figs. 7(c) and 7(d). It can be concluded that d_m,max and d_m,min both increase with increasing α, while on the contrary, d_m,max and d_m,min gradually decrease as d₀ increases. Inspired by the above analysis, the systematic errors brought by the two-step method in near-distance situation can be effectively reduced by virtue of the optimization of system parameters in practical applications.

 

Fig. 7. The relation between distance d_m and the parameters of two prisms. (a) Distance d_m in various directions of the emergent beam. (b) The relation between the distance d_m and the pitch angle ρ. (c) d_m,max and d_m,min as functions of different wedge angle α. (d) d_m,max and d_m,min as functions of different distance d₀.

Download Full Size | PDF

In some specific cases when the target moves in a plane at the certain distance D₂, the spatial position of the target can be accurately derived by providing the imaging feedback for the ray tracing model in Section 2.2. Given the target location in space, the iterative refinement method is used to solve the rotation angles of two prisms without sacrificing the real-time performance of the system, since the method requires the calculation time within several milliseconds for the accuracy better than 0.1 µm [19]. By combining the inverse ray tracing process with the iterative refinement method as shown in Fig. 8, it is viable to accomplish direct and high-speed boresight adjustment. It should be noted that the system calibration is needed before the tracking process to obtain the camera parameters and the prism system parameters, which are necessary for the boresight adjustment process. At the first stage, the feature recognition is conducted to obtain the pixel coordinates of the target imaging point from the image information. The second stage requires the inverse ray tracing process starting with the imaging point, and ending with the emergent beam vector and the exiting point from the prism Π₂ to the target. At the third stage, the target location in space is determined from the inverse ray tracing results and the known distance D₂, which can be provided for the iterative refinement method to calculate the consequent rotation angles of two prisms. Finally the prisms are rotated according to the orientations obtained through iterative calculation, such that the camera boresight can be steered towards the target after just once adjustment. During the calculation process of the iterative refinement method, the two prisms remain stationary until the calculation is completed and the acquired orientations are used to drive the two prisms.

 

Fig. 8. Flow chart of the boresight adjustment process.

Download Full Size | PDF

Assuming that the target moves in a spiral trajectory, Fig. 9 shows the position of the center of FOV obtained by the two-step method and the iterative refinement method, respectively. It is obvious that in the near-distance tracking situation, the boresight adjustment deviation occurs due to the approximate solution by the two-step method. Meanwhile, when the target movies away from the Z_P-axis, the deviation grows with increasing pitch angle of the emergent beam, which is consistent with the analysis in the previous analysis. In this example, it takes less than nine iterations to obtain the prism orientations for any target point in the given trajectory when the allowable threshold is set as 0.1 µm. The boresight adjustment accuracy by this method is decided by the given allowable threshold according to the previous conclusions [19]. Therefore, if a given allowable threshold is set small enough, the exact solutions of prisms’ orientations can be derived to meet the requirement of high-precision target tracking.

 

Fig. 9. Comparison of the calculation accuracy of two-step method and the iterative refinement method, the first set of solutions is taken as an example. Where trajectory I represents the trajectory of the center of FOV by using the two-step method, while trajectory II represents the one by using the iterative refinement method.

Download Full Size | PDF

3.2 Dynamic characteristics of the boresight adjustment process

The dynamic characteristics of the boresight adjustment should be investigated to design the control strategy for the imaging tracking process. Considering the rotation characteristics of double prisms, the velocity of the target is respectively decomposed into the radial velocity v_r and the tangential velocity v_t, as depicted in Fig. 10. Since the pitch angle ρ of the emergent beam is determined by the deviation angle between two prisms, the radial velocity v_r of the target depends on the relative rotational speed of two prisms, expressed as Δω_r = |ω_r1 - ω_r2|. On the other hand, the azimuth angle φ is achieved by rotating two prisms at the same speed when the deviation angle of two prisms remains constant. Therefore, the tangential velocity v_t is only related to the rotational speed when two prisms rotate synchronously, namely ω_r = |ω_r1| = |ω_r2|. On this basis, the ratios Δω_r/v_r and ω_r/v_t are discussed to further study the dynamic laws of the boresight adjustment process.

 

Fig. 10. Schematic of the decomposition of target velocity along radial and tangential directions, where M₅ represents the actual target point, v represents the actual velocity of the target, v_r and v_t respectively represent the radial and tangential velocity of the target.

Download Full Size | PDF

Under the system parameters given in Section 2, Fig. 11(a) describes the change rate of Δω_r/v_r as a function of pitch angle ρ. It can be seen that the ratio Δω_r/v_r varies severely with the pitch angle ρ increasing from 0° to ρ_max, and the ratio Δω_r/v_r tends to infinity when ρ reaches ρ_max. While the ratio ω_r/v_t decreases with increasing pitch angle ρ, as shown in Fig. 11(b). When the pitch angle ρ approaches to 0°, the ratio ω_r/v_t tends to infinity as well. It can be concluded that the required rotational speeds of two prisms increase dramatically when the target moves close to the center and outer edge of the tracking region, which puts forward higher requirements for the performance of the driving motors. Apart from the relation between prism rotation and target motion, the available tracking region can be redefined according to the ratios Δω_r/v_r and ω_r/v_t for the limited speed of the driving motors. As depicted in Fig. 12, there are two restricted areas corresponding to the center and outer edge of the tracking region, where the smooth tracking cannot be accomplished.

 

Fig. 11. Ratios of the rotational velocity of the prisms to the radial velocity of the target Δω_r/v_r and the tangential velocity of the target ω_r/v_t, where ρ_max = 10.48°. (a) Δω_r/v_r, (b) ω_r/v_t.

Download Full Size | PDF

 

Fig. 12. Diagram illustrates the redefinition of the tracking region, where R1 represents the actually available tracking region for smooth tracking, R2 represents the blind zone, R3 and R4 represent the restricted areas according to ratios ω_r/v_t and Δω_r/v_r, respectively.

Download Full Size | PDF

As a case example, it is assumed that the maximum angular velocity of the prism is 50 rpm and the maximum velocity of the target is 200 mm/s. Firstly, suppose that the tangential velocity of the target reaches the maximum, in which case v_r = 200 mm/s and v_t = 0. When the distance D₂ is set to 500 mm and the angular velocity of two prisms reaches the maximum, the ratio Δω_r/v_r is calculated to be 0.05233 and the corresponding maximum pitch angle for smooth tracking is 9.57°, denoted as ρ_o1 = 9.57°. It can also be calculated that ρ_o2 = 10.38° and ρ_o3 = 10.44° with target distance D₂ = 1000 mm and 1500 mm, respectively. Similarly, when the radial velocity of the target reaches the maximum, the minimum pitch angles corresponding to different target distance are obtained, expressed as ρ_i1 = 4.371°, ρ_i2 = 1.461° and ρ_i3 = 0.88°. In fact, the smooth tracking can only be implemented within a certain range of pitch angle ρ_i and ρ_o for a given system. Moreover, it can be concluded that the available tracking region extends with increasing target distance D₂. Furthermore, motors with higher speed can increase the ratios Δω_r/v_r and ω_r/v_t to expand the actual tracking region as well. In addition, other components can be adopted for the extension of the tracking region, such as a third prism [23,24], but it also complicates the design and control of the system.

4. Experiment validation

Figure 13 shows the test platform. The white circular marker attached to the end-effector of the manipulator acts as the tracked target. All functions are integrated into the software, including image information acquisition, boresight adjustment algorithm, and serial communication.

 

Fig. 13. Experiment platform. (1) CCD camera; (2) camera bracket; (3) rotation double-prism system; (4) 3 degree-of-freedom manipulator; (5) computer; (6) motor control box.

Download Full Size | PDF

In the experiments, the maximum velocity of the end-effector of the manipulator and the maximum angular velocity of the motor are set to 50 mm/s and 150°/s respectively, with the target distance D₂ = 1000 mm and the field angle of the camera θ_c = 23.32°. The structural parameters of each prism are designed to provide sufficient imaging tracking range while facilitating the manufacturing and assembly process, among which D_p = 80 mm, α = 10° and d₀ = 5 mm. According to the analysis in Section 3.1, increasing the distance D₁ will enlarge the error caused by the change of exiting point position when tracking far-distance target based on the approximate formula of beam refraction. Considering both the minimization of distance D₁ to reduce the error in far-distance condition and the convenience of manufacture and assembly, the distance D₁ is finally set to 67 mm. However, the distance between the exiting point and the center of surface 4 is about 5 mm as analyzed in Section 3.1, which cannot be ignored under high-accuracy tracking condition, indicating that the iterative refinement method is valuable in this situation. Besides, based on the analysis in Section 3.2, the range of pitch angles for smooth tracking is between 0.37° and 10.46°. The maximum moving distances of the manipulator along x and y directions on the XOY plane are 307 mm and 255 mm, respectively.

4.1 Static tracking experiment

During the experiment process, the camera boresight is altered just one time by rotating two prisms according to the imaging feedback, and then the center of camera FOV is moved towards the target center, as shown in Fig. 14. Using the least square method (LSM), the experimental deviation distribution is described directly in Fig. 15(a), in which the red solid points represent the error distributions and the blue circle represents the experimental error distribution circle with a radius of 6.4097 pixels. As shown in Fig. 15(a), the error range obtained by LSM is plotted as a green circle with the center of (-0.0219, -0.1353) and the radius of 4.0047 pixels. In order to evaluate the accuracy of the boresight adjustment, the total deviation is defined as $\Delta e = \sqrt {\Delta {u^2} + \Delta {v^2}} $, where Δu and Δv represent the pixel deviations of target imaging point from the image center in different directions. Figure 15(b) shows the pixel deviations of 35 test points, and the root mean square error (RMSE) is calculated to be 4.0058 pixels, which is displayed with a green dotted line. The experimental deviation is mainly caused by calibration error, relative position error between camera and prism, geometric parameter errors of two prisms, and errors in motor and encoder. Regardless of all these various error sources, the experimental results still exhibit the feasibility and high-precision of the proposed method.

 

Fig. 14. Images captured before and after adjustment during the experiment, where (a)–(d) are images captured during the experiment of 4 groups among the total 35 test points.

Download Full Size | PDF

 

Fig. 15. Boresight adjustment accuracy. (a) Error distribution range. (b) Total pixel errors among 35 test points.

Download Full Size | PDF

4.2 Dynamic tracking experiment

In this experiment, the camera boresight is altered through continuous rotation of two prisms driven by servo motors in order to dynamically track a moving target. During the dynamic tracking process, the end-effect of the manipulator passes through Q₁ (-230mm, -70mm), Q₂ (-200mm, -20mm), Q₃ (-140mm, 100mm) and Q₄ (-244mm, -90mm) successively in the manipulator coordinate system. Once the camera boresight is pointed to the target, the imaging point of the target will locate at the center of the image in the ideal case. Thus the pixel deviation between the target imaging point and the image center is utilized as the feedback for the dynamic control of the tracking system. Figure 16 illustrates the closed-loop control block of the tracking system, where (u₀, v₀) and (u, v) respectively represent the pixel coordinates of the image center and the target imaging point, and (θ_r1, θ_r2) are the rotation angles of two prisms obtained by the proposed boresight adjustment principle in Section 3.1. During the experiment process, the target is locked in the central FOV of the camera as much as possible by the adjustment of camera boresight. The total pixel deviation Δe is defined as the pixel deviation between the target imaging point and image center. The root mean square error (RMSE) of the whole dynamic tracking process is used to evaluate the performance of the tracking system in dynamic tracking for near-distance target.

 

Fig. 16. Control block diagram of the dynamic tracking system.

Download Full Size | PDF

Figure 17 shows the images recorded every four seconds during the tracking process. Figure 18(a) describes the pixel coordinates of the target imaging point during the experiment. The solid lines represent the pixel coordinates of the target imaging point during the tracking process. While the dotted lines represent the coordinates of the image center, which is considered as the expected boresight projection in the image plane. The corresponding relationship between the four key points (Q₁ ∼ Q₄) and the experimental time is also shown in Fig. 18(a), from which the motion direction of the manipulator at different times can be obtained. It is obvious that the target imaging point always locates near the image center in the whole dynamic tracking process. At about ten seconds after the beginning of the experiment, there is a sudden variation of the pixel coordinates of the target imaging point as shown in Fig. 18(a), which is caused by the abrupt change of the movement direction of the target. However, it is notable that the system can perfectly adapt to this unexpected disturbance and exhibits strong robustness. Figure 18(b) shows the total pixel deviation during the whole tracking process, which is plotted as a red solid line. The green dotted line plotted in Fig. 18(b) represent the RMSE during the whole tracking process, which is calculated to be 8.487 pixels. The experimental results indicate that the tracking system is feasible and robust in near-distance dynamic target tracking applications.

 

Fig. 17. Target images captured at different times during the dynamic tracking process, where (a) ∼ (e) are five images captured at an interval of four seconds.

Download Full Size | PDF

 

Fig. 18. The experimental results of the dynamic tracking process. (a) The pixel coordinates of the target imaging point in the tracking process, and Q₁ (-230 mm, -70 mm), Q₂ (-200 mm, -20 mm), Q₃ (-140 mm, 100 mm) and Q₄ (-244 mm, -90 mm) are four points that the manipulator passes through successively at different times. (b) The total deviation of the target imaging point in the dynamic tracking process.

Download Full Size | PDF

5. Conclusion

An effective imaging tracking model from a target to a rotation Risley prism pair embedded with a camera is derived by the beam vector propagation method. A theoretical model of the Risley-prism-based imaging tracking prototype is established to illustrate the law of boresight adjustment and imaging feedback. Considering the system error introduced by using the two-step method to solve the rotation angles of double prisms, a boresight adjustment principle combined the inverse ray tracing and the iterative refinement method is further developed to accomplish direct, fast controlling of prisms for tracking a near-distance target. The analysis of dynamic characteristics of the tracking model reveals the coupling mechanisms between the imaging feedback and the prism rotation, which offers a theoretical foundation for the system optimization and the formulation of control strategy. The experimental results well validate the flexibility and robustness of the tracking prototype, providing a useful guidance for further developments of the Risley-prism-based imaging tracking in broad fields, such as industrial detection and military reconnaissance.