## Abstract

Benefiting from rapid development of imaging sensor technology, modern optical technology, and a high-speed computing chip, the star tracker’s accuracy, dynamic performance, and update rate have been greatly improved with low power consumption and miniature size. The star tracker is currently one of the most competitive attitude measurement sensors. However, due to restrictions of the optical imaging system, difficulties still exist in moving star spot detection and star tracking when in special motion conditions. An effective star tracking method based on optical flow analysis for star trackers is proposed in this paper. Spot-based optical flow, based on a gray gradient between two adjacent star images, is analyzed to distinguish the star spot region and obtain an accurate star spot position so that the star tracking can keep continuous under high dynamic conditions. The obtained star vectors and extended Kalman filter (EKF) are then combined to conduct an angular velocity estimation to ensure region prediction of the star spot; this can be combined with the optical flow analysis result. Experiment results show that the method proposed in this paper has advantages in conditions of large angular velocity and large angular acceleration, despite the presence of noise. Higher functional density and better performance can be achieved; thus, the star tracker can be more widely applied in small satellites, remote sensing, and other complex space missions.

© 2016 Optical Society of America

## 1. INTRODUCTION

Along with the evolution of imaging sensor technology, image processing algorithms, and computing chip function, the star tracker has been one of the most promising attitude measurement devices widely used in space missions such as Earth observation, deep-space exploration, and celestial navigation for its high accuracy, optimized dynamic performance, increasing update rate, low power consumption, and miniature size. In accordance with the latest research results, and product manuals of related institutes and companies, the accuracy of the star tracker can currently reach up to $1\u2013{10}^{\prime \prime}$ (2.54–25.4 cm) ($3\sigma $) [1–12]. The mass of the star tracker can reach 0.1–3 kg. The update rate can reach 1–10 Hz, and even can be up to 100 Hz [13,14] in maximum.

The star tracker has absolute attitude determination and is drift free. These are supported by its working principle. The star tracker considers stars as its measuring targets. On one side, the vector $w\mathrm{i}$ of the star image in the star tracker coordinate system can be calculated by combining the information of the principal point and the focal length. On the other side, star direction vector $v\mathrm{i}$ in the celestial coordinate system is very accurate, and remains almost unchanged after many years of astronomical observations. The navigation star catalog is stored in the star tracker and comprises thousands of star direction vectors. Through star identification, the vectors in the two coordinate systems can be connected. When the number of identified stars is more than two, the optical attitude matrix $A\mathrm{q}$ in the inertial space of the star tracker can be determined using the QUEST algorithm by solving the relationship between vector $w\mathrm{i}$ and $v\mathrm{i}$.

However, dynamic conditions bring great challenges to star extracting and star tracking due to the restrictions of the optical imaging system and the existing star tracking strategy. The process of star extracting and tracking not only becomes difficult, it may result in failure as well, inevitability affecting subsequent functions of the star tracker such as star identification and attitude determination. So an effective star extracting algorithm and a star tracking strategy particularly need to be studied, enabling the star tracker to perform better.

Processing algorithms at the star image level have attracted considerable attention recently, and the dynamic problems have been partially solved [15–21]. However, there are remaining research difficulties, and research needs to go a step further, as follows:

- (1) Star tracking and extracting may have problems under certain conditions such as large angular velocity or large angular acceleration, though star tracking algorithms have been discussed in Refs. [22–24] by using angular velocity prediction. Solutions are needed to avoid losing tracking and to provide an effective position and size of the region of interest for detecting the star spot when the angular velocity has a sudden change.
- (2) The noise surrounding a star spot region may sometimes be mistaken as a star spot when in tracking mode, and this may affect the results.

An effective star tracking method based on optical flow analysis for star trackers is proposed in this paper. The features of this method include the following: it can acquire accurate star location region, effectively avoid the noise of the image sensor, and be highly reliable even under conditions of high angular velocity and high angular acceleration. Optical flow analysis between two frames is adopted to conduct star capture and tracking. The extended Kalman filter (EKF) is combined with star vectors in the star tracker coordinate system to determine the angular velocity to provide information for region prediction of the star spot. Compared to former related algorithms, this method has advantages when the angular velocity is relatively high, the star spot is hard to extract, and the motion of the star tracker is not uniform.

## 2. STAR TRACKING METHOD UNDER DYNAMIC CONDITIONS

We propose an effective star tracking method based on an adaptive tracking window and the EKF, as illustrated in Fig. 1. The tracking reference window can be dynamically adjusted according to the information of optical flow and the change of angular velocity. It can provide a search scope to avoid failure in star image extraction. The EKF is also used to estimate the angular velocity, which could supply predictive information for star tracking.

#### A. Adaptive Reference Window Setting for Star Tracking

The adaptive reference window setting, which includes size and position, considers information of angular velocity and optical flow. It ensures the extraction in a limited window, so that the star image processing time can be reduced. Meanwhile, accurate position or appropriate size can avoid mistaken star identification, and wrong attitude and angular velocity determination.

In the process of an adaptive reference window setting, we need to determine the position of the star spot using a spot-target-based optical flow method and then determine the size of the reference window according to the estimated angular velocity.

The preliminary star position determination method is as follows. The brightness of one star point $(x,y)$ at time $t$ is defined as $E(x,y,t)$. When the star tracker is in motion mode, the brightness of this particular star point is expressed as $E(x+{d}_{x},y+{d}_{y},t+{d}_{t})$ at time $t+{d}_{t}$. It is considered constant [25] as shown in Eq. (1), and this is the basis of the derivation,

When ${d}_{x}$, ${d}_{y}$ and ${d}_{t}$ are small enough, the equation can be obtained in Eq. (2), as follows:

Abbreviations ${E}_{x}$, ${E}_{y}$, and ${E}_{t}$ are used for expressing the partial differentiations of image brightness with respect to $x$, $y$, and $t$, respectively. $u$ is equal to ${d}_{x}/{d}_{t}$, and $v$ is equal to ${d}_{y}/{d}_{t}$. Then, we can obtain the relationship in Eq. (3), as follows:

There are several approaches to estimate $u$, $v$ [26,27]. Consider that the optical flow of the star image belongs to sparse optical flow. An approach based on Lucas and Kanade’s method [28] is adopted in Eq. (4). ${q}_{1}\sim {q}_{n}$ are the pixels inside a rectangular window. There are strict conditions for using Lucas and Kanade’s method. The main shortcomings in star tracking can be summarized in one problem: that is, the motion may not be in accordance with the hypothesis of small movement. To avoid failed calculation brought by large movement, an image pyramid decomposition method in multi-resolution [29] as shown in Fig. 2 is adopted. Pyramid decomposition conducts down sampling from coarse to fine in multi-resolutions. The motion of the object on the image can be reduced through reducing the size of the star image; thus, the “small movement” hypothesis can be satisfied. Two layers ($L=2$) of the pyramid are enough for the experiments in this paper:

Owing to the existence of movement, the values of optical flow around the star spot are different with other positions. Thus, the vector synthesized by $(u,v)$ can be used to conduct thresholding to distinguish star spots from the noise or the background. To avoid the influence brought by the occasional discontinuous optical flow result, the dilation operation of mathematical morphology is adopted in addition to ensure the effective region determination. The concrete implementation is similar with the dilation operation of a two-dimensional gray image [30].

Figure 3(a) is a raw display of the result of optical flow analysis, and Fig. 3(b) includes the thresholding result, and reference window setting process.

#### B. Angular Velocity Optimal Estimation for Region Prediction of the Star Spot

To obtain a better performance, angular velocity information is also needed to be combined with the optical flow analysis result so that accurate star spot tracking can be performed under conditions of large angular velocity and the existence of noise.

### 1. Angular Velocity Measurement

There are several ways to realize angular velocity measurement by the star tracker only. Liebe *et al.* [31] proposed a concept of a stellar gyroscope, which could eliminate the need for conventional inertial gyros and revolutionize attitude determination systems on all spacecraft. Feasibility analysis and operation mode design regarding this possibility were discussed. Singla *et al.* [32] presented two different algorithms for the estimation of angular rates in the absence of gyro rate data. The first used an attitude-dependent dynamical model, and the other used rapid update rates of star cameras and star vectors in the sensor coordinate system. Models and simulations were described. Bar-Itzhack [33] divided different common algorithms for estimating the angular rate of spacecraft into the derivative approach and the estimation approach. The connections between the two approaches were analyzed. Crassidis [34] and Jo *et al.* [35] proposed angular velocity determination methods that could be directly derived from the body vector measurements based on the least-squares approach and the Wahba problem separately. Liu *et al.* [36] and Rao *et al.* [37] adopted the Kalman Filter to achieve angular velocity estimation in real time without information on the star reference vectors or attitude.

The angular velocity estimation method directly through star vectors does not need the process of star identification and attitude estimation. It needs only to use the star vector $w(t)$ in the star tracker coordinate, thus making it more convenient. The angular velocity can be expressed as Eq. (5) [36]:

${\overline{\sigma}}_{n}^{2}\equiv 2{\sigma}_{n}^{2}/\mathrm{\Delta}{t}^{2}$, and ${\sigma}_{n}^{2}{I}_{3\times 3}$ is the measurement noise covariance of the star vector $w$, where ${I}_{3\times 3}$ is a $3\times 3$ identity matrix. $[w{(t)}_{n}\times ]$ is the cross-product matrix to $w{(t)}_{n}$.

### 2. Angular Velocity Optimal Estimation Based on Extended Kalman Filter

The EKF is adopted to avoid the noise effect on the estimation accuracy. We use ${\omega}_{1}$, ${\omega}_{2}$, ${\omega}_{3}$ as the state vectors:

The basic discretization form of the EKF is shown in Eq. (6):

The state equation can be expressed as Eq. (7):

Then, the measurement equation can be expressed as Eq. (8) based on Eq. (5). This expression is explicit with the concise measurement equation:

Additionally, $\phi $ and $H$ are Jacobian matrices that denote the partial derivative of $f$ and $h$ with respect to $x$ separately. ${\phi}_{k/k-1}$ and ${H}_{k}$ are $3\times 3$ identity matrices.

${\gamma}_{k-1}$ and ${\chi}_{k}$ are considered as Gaussian white-noise processes, the means and covariance functions of which are assumed to agree with Eq. (9):

The values of ${Q}_{k-1}$ and ${R}_{k}$ in this method are adjusted according to the value of $(u,v)$ and the number of available stars in real time. Thus the method can be adapted to either dynamic or static conditions. When in stationary motion conditions, reducing the value of ${Q}_{k-1}$ or increasing the value of ${R}_{k}$ can increase the confidence of the prediction model and avoid the disturbance brought by the occasional false identified star spot. When in static or large acceleration conditions, increasing the value of ${Q}_{k-1}$ or reducing the value of ${R}_{k}$ can increase the confidence of the measurement result, and ensure the accurate attitude determination and effective star tracking. Considering that the accuracy of the star tracker is 5 arcsec, the accuracy of the angular velocity obtained by the star tracker is around 0.0014°/s. Therefore, in the dynamic conditions, the value of ${Q}_{k-1}$ is set as 0.002°/s in the following experiments, and the value of ${R}_{k}$ is set eight times as large as the value of ${Q}_{k-1}$. While in static or large acceleration conditions, the values of ${Q}_{k-1}$ and ${R}_{k}$ are exchanged.

## 3. EXPERIMENTS

The star tracker used in the experiment is developed by Tsinghua University as shown in Fig. 4. The parameters of the star tracker are listed in Table 1.

The real night sky experiments were conducted at the *National Astronomical Observatories, Chinese Academy of Sciences* on 15 December 2015 (Beijing Time). The observation targets are real stars. The star tracker is fixed on a turntable as shown in Fig. 5, and the rotation angular velocity of the star tracker could be adjusted along with the controlling of the turntable. Thus, the star tracking performances with the common-used method and the proposed method are compared. The working range and accuracy of the star tracker can be verified.

Figures 6 and 7 display the gray value between two adjacent star images. The gray value is quantified from 0 to 255. Through the images, we can see that if the turntable remains stationary, the star tracker moves along with the rotation of the earth; the angular velocity is so small that it is hard to observe the motion of the star spot on the image plane. If the star tracker rotates along with the turntable, the star spot moves apparently. If angular acceleration exists, the image position of a star moves nonuniformly as shown in Fig. 8, and is hard to predict and track.

The optical flow based on the gray gradient is effective in motion distinguishing of the star image. Figure 9(a) is the result of optical flow analysis between two adjacent star images. The small yellow arrows denote the gray gradient field and can be the detection basis of a star spot in the two star images. To reveal the relationship between the results obtained by optical flow analysis and the true star spot, the selected gray gradient field is added to the gray star image in Fig. 9(a). It must be noted that the optical analysis is conducted by two star images, so the selected gray gradient fields appear in pairs, wherein one belongs to the former star image and the other belongs to the latter one. The star image in Fig. 9(a) is a combination result by gray gradient fields and gray star image for visual display. In Fig. 9(b), the optical flow with color display is shown. It can be seen that the regions detected by optical flow analysis are all identified star spots. The continuous results of the optical flow analysis are shown in Visualization 1.

Based on the star image analysis and star spot position tracking results, the angular velocity can be estimated through the EKF. The angular velocity measurement results are shown in Fig. 10. Dynamic performance is the main focus, so the measurement ranges are set at 1.4776°/s, 1.6856°/s, 1.9018°/s, and 2.1192°/s separately. In the experiment process, the turntable rotates repeatedly. The process involves three main motion modes, which include rotating at a constant angular velocity, static state, and accelerating state. Since the turntable is set as rotating across one axis ($y$ axis), the obtained angular velocity of this axis is particularly shown. The angular acceleration of this axis can reach $7.5731\xb0/{\mathrm{s}}^{2}$.

When the maximum angular velocity is 2.1192°/s, the measurement error of the angular velocity is calculated as shown in Fig. 11. The statistics of the measurement error without the EKF is 0.0059°/s ($1\sigma $), while it has been enhanced to 0.0024°/s ($1\sigma $) with the EKF. Figure 12 displays the comparison results between the common-used star tracking method and the proposed star tracking method.

## 4. CONCLUSIONS

An effective star tracking method is proposed in this paper. The method uses spot-target-based optical flow between adjacent star images to distinguish the region of the star spot and obtain an accurate star spot position. Star vectors and the Kalman filter are combined to determine the angular velocity information. The experiment results have verified that the proposed method is drift free, highly accurate, effective, robust, and of great significance to the spacecraft control system. The method proposed in this paper can realize effective attitude determination when angular velocity is 2.1192°/s or even higher, and the angular acceleration is $7.5731\xb0/{\mathrm{s}}^{2}$. It is an advanced indicator of an active pixel sensor CMOS star tracker. If exposure time is reduced, and the image processing algorithm is further optimized, the slew rate up-limitation can reach 3–4°/s.

Compared to the existing star tracking method, this approach in this paper can improve the accuracy of star extraction, avoid noise interference, increase correct identification rate, achieve accurate tracking, and address the impact brought by large angular velocity and large angular acceleration.

## Funding

National Natural Science Foundation of China (NSFC) (61605099, 61377012, 51522505); China Postdoctoral Science Foundation (2015M570091, 2016T90090).

## REFERENCES

**1. **U. Schmidt, T. Fiksel, A. Kwiatkowski, I. Steinbach, B. Pradarutti, K. Michel, and E. Benzi, “Autonomous star sensor ASTRO APS: flight experience on Alphasat,” CEAS Space J. **7**, 237–246 (2015). [CrossRef]

**2. **U. Schmidt, C. Elstner, and K. Michel, “ASTRO 15 star tracker flight experience and further improvements towards the ASTRO APS star tracker,” in *AIAA Guidance, Navigation, and Control Conference and Exhibit* (American Institute of Aeronautics and Astronautics, 2008), paper 6649.

**3. **U. Schmidt, “ASTRO APS-the next generation Hi-Rel star tracker based on active pixel sensor technology,” in *AIAA Guidance, Navigation, and Control Conference and Exhibit* (American Institute of Aeronautics and Astronautics, 2005), p. 5925.

**4. **L. Blarre, J. Ouaknine, L. Oddos-Marcel, and P. Martinez, “High accuracy Sodern star trackers: recent improvements proposed on SED36 and HYDRA star trackers,” in *AIAA Guidance, Navigation, and Control Conference and Exhibit* (American Institute of Aeronautics and Astronautics, 2006), p. 6046.

**5. **D. L. Michaels, “Ball aerospace star tracker achieves high tracking accuracy for a moving star field,” in *Aerospace Conference* (IEEE, 2005), pp. 1–7.

**6. **L. W. Cassidy, “HDOS HD-1003 star tracker,” Proc. SPIE **2466**, 93–99 (1995). [CrossRef]

**7. **F. Boldrini, D. Procopio, S. P. Airy, and L. Giulicchi, “Miniaturised star tracker (AA-STR) ready to fly,” in Proceedings of the 4S Symposium: Small Satellites, Systems and Services (ESA SP-571), La Rochelle, France, 2004, p. 571.

**8. **M. Betto, J. L. Jørgensen, P. S. Jørgensen, and T. Denver, “Advanced stellar compass deep space navigation, ground testing results,” Acta Astronaut. **59**, 1020–1028 (2006). [CrossRef]

**9. **L. Maresi, R. Noteborn, O. Mikkelsen, R. Nielsen, and T. Paulsen, “A compact autonomous medium resolution, high accuracy star tracker for earth remote sensing spacecraft,” in *4th ESA International Conference on Spacecraft Guidance, Navigation and Control Systems* (European Space Agency, 2000), pp. 531–536.

**10. **T. E. Paulsen and L. Maresi, “Calibration and verification of the TERMA star tracker for the NEMO satellite,” in *AIAA Space Conference and Exposition* (American Institute of Aeronautics and Astronautics, 2000), p. 5338.

**11. **P. Oosthuizen, S. Fellowes, C. Collingwood, S. Lalani, A. Cropp, and S. Gleason, “Development and on-orbit results of the SSTL low cost commercial star tracker,” in *AIAA Guidance, Navigation and Control Conference and Exhibit* (American Institute of Aeronautics and Astronautics, 2006), p. 6045.

**12. **T. Sun, F. Xing, and Z. You, “Optical system error analysis and calibration method of high-accuracy star trackers,” Sensors **13**, 4598–4623 (2013). [CrossRef]

**13. **A. Katake and C. Bruccoleri, “StarCam SG100: a high-update rate, high-sensitivity stellar gyroscope for spacecraft,” Proc. SPIE **7536**, 753608 (2010). [CrossRef]

**14. **F. Xing, Z. You, and T. Sun, “Method for determining attitude of star sensor based on rolling shutter imaging,” U.S. patent 20140232867A1 (21 August 2014).

**15. **T. Sun, F. Xing, Z. You, X. Wang, and B. Li, “Smearing model and restoration of star image under conditions of variable angular velocity and long exposure time,” Opt. Express **22**, 6009–6024 (2014). [CrossRef]

**16. **L. Ma, F. Bernelli-Zazzera, G. Jiang, X. Wang, Z. Huang, and S. Qin, “Region-confined restoration method for motion-blurred star image of the star sensor under dynamic conditions,” Appl. Opt. **55**, 4621–4631 (2016). [CrossRef]

**17. **T. Sun, F. Xing, Z. You, X. Wang, and B. Li, “Deep coupling of star tracker and MEMS-gyro data under highly dynamic and long exposure conditions,” Meas. Sci. Technol. **25**, 085003 (2014). [CrossRef]

**18. **J. Yan, J. Jiang, and G. Zhang, “Dynamic imaging model and parameter optimization for a star tracker,” Opt. Express **24**, 5961–5983 (2016). [CrossRef]

**19. **L. Ma, D. Zhan, G. Jiang, S. Fu, H. Jia, X. Wang, Z. Huang, J. Zheng, F. Hu, W. Wu, and S. Qin, “Attitude-correlated frames approach for a star sensor to improve attitude accuracy under highly dynamic conditions,” Appl. Opt. **54**, 7559–7566 (2015). [CrossRef]

**20. **L. Kazemi, J. Enright, and T. Dzamba, “Improving star tracker centroiding performance in dynamic imaging conditions,” in *IEEE Aerospace Conference* (IEEE, 2015), pp. 1–8.

**21. **J. Zhang, Y. Hao, L. Wang, and D. Liu, “Studies on dynamic motion compensation and positioning accuracy on star tracker,” Appl. Opt. **54**, 8417–8424 (2015). [CrossRef]

**22. **M. A. Samaan, D. Mortari, and J. L. Junkins, “Recursive mode star identification algorithms,” IEEE Trans. Aerosp. Electron. Syst. **41**, 1246–1254 (2005). [CrossRef]

**23. **J. Jiang, G. Zhang, X. Wei, and X. Li, “Rapid star tracking algorithm for star sensor,” IEEE Aerosp. Electron. Syst. Mag. **24**(9), 23–33 (2009). [CrossRef]

**24. **T. Ye and F. Zhou, “Autonomous space target recognition and tracking approach using star sensors based on a Kalman filter,” Appl. Opt. **54**, 3455–3469 (2015). [CrossRef]

**25. **J. L. Barron, D. J. Fleet, and S. S. Beauchemin, “Performance of optical flow techniques,” Int. J. Comput. Vis. **12**, 43–77 (1994). [CrossRef]

**26. **B. K. P. Horn and B. G. Schunck, “Determining optical flow,” Artif. Intell. **17**, 185–203 (1981). [CrossRef]

**27. **G. Fasano, G. Rufino, D. Accardo, and M. Grassi, “Satellite angular velocity estimation based on star images and optical flow techniques,” Sensors **13**, 12771–12793 (2013). [CrossRef]

**28. **B. D. Lucas and T. Kanade, “An iterative image registration technique with an application to stereo vision,” in *Proc. 7th International Joint Conference on Artificial Intelligence (IJCAI)* (Morgan Kaufmann, 1981), pp. 674–679.

**29. **J.-Y. Bouguet, *Pyramidal Implementation of the Affine Lucas Kanade Feature Tracker Description of the Algorithm* (Intel Corporation, 2001).

**30. **T. Sun, F. Xing, Z. You, and M. Wei, “Motion-blurred star acquisition method of the star tracker under high dynamic conditions,” Opt. Express **21**, 20096–20110 (2013). [CrossRef]

**31. **C. C. Liebe, K. Gromov, and D. M. Meller, “Toward a stellar gyroscope for spacecraft attitude determination,” J. Guid. Control Dyn. **27**, 91–99 (2004). [CrossRef]

**32. **P. Singla, J. L. Crassidis, and J. L. Junkins, “Spacecraft angular rate estimation algorithms for star tracker-based attitude determination,” Adv. Astronaut. Sci. **114**, 1303–1316 (2003).

**33. **I. Y. Bar-Itzhack, “Classification of algorithms for angular velocity estimation,” J. Guid. Control Dyn. **24**, 214–218 (2001). [CrossRef]

**34. **J. L. Crassidis, “Angular velocity determination directly from star tracker measurements,” J. Guid. Control Dyn. **25**, 1165–1168 (2002). [CrossRef]

**35. **S. Jo, Y. Choi, and H. Bang, “Optimal angular velocity estimation of spacecraft using only star tracker measurements,” J. Guid. Control Dyn. **38**, 342–346 (2015). [CrossRef]

**36. **H. Liu, J. Yang, W. Yi, J. Wang, J. Yang, X. Li, and J. Tan, “Angular velocity estimation from measurement vectors of star tracker,” Appl. Opt. **51**, 3590–3598 (2012). [CrossRef]

**37. **G. N. Rao, T. K. Alex, and M. S. Bhat, “Incremental-angle and angular velocity estimation using a star sensor,” J. Guid. Control Dyn. **25**, 433–441 (2002). [CrossRef]