1. Introduction

Measurement of vibrations and displacements of structures under dynamic excitations or impacts is an important topic in structural engineering [1]. Movement of structures is related to the strains and stresses in the structure; therefore, to its security level [2]. Moreover, modal analysis and damping time provide important information about mechanical behavior of the different structures.

Despite the importance of displacement measurements, there has not been any good solutions found for all of the frequency range. For low frequencies ($\sim 80\mathrm{Hz}$), linear variable differential transformers provide direct value of the displacement. However, they must be fixed to the structure and anchored to an external structure, which must be stable enough to avoid noise in the sensor measurement. These arrangements are not always possible due to security reasons or terrain constraints; therefore, non-contact methods are preferred.

Accelerometers are often used to analyse the dynamics of a vibrating body. Although they measure by direct contact, their mass is small enough not to modify the natural vibration of the structure they are measuring. Their dynamic range and acquisition rate can be very high, so they are adequate for structural modal analysis. Nevertheless, double integration of the signal is not always possible and thus, absolute information about displacement cannot be easily obtained [3]. Laser vibrometers and radar interferometers [4–7] provide a good solution for measuring displacements in all range of frequencies. Unfortunately, these devices tend to be very expensive and are not cost-effective for many applications. Additionally, Doppler based technologies measure velocities instead of direct displacements so numerical integration is, again, needed.

Vision based systems have emerged as a good alternative for measuring real displacements in structures [8]. However, limitations in temporal and spatial resolution make them inapplicable to many problems of great interest like analysis of vibrations under accidental actions [9], where frequencies in the mid-range (50–500 Hz) are of utmost importance.

To the best of our knowledge, no paper in the literature describes image based methods for measuring submillimetric oscillations at that range of frequencies. Recently, some of the authors of this paper presented a novel technique for directly measuring the displacement of a vibrating column with a high-end scientific high speed camera (X-PRI from AOS Technologies AG) [9]. The method showed to be robust and capable of measuring all parameters of the main vibration mode, i.e. amplitude, frequency and damping constant. Images were acquired at 1000 fps so the method allows measuring direct displacement of the structure in the low and mid-range frequencies. Although the presented optical setup is simpler and cheaper than traditional systems [4–7], the achieved spatial resolution was relatively low. Additionally, scientific high-resolution fast cameras cannot always be considered as being low-cost devices and their availability is not widespread.

Vision based systems with high speed cameras appear as a good solution provided that the problem of limited spatial resolution can be solved. In this paper, we propose a method for object tracking with sub-pixel accuracy that solves this problem. Additionally, the method can be implemented with low-cost devices; thus, being a close to optimal solution to the problem of direct measuring displacements in the mid-frequency range.

The main problem of using digital cameras to quantitative measure small displacements in structures is due to hardware constraints. Resolution of the sensor is usually linked to the processing speed. Actual image systems have severe limitations to simultaneously capture, process and store large amount of data. This fact impedes both very high temporal and spatial resolution. In the case of structural analysis, natural frequencies of civil structures are typically below 100 Hz. Although it is not a very high rate, cameras working slower than 250 fps are considered low-end; thus, their performance is limited, typically offering VGA resolution or even lower, [10] which is not accurate enough to measure small amplitude vibrations.

Depending on the particular application, improving the resolution of the camera may be uneconomical or even impractical. However, one can take advantage of some “a priori” information about the system and, afterwards, use image processing techniques to increase the performance of the system. Thus, software methods may be used to compensate limitations in the hardware.

Sub-pixel techniques allow increasing the nominal resolution of the image sensors. Basically, these techniques consist of capturing and digitally post-processing scenes containing targets of known shape. These targets are recognized and extracted from the scene. Then, through smart interpolation of their different features, it is possible to decrease the uncertainty in the object position, even below the basic pixel unit. Therefore, a fully automated tracking method with sub-pixel accuracy includes two basic steps (1) target recognition, and (2) location through features extraction [11,12]. From both tasks, target recognition is the most complicated since it requires of a proper analysis and segmentation of the image. This process can be simplified and accelerated by using specific targets that are easy to recognize [13]. Among all possible targets elliptical objects including circles are usually preferred since its shape is preserved under lateral shifts and rotations.

The first step of all algorithms for object location in the scene consists of target isolation from the background image that allows obtaining a distinguishable “blob” object. This step is usually performed through binary thresholding, hard-clipping, or edge extraction. Once the target is isolated several techniques can be applied for sub-pixel location of the blob [11,14,15]. Among them those based on centroid calculation seem to be the most accurate and reliable of methods [16,17].

Centroid detection algorithms provide good results when the target illumination is symmetric. However, their performance can be seriously affected in presence of irregular illumination and noise. Other methods based on image intensity, like those based on image correlation or template matching, may be affected by similar problems [18]. These drawbacks become evident in the particular characteristics of images from fast cameras. Under constant illumination, an increase in the frame rate imposes low exposure levels, so captured images suffer from low contrast and high noise level.

Here we are interested in tracking vibrations in civil structures [9], so all measurements are taken outdoors. This fact imposes serious restrictions on the scene illumination. The tracked target may receive direct daylight, which is unstable due to the presence of clouds, changing angle of the sun, and uncontrolled presence of shadows. Additionally, the camera cannot always be situated in an optimal position due to security or terrain restrictions. All these conditions limit the quality of the image, and impose severe restrictions on the target selection.

Contour detection methods for object tracking are a reliable and robust alternative to centroid-based methods. Under ideal conditions their sub-pixel detection capability is not as good as these last techniques [11,12] since only the target borders are used and the inner structure is neglected. However, contour based methods are more robust since they are not as affected by noise in the image. Additionally, the detected contour can be fitted to a known geometrical shape, which provides additional information along with the target location. In our case, we have selected a target composed by a black elliptical contour line over a white background. Highly-contrasted binary targets permit the application’s very aggressive image processing algorithms, such as local binarization, thus correcting changes in light incidence angle or irregular illumination over the scene.

The novelty of our proposal is to demonstrate the possibility of direct measurement of structural micro-vibrations in a wide frequency range. The method here presented is highly cost-effective since it uses a commercial digital pocket camera. In particular, we use a standard Casio Exilim EX-FH20, which is able to capture high speed video. This camera is available in the market for less than $400 and is able to acquire images at 240, 420, and 1000 fps by reducing the spatial resolution of the camera from $3456\times 2592\mathrm{px}$ (still mode) to $480\times 360$, $224\times 168$ and $224\times 54\mathrm{px}$, respectively. The use of such low-end technology requires intensive image processing, but results are accurate and reliable, avoiding the need of numerical integration of the signal. An additional advantage of the method is that it allows direct observation of the phenomenon permitting a better interpretation of the result.

The paper is developed as follows. In Section 2, we describe the methodology and define the main variables. In Section 3, the accuracy of the method is demonstrated through numerical analysis. A simple lab experiment is presented in Section 4 in order to prove the method. The performance of the technique outdoors is checked in Section 5 where we use the measure of the vibrations of a steel column after an impulsive impact. Main parameters of the vibration are calculated and compared with results obtained through dynamic analysis made by a Finite Element Model (FEM). Finally, in Section 6, the main conclusions are outlined.

2. Method

Sub-pixel methods are widely known and academic and commercial applications can be easily found. Usually, these methods are based on the numerical calculation of the centroid coordinates of an object by using its complete structure or its contour. In our case, we are only interested in this last method, but restricted to objects of known shape. Hence, the observed pixelated image can be fitted to the theoretical model which defines this shape and, therefore, the location accuracy is statistically increased (see Fig. 1).

 

Fig. 1. Detection of a target (black line) is done by a set of pixels, here marked with a grid. A displacement of $1/2$ pixel to the left (gray line) is detected by a change in some of the detection pixels, now painted in gray. Although the structure has not moved a whole pixel, the geometrical bindings imposed on the target allow tracing the new contour and obtaining the new position.

Download Full Size | PDF

Among all possible targets, the most appropriate are those which preserve the topology at all possible movements [5]. The simplest one which fulfills this characteristic is an ellipse with general equation

The coefficient $B$ equal to zero indicates that the ellipse’s axes are parallel to the coordinate axes. The center position $(x_c,y_c)$, and the long and short axes $(a,b)$ can be determined as [19]:

In case of $B$ being different to zero in (1), we position the ellipse by making a rotation of angle ($-\theta$) (angle between the major axis with the horizontal one) in order to set ellipse’s axes parallel to coordinate ones. In that case, the angle can be obtained through:

The center of the ellipse provides localization of the object inside the image plane. Orientation of the ellipse informs about in-plane rotations, whereas the axis lengths may both be associated to real changes in the ellipse shape (deformations) or provide information about rotations around an axis contained in the image plane. This kind of movements will be seen as a change in the perspective and, consequently, in the apparent eccentricity.

To proceed with the detection, the target is attached to the object under study and a video sequence is registered. The sequence is processed frame-by-frame. Convenient thresholding permits the isolation of the target contour and tracking is performed by fitting the extracted line to Eq. (1). In this work, we only focus our attention in tracking lateral shifts, i.e. detecting the ellipse center. Coming back to Fig. 1, the change in only one pixel changes the statistics of the whole set. Geometrical restrictions imposed by the Eq. 1 (the contour must remain an ellipse) induces small changes in the calculated center in Eq. (2) although the target displacement is smaller than one pixel, so we increased the nominal resolution of the sensor. According to previous works in [11], the resolution of this method is around 0.08 px.

3. Numerical Simulations

The accuracy of the method has been numerically tested before performing real experiments. To this end, we have applied the sub-pixel method on circular targets. A simple noise model has been introduced in the simulation in order to see how the performance is affected.

Low-end cameras, like the Casio here used, are not specialized, but compact automatic devices, which give little control over the recording parameters. Therefore, aside of limitations in spatial and temporal resolution, which can be conveniently addressed, the use of compact cameras presents other problems, which also may limit the performance of the detection method. These problems include focusing accuracy, high ISO-noise and video compression algorithms.

Inaccurate focusing and consequent image blurring causes the contour lines to be diffused and, after an image thresholding, it may results in lines thickening. After this, processing noise coming from low illumination appears as salt-and pepper and is randomly distributed through the image. Majority of this noise can be easily removed with a median filter or a morphological opening [20] and only those dots in contact with the target will remain. In this case, the problem can be reduced to a local thickening/thinning of the contour lines.

Video compression is the effect that most degrades the image. Typically, consumer cameras use AVI-JPEG compression algorithms in order to avoid excessive file sizes. Roughly speaking, this video compression algorithm consists of a quantization of the discrete cosine transform on predefined pixel blocks in an image. Each of the blocks is considered independently of the adjacent structures; thus, the final compressed image may have a tiled appearance. The main difference with the standard MPEG algorithm is that AVI-JPEG compression is done frame-by-frame so distortions are not propagated in time.

JPEG algorithm with very high compression rates may produce a strong distortion in complex images. Nevertheless, scenes used in our application just contain a fine elliptical line with slow variations between frames. Thus, after compression, the line will appear broken and noise corrupted, but with a still clear recognizable shape. This tile-effect decreases the fitting accuracy since increases the dots dispersion.

The calculation of the target position and orientation is done by taking the position of each black pixel on the image and fitting the whole set to an analytical ellipse. Therefore, any pixel outside the theoretical line will contribute to the increase of the fitting error. In this sense, the effect of discontinuities of the line due to compression or increased line thickness should be considered as particular cases of a more general case of dots misplacement.

This noise effect has been modelled here by corrupting the coordinates of each dot by a uniformly distributed random noise distribution. Notice that the different situations above described (image blurring, ISO noise and JPEG corruption)–can be reduced to a problem of line thickness and their difference is just a question of dots density. In this sense, we believe that our noise model is accurate enough to determine the theoretical limits of our proposal.

Let us consider the coordinate of each pixel in the contour $(x,y)$. Following our model, these dots will be randomly shifted according to:

The numerical simulations have been implemented as follows: different size targets consisting on circular contour of 1 px thickness have been implemented on a discrete mesh and then least square fitted to (1). The generating analytical function is then horizontally shifted 1 full pixel in steps of 0.005 px and the resulting contours are consecutively fitted to (1) to re-calculate the new center positions. The process is repeated for diameters from 10 px to 200 px and for different dispersion levels, according to Eq. (5). Absolute differences between the theoretical and estimated positions are calculated. For each diameter-cycle the positioning error distribution is obtained. The accuracy of the method is established in three times the standard deviation of the error distribution. Although this is a very conservative criterion, estimation of the position above this error level guarantees the correct positioning of the target. Results of the simulations for noise-free target ($w=0$) and for $w=2$ and $w=5$ are represented in Fig. 2.

 

Fig. 2. Accuracy of the sub-pixel method for different dispersion errors. The rough appearance of the graph is due to the discrete nature of the calculations.

Download Full Size | PDF

We can see in Fig. 2 that for circular targets of diameter larger than 100 px, the localization error is below 0.2 px even for high noise. If one manages to make an adequate image filtering and obtaining a target contour of a 1 px thick line, which is equivalent to a free of noise target, localization can be lower than 0.05 px for diameters larger than 30 px and below 0.02 px for target diameters larger than 100 px. This means an improvement of 20 to 50 times over the nominal resolution of the camera sensor. Therefore, an adequate pixel to millimeter ratio may provide micrometric accuracy by using simple devices.

4. Validation Results

In order to check the real performance of our proposal, a moving circular target sequence has been captured simultaneously by the Casio camera and a professional camera AOS-XPri. Casio recording mode was set to 420 fps with a spatial resolution of $224\times 168\mathrm{px}$ while the AOS camera was set to 250 fps at $800\times 560\mathrm{px}$. This camera used a Navitar Zoom 7000 lens in order to focus the object while the Casio used the built-in objective.

A circular target with 5 cm of diameter was placed at 10 m from both cameras and was manually displaced horizontally with a micro-metric screw. The total displacement was 1 mm left and right. Since the movement is not perfectly controlled, no additional information from the trajectory can be used to diminish the error; therefore, the accuracy estimation is done in the worst possible case. Conversion factor for this distance was $1.25\mathrm{px}/\mathrm{mm}$ for the AOS camera and $0.42\mathrm{px}/\mathrm{mm}$ for the Casio.

In Fig. 3(a) (Media 1) and 3(b) (Media 2), we represent the captured videos by the AOS and the Casio cameras, respectively. In the first case, the captured sequence is stored without compression; thus, a sequence of 10 s takes around 1 GB. In the second case, the sequence is stored in AVI-JPEG RGB video, so each frame is compressed and, consequently, deteriorated. A 10 s sequence from the pocket camera needs less than 70 MB of storage, so it is faster to manage and process, but at the cost of degrading the image quality. Because the total displacement of the target (1 mm) is at the resolution limit of both cameras, the movement is hard to see in both sequences. In order to allow eye detection of this movement, one must move the video slider by hand causing to jump from the beginning to the middle and to the end of the sequence.

 

Fig. 3. Video sequence for object tracking. Both sequences have been taken from 10 m distance. Target diameter is 5 cm. (a) Sequence from AOS camera, 250 fps, $100\mathrm{px}\times 100\mathrm{px}$ and no compression 8 bits AVI format (Media 1). (b) Sequence from CASIO camera, 420 fps, $35\mathrm{px}\times 35\mathrm{px}$ and AVI-jpeg RGB format (Media 2). This last sequence has been magnified and equalized in order to improve the visualization.

Download Full Size | PDF

Detection and tracking of the target is done off-line. One point from the center of the target is selected and a convenient threshold for this frame is calculated according to the background level. The binary image is then cleaned through an opening filter and the contour is detected through boundary detection and tracing [20]. The final one-pixel thick contour is then fitted to Eq. (1) and its center is calculated through Eq. (2). The obtained point is used for threshold determination in the next frame.

Although the eye can barely appreciate the target shift, the method is able not only to detect the image displacement, but also the inner structure of the movement. In Fig. 4 we show the horizontal coordinate calculated from the sequence for both cameras. One can see that, despite the differences in the resolution and image quality, both are able to detect the object position. As it is expected, information from the pocket camera is noisier, but the signal is still recognizable. Notice that both cameras detect even the small pauses in the hand-rotated screw. The sub-pixel accuracy of the method can be determined from the last part of the signal, where the target is static. The absolute detection threshold can be estimated by taking three times the standard deviation of the position during the last 1 s interval; thus, obtaining a real resolution of 0.15 mm for the pocket camera and 0.03 mm for the professional one. In both cases, the method is able to see variations of around 0.05 px. By considering that our captured targets have diameters of 24 px for the Casio and 63 px for the AOS camera, the accuracy values obtained are congruent with the the theoretical values presented in Fig. 2 for noise-free targets.

 

Fig. 4. Target position obtained from sequence in Fig. 2(a) AOS and Fig. 2(b) Casio.

Download Full Size | PDF

This experiment clearly shows that the pocket camera is able to measure with great accuracy. We would like to emphasize that the movement was measured from a distance of 10 m and, for this camera, the obtained angular resolution is 2 arc sec or, equivalently, 1 mm at 100 m, making it suitable for use in long distance measurements. Although the AOS camera is almost four times more accurate and 20 times cheaper, thus, being more cost effective for many applications.

5. Vibration Measurement in Real Structures

The sub-pixel method with the pocket camera has also been applied to the measurement of the main vibration parameters of a column in a small structure. We selected a hollow steel column of 2.1 m high composed by two welded beams. As can be seen in Fig. 5(a) the two ends of the column are pinned to the floor and to emergency stairs. A steel ball of 0.44 kg was used as projectile for the impact. The ball was mounted in a pendulum of 1.5 m hanging from the upper part of the column. By releasing the ball from a distance of 90 cm from the column, we achieved an impact at $2.3\mathrm{m}/\mathrm{s}$ [see Fig. 5(b)]. The vibration produced by the impact was registered by the Casio camera focusing on a target attached to the column.

 

Fig. 5. (a) Measured structure. (b) Position of the camera and pendulum. (c) Scheme of the experiment and model use for the Finite Element simulation.

Download Full Size | PDF

Before starting the experiment, the setup was implemented in a FEM using LS-Dyna code [21]. Both column and ball were modeled with solid elements. Steel was modeled as an elastic-plastic material with a Young modulus of 210 Gpa, Poisson modulus 0.3, density $7850\mathrm{kg}/{\mathrm{m}}^3$, and yield strength 275 Mpa. A similar experiment was done in [8] with other another camera and different detection techniques. Since the mathematical model is the same, details will not be repeated here. In Fig. 5(c), we depict a schematic representation of the model.

We calculated the displacement of the column at the impact height through the numerical analysis. As it is explained in [8], the vibration can be considered monomodal so it can be easily described as an attenuated oscillation

The real displacement of the column after an impulsive load was measured with the Casio camera placed at a lateral side of the column. Since the total movement was predicted to be very small, we placed the camera at 15 cm from the target and the macro mode was used [see Fig. 5(b)] Furthermore, since the expected vibration frequency was not very high the acquisition speed was set to 240 fps at a resolution of $480\times 360\mathrm{px}$.

The circular contour of the target that can be seen in Fig. 5(b) can be extracted by hard-clipping and boundary tracing. Fitting this contour to the ellipse in Eq. (1), we obtain the centre coordinates, which are depicted in Fig. 6. Movement of the column occurs in transversal direction and the impact can be clearly seen in the movement of the $x_c$ coordinate. One can also appreciate that the target slightly moves in the vertical direction with a delay of 0.04 s with respect to the beginning of the horizontal vibration. This residual movement comes from transmitted vibration from the floor to the camera. Undoubtedly, this effect will add some noise to the signal, but will not interfere with the registered target movement.

 

Fig. 6. Coordinates of the tracked center of the target.

Download Full Size | PDF

As it was done in the previous section, the accuracy of the method was checked by analyzing a sequence of the static target. Both $x$ and $y$ coordinates oscillate around a fixed position about 0.07 px ($3\times \mathrm{STD}$). By considering that the whole target has a size of 2 cm, the obtained resolution is $17\mathrm{px}/\mathrm{mm}$; hence, giving a spatial resolution of around 4 µm.

Results obtained from the coordinate $x_c$ were fitted to the Eq. (5). Fourier transform of the obtained signal is shown in Fig. 7(a). As it happened with the simulation, the movement can also be considered monomodal. In Fig. 7(b), we depict an example of the obtained data together with the fitting graph (a monomodal attenuated oscillation). In Table 1, we show the values obtained for three different trials together with the 95% confidence bounds. The value for the constant $\varphi$ has been omitted from the analysis since it is different for each case and it does not add any relevant physical information.

 

Fig. 7. (a) Fourier transform of the vibration of the column obtained from the horizontal position of the target (see Fig. 5). (b) Data fitting to Eq. (5).

Download Full Size | PDF

Table 1. Results from $F=\mathbf{\text{the Target Position to an Attenuated Movement}}$

View Table

By comparing these results with those from the simulation one can see that there is a good coincidence between the model and the experiment. Only a displacement in the frequency is observed but this can be explained since the model does not take into account all boundary conditions of the real structure (damping of the soil, real structure, material, welding properties, etc).

We would like to underline that, with a pocket camera, one can measure vibrations with an accuracy of few micrometers, which is much higher than other more expensive devices. Although distance from the camera to the element is really short, this can be improved by adapting a telescopic system to the camera and placing it further from the experimental setup.

6. Conclusions

In this paper, we have presented a simple and very efficient method for microvibrations measurements that improves others from the references. We have demonstrated the possibility of using a pocket camera for high speed object tracking. On the one hand, the application of contour-based methods reduces the influence of noise and irregular illumination on the target. On the other hand, the use of targets of known geometrical shapes allows obtaining sub-pixel accuracy through least square fitting of the object contour. To the best of our knowledge, this is the first method that is able to measure micrometric vibrations in a wide frequency range. Additionally, the proposal is very cost-effective since it can be implemented on a pocket camera and with no specific software.

Two experiments prove the usability of these cameras for target tracking. In one experiment, we used a high frame-rate with low spatial accuracy. The object was situated relatively far from the cameras and it was moved without a predefined trajectory. The pocket camera showed to have a very good performance even in situation where a naked eye cannot appreciate any movement. We achieved an accuracy improvement of more than one order of magnitude, being able to record variations of 2 arc sec in long distances or few micrometers in short distances. In the second experiment, the target was attached to a real structure. The vibration of a steel column after an impulsive load was measured and their main parameters determined with high accuracy.

We would like to underline that these methods can be also used in high-end cameras, where spatial resolution and image quality is higher, therefore, higher gain is expected. Nevertheless, high-end cameras are more expensive and their use is not advisable due to the environment (dust, impact danger, exposition to climatology). Additionally, these cameras usually require of a complementary computer to download the sequences and external power suppliers. On the contrary, pocket cameras are cheaper and smaller. They use batteries and small memory cards that can be easily replaced. Thus, many preliminary trials can be done with these cameras and preserve the high end camera for final experiments.