Automatic scratch detector for optical surface

Hong-Yan Zhang; Zi-Hao Wang; Hai-Yan Fu

doi:10.1364/OE.27.020910

1. Introduction

Machine vision is widely utilized to detect the surfaces of the materials of interest in the field of non-destructive testing (NDT) [1–3]. Most of the available approaches are designed to deal with metal [4], wood [5], textured materials [6] or living tissue in medicine and biology [7]. However, optical components, such as lenses and mirrors, are rarely involved.

Scratches, the major defects on optical surface, have serious impacts on optical system since they can not be cleaned or removed with NDT methods. The optical parameters such as refractive index, light transmittance will be changed by scratches [8]. According to the international standard of surface imperfection tolerances [9], the length of the scratch is an important indicator of the quality of optical systems or elements. Thus, the measurement precision of scratches is essential for the quality evaluation of optical system, which leads to the key problem of analyzing the scratches quantitatively. At present, for manufacturing optical components, scratch testing is mainly dominated by the naked-eyes visual testing, which is also named with human visual inspection (HVI). However, the detecting efficiency of the HVI method is very low since the workers’ visual acuity is seriously decreased by fatigue or high strength workload, and the subjective evaluation is arguable due to the acuity of the workers’ eyesight and experiences [10]. In addition, scratches have different directions and depths. In the process of detection, workers need to not only frequently change the incident angle of light but also move the glass-like surface itself, which leads to a fact that the visual inspection of scratches is the most time-consuming step when manufacturing optical components.

There are several conventional optical techniques for surface defect detection. Germer [11] and Gu [12] proposed the defect detecting method based on the polarization of light. In their method, the emphasis is put on the qualitative analysis of the existence of defects. However, there is lack of quantitative evaluation for the results of detection. Schneider et al. [13] proposed a fast defect detection method with the help of single sided diffraction. However, the imaging quality is still needed to be analysed even if the defects are highlighted successfully. Sanchez-Brea [14] proposed a wire surface defect detection method stimulated by the fact of intensity variations on the scattering cone generated. Unfortunately, the detection device needed is expensive and complex, which limits its applications.

Images of dark field are much easier to be processed and the information for quantitative defect analysis can be obtained directly. A perfect or ideal optical surface is a smooth glass-like surface, thus the scattering light at the defect will enter into the imaging system (camera). On the contrary, the light reflected by the defect-free surface will not enter into the aperture of the imaging system. In consequence, the image regions corresponding to the defects or dusts on the optical surface appear as bright white pixels. On the contrary, the dark gray pixels corresponds to the defect-free or clean region of the surface. Here we remark that the path of the light ray in dark field system is easy to be designed and verified with optical design software such as CODE V and ZEMAX.

In recent decade, lots of researches on defect detection on optical surface appeared with the birth of dark field imaging system. In 2007, Yang [15, 16] not only designed a microscopic dark field scattering imaging (MDSI) system to capture the candidate images with scratches of interest, but also designed a detection algorithm for detecting strong scratches. However, this imaging system introduces noise inevitably while it works. The algorithm may miss a lot of weak or non-notable scratches and the estimation of the length of scratches fluctuates apparently. In 2015, Li [17] proposed the image filtering and difference scheme for scratch extraction. For this method, the noise in the images can be efficiently removed. However, some weak scratches may also be removed unexpectedly. Thus the accuracy of scratches detection is not satisfactory for the practical purpose. In 2016, Tao [18] proposed a binarization method for dark field images by combining the LSD method and Hough transform directly. However, there are two disadvantages for Tao’s method: the detection result is not accurate enough and it is very sensitive to threshold parameters.

The key objective of scratch detection is locating the scratches and evaluating them. The evaluation includes counting the number of scratches and determining the length of the scratches. In machine vision, the fundamental strategy for scratches detection is to convert the detection task into line detection problem. There are some typical general line detection (GLD) algorithms based on Hough transform [19–21], LSD [22] and ED lines [23].

These GLD algorithms are not appropriate for continuous straight line segments because of three deficiencies:

they locate the scratches well but the output of detection consists of broken lines in most cases;
some extra process need to be applied to connect these line segments according to semantics;
the weak scratches are difficult to be distinguished since the values of intense of pixels in a given scratch regions may be slightly different from those in non-scratch regions.

Once the binarization method is applied, the scratches will appear as some co-linear points. In the middle part of a scratch, the co-linear points can be connected to segments. However, at both ends of the scratches, these points are more dispersed and will be difficult to be distinguished from the noise points. Moreover, the closure of the GLD outputs may not contain all the scratches, thus, locating the endpoints of each scratch is much more difficult than locating the scratch itself. Particularly, the position of the endpoints has a crucial influence on the accuracy of detection: the length is closely related with the endpoints of the line segment of interest. Moreover, the GLD algorithm may fail since some shallow scratches appear as co-linear points or short segments on the surfaces of glass-like material, which will be projected onto the corresponding digital image (eg., generated by the MDSI system).

Therefore, GLD algorithms can not be used for quantitative analysis and we can not use them directly when evaluating the glass-like surfaces with machine vision inspection (MVI) methods.

In our application, the surfaces of interest are those of neodymium glasses. This optical component is used for laser fusion system and the optical component is quite large. The dimension of the tested surface is 810 mm × 460 mm. Scratches are generated during the process of grinding, polishing and translating the optical components. The MDSI system is used as a front end for capturing digital images, which provide original information for detecting scratches on optical surface fast and accurately. The system consists of a ring light source, a zoom microscope, XY-DOF translation stage and a high resolution CCD. The ring light source allows the imaging system to capture scratches in all directions. The images collected by the microscope are clear and weak scratches can be highlighted. Since the field of view (FOV) of the microscope is small (14.42 mm × 14.42 mm), the large-scale optical component is placed on a translational platform, which allows the microscope to scan the entire surface for complete inspection. For more details about MDSI, please see Liu [24].

In this paper, an accurate and efficient algorithm is designed, implemented, verified and evaluated for detecting the scratches on the optical surface. Our scratch detection algorithm includes four consecutive steps:

Coarse Locating — determine the coarse position of the scratches;
Classifying the seeds for each scratch;
Collecting all of the points belong to scratches;
Accurate Locating — determine the accurate position of scratch endpoints.

Our main contributions lies in three folds:

The scratches and abstract line segments on binary image are modelled and represented mathematically.
A procedure is designed to connect the broken segments (the output of the GLD algorithm).
The key information, which includes location, numbers and length with satisfied accuracy, that specifying the scratches are generated automatically for evaluation.

The contents of this paper are organized as follows: Section 2 gives the fundamental notations and concepts for modelling the scratches; Section 3 describes the modules of the scratch detection algorithm with details; Section 4 gives the verification and validation of the algorithm via concrete examples; and finally conclusions are made in Section 5.

2. Notations and concepts about scratches

2.1. Image captured from MDSI system

A grayscale image is specified by a function defined on a 2-dimensional finite region. Mathematically, we have

\begin{array}{l} ℐ : Ω \to {0, \dots, 255} \\ {[x, y]}^{⊤} \mapsto ℐ (x, y) \end{array}

where

Ω = {{[x, y]}^{⊤} : 0 \leq x < W, 0 \leq y < H},

is the image region, W is the width and H is the height of the image ℐ. We use p = [x, y]^⊤ to denote the image point or pixel at location (x, y) and ℐ(x, y) = ℐ(p) to denote the intensity of the grayscale image at image point p.

In this paper, the grayscale images of large scale optical components are generated by the MDSI system and W = H = 2724 as demonstrated in Fig. 1. For easier processing, these grayscale images are binarized via an operation

ℬ_{t} : {0, 1, 2, \dots, 254, 255} \to {0, 255}

according to an adaptive threshold [18] t such that

\begin{array}{l} ℐ_{b} = ℬ_{t} \circ ℐ : & Ω \to {0, 255}, \\ p \mapsto ℐ_{b} (p) = {\begin{array}{l} 0, & for ℐ (p) \geq t; \\ 255, & for ℐ (p) < t . \end{array} \end{array}

The binarization of Fig. 1 is shown in Fig. 2.

Fig. 1 Original images from MDSI system (enhanced).

Download Full Size | PDF

Fig. 2 Binary images (enhanced).

Download Full Size | PDF

2.2. Representation of line segment and scratch

A line segment in an image is determined by its endpoints. Formally, let p_s = [x_s, y_s]^⊤ ∈ Ω be the start point and p_e = [x_e, y_e]^⊤ ∈ Ω be the end point, the corresponding line segment can be represented by

L = 〈 p_{s}, p_{e} 〉

with length

ℓ = ‖ l ‖ = ‖ p_{e} - p_{s} ‖ = \sqrt{{(x_{s} - x_{e})}^{2} + {(y_{s} - y_{e})}^{2}},

and direction vector

l = p_{e} - p_{s} .

Similarly, we use

S = 〈 ℓ; p_{s}, p_{e} 〉

to denote a non-closed scratch in a binary image, i.e., S ⊂ ℐ_b, in which a length attribution ℓ is added if compared with the line segment L. Usually, for a scratch S = 〈ℓ; p_s, p_e〉 we have

ℓ \geq ‖ p_{e} - p_{s} ‖ = ‖ l ‖

since it may be a line segment or curve.

The data structures of the line segment L and scratch S used in this paper is summarized in Table 1.

Table 1. Data Structures for Line Segment L = 〈p_s, p_e〉 and Scratch S = 〈ℓ; p_s, p_e〉

View Table | View all tables in this article

For two line segments $L_{i} = 〈 p_{s}^{i}, p_{e}^{i} 〉$ and $L_{j} = 〈 p_{s}^{j}, p_{e}^{j} 〉$ as illustrated in Fig. 3, let θ_ij be the included angle of them, then we have

sin θ_{i j} = \frac{det ([l_{i}, l_{j}])}{‖ l_{i} ‖ \cdot ‖ l_{j} ‖}

where det([a, b]) = a₁b₂ − a₂b₁ for a = [a₁, a₂]^⊤, b = [b₁, b₂]^⊤. Furthermore, the degree of proximity (DOP) of these two line segments is defined by

\begin{array}{l} {DOP}_{i j} & = DOP (L_{i}, L_{j}) = DOP (〈 p_{s}^{i}, p_{e}^{i} 〉, 〈 p_{s}^{j}, p_{e}^{j} 〉) \\ = min {‖ p_{e}^{i} - p_{e}^{j} ‖, ‖ p_{e}^{i} - p_{s}^{j} ‖, ‖ p_{s}^{i} - p_{e}^{j} ‖, ‖ p_{s}^{i} - p_{s}^{j} ‖}, \end{array}

i.e., the minimal distance between the endpoints of L_i and L_j.

Fig. 3 Two line segments with a included angle.

Download Full Size | PDF

2.3. Dark points

The set of dark points 𝒟 in the image ℐ_b is specified by

𝒟 = {ℐ_{b}}^{- 1} (0) = {p \in Ω : ℐ_{b} (p) = 0} .

The dark points are visible in the binary image and they will be concerned frequently in scratch detection.

In the presence of dusts on optical components, the grayscale images generated by the MDSI system are always polluted by noises. Obviously, this impact will be propagated to the binary images, as the random noise points illustrated in Fig. 2. Usually, The scratches are represented as line segments in the images, thus the line detection algorithms can be used to detect them. It should be noted that the concept of endpoint for a scratch is different from that of a line segment since the former is not well-defined. For a physical scratch, an “endpoint” must be definite instead of some possible points. However, for its representation in the binary image, it may be some points in a set of co-linear points and/or noise points, as shown in Fig. 2 (a). It is easy to find that the dark points may be generated from noise or scratches, viz.,

𝒟 = {points of the scratches} \cup {noise points} = 𝒟^{S} \cup 𝒟^{N}

subject to 𝒟^S ∩ 𝒟^N = ∅. A direct result of this phenomena is that the GLD algorithm may fail because of the difficulty of distinguishing endpoints from the noise points.

2.4. Performance parameter of scratch detection

Although the HVI is criticized for its low efficiency, its detecting results are widely accepted. Let ℓ_hvi be the total length of all scratches obtained by the HVI method and ℓ_mvi be the counterpart by the MVI method.

For the purpose of evaluating the detecting results, we define the error rate of scratch detection by

E = \frac{| ℓ_{mvi} - ℓ_{hvi} |}{ℓ_{hvi}} \times 100 % .

If the error rate E < E_max = 5% for a sufficiently large amount of samples of optical surface components, then the detector is Accepted as a reliable detector, otherwise it will be Rejected.

3. Scratch detecting algorithm

For the purpose of automatic and rapid processing, we designed a flowchart for scratches detecting with the help of computer vision, see Fig. 4. There are four key modules in our MVI method:

coarse locating of scratches,
classifying scratches,
removing noise point and
positioning proper endpoints.

We now give some necessary interpretations. Firstly, we take the LSD for initial or coarse positioning the scratches after loading the binary images. Then, a classification is used to find “seeds” for each scratch. After that, the RANSAC algorithm [25] is utilized to select the points belonging to the scratches. Finally, a search algorithm is applied to determine the endpoints of the scratches, calculate and output the scratch information of interest.

Fig. 4 Flowchart for detecting scratches.

Download Full Size | PDF

3.1. Coarse locating of scratches with LSD algorithm

LSD is a fast algorithm based on computing grayscale gradient with sub-pixel accuracy for detecting line segments in digital images. Compared with the line detection algorithms based on the Hough transformation, LSD algorithm have three merits: it is more faster, more accurate and does not depend on tuning parameters. The output of LSD algorithm is a set of line segments denoted by

ℒ = \cup_{i = 1}^{n} L_{i} = \cup_{i = 1}^{n} {〈 p_{i}, p_{i}^{'} 〉 : p_{i}, p_{i}^{'} \in ℐ_{b}} .

When the LSD algorithm is used to find scratches, there are two unexpected phenomena:

it usually detects several line segments for a given scratch since it relies on gradient computation heavily;
the line segments detected may be distributed on both sides of the scratch and may have different slopes, see Fig. 6.

The method of eliminating these unexpected phenomena depends on tuning the parameters of LSD algorithm.

Although there are seven built-in parameters in the LSD algorithm, we found that not all of them play important roles in our application. Our experiments show some fascinating results:

Only two parameters are essential: one is the moderate scaling parameter s, the other one is the standard deviation σ of Gaussian filter.
We may not find any line segment with default parameters in the LSD, thus it is necessary to tune the parameters carefully. Once these two parameters are well-tuned, the impact of noise can be reduced and the detection accuracy can be improved significantly.
For each scratch S, the set ℒ contains 3 ∼ 5 line segments and the positions of these line segments give a coarse localization of S.

3.2. Classifying segments from different scratches

To distinguish the line segments of different scratches, we propose a line segment classification algorithm to classify the line segments, see Algorithm 1.

Algorithm 1. Line Segment Classification, {ℒ_ctg, K} = LineSegmentClassifier(ℒ, θ_max, DOP_max, r_min).

View Table | View all tables in this article

For the design of this algorithm, there are three key elements of significance for the line segments L_i and L_j :

included angle θ_ij of L_i and L_j,
degree of proximity DOP_ij of L_i and L_j, and
scratch account ratio r_ij of L_i and L_j.

For the first element, we calculate the sine of the included angle sin θ_ij between two LSD output line segments $L_{i} = 〈 p_{s}^{i}, p_{e}^{i} 〉$ and $L_{j} = 〈 p_{s}^{j}, p_{e}^{j} 〉$ by (6). Let θ_max ∈ [0, π) be a critical angle, then we can set a threshold value sin θ_max. If |sin θ_ij| > sin θ_max, we label these two line segments as different categories of scratches. Otherwise, we regard them as “co-linear” segments, see Fig. 5. Fig. 5(a) shows two scratches S₁ and S₂ and Fig. 5(b) simulates the detecting result of LSD. The included angles are θ₂₄ = θ(l₂, l₄) and θ₂₅ = θ(l₂, l₅) respectively. Obviously, |sin θ₂₄| < |sin θ₂₅|, thus l₂ and l₅ belong to different scratches. It is necessary to verify whether L₂ and L₄ belong to the same scratch.

Fig. 5 Classification of the line segments by the included angle.

Download Full Size | PDF

Fig. 6 Classification of the line segments by the DOP.

Download Full Size | PDF

In order to verify whether these “co-linear” line segments belong to the same scratch, we calculate the DOP of them according to (7). Let DOP_max be the maximum DOP, i.e., a threshold value. If DOP_ij < DOP_max, we regard them as the same category, as shown in Fig. 6. Fig. 6(a) shows two scratches S₃ and S₄ and Fig. 6(b) simulates the detection result of LSD. We can see that the |sin θ₁₅| is small, thus we need to calculate ${DOP}_{15} = ‖ p_{e}^{1} - p_{s}^{5} ‖$ . Similarly, we compute the distance ${DOP}_{13} = ‖ p_{e}^{1} - p_{s}^{3} ‖$ between L₁ and L₃. Obviously, DOP₁₃ < DOP₁₅. Hence, compared with the threshold, we find that L₁ and L₃ belong to the same scratch. We need further verification to see whether L₁ and L₅ belong to different scratches. Otherwise, we need to check whether there is a real scratch between these two “co-linear” segments.

Firstly, we choose a rectangle R_ij which contain the line segment L_i: one of its symmetric axis is the direction vector l_i with length ℓ_i and width 5 pixels, as illustrated in Fig. 7. Fig. 7(a) simulates the detecting result of LSD. L₁ and L₅ are “co-linear”, but DOP₁₅ > DOP_max. We need to verify whether there are scratches in the rectangular area specified by the points $p_{e}^{1}$ and $p_{s}^{5}$ . We determine a rectangular area in binary image and calculate the scratches ratio r₁₅. Fig. 7(b) shows that r₁₅ = 30% < r_min = 50%. Thus, the L₁ and L₅ belong to different scratches. We then divide R_ij into n_t equal parts along l_i, see Fig. 7 (b). Formally, we have

R_{i j} = \cap_{k = 1}^{n_{t}} R_{i j}^{k}

such that

R_{i j}^{k} \cap R_{i j}^{q} = \emptyset, \forall k \neq q .

Let

δ (R_{i j}^{k})

be the indicator for

R_{i j}^{k}

such that

δ (R_{i j}^{k}) = {\begin{array}{l} 1, & \exists p \in 𝒟 \cap ℛ_{i j}^{k}; \\ 0, & otherwise . \end{array}

Let n_s be the number of small rectangles which have dark points, viz.,

n_{s} = \sum_{k = 1}^{n_{t}} δ (R_{i j}^{k}) .

We now define the scratch account ratio for the rectangle R_ij as

r_{i j} = r (R_{i j}) = \frac{n_{s}}{n_{t}} \times 100 % .

In our detection problem, we set n_t = 10 as the default value. Let r_min = 50% be the threshold. If r_ij ≥ r_min, L_i and L_j are considered to be part of the same scratch category. Otherwise, they belong to different categories, as illustrated in Fig. 7.

Fig. 7 Verification of scratches.

Download Full Size | PDF

Once the classification is finished, each category represents a scratch, and we regard each category as a scratch seed. Then, we can find the two endpoints of each scratch seed. However, as Fig. 8 shows, if we only use one certain set of the built-in parameters of LSD, the closure of scratch seed may not relates with the corresponding scratch. Fig. 8(a)∼(c) are the original binary images, they all have at least one scratch. The red parts in Fig. 8(d)∼(f) are the LSD detection results, the scratch seeds. We can see that LSD can not cover the whole scratches clearly. In consequence, it is difficult to find the proper endpoints of the scratches. This disadvantage has serious impact on the detection performance which is characterized by the total length ℓ_mvi of all scratches.

Fig. 8 LSD detection result in red color.

Download Full Size | PDF

3.3. Removing noise points with RANSAC method

In order to overcome the disadvantage mentioned above, we expand the search grid according to the output of Algorithm 1 (Line Segment Classification Algorithm). We create a rectangular region of interest (ROI), denoted by Σ, to contain the whole scratch: it covers the part detected by LSD and its extended part is along the direction of the line segment. The set of dark points 𝒟_Σ in this ROI will be

𝒟_{Σ} = Σ \cap 𝒟 .

In general, this set contain both the points in a scratch and noise points simultaneously. With the help of (9), we immediately have

𝒟_{Σ} = (Σ \cap 𝒟^{S}) \cup (Σ \cap 𝒟^{N}) = 𝒟_{Σ}^{S} \cup 𝒟_{Σ}^{N}

where

𝒟_{Σ}^{S} = Σ \cap 𝒟^{S}

and

𝒟_{Σ}^{N} = Σ \cap 𝒟^{N}

. This implies that each scratch is perturbed by noise and the existence of noise increases the difficulty of detecting scratches. It is easy to deduce that the noise lowers the performance of scratch detection and it is necessary to remove the impact of

𝒟_{Σ}^{N}

. Due to the shape propriety of the scratch, we use the RANSAC method [25] to implement the operation

𝒟_{Σ}^{S} = 𝒟_{Σ} - 𝒟_{Σ}^{N}

approximately. The outline of the procedure is as follows:

Firstly, we sort these points by their coordinates in the slope direction.
Secondly, we divide them into n_r parts such that each part contains $⌈ n_{r}^{- 1} | 𝒟_{Σ}^{S} | ⌉$ dark points where |A| denotes the number of elements in a finite set A and ⌈x⌉ denotes the the ceiling of real number x, i.e., $⌈ x ⌉ = k, k - 1 < x \leq k, k \in ℤ .$ Fig. 9 illustrates the partition of dark points in $𝒟_{Σ}^{S}$ , in which we set n_r = 3 as the default value. Fig. 9(a) emulates the detecting result of LSD. We define the rectangular area surrounded by L₁ and L₂. In Fig. 9(b), we collect all the points in this area from binary image. And, according to the number of points, we divide the region into three equal parts. In Fig. 9(c), we use RANSAC in each part. In Fig. 9(d), with the result of RANSAC, we can obtain the more precise area which contain all the scratches with less noise. Therefore, each part of the ROI 𝒟_Σ can be modeled well by a line segment even if the scratch has a little curvature since the length of each part is small enough.
Thirdly, we take the RANSAC method to fit the scratch with the line model by choosing two points randomly in each part and removing the noise points automatically.
Finally, we can obtain n_r parts ${S_{n}^{k}}_{k = 1}^{n_{r}}$ of the n-th scratch S_n, or equivalently $𝒟^{S_{n}} = \cup_{k = 1}^{n_{r}} 𝒟^{S_{n}^{k}} .$

At the same time, we also get the candidate lines L_n as the output of RANSAC procedure. Our experiments show that three parts are enough to present the existing curved scratches, as demonstrated in Fig. 10.

Fig. 9 Collecting points of straight scratches.

Download Full Size | PDF

Fig. 10 Collecting points of curved scratches.

Download Full Size | PDF

Once the dark points caused by noise is removed successfully, we can determine the structure of 𝒟^S. Suppose there are K scratches in an image ℐ_b, then (9) and (17) imply that

𝒟^{S} = \cup_{n = 1}^{K} 𝒟^{S_{n}} = \cup_{n = 1}^{K} \cup_{k = 1}^{n_{r}} 𝒟^{S_{n}^{k}} .

Note that the value of K is unknown and it can be determined automatically in Algorithm 1.

3.4. Positioning proper endpoints

Once the noise points are removed, we sort the points in 𝒟^S_n by a strategy of two steps:

Compute the slope coefficient α_n of L_n;
Sort the dark points: If |α_n| > 1, then the dark points ${p_{i} = {[x_{i}, y_{i}]}^{⊤}}_{i = 1}^{N}$ in 𝒟^S_n are sorted with increasing order by the vertical coordinates y_i, viz., $\begin{array}{l} {Order}_{y} & : 𝒟^{S_{n}} \times 𝒟^{S_{n}} \to {True, False} \\ (p_{i}, p_{j}) \mapsto {\begin{array}{l} True, & if p_{i} . y < p_{j} . y; \\ False, & otherwise . \end{array} \end{array}$ Otherwise, these points are sorted by the horizontal coordinates x_i similarly, $\begin{array}{l} {Order}_{x} & : 𝒟^{S_{n}} \times 𝒟^{S_{n}} \to {True, False} \\ (p_{i}, p_{j}) \mapsto {\begin{array}{l} True, & if p_{i} . x < p_{j} . x; \\ False, & otherwise . \end{array} \end{array}$

Then, we apply Algorithm 2 to obtain the endpoints as follows.

Algorithm 2. Proper Endpoints Positioning, {p_s, p_e} = EndpointSelector(𝒟^S_n, L_n, d_max, ℓ_min).

View Table | View all tables in this article

In Algorithm 2, the two endpoints are found one by one. Firstly, we begin with the first point p₁ as start endpoint candidate p_sc. The distance between p_i and p_i+1 are computed by

dist (p_{i}, p_{i + 1}) = ‖ p_{i + 1} - p_{i} ‖ .

If dist(p_i, p_i+1) ≤ d_max, where d_max is the maximum acceptable separation between p_i and p_i+1, then we regard that these two points are not separable and continue to calculate dist(p_i+1, p_i+2). Otherwise, such as the case where dist(p_j, p_j+1) > d_max, we calculate the distance dist(p_sc, p_j) between p_j and p_sc. This distance has two possibilities: it may be the length of a part of scratch which consists of “adjacent” points, or the diameter of a lump of noise points. It depends on the value of this distance:

If the distance is less than ℓ_min and the minimum length of the scratch should be recorded. We just abandon these points and set the p_j+1 with p_sc, and then repeat previous steps.
Otherwise we find the start point p_s, which is the last endpoint candidate p_sc, see Fig. 11. In Fig. 11(a), we assume p₁ as p_sc. Since dist(p₁₁, p₁₂) > d_max, we have to calculate dist(p_sc, p₁₁) = dist(p₁, p₁₁). This value is shorter than ℓ_min, therefore we abandon the points on the line segment $\bar{p_{1} p_{11}}$ , and regard p₁₂ as p_sc. In Fig. 11(b), since dist(p₃₉, p₄₀) > d_max, we calculate dist(p_sc, p₃₉) = dist(p₁₂, p₃₉). This value is greater than ℓ_min, hence we have p_s = p_sc = p₁₂.

Fig. 11 Seeking for start endpoint.

Download Full Size | PDF

Secondly, we determine the other endpoint similarly. However, this time we begin with the last point of the sorted points p_N.

The time complexity of Algorithm 2 is 𝒪(n_d), thus the two endpoints can be determined quickly.

3.5. Implementation of scratch detecting algorithm

With four modules constructed above, the complete scratch detecting algorithm is described in Algorithm 3, which corresponds to the flowchart in Fig. 4. To locate the scratches, we use the LSD algorithm. After classifying the scratches, we use a RANSAC procedure to collect the points in scratches. Finally, we found the endpoints of the scratches as accurately as possible. As mentioned in Algorithm 3, there are five parameters need to be set. Actually, for any similar problem, these parameter can be set easily by following process. First, according to the detection requirement, ℓ_min (the minimum length of the scratch to be recorded) is determined. Then, the degree of proximity DOP_max and the maximum acceptable separation d_max are determined based on ℓ_min. Actually, we can set

{\begin{array}{r} {DOP}_{max} = α \cdot ℓ_{min} \\ d_{max} = β \cdot ℓ_{min} \end{array}

For our images,

α \approx 3, β \geq 2 .

is an appropriate choice.

Algorithm 3. Scratch Detection for Binary Image, 𝒮 = ScratchDetector(ℐ_b, θ_max, DOP_max, r_min, d_max, ℓ_min).

View Table | View all tables in this article

The scratch account ratio r_min is similar with the duty ratio, which indicates the proportion of area that may be scratched through, thus 50% is a reasonable threshold. The last parameter θ_max is used to distinguish scratch categories by computing the difference of angle. Since the LSD results can be located at both sides of scratch, the setting of this parameter is related to ℓ_min and the width of scratch w will be

θ_{max} \geq arcsin (\frac{w}{ℓ_{min}}) .

In our problem, the default values for the five parameters are set by θ_max = 13° = 0.22689 rad, r_min = 50%, DOP_max = 50 pixels, d_max = 30 pixels, ℓ_min = 15 pixels.

4. Experiment and result analysis

4.1. Experiment results

To evaluate the performance, we applied Algorithm 3 on about 1700 images (2724 × 2724 pixels), which are generated from 7 different optical surfaces (810 mm × 460 mm for each).

During the experiment, we found that the connection between scratches plays a very important role in the final results. If several co-linear segments correspond to the same scratch, then it is necessary to connect and merge them into a long scratch. Otherwise, the length of the scratch will probably be calculated as less than the actual length. Thanks to the judging condition in Algorithm 1, we have our own standard to decide the attribution different line segments. This phenomena is clearly demonstrated in Fig. 12. This figure shows the original images, LSD test results and our test results. Fig. 12(a)∼(c) are the images with LSD detection results. The rectangular areas in Fig. 12(d)∼(f) are the area that include scratches and the black line segment is the scratch that we found. In Fig. 12(c), the middle area of the scratch is too weak, thus, the corresponding area in Fig. 12(f) is shown in co-linear points, which cannot be detected by LSD. For our algorithm, it can not only determine the position relationship between the found strong scratches successfully but also can connect them very well via Algorithm 3.

Fig. 12 Connection of scratches.

Download Full Size | PDF

Fig. 13 shows that our algorithm can find the scratches’ endpoints which coincides with the realistic situation very well. Fig. 13(a)∼(c) are the images with LSD detection results. The rectangular areas in Fig. 13(d)∼(f) are the area that include scratches and the black line segment is the scratch that was found. Due to the fact that the contrast of original image is too low, see Fig. 13(a)∼(c), the scratch (especially for the head and tail area) is discontinuous, see Fig. 13(d)∼(f). For this case, LSD can only find the seeds on the scratches. The length of these seeds is far less than the length of the scratch. In Algorithm 2, a pixel-level linear search algorithm is used to find true endpoints of scratch. Actually, Fig. 13(a)∼(c) show that there is no clear standard for the endpoint selection by the HVI method. That means for the same image but different people, an inspector person may mark different endpoints. However, by using our algorithm, the stability of the scratch detection results is good, which can be said that this is one of the advantage over human vision detection. This process also improves the accuracy of determining the total length of the scratch concerned.

Fig. 13 Find real endpoints of scratches.

Download Full Size | PDF

Actually, the LSD algorithm just works well for the straight scratches (line segments) whose curvatures are near zero. However, there exists large curvature scratches. Fig. 14 indicates that the algorithm can deal with large curvature scratches. Fig. 14(a)∼(c) are the images with LSD detection results shown in red. The rectangular area in Fig. 14(d)∼(f) are the area that include scratches and the black line segment is the scratch that was found.

Fig. 14 Dealing with large curvature scratches.

Download Full Size | PDF

For these kind of scratches, the LSD algorithm treat them as a collection of small independent straight scratches. In this way, it breaks the continuity of the scratches and increases the difficulty of calculating the length of the scratch. Physically and geometrically, a large curvature scratch can be considered as several linked long scratches whose ends connects each other. For more original images with acceptable resolution, please see the images on the web site https://github.com/PaperImageData/ScratchDetector shared by the authors.

4.2. Comparison of detecting algorithms

We compared our scratch algorithm with LSD [22] and the algorithm in [18]. Table 2 shows the results by different detection algorithms. We use 5 group of images data and collect the detection results. Compared with the GLD such as LSD [22], our algorithm can detect the correct number of scratches in most cases. Meanwhile, the total length of scratches detected with LSD algorithm is far less than its counterpart obtained with HVI method. Compared with the algorithm in [18], our method is more stable and reliable, since for all experiments, we have a small error rate E < 5%. Obviously, the accuracy of our MVI result is satisfactory when compared with that of HVI.

Table 2. Comparison of Performance for Different Scratch Detection Algorithms

View Table | View all tables in this article

We implemented the detection algorithm with the C++ programming language and the open source software OpenCV-3.2 [26] on a Ubuntu 16.04 GNU/Linux operating system with Intel i7-6700K CPU and 32GB DDR4 RAM. The processing speed is about 0.138 second per binary image with 2724 × 2724 pixels in scratch detection.

5. Conclusions

Scratches detection on optical surface is a challenging problem in optics engineering. In this paper, we proposed an algorithm for detecting scratch defects on precise optical component surfaces. This algorithm can deal with the cases that GLD algorithms are not suitable for, such as the scratch has punctiform in the head and tail area, discontinuous scratches segments and etc.. Particularly, compared with LSD algorithm, our algorithm can give a semantic scratch detection and provide higher accuracy in quantitative analysis. Compared with the HVI method, our MVI method is much more accurate, fast, convenient and economical. Due to the physics characters of the surface, we believe this algorithm can also be used in other glass-like surface scratch detection.

Funding

Hainan Provincial Natural Science Foundation of China (HPNSFC) (2019CXTD405); National Natural Science Foundation of China (NSFC) (61201085, 61562023, 61502127).

References

1. L.-X. Yuan, Z.-T. Zhang, and X. Tao, “The development and prospect of surface defect detection based on vision measurement method,” in World Congress on Intelligent Control and Automation, (2016), pp. 1382–1387.

2. P. Mariño, V. Pastoriza, and M. Santamaría, “Machine vision method for online surface inspection of easy open can ends,” in Proc. SPIE, vol. 6382 (2006), pp. 638206. [CrossRef]

3. H.-H. Jiang and G.-F. Yin, “Surface defect inspection and classification of segment magnet by using machine vision technique,” Adv. Mat. Res. 339, 32–35 (2011).

4. J.-W. Dong, Q.-J. Wu, W.-W. Jiang, and Q. Xu, “The review of new NDT methods of metal material fatigue monitoring,” Int. J. Hybrid Inf. Technol. 8, 225–232 (2015). [CrossRef]

5. S. Radovan, P. George, M. Panagiotis, and G. Manos, “An approach for automated inspection of wood boards,” in Proc. 2001 ICIP (Cat. No.01CH37205), vol. 1 (2001), pp. 798–801.

6. G.-H. Hu, G.-H. Zhang, and Q.-H. Wang, “Automated defect detection in textured materials using wavelet-domain hidden markov models,” Opt. Eng. 53, 093107 (2014). [CrossRef]

7. E. H. Shortliffe and M. S. Blois, “The computer meets medicine and biology: emergence of a discipline,” Biomed. Infor. pp. 3–45 (2006). [CrossRef]

8. P. Cormont, A. Bourgeade, S. Cavaro, T. Doualle, G. Gaborit, L. Gallais, J.-L. Rullier, and D. Taroux, “Process for repairing large scratches on fused silica optics,” in Proc. SPIE, vol. 9633 (2015), p. 9633A.

9. “Optics and Photonics — Preparation of Drawings for Optical Elements and Systems — Part 7: Surface Imperfection Tolerances,” Standard, IOS (2008).

10. X. Tao, Z.-T. Zhang, F. Zhang, and D. Xu, “A novel and effective surface flaw inspection instrument for large-aperture optical elements,” IEEE Trans. Instrum. Meas. 64, 2530–2540 (2015). [CrossRef]

11. T. A. Germer, “Angular dependence and polarization of out-of-plane optical scattering from particulate contamination, subsurface defects, and surface microroughness,” Appl. Opt. 36, 8798–8805 (1997). [CrossRef]

12. Z.-H. Gu, “Detection of a small defect on a rough surface,” Opt. Lett. 23, 494–496 (1998). [CrossRef]

13. V. M. Schneider, M. Meljnek, and K. T. Gahagan, “Fast detection of single sided diffracted defects in display glass,” (OSA, 2009), pp. 638–644.

14. L. M. Sanchez-Brea, P. Siegmann, M. A. Rebollo, and E. Bernabeu, “Optical technique for the automatic detection and measurement of surface defects on thin metallic wires,” Appl. Opt. 39, 539–545 (2000). [CrossRef]

15. L. Li, D. Liu, P. Cao, S. Xie, Y. Li, Y. Chen, and Y. Yang, “Automated discrimination between digs and dust particles on optical surfaces with dark-field scattering microscopy,” Appl. Opt. 53, 5131–5140 (2014). [CrossRef] [PubMed]

16. D. Liu, Y.-Y. Yang, L. Wang, Y.-M. Zhuo, C.-H. Lu, L.-M. Yang, and R.-J. Li, “Microscopic scattering imaging measurement and digital evaluation system of defects for fine optical surface,” Opt. Commun. 278, 240–246 (2007). [CrossRef]

17. C. Li, Y.-Y. Yang, H.-L. Xiong, D. Liu, S.-B. Xie, Y. Li, J. Bai, Y. Shen, H.-Z. Jiang, and X. Liu, “Dual-threshold algorithm study of weak-scratch extraction based on the filter and difference,” High Power Laser Part. Beams 27, 1–8 (2015). In Chinese.

18. X. Tao, D. Xu, Z. T. Zhang, F. Zhang, X. L. Liu, and D. P. Zhang, “Weak scratch detection and defect classification methods for a large-aperture optical element,” Opt. Commun. 387, 390–400 (2016). [CrossRef]

19. J. Illingworth and J. Kittler, “The adaptive Hough transform,” IEEE Trans. PAMI 9, 690–698 (1987). [CrossRef]

20. N. Kiryati, Y. Eldar, and A. M. Bruckstein, “A probabilistic Hough transform,” Pattern Recogn. 24, 303–316 (1991). [CrossRef]

21. N. Guil, J. Villalba, and E. L. Zapata, “A fast Hough transform for segment detection,” IEEE Trans. IP 4, 1541–1548 (1995).

22. R. G. von Gioi, J. Jakubowicz, J.-M. Morel, and G. Randall, “LSD: a fast line segment detector with a false detection control,” IEEE Trans. PAMI 32, 722–732 (2010). [CrossRef]

23. C. Akinlar and C. Topal, “EDLines: A real-time line segment detector with a false detection control,” Pattern Recogn. Lett. 32, 1633–1642 (2011). [CrossRef]

24. D. Liu, S. Wang, P. Cao, L. Li, Z. Cheng, X. Gao, and Y. Yang, “Dark-field microscopic image stitching method for surface defects evaluation of large fine optics,” Opt. Express 21, 5974–5987 (2013). [CrossRef] [PubMed]

25. M. A. Fischer and R. C. Bolles, “Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography,” Commun. ACM 24, 381–395 (1981). [CrossRef]

26. A. Kaehler and G. Bradski, Learning OpenCV 3: Computer Vision in C++ with the OpenCV Library (O’Reilly Media, 2017).

Notation	Attribution	Description
p_s = [x_s, y_s]^⊤	`L.startpoint`	Start Point of L
p_s = [x_s, y_s]^⊤	`S.startpoint`	Start Point of S
p_e = [x_e, y_e]^⊤	`L.endpoint`	End Point of L
p_e = [x_e, y_e]^⊤	`S.endpoint`	End Point of S
l = p_s − p_s	`L.direction`	Direction of L
l = p_s − p_s	`S.direction`	Direction of S
ℓ = ‖l‖	`L.length`	Length of L
ℓ = ‖l‖	`S.length`	Length of S
ctg	`L.ctg`	Category of L

# of image group	Scratch num. (HVI)	ℓ_hvi(pixel)	Algorithm	Scratch num. (MVI)	ℓ_mvi(pixel)	E
1	3	3158.362	LSD [22]	31	—	—
			Tao [18]	4	3179.115	0.66%
			Ours	3	3112.673	−1.45%

2	2	1928.128	LSD [22]	12	—	—
			Tao [18]	2	1928.282	0.01%
			Ours	2	1928.865	0.04%

3	2	1596.565	LSD [22]	19	—	—
			Tao [18]	2	1831.135	14.69%
			Ours	2	1532.661	−4.00%

4	6	5799.83	LSD [22]	41	—	—
			Tao [18]	9	5070.034	−12.58%
			Ours	7	5576.723	3.85%

5	7	3427.567	LSD [22]	24	—	—
			Tao [18]	5	4793.467	39.85%
			Ours	7	3532.917	−3.07%

Notation	Attribution	Description
p_s = [x_s, y_s]^⊤	`L.startpoint`	Start Point of L
p_s = [x_s, y_s]^⊤	`S.startpoint`	Start Point of S
p_e = [x_e, y_e]^⊤	`L.endpoint`	End Point of L
p_e = [x_e, y_e]^⊤	`S.endpoint`	End Point of S
l = p_s − p_s	`L.direction`	Direction of L
l = p_s − p_s	`S.direction`	Direction of S
ℓ = ‖l‖	`L.length`	Length of L
ℓ = ‖l‖	`S.length`	Length of S
ctg	`L.ctg`	Category of L

# of image group	Scratch num. (HVI)	ℓ_hvi(pixel)	Algorithm	Scratch num. (MVI)	ℓ_mvi(pixel)	E
1	3	3158.362	LSD [22]	31	—	—
			Tao [18]	4	3179.115	0.66%
			Ours	3	3112.673	−1.45%

2	2	1928.128	LSD [22]	12	—	—
			Tao [18]	2	1928.282	0.01%
			Ours	2	1928.865	0.04%

3	2	1596.565	LSD [22]	19	—	—
			Tao [18]	2	1831.135	14.69%
			Ours	2	1532.661	−4.00%

4	6	5799.83	LSD [22]	41	—	—
			Tao [18]	9	5070.034	−12.58%
			Ours	7	5576.723	3.85%

5	7	3427.567	LSD [22]	24	—	—
			Tao [18]	5	4793.467	39.85%
			Ours	7	3532.917	−3.07%

Automatic scratch detector for optical surface

Abstract

1. Introduction

2. Notations and concepts about scratches

2.1. Image captured from MDSI system

2.2. Representation of line segment and scratch

2.3. Dark points

2.4. Performance parameter of scratch detection

3. Scratch detecting algorithm

3.1. Coarse locating of scratches with LSD algorithm

3.2. Classifying segments from different scratches

3.3. Removing noise points with RANSAC method

3.4. Positioning proper endpoints

3.5. Implementation of scratch detecting algorithm

4. Experiment and result analysis

4.1. Experiment results

4.2. Comparison of detecting algorithms

5. Conclusions

Funding

References

Cited By

Figures (14)

Tables (5)

Equations (31)

Optics Express