Xi'an Technological University
Subject: Computer Science, Software Engineering
eISSN: 2470-8038
SEARCH WITHIN CONTENT
Zhang Wenqi ^{*} / Bai Wanmin ^{*} / Yu Jun ^{*} / Gao Shouyi ^{*}
Keywords : Sea-sky Line, Mathematical Morphology, Edge Detection, Hough Transformation
Citation Information : International Journal of Advanced Network, Monitoring and Controls. Volume 3, Issue 3, Pages 85-92, DOI: https://doi.org/10.21307/ijanmc-2019-010
License : (CC-BY-NC-ND 4.0)
Published Online: 01-October-2019
To solve the problem of low accuracy and robustness of sea-sky line detection, this paper presents a method of sea- sky line detection based on the mathematical morphology. Firstly, the mathematical morphology closed-open operation is used to filter and denoise the sea-sky image. Then the Canny operator is used to obtain the sea-sky boundary of the image, Then mathematical morphological operation is used to remove some disturbing points. Finally, the sea-sky line is detected by Hough transform. The experimental results show that the algorithm can accurately and efficiently detect the sea-sky line under the complex sea-sky background.
The sea-sky line is the dividing line between the sea and the sky. In general, an image of sea-sky background mainly includes three regions, those are brighter sky area, the darker sea area, and the sea-sky line area from light to dark [1]. If the low altitude investigation is carried out, the target on the sea usually appears in the area of sea-sky line. By detecting and obtaining the sea-sky line, it can reduce the calculation amount of target detection and shorten the calculation time. At the same time, it can distinguish between the sky area and sea area, that is meaningful to the simulation experiment of target detection on the sea.
The sea-sky line detection is influenced from marine environment greatly. The main influencing factors are as follows :
(1) The strong watermark interference caused by the wave, that makes the gray-value of the wave which is close to the pixel point gray-value of the sea-sky line, so that the extraction of the sea-sky line is difficult.
(2) When the background images contain mountains, ships and so on, which will interfere with the detection of sea-sky lines;
(3) When the atmospheric visibility is low, the sea-sky boundary is blurred, which leads to difficult y on detecting sea-sky line.
In order to detect the sea-sky line accurately, we need to know about its characteristics as follows.
(1) The area of sea-sky line is between the sky and the sea. Its brightness is more intense than the other two parts. Grayscale changes strongly in vertical direction as well as varies in horizontal direction slowly.
(2) The sea-sky line is usually not a straight line but a gradual change band.
At present, there are many reference documentation on sea-sky line detection. For example, Liang D and others use the algorithm of OTSU segmentation and clustering in order to detect the sea-sky lines[2]. Because the OTSU segmentation algorithm could not accurately segment the sea-sky background images with imbalanced illumination. It makes the sea-sky line detection error in this kind of image; H. Wang and others use the algorithm of combining the Sobel operator with the straight line fitting to carry out the sea-sky line detection[3]. This method can be used to extract the sea-sky line in simple background, but it is difficult to get a satisfactory extraction effect in some complicated situations. Wang Bo and others use the algorithm of gradient saliency region growth to detect the sea-sky lines[4], but the sea surface splash and water wave will interfere with the image gradient calculation. For the complex images of sea conditions, these methods are limited in certain degree.
In order to improve the robustness and accuracy of sea- sky line detection, the method of sea-sky line detection based on mathematical morphology is proposed. This method can improve the robustness of sea-sky line detection in the complex sea-sky background. The mathematical morphology are used to denoise sea-sky images and remove interference points, which can reduce computation and improve the accuracy of sea sky detection.
The mathematical morphology is a nonlinear image processing and analysis theory. It is characterized by geometrical method, which is more suitable for the processing and analysis of visual information. The basic idea of the mathematical morphology is to measure the availability of the target image region and the effectiveness of the filling method by using a certain form of structural elements. Then it extracts more essential information of the related characteristics of the image morphology, which can achieve the purpose of the target image analysis and recognition. The mathematical morphology can eliminate the unrelated morphological and structural attributes in the target image and retain the basic nature of the morphological and structural properties to simplify the target image data, so that it has the characteristics of fast parallel speed and easy implement in hardware. The algorithm has the natural parallel structure. It realizes the parallel of morphological analysis and process, which greatly improves the speed of image analysis and process.
At present, the mathematical morphology has been widely used in the fields of pattern recognition, machine vision, microscopic image analysis, medical image processing, computing and data processing and so on. It has obvious advantages in image processing problems such as filtering noise reduction, image enhancement, edge detection, image segmentation, feature extraction, texture analysis, image restoration and reconstruction, and image compression and so on.
The mathematical morphology is composed of a set of morphological algebraic operators, whose basic operations are shown as follows: expansion, erosion, opening and closing. These operations have different characteristics in binary and grayscale images [5]. These basic operations can also be derived and combined into various practical algorithms of the mathematical morphology, which can be used to analyze and process the shape and structure of the image.
The most basic morphological transformation of the mathematical morphology includes expansion and corrosion, which can achieve many functions, such as filtering noise, dividing the independent elements and bridging the adjacent elements in the target image. The mathematical morphology can also be used to find the maximum or minimum region of the obvious block in the target image and get the gradient of the target image.
The expansion operation is to calculate the local maximum. While the corrosion operation is to calculate the minimum value of the pixel in the area. The two operations are a pair of mutually dual operations [6].The expansion operation is a process to expand the edge to the outside. It can be used to fill the small holes in the target image and transform the background edge into the target edge, so that the goal is increased and the background is reduced. The corrosion operation is a process that removes the unrelated edge points and makes the edge shrink inward. It can eliminate the small bulges in the target image and reduce the target and increase the background.
The other morphological operations are composed of two basic morphological transformations[6-7], such as opening and closing. The f(x, y ) is set as an input image, b(x, y)is set as a structural element. The structure element b is used to handle the input the image f(x, y ).As the formula(1) shown, this is an expansion operation. As the formula (2) shown, this is a corrosion operation.
Definition 1 The image f is expanded by using the structural element b, writed as f ⊕ b.
Definition 2 The image f is Corroded by using the structural element b, writed as f ⊖ b.
Based on the two basic morphological transformations of expansion and corrosion, many mathematical morphological clusters can be constructed. While open and closed operations are the two basic operations in the cluster [7-8]. The open operation firstly corrodes the image and then expands it, as shown in the formula (3). The closed operation firstly expands the image and then corrodes it, as shown in the formula (4).
Definition 3 The image f is opened by using the structural element b, writed as f ∘ b
Definition 4 The image f is closed by using the structural element b, writed as f • b
As shown in Fig.1, the Fig.(a) is a square structural element, whose size is 2 x 2. The Fig.(b) is the objective matrix that we want to perform the mathematical morphology operations. The Fig.(c) is the result diagram of the corrosion operation. We use the structural elements that is shown in the Fig.(a) to handle the target matrix of the Fig.(b). The Fig.(d) is the result graph of the expansion operation that we use the structural elements that is shown in the Fig.(a) to handle the target matrix of the Fig.(b).
The open operation can remove isolated points, burrs and small bridges (that is, the small points connected to two blocks), which can be used to segment large areas and smooth the edges of large area. While the total position and shape are constant. The closed operation can fill the small holes in the object and achieve the purpose of stitching small cracks to connect the adjacent objects and smooth edges. While the total position and shape are constant. The open operation and closed operation are also a pair of dual operations.
The closed operation can be filled with low grayscale black holes. And the open operation will inhibit the white point( noise) with high gray value. The operation that the closed operation is performed firstly, followed that the open operation is used to make the de-noising effect better and smooth edge. Therefore, the closed - open operation are choosen in the this article to filter and reduce the noise of the sky-sea images.
In any condition, the mathematical morphology algorithm is composed of two basic problems: mathematical morphology operation and structural element selection. The definition of mathematical morphology makes the operation rules of mathematical morphology constant. Therefore, the selection of morphological and structural elements determines the purpose and effect of mathematical morphology algorithm. Throughout, The determination and optimization of structural elements have become hot topics and difficulties in the study of the mathematical morphology.
The choice of the morphological structure elements can be divided into two aspects: the size and the shape of the structural elements. Generally speaking, the structural elements must be geometrically simpler than the original image. And they are bounded; Besides, the convexity of structural elements is also important. Based on the selection principle of structural elements, we usually choose some small simple collections, such as square, diamond, circle and so on. As shown in the Fig. 2, there are some examples of structural elements.
If the structural elements are not properly selected, they can not effectively process pictures. And the result will not have the desired effect. That mainly include the following two conditions.(1) When the size of the selected structure element is too small, the open operation cannot effectively eliminate the larger high grayscale noise point; For larger, low-gray black holes, that cannot be effectively bridged by the closed calculations. (2) When the size of the selected structure element is too large, on the one hand, the open operations will excessively eliminate pixel points on the edges of the image and cause false breaks. On the other hand, the closed operations will over-combine black holes and generate interference information [9-10].
Therefore, the use of single size structure elements can easily lead to the edge location of the target image is not accurate enough and the denoising effect is not ideal. In addition, due to the existence of a constraint relation on the edge of the image, the noise of the image is generated randomly.
When the structural elements is used to measure the target image, a geometric shape similar edge point can always be found near the edge points of the image. Thus, it is not effective to retain the edge segmentation information of the image by using a single morphological structure element to extract the edge of the target image.
Consequently, when the sea-sky line is extracted in the target image. If the single size and shape structure elements are used for image processing, the location is not accurate, the de-noising effect is not ideal, and the detail information of the sea-sky line can not be retained effectively. In conclusion, this paper adopts multi-dimensional, multiple-shape structure elements to process sea-sky images.
The sea-sky line detection algorithm in this paper is based on the mathematical morphology. The overall flow diagram is shown in the Fig.3. Firstly, the sky-sea image is preprocessed. The mathematical morphology is used to filter the target sea-sky background image and denoise the interference of the sea-sky lines; Secondly, the Canny operator is used to extract the sea sky boundary of the preprocessed sea-sky background image. Follow that the mathematical morphology is used again to remove the interference points, so that the sea-sky line detection is more efficient and accurate. Finally, the Hough line detection and the least square method are used. Linear fitting is used to get the final sea-sky lines.
Step 1: We preprocess the image to solve the interference problems caused by strong watermark and uneven illumination. In which, the structural elements of closed operation and open operation are square structural elements.
Step 2:We use the Canny operator to extract the preprocessed sea- sky pictures.
Step 3: The mathematical morphology is carried out to remove the interference points, so that the sea-sky line detection is more efficient and accurate. The structural elements is the linear structural elements.
Step 4: We use Hough straight line detection and the least square fitting method to get the sea-sky lines.
The followings focuse on mathematical morphology of image preprocessing, interference point elimination, as well as the Hough line detection and other major steps to describe.
In the process of the initial image collection, due to optical system distortion, relative motion, weather and other reasons, the noise is inevitable. In the process of transmission, noise can pollute the image and noise points have a certain bad effect on edge detection [11]. Generally, the Gauss filter is used to remove noise. However, this paper uses the mathematical morphology filter to preprocess the image, which can make the brightness of the target image more uniform and remove the interference of the watermark. At the same time, that can preserve the structure gradient information of the image better. In this paper, a closed-open filter(COF) is constructed through the combination of opening and closing. The definition is shown in the formula (5).
The mathematical morphological filters have the properties of transitivity, translation invariance, idempotency and duality. The structure element chooses the larger square structure elements, because the noise element of uneven illumination and the strong watermark is larger.
The edge detection by the Canny operator is a technology to extract useful structural information in different visual objects. And that greatly reduces the amount of data to be processed. It is now widely used in various computer vision systems. The Canny operators are used in different visual systems to detect edges, but the requirements on the edge detection are similar, so the wide application of edge detection technology can be realized[12].
For the grayscale image that has been preprocessed by the method of step 1, the gradient intensity and direction of each pixel in the image are calculated. The edges of the image can be directed to all directions, so the Canny algorithm uses four operators to detect the horizontal, vertical and diagonal border in the image. The operator of edge detection returns the first order value of horizontal Gx and vertical Gy direction, thereby the gradient G and direction theta of pixels can be determined, as shown in formula (6) and formula (7).
In formula (6),G is gradient strength. In formula (7), the arctan is an inverse tangent function.Besides, the theta represents gradient direction
The Non-Maximum Suppression is applied to eliminate the spurious response that is caused by edge detection. The Double-Threshold detection is used to determine the real and potential edges. Finally, the edge detection is completed by suppressing isolated weak edges.
The Canny operator is used to extract edges, so that all possible edges can be obtained to ensure the accuracy of edges.
After using the Canny operator to extract the edges, the images with two value are obtained. There are still a lot of small noise points in the the images with two value. That causes interference to the next detection and fitting of sea- sky lines. Therefore, we use the mathematical morphology operation to remove interference points.
Because the mathematical morphological operation is sensitive to the size and shape of structural elements, the appropriate structural elements must be selected. Meanwhile, because the target is to obtain a sea-sky line, so the linear structure element is used to remove the noise points, so that the interference points can be removed. At the same time, the edge points of the target are not mistakenly removed [13]. The linear structural elements used in this paper are shown in the formula (8).
In formula (8), these character se represents the structure element, and the strel() is the function that created the structure element. In which, the ‘line’ represents a linear structure element, Meanwhile, the x and y determine the size and direction of the structure element that we choosed. Of which the linear structure element x and y have the following relationship, as shown in formula (9):
According to the characteristics of sea-sky line, this paper selects the linear structural elements, which can effectively remove interference points, reduce amount of calculation in Hough line detection and improve its accuracy and efficiency.
The basic idea of the Hough transformation is the duality of the point to the line. After the image transformation, the images in the image space are transformed into the parameter space [14]. In the x-y image space, a straight line y = Ax + B (Of which, A is the slope, B is intercept) corresponds the points in the ρ - θ parameter space.
In the image space, the point on a straight line is a sinusoidal curve in the Hough parameter space; Many points on the same line in the image space are a sinusoidal cluster in the Hough parameter space and the curve clusters are intersected to a point, which is called the peak point. The peak point in the Hough parameter space corresponds to a straight line in the image space. As shown in Fig.4, this is the image space; As shown in Fig.5, this is the parameter space. The Hough transformation is converted from the image space of Fig. 4 to the parameter space of Fig.5.
Therefore, Therefore, the Hough transformation transforms the straight line detection problems in the image space to the points detection in the parameter space. There are a number of possible lines in the sea-sky image, but the sea-sky line is throughout the image. Thus, the sea-sky line is The longest line segment in the sea-sky image, corresponding the local maximum value in the Hough parameter space. By detecting the local maximum in the Hough parameter space, we can find a corresponding line in the x-y image space, that is, the sea-sky line [15].
Through the Hough Line detection, the longest line segment is extracted. However, the sea sky line is a straight line through the whole picture. So we have to extract and fit the points in the line segment and get the final sea-sky line. through the whole image. The sea-sky line is gotted by selecting some points and making straight line fitting by the least square method. In this paper, the least square method is used to fit the straight line. The least square method is a mathematical optimization technique. It searches for the best function matching of data by minimizing the sum of squares of errors. The least squares method can be used to obtain the unknown data simply and make the sum of squares between the obtained data and the actual data minimum.
To verify the results of this method, we selected three sea-sky background images in different environments, as shown in the Fig.6. The Fig. (a) is a sea-sky image with lower visibility; The Fig. (b) is a sea-sky image with strong watermark; The Fig. (c) is a sea-sky image with uneven illumination. After the operations are performed on the matlab 2015a software, two mathematical morphological processing cases are compared and analyzed.
The mathematical morphology operations are used in the pretreatment of sea-sky image.Two preprocessing methods are used. One way is to use the Gauss filter to reduce noise and then conduct sea-sky lines detection. The results are shown in Fig.7. The other way is using the mathematical morphological filter to reduce noise and then conduct the sea- sky line detection, the results are shown in Fig.8. As seen from the Fig. 7(b), the former method is not ideal for detecting sea-sky pictures with strong water marks, and there is an error. From the fig. 8, we can see that the method of this paper can accurately detect the sea-sky line in different environment.
The images are results of sea-sky lines detection in Fig.8.The sea-sky-line detected by the algorithm in this paper. In order to measure the processed image quality, we usually refer to the PSNR value to determine whether a particular processing program is satisfactory enough.Thus, to quantitatively evaluate the experiment results, the experiment performance of the sea-sky line detection in three sea-sky backgound images are compared. The peak signal to noise ratio (PSNR) is used. The PSNR is the ratio of the variance to the information and noise, when the value of PSNR is between the 30dB and 40dB,which means less noise. When the value of PSNR is 40dB, which means better picture processing effect [16]. The expression of the PSNR is shown in the formula(10).
In which f_{max} is the maximum grayscale value of function f(x, y). MSE is a mean-square error that reflects the variance between the estimate and the estimated amount, as shown in the formula (11).
The obtained quantitative evaluation experiment data are shown in Table 1.
From the numerical change of PSNR in Table 1, we can see that the quality of the picture is in high level in most cases, for the sea-sky background pictures are processed by the Gauss filter. However, from Fig. 6 (c), we can see that the value of PNSR is lower than 30 dB. Thus, for the unevenly illuminated sea- sky background pictures, the Gauss filter can not effectively process and solve the uneven illumination problem; For Fig. 7 (b), the PSNR values of the pictures after the Gauss filter processing are higher than the result of the pictures after the mathematical morphological filter processing. As a result, it can be seen that the quality the pictures after the Gauss filter processing is relatively higher than that after morphological filter processing. It is also shown that the Gauss filter can effectively remove the interference of strong watermarks on the image processing. However, according to the Fig.6 (b) effect diagram of the sea-sky lines detection after the Gauss filter processing, it is known that although the pictures are processed by the Gauss filter. They get better quality of the images. It effectively removes the interference of the strong watermarks, but also loses more detail information of the edge of the sea and the sky, so that the effective points of the edge are also erroneously removed. There is not an accurate sea-sky line to be obtained. As seen from the Table 1, the PSNR values of the images that are processed by the mathematical morphological filter in this paper are all between 30 dB and 40 dB, which indicates that the image processing quality is better. At the same time, combining with figure 8, we can know that the algorithm in this paper can effectively preserve the details of the sea-sky boundary by the algorithm in this paper. The more accurate sea-sky lines can be obtained. The sea-sky lines in different sky-sea background images are measured. It objectively reflects the feasibility and superiority of the algorithm in this paper.
The mathematical morphology is applied to the interference point removal after edge detection by the Canny operator. After the interference points are removed, the Hough transformation is used to extract the sea-sky lines so as to achieve the final detection and fitting of the sea-sky lines. We get the detection result of the sea-sky lines finally,as shown in Fig. 8. We obtain the two-value pictures after edge detection by the Canny operators.Thus, the mathematical morphology is applied to the two-value pictures.The experimental data of Table 2 are analyzed by comparing the number of effective points reserved before and after second mathematical morphology processing in the experiment.
From table 2, it can be seen that after the second morphological processing, based on the special structure element, the interference points of the sea-sky lines can be reduced effectively. Because of the edge extraction using the Canny operator, all the possible edges are detected. However, only one of the required sea-sky line is needed. At the same time, some interference points are produced when the edge detection is performed. These are unavoidable. Under the premise of guaranteeing the valid the sea-sky line information point ratio (B/A), the effective detail of sea-sky lines are protected, the possible points (A) are effectively reduced. From the data change in Table 2, we can see that the interference information produced by edge extraction is the lower for images with lower visibility. For images with uneven illumination, there are more interference information produced by the operation of preprocessing and edge extraction. These interference information will cause the interference problems on straight line detection and fitting. That makes the sea –sky line extraction difficult and inaccurate. At the same time, it shortens the time of straight line detection and improves the efficiency of the algorithm in this paper. The mathematical morphologic is used, the computation amount of Hough detection is reduced. Meanwhile, the algorithm in this paper reduces the interference on the sea-sky line fitting. Thus, the algorithm in this paper ensures the efficiency and accuracy of the sea-sky line detection.
Therefore, the method of this paper achieves the expected effect and the extraction effect of sea-sky lines is ideal.
This paper presents a method of the sea-sky line detection based on the mathematical morphological. Firstly, the image is preprocessed by mathematical morphological filtering. Followed that, the Canny operator is used to extract the sea-sky boundary. Secondly, the Mathematical morphology processing is once more used to remove the interference points on the sea-sky line; Finally, the sea-sky line is detected by the Hough transform and fitted by the least square method. The experimental results show that, this algorithm can detect the sea-sky lines, as well as the robustness is better, accuracy is higher. It can effectively cope with the complex marine environment and weather effects.