In order to study the changes of molten pool morphology during laser cladding, an online monitoring system for laser cladding molten pool was built. The molten pool image was obtained by coaxial assembly of COMS camera and laser equipment. Based on the analysis of the grayscale histogram distribution of the molten pool, the adaptive threshold segmentation method of triangular threshold segmentation was used to binarize the molten pool image. The edge of the molten pool image was retrieved by Canny operator, and the length and width of the molten pool area were obtained by the minimum circumscribed rectangle algorithm. Nine groups of single-pass cladding orthogonal experiments were carried out with 45 steel as substrate and 420 stainless steel as cladding powder. The experimental results show that the average error between the molten pool width measured under the monitoring system and the actual cladding width measured under an electron microscope is 4.5%, which verifies the effectiveness of the visual monitoring system. The range analysis of the molten pool width obtained under the monitoring system shows that the laser power has the greatest influence on the molten pool width, followed by the scanning speed, and finally the powder feeding rate; the molten pool width increases with the increase of laser power, and decreases with the increase of scanning speed and powder feeding rate. The molten pool information and change rules obtained by the monitoring system can be used as reference variables for real-time control of laser cladding, laying the foundation for closed-loop control of laser cladding.
1. Technical background
As an advanced material forming technology, laser cladding works by forming a high-temperature molten pool on the surface of the substrate through the action of a high-energy laser beam. The nozzle transports the metal powder into the molten pool in a directional manner, undergoes a melting and solidification process, and finally obtains a deposited cladding entity. This technology has unique advantages in metal part repair, rapid prototyping, surface modification, and metal additive manufacturing. However, the laser cladding process is unstable, and there are obvious changes between the cladding layers even with the same working parameters. This poor reproducibility is caused by the fact that laser cladding is highly sensitive to the slight effects of operating parameters (such as laser power, cladding speed, and powder feeding rate), and there is a complex coupling relationship between the parameters, so there are certain limitations in actual work.
In order to obtain more stable cladding quality, it is very important to monitor the laser cladding process in real time. Hong Lei et al. [8] used a photoelectric sensor to monitor the plasma blue-violet light signal generated during the laser cladding process, analyzed the relationship between different process parameters and the light intensity signal, and obtained the light intensity signal range with good cladding layer quality under experimental conditions. However, when the laser power is less than the threshold p, the blue-violet light signal is not greatly affected by the laser power. Therefore, this signal is not suitable for low-power laser cladding process monitoring. In addition, the blue-violet light signal intensity range corresponding to the good cladding quality of different cladding materials is different. A large number of observations are required to obtain a suitable range of changes. Hu Xiaodong et al. [9] designed a new photoelectric sensor, established the corresponding relationship between the sensor voltage signal and the powder flow rate, and controlled the cladding process by monitoring the powder flow rate to achieve the purpose of stable cladding. Song Wei et al. [6] used a CCD camera to obtain the size information and temperature distribution of the molten pool, and obtained the relationship between the cladding parameters and the molten pool size. Miyagi M. et al. [10] integrated a photodiode into the laser processing head for monitoring and found that the thermal radiation signal had a strong correlation with the change of the molten pool width. A PID controller was designed to control the output of the laser power, thereby controlling the cladding shape. Sun Huajie et al. [11] built a color CCD camera temperature closed-loop control system based on colorimetric temperature measurement, which can effectively eliminate the heat accumulation effect in the laser cladding process and achieve the expected cladding quality. However, when the laser power exceeds 1800W, the image grayscale corresponding to the R channel reaches the maximum grayscale value of 255. The image grayscale and the molten pool temperature cannot form a one-to-one correspondence, resulting in temperature measurement failure. Smurov I. et al. [12] used a pyrometer and an infrared camera to measure the brightness and temperature information of the molten pool, established the relationship between the brightness temperature information and the molten pool morphology, and realized the control of the cladding process.
As the basic unit of the cladding entity, the molten pool exists in the entire cladding cycle, and the morphological characteristics of the molten pool can directly reflect the final cladding results. Therefore, this paper selects the molten pool morphology as the monitoring object and develops a laser cladding molten pool online monitoring system based on COMS industrial camera and OpenCV (open source computer vision and machine learning software library). The system uses a comprehensive image algorithm to process the input molten pool image, which can effectively segment the molten pool area and extract the molten pool area and length and width of the molten pool. Finally, the entire cladding process and algorithm processing results are observed through the system’s interactive interface. The monitored molten pool information can be used as a reference variable for real-time control of laser cladding, laying the foundation for closed-loop control of laser cladding.
2 Monitoring platform and cladding materials
The COMS color camera model used is Baslera2A192051gcBAS, with a maximum resolution of 1920×1200. The camera provides an SDK (Software Development Kit) based on the C++ programming language, which can be used for secondary development of the camera. The experiment uses a coaxial installation of the COMS camera, and the overall architecture is shown in Figure 1. The coaxial assembly camera can ensure that the molten pool and the camera remain relatively still during processing, and there is no need to correct the image, so the detection field of view and accuracy must be better than the side axis assembly.
Before the experiment starts, move the laser head to the working position, start the camera, and adjust its exposure and focal length so that the captured image can be clearly imaged within the field of view. During the laser cladding process, the light emitted by the molten pool is reflected by two preset beam splitters placed at an angle of 45°, and finally directly enters the camera COMS chip. The light signal captured on the pixel unit is converted into a digital signal through a series of conversions and input into the computer through a network cable for image processing.
The experiment uses 45 steel with a size of 200mm×100mm×10mm as the cladding substrate. Before the experiment, the surface of the cladding substrate is polished with a sandblasting machine to remove the oxide layer and other impurities on the surface. 420 stainless steel is used as the cladding powder, and its chemical composition is shown in Table 1.
3 Image processing
When acquiring the molten pool image, some interference factors such as laser light, plasma and powder splashing will be collected and transmitted to the computer together with the molten pool image [14]. In addition, the instability of the recording device and the transmission device will also interfere with the collected molten pool image [15]. These interferences will mislead the analysis of the molten pool characteristics. Therefore, in order to accurately extract the characteristic information contained in the molten pool, the original molten pool image needs to be processed. The overall process of molten pool image processing is shown in Figure 2.
3.1 Image grayscale
During laser cladding, the image presented in the molten pool area is different from the highlight part of other areas. Compared with the chromaticity information of the image, the brightness information of the image can better reflect the characteristics of the molten pool. Therefore, it is necessary to convert the color image captured by this camera into a single-channel grayscale image. The reduction in the number of channels reduces the amount of calculation, which is beneficial to the subsequent algorithm processing. The weighted average method is used to calculate the image grayscale. The calculation formula is: Gray = 0. 299 × R + 0. 587 × G + 0. 114 × B (1)
Where Gray is the grayscale value of the pixel after weighted calculation; R, G, and B are the grayscale values of the red, green, and blue channels of the pixel respectively.
3.2 Filtering and denoising
During the process of molten pool image acquisition and transmission, it is inevitable that it will be affected by noise. It is necessary to denoise the grayscale molten pool image. As a nonlinear spatial filter, the median filter has the advantage of effectively removing sudden pixel points in the image and retaining the edge of the image, which is beneficial to the subsequent edge detection. The gray value of the coordinate point (x, y) after median filtering is: See formula (2) in the figure
Where, Sxy is the coordinate of all pixels in the area centered at point (x, y); g (s, t) is the gray value of the original pixel at this coordinate.
3.3 Adaptive threshold segmentation
In the molten pool image, the gray value of the pixels located in the molten pool area is higher than that in other areas. Therefore, a reasonable gray threshold can be used to separate the molten pool area from the molten pool image. During the laser cladding process, the gray value of the molten pool image changes all the time. The fixed threshold segmentation method cannot accurately separate the molten pool area of all images. In order to improve the detection accuracy, the molten pool image is segmented by adaptive threshold segmentation. The gray histogram distribution of the molten pool image is shown in Figure 3.
The gray distribution of the molten pool image in Figure 3 is mainly concentrated in the highlight area above 250, which is a typical single-peak gray histogram. Therefore, the triangular threshold segmentation method is used to process the molten pool image. The principle is to draw a straight line from the highest point to the lowest point of the grayscale in the histogram, and then calculate the vertical distance from the histogram vertex corresponding to each grayscale to the straight line, and select the grayscale value corresponding to the farthest point as the image threshold. The explanation of the triangular value segmentation code is shown in Table 2. The binary image after adaptive threshold segmentation is shown in Figure 4.
3.4 Morphological processing
After binarizing the image, a binary image with two sets of molten pool area and background area is obtained (see Figure 5). It can be seen from Figure 5a that there are hollow black spots caused by noise inside the molten pool area, and there are small light spots reflected by powder splashing on the edge contour. These defects will affect the extraction of the molten pool contour, so the molten pool image needs to be further refined. The closed operation in morphological processing is used to expand the image first and then erode it, which can remove small holes in the connected area; the open operation first erodes the image and then expands it to eliminate the small discrete points on the edge of the contour, disconnect the narrow gaps on the edge of the image, and make the contour smoother. Both operations will not change the area of the molten pool area. After the treatment, a closed molten pool area is obtained, as shown in Figure 5b.
3.5 Edge Extraction
After separating the molten pool area, retrieve the edge of the molten pool area. The junction of the molten pool area and the background area is the dividing point where the pixel grayscale changes dramatically. The set of these pixel points is the edge of the molten pool area. The Canny operator is used to detect the edge of the molten pool image, and the two-dimensional Gaussian filter is used to smooth and denoise the image. The filter expression is: see formula (3) in the figure
Where, (x, y) is the pixel coordinate of the image; α is the variance, which is used to control the smoothness.
Use the first-order partial derivative finite difference to calculate Jx and Jy. According to Jx. and Jy, calculate the gradient amplitude A (x, y) and direction θ, and we have; see formulas (4)-(7) in the figure
After obtaining the gradient amplitude, non-maximum suppression is performed, and the high and low double threshold methods are used to determine the image edge. After processing, a closed annular area may be obtained, and the result is shown in Figure 6.
3.6 Extraction of melt pool length and width
The melt pool is an irregular ellipse, and its length and width cannot be measured directly. Therefore, the minimum enclosing rectangle algorithm is used to obtain the length and width information of the melt pool.
According to the edge contour of the melt pool, the upper, lower, left and right boundaries of the melt pool are found to establish the initial enclosing rectangle. Let the upper boundary equation be x=x1, the lower boundary equation be x=x2, the left boundary equation be y=y1, and the right boundary equation be y=y2.
The center coordinates O(x0, y0) of the initial enclosing rectangle are determined by the four boundaries. Then: see formula (8) in the figure
Using O(x0, y0) as the coordinate origin, two mutually perpendicular central principal axes are established. The coordinates of the two points at the vertical end are A(l, y0) and B(c, y0), and the coordinates of the two points at the horizontal end are C(x0, l) and D(x0, k).
Rotate the main axis by θ degrees around the center point O(x0, y0). Assume that the coordinates of the four endpoints of the main axis after rotation are A'(xa, ya), B'(xb, yb), C'(xc, yc), and D'(xd, yd). Then: See formulas (9)-(12) in the figure.
Translate the main axis. When 0°<θ<45°, the horizontal x value moves up or down, and the vertical y value moves left or right. When 45°<θ<90°, the horizontal y value moves left or right, and the vertical x value moves up or down.
By rotating and translating the main axis multiple times, the area of the enclosing rectangle is calculated, and finally the rectangle with the smallest area is selected as the minimum enclosing rectangle of the image. The minimum enclosing rectangle processing is shown in Figure 7.
3.7 Molten pool monitoring system and camera calibration
After processing, the relevant information of the molten pool area, molten pool length, and molten pool width of the molten pool image can be correctly extracted. In order to monitor the molten pool morphology in the laser cladding process in real time, a laser cladding molten pool image acquisition and online monitoring system was built. The monitoring system is based on the Windows platform and is developed using C++ programming, OpenCV open source visual processing library and Qt application. The left part of the interface can dynamically display the original molten pool image and the processed molten pool image in real time. The right side can output the relevant information of the molten pool area, molten pool length and molten pool width of the current molten pool area. The result curve can draw a line chart of the molten pool area. The interactive main interface of the monitoring system is shown in Figure 8.
Click the setting button to modify the camera-related parameters. The exposure and gain can be adjusted in real time according to the imaging results, and the camera acquisition mode can also be adjusted. The calibration module below can manually set the calibration point, and finally obtain the size corresponding to each pixel by counting the number of pixels between two points and the given length.
The experiment uses a calibration plate composed of black and white square blocks with a side length of 1.5mm to calibrate the image. Adjust the camera to the same working position as during laser cladding, and place the calibration plate under the camera lens for acquisition, as shown in Figure 9.
Set two calibration points P1 and P2, and count the number of pixels corresponding to the 1.5mm long calibration block as 222 pixels. Then the size of each pixel is 1.5/222mm, the molten pool area S = the number of pixels in the molten pool area × (1.5/222)², the molten pool length L = the number of pixels of the minimum circumscribed rectangle length × (1.5/222), and the molten pool width W = the number of pixels of the minimum circumscribed rectangle width × (1.5/222).
