Laser cladding technology is an additive manufacturing technology that manufactures or repairs metal parts by melting and depositing powder. Under a specified path, the technology uses a high-energy laser beam to quickly melt and solidify the powder on the surface of the substrate, so that the powder and the substrate are combined to form a functional coating with metallurgical bonding characteristics. Laser cladding has the advantages of high operational flexibility, high bonding strength, high material utilization rate and excellent comprehensive performance. It can significantly improve the wear resistance and corrosion resistance of the surface of the material substrate. It is widely used in aerospace, automobile and even national defense. “Made in China 2025” clearly pointed out that the vigorous development of remanufacturing has promoted the rapid development of laser remanufacturing technology, and the requirements for the forming morphology and performance of laser cladding are increasing.

There are many complex reactions such as physical and chemical reactions in the laser cladding process, which are easily affected by many internal and external factors. In the actual operation process, due to the lack of feedback and control system, the cladding process will be affected by internal factors such as laser power, scanning speed and powder feeding rate. In addition, it is also affected by environmental factors and interference during the processing. The processing stability is poor, and defects such as pores and cracks are easily formed, which seriously limits the development of this technology and its commercial application. Therefore, real-time monitoring and feedback control of the laser cladding process is one of the current research hotspots.

The use of closed-loop control systems in laser cladding helps to greatly improve the quality of the cladding layer and reduce costs. During the cladding process, the molten pool image is collected by a visual camera, the molten pool image is processed and the molten pool geometry information is extracted. The size, shape and brightness of the molten pool are observed to adjust the process parameters such as laser power in real time to keep the molten pool stable and obtain good cladding quality. This paper summarizes the latest research progress in the use of molten pool geometry information as feedback signals for real-time monitoring of laser cladding molten pools at home and abroad, focusing on the collection and processing of molten pool images, and provides a reference for real-time online monitoring and intelligent control of laser cladding.

1 Research status of laser cladding molten pool monitoring and closed-loop control

During the cladding process, there are complex physical and chemical phenomena of optoelectronic multi-parameter coupling between the laser beam, cladding material and substrate. The economic development of laser cladding process application is hindered due to the lack of ability to accurately predict the complex physical phenomena related to the process. During the cladding process, there are phenomena such as heat transfer and convection in the molten pool. The temperature and morphology of the molten pool are important factors affecting the quality of cladding forming. Therefore, molten pool monitoring is an important prerequisite for realizing laser cladding automation.

The stability of the molten pool significantly affects the forming quality and forming stability of the cladding layer. When the molten pool is not properly controlled, many defects such as pores and cracks will occur. This is largely attributed to the lack of molten pool monitoring and closed-loop control algorithms during the cladding process. Therefore, it is particularly important to monitor and accurately control the molten pool. With the rapid development of deep learning and image processing technology, non-contact detection technologies such as vision, acoustic emission, temperature field detection and spectral analysis are widely used in molten pool monitoring, as shown in Table 1. The molten pool can reflect the stability of the cladding process and judge the quality of the cladding layer. Through molten pool monitoring, the current cladding state, such as the generation of defects such as cracks, can be identified, thereby reducing the instability of the cladding process caused by molten pool size errors and process parameters, and improving the cladding molding quality.

As one of the non-contact detection technologies, visual monitoring is less affected by the processing materials, processing methods and environment during the monitoring process, with a short response time and high accuracy and sensitivity. It can collect rich molten pool image information, which is helpful for the subsequent realization of laser cladding closed-loop control. It is one of the common methods for domestic and foreign scholars to monitor the molten pool for laser cladding.

1.1 Research status of laser cladding molten pool monitoring
Molten pool monitoring is to monitor various dimensional parameters of the molten pool morphology, establish a feedback control system, maintain the stability of the cladding process, and reduce defects such as dimensional anomalies and deformation. As shown in Figure 1, the monitoring system is mainly composed of a high-speed camera, a lens, a filter and a lighting system. High-speed cameras are used to capture and image the molten pool during the cladding process. Digital image processing technology is used to obtain molten pool information (such as molten pool morphology, molten pool width, molten pool height, etc.). This information can be used as the output of laser cladding closed-loop control, laying the foundation for the realization of laser cladding closed-loop control.

The molten pool image contains rich molten pool information, such as the length, width, height and molten pool area of ​​the molten pool, which is closely related to the quality of the cladding layer. Kao et al. monitored the laser cladding process using a coaxial camera and proved that there is a correlation between the coaxial monitoring molten pool image and the quality of the cladding layer. Yong et al. extracted dimensional information such as molten pool area, width and height through visual monitoring, and used APDL for simulation. By comparing the monitoring and simulation results, the effectiveness of the visual monitoring system was verified.

There are many interference factors in the molten pool image. Obtaining a clear molten pool image is an important prerequisite for extracting molten pool information. In order to eliminate the interference caused by camera positioning on image acquisition, domestic and foreign scholars have conducted a series of studies. For example, Chkalov et al. studied the optimal layout of the laser exposure area lighting system and CCD camera. The results showed that the clearest molten pool image can be obtained when the visual camera and the lighting system are both distributed on the back side of the molten pool. Li et al. [35] proposed a molten pool image position calibration method based on image processing technology to eliminate the positioning error caused by the tilt of the CCD industrial camera and obtain an accurate molten pool contour, which laid the foundation for subsequent image processing.

In addition, there are interferences such as spatter, plume, noise and flare in the cladding process. The molten pool image needs to be processed to obtain a clear molten pool image. For example, Song et al. used a high-speed camera to build a near-axis real-time molten pool monitoring system and proposed a phase-consistent molten pool edge extraction method to handle interferences such as blurred areas and flares. Le et al. used a CMOS high-speed camera to build an off-axis monitoring system to achieve the calculation of the molten pool size and the removal of spatter interference. Jiang Shujuan et al. detected the molten pool width based on Kalman filter technology, reducing noise interference in the measurement process, thereby significantly improving the detection accuracy of the molten pool width.

Real-time monitoring of the molten pool can realize the measurement and prediction of molten pool characteristics such as molten pool width. It can also realize the detection of defects and the control of cladding layer quality by studying the laws of molten pool geometric characteristics and laser power, scanning speed and powder feeding rate. Sampson et al. showed through experiments that there is a complex interaction between laser power, powder feeding rate and scanning speed. When the laser power is high, increasing the powder feeding rate will lead to an increase in the molten pool width. When the laser power is low, the molten pool width remains unchanged. Regardless of the laser power, increasing the scanning speed will lead to a decrease in the molten pool width. Chen et al. [41] studied the relationship between process parameters such as laser power and scanning speed and molten pool area, and realized accurate detection of different types of forming defects based on the molten pool area. Concalves et al. used a coaxial camera to capture the molten pool image, used the molten pool image as input, estimated the height and width of the cladding layer, developed and compared 6 convolutional neural network (CNN) architectures, and found that the determination coefficient of the Adadelta neural network architecture was higher than 0.98, with the best prediction performance.

1.2 Current status of closed-loop control of laser cladding
The closed-loop control of the laser cladding process can predict and control the defects generated in the laser cladding process, maintain the stability of the molten pool, and improve the cladding quality. In the laser cladding process, the defects (pores, cracks, etc.) in the cladding process are timely discovered by detecting the molten pool morphology (width, height, contour, etc.), and the cladding quality is improved by adjusting certain correction parameters through feedback control.

There are two commonly used closed-loop control methods. The first method is to compare the expected value with the online molten pool characteristics, and then adjust the process parameters such as laser power in time according to the deviation e based on the control algorithm, so as to stabilize the molten pool size and realize the closed-loop control of the molten pool process. The principle is shown in Figure 2. Arias et al. designed and developed a laser cladding control system based on FPGA, using a CMOS camera to measure the molten pool width, and realized closed-loop control of the laser cladding process by adjusting the laser power. Liu Xuyang built a real-time monitoring and control system for the molten pool based on a CMOS camera, compared the difference between the monitored molten pool width and the expected molten pool width, and adjusted the scanning speed in the cladding process in real time through a PID controller to achieve real-time monitoring and closed-loop control of the laser cladding process. Yang Yongxing [46] used a CCD camera to capture the molten pool image, and used a fuzzy control method to control the scanning rate, and controlled the cladding layer height by adjusting the scanning rate.

The second method is to adjust the cumulative error so that the final result converges to the expected value. Moralejo et al. developed a feedforward proportional integral controller that can acquire and process the molten pool image in real time based on a CMOS camera, with laser power as input and cladding layer width as output, to achieve closed-loop control and conduct real-time verification in the processing of variable-width cladding parts. Ding et al. installed a CCD camera on the side of the laser head to monitor the top of the molten pool, combined with feedforward compensation and PID controller, to achieve closed-loop control of the molten pool size. Shi Tuo et al. used P and PI controllers to calculate the deviation between the actual layer height and the expected value to adjust the scanning speed, laser power and other process parameters, and control the actual layer height to continuously converge to the expected value. Table 2 summarizes the different feedback methods in the cladding process. The first method has a fast response speed and can improve the stability of the molten pool and the forming accuracy of the cladding layer during the cladding process. However, in the process of layer-by-layer accumulation of the cladding layer, the cumulative error is large, and the final forming accuracy is poorly controllable. Compared with the first method, the second method has a slower response speed, but the final forming accuracy is controllable.

2 Molten pool image processing and feature extraction

In the laser cladding process, the molten pool is generated by the powder on the substrate under the action of a high-power laser beam. Its geometric morphology reflects the quality of the cladding layer. Real-time monitoring of the molten pool and feedback control are important means to maintain the stability of the molten pool to improve the quality of the cladding layer. During the molten pool monitoring process, high-quality molten pool images can reflect more molten pool information, such as molten pool area, width and height. Since there are many interferences in the cladding process, such as powder splash, plume, noise, flare and light reflection, it is challenging to capture high-quality molten pool images and extract molten pool information. Therefore, image processing technology is one of the important components of molten pool monitoring.

2.1 Molten pool image preprocessing

2.1.1 Image denoising
Image denoising is one of the key steps in image preprocessing. During the cladding process, there are a lot of noise signals such as splash and smoke. Therefore, it is necessary to use image filtering methods to remove noise and other interferences in the molten pool image, so as to highlight the molten pool area, reduce the influence of arc light, uneven brightness and noise on molten pool identification, obtain the clearest molten pool image, and facilitate the subsequent extraction of molten pool information.

Filtering the image is a common means to remove noise interference. Commonly used filtering methods include linear filtering and nonlinear filtering, including mean filtering, Gaussian filtering, median filtering and bilateral filtering. Domestic and foreign scholars have tried various filtering algorithms to deal with noise interference such as splashes in the molten pool image and compared their noise reduction effects. For example, Hui Wanyu et al. denoised images with added salt and pepper noise, Gaussian noise, and salt and pepper Gaussian mixed noise, and compared the denoising effects of Gaussian filtering and median filtering methods, and concluded that the denoising effect of median filtering is better than that of Gaussian filtering. Table 3 summarizes the characteristics of different filtering methods in the denoising process.

In summary, linear filtering does not retain edge information while reducing noise, which can easily cause image blur and destroy image details, while nonlinear filters can effectively protect image details and edges while suppressing noise. Therefore, nonlinear filtering is often used in most molten pool image processing. For example, the median filtering algorithm is simple and has a good noise reduction effect in random noise processing. It can achieve the purpose of removing noise interference in the image while retaining the edge details of the molten pool to the greatest extent and restoring the molten pool information.

2.1.2 Image Enhancement
After the noise interference in the molten pool image is removed by image filtering, some details in the image are smoothed. At this time, the molten pool image needs to be enhanced to increase the effective detail gray value, thereby increasing the contrast between the molten pool and the surrounding area, highlighting the detail features of the molten pool, and making the enhanced image have a better visual effect and easier to be recognized by the machine, laying the foundation for image segmentation. Due to the real-time nature of the image enhancement algorithm, a variety of image enhancement algorithms have emerged in recent years, as summarized in Table 4.

In summary, a single image enhancement algorithm also has certain disadvantages while enhancing the image. For example, the histogram equalization algorithm has good enhancement effect, high real-time performance and high efficiency, but it is easy to lose image details and even enhance the noise signal in the image. The advantages and disadvantages of the image algorithm should be comprehensively considered, and the image algorithm should be optimized and improved or a combination of multiple algorithms can significantly improve the image features of the laser cladding molten pool, laying a good foundation for subsequent image segmentation and molten pool feature extraction. For example, Mao Wei et al. considered the advantages and disadvantages of a single algorithm and combined with the processing characteristics of laser cladding, and proposed the Heat-HE infrared image enhancement algorithm based on the histogram equalization algorithm. Under the premise of removing noise and other interference, the image features of the cladding layer were significantly enhanced.

2.2 Image segmentation
Image segmentation is to divide the image area into several sub-areas, each sub-area does not intersect with each other and the internal features of the same area have a certain correlation. After image denoising and enhancement, the contrast between the molten pool and the surrounding area is improved, the molten pool image is clearer, and the details are more prominent. At this time, the image contains the molten pool area and the background area. In order to obtain the molten pool area, it is necessary to perform image segmentation on the molten pool and the background area. The main image segmentation methods include threshold segmentation and edge segmentation, etc., as summarized in Table 5.

Threshold segmentation uses the difference in grayscale values ​​between the molten pool and the background to set pixels greater than the threshold as targets, and accurately segment the image into two parts: the target and the background. The key lies in setting a suitable grayscale threshold. For example, Meng Qingdong determined the appropriate grayscale threshold by determining the minimum fuzzy degree in fuzzy C-means clustering based on fuzzy set theory. However, the cladding process is complex, and a single threshold segmentation method sometimes cannot accurately segment the molten pool. Multiple segmentation algorithms can be combined according to image features. For example, Dong Fangyu et al. divided the image into three parts: the molten pool, plume and background, and combined the K-means algorithm and the maximum inter-class variance (Otsu) double threshold segmentation method to segment the image, which improved the accuracy of the molten pool.

Edge segmentation is based on the difference between the target area and the surrounding background, and the target area edge is separated from the background environment by pixel difference. In the molten pool image, the grayscale change of the molten pool edge is the most drastic. This feature can be used in combination with the edge detection algorithm to extract the molten pool edge. For example, Wang Renjie et al. extracted the complete molten pool outline based on the Canny edge detection algorithm. However, for images without obvious edges in the melt pool image, the traditional edge detection algorithm has poor extraction effect. The algorithm can be improved to improve the algorithm performance. For example, Shan Jun et al. improved the traditional ant colony algorithm to successfully extract the edge of the melt pool in view of the defect that the traditional ant colony algorithm easily misses edge points. Yan et al. [82] optimized and improved the mathematical morphology edge detection algorithm and compared it with various processing algorithms. It was concluded that this algorithm can better extract the edge of the melt pool and has good noise resistance.

2.3 Melt pool information extraction

After image segmentation, the image is divided into two parts: the melt pool area and the background area. At this time, it is necessary to further obtain the size information of the melt pool, such as the melt pool area, length and width.

In the melt pool image, the size measurement is based on pixels, and the camera installation angle, installation distance and installation error in the melt pool shooting system will also cause image distortion. In order to obtain the real size information of the melt pool, the melt pool image needs to be physically calibrated. As shown in Figure 3, Li Xin placed the ruler under the cladding head and calibrated it by using the same monitoring equipment to measure the number of pixels corresponding to the scale, and obtained the conversion ruler of 0.0164mm/pixel; Guo Bo used the MATLAB calibration toolbox to calibrate the visual camera, determined the parameters of the visual camera, and the calibration error ε∈[-0.5671, 0.4389]. Yu, Yang Zhengyu and others used a black and white chessboard calibration plate to calibrate the camera, and obtained that the ratio of camera pixels to actual size was 1: 0.149 mm, and the height of the molten pool was obtained by subtracting the substrate thickness from the molten pool coordinates.

The purpose of capturing and processing the molten pool image is to obtain the geometric feature information in the molten pool, such as the length, width, height and area of ​​the molten pool. After image calibration, the feature extraction of the molten pool can be performed to obtain the characteristic size of the molten pool. As shown in Figure 4, Yang et al. used the minimum circumscribed rectangle method to replace the length of the molten pool with the length of the rectangle and the width of the rectangle with the width of the molten pool, thus accurately extracting the length and width of the molten pool; Kim et al. used the ellipse fitting method to locate the center of the ellipse as the center of the molten pool, and obtained the length and width of the molten pool corresponding to the major and minor axis dimensions of the molten pool.

Extracting the molten pool characteristics is the purpose of real-time monitoring of the molten pool, and it is also an important prerequisite for achieving closed-loop control of the laser cladding process. Real-time monitoring of a molten pool feature and real-time control of the molten pool feature by adjusting the process parameters can predict and control the defects in the cladding process, thereby maintaining the stability of the cladding process and improving the quality of the cladding layer.

3 Conclusion and Outlook

There are many complex reactions such as physics and chemistry in the laser cladding process, which are easily affected by multiple internal and external factors, thus affecting the cladding quality. The molten pool image can be collected by a visual camera, and the molten pool image can be processed and the molten pool geometric information can be extracted to realize the monitoring and control of the molten pool, thereby ensuring the stability of the molten pool geometric morphology during the cladding process and obtaining good forming quality. In terms of laser cladding molten pool monitoring, the following aspects can be studied:

(1) Multi-information fusion monitoring system: In the visual monitoring system, the information of the molten pool is collected, and only the molten pool morphology information is obtained. The information is relatively simple. It can be combined with spectroscopy, acoustic emission, temperature sensing and other technologies to establish a multi-information fusion monitoring system integrating sound, light and temperature signals. The molten pool is monitored in all directions during the cladding process to improve the quality of the cladding layer.

(2) Internal defect detection: The laser cladding online monitoring technology based on visual imaging is mostly used to monitor the molten pool morphology. There is less research on internal defect monitoring. Based on the molten pool morphology transformation, a corresponding relationship with the internal defects of the cladding layer can be established to achieve real-time monitoring and prevention of defects.