Multimodal deep learning-based on and off-axis melt pool monitoring for layer height and surface metrology predictions in directed energy deposition

Directed energy deposition—laser beam / metal with wire feedstock (DED-LB/M with wire) is an industrial additive manufacturing (AM) technology. It enables the fabrication of near-net-shape metal components with high deposition rates,
good material efficiency, and high precision. However, the inherent process complexity, characterized by the dynamic interaction between laser, melt pool, and feedstock, presents challenges for monitoring and control. These challenges are further amplified in multilayer deposition, where evolving part geometry and thermal accumulation introduce additional variability. The present study introduces a comprehensive real-time melt pool monitoring framework for multilayer deposition in DED-LB/M with wire with a 3 kW fiber laser and Inconel alloy 625 wire feedstock material, and its application in bead geometry and surface metrology prediction. The melt pool detection, segmentation and analysis were performed precisely by developing a convolutional neural networks (CNNs)-based model. The inference time and computational complexity of the developed model were analyzed and the capability of the model for real-time monitoring was confirmed. The developed YOLOv11-based model improved melt pool boundary prediction, increasing the segmented mask mean average precision (mAP50–95) by 3.3% over the original YOLOv11. It achieved an average processing speed of over 61 frames per second (fps) on an NVIDIA RTX A2000 GPU, confirming its real-time monitoring capability. Additionally, integrating a lightweight multimodal CNN, using Feature-wise Linear Modulation (FiLM) to embed process parameters into the image stream, within the segmentation head enabled accurate prediction of surface waviness and layer height, achieving testing dataset RMSE values of 21.37 µm and 7.34 µm, respectively. The proposed methodology enables accurate melt pool monitoring throughout multilayer deposition, establishing a foundation for improved quality assurance and process control. The results highlight the critical role of advanced vision-based monitoring techniques in addressing the challenges of multilayer AM and enhancing final part quality.