An image-based machine learning approach for evaluating the efficacy of cleaning treatments in heritage conservation

This paper reports the first application of machine learning (ML) for the quantitative assessment of cleaning efficacy in a cultural heritage conservation context. The study was conducted within the framework of the EU Horizon MOXY project in which the extent to which plasma-generated atomic oxygen (AO) can be beneficially employed for the removal of unwanted materials from various substrates (textile, acrylics, oils, plaster etc.) was extensively explored. In particular, research assessed if AO cleaning can be added to the conservator’s toolbox for a number of specific cleaning challenges and this by benchmarking the performance of the innovative cleaning method under study with established cleaning approaches. In a first stage, simplified model systems (SMS) were prepared and treated to allow for characterization of the cleaning efficacy. Objectively defining how effectively a method removes unwanted material from such mock-ups requires unambiguous and quantifiable metrics. Although assessing the aptness of an innovative cleaning method is a multifaceted endeavour, in this work, we focus only on the cleaning efficacy, by investigating three non-invasive methodologies for evaluating the degree of soil removal from silk fabrics. Two of the approaches are spectral-based and well-established in the field: CIELAB colorimetric analysis and statistical evaluation of histogram brightness moments. The third one relies on an existing image-based supervised machine learning segmentation method, Trainable Weka Segmentation (TWS), here for the first time applied in a cultural heritage context. It is available as open-source plug-in in Fiji, making it accessible to a wide range of users. The performances of the ML method were validated by comparing two models, model 1 obtained through the conventional approach of training TWS on limited regions of interest from a single image and model 2, trained on all pixels from a large, diverse set of training images. Model 1 demonstrated robust performance, particularly in addressing the detection limitations posed by highly textured substrates. Furthermore, other analytical techniques, like infrared imaging and spectroscopy, gravimetric measurements and hyperspectral imaging, were tested but resulted unsuitable for an actual surface cleaning quantification. As such, our research did not only allow selecting the machine learning approach for evaluating the cleaning results on SMS within MOXY, but it is anticipated to be a valuable tool for future cleaning tests in the conservation field as well.