Combining multiple NUV-MIR spectral unmixing techniques for identifying mixtures of colorants in art and cultural heritage

Reflectance spectra in various frequency ranges (near-UV, visible and IR) are widely used in cultural heritage studies to characterize colorants (pigments and dyes) in paintings, manuscripts, and other artefacts. However, identifying the constituents of a paint mixture can be challenging, as the spectral features of a mixture may not match those of the individual substances. In conventional colorant analysis, a spectrum measured from a painted object is compared to reference spectra of known mixtures to identify the constituents of the paint examined. However, the number of reference spectra needed for a reliable mockup-based identification method depends on the research question, mixture complexity, and data quality. For complex mixtures, up to 500 distinct spectra may be needed, making this approach time-consuming and expensive.
To overcome these challenges, spectral unmixing techniques, which can decompose the spectrum of a mixture into its individual components, have been used. Spectral unmixing is based on the principle that a mixture spectrum is a linear or nonlinear combination of the spectra of its individual components. The method requires a library of reference spectra for the individual components, which are typically obtained by measuring the spectra of pure pigments and dyes or by using computational methods to simulate their spectra. Linear mixture analysis (LMA), non-negative matrix factorization (NMF), and vertex component analysis (VCA) are some of the techniques that have been successfully applied to identify colorants in paintings. These techniques can be used for data pre-processing, endmember extraction, and abundance estimation.
Despite their success, the accuracy of these techniques can be impacted by the presence of other compounds not considered during the analysis, such as extenders, mordants, and binding media. These compounds can interfere with the spectral features of the individual components and can lead to inaccurate estimates of their spectra and abundances. To address this issue, a new method for identifying paint mixtures using machine learning and multiple spectral unmixing techniques is proposed. This method consists in applying a variety of unmixing techniques simultaneously, evaluating and interpreting the performance of each technique, and validating and combining the results. The proposed method was trained on a number of swatches of historical and modern colorants, bound in different media, produced by various paint manufacturers between the second half of the 20th century and the early 2000s, and was applied to a dataset of NUV-MIR (350-25000 nm) reflectance spectra collected from a set of selected European paintings and Japanese prints. The spectral data was acquired using a combination of UV-Vis-NIR reflectance spectroscopy and external reflection (ER)-FTIR spectroscopy.
The results showed that all three unmixing techniques were able to identify the major colorants and binding media present in the mixtures with relatively high accuracy. LMA and NMF were particularly effective in estimating the abundance of the individual colorants, while VCA was more robust to the presence of noise and other interference. Additionally, the combination of the three techniques provided a more comprehensive and accurate identification of the materials in the mixtures, as each technique captured different aspects of the spectral data. To validate the results, the estimated compositions of the paint layers were compared with the compositions determined by Raman and X-ray fluorescence (XRF) spectroscopies. The comparison showed a high degree of agreement between the estimated and actual compositions, indicating the reliability of the proposed method.
One limitation is that the accuracy of the method is strongly dependent on the quality of the spectral data, which may be affected by factors such as the surface unevenness of the object, the presence of varnish and retouching, and the measurement conditions. To address this weakness, future research will focus on developing more efficient and automated procedures for data acquisition, as well as on integrating the proposed method with other analytical techniques for a more comprehensive and multi-scale characterization of pigments and other materials in art and cultural heritage.
In conclusion, the proposed method for identifying mixtures of artist’s materials in cultural heritage studies using machine learning and spectral unmixing techniques has been demonstrated to be effective and reliable for the analysis of paints containing various amounts of pigments and dyes. By carefully selecting and applying suitable techniques, it is possible to obtain accurate and meaningful results in relation to the identification and quantification of colorants in mixtures, which can provide valuable insights into the materials and techniques used by artists as well as their historical and cultural context.