Analysis and Applicability of Results

Copious datasets provide a possibility for comprehensive and unbiased testing of the obtained ML models with data that they haven’t been previously exposed, the same way as it normally would occur in any engineering application. Figure 4 presents the testing results for the TabNet model for the tensile strength temperature dependency for a wide range of 687 titanium alloys, where percentage of titanium varies from 55.3 to 99.8%.

While testing results and quality parameters differ from model to model and from property to property, the accuracy achieved in the terms of Mean Absolute Percentage Error (MAPE) has a median value of 4 to 5%, while correlation coefficient R2 is in most cases above 90%, and frequently above 95%. While absolute accuracy and certainty can never be guaranteed, the achieved levels are generally acceptable for most engineering applications and CAE simulations. They are comparable with other data sources that are being frequently used by engineers, such as producers’ specifications and curated data collections, and generally demonstrate higher accuracy than MMPDS B-Basis values, which are used in aerospace industry [].

Figure 4. Results of Tabnet model for the tensile strength temperature dependency of titanium alloys using (a) regular and (b) extended dataset

Beside testing of accuracy, model quality assurance includes various other metrics and considerations: applicability domain, sensitivity analysis, feature importance, explainability and confidence intervals at the datapoint level (Figure 5).

Figure 5. a) Sensitivity analysis of temperature effect on the heat capacity of aluminum alloys; b) Feature importance for thermal expansion of copper alloys; c) Features contributions to 50 predictions for the elastic modulus of titanium alloys (TabNet) d) Confidence intervals for shear modulus of stainless steels.

For successful applicability of these models, they need to be integrated with an interface that is efficient and easy to use for the engineer. This has been implemented through Total Materia Predictor, a specialized product within the Total Materia family [], where the user can simply select a material or chemical composition, define the mechanical or physical property to be modeled and optionally add some other inputs such as product and heat treatment. As a result, modeled values of the property are obtained, together with all relevant parameters of the quality of the model, Figure 6.

Figure 6. (a) Defining input parameters for a Tabnet ANN model

Figure 6. (b) Model results for the tensile strength temperature dependency for an aluminum alloy

Once obtained, the property points can be simply exported to Excel and then used further. Alternatively, by using tools within Total Materia such as Console, they can be saved, comparted with other materials or conditions, formatted into a report, shared and exported directly into a CAE solver of choice.