Interpetable Support Vector Machines in Regression and Classification – Application in Process Engineering

Abstract

Tools from the armoury of soft computing have been in focus of researches recently, since soft computing techniques are used for fault detection (classification techniques), forecasting of time-series data, inference, hypothesis testing, and modelling of causal relationships (regression techniques) in process engineering. These techniques solve two cardinal problems: learning from experimental data by neural networks and support vector based techniques and embedding existing structured human knowledge into fuzzy models. Support vector based models are one of the most commonly used soft computing techniques. Support vector based models are strong in feature selection and to achieve robust models and fuzzy logic helps to improve the interpretability of models. This paper deals with combining these existing soft computing techniques to get interpretable but accurate models for industrial purposes. The paper describes that trained support vector based models can be used for the construction of fuzzy rule-based classifier or regression models. However, the transformed support vector model does not automatically result in an interpretable fuzzy model because the support vector model results in a complex rulebase, where the number of rules is approximately 40-60% of the number of the training data. Hence, reduction of the support model-initialized fuzzy model is an essential task. For this purpose, a three-step reduction algorithm is used on the combination of previously published model reduction techniques. In the first step, the identification of the SV model is followed by the application of the Reduced Set method to decrease the number of kernel functions. The reduced SV model is then transformed into a fuzzy rule-based model. The interpretability of a fuzzy model highly depends on the distribution of the membership functions. Hence, the second reduction step is achieved by merging similar fuzzy sets based on a similarity measure. Finally, in the third step, an orthogonal least-squares method is used to reduce the number of rules and re-estimate the consequent parameters of the fuzzy rule-based model. The proposed approach is applied for classification problems and applied for Hammerstein system identification to illustrate the effectiveness of the technique.