Analysing the efficacy of feature elimination and adaptive sampling in meta-model based exploration of agent-based simulation models

Abstract

In this thesis, an advanced procedure is constructed to investigate agent-based simulation models. A meta-modeling approach, that utilizes adaptive sampling and feature elimination methods is used in the proposed procedure. The procedure aims to build a machine learning model that replicates the input-output relationships of the original agent-based simulation model and accurately predicts the output of inter est. Thanks to feature importance measurements, the proposed procedure also enables researchers to analyse the relationships between the agent-based simulation model pa rameters and the output of interest. The Random Forest algorithm is used for building the meta-model. The adaptive sampling method is utilized to create a high-quality data set to train the meta-model. The feature elimination process is applied to enable meta-model to prevent the curse of dimensionality and keep the focus on important fea tures regarding the output of interest. The proposed procedure is applied to a complex agent-based meta-model to evaluate its performance. A recent agent-based simulation model, that is analyzing socio-dynamic systems, is selected for application considering its probabilistic nature and wide range of parameters. Moreover, previously proposed meta-modeling approaches in the literature are reviewed and performance comparisons are assessed with the proposed procedure. Both the accuracy of output predictions and the validation of feature elimination decisions are analysed in detail. The conducted experiments and analysis showed that the proposed advanced procedure estimates the output of the original simulation model in an accurate and efficient way, and it out performed the previously proposed meta-modeling approach in terms of accuracy.