Benchmarking Machine Learning Algorithms for Customer Churn Prediction in SaaS Platforms Serving SMEs: An Indonesian Case Study

Abstract

Customer churn is a critical challenge for Software-as-a-Service (SaaS) platforms serving small and medium-sized enterprises (SMEs), as it reduces recurring revenue and raises acquisition costs. While churn prediction has been studied extensively in telecommunications and finance, limited work focuses on SaaS SMEs in emerging markets such as Indonesia. This paper benchmarks four machine learning algorithms: Logistic Regression, Decision Tree, Random Forest, and XGBoost, to assess their effectiveness in predicting churn using a dataset of 18,491 SME customers from an Indonesian SaaS platform. The study applies a structured modeling pipeline including data preprocessing, feature engineering, and stratified train–test validation. Because churn data is imbalanced, model performance was evaluated using not only accuracy-related metrics but also ROC-AUC, PR-AUC, recall, F1-score, and F2-score, which emphasize the ability to detect at-risk customers. Results show that ensemble models consistently outperform baselines. XGBoost achieved the best overall performance (ROC-AUC = 0.93; PR-AUC = 0.93; Recall = 0.98; F1 = 0.91), demonstrating its robustness in identifying churners while maintaining high discrimination. These findings confirm that advanced ensemble methods are well-suited for churn prediction in SaaS platforms serving SMEs, where heterogeneous customer behavior and short subscription cycles intensify attrition risk. The contribution of this study is a technical benchmark that highlights the strengths and trade-offs of common classifiers in this domain. By presenting an Indonesian case study, it provides empirical evidence to guide both researchers and practitioners in developing predictive churn models that support data-driven retention strategies for SaaS providers in emerging markets.