Perbandingan Ketahanan Lintas-Domain Indobert dan SVM Pada Ulasan Game dan Media Sosial Berbahasa Indonesia
Original Full-Text Article
Download published version for reading and archivingAbstract
Sentiment classifiers for Indonesian app reviews are rarely tested on any app other than the one that provided their training data. This study contrasts a linear support vector machine (SVM) with TF-IDF features and a fine-tuned IndoBERT. Both models were trained on Roblox reviews and then tested on held-out Roblox reviews as well as on Minecraft and TikTok reviews from Google Play. Each app contributed 2,500 class-balanced reviews, with labels derived from ratings. Performance was measured by macro-F1 with 95% bootstrap confidence intervals, and domain distance was assessed using five measures. On Roblox itself, IndoBERT and SVM showed no clear difference (0.867 vs 0.850). Across domains, IndoBERT scored roughly 0.065 higher and lost about 0.048 less. TikTok lay farther from Roblox on every distance measure and suffered larger drops, which is consistent with a distance effect. Domain-adaptive pretraining reduced the TikTok drop by 0.020 (macro-F1 rose from 0.794 to 0.810) but had no effect on Minecraft. In practical terms, IndoBERT is the safer option for developers who apply a classifier to other apps, while SVM is enough as a cheap in-domain baseline.
Keywords:
Author Biographies
Program Studi Sistem Informasi, Fakultas Ilmu Komputer, Universitas Sriwijaya, Kota Palembang, Provinsi Sumatera Selatan, Indonesia.
Program Studi Sistem Informasi, Fakultas Ilmu Komputer, Universitas Sriwijaya, Kota Palembang, Provinsi Sumatera Selatan, Indonesia.
Program Studi Sistem Informasi, Fakultas Ilmu Komputer, Universitas Sriwijaya, Kota Palembang, Provinsi Sumatera Selatan, Indonesia.
Program Studi Sistem Informasi, Fakultas Ilmu Komputer, Universitas Sriwijaya, Kota Palembang, Provinsi Sumatera Selatan, Indonesia.
Program Studi Sistem Informasi, Fakultas Ilmu Komputer, Universitas Sriwijaya, Kota Palembang, Provinsi Sumatera Selatan, Indonesia.
How to Cite
This work is licensed under a Copyright (c) 2026 Dyo Arya Purnama, Alsella Meiriza, Hardini Novianti, Rizka Dhini Kurnia, Ken Ditha Tania .
This is an open-access article distributed under the terms of the Creative Commons Attribution 4.0 International License .
- Share: You are free to copy, distribute, and transmit the work in any medium or format.
- Adapt: You are free to remix, transform, and build upon the work for any purpose, even commercially.
- Attribution: You must give appropriate credit, provide a link to the license, and indicate if changes were made. You may do so in any reasonable manner, but not in any way that suggests the licensor endorses you or your use.
References
Total: 19 References- Agro, M. T., & Aldarmaki, H. (2023). Handling realistic label noise in BERT text classification. Proceedings of the 6th International Conference on Natural Language and Speech Processing (ICNLSP), 11–20. https://doi.org/10.48550/arXiv.2305.16337.
- Athyabunnaja, S., Tamrin, T., & Mulyo, H. (2026). Implementation of IndoBERT for text-based emotion classification on TikTok application reviews in Google Play Store. Journal Dinda: Data Science, Information Technology, and Data Analytics, 6(1), 18–24. https://doi.org/10.20895/dinda.v6i1.10174.
- Calderon, N., Porat, N., Ben-David, E., Chapanin, A., Gekhman, Z., Oved, N., Shalumov, V., & Reichart, R. (2024). Measuring the robustness of NLP models to domain shifts. Findings of the Association for Computational Linguistics: EMNLP 2024, 126–154. https://doi.org/10.18653/v1/2024.findings-emnlp.7.
- Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2019). BERT: Pre-training of deep bidirectional transformers for language understanding. Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, 1, 4171–4186. https://doi.org/10.18653/v1/N19-1423.
- Du, W. (2025). When +1% is not enough: A paired bootstrap protocol for evaluating small improvements [Preprint]. arXiv. https://doi.org/10.48550/arXiv.2511.19794.
Similar Articles
Articles sharing related keywords and machine learning classifications:
Most read articles by the same author(s)
Other papers published by author(s) in this journal: