Vol. 3 No. 2 (2026) • Articles
Open Access

Perbandingan IndoBERT dan mBERT Untuk Klasifikasi Toksisitas Pada Chat Roblox dan Minecraft Berbahasa Indonesia

Idris Sonny
Sriwijaya University image/svg+xml
Allsela Meiriza
Sriwijaya University image/svg+xml
Hardini Novianti
Sriwijaya University image/svg+xml
Mira Afriani
Sriwijaya University image/svg+xml
Rizka Dhina Kurnia
Sriwijaya University image/svg+xml
Published: November 25, 2026 Pages: 557-569
Original Full-Text Article
Download published version for reading and archiving
Abstract

Toxic chat in online games exposes young players to harassment, while moderation of Indonesian chat is hampered by slang and non-standard spelling. This study compares monolingual IndoBERT and multilingual mBERT for four-class toxicity classification (neutral, violence, racist, harassment) of Indonesian Roblox and Minecraft chat to identify the more suitable model for moderation. Both models were fine-tuned with identical settings on the Indonesian Chat Dataset (10,702 chats) using three seeds, and evaluated with accuracy, macro-averaged precision, recall, F1-score, and McNemar’s test. IndoBERT reached a mean macro F1-score of 0.8541 ± 0.0019 versus 0.8263 ± 0.0090 for mBERT, significant in two of three seeds. Neutral was the hardest class, and IndoBERT let 5.8% of toxic chats pass as neutral versus 8.3% for mBERT. Excluding 106 duplicated test chats did not change the ranking. Under these settings, IndoBERT performed better, although labels lack inter-annotator agreement and data cover two games only.

Article Metrics & Downloads Graph
Monthly Download Trends:
Author Biographies
Idris Sonny Sriwijaya University

Program Studi Sistem Informasi, Fakultas Ilmu Komputer, Universitas Sriwijaya, Kota Palembang, Provinsi Sumatera Selatan, Indonesia.

Allsela Meiriza Sriwijaya University

Program Studi Sistem Informasi, Fakultas Ilmu Komputer, Universitas Sriwijaya, Kota Palembang, Provinsi Sumatera Selatan, Indonesia.

Hardini Novianti Sriwijaya University

Program Studi Sistem Informasi, Fakultas Ilmu Komputer, Universitas Sriwijaya, Kota Palembang, Provinsi Sumatera Selatan, Indonesia.

Mira Afriani Sriwijaya University

Program Studi Sistem Informasi, Fakultas Ilmu Komputer, Universitas Sriwijaya, Kota Palembang, Provinsi Sumatera Selatan, Indonesia.

Rizka Dhina Kurnia Sriwijaya University

Program Studi Sistem Informasi, Fakultas Ilmu Komputer, Universitas Sriwijaya, Kota Palembang, Provinsi Sumatera Selatan, Indonesia.

How to Cite
Sonny, I., Meiriza, A., Novianti, H. ., Afriani, M., & Kurnia, R. D. (2026). Perbandingan IndoBERT dan mBERT Untuk Klasifikasi Toksisitas Pada Chat Roblox dan Minecraft Berbahasa Indonesia. Jurnal Manajemen Dan Teknologi, 3(2), 557-569. https://doi.org/10.63447/jmt.v3i2.2196
License

This work is licensed under a Copyright (c) 2026 Idris Sonny, Allsela Meiriza, Hardini Novianti, Mira Afriani, Rizka Dhina Kurnia .

Creative Commons Attribution 4.0 International License (CC BY 4.0)

This is an open-access article distributed under the terms of the Creative Commons Attribution 4.0 International License .

  • Share: You are free to copy, distribute, and transmit the work in any medium or format.
  • Adapt: You are free to remix, transform, and build upon the work for any purpose, even commercially.
  • Attribution: You must give appropriate credit, provide a link to the license, and indicate if changes were made. You may do so in any reasonable manner, but not in any way that suggests the licensor endorses you or your use.
Copyright & Retention: Authors retain copyright without restrictions and grant this journal the right of first publication under an open-access model. The journal retains non-exclusive publishing rights for archiving, indexing, and scholarly dissemination.

References
Total: 20 References
  1. Dharmawan, S., Mawardi, V. C., & Perdana, N. J. (2023). Klasifikasi ujaran kebencian menggunakan metode feedforward neural network (IndoBERT). Jurnal Ilmu Komputer dan Sistem Informasi, 11(1). https://doi.org/10.24912/jiksi.v11i1.24066.
  2. Ghosh, K., & Senapati, A. (2025). Hate speech detection in low-resourced Indian languages: An analysis of transformer-based monolingual and multilingual models with cross-lingual experiments. Natural Language Processing, 31(2), 393–414. https://doi.org/10.1017/nlp.2024.28.
  3. Hakim, A. N., Sibaroni, Y., & Prasetyowati, S. S. (2024). Detection of hate-speech text on Indonesian Twitter social media using IndoBERTweet-BiLSTM-CNN. In 2024 12th International Conference on Information and Communication Technology (ICoICT) (pp. 374–381). IEEE. https://doi.org/10.1109/ICoICT61617.2024.10698615.
  4. Hibatullah, H., Ballı, T., & Yetkin, E. F. (2025). Verbal harassment detection in online games using machine learning methods. Entertainment Computing, Article 101009. https://doi.org/10.1016/j.entcom.2025.101009.
  5. Ibrohim, M. O., & Budi, I. (2023). Hate speech and abusive language detection in Indonesian social media: Progress and challenges. Heliyon, 9(8), Article e18647. https://doi.org/10.1016/j.heliyon.2023.e18647.
  6. Isa, S. M., Nico, G., & Permana, M. (2022). IndoBERT for Indonesian fake news detection. ICIC Express Letters, 16(3), 289–297. https://doi.org/10.24507/icicel.16.03.289.
  7. Kartika, B. V., Martin, J. A., & Kusuma, G. P. (2023). Fine-tuned IndoBERT based model and data augmentation for Indonesian language paraphrase identification. Revue d'Intelligence Artificielle, 37(3), 733. https://doi.org/10.18280/ria.370322.
  8. Kordyaka, B., Karaosmanoglu, S., & Laato, S. (2025). Defining toxicity in multiplayer online games: A systematic literature review. Computers in Human Behavior Reports, Article 100698. https://doi.org/10.1016/j.chbr.2025.100698.
  9. Min, B., Ross, H., Sulem, E., Veyseh, A. P. B., Nguyen, T. H., Sainz, O., Agirre, E., Heintz, I., & Roth, D. (2023). Recent advances in natural language processing via large pre-trained language models: A survey. ACM Computing Surveys, 56(2), 1–40. https://doi.org/10.1145/3605943.
  10. Nabiilah, G. Z., Alam, I. N., Purwanto, E. S., & Hidayat, M. F. (2024). Indonesian multilabel classification using IndoBERT embedding and MBERT classification. International Journal of Electrical and Computer Engineering, 14(1), 1071–1078. https://doi.org/10.11591/ijece.v14i1.pp1071-1078.
  11. Nabiilah, G. Z., Prasetyo, S. Y., Izdihar, Z. N., & Girsang, A. S. (2023). BERT base model for toxic comment analysis on Indonesian social media. Procedia Computer Science, 216, 714–721. https://doi.org/10.1016/j.procs.2022.12.188.
  12. Pamungkas, E., Putri, D., & Fatmawati, A. (2023). Hate speech detection in Bahasa Indonesia: Challenges and opportunities. International Journal of Advanced Computer Science and Applications, 14(6), 1175–1181. https://doi.org/10.14569/IJACSA.2023.01406125.
  13. Prestiliano, J., Azhari, A., & Nurwidyantoro, A. (2026). Multimodal deep learning violence detector for child-friendly online game. International Journal of Serious Games, 13(1), 41–61. https://doi.org/10.17083/9k96e890.
  14. Rahmawati, A., Alamsyah, A., & Romadhony, A. (2022). Hoax news detection analysis using IndoBERT deep learning methodology. In 2022 10th International Conference on Information and Communication Technology (ICoICT) (pp. 368–373). IEEE. https://doi.org/10.1109/ICoICT55009.2022.9914902.
  15. Suharjanto, D., & Sumarsono, S. (2025). Optimalisasi DoRA untuk deteksi ujaran kebencian berbahasa Indonesia berbasis transformer. Jurnal Nasional Teknologi dan Sistem Informasi, 11(3), 341–349. https://doi.org/10.25077/TEKNOSI.v11i3.2025.341-349.
  16. Susanto, L., et al. (2024). IndoToxic2024: A demographically-enriched dataset of hate speech and toxicity types for Indonesian language. arXiv preprint arXiv:2406.19349. https://doi.org/10.48550/arXiv.2406.19349.
  17. Tobing, C. J. L., Wijayakusuma, I. G. N. L., & Harini, L. P. I. (2025). Perbandingan kinerja IndoBERT dan mBERT untuk deteksi berita hoaks politik dalam bahasa Indonesia. JST (Jurnal Sains dan Teknologi), 14(1), 114–123. https://doi.org/10.23887/jstundiksha.v14i1.92126.
  18. Wijkstra, M., Rogers, K., Mandryk, R. L., Veltkamp, R. C., & Frommel, J. (2024). How to tame a toxic player? A systematic literature review on intervention systems for toxic behaviors in online video games. Proceedings of the ACM on Human-Computer Interaction, 8(CHI PLAY), 1–32. https://doi.org/10.1145/3677080.
  19. Yang, Z., Grenon-Godbout, N., & Rabbany, R. (2023). Towards detecting contextual real-time toxicity for in-game chat. In Findings of the Association for Computational Linguistics: EMNLP 2023 (pp. 9894–9906). Association for Computational Linguistics. https://doi.org/10.18653/v1/2023.findings-emnlp.663.
  20. Yanti, O. D., Alfariani, S., Celesta, S. N. M., Tania, K. D., & Rifai, A. (2026). Comparative analysis of the performance of machine learning methods and text embedding techniques in classifying toxic conversations in the Roblox game. Journal of Applied Informatics and Computing, 10(3), 2775–2789. https://doi.org/10.30871/jaic.v10i3.12646.
Similar Articles

Articles sharing related keywords and machine learning classifications:

Perbandingan Ketahanan Lintas-Domain Indobert dan SVM Pada Ulasan Game dan Media Sosial Berbahasa Indonesia
Analisis Perbedaan Tarif Rumah Sakit, Tarif INA-CBG’s, dan Tarif IDRG pada Pasien Diabetes Melitus Tipe 2 Rawat Inap di RS X
Klasifikasi Kualitas Karet Menggunakan Metode Random Forest Pada Perusahaan XYZ
Most read articles by the same author(s)

Other papers published by author(s) in this journal:

Klasifikasi Kualitas Karet Menggunakan Metode Random Forest Pada Perusahaan XYZ
Perbandingan Ketahanan Lintas-Domain Indobert dan SVM Pada Ulasan Game dan Media Sosial Berbahasa Indonesia