Sentiment Analysis of Customer Comments in Banking using BERT-based Approaches

Abstract

Customer comments collected by companies through various channels are useful resources for understanding customer satisfaction. The continuous increase in the amount of comments makes manual analysis infeasible. In this study, the comments of customers, written in Turkish, regarding banking services collected through NPS questionnaires were analyzed using Natural Language Processing methods. BERT-based sentiment classification models were developed and compared with traditional methods for the banking domain. The effectiveness of the methods was investigated in a low-resource setting, where (i) there is a small amount of labeled training data and (ii) there is no labeled training data in the target domain. For the first case, the results showed that BERTurk-based model performs better than the traditional models and its performance is affected less from the decrease in training data size. For the second case, training with out of domain data from Twitter was explored. In addition, zero-shot learning with XLM-Roberta, which was pertained for natural language inference, was investigated. While using out of domain data resulted in poor performance, the zero-shot learning approach achieved promising results for sentiment classification in the banking domain.