Recursive deep learning for Turkish sentiment analysis

Abstract

In this thesis, Recursive Deep Learning models have been implemented for Turkish sentiment analysis. Although natural language processing has made progress recently, representing compositional meanings is a challenging task. The traditional deep learning methods claim sentences as an ordinary linear structure, i.e. chains or sequences. In this thesis, tree-structured representations of the language have been developed to improve the compositional semantics of the Turkish language considering the morphological structure of the words. To this end, a novel Morphologically Enriched Turkish Sentiment Treebank (MS-TR) has been constructed to encode sentences structure. MS-TR is the first fully-labelled sentiment analysis treebank, which has four different annotation levels, including morph-level, stem-level, token-level, and review-level. Recursive Neural Tensor Networks (RNTN), which operate over MS-TR, have achieved much better results compared to the machine learning methods. In addition to the RNTN model, an advanced tree-structured LSTM model (ACT-LSTM) has been proposed as a novel recursive deep architecture. ACT-LSTM combines both attention and memory mechanisms over recursive tree structures, which learn latent structural information while learning more important parts of the sentences. ACT-LSTM has been compared with advanced chain-structured models to decide which architecture is better. As a third main contribution, a novel metaheuristic training algorithm has been proposed to overcome the vanishing and exploding gradients (VEG) problem, which is usually observed while training models. An enhanced ternary Bees Algorithm (BA-3+) has been implemented, which maintains low time complexity for large dataset classification problems by considering only three individual solutions in each iteration. The algorithm utilises the greedy selection strategy of the local solutions with exploitative search, stabilises the problem of VEGs of the decision parameters using SGD learning with singular value decomposition, and explores the random global solution with explorative search. BA-3+ has achieved faster convergence, avoiding getting trapped at local optima compared to the classical SGD training algorithm.