Text sentiment analysis methods (3) - Deep learning-based sentiment analysis methods

Introduction of deep learning-based sentiment analysis methods

The sentiment analysis methods based on deep learning are performed using neural networks, and the typical neural network learning methods are: Convolutional Neural Network (CNN), Recurrent Neural Network (RNN), Long Short-Term Memory (LSTM) and so on.

By subdividing the deep learning-based sentiment analysis methods, they can be divided into: single neural network sentiment analysis methods, hybrid (combined, fusion) neural network sentiment analysis methods, sentiment analysis by introducing attention mechanism and sentiment analysis using pre-trained models.

1. Single neural network sentiment analysis:

In 2003 Bengio et al. proposed a neural network language model, which uses a three-layer feedforward neural network to model the language. The neural network mainly consists of an input layer, a hidden layer, and an output layer.

Each neuron in the input layer of the network represents a trait, the number of hidden layers and hidden layer neurons are set manually, and the output layer represents the number of categorical labels, a basic three-layer neural network is shown below.

The essence of the language model is to predict the content of the next word based on the contextual information without relying on the manually labeled corpus, from which it can be found that the advantage of the language model is the ability to learn rich knowledge from the large-scale corpus.

This approach can effectively solve the problem of ignoring contextual semantics in traditional sentiment analysis-based methods.

2. Sentiment analysis by hybrid (combined, fused) neural networks:

In addition to the research on approaches to single neural networks, a number of scholars have combined and improved these approaches and used them in sentiment analysis after considering the advantages of different approaches.

Compared with sentiment analysis methods based on sentiment lexicons and traditional machine learning, the approach using neural networks has significant advantages in text feature learning, which can actively learn features and actively retain information about words in the text to better extract the semantic information of the corresponding words to effectively achieve sentiment classification of text.

As the concept of deep learning was proposed, many researchers have continuously explored it and got a lot of results, so the text sentiment classification methods based on deep learning are expanding.

3. Sentiment analysis with the introduction of attention mechanism:

Based on neural networks, in 2006, Hinton et al. pioneered the concept of deep learning to improve the performance of learning by learning key information in the data through deep network models to reflect the characteristics of the data.

Deep learning-based methods use continuous, low-dimensional vectors to represent documents and words, and thus can effectively solve the problem of sparse data. In addition, deep learning-based methods are end-to-end methods that automatically extract text features and reduce the complexity of text construction features.

Deep learning methods have made significant progress in the field of natural language processing, such as machine translation, text classification, and entity recognition, in addition to remarkable results in the fields of speech and image. The research on text sentiment analysis methods belongs to a small branch of text classification.

By adding attention mechanism to deep learning methods for sentiment analysis tasks, it can better capture contextually relevant information, extract semantic information and prevent the loss of important information, which can effectively improve the accuracy of text sentiment classification.

The current stage of research is more about fine-tuning and improving the pre-training model so as to enhance the experiments more effectively.

4. Sentiment analysis using pre-trained models:

A pre-trained model is a model that has been trained with a dataset. By fine-tuning the pre-trained model, better sentiment classification results can be achieved, so most of the latest methods use pre-trained models, and the latest pre-trained models are: ELMo, BERT, XL-NET, ALBERT, etc.

By making full use of the large-scale monolingual corpus compared with the traditional methods, the pre-training method using language models can model multiple meanings of a word, and the process of pre-training using language models can be regarded as a sentence-level contextual word representation.

By pre-training a large-scale corpus using a unified model or adding features to some simple models, good results have been achieved in many NLP tasks, indicating that this approach is significantly effective in alleviating the problem of reliance on model structure.

There will be more research on natural language processing tasks in the future, especially on sentiment mining of text. Most of the latest approaches to sentiment analysis are based on fine-tuning of pre-trained models and have achieved good results.

Therefore, it can be predicted that future sentiment analysis methods will focus more on researching deep learning-based methods and achieving better sentiment analysis results by fine-tuning the pre-training models.