One sentence is positive. One sentence is negative. What’s the sentiment of the entire text block?
Sentiment analysis tells you if text conveys a positive, negative, or neutral message. When applied to a stream of social media messages from an account or a hashtag, for example, you can determine whether sentiment is overall favorable or unfavorable. If you examine sentiments over time, you can analyze them for trends or attempt to correlate them against external data. Based on this analysis, you could then build predictive models.
This is the second article in a series on performing sentiment analysis in Java by using the sentiment tool integrated into Stanford CoreNLP, an open source library for natural language processing (NLP).
In the first article, “Perform textual sentiment analysis in Java using a deep learning model,” you learned how to use this tool to determine the sentiment of a sentence from very negative to very positive. In practice, however, you might often need to look at a single aggregate sentiment score for the entire text block, rather than having a set of sentence-level sentiment scores.
Here, I will describe some approaches you can use to perform analysis on an arbitrarily sized text block, building on the Java code presented in the first article.
Scoring a multisentence text block
When you need to deal with a long, multisentence text block (such as a tweet, an email, or a product review), you might naturally want to have a single sentiment score for the entire text block rather than merely receiving a list of sentiment scores for separate sentences.
One simple solution is to calculate the average sentiment score for the entire text block by adding the sentiment scores of separate sentences and dividing by the number of sentences.
However, this approach is not perfect in most cases since different sentences within a text block can affect the overall sentiment differently. In other words, different sentences within a block may have varying degrees of importance when you calculate the overall sentiment.
There is no single algorithm for identifying the most-important sentences that would work equally well for all types of texts; perhaps that is why Stanford CoreNLP does not provide a built-in option for identifying the overall sentiment of a multisentence text block.
Fortunately, you can manually code such functionality to work best for the type of text you are dealing with. For example, text samples of the same type usually have something in common when it comes to identifying the most-important sentences.
Imagine you’re dealing with product reviews. The most-important statements—from the standpoint of the overall review sentiment—typically can be found at the beginning or the end of the review. The first statement usually expresses the main idea of the review, and the last one summarizes it. While this may not be true for every review, a significant portion of them look exactly like that. Here is an example.
I would recommend this book for anyone who wants an introduction to natural language processing. Just finished the book and followed the code all way. I tried the code from the resource website. I like how it is organized. Well done.
The Stanford CoreNLP sentiment classifier would identify the above sentences as follows:
Sentence: I would recommend this book for anyone who wants an introduction to natural language processing.
Sentiment: Positive(3)
Sentence: Just finished the book and followed the code all way.
Sentiment: Neutral(2)
Sentence: I tried the code from the resource website.
Sentiment: Neutral(2)
Sentence: I like how it is organized.
Sentiment: Neutral(2)
Sentence: Well done.
Sentiment: Positive(3)
As you can see, the first and the last sentences suggest that the review is positive. Overall, however, the number of neutral sentences in the review outnumber the positive statements, which means that an arithmetic linear average, where you give the same weight to each sentence, does not seem to be a proper way to calculate the overall sentiment of the review. Instead, you might want to calculate it with more weight assigned to the first and the last sentences, as implemented in the example discussed below.
The weighted-average approach
Continuing with the sample Java program introduced in the first article, add the following getReviewSentiment() method to the nlpPipeline class, as follows:
import java.util.*;
...
public static void getReviewSentiment(String review, float weight)
{
int sentenceSentiment;
int reviewSentimentAverageSum = 0;
int reviewSentimentWeightedSum = 0;
Annotation annotation = pipeline.process(review);
List<CoreMap> sentences = annotation.get(CoreAnnotations.SentencesAnnotation.class);
int numOfSentences = sentences.size();
int factor = Math.round(numOfSentences*weight);
if (factor == 0) {
factor = 1;
}
int divisorLinear = numOfSentences;
int divisorWeighted = 0;
for (int i = 0; i < numOfSentences; i++)
{
Tree tree = sentences.get(i).get(SentimentAnnotatedTree.class);
sentenceSentiment = RNNCoreAnnotations.getPredictedClass(tree);
reviewSentimentAverageSum = reviewSentimentAverageSum + sentenceSentiment;
if(i == 0 || i == numOfSentences -1) {
reviewSentimentWeightedSum = reviewSentimentWeightedSum + sentenceSentiment*factor;
divisorWeighted += factor;
}
else
{
reviewSentimentWeightedSum = reviewSentimentWeightedSum + sentenceSentiment;
divisorWeighted += 1;
}
}
System.out.println("Number of sentences:\t\t" + numOfSentences);
System.out.println("Adapted weighting factor:\t" + factor);
System.out.println("Weighted average sentiment:\t" + Math.round((float) reviewSentimentWeightedSum/divisorWeighted));
System.out.println("Linear average sentiment:\t" + Math.round((float) reviewSentimentAverageSum/divisorLinear));
}
The getReviewSentiment() method shown above illustrates how to calculate the overall sentiment of a review using two approaches, calculating both a weighted average and the linear average for comparison purposes.
The method takes the text of a review as the first parameter. As the second, you pass a weighting factor to apply to the first and the last sentences when calculating the overall review sentiment. The weighting factor is passed in as a real number in the range [0, 1]. To apply the scale to fit a particular review, you recalculate the weighting factor by multiplying the passed value by the number of sentences in the review, thus calculating the adapted weighting factor.
To test the getReviewSentiment() method, use the following code:
public class OverallReviewSentiment
{
public static void main(String[] args)
{
String text = "I would recommend this book for anyone who wants an introduction to natural language processing. Just finished the book and followed the code all way. I tried the code from the resource web site. I like how it is organized. Well done.";
nlpPipeline.init();
nlpPipeline.getReviewSentiment(text, 0.4f);
}
}
This example passes in 0.4 as the weighting factor, but you should experiment with the value passed in. The higher this value, the more importance is given to the first and last sentences in the review.
To see this approach in action, recompile the nlpPipeline class and compile the newly created OverallReviewSentiment class. Then, run OverallReviewSentiment, as follows:
$ javac nlpPipeline.java
$ javac OverallReviewSentiment.java
$ java OverallReviewSentiment
This should produce the following results:
Number of sentences: 5
Adapted weighting factor: 2
Weighted average sentiment: 3
Linear average sentiment: 2
As you can see, the weighted average shows a more relevant estimate of the overall sentiment of the review than the linear average does.
Sequential increases in weight ratios
When it comes to storylike texts that cover a sequence of events spread over a time span, the importance of sentences—from the standpoint of the overall sentiment—often increases as the story goes. That is, the most important sentences in the sense of having the most influence on the overall sentiment conveyed by the story are typically found at the end, because they describe the most-recent episodes, conclusions, or experiences.
Consider the following tweet:
The weather in the morning was terrible. We decided to go to the cinema. Had a great time.
The sentence-level sentiment analysis of this story gives the following results:
Sentence: The weather in the morning was terrible.
Sentiment: Negative(1)
Sentence: We decided to go to the cinema.
Sentiment: Neutral(2)
Sentence: Had a great time.
Sentiment: Positive(3)
Although the tweet begins with a negative remark, the overall sentiment here is clearly positive due to the final note about time well spent at the movies. This pattern also works for reviews where customers describe their experience with a product much like a story, as in the following example:
I love the stories from this publisher. They are always so enjoyable. But this one disappointed me.
Here is the sentiment analysis for it:
Sentence: I love the stories from this publisher.
Sentiment: Positive(3)
Sentence: They are always so enjoyable.
Sentiment: Positive(3)
Sentence: But this one disappointed me.
Sentiment: Negative(1)
As you can see, more comments here are positive, but the entire block has an overall negative sentiment due to the final, disapproving remark. As in the previous example, this suggests that in a text block like this one, later sentences should be weighted more heavily than earlier ones.
For the ratio, you might use the index value of each sentence in the text, taking advantage of the fact that a later sentence has a greater index value. In other words, the importance increases proportionally to the index value of a sentence.
A matter of scale
Another important thing to decide is the scale you’re going to use for sentiment evaluation of each sentence, as the best solution may vary depending on the type of text blocks you’re dealing with.
To evaluate tweets, for example, you might want to employ all five levels of sentiment available with Stanford CoreNLP: very negative, negative, neutral, positive, and very positive.
When it comes to product review analysis, you might choose only two levels of sentiment—positive and negative—rounding all other options to one of these two. Since both the negative and the positive classes in Stanford CoreNLP are indexed with an odd number (1 and 3, respectively), you can tune the sentiment evaluation method discussed earlier to round the weighted average being calculated to its nearest odd integer.
To try this, you can add to the nlpPipeline class as follows:
public static void getStorySentiment(String story)
{
int sentenceSentiment;
int reviewSentimentWeightedSum = 0;
Annotation annotation = pipeline.process(story);
List<CoreMap> sentences = annotation.get(CoreAnnotations.SentencesAnnotation.class);
int divisorWeighted = 0;
for (int i = 1; i <= sentences.size(); i++)
{
Tree tree = sentences.get(i-1).get(SentimentAnnotatedTree.class);
sentenceSentiment = RNNCoreAnnotations.getPredictedClass(tree);
reviewSentimentWeightedSum = reviewSentimentWeightedSum + sentenceSentiment*i;
divisorWeighted += i;
}
System.out.println("Weighted average sentiment:\t" + (double)(2*Math.floor((reviewSentimentWeightedSum/divisorWeighted)/2) + 1.0d));
}
Test the above method with the following code:
public class OverallStorySentiment
{
public static void main(String[] args)
{
String text = "The weather in the morning was terrible. We decided to go to the cinema. Had a great time.";
nlpPipeline.init();
nlpPipeline.getStorySentiment(text);
}
}
Recompile nlpPipeline and compile the newly created OverallStorySentiment class, and run OverallStorySentiment as follows:
$ javac nlpPipeline.java
$ javac OverallStorySentiment.java
$ java OverallStorySentiment
The result should look as follows:
Weighted average sentiment: 3.0
This test uses a single sample text to test the sentiment-determining method discussed here. For an example of how to perform such a test against a set of samples, refer back to the first article in this series.
Source: oracle.com
Posts
0 comments:
Post a Comment