2.1 Classical Resources on Sentiment

One of the first and most widely used resources is the Subjectivity Lexicon [6],³ which is a list of subjectivity clues compiled from several sources, annotated both manually and automatically. This lexicon is the core of OpinionFinder, one of the first systems for the automatic detection of polarity. Another widely used resource is the Opinion Lexicon⁴ compiled by Bing Liu. The list contains approximately 6800 English words that are classified as either positive or negative. Both resources were compiled manually and are thus quite accurate. However, they make two simplifications: first, they encode sentiment information in terms of a sharp division between a positive and negative sentiment value rather than providing a scale of positivity/negativity; second, they associate sentiment with words rather than with their senses. We will discuss the latter issue first, and the former later.

The lack of sense distinctions is quite a limitation from a semantic perspective. Indeed, lexical entries are often polysemous, so the same string might actually have a completely opposite sentiment depending on the context in which it is used. For example, “crazy” can be used in a negative as well as a positive way, which is strictly context dependent. Creating a flat list where words are assigned a binary polarity value will not account for the complexity of such semantic aspects.

One step further in creating sentiment resources is thus assigning polarity values at the sense level rather than the word level. This is the principle behind the annotation scheme developed in [7], which gave rise to the Subjectivity Sense Annotations,⁵ a sense-aware lexicon. This resource actually addresses sentiment at one level up with respect to polarity, as it classifies a given word sense as objective or subjective, without specifying, in the latter case, its polarity value. As a plus, though, this lexicon includes part-of-speech information. Apart from sense distinctions of exactly the same lexical entries, there can also be ambiguities related to parts of speech, as a given word could be, for example, a noun and also an adjective, and could exhibit different polarity features accordingly. A good example is “novel,” which has a neutral polarity when used as a noun but a rather positive one when used as an adjective.

The creation of SentiWordNet [8] addresses ambiguities both within and across parts of speech. SentiWordNet is an extension of WordNet [9], where three sentiment scores are assigned to each synset, thus to word senses: positivity, negativity, and objectivity. In addition to the assigning of sentiment to senses, one nice feature of SentiWordNet is that sentiment is a gradual concept rather than a categorical one, thus addressing the first limitation of the simplest resources highlighted above. The same synset can exhibit the three values (positivity, negativity, neutrality) with different degrees, summing up to 1. The main drawback of SentiWordNet is that its sentiment scores are for the most part automatically assigned, so the presence of noise is not trivial. It is also known to have a positive bias [10], and so is not so good for detecting negative opinions.

The very first sentiment-aware resource we know of—namely, General Inquirer⁶ [11], whose compilation was started in 1966—accounted for sense distinctions, although polarity was conceived as a binary value rather than a gradual concept.

Another way to indirectly address word sense disambiguation is to have domain-dependent lexica. Often a given word will exhibit only one sense in a specific domain (the classic example of “bank” is indicative here, as in a financial context it is unlikely this term would be used in its “river bank” sense). There are two large lexica that are built around very specific domains, exploiting contributors’ reviews: the Yelp Lexicon, with a focus on restaurant reviews, derived from the very large Yelp challenge dataset,⁷ and the Amazon Lexicon, built on laptop reviews posted on Amazon from June 1995 to March 2013 [12, 13]. Additionally, there are sentiment lists tailored for the financial domain, developed chiefly by Loughran and McDonald [14] after they noted that three quarters of the words with a negative polarity contained in standard, domain-independent resources did not actually have a negative connotation in the financial domain.

Among the resources that conceive polarity as a gradual rather than categorical value, we mention AFINN, which is a manually compiled list of 2477 English words and phrases rated with an integer between − 5 (very negative) and + 5 (very positive),⁸ and the Sentiment140 lexicon, which contains about 60,000 unigrams and 10 times more bigrams. Differently from the AFINN lexicon, Sentiment140 is compiled automatically by exploiting tweets with emoticons. This resource is part of a suite of lexica all automatically compiled by Said Mohammad and coworkers,⁹ under the NRC general label [15, 16], which also includes the NRC Hashtag Sentiment Lexicon, built by exploiting the sentiment of hashtags, and two more “double” resources built on the previous ones; namely, the NRC Hashtag Affirmative Context Sentiment Lexicon (and NRC Hashtag Negated Context Sentiment Lexicon), and the Sentiment140 Affirmative Context Lexicon (and Sentiment140 Negated Context Lexicon), where the average sentiment of a term occurring in an affirmative context is separate from the average sentiment of the same term occurring in a negated context.

Within the NRC tools there is also a small manually (crowdsourced) produced lexicon of about 1500 entries, the MaxDiff Twitter Sentiment Lexicon [17]. Handmade resources are always smaller than automatically built ones, as they are costly to create, but they are obviously more reliable. The ways of reducing noise in the automatic acquisition of resources have been explored by, for instance, the use of sentiment proxies such as emoticons [18] or, usually, the adoption of a semiautomatic approach, bootstrapping manually annotated sets of seeds [19, 20]. This is also the approach behind the original creation of SentiWordNet. Finally, using seeds, Chen and Skien [21] automatically produced sentiment lexica for more than 130 languages, reporting an overlap with existing resources of more than 90%.

For the same economical reasons we have just mentioned, and because tools for basic text processing are more advanced for English than for other languages, the creation of lexica for other languages can benefit from the automatic porting of data and techniques from already existing English resources [22]. An example is the transfer of SentiWordNet to other languages. Because English Wordnet is easily aligned to WordNets in other languages, porting SentiWordNet becomes rather straightforward, and it has been shown that sense transfer is robust across languages in about 90% of cases [23] (see also [24] for a WordNet-based, almost unsupervised approach for generating polarity lexica in multiple languages). Transfer of resources can be done in two ways: to create a stable, reusable resource, or “on the fly.” An example of the former is Sentix for Italian (via SentiWordNet), which is currently the most commonly used resource for this language in sentiment analysis systems [25, 26]. As for the latter, resources are translated and used in the context of a sentiment analysis system, but are not necessarily stabilized and distributed for further use; for example, Hernández Farías et al. [27] use a large number of English lexica/resources automatically translated into Italian to exploit information as features in their supervised system, but no specific evaluation or refining of such resources is performed.

2.2 Beyond the Polarity Valence: Emotion Lexica, Ontologies, and Psycholinguistic Resources

Recently, a variety of affective lexica have been proposed to offer information about affect expressed in text according to finer levels of granularity (eg, referring not simply to positive or negative sentiment polarity but to emotional categories such as joy, sadness, and fear). Moreover, a variety of psycholinguistic resources are available that can give some additional measure about the emotional disclosure in social media texts, according to different theoretical perspectives on emotions. All such affect-related resources could be useful with the purpose to increase the coverage of different aspects of affect in textual content. We organize the description of such resources into three groups: the first group is related to information about emotions by referring to a finer-grained categorization model; the second group includes psycholinguistic resources and other resources that refer to different perspectives on affect, according to dimensional approaches to emotion modeling; the third group includes knowledge-based resources and ontologies, which have been developed with the twofold aim to help sentiment analysis systems to grasp the conceptual and affective information associated with natural language opinions, and to use Semantic Web and linked data technologies to provide structured, reusable, and meaningful sentiment analysis results.

2.3 Social Media Corpora Annotated for Sentiment and Fine Emotion Categories

The growing interest in the development of automatic systems for sentiment analysis has also prompted the production of annotated corpora that could be exploited for the development of such systems in a learning fashion (see also Section 3).

The design of schemes for the annotation of corpora is always a task in the field of data classification, which leads to theoretical assumptions about the concepts to be annotated. It defines what kind of information must be annotated, the inventory of markers to be used, and the annotation’s granularity. As highlighted in [48], in annotated corpora for sentiment analysis this is especially challenging. Research in psychology outlines three main approaches to the modeling of emotions and sentiments: the categorical, the dimensional, and the appraisal-based approach. The most widespread are the categorical and the dimensional ones, which describe emotions by marking a small set of discrete categories and scoring properties in a continuous range of values. Accordingly, the kinds of knowledge usually annotated are the sentiment’s polarity (positive vs. negative), category (happiness vs. sadness), the source and target of the sentiment, and its intensity. Annotations can be based on simple broad polarity labels, possibly equipped with intensity ratings, which also helps us to classify texts where mixed sentiments are expressed. They can also be based on labels representing different emotions.

The resources developed within the SemEval evaluation campaigns (regularly since 2013) comprise tweet collections annotated for subjectivity, polarity, emotions, irony, and aspects/properties, and are not restricted to English [25, 26, 48, 49]. In addition to the SemEval related data, other collections have been constructed, especially on news and events or product reviews, which are obviously a very good source of opinionated texts, and a field of interest for any business-related application that works with users’ opinions. For example, a widely used dataset is the Movie Review Data,¹⁹ a collection of movie reviews [50]. Sentiment labels (positive vs. negative) are assigned to the global review, while in an extension of this corpus, annotation is done at the sentence level [51]. Another classic resource is the MPQA corpus,²⁰ based on [52], which contains news articles from a wide variety of sources and which was manually annotated for opinions and other private states. This corpus has been expanded very recently with finer-grained information, and now includes entity and event target annotations [53], in line with the increasing interest in aspect-based sentiment analysis. A pioneer work in this sense is [54]: given a product, Hu and Liu [54] mine and summarize reviews related to specific features of that product, on which customers have expressed their positive or negative opinion, so that sentiment is associated with aspects of rather than the whole product. Extraction of opinions on aspects (specifically service, location, and rooms of hotels) is also the focus in [55], where the authors collected and processed 10,000 TripAdvisor reviews.

Interest in properties and aspect-based sentiment analysis is increasing also in Twitter-based work, as shown by recently organized evaluation campaigns on this [3]. Indeed, even if a lot shorter in characters than a product review, the same tweet can contain both positive and negative information, possibly relating to different entities that are mentioned. Niek Sanders’s Twitter dataset²¹ is not aspect annotated but a specific topic is assigned to each tweet, and this topic can be possibly used as a proxy for the entity with which the sentiment is associated, as is also done in [25], where specific hashtags are used as proxies.

Still in term of proxies, it is interesting to note that to save annotation effort in the creation of sentiment datasets, there have been experiments with distant learning, where class labels are not assigned manually but are rather derived from other information available. Performing sentiment analysis on tweets, Go et al. [56] trained a few classifiers using emoticons as noisy labels, and achieved an accuracy of about 80%. Recently, a new Twitter corpus was released by Mohammad et al. [57], where a multilayer scheme is applied. It contains a set of tweets with annotations concerning different aspects: sentiment (positive or negative), but also finer-grained annotation of emotions (anger, anticipation, disgust, fear, joy, sadness, surprise, and trust), and in addition purpose (to point out a mistake, to support, to ridicule, etc.) and style (simple statement, sarcasm, hyperbole, understatement). The corpus is not generic but focused on the political domain. Mohammad et al. collected tweets labeled with a set of hashtags pertaining to the 2012 US presidential election. The tweets were annotated manually by reliance on crowdsourcing platforms. This is the first dataset including a fine-grained annotation concerning stylistic features related to irony, and also a multilayer annotation concerning affect, since both annotations on sentiment polarity and on emotions are provided. For style, only 23% of the tweets were labeled with a style tag pertinent to the expression of irony, whereas most of them were annotated with the label simple statement, which can be interpreted as a tag for marking nonironic expressions.

Previous corpora annotated for fine-grained emotion categories, such as the one proposed for SemEval 2007 task14 [31], were not focused on social media data. The availability of such a new kind of datasets opens the way to the possibility of building machine learning systems that predict emotion and purpose labels in unseen tweets, as proposed by Mohammad et al. [57], who also presented the first results and a baseline for such new sentiment-related tasks on Twitter.

3.1 Lexical Information

Independently of the approach (rule based or statistical), virtually all systems for sentiment analysis rely on information derived from lexical resources. In machine learning such information is used as features, normally in combination with other features, too. The best performing system for subtask B of SemEval 2015’s task 10, which is the most standard message-level polarity task, is an ensemble classifier that builds on three well-performing classifiers at SemEval 2013 and one at SemEval 2014 [58]. Although the classifiers were handpicked with attention not only to performance but also to significant differences in feature combinations, all four classifiers employ polarity dictionaries of some sort and in some way. Some systems rely on one resource only; for example, Günther and Furrer [59] only use SentiWordNet, while other systems will try to use and combine information from all available sources [15, 60].

As a general strategy, lexical information is collected at the word level (eg, the categorical or gradual polarity of a given token) for each word of the whole tweet, and is then propagated via different combinations at the sentence/message level. Such combinations yield values that are then used as different features. For example, feature values can be the number of tokens in the text with a positive score or the number of those with a negative score, but also derived values such as the global polarity score of a tweet obtained as the average of the tokens’ polarities. Also, maximum and minimum scores, when available, are used as features.

Most statistical approaches are support vector machine models that implement such features and similar features, and Taboada et al. [10] highlight that information from basic unigrams appears to be the most useful. Although this shows that words in themselves are very informative for this task, the intrinsic limitations in the use of information from a plain dictionary lookup are quite evident if one thinks that words are used in a specific (syntactic) context, and their polarity can change substantially according to how they relate to other words in the text. Mainly because of the lack of tools that can deal with Twitter, few systems perform word sense disambiguation, so the richer sense-annotated lexica cannot be exploited to their full potential. A way of addressing polysemy without disambiguation was suggested by Basile and Nissim [25], who introduced the concept of polypathy (calculated as the standard deviation of the polarity scores of the possible senses of a lemma) as an indicator of variance of polarity scores across a lemma’s synsets, which can then be used as a feature or threshold value (tokens that have a too high polypathy could, for example, be ignored, as they provide conflicting information).

Another crucial limitation of not taking context into account is negation [61] (recall also the separate lexica for negated and affirmative contexts reviewed earlier). To cope with this aspect, Taboada et al. [10] incorporate information from what they call contextual valence shifters, showing an increase in performance. This is a first step toward deeper language processing, which is considered more and more necessary, even for short texts such as tweets.

3.2 Distributional Semantics

In the context of sentiment analysis the idea of exploiting the distributional hypothesis—namely, the assumption that words that occur in the same contexts tend to have similar meanings [62, 63]—simply boils down to the fact that similarity models that predict, for example, that “amazing” and “wonderful” are similar could be extended to predict that if “amazing” has a positive value, so too will “wonderful,” and will be at the opposite end of spectrum to, say, “terrible” and “awful.” However, general similarity models usually take into account the lexical and morphosyntactic context of a word but not necessarily the text’s polarity. Therefore similarity is potentially accurate at a syntactic and more general semantic level but not necessarily in a sentiment-aware way, so “good” and “bad” end up being very similar. This is true for classic distributional similarity models as well as for the more recent and successful distributed vector representations known as word embeddings, at least in their standard formulation [64, 65].

As a first attempt at directly incorporating sentiment in learning the distributional context of a word, so that words expressing similar sentiment do indeed end up having similar vector representations, Maas et al. [66] developed a model where they give a sentiment predictor function to each word. On the positive/negative classification of tweets, their system is shown to perform better than models that incorporate embeddings trained in a nonsentiment-aware fashion. An even more powerful and recent model, which outperforms that of Maas et al. [66], was proposed by Tang et al. [67], who train a neural network by associating each n-gram with the polarity of a sentence (thus beyond the word level, as in [66]), and show that sentiment-specific word embeddings effectively distinguish words with opposite sentiment polarity. This model performs better than other models that use generally learned embeddings.

3.3 Entities, Properties, and Relations

Interest in finer-grained sentiment analysis has also necessarily prompted the need for finer-grained semantic analysis, and therefore deeper language processing. For example, aspect-based sentiment analysis must rely on the identification of specific entities and/or properties of entities in reviews or tweets. To do this, standard techniques for entity detection and classification are employed, such as sequential taggers, possibly retrained for specific domains. Particular attention to (named) entities in sentiment analysis is also shown by the OpeNER EU-funded project,²² which focuses on named entity recognition within sentiment analysis.

Further, relations between the entities and events involved must be identified, so as to know what is said of which entity. An obvious way to do this is to exploit dependency relations, although deeper processing of tweets is not so simple because of the idiosyncratic and often ungrammatical language such short texts contain (although recent work based on learning neural knowledge graph embeddings shows an error reduction of more than 26% in semantic parsing of tweets [68]). Similar issues arise when one is developing systems to detect stance, as in order to assess the opinion of someone toward a given target, all relations between the entities involved must be correctly identified and associated with the sentiment expressed. However, a deeper linguistic analysis of text is also beneficial, if not necessary, for standard message- or text-level sentiment analysis, as it helps to treat the issue of contextual valence shifters mentioned in Section 3.1 by also accounting for word order and sentence structure. To this end, the Natural Language Processing Group at Stanford University developed a sentiment treebank.²³ This treebank has been used to train a recursive neural network built on top of grammatical structures [69], achieving an increase of 5 percentage points on sentence polarity classification. On fine-grained sentiment level they obtained a 9.7% improvement over a bag-of-words baseline, and overall showed the ability to accurately capture the effects of negation and its scope at various levels in the tree structures.

3.4 Concept-Level Sentiment Analysis: Reasoning With Semantics

Approaches enabling concept- and context-based analysis can lead to a better understanding of opinions expressed in textual data, therefore reducing the gap between unstructured information and structured machine-processable data. Concept-level sentiment analysis exploits large semantic knowledge bases (eg, ontologies and semantic networks), together with natural language processing tools and techniques, thus stepping away from blind use of keywords and word co-occurrence counts. Rather, it relies on the implicit features associated with natural language concepts. Unlike purely syntactic techniques, concept-based approaches are able to detect also sentiments that are expressed in a subtle manner (eg, through the analysis of concepts that do not explicitly convey any emotion) but that are implicitly linked to other concepts that do so.

The accent on concept-level sentiment analysis is recent. We can observe different knowledge representations of the concepts related to words and of their affective load, as well as different ways in which systems that embrace this perspective on sentiment analysis exploit and reason about such knowledge, depending also on the specific task they address.

Also, this perspective on sentiment analysis offers a challenge to the Semantic Web community. A periodic shared task on concept-level sentiment analysis has been launched recently [5], and some of the systems described below explicitly rely on the use of Semantic Web and linked data resources and tools to enable automated reasoning (eg, ontological reasoning on the taxonomic structure of an ontology of affect) and semantic metadata processing.

Let us start with Sentilo,²⁴ an unsupervised, domain-independent system that relies on a knowledge-based approach where existing tools for natural language processing and publicly available lexical/affective resources are combined with Semantic Web techniques for representing and reasoning about knowledge involved in opinion sentences. Sentilo exploits the semantic graph representation of a sentence enriched with opinion-related information (eg, opinion holder, topics, sentiment scores). Such a semantic representation allows Sentilo to address the finer-grained tasks of detecting holders and topics of an opinion, and additionally to identify and distinguish both main topics and subtopics. Specifically, given an input opinion sentence (natural language text), the system returns the corresponding semantic representation as a FRED graph (RDF/OWL graph) annotated with concepts from a semantic model of opinions. Annotations concerning the opinion holder, opinion features, and opinion topics are produced on the basis of a set of heuristic rules exploiting the FRED semantic representation and by use of lexical and affective resources, such as SentiWordNet and SenticNet. An evaluation of Sentilo as the opinion holder and main/subtopic detection tool has been presented [70]. The system is able to achieve good performance compared with the other tools, highlighting promising results on opinion holder (F1: 95%), subtopic (F1: 78%), and topic (F1: 68%) detection. Sentilo is also available as a REST service that returns RDF as output. This is important to make the output of the system machine processable and reusable.

In other systems the accent is on the use of conceptual and affective knowledge of words, the focus being on sophisticated representations of the affective space that can enable commonsense or ontological reasoning on networks or taxonomies of concepts. This is, for instance, the case for many systems exploiting the resources of the SenticNet suite of resources [71]. Another system that can be placed in this category is ArsEmotica,²⁵ which is an example of a domain-dependent system focused on cultural heritage. It applies sentiment analysis to resources from online art collections, and exploits tags given by the visitors on social platforms as an additional source of information [46]. The extraction of the emotional semantics from social tags is driven by an ontology-based representation of the emotional categories presented in the previous section. Methods and tools from the Semantic Web and natural language processing provide the building blocks for the creation of a semantic social space to organize artworks according to the ontology of emotions. The output is encoded into World Wide Web Consortium ontology languages. The semantic representation is not limited to affect but concerns also aspects of the specific art domain. This gives the twofold advantage of enabling tractable reasoning on the relationships between emotions and other interesting dimensions of the domain (eg, artists and artworks), and fostering the interoperability and integration of tools developed in the Semantic Web and linked data community. The system has been evaluated against a dataset of tagged multimedia artworks from the ArsMeteo Italian art portal.²⁶ A SPARQL end point has been implemented to explore the collection and to extract information about the relationships among emotions, artworks and authors by posing queries such as: “Give me the artworks stirring emotions similar to sadness and belonging to the music genre.” In some systems the focus on conceptual semantics, not limited to affect, is also combined with issues related to the contextual semantics of words. Such approaches do not offer fixed sentiment polarities but assign context-specific sentiment orientation to words. An interesting proposal in this line is SentiCircles, a lexicon-based approach for the detection of sentiment in Twitter posts at both entity level and tweet level [72]. SentiCircle builds a dynamic representation of context to tune the preassigned strength and polarity of words in the lexicon, and incorporates both contextual semantics (ie, semantics inferred from the co-occurrence of words) and conceptual semantics (ie, semantics extracted from background ontologies).