Like AI Cares: Helping Computers Detect Sarcasm

This article is part of the Academic Alibaba series and is taken from the WWW 2019 paper entitled “Sarcasm Detection with Self-matching Networks and Low-rank Bilinear Pooling” by Tao Xiong, Hongbo Zhu, Peiran Zhang, and Yihui Yang. The full paper can be read here.

How can you tell when someone is being sarcastic online? When you can’t rely on facial expressions, tone of voice, and other contextual information, identifying sarcasm can be difficult. If even humans struggle with this, imagine how challenging it is for computers. Accurate computer analysis of human language is impossible without knowing which expressions to take at face value. And considering how common sarcasm has become in online communication, computational linguists have their work cut out for them.

Conflicting Sentiments

In any effort to detect sarcasm, a key ingredient to check for is “incongruity” — that is, is there a disparity between the literal meaning of the sentence and the author’s underlying intention? One way for computers to detect incongruity is to compare words within a sentence, looking for positive expressions accompanying a negative scenario or vice versa. For example:

I had the pleasure of being awakened by my neighbor’s screaming cockatoo.

“Pleasure” is a positive expression, but being awakened by a loud noise is something people generally don’t enjoy.

However, simply identifying conflicting sentiments together in a sentence isn’t enough to prove that the sentence is sarcastic. The composition of the sentence is also a key factor. For example, this sentence also contains a positive description and a negative concept, but obviously isn’t sarcastic:

I like all birds except my neighbor’s screaming cockatoo.

Putting It All Together

The Alibaba tech team, in collaboration with Ant Financial, has developed a new model that combines both of the aforementioned factors by generating two separate sentence representations: one capturing incongruity information and one capturing information about sentence composition.

The incongruity information comes from an innovative “self-matching network.” This network analyzes every word-to-word pair in the input sentence, searching for potential conflicting sentiments.

The compositional information comes from a bidirectional long short-term memory (LSTM) network, which is especially suited for analyzing sentence composition.

Because the self-matching network also ends up containing some compositional information, directly combining the two networks is likely to result in redundancies. For this reason, the two networks are concatenated using a pooling method that controls for potential redundant information without compromising the discriminative power of the self-matching network.

Sarcastic sentences reported by the self-matching network

The model’s performance is impressive. Tests run on several publicly available datasets from sources such as Twitter and Reddit have demonstrated that this model outperforms existing baselines in precision, recall, F1 score (i.e., false positives and false negatives), and accuracy — in some cases, by as much as 10%.

All Done? Yeah, Right.

The self-matching-based sarcasm detection model, while powerful, has some limitations. In particular, the model is less capable of identifying sarcasm in sentences that rely heavily on external information. In addition, rhetorical questions can result in false positives, as they are difficult to distinguish from sarcastic expressions.

Two false negatives and a false positive reported by the model

The Long Road Ahead

Computational linguistics is just getting started on sarcasm detection, and there is still a lot of work to be done. But Alibaba and Ant Financial’s innovative approach is steering the field in the right direction. New models like the one they have proposed will improve the way social media and e-commerce platforms draw conclusions from user language, thus better informing sentiment analysis, opinion mining, and advertising decision-making.

Great job, geniuses — and we mean that in the sincerest way possible.

The full paper can be read here.

Alibaba Tech

First hand and in-depth information about Alibaba’s latest technology → Facebook: “Alibaba Tech”. Twitter: “AlibabaTech”.