Geek Out with Meltwater on NLP and Sentiment Analysis for PR and Marketing; Off Script No. 48: Dr. Tim Furche

An expert at Meltwater says natural language processing (NLP) is getting better at
understanding  context, which can help PR assess large volumes of content

Sometimes bad words can mean good things.

For example, the word “bomb” might have different connotations – associated with destruction or security – depending on your point of view.

However, if someone posts to Twitter, “Rocky Road ice cream is the bomb!” that’s probably a compliment.

It’s a compliment that’s likely to draw fire too because, as critics might respond, “Putting marshmallows in ice cream is disgusting.”

“Boom!” chimes in a third.

How would a natural language processing (NLP) algorithm tone or assess the sentiment of ice cream chatter on social media?

Sorting out what is positive, and negative is a big challenge that NLP faces. NLP is used in a variety or PR technologies for media monitoring. Meltwater has been working on adding context. Instead of toning a whole article based on a single word, it’s trying to assign tone to words and phrases to understand the whole article.

I put some questions to the company about NLP and Dr. Tim Furche stepped up. He is a senior director of Engineering and Head of Data Science for the company – and was the co-founder of Wrapidity, who Meltwater acquired in 2017. He teaches at the University of Oxford and he’s my guest for this 48^th edition of the Off Script Q&A series.

1. In just a few sentences, how would you describe natural language processing (NLP) to a child in Kindergarten, a high student, and a newly minted college graduate with a degree in computer science?

TF: [To a child]: Natural language processing is about teaching computers and robots to understand the language of the humans so that they can help us do things without having to type, touch a screen, or use a mouse. For example if I ask my phone to “call me a taxi”, I want it to phone a taxi company and send a car over, not start calling me Mr. Taxi.

[To a high-school student]: Imagine your last conversation with Siri or Alexa. That’s largely powered by advances and techniques from natural language processing.  It’s the same when Google tries to provide you with a direct answer to your search. Basically, any time you process any form of text or voice. It’s all about natural languages like English, Mandarin, or Swahili, in contrast to programming languages such as Javascript, Lua, or Python that you might have used to create a website or script a game.

[To a college graduate]: Natural language processing (NLP) is a field of study in computer science and linguistics that is often considered a branch of artificial intelligence. It studies the problems of interpreting and manipulating human (aka natural) language. The goal is to teach computers to ‘understand’ statements in human languages well enough for specific tasks such as a voice assistant, a search engine, or measuring the sentiment of a conversation. It draws from many disciplines, including computer science, computational linguistics, and statistics, in its pursuit to fill the gap between human communication and computer understanding.

2. How would you characterize the relationship between NLP and artificial intelligence (AI)?

TF: Formally, natural language processing (NLP) is a subfield of artificial intelligence (AI), just like image recognition, machine learning (ML) or deep learning. Conversationally often NLP, AI, and ML are used interchangeably. That’s largely because a lot of practical applications of AI are actually applications of NLP and a lot of the techniques used in NLP in particular in the last few years are using ML.

To further dive into the relationship, AI has been historically used for all kinds of applications, anything that can be seen as showing some form of ‘intelligence’ and is realized in computers. Intelligence itself is a rather fuzzy context and we are still discussing whether certain forms of problem solving in plants are a sign of intelligence.

As such there isn’t a single agreed definition of AI but being able to interact with human created artefacts such as text and images is certainly at the core of it. With NLP, machines can make sense of written or spoken text and perform tasks like translation, entity extraction, and linking (the Apple example), topic classification, sentiment analysis, and more.

Modern NLP tends to use a lot of machine learning around many of these tasks, where machine learning is the process of using algorithms to teach machines how to solve complex tasks given a set of examples and feedback without explicitly being programmed.

3. What is the value of NLP to PR (and marketing)?

TF:  PR (and marketing) is naturally concerned specifically with human language and, both textual and oral, conversations between humans. Natural language processing is a, if not the, crucial tool in helping PR and marketing professionals to track these conversations and to derive actionable insights.

It starts with the basics – identifying how we talk about what. For that NLP mines topics of conversation and identifies mentions about people, companies, products, locations, events, linking them to known entities such as Barack Obama or the Apple company. Having identified and disambiguated these entities, allows the PR or marketing professional to search for conversations they are involved or interested in and also to perform analytics, e.g., about the relative engagement generated by different entities in a conversation.

NLP can go further and help users analyze the sentiment, emotions, and other forms of tonality modifiers like sarcasm or irony related to the conversations they are interested in. It also allows PR and marketing teams to identify which aspects of an entity are heavily talked about and in which context. This allows analysis to the point of ‘Apple’s iPhone is compared a lot with Samsung’s S10’ and ‘Battery life is often mentioned as a positive aspect of the iPhone’.

4. NLP has traditionally had some challenges with accuracy. What are some of those challenges and what is the cause?

TF: Natural language is inherently imprecise, ambiguous, and contextual. It requires not only the ability to understand its linguistic features, but often also world-knowledge to make sense of it.

Recognizing that a sentence like “I waited 1h for my burger! Great job!” is likely sarcastic and requires understanding that waiting an hour for a burger is extraordinarily long. Just like for an alien from another world that doesn’t know anything about human restaurants. Similar challenges arise around humor and irony, but also in situations where cultural or conversational context is assumed.

There are many arguments about the formal complexity of NLP and we know that certain NLP tasks are indeed in a class of problems (NP-complete problems) for which precise solutions are infeasible for current computers at large problem sizes. NLP has also been postulated as an AI-complete problem, in as much as solving NLP perfectly would require a computer with human-level intelligence.

Many of the problems NLP addresses are problems where humans perform far from perfectly. Modern language models such as OpenAI’s GPT-3 and Google’s Switch Transformers with billions and trillions of parameters are now showing human (and sometimes super-human) levels of accuracy on complex NLP tasks but require millions of dollars to build (and operate) as well as large amounts of data that most companies don’t have or can’t afford.

Another known problem about these language models is bias. Since they require massive amounts of data, they don’t rely on ‘curated’ datasets. Instead, they instead learn from ‘open’ data, typically from the Internet… not the greatest teacher, unfortunately.

What is often at the heart of the perception of inaccuracy is a lack of precise problem definition. For instance, sentiment analysis at a document level is about determining if the document explicitly expresses a negative sentiment.

A sentence like ‘E-cigarette use has risen rapidly, especially among young adults’ does not express a negative sentiment in any way, in fact, ‘rising’ is often associated with positive feelings, e.g., when talking about stock markets or currencies.

However, from a public health perspective (and possibly for an e-cigarette brand trying to avoid regulatory actions) such a statement is clearly negative. What is needed in these situations is a better understanding of the use case and use case-specific NLP rather than the suggestion that the general-purpose NLP is not performing as expected.

5. Meltwater has recently made some enhancements to NLP – tell us about those.

TF: These are amazing days for NLP at Meltwater. We are constantly increasing the coverage of our NLP to new languages, such as Indonesian, Thai, and Russian in 2020 alone. We have also rolled out three improvements to our existing NLP services including, event detection and resolution, deep learning sentiment and fine granular sentiment around entities our customers care about.

To understand the scale of NLP at Meltwater, you need to know a bit more about the vast amount of data we are making available every day, from social media posts (Twitter, Facebook, Instagram, Reddit, YouTube), to editorial articles, blog and forum posts and reviews, to broadcast transcripts from TV, radio, and podcasts.

In total we process over 1 billion documents per day and make more than 10 billion calls to our NLP services, performing tasks ranging from the low-level tokenization, sentence splitting, morphology analysis, POS tagging, noun-phrase chunking, and dependency parsing, to the high-level named entity recognition, knowledge-graph linking, key-phrase extraction, document categorization, event detection and, finally, sentiment and emotion analysis.

We do all of that with a budget that is minuscule in comparison to Google, Facebook, or most other data providers with a similar scale.

Recently, we added the ability to detect real-world events such as an acquisition in each of the billion incoming documents. This is different from most of our NLP stack because events are aggregated across all documents first before they are exposed to customers. This allows us to extract and combine information from different sources to build a complete and confident picture of the real-world event.

As an example, one document might mention that one company acquired another on a certain date, another document might mention the price, and yet another some further details such as expected completion date of the acquisition. By combining multiple documents, we can also build a higher confidence that the event did take place indeed and wasn’t just speculation or fake news.

Customers can subscribe to 18 different types of these events through our smart alerts feature.

Our sentiment analysis has long been based on a Bayesian machine learning algorithm that performed decently and could be scaled fairly well.

Recent advances in deep learning and our amazing NLP team have made it possible for us to move to a much richer and more capable Glove+CNN deep learning model that performs 20-25% better on our benchmark datasets and has reduced the number of overrides (where our customers correct the sentiment manually) by over 50%.

If you want to learn more about that new sentiment model, please check out our engineering blog.  We are currently rolling out these new deep learning models to all of our languages and extending them to our entity extraction, with BERT-like models being shipped later in the year.

We are also bringing more fine-granular sentiment analysis to our customers that allows them to understand not just sentiment at the document level but also at the level of sentiment around entities such as companies or people that they care about. That way they can see if, e.g., in a conversation the majority of posts are negative in general, but actually positive about their own brand, as well as comparing sentiment around themselves and their competitors.

6. In 5-10 years, what are some things PR will be able to do with NLP that we cannot do today?

TF: We can see at least two major ways in which progress in NLP will change the way PR or marketing professionals are doing their jobs.

[The first area is] NLP and machine learning is getting better at quickly adapting to the specific context and needs of the user, brand, company or other entity on whose behalf the PR or marketing professional operates.

Better understanding the context of users and posts will also allow us to do a better job at understanding and classifying types of actors and posts. This will allow PR and marketing professionals to dynamically screen out posts from bots, fake accounts, and other groups of users with suspicious or simply common behavior patterns. It will also allow for the screening of specific posts with adverse or suspicious content (such as fake news, slander, trolling posts etc.) taking into consideration the context of the entity.

The second area where we see NLP to change the way our industry works is in natural language generation (NLG). Already NLG is used by some news agencies and marketers for generating formulaic documents (like analyst call summaries) or slightly personalized messaging.

In the future, we will see far more sophisticated targeting of PR and marketing messages to the individual audience through the use of NLG which allows us to match our messaging to the interests and needs to the individual that is indistinguishable from having a human craft a personal message to each of the audience members.

* * *

Thank you for taking the time to participate, Dr. Tim. I look forward to watching the progress Meltwater makes in NLP.

If you enjoyed this post, you might also like:
PR Agency Owner Started Making Homegrown PR Software; Today it’s a VC-backed Startup; Off Script No. 47: Zach Cutler Co-founder and CEO at Propel

Image credits: Meltwater and Unsplash