I had the absolute pleasure of attending Antidote last Sunday at the Sydney Opera House, alongside our favourite data specialist, Elisa Choy (the DataWhisperer). We attended an expert panel designed to unpack the problems with polling (aptly named ‘The Problem with Polling’), which focussed on how, in particular, it all went so wrong in predicting the recent federal election. What a poll bomb it was!
On the panel hosted by David Speers, was everyone’s favourite election analyst Antony Green, along with the data scientist who predicted the federal election result by analysing 2 million tweets, Bela Stantic, and social researcher, Rebecca Huntley.
In discussing just how it went wrong, it was surprising to hear just how small, self-selecting and limited in diversity the sample of Australian voters were, who were used as the poll sample.
Approximately 2,000-3,000 typically inner city folk were surveyed, forming the basis of the polls (and no doubt numerous key decisions). It’s frightening stuff. Would you trust major decisions on the opinions of 0.0002% of the population?
It’s no wonder, then, that Bela Stantic’s analysis of social media listening was so much more accurate in its predictions.
Stantic explained that by analysing approximately 2 million tweets from around 500,000 users, he was able to accurately predict that the Coalition would be returned to government.
Unlike the polling—which had a sample size of max 3,000, required participants to ‘opt-in’ to being surveyed, and typically needed landline access—there were no such hurdles to analysing their tweets. Who has a landline these days?
As Stantic explained:‘Don’t ask for an opinion, but rather monitor people’s behaviour. People are more likely to respond truthfully, indirectly.’
While both Elisa and I agree with Bela Stantic, we also feel strongly that it’s just as important to monitor using a diverse data-pool. And, while what people say or like on Twitter provides some insight into what they feel and how they behave, it far from reveals the true story. I have a Twitter account (which to date has only been used to share my displeasure at Emirates appalling ‘business class’ experience, which included a 2 3 2 seating on the long haul to Europe—still shocking, but totally irrelevant here), but Elisa doesn’t and we both have strong opinions about almost everything, particularly politics.
The Problem with Polling is widespread
Recently, Elisa and I have been working on a new and improved methodology intended to use the insights gained by Artificial Intelligence to better understand and predict behaviour.
We’ve been looking at television programming as a case study (see our recent piece on predicting who audiences want as the next MasterChef Australia judges), where there already exists a key set of data (TV ratings) that are relied upon to make programming decisions, often in combination with social listening.
The problem for us is the same as what is plaguing the polls. This self-volunteered sample from which the ratings are devised can by no means be an accurate and diverse view of how people are engaging with TV shows.
And, social listening isn’t the answer. Of course, people are likely to amplify their engagement on social media, but how truthful is what they say, and how valuable is that data?
Facebook likes are especially problematic. We were tracking a popular TV show over a period of time only to see a huge increase in likes, while the ratings fell through the floor. Do you remember the likes you clicked a year ago?
Why? Because ‘likes’ are mindless and are not predictive of actual meaningful engagement or behaviour. Like doesn’t mean love, nor does it reflect any depth of emotion, but is often an arbitrary action.
So, what’s the answer… to elections, TV hits and more? We believe it’s language-based data, analysed in real time by artificial intelligence. Only when you can understand how people feel about a subject, and how deeply those feelings go, can you begin to predict how this may impact their behaviour.
Thanks to AI, this deeper level of analysis can be easily measured, and when these insights are used to inform key decision-making, such as policies and TV programs, a higher rate of stickiness and engagement can be expected.
Stay tuned…we’re working on something else to demonstrate the power of AI for Market Intelligence.
#artificialintelligence #polling #surveys #data #creativeintelligence