Photo by Kelly Sikkema on Unsplash
Rolling into the end of October, it felt like predictions that were being forecast about the upcoming (now past-tense) 2020 Presidential election were almost fact.
And then Super Tuesday came. And Super Wednesday. And Super Thursday. . .
Although aggregators like FiveThirtyEight were (in the end) correct in their projections, there were also some big areas that they were blindsided—areas that made the 2020 Presidential election more of a nail-biter than anyone was expecting. This has us thinking about what lessons we can learn on behalf of our association clients, namely, that accurate data is crucial to creating predictions that work.
From where we sit, there are five main problems that analysts ran into that made the data going into the presidential election skewed.
- Small Sample Sizes
- Incomplete Data
- Failure to Segment
- Lack of Communicating About Uncertainty
- Underestimating the Impact of the Prediction on Data
These are the same key issues that we see with associations who are using predictive analytics to develop strategies for their organizations.
Predictive Analytics Fail #1: Small Sample Sizes
As Dan Valenzuela recently quotes in his article “What if Stories Are More Predictive Than Data,” a friend of his stated: “How can I trust a poll using only 700 people to predict how a whole [frickin’] state will vote?”
The fact is, while pollsters did their best to represent the demographics of the states in which they were working, there are a lot of limitations to gathering a complete data sample over the phone or in front of Walmarts. There are very human reasons for some people not to want to participate—not enough time, not enough resources, protection of personal space and rights—and the pandemic made this issue even worse.
In addition, we have to take into consideration the infrequency of presidential elections—just over a dozen in the recent polling era. That’s not many data sets to choose from. That kind of infrequency creates a lot of uncertainty, even if the data was more complete.
A more robust data set is the solution. Large and reliable sets of data that cover not 5% of the population (or in our case, your association memberships), but 100%—or at least 99.9%.
For associations, data like how many CEUs, how many purchases made, and whether or not they are accessing the association online portal are like mini-votes they are making (or not making) for your organization. You need to be collecting and analyzing all of them frequently and completely, or else accurate behavior predictions are virtually impossible.
Predictive Analytics Fail #2: Incomplete Data
As with a too-small sample size, incomplete data means that your predictions aren’t based in reality and are basically useless for building strategy upon.
A perfect example of this is the polling averages that were reported leading up to the presidential election. Aggregators like RealClearPolitics (showed Biden up by 8 points the week of the election) and FiveThirtyEight (Biden +7) had no concept that the final race would come down to just a few hundred thousand ballots across seven key states—a race that is still too close for some to call.
One of the big issues that was presented even before election day was the concept of “shadow supporters” of Donald Trump, people who were planning on voting for him, but not willing to tell pollsters (or anyone else, for that matter). Regardless of the social or psychological reason for this withholding of information, this was one of the key areas of failure in the accuracy of the predictive models. How can you possibly predict behavior if you’re not getting a straight answer about how people are feeling?
This is another area where associations sometimes are blindsided by their data. Sure, there are exit forms from CEU courses and annual questionnaires about areas like quality of education, cost of membership, value to the individual member, etc. But how many of the members are consistently providing answers? Is it just the few outspoken supporters (or non-supporters)?
Remember, on a typical bell curve, the people most likely to answer those questionnaires and fill out the suggestion box form are the top and bottom 15%—the people who either love you or hate you. That means that there is a full 70% of “shadow voters” who may never tell you how they feel. They’ll just leave.
By focusing on the right kind of data points, you can actually bypass this issue and look directly at actions. Which actions are people who are engaged with your association participating in? Which actions precede a member’s loss? That way, you’re getting the complete picture.
Predictive Analytics Fail #3: Failure to Segment
If you were going to host a pre-COVID party and were in charge of drinks, you would be an idiot to buy six 24-packs of the same thing. Why? Because not everyone you know likes the same thing as everyone else. Hopefully, you would have an idea of what percentage of your guests liked hard lemonade versus a local IPA and who would only drink soda or water.
Although analysts for the 2020 presidential election were able to segment generally, there were huge blind spots that they didn’t account for.
For example, Polly, the AI system in charge of the analytics for Advanced Symbolics’ predictions, was wildly inaccurate in predicting the closeness of the election results, giving Biden a 332-166 lead in the electoral college as well as Florida, Texas, and Ohio (all states that voted red).
One of the key problems the AI had was its inability to dissect subsections of voters who might be galvanized by different reasons. In the case of Polly’s Florida prediction, it lumped all voters from south of the border (including Cuba) as “Hispanic,” failing to recognize the huge Republican Cuban contingency.
Associations have a similar problem when it comes to reviewing data. You might see drop-off levels in the over 65 demographic when it comes to your website use (getting on the member area, signing up for courses, etc.) Although you might assume that those members aren’t very comfortable with the technology, you may be ignoring a vital subsection of 65+ members who are very web-savvy but just are choosing not to deal with your outdated or glitchy software.
Accurate segmentation is the key to developing association marketing strategies that connect with your members, and can only be done with the right kind of data points and analysis.
Just the Tip of the Predictive Iceberg
Of course, these three missteps are just a few of the errors that major polls dealt with in projecting the presidential election outcome. In our next post, we’ll talk about how communication, and feedback loops also played major roles in predicting (or not predicting) the future—and what associations can do differently in their own organizations.
To learn more about how Tasio helps associations with predictive analytics and data management for better member retention and overall strategy, download their free ebook.