Start the Churn Predictions, Hal

Written by Kerry Cosby on February 1, 2018

When we think of artificial intelligence, our first thoughts tend toward Hal in Stanley Kubrick’s classic 2001: A Space Odyssey.

But we should probably think more along the lines of predictive algorithms that help us understand who will leave or stay with our association.

Over the years, association management professionals have used a number of methods to understand why members stay or leave, but getting to the point where, out of 30,000 individuals, we can pinpoint who will leave, and when, has been a challenge. The problem is that member satisfaction surveys and exit surveys are geared toward general understandings, not predicting what specifically John or Jane Doe will do at renewal time.

Their decisions to stay or leave are simply too complex for such blunt instruments. Every year, Jane (and her 29,999 colleagues) is weighing a number of factors to determine whether or not she hits that renew button on the association’s webpage. And you might or might not be in control of the factors most significant for her choice.

Jane could be facing layoffs this year, due to a downturn in the industry or the broader economy. Or she has entered the gig economy, and as a contractor, no longer has a company paying her membership dues. Or maybe Jane is a global member and the conversion rate of her country’s currency has fallen against yours, and the membership price is just too far out of her reach this year.

We puny humans simply cannot go through all of the membership (and macroeconomic) data to sort out those people who will stay and who will go. But a machine-learning algorithm can.

Machine Learning—the Backbone of Your AI

Machine Learning is a process by which a computer system is able to learn from data provided, examples or experience, rather than needing to be given the logic by a human programmer.

With algorithms like Support Vector Machines (SVM), neural networks or Bayesian classifiers, the program can learn much more deeply about your members than you can through surveys alone. Furthermore, it can take into account many more variables in classifying your members into those who will renew and those who won’t.

I don’t mean to say that surveys are useless. In fact, later in this article I will argue a way to enhance artificial intelligence by employing the surveys conducted over the years.

Figure 1: Support Vector Machine sowing churners

Experimenting with Offerings

Recently, my organization invested the time to create a Support Vector Machine to predict membership churn. A Support Vector Machine uses lines or planes in multi-dimensional space to separate the groups who will renew from those who won’t.

Such an AI program might predict that out of 30,000 members, there would be roughly 5,400 churners. Unless the program is perfect—which it will not be—despite the program’s predictions a probable number of expected churners will renew. If this is the case, is the AI any better than guessing?

To understand the answer to this question, the organization needs to monitor its AI’s accuracy while planning activities to support at-risk members. The best way to do this is presented in Figure 2, which shows the difference in renewal rates over time for the members the AI predicts to churn and those it predicts to renew.

If the program had chosen the members of these two groups at random, there would be no or very little difference between the respective renewal rates. In other words, the two lines in Figure 2 would roughly lie on top of each other.

The situation can be described in terms of body temperature. People’s body temperatures change all the time, but on average they are 98.6 degrees Fahrenheit. Measuring the body temperatures of a random group of people (if the sample were large enough) would reveal a range of temperatures throughout the day with an average of 98.6 degrees. However, if the sample were selected based on whether the people had the flu, we might expect the temperatures to track higher than the healthy group with an average above 100.4 degrees.

In the same way, if the churn predictions are good, we would expect that at any given moment during the renewal campaign, predicted churners would renew at a significantly different rate than predicted “renewers.”

Figure 2 demonstrates not only a significant difference, but also a difference that grows over time. From this, we can tell whether the algorithm is good. But it tells us nothing about any (pro)active attempts to keep these members.

Figure 2: Comparison of renewal rates

The Blue Pill or the Red Pill

Once an association knows it has a good algorithm, it can start experimenting with ways to convince at-risk members to stay—even before they have decided to leave.

And this is where surveys come in handy. They provide the potential set of offerings for at-risk members. If the surveys reveal that younger members who leave the association will miss publications and are interested in conferences and workshops, the association can randomize offerings within the group to see which entices more members to renew.

The association might also look into dividing churners into an experimental group and control group (a division of 80% & 20% respectively) to enable membership managers to assess their offerings.

Figure 3 shows a difference in renewal rates between the experimental group and the control group at the moment the organization starts its offerings to the experimental group. In this way, the association sees which options are best while assessing the effort overall.

Figure 3: Comparison of Renewal Rates for Members predicted to churn

From Predictive to Prescriptive Analytics

After a year or two of running such experiments, an association may be able to move beyond simply prediction and let the algorithm start digging through the data to make its own prescriptions on what should be offered to various at-risk members—creating an offering tailored to their expected preferences.

Just as artificial intelligence is able to dig more deeply into the influences that lead members to leave, it can also understand more deeply what has a better chance of convincing at-risk members to stay.

As such, working with AI is not just creating a predictive algorithm. Rather, it is the further use of data analytics to create a more detailed and precise understanding of your members, and with this to determine the best methods for retaining them.

All graphs are simulated data to illustrate the processes of prediction and analysis.

Kerry spoke in the “Exploring New Markets: How Research, Analytics and Risk Assessment Can Help” session during SURGE Spring, an interactive virtual summit hosted by on May 2nd-4th. Click here to watch the sessions on demand.