From Smart Signals to Conversion, Churn & Health Scores

Topics

Overview

The goal of a PLG platform is to detect accounts that are most likely to buy, to expand, or to churn; and to act on them (and their users) in a way that best guarantees a positive outcome.

The principles are simple: Analyse past conversions, expansions and churns, and see which elements (properties, events, screens, web pages...) combined (!) are unique to these stage movements during a running window of 30 or 90 days, whichever best fit your business. Then, use these combined elements, taking their each impact into consideration, to analyse each account/user in each stage, and give 2 likelihood-to scores:

  • A conversion score: Likelihood that an account/user will move into a positive direction, being either from free→paying, from trial→paying, or from paying→more paying (=expansion).

  • A churn score: Likelihood that an account/user will move into a negative direction, being either from paying→churn, from trial→trial expired, or from free→churn.

  • A health score: General indication that takes both positive and negative movement likelihood into consideration.

The elements combined are called Signals and they are the source of above scores being computed.

Specific to the SaaS market we are serving, is that the whole signal→scoring process cannot be a black box. Because on the one hand, Growth Managers have clearly indicated that they wanted to be able to create and adapt own signals, based on business intuitions. On the other hand, they also indicated that getting some help to create these signals — read: having journy.io do the job through machine learning — was paramount, as they realized they couldn't possibly find all logical signal that could have impact on stage movements. The Signals created through machine learning are called Smart Signals.

Important! Our ML Engine needs a minimum threshold of 3 signals per stage and 5 past conversions in order to calculate conversion, expansion and churn scores. These scores will be greyed out when not enough data.

Signals anatomy

Each Signal, whether Smart or user-defined, is identified by:

  • Its name: name that clearly and easily reflect its conditions.

  • Its Signal group: The group the Signal belongs to, and that will determine whether a Signal is publicly visible throughout journy.io. E.g. in the 360 customer view.

  • Its conditions.

  • Its stage associations.

  • Its impact, both on conversion/expansion and churn.

Some very important notes about conditions and impact:

  • Signal conditions always need to represent positive outcomes.

    E.g. OK ✅ (for Retention) Login > 3 times last 7 days.

    NOT OK ❌ (for churn) Login ⪬ 3 times last 7 days.

  • Positive green portion reflects how much impact the Signal has on positive movement when active! (conditions = true)

  • Negative red portion reflects how much impact the Signal has on negative movement when NOT active! (conditions = false)

image-20240410-130414.png

The Signal group is characterized by its name, a description, and foremost the option to show all belonging Signals throughout journy.io. Some Signals are important to show (e.g. one wants to show which onboarding events already occured) while others can easily stay hidden as it only makes sense for machine learning reasons and have little logical sense.

Smart Signals

As mentioned before, journy.io can create Signals based upon past stage movements, upon holding enough data into its model. What's great about the Smart Signal's ML-model is that it only requires very little training data to start providing meaningful results. The threshold is set at only 5 stage movements per stage. And of course, the more data the model is trained with, the better it becomes. If a Signal's impact becomes irrelevant for certain movements, we will automatically exclude it from the model.

image-20240410-130444.png

Going to Settings → Signals, you'll clearly see the button 'Smart Signals' with the amount of Smart Signals to be considered. Pressing the button will show you these Signals.

The repeatable process is straightforward:

  • Find the rules, grouped by stage.

  • Select the rules you want to add.

  • Optionally, change the date range. For some date ranges, you might have a figure that makes more sense for e.g. last 7 days rather than last 30 days. Making the date range smaller also makes the system react quicker on changed.

  • Sometimes, a Signal will be greyed out, because similar Signal already exist and you'd want to avoid same/double Signals.

  • Select the Signal group you want to add the Signals to.

  • Press 'Add Smart Signals'

You'll now find new Signals in chosen group. They will already have impact computed on past stage movements.

Conversion, Churn and Health Scores

Out of the Signals created, journy.io will automatically compute conversion, churn and health scores. These actionable computed properties can serve to trigger Playbooks,

serve as base for Segments and be sync’d downstream your PLG motion, to custom fields in marketing automation, CRM and ticketing systems...

And as they are scores, you can expect them in the Settings → Scores section. Obviously uneditable.

You will also see them in 360 customer views.

As well as in the Inbox sidebar.

Signals throughout journy.io

On several screens, you'll see a simple indicator ⚡️ that — upon hoovering — will display the Signals being active or NOT active for the account/user at stake, together with its impact to Conversion and Churn. If the Signal is active/set you will see a green bar indicating how much it contributes to the Conversion score; when it's NOT active/clear, you will see a red bar indicating how much it contributes to the Churn score.

The more green and less red, the better. The more red and less green, the worst. The more evenly green and red, the more indecisive. And this reflects real behaviour, as some features being used might indicate a likelihood to buy, while yet the lack of using other important features might indicate a likelihood to churn.

Also in the 360 customer view, you'll clearly find Signals back in the sidebar.

Here are Signals in the Stage View.

Health score vs Health Profile throughout journy.io

The health score eventually reflects whether to be leaning to expecting a buy or a churn. And while the Heath score is unquestionably clear about these likelihoods, the way an account/user appears to be in Good, Bad or Normal health might need some additional reflection.

By default, the 'Default Health Profile' is directly derived from Health scores. However, the SaaS market has a tendency to be seeing a Good, Bad and Normal account by more than just its likelyhood to buy or churn. We therefore kept the option for a journy.io subscriber to define his own business-typical Health profiles.