Highlights Testing communication chains helps determine the most efficient ways of interacting with the customer Previously, in order to test the workflow, it was necessary to add a flow split node, then unload actions and manually count statistics for each branch. Now all these actions can be completed via the new A/B test widget: split the branches and select the main test […]
Beethoven’s Story: Using a Neural Network to Forecast Best Sending Time and Reactivating an Invalid Database
Pet store chain. Founded in 1993
Goals
Increase online store revenue
Solutions
Switch to a new CDP
IT stack.
ERP, Mindbox marketing automation platform
Results
Target revenue was exceeded by 30% and reached the pre-CDP switch level
Time to value.
1 year
Noteworthy features
The Next Best Action algorithm helped determine the best sending date for emails, which then became more effective than trigger-based campaigns
September 26, 2022
The following case study is from Mindbox, the original brand behind Maestra’s technology
In 2020, the Beethoven pet store chain switched to a new CDP with the aim of enhancing its communication with customers and increasing the online store’s revenue. The switch to the new platform allowed trigger warnings to be set up, prevented revenue loss and unified and segmented the database. It also provided the opportunity to test a new neural network algorithm that forecasts the best sending time for emails to increase repeat purchases. In this article we will be analyzing:
- What triggered the decision to switch CDP;
- How the neural network’s LSTM model (the Next Best Action algorithm) increased the mailing quality, making them more efficient than trigger campaigns and helped increase mailing revenue by 8.5 times;
- How to segment the database, optimize bulk campaigns, and what to do with invalid email addresses;
- Future plans for improving the channel.
Results after Switching to the ew CDP
-
Target revenue was exceeded by30 %and reached the same level as before the implementation
-
The CDP now contains2.5 xmore valid customers thanks to the unified database and data cleansing
-
Revenue from emails increased by8.5 xwith the Next Best Action algorithm (based on last-click attribution in A/B tests)
-
The brand now sends23 xmore targeted emails (compared to trigger emails), aimed at boosting repeat purchases with the Next Best Action algorithm
Why Beethoven Decided to Switch to Mindbox
Beethoven decided to start developing direct marketing a year ago. The existing CDP could not ensure mailing growth because it didn’t allow for workflows to be customized or processes to be automated. For instance, segmentation was set up manually because it was impossible to change the selection parameters for product recommendations, and the overlapping of trigger campaigns wasn’t tracked. To better understand the needs of their customers and manage the settings themselves, Beethoven decided to switch to Mindbox.
At some point, I realized that if we want to grow, we need better partners, so we decided to switch to Mindbox. We’re launching the app soon, so I’m interested in working with recommendations for the app, and sending push notifications and triggers, among other features. It’s much easier to set it all up in Mindbox, and I’m sure we’ll be able to sort it all out simply using the platform.
So far, we have moved standard triggered flows like “Abandoned Cart” and “Abandoned Views, ” taken a look at how they work and set up triggers, time intervals and a priority sequence. I am grateful to the Mindbox manager who supervised these settings. The only thing that remains for us is to see how the recommendations work.
Before the switch to Mindbox
After the switch to Mindbox
Workflows
There was no understanding of how workflows and algorithms worked
Transparent workflows that can be changed and customized according to the company’s needs
Segmentation
Pre-configured segmentation conditions, manual list uploads
The conditions and attributes for segmentation are set by the company; the platform runs segmentation and sends campaigns
Recommendations
Pre-configured algorithms with recommendations meant that it was impossible to analyze their operation or change the settings manually
Selecting products and recommendation algorithms is now possible. For example, if the email conversion is low, it’s possible to change the product and display only discounted or new products
Trigger Chain Management
The system didn’t track whether emails from different trigger chains were duplicated and still sent a similar email
If an email is sent to a customer and is duplicated in another trigger chain, a similar email won’t be sent
Attributions
It was impossible to track the true effectiveness of the mailing because the system only counted clicks in the email itself
The platform attributes the customer’s purchases to the mailing list if the customer has made no other actions for 30 days. For example, if a customer received a mailing, and a couple of days later purchased the product directly through the website, the platform will attribute the purchase to the mailing
Using a Neural Network to Help Forecast the Best Sending Time for an Email with a Repeat Purchase Offer
Pet owners order products such as pet food and litter with a certain frequency, as they have to replenish them from time to time once they’ve run out. The task is to predict when the customers will need the product next and to send them a correspondening email with an offer.
The Next Best Action neural network algorithm helped forecast the probable purchase date for each customer. The neural network analyzes the interval between the customer’s purchases and how often the customer opens emails and makes purchases after receiving them.Then it suggests the Best Sending Time for an email with recommendations. The email lists the products that the customer purchased previously and contains a “Repeat My Previous Order” button. After clicking the button, the customer is redirected to a page with a cart with the same preselected products.
We conducted an A/B test to evaluate the results of the algorithm. Some of the customers received emails based on the average time between purchases; others received emails based on the neural network’s forecast. The significance level of the A/B test results is 95%. The NBA algorithm produced higher results compared to the trigger, which sets a fixed time between email sendings:
Indicator
Increase
Targeted emails
23 times in absolute value
Open rate
17 times in absolute value
Revenue for email sending
1.2 times in absolute value
Conversion rate
1.5 times as a percentage
-
The revenue from the NBA algorithm reached
26.9 % of the total revenue generated by automated campaigns
In February, the NBA algorithm was launched for the entire database and, in just 2 weeks, it overtook all triggers in terms of efficiency and reached 26.9% of the total revenue received from automated campaigns. The only original trigger that remained competitive with the NBA algorithm was the one that sends an email with a promo code 15 days after a customer’s purchase. If the positive results we see from using the NBA algorithm persist, we will consider disabling the trigger with a promo code for a future order and replace it with an algorithm. This step will increase our margins without sacrificing any customer benefits. The customer will still receive reminders about orders.
An email with a “Repeat My Previous Order” button is sent when we predict the customer is about to run out of their previously ordered goods. The Best Sending Time is calculated by the NBA neural network algorithm.
Improved Segmentation
Previously, only 30% of the entire customer base was segmented, and the database did not include any data from ERP software used by Beethoven. Moreover, in order to create a segmented campaign, an email list had to be requested from the service manager, and the mailing had to be configured manually from an Excel spreadsheet.
We’ve now collected data on the existing five pet segments (cats, dogs, birds, small pets, and fish), configured the correlations between orders and views and covered 100% of the database. Now we know the interests of our customers for certain and can interact with them in a targeted manner.
To engage the target audience more accurately, trigger campaigns for new products are also divided into categories: cats, dogs, cats and dogs, birds, small pets, fish, and the pet pharmacy. An email with new products is sent if the customer has not made any purchases within the last month. The segment is recalculated every day and includes new products that have not been included in campaigns over the last 30 days. A trigger checking the availability of the products in the segment and whether the customer received an email with them the previous week is launched weekly on Thursdays.
The campaigns for the “Cats” and “Dogs” segments are sent before the “Cats and Dogs” segment, in order to avoid email duplication for those who have several pets.
To reactivate customers who have not made orders for a long time, in addition to new products, we also offer a discount.
Bulk Campaign Optimization
-
Bulk campaigns are sent
2 times a month, which helps to save customers who are interested in communication in the database
Now, bulk emails are automated and sent only twice a month. This saves marketing resources and keeps only the customers who want emails in the database. In addition, infrequent emails to inactive customers positively affect the spam profile of the store’s domain.
We only send out promotional campaigns (e.g., product discounts). The company automatically generates emails from a template banner in which the sizes of discounts and featured products and brands are inserted. Then, the email is uploaded to the CDP. The marketer changes the templates every few months to diversify the mailing.
The mailing list goes either through the entire database at once, or by segments for owners of cats or dogs. Because mass emails are rarely sent and trigger emails are sent only to those who visited the website, the company is able to avoid spamming customers. In addition, it helps gently remind churned customers about the company.
Bulk campaigns are sent by segments depending on the customer’s pet
Trigger campaigns overtook bulk campaigns in December 2020, which signified the growth of the active customer base. The channel grew due to the new CDP as well as due to additional advertising and affiliate programs.
Reactivating Customers with Invalid Email Addresses
-
The open rate of campaigns sent to invalid emails amounted to
66 %
The customer’s email address can be entered in the list of invalid email addresses if the platform detects an incorrect email address (for instance, gmail.net instead of gmail.com), or if the mailing fails to deliver for any other reason. This measure is necessary to understand the real size of the customer base available for communication.
Some customers with invalid addresses can be activated. The email may be valid and belong to a real customer that had a full inbox at the time of receiving the email, or the server may have rejected the email for other reasons. To give these contacts a second chance, an email is sent once a month only to invalid addresses. Some of the emails were never delivered, hence, these customers were excluded from the campaigns. However, most of them were actually delivered. The open rate for delivered emails averaged 66%; the click rate was 14%, and some emails even led to purchases.
At the same time, it is crucial not to send more than one email a month to invalid customer emails and exclude email addresses that are clearly invalid, such as those with non-existent domain names. Otherwise, it may have a negative effect on the spam status of the sender’s domain.
Developing Beethoven’s Digital Channel: Next Steps
- Increase revenue from campaigns by 15% in 2021;
- Launch end-to-end recommendations on the website and in the app;
- Launch AMP mailing lists;
- Transfer service emails to the platform.