Two versions, one winner – The A/B test
No Comments
You have integrated SMS into your marketing and want to make sure that your campaign runs as well as possible? A/B tests are a great tool for optimization in SMS marketing. They effectively show the strengths and weaknesses of different approaches and ensure that your campaign is actually speaking to your target group. In addition, the tests help you to better assess your target group, especially if you are in a phase where your customer base is growing quickly. If your A/B tests are successful, you can be sure that the winner version will achieve the best results.
We want to help you conduct your first A/B test in SMS marketing, so that you can fully exploit the potential of your campaign.
What is A/B testing?
Very briefly: In A/B testing, two versions of an advertisement are played out to a certain number of people and thus tested against each other. The ads differ only in one single detail, because this allows you to draw precise conclusions as to why ad A performs better than ad B. Of course, other media can also be tested, such as newly designed websites or shopping carts. In this article, however, we will focus on testing ads in general and marketing SMS in particular.
The beginning
Before you can start with the actual test, you need a basis. Therefore, first consider a hypothesis that the test is intended to confirm or refute. Ideally, this should not be based on a gut feeling, but on key figures that you already know. If you cannot yet rely on your figures, you should at least make sure that your hypothesis is logical and fits your target group.
Your hypothesis could be
Our primary target group is more likely to respond to high discounts than to low prices.
Create two versions of an advertisement
In your A/B test, you should test a new ad against a control version, of which you already know how it will perform in the test from past experience. This will ensure that you have a solid starting point from which to compare the results. Also, this allows you to quickly identify any anomalies in the test. Anomalies can vary in severity and be completely unexpected, depending on your industry and target audience.
In our example, you would now change an advertisement whose values you already know. Let’s assume that in the known ad you are promoting a new collection with 10% discount. Create a new ad on this basis, in our example for a new towel collection. Change this ad so that the corresponding monetary value is displayed. Now there are two different versions of the ad that can be tested against each other, version A and version B.
The test groups
Admittedly, setting the test group size and duration takes a little time. You need a current conversion rate, a desired increase (in percent) of this conversion rate and then you have to put them into a meaningful relationship. Fortunately, there are not only convenient calculators to help you determine the test group size, but also detailed explanations on how to determine the required key figures. In any case, try to get a basic understanding of the statistics on which the A/B test is based. The better informed you are, the fewer errors you will make. Also make sure that the people who will receive the A/B tests have never seen any of the versions before, because this would significantly falsify the result.
The evaluation
When your test has run for the necessary period of time, you can finally evaluate the results! Of course it is important that you can interpret them correctly. Therefore you should first check whether the difference in the test result is statistically reliable. Again, there are calculators to check the statistical relevance. Only if this relevance is given, you can use the result as a reason to align your campaign accordingly. If the results are not relevant, you should make a new hypothesis and test it. Do not use an advertisement of an A/B test for a new test if there is a chance that former test participants will see the advertisement again.
You should be aware in advance that A/B tests are often inconclusive because the result is not statistically relevant. However: The stronger the base of your hypothesis is, the less often the result will be irrelevant. Also, don’t be discouraged if your tests turn out differently than you had hoped. You can learn something from all test results, even those that do not achieve statistical relevance.
What can you test?
Do you still need inspiration about which details of your SMS campaign you could improve? No problem: Here are our tips for possible A/B tests
Timing
The timing of your SMS can make a big difference. Try sending on different times of day or days of the week. However, make sure not to send SMS too late or too early in the day. You probably do not want to receive a message at five o’clock in the morning, either. Also be careful if the recipients live in another country: Always calculate the time zone of the destination country! Note that in some countries it is forbidden for companies to send SMS at certain times. In some cases, your SMS may not be delivered at all during these periods, but will end up in a queue. Be sure to inform yourself in advance about the particularities of the destination countries.
Tonality and different approaches
Especially for young companies, tonality can make a big difference in the success of their advertising efforts. Test whether your target group is more likely to respond to offers that contain a value proposition or to those that convey an urgency, for example offers that are limited in time. It can also be decisive whether you are issuing a call to action or if you are rather making the recipients curious. Finally, especially if your company is still very young, you can test whether the target group responds better to emotional or rational messages.
Personalization
In most cases a personalized message is a good idea. When you address customers personally, they feel more acknowledged as an individual and build a better relationship with your company. However, under certain circumstances, personal details may be undesirable, especially if you sell sensitive products. The acceptance of certain target groups can also vary greatly. You could, for example, test whether your customers want offers that are tailored to their marital status, place of residence or gender.
However, you’re usually good to go with using the name, which you can insert using wildcards.
In the end there is only one thing left to do: Be happy about your optimized campaign. By the way, you should repeat A/B tests from time to time, especially if a rebranding is taking place or your customer base is growing strongly. With our performance and conversion tracking, you can always keep track of how well your current campaign is performing so that you can take appropriate action in good time.
All the best,
Source of header picture: iStock/phototechno