Skip to main content

Documentation Index

Fetch the complete documentation index at: https://docs.messagesender.ai/llms.txt

Use this file to discover all available pages before exploring further.

When you’re not sure which version of a message will land best, an A/B test lets the data decide. Amplifi sends two different message variants to a small, randomly selected sample of your audience. After reviewing the results, you choose the winning variant and send it to the remaining contacts. This approach reduces risk and helps you learn what resonates with your audience before going wide.

​
How A/B testing works

Amplifi splits your audience automatically when you enable A/B testing:
  • Variant A goes to 5% of your audience
  • Variant B goes to 5% of your audience
  • The remaining 90% is held back until you pick a winner
The sample contacts are selected randomly, so both variants reach a representative slice of your audience. Once the sample sends complete, you review performance and select the winning variant to send to the other 90%.
A/B testing works best with larger audiences where a 5% sample gives you statistically meaningful results. With very small lists, the sample sizes may be too small to draw clear conclusions.

​
Set up an A/B test

1

Start composing a campaign

Go to Campaigns and click Create Campaign. Choose your audience and set up your campaign as usual.
2

Enable A/B testing

In the campaign composer, enable the A/B test option. This adds a second message body field to the form.
3

Write Variant A

Write your first message version in the Variant A field. This is typically your control message — the version you’d normally send.
4

Write Variant B

Write your second message version in the Variant B field. Change whatever you want to test: the opening line, call to action, tone, length, or whether you include a link.
5

Launch the A/B test

Schedule or send the campaign. Amplifi immediately sends Variant A to 5% of your audience and Variant B to another 5%. The other 90% receives nothing yet.

​
Compare variants and pick a winner

Once the sample messages have been sent, go to the campaign detail page to review performance.
Amplifi shows you side-by-side stats for each variant:
  • Sent count — how many messages went out per variant
  • Delivery rate — percentage of sent messages confirmed delivered
  • Reply rate — percentage of recipients who replied
  • Link click rate — if you included a tracked link, the percentage who clicked
Look for meaningful differences across these metrics. A variant with a higher delivery rate and higher reply rate is a clear winner. If results are mixed, consider which metric matters most for your campaign goal.

​
Tips for effective A/B tests

Test one thing at a time. If Variant A and Variant B differ in multiple ways — the opening, the call to action, and the link placement — you won’t know which change drove the difference. Isolate one variable per test for cleaner results.
Give the sample sends enough time to collect replies and link clicks before picking a winner. Reply rates in particular can take a few hours to stabilize after delivery.