A/B Testing

What is A/B testing and how it helps

A/B testing lets you test up to five different email variants (A, B, C, D, E) with different subject lines or email content. This helps you identify which version performs better for opens and replies in your cold email campaigns.

How it works

Step 1: Add email variants during campaign creation

1. Navigate to campaign creation and add an email step
2. Click "Add Variant" to create additional versions (up to 5 total)
3. Customize each variant with different:
   • Subject lines
   • Email body content
4. Save your variants

Step 2: Launch your campaign

1. Start your campaign using Campaign Soft-Start or regular launch
2. SmartReach automatically splits prospects evenly across active variants
3. Each variant is sent randomly to approximately equal groups
   • Example: 200 prospects with 2 variants = 100 prospects per variant

Step 3: Check results and optimize

1. Select your campaign from the dashboard
2. Click "Reports"
3. Scroll down to view detailed results for each variant
4. Check the template section for additional analytics

Step 4: Remove underperforming variants

1. Open the email step editor
2. Click on the underperforming variant
3. Click the delete icon to remove it permanently, or
4. Use the toggle switch to pause it temporarily

What happens when you delete a variant:

• The variant and its stats are permanently deleted
• Remaining variants get renamed (if you delete variant B, variant C becomes the new variant B)
• Future prospects receive emails using only the remaining variants

For example: Let's say you have a campaign which has a Step-1 with two variants A and B. You have added 100 prospects to the campaign and started the campaign with the Campaign Soft-Start. Let's say that on the first day, 20 prospects each have received variants A and B (so 40 prospects have received the Step 1 till now). Now, you delete variant B from the Step-1. Then the other 60 prospects in the campaign will receive variant A of Step-1.

Best Practices for A/B Testing

Test One Element at a Time: Focus on testing either the subject line (for open rates) or email body elements (e.g., headings, body text, call-to-action, images, layout) for reply rates to clearly identify what impacts results.

Random Distribution: Each step’s variants are randomized independently, so a prospect receiving variant A in one step may receive variant B in another thus you should be able to track performance of each variant with precision.

FAQs

Q. What should I A/B test first?

Test multiple elements at a time for clearer results:

Subject lines - to improve open rates
Email body elements - to boost reply rates (headings, content, call-to-action, images, layout)

Q. Can I add variants after starting my campaign?

Yes, you can add new variants to existing campaigns. New prospects will be distributed across all active variants, including newly added ones.

Q. How long should I run an A/B test?

Run your test long enough to gather meaningful data. Wait until each variant has been sent to at least 50-100 prospects before making decisions about performance.