A/B Testing & Personalization
AI-Driven Experiment Design & Dynamic Content
The Gut Feel Trap
A marketing manager at a DTC brand is convinced that red CTA buttons convert better than green ones. "Red means urgency," he says. He has used red buttons for two years. No one has ever tested it. When they finally run an A/B test, green outperforms red by 23%. Two years of suboptimal conversions because of an untested assumption.
This story plays out across marketing teams daily. Decisions about headlines, offers, images, send times, and pricing are made on gut feel — or worse, on "best practices" imported from a single blog post that tested on a completely different audience. A/B testing replaces opinions with data. AI makes testing faster, smarter, and more impactful.
A/B Testing Fundamentals
The Core Framework
Every A/B test has four elements:
What AI Adds to Testing
Without AI, testing is slow and linear. You test one thing, wait 2 weeks for significance, then test the next thing. AI transforms this:
| Traditional Testing | AI-Powered Testing |
|---|---|
| Test 1 thing at a time | Test 10+ variants simultaneously (multi-armed bandit) |
| Wait for full statistical significance | Shift traffic to winners in real-time |
| Human decides what to test | AI identifies highest-impact test opportunities |
| Fixed traffic split | Dynamic allocation — winning variants get more traffic automatically |
| One audience sees one test | Different segments see different tests simultaneously |
Enterprise Testing Platforms
| Platform | Strengths | Best For | Starting Price |
|---|---|---|---|
| Optimizely | Full-stack experimentation, feature flags, server-side | Enterprise, product teams | Custom pricing |
| VWO | Visual editor, heatmaps, session recordings, testing | Mid-market, marketing teams | $99/month |
| Dynamic Yield | AI-powered personalisation + testing, recommendations | E-commerce, retail | Enterprise pricing |
| **Google Optimize** (sunset, now in GA4) | Free A/B testing integrated with GA4 | Small teams, budget-conscious | Free (via GA4) |
| LaunchDarkly | Feature flags, progressive rollouts, experimentation | Engineering-led teams | $10/month per seat |
What to Test in Western Markets
Pricing Display (Massive Impact)
| Test | Why It Matters |
|---|---|
| Monthly vs annual pricing | "$8.25/month" vs "$99/year" — monthly framing feels cheaper. Annual framing anchors total commitment. Test which drives more signups AND retention. |
| Dollar off vs percent off | "$15 off" vs "25% off" — same discount, different perception. Below $100, percentage feels bigger. Above $100, absolute amount feels bigger. |
| Free shipping threshold | "Free shipping over $50" vs "$5.99 shipping" — test where the psychological tipping point is. Amazon Prime has trained consumers to expect free shipping. |
| Currency localisation | "$49" (US) vs "£39" (UK) vs "A$69" (AU) — test whether round numbers or precise conversions perform better in each market |
Email & SMS Timing
Engagement varies dramatically by time and market:
| Time Slot (Local) | Open Rate | Best For |
|---|---|---|
| 6-7 AM (early risers) | 25-35% | Daily digests, content newsletters |
| 10-11 AM (mid-morning) | 30-40% | B2B emails, product launches |
| 12-1 PM (lunch break) | 25-35% | Flash sales, engagement content |
| 5-6 PM (commute/wind-down) | 20-30% | After-work deals, personal products |
| 8-9 PM (evening browse) | 35-45% | Fashion, beauty, entertainment |
AI runs timing tests automatically — sending messages at different times to different segments and converging on optimal windows for each audience type.
Creative Elements
Open data/ab-test-results.csv in the code panel. This dataset shows 20 completed A/B tests for a DTC brand — with variants, traffic, conversions, confidence levels, and lift percentages. Study which tests produced the biggest wins and why.
Personalisation: Beyond Testing
A/B testing finds the single best option for everyone. Personalisation shows different content to different people. AI makes personalisation possible at scale.
Personalisation Platforms
| Platform | Strengths | Starting Price |
|---|---|---|
| Klaviyo | Email + SMS personalisation, predictive analytics, DTC-focused | $20/month |
| Iterable | Cross-channel orchestration, AI-powered send time, behavioural triggers | Enterprise pricing |
| Braze | Real-time personalisation, in-app messaging, cross-channel | Enterprise pricing |
| Dynamic Yield | Website personalisation, recommendations, testing | Enterprise pricing |
Personalisation Rules and Triggers
AI-powered personalisation operates on rules:
| Trigger | Action | Goal |
|---|---|---|
| Visitor has viewed 3+ products without adding to cart | Show social proof popup: "X people bought this today" | Reduce hesitation |
| Cart value is between $40-49 | Show "Add $X more for free shipping" banner | Increase AOV |
| First-time visitor from Google Search | Show "New here? Get 10% off your first order" modal | Convert new visitors |
| User visited 3 times without purchasing | Trigger email with "Still thinking it over?" + bestsellers | Convert fence-sitters |
| User's last purchase was 30+ days ago | Send re-engagement email with "We miss you" + personalised picks | Win back dormant users |
| Black Friday is 7 days away | Show countdown timer + early access for email subscribers | Capture seasonal demand |
GDPR and Privacy Compliance
Personalisation in Western markets requires careful attention to privacy:
AI personalisation must work within these constraints. The best platforms (Klaviyo, Braze, Iterable) have GDPR and CCPA compliance built in — but the responsibility for correct implementation remains with your team.
Open data/personalization-rules.json to see a complete personalisation ruleset for a DTC brand — with 15 trigger-action pairs, segment definitions, and content templates for each rule.
Key Takeaways
This is chapter 5 of AI for Marketing Professionals (Global).
Get the full hands-on course — free during early access. Build the complete system. Your projects become your portfolio.
View course details