Case Study · CRO & Personalization

50 A/B tests a year, run as a continuous conversion engine.

Optum's product pages and acquisition funnels needed constant tuning to keep users engaged and moving toward their health goals, not bouncing off generic layouts.

Role Led test strategy and prioritization — turned product-analytics hypotheses into a monthly experiment pipeline

2017-2018 50 tests run in 2019 70 tests run in 2020 47% avg. activity lift (2019) 14% avg. conversion lift (2020)

User journeys had to actively guide people to their health goals

Optum's digital products needed to optimize how users moved through topic pages, search, and app-download flows so the experience matched each customer's underlying objective.

CAPABILITY NEEDED

Journey-level optimization

Provider search, topic pages, and landing pages all needed structured experimentation to find layouts that moved users toward their goals.

Static layouts = users disengaging before reaching their goal
PERSONALIZATION GAP

One-size-fits-all content

Users with different category affinities (e.g. mental health, depression topics) were shown the same generic content regardless of demonstrated interest.

Same content for every user = missed chance to guide them
WHY VENDOR TOOLS FAIL

Off-the-shelf tools lack health-domain nuance

Generic CRO tooling can run tests, but prioritizing and interpreting them for sensitive health topics requires in-house strategy and analysis.

Off-the-shelf CRO tools = can't prioritize for sensitive health topics

Running 50+ tests a year requires a repeatable pipeline, not one-off experiments

At 50-70 tests a year, every stage — generating hypotheses, choosing which to build, and handling sensitive topics — needed its own process.

VOLUME

High test throughput

Sustaining 50-70 tests per year meant every step from hypothesis to insight needed a defined, monthly-reviewed process.

50-70 tests a year = no room for ad-hoc execution
PRIORITIZATION

Competing hypotheses across teams

Product analytics teams surfaced hypotheses faster than they could be built and shipped, requiring a prioritization layer before development.

Hypotheses outpaced build capacity = needed a prioritization step
SENSITIVITY

Sensitive topic pages

Tests on pages like "depression" topic content needed careful layout variants to find engagement gains without compromising user experience.

Depression-topic pages = layout changes needed extra care

A monthly-cadence pipeline from hypothesis to insight

A six-stage loop — recommend, hypothesis, prioritize, develop, go live, analyze — ran on a monthly review cycle, with strategy and execution owned end-to-end.

1
Hypothesis & prioritization insights → test queue

Product analytics teams identified hypotheses; strategized and prioritized which to build each month based on expected impact.Hypothesis · prioritization

2
Build & cross-team execution

Collaborated across teams to build test variants — from CTA layout changes to multivariate landing page tests (MVT).A/B · MVT

3
Targeted, affinity-based variants

Designed variants around observed category affinity, offering next-best-action content based on a user's demonstrated interests.Personalization · next-best-action

4
Go live, analyze, repeat monthly

Shipped tests, analyzed results monthly, and fed insights back into the next round of hypotheses — a continuous loop.Analysis · monthly review

Consistent double-digit lifts across two years of testing

50
Tests run in 2019
47% avg. activity lift
85%
Tests with significant lift
2019 acceptable-lift rate
70
Tests run in 2020
14% avg. conversion lift
36%
Positivity rate
2020 results

From one-off page tweaks to a year-round optimization engine

⚡

A scaled, repeatable optimization engine

Optum moved from one-off page tweaks to a monthly pipeline running 50-70 experiments a year across search, topic pages, and app-download flows — with most tests landing significant, positive lifts.

⊕

Affinity-based personalization

Next-best-action content based on demonstrated category affinity gave users more relevant paths through sensitive topic areas.

↻

NPS contribution via engagement

More engaging content helped customers progress toward their health goals, contributing marginally to overall NPS improvement.

Let's talk

Tackling something like this?

Tell me your biggest data or AI bottleneck and I'll send you how I'd approach it — the same way I worked this problem. No résumé dump, no pitch. Just a useful conversation.

Start the conversation →