Joyn's event pipeline was an unbounded, unbilled firehose — 1B calls a month on track to cost $1.05M over three years. Rebuilt in seven deliberate steps into a governed, cost-controlled, and now agent-operated data platform — without a single big-bang migration.
Role Led the CDP migration end-to-end — cost diagnosis, schema governance, and the agentic tracking pipeline
Cost, quality, and identity were each broken on their own — and each was making the other two worse.
A heartbeat fired every 10 seconds to track playback. Raw 2KB payloads were routed unfiltered to every downstream tool — and CRM, analytics, and ad platforms all bill by volume.
Engineers could push any payload to production. Malformed events regularly broke downstream pipelines, taking marketing, CRM, and analytics down together.
Behaviour was siloed across six touchpoints — web, mobile, CTV, and offline billing — with no way to join them. LTV and cross-channel campaigns were guesswork.
Each phase solved the most pressing problem of its moment — first cost, then governance, and finally the way events themselves get created. The track shifts from teal to purple where the platform stops being something engineers operate and starts operating itself.
Moved raw heartbeat events off the legacy stack into Google Cloud Functions. Fast to ship — but invocation cost scaled linearly with every ping, so it stayed too expensive at Joyn's volume.
Migrated heartbeat processing to Cloud Run for containerised, always-warm compute. Per-call infrastructure cost dropped sharply, giving the team a stable, scalable base to build on.
With infrastructure cost under control, tracking grew beyond heartbeats — screen views first, then broader behavioural events. These non-heartbeat events added roughly 30% more event volume on top of the heartbeat baseline, producing Joyn's first complete picture of how users actually navigate the product — and making disciplined cost and schema control matter across every event type, not just playback.
The boldest optimisation: instead of a fixed 10-second ping for the whole session, the client transmits on a Fibonacci schedule — 0s, 10s, 20s, 30s, 50s, 80s, and onward. High resolution early, widening intervals as confidence builds. Total server calls fell dramatically with no loss of meaningful playback signal.
Segment deployed as the central CDP. A formal tracking plan with hard schema validation replaced the free-for-all. Clean, structured data now routes by intent — heartbeats to Mixpanel only; conversion events to Braze, Adjust, Facebook, and Google.
We wrote our own frontend tracking library to own event transmission, with the event specification compiled directly into it. A control failure — wrong field, wrong type, missing property — surfaces as a compiler exception at development time, not as a broken pipeline in production days later. It also ended the endless refinement debates over what value a field should carry: the answer is the specification, and the compiler enforces it. Fewer meetings, harder guarantees.
Event creation is now an agentic workflow. A developer describes intent in plain language; agents handle the rest end to end.
Quality is now enforced in three layers that reinforce each other: the compiler rejects bad events at development time, schema validation gates them in CI/CD, and the spec agent prevents malformed or duplicate events from ever being authored. Errors are designed out before they exist — not debugged after they ship.
Malformed events are caught at the compiler and at schema validation — before they reach any downstream system.
Tracking changes auto-validate in CI/CD, and agents draft and open the merge requests. No manual QA loop, no spec-debate meetings.
Six touchpoints — web, mobile, CTV, and billing — merged into one identity, making accurate LTV possible for the first time.
Sub-second clean-data delivery into Braze enabled real-time, event-driven campaigns.
Offline billing events joined with online acquisition signals — giving the growth team accurate CAC and multi-touch attribution across the full funnel.
Let's talk
Tell me your biggest data or AI bottleneck and I'll send you how I'd approach it — the same way I worked this problem. No résumé dump, no pitch. Just a useful conversation.
Start the conversation →