Case Study · Recommendations Platform

Cutting vendor spend $125K/yr while tripling recommendation CTR.

Activation was stuck at 45% and recommendations were a vendor black box nobody could see, tune, or own. Replaced it with a modular, explainable in-house platform spanning ranking, presentation, and a GenAI discovery assistant.

Role Built and owned the in-house recommendation platform end-to-end — architecture, ranking, experimentation, and the GenAI assistant

2021-current +200% CTR on rec lanes +114% watch time attribution $125k/yr saved vs. vendor 2 new markets (CH, AT)

Activation was stuck at 45% and nobody owned the fix

Recommendations are a core activation lever, but the vendor running them gave product and marketing no way to see, test, or improve them.

ACTIVATION

Two-minute watch rate was flat

Only 45% of users reached the Activation threshold of two minutes of viewing, and recommendation lanes weren't moving that number.

45% activation, flat for years = recs aren't moving the core metric
VENDOR

Ranking logic was a black box

A third-party recommender couldn't be inspected, tuned, or tied to Joyn's own catalog and editorial signals.

Ranking logic invisible = no way to test or improve it
OWNERSHIP

No product owner for recs

With nobody able to see or test ranking logic, engineering, data science, product, and marketing had no shared roadmap.

5% CTR, no owner = recs ignored by every team

What kept the team locked to the vendor

Cost, opacity, and a monolithic design meant every fix required a vendor ticket instead of an internal experiment.

COST

$135k/year for a closed system

The vendor contract carried a fixed annual cost regardless of whether the recommendations actually performed.

$135k/yr fixed cost = paid the same whether it worked or not
TESTING

No embedded A/B framework

Experimentation ran through Optimizely, separate from ranking, so testing new recommendation ideas was slow and disconnected.

Testing lived outside ranking = every experiment slow and disconnected
SCALE

Single-market architecture

The existing setup wasn't designed to extend cleanly to new markets like Switzerland and Austria.

Built for one market = CH/AT expansion blocked

A modular platform: candidates, ranking, presentation

The recommendation engine was rebuilt as self-contained layers so teams could test and ship independently, then extended with automated personalization and a GenAI assistant.

1
Modular engine candidates → ranking → presentation

Split the recommender into candidate generation, ranking & personalization, and presentation, letting each layer be built and tested in parallel.In-house recsys · modular architecture

2
Smart curator + cold-start lane

Reordered existing lanes by recency and user signals, and shipped a dedicated machine-curated discovery lane for new users.Presentation layer · cold-start

3
Embedded experimentation + explainability

Built A/B testing directly into the platform API to replace Optimizely, and shipped a Streamlit tool exposing ranking scores with editorial override.Streamlit · A/B framework

4
Viewing-pattern reinforcement + GenAI assistant

Detected recurring viewing habits to prioritize matching content at predicted times, and launched a conversational discovery assistant with trace-based evaluation.GenAI · pattern detection

An owned, explainable recommendation platform

+200%
CTR on rec lanes
5% → 15%
+114%
Watch time attribution
7% → 15% of total
+7%
Activation lift
Pattern reinforcement
$125k
Annual savings
$135k vendor → $10k in-house

From a single fix to a multi-market platform

⚡

A scalable, multi-tenant platform built once

The same modular architecture rolled out to Switzerland and Austria on shared infrastructure, and the embedded A/B framework now serves non-recommendation use cases across the product.

↻

10pt new-user activation lift

A machine-curated discovery lane for new users raised activation by 10 percentage points from a sub-25% baseline.

⤳

Reverse recommendations for marketing

A new model identifies which users are most likely to watch specific content, powering lookalike and retargeting campaigns for CRM.

⊕

GenAI discovery at 100k users

A conversational assistant for content discovery reached 100,000 users in four months, improving via trace-based evaluation.

⟲

Real-time observability and rollback

Monitoring dashboards track recommendation quality and system health, enabling fast rollback of underperforming models.

Let's talk

Tackling something like this?

Tell me your biggest data or AI bottleneck and I'll send you how I'd approach it — the same way I worked this problem. No résumé dump, no pitch. Just a useful conversation.

Start the conversation →