Case Study · Marketing Data Pipeline

Lifting campaign ROI 21% by streaming 300K subscribers into GA4.

Subscription behavior lived outside GA4 and the 300K-user dataset exceeded what Excel could process, leaving marketing unable to segment campaigns by real subscriber behavior. I built a Python and Spark pipeline on Databricks to stream that data into GA4 continuously.

Role Built and owned the GA4 subscription data pipeline end-to-end — Spark querying, Python transformation, and automated scheduling on Databricks

2021-current +21% campaign ROI 300K users processed Spark SQL on Databricks ga4mp API · Python
01 — The Business Driver

Subscription data never reached GA4

Marketing needed subscription behavior in GA4 to optimize ad campaigns, but the data lived elsewhere and couldn't get there at scale.

GAP

No subscription data in GA4

GA4 lacked the subscription event and property data marketing needed to optimize campaign targeting.

No subscription events in GA4 = campaigns target blind
SCALE

Excel couldn't handle the dataset

The 300K-user subscription dataset exceeded what spreadsheet-based processing could realistically transform.

300K-row dataset = too large for spreadsheet processing
ROI

Campaign targeting was under-optimized

Without granular segmentation data, ad spend couldn't be directed toward the highest-value subscriber segments.

No behavioral segments = ad spend not aligned to value
02 — The Obstacles

What stood between raw data and GA4

Getting subscription data into GA4 meant solving three problems at once: querying at scale, reshaping for GA4's event format, and delivering on a schedule.

QUERY

Large-scale querying needed Spark

Efficiently pulling subscription data from source databases at 300K-user scale required distributed query execution.

300K users to query = single-machine tools fall over
FORMAT

GA4 requires a specific event shape

Raw subscription records had to be transformed and batched into the event and user-property format GA4's API expects.

Raw subscription records = unusable by GA4's API as-is
DELIVERY

One-off exports wouldn't scale

Marketing needed continuous, automated delivery of fresh subscription data, not manual periodic exports.

Manual exports = GA4 data goes stale between runs
03 — The Solution

A Python + Spark pipeline streaming into GA4

Built on Databricks, the pipeline queries, batches, transforms, and ships subscription data to GA4 on an automated schedule.

1
Query at scale database → Spark SQL

Used Spark SQL on Databricks to efficiently query subscription data for 300K users directly from source databases.Spark SQL · Databricks

2
Batch and transform with Python

Broke the 300K-user dataset into manageable batches with Spark, then transformed records into GA4's required event format with Python.Python · Spark batching

3
Stream to GA4 via Measurement Protocol

Used the ga4mp API in Python to send transformed events and user properties into GA4 as granular, real-time records.ga4mp · Python

4
Automated, continuous scheduling

Scheduled the pipeline on Databricks for continuous automated processing, keeping GA4 data current without manual exports.Databricks · job scheduling

04 — Quantified Impact

Granular subscription data, live in GA4

+21%
Campaign ROI
Improved targeting
300K
Users processed
Per pipeline run, continuously
05 — Strategic Outcomes Unlocked

From a spreadsheet ceiling to a continuous data pipeline

⚡

Marketing gained a real-time subscription signal

Rich, granular subscription data now flows continuously into GA4, giving marketing a durable foundation for segmentation and conversion tracking far beyond the original campaign.

⊕

A reusable large-scale ETL pattern

The Spark + Databricks batching pattern generalizes to other large datasets that previously hit Excel's limits.

↻

Continuous data freshness

Scheduled Databricks jobs keep GA4 data current automatically, removing manual export work entirely.

Let's talk

Tackling something like this?

Tell me your biggest data or AI bottleneck and I'll send you how I'd approach it — the same way I worked this problem. No résumé dump, no pitch. Just a useful conversation.

Start the conversation →