See how beehiiv turned subway ads from speculative spend into a proven growth tactic. By running the first statistically validated study on OOH performance, they uncovered ~100,000 incremental new users and built a data-backed case for future investment—with clear, predictable ROI.

Client Overview: beehiiv

• Industry: Email & Newsletter Platform

• Marketing Channels: 5+ platforms, 30+ campaigns including digital and out of-home

• The Bold Move: $300,000 New York subway advertising campaign without traditional measurement capabilities

The Challenge: The $300K Question Nobody Could Answer

Beehiiv faced the classic brand marketing dilemma: they were getting texts and messages from people seeing their subway ads, generating buzz and awareness.

But the executive question remained unanswered:

Did this $300,000 investment actually drive incremental business, or were we just entertaining commuters?

Traditional measurement approaches failed:

• Platform metrics: Non-existent for OOH

• Surveys: Anecdotal at best

• Attribution models: Completely blind to offline channels

• MMMs: Too slow to capture campaign-specific impact

Without answers, beehiiv couldn't determine whether to:

• Continue OOH investments

• Scale to other markets

• Reallocate to digital channels

• Adjust creative or targeting strategies

The Solution: BlueAlpha's OOH Action System

BlueAlpha deployed its measurement-to-action framework to transform OOH from unmeasurable brand spend into a quantifiable growth channel.

Phase 1: Experiment Design & Data Architecture

Timeline: Pre-campaign setup (1 week)

• Established New York City as treatment geography

• Identified control markets with similar baseline characteristics

• Set up real-time data pipelines for GA4, signup, and purchase metrics

• Created predictive models for expected performance without OOH

Phase 2: Live Campaign Monitoring

Timeline: March 24 - April 21 (4-week campaign) + 2-week cool-off period

• Daily data capture across treatment and control regions

• Weekly variance analysis to detect early signals

• Anomaly detection to flag confounding factors

• Real-time dashboard for stakeholder visibility

Phase 3: Causal Impact Analysis

Timeline: Post-campaign (1 week)

• Bayesian structural time series modeling

• Synthetic control method validation

• Incrementality calculation with confidence intervals

• ROI analysis by funnel stage

The Results: From Unmeasurable to Undeniable

Traffic Impact: Massive Awareness Spike

• ~ 100,000 incremental new website users

• ~ $4 cost per incremental visitor

• 95% statistical confidence

This validated that the subway ads weren't just wallpaper – they drove measurable digital engagement at scale.

Signup Impact: Quality Traffic Confirmed

• 100+ incremental signups

• ~ $2,700 cost per incremental signup

• 5.3x higher than digital baseline

While expensive compared to digital, this proved OOH could drive action, not just awareness.

Purchase Impact: The Revenue Reality Check

• 15-20 incremental purchases

• ~ $17,000 cost per incremental purchase

• $19,000-25,000 in direct revenue (at $1,250 LTV)

• ~ 93% immediate ROI

The harsh truth: OOH generated real customers but at a much-higher-than average acquisition cost for direct response.

The Actions: Turning Insights into Execution

BlueAlpha didn't just deliver a measurement report. The system generated specific, executable recommendations:

Immediate Actions Taken

1. Portfolio Rebalancing

2. Campaign Optimization Framework

3. Creative Strategy Tune-up

Strategic Decisions Enabled

Market Expansion Confidence

With validated measurement capabilities, beehiiv could now:

• Test OOH in new markets with clear success criteria

• Compare market-by-market efficiency

• Build predictive models for OOH performance

• Justify brand investments to stakeholders with data

Budget Planning Precision

The insights enabled:

• Separate performance and brand budget allocation

• Quarterly OOH testing budget with clear ROI expectations

• Data-driven cases for board-level budget discussions

• 
  

Why It Worked: The Action System Difference

1. Geo-Experimentation at Scale

BlueAlpha's DMA-level modeling turned New York into a massive A/B test, isolating OOH impact from other marketing activities and organic growth.

2. Multi-Signal Triangulation

By tracking traffic, signups, and purchases simultaneously, the system revealed the full funnel impact - critical for understanding OOH's role in the customer journey.

3. Rapid Iteration Cycles

Weekly monitoring meant beehiiv could have paused or optimized mid-campaign if needed, rather than waiting months for post-campaign analysis.

4. Transparent Methodology

No black boxes. Beehiiv's team understood exactly how incrementality was calculated, building confidence in the tough decision to limit OOH spending.

5. Action-First Reporting

Results weren't buried in slides. Each insight came with a specific recommendation: scale, pause, test, or optimize.

The Breakthrough: Why This Matters for Marketing

This case represents a watershed moment in marketing measurement. For the first time, a mid-market company could:

• Prove OOH incrementality with the same rigor as digital channels

• Make same-quarter decisions instead of annual planning cycles

• Separate brand from performance with actual data, not intuition

• Test boldly knowing they'd have answers, not just anecdotes

As EJ White noted on the Numbers and Narratives Podcast: "Brandformance is the hot word these days, but nobody has ever been able to show actual performance lift from brand campaigns. We were able to show statistically valid proof."

Fast Facts

• $300,000 subway advertising investment analyzed

• ~ 100,000 incremental website visitors detected

• 95% statistical confidence achieved

• 6 weeks from campaign launch to full insights

• First ever statistically validated OOH measurement for the company

• Weekly optimization recommendations delivered

About BlueAlpha

Founded by former Tesla leaders, BlueAlpha is the marketing action system that transforms measurement into execution. We don't just tell you what happened – we tell you what to do next. Our platform combines always-on incrementality testing, AI-powered decision engines, and human expertise to turn marketing chaos into strategic clarity. From subway ads to social media, we measure what matters and deliver the actions that drive growth.

Stop measuring. Start executing.

Transform Your OOH Investments from Hope to Science

Ready to measure the unmeasurable? Whether it's billboards, transit ads, or TV spots, BlueAlpha turns brand spending into data-driven decisions.

• Book an OOH Measurement Strategy Session

• Read the Full beehiiv Digital Channels Case Study

• Follow BlueAlpha on LinkedIn