Incrementality

The counterfactual question

Every outbound program produces two kinds of outcomes: those it caused, and those that would have arrived anyway — the prospect who was already searching, the renewal that was already committed. Incrementality is the first kind only. Measured, an incrementality-measurement vendor, defines it as the sales a tactic drives beyond what would have occurred without it; the definition holds across channels because the counterfactual is the same everywhere.

Attribution answers a different question. It looks backward at touches that all happened — the ad, the email, the call — and divides credit among them by rule. No attribution model, however elaborate, observes the world in which the touch never occurred; it can only re-slice the observed one. Incrementality is forward-looking by construction: the measurement is designed before the outcome exists, by choosing whom not to touch.

Measurement by withholding

The instrument is the randomized holdout. Leads are split at random; the treated group receives the program, the holdout receives nothing; the difference in outcome rates — written τ — is the incremental effect. Randomization is what licenses the causal reading: because the two groups differ only by the coin flip, whatever gap opens between them is the program’s doing, up to the stated confidence.

In B2B this is understood but historically underused. Dreamdata’s treatment of B2B incrementality notes that formal experiments have historically demanded dedicated teams and specialized software — the withheld group costs pipeline today in exchange for certainty later, and someone has to keep the assignment ledger honest. The practical questions are how large the holdout must be, how long the outcome window runs, and how to read the result without inflating it.

What the numbers look like in cold outbound

Effects in cold outbound are small in absolute terms, so the evidence requirements are concrete. RevenueOS’s published minimum-n table (generated 2026-07-03, at α = 0.1 — a 90% anytime-valid confidence sequence) prices them. In its conservative cell A1, a planned 0.1% organic positive-reply rate in the holdout against 0.64% per lead in the proof cell gives τ = 0.54%, and reaching the w = τ precision target takes 21,928 evidence rows. The n counts randomized leads in the proof and holdout cells — not emails sent, and not all enrolled leads.

Two honesty notes travel with that number. The 0.1% organic baseline is a conservatism assumption, not an observation — holdout leads sit in no email campaign, so their organic positive replies are structurally unobservable today, and the table says so in plain text. And the estimand is windowed: only outcomes inside each lead’s attribution window count, so a meeting that lands months later is not quietly claimed.

How RevenueOS uses this

Panel A of RevenueOS’s published minimum-n table is program incrementality in exactly this sense: a proof cell of mailed leads contrasted against a never-mailed holdout, with the per-lead difference fed into an anytime-valid confidence sequence that can be read at any time without inflating the error rate. The proof cell’s arms are uniformly rotated rather than Thompson-optimized, so the shipped contrast reads as a lower-bound proxy for the optimized program — a deliberately modest estimand, stated before a partner asks.

RevenueOS is a proof layer for cold outbound: it runs the randomized holdout, keeps the assignment ledger, and publishes recomputable confidence bands on the incremental effect rather than an attribution report. As of July 2026 the firm is at design-partner stage; the figures above are planning floors from its published table, not customer results.

Related terms

Randomized holdout · Causal lift · Minimum detectable effect