Methods · revised 2026-09-02 · every figure carries its source
The math, in writing
Design-partner stage, on the record: the machinery on this page is built, verified, and simulated — customer enrollment opens with the pilot, and no figure below is a customer result.
Why can’t a dashboard tell you which email angle works?
Because outbound metrics are checked continuously. A classical significance test assumes you look once, at a preplanned sample size; a dashboard is looked at every day. Checking a running experiment until it looks good — the peeking problem — inflates false positives far past the promised rate, so an angle that merely got lucky for a week gets scaled as a winner.
What is an anytime-valid confidence sequence?
A confidence sequence is a confidence interval you may legally read at any time. RevenueOS uses anytime-valid sequences at α = 0.1: a band that starts wide, narrows as evidence accrues, and keeps its 90% coverage guarantee no matter how often anyone checks it. A claim closes only when its band excludes the no-effect point — zero on the difference scale, 1.00× on the ratio scale — and stays open, honestly, until it does.
How many leads does a claim need before it closes?
The floors below come from simulating the shipped proof engine — two hundred Monte-Carlo runs per grid point — never from a textbook power formula. Panel A prices program incrementality against a never-mailed holdout; Panel B prices arm-versus-arm lift, where small effects on small bases are honestly expensive. Rows count leads randomized into the proof and holdout cells, not emails sent.
Generated 2026-07-03 · α = 0.1 · design assumed: 10% holdout / 10% proof cell (the 25%/25% design accrues rows 3.0× faster at the same sends budget). “≥” marks a lower bound where the floor never crossed inside the simulated grid.
| Cell | Rung | Planned rates (holdout → proof) | τ | Rows n (w=τ, median [p25–p75]) | Rows n (w=2τ, bare) | 3K sends/mo | 5K sends/mo | 10K sends/mo | 20K sends/mo |
|---|---|---|---|---|---|---|---|---|---|
| A1 | positive-reply | 0.1% → 0.64% (conservative (Sales.co "1 in 157" ≈ 0.64%/lead — see sources note)) | 0.54% | 21,928 [21,928–21,928] | 5,484 | 164 | 98.7 | 49.3 | 24.7 |
| A2 | positive-reply | 0.1% → 1.2% (mid (1.2%/lead)) | 1.1% | 9,260 [9,260–9,260] | 2,316 | 69.4 | 41.7 | 20.8 | 10.4 |
| A3 | positive-reply | 0.1% → 2.2% (healthy (2.2%/lead)) | 2.1% | 4,476 [4,476–4,476] | 1,120 | 33.6 | 20.1 | 10.1 | 5.0 |
| A4 | meeting | 0.05% → 0.48% (mid program × 40% booking (0.48%/lead)) | 0.43% | 24,776 [24,776–24,776] | 6,196 | 186 | 111 | 55.7 | 27.9 |
| A5 | meeting | 0.05% → 0.88% (healthy program × 40% booking (0.88%/lead)) | 0.83% | 11,648 [11,648–11,648] | 2,912 | 87.4 | 52.4 | 26.2 | 13.1 |
| Cell | Rung | Planned rates (holdout → proof) | τ | Rows n (w=τ, median [p25–p75]) | Rows n (w=2τ, bare) | 3K sends/mo | 5K sends/mo | 10K sends/mo | 20K sends/mo |
|---|---|---|---|---|---|---|---|---|---|
| B1 | positive-reply | 2% → 2.5% (1.25× lift on 2% base) | 0.5% | ≥250,000 [≥250,000–≥250,000] | 77,904 | ≥1,875 | ≥1,125 | ≥563 | ≥281 |
| B2 | positive-reply | 2% → 3% (1.5× lift on 2% base) | 1% | 86,496 [86,496–86,496] | 10,812 | 649 | 389 | 195 | 97.3 |
| B3 | positive-reply | 2% → 4% (2× lift on 2% base) | 2% | 12,944 [12,944–25,888] | 3,236 | 97.1 | 58.2 | 29.1 | 14.6 |
| B4 | meeting | 0.8% → 1% (1.25× lift on 0.8% base (2% reply × 40%)) | 0.2% | ≥250,000 [≥250,000–≥250,000] | 194,784 | ≥1,875 | ≥1,125 | ≥563 | ≥281 |
| B5 | meeting | 0.8% → 1.2% (1.5× lift on 0.8% base) | 0.4% | 216,360 [216,360–216,360] | 27,048 | 1,623 | 974 | 487 | 243 |
| B6 | meeting | 0.8% → 1.6% (2× lift on 0.8% base) | 0.8% | 64,848 [32,424–64,848] | 8,108 | 486 | 292 | 146 | 73.0 |
What is a firewalled proof cohort?
Before a program sends anything, RevenueOS randomly assigns a slice of its leads to a holdout that is never mailed and a proof cell whose arms rotate uniformly. The optimizer never touches either cell, so the comparison stays causal: whatever difference appears between proof and holdout was caused by the sending, not selected by the algorithm that benefits from looking good.
Why do null results publish?
A proof layer that only reports wins is an advertising layer. Program incrementality reads on a difference scale — its band closes against zero. Angle lift reads on a ratio scale — its band closes against 1.00×, the no-effect line. When a band never excludes its no-effect point, RevenueOS publishes the null with the same arithmetic — an agency rationing sends needs the nothings as much as the wins.
What actually happens between connecting an account and a published claim?
One send passes through five hands, chained on one spine, and every custodian is on the record: the system drafts a candidate and scores it, a person releases it or does not, the outcome lands back as a typed event, and the claim publishes in whatever state the evidence has earned. The same record id runs the length of the chain, so any hand can be reconstructed.
- 01
Connect
Plug into Smartlead, Instantly, or SendGrid, plus your CRM — HubSpot, Salesforce, or Copper. No parser goes live until it proves itself against captured events from real accounts.
- Record
- cand_7f3a
- Entry
- drafted · m. okonjo · halden logistics
- Custodian
- system · logged
- Logged
- wk 27 · tue 09:12
- 02
Rank
The engine scores candidate sends from live evidence — and shows its reasoning in plain numbers, not a black box with a confidence costume. Learned inference stays off the page until the loop has validated it.
- Record
- cand_7f3a
- Entry
- action confidence 0.67 · labeled
- Custodian
- system · logged
- Logged
- 09:13
- 03
Sign
A person chose thisA human approves, edits, or rejects. Sends leave through your own rails; we operate to the 2026 deliverability doctrine — under 50 per mailbox per day, complaints held under Google’s 0.1% line.
- Record
- snd_7f3a
- Entry
- released · a approve
- Custodian
- g. mahn
- Logged
- 09:14
- 04
Observe
Replies, positive replies, and meetings land back as typed outcomes — fast rungs first, so outcome events start accruing in weeks. A verdict takes the sample the design requires, and the minimum-n table on this page sizes it before you sign anything.
The reply itself stays in your sending tool’s inbox — Smartlead, Instantly. RevenueOS is send-side: the proof layer, not an inbox, and it reads the outcome, not the thread. On SendGrid it reads delivery and engagement events only — reply outcomes come from your sequencer or your CRM.
- Record
- snd_7f3a
- Entry
- positive.reply landed · rung 3
- Custodian
- system · logged
- Logged
- thu 11:02
- 05
Publish
Claims publish in the state the evidence has earned — validated, null, or still collecting — whether or not the band ever clears the gate. Nulls publish too.
- Record
- snd_7f3a
- Entry
- counted → clm_112 · still-collecting
- Custodian
- system · logged
- Logged
- wk 28
Disposition: entry closed — in the book
each entry above carries its own custodian and the minute it was written.
The full methodology — firewalled proof cohorts, intervals that stay honest when you peek daily, the minimum-n table above — ships with every pilot, in writing.
Where do the numbers come from?
The statistical machinery is the published literature on time-uniform confidence sequences and betting-based estimation; the market rates are dated 2026 benchmark publications, cited below with their disagreements noted rather than smoothed over. Every figure on this page carries its source, and the table’s generator script is named so the arithmetic can be recomputed rather than believed.
- Howard, Ramdas, McAuliffe & Sekhon — “Time-uniform, nonparametric, nonasymptotic confidence sequences”, Annals of Statistics 49(2), 2021
- Waudby-Smith & Ramdas — “Estimating means of bounded random variables by betting”, Journal of the Royal Statistical Society: Series B
- Instantly — Cold Email Benchmark Report 2026
- Prospeo — cold-email reply-rate study (live-verified 2026-07-03)
- Generator: backend/scripts/proof/minimum-n-table.ts — seeded, deterministic, 200 Monte-Carlo runs per grid point. The table above is drift-locked to its output.
RevenueOS — the proof layer for cold outbound · the first entry · the pilot, on one page