Glossary · revised 2026-07-13· defined with the product’s math

Minimum detectable effect

The minimum detectable effect (MDE) is the smallest true effect a given experimental design can reliably distinguish from zero at its planned sample size and confidence level. It is the budget line of experimentation: an effect below the MDE can be real and still produce a null read, so a null result from an underpowered design is uninformative, not exonerating. Honest testing states its MDE before the first send.

The budget line of experimentation

Every design has a smallest effect it can see. Fix the sample size and the base rate and the MDE falls out: true effects larger than it surface reliably, while effects smaller than it mostly read as noise. Run the relation the other way and it becomes a price list — name the lift you need to detect, and the same arithmetic quotes the rows it costs.

In cold outbound the quotes are steep, because base rates are small and the absolute gap a lift creates — τ, the base rate times the lift minus one — is smaller still. On a 2% positive-reply base, a 1.25× lift is a gap of τ = 0.5%, and resolving it takes at least 250,000 randomized rows; that floor never crossed inside the simulated grid’s 250,000-row cap. A 1.5× lift on the same base needs 86,496 rows, and a 2× lift needs 12,944 — all at the table’s primary precision standard (band width w = τ), with rows counting leads randomized into the measured cells, not emails sent. Small lifts on small bases cost quarters; big lifts on healthy bases cost weeks.

Below the line, a null is not an acquittal

The MDE is what gives a null result its meaning. If a design can only resolve a 2× lift and the true lift is smaller, the read will come back null most of the time — the design failing to see, not the effect failing to exist. Treating that null as evidence of no effect is the quiet failure mode of underpowered testing: the variant may be better, worse, or indistinguishable from the control, and the data cannot say which.

The discipline that follows is procedural rather than statistical: state the smallest effect the design can resolve before the first send, and decline to interpret a null on anything below it. An experiment that could not have resolved the question it was asked is not evidence about the answer — it is evidence about the budget.

From MDE curve to CI-width floor

In fixed-horizon testing the MDE comes from a power formula: choose a significance level and a power target, and a closed-form curve maps sample size to detectable effect. RevenueOS runs anytime-valid confidence sequences instead of fixed-horizon tests, so its planning object is different. The binding go/no-go is a CI-width floor — the number of rows at which the anytime-valid band at α = 0.1 becomes narrow enough to resolve the planned effect — computed by inverting the width of the shipped betting confidence sequence (the estimator of Waudby-Smith and Ramdas, 2024) with seeded Monte-Carlo simulation, 200 runs per grid point, rather than by a textbook formula.

An analytic MDE curve is still computed alongside the floor, but it is informational only; it never gates a decision. Two conventions carry into the published numbers: a “≥” entry means the floor never crossed inside the simulated grid, and the ×2-geometric search quantizes each floor upward by as much as 2×, so adjacent grid steps should be read as the uncertainty — the floors are conservative, not exact integers.

Panel B of the published minimum-n table — arm-lift floors at the w=τ precision target (α = 0.1, generated 2026-07-03).
Cell	Planned rates (holdout → proof)	τ	Rows n (w=τ, median [p25–p75])
B1	2% → 2.5% (1.25× lift on 2% base)	0.5%	≥250,000 [≥250,000–≥250,000]
B2	2% → 3% (1.5× lift on 2% base)	1%	86,496 [86,496–86,496]
B3	2% → 4% (2× lift on 2% base)	2%	12,944 [12,944–25,888]
B4	0.8% → 1% (1.25× lift on 0.8% base (2% reply × 40%))	0.2%	≥250,000 [≥250,000–≥250,000]
B5	0.8% → 1.2% (1.5× lift on 0.8% base)	0.4%	216,360 [216,360–216,360]
B6	0.8% → 1.6% (2× lift on 0.8% base)	0.8%	64,848 [32,424–64,848]

How RevenueOS uses this

RevenueOS publishes its floors before it sells against them. The pilot-kit minimum-n table (generated 2026-07-03; seeded and deterministic) states, per outcome rung and per planned effect, the rows at which the α = 0.1 anytime-valid band resolves the claim; Panel B above renders from the same source module the published table is drift-locked to, so this page cannot silently disagree with the document a partner reads. Those floors are why the pilot has a public qualification gate — 5,000+ monthly sends across 2+ programs. Below the floor, the honest answer is the audit, not the invoice: a prospect whose volume cannot resolve the planned claim is offered the running confidence band and the dated floor it is accruing toward, not a promised significance date.

Inside the product the MDE curve is deliberately demoted: it appears as informational context, while the CI-width floor is the only number that gates a go/no-go. RevenueOS is at design-partner stage (July 2026) and publishes design floors, not customer results.

Sources

the full statistical machinery, in writing: /methods · every term: /glossary