Told $2k is the minimum to test native properly. Is that a real number?

Got on a call with a guy who runs native at what sounds like real volume and he told me flat out that under $2k you're not testing, you're guessing.

That's more than I have. I was planning to put $400 across two widget sets and see what happens.

Is $2k a real number or is it the number people say so you don't come back and tell them their advice didn't work? Genuinely asking, I'm not being sour about it.

It's a real number but it's the wrong framing, and the caveat matters more than the figure.

$2k is roughly what it costs to get to a decision you can trust on native, because the unit isn't dollars, it's conversions. You need enough conversions per widget to tell a 1.2% CTR from a 0.9% one, and on native at typical CPCs that lands somewhere around $1.5-2k before you can say anything without lying to yourself.

What $400 buys you is a bin test - which widgets are alive at all, which creatives get any click. That's genuinely useful and it's not guessing. It just isn't the same question.

ok that reframe actually helps a lot, thank you

Numbers from my last four native tests, all same vertical, 2024:

Spend to first read	Widgets cut	Kept	Verdict held?
$380	41 of 60	19	yes
$1,640	6 of 19	13	yes
$2,900	4 of 13	9	no, 2 came back
$410	38 of 55	17	yes

The cheap passes are the reliable ones because they're answering an easy question. The expensive pass is where I've been wrong most often, which is the opposite of what you'd expect and I don't have a good explanation for it.

Anyway 400 is fine for start. Nobody spend 2k on first test, this is story people tell.

The two-thousand figure comes from a real calculation that everyone repeats without doing, so let me do it, because once you've seen it you can work out your own number instead of borrowing someone else's.

The unit is conversions, not dollars

You are trying to detect a difference in conversion rate between two things. The sample size you need depends on the baseline rate and the size of the difference you care about. Roughly, to detect a 20% relative lift on a 2% baseline at any confidence worth acting on, you need somewhere in the region of 250-350 conversions per arm. That is the number. Everything else is arithmetic.

Turning conversions into spend

Take your payout, your expected conversion rate and your CPC. If you're paying $0.35 a click and converting at 2%, one conversion costs you $17.50 in traffic. Three hundred of them is $5,250 per arm, and now the two-thousand-dollar advice looks generous rather than gatekeeping.

Except you are almost never testing to that standard, and you shouldn't be, because:

There are three different tests and people conflate them

1. Is anything alive? Do any of these widgets serve, do any of these creatives get clicked. Twenty to forty clicks per widget answers this. A few hundred dollars total. This is what $400 buys and it is a real test.
2. Which half do I cut? Ranking widgets or creatives against each other to drop the bottom half. You need far less certainty to cut than to scale, because the cost of wrongly cutting a mediocre widget is small. Somewhere around 1-3 conversions per unit is enough for a first cull.
3. Is this profitable enough to scale? This is the one that costs $2k and up, and it's the only one where the statistics actually bind, because now a wrong answer costs you a five-figure scale-up.

The guy on your call was answering question three. You asked question one.

Where I'd actually spend $400

All of it on step one, on as few widgets as you can stand, with one creative concept in three variants. Do not spread $400 across sixty widgets — you'll get four clicks each and learn nothing at all, which is the actual failure mode here, not the budget size.

I have not run native below about $300 in a single test so I can't tell you where the floor is. It exists somewhere below where I've looked.

Summary: $400 answers whether it's alive, $2k answers whether it's worth scaling, and spending $400 as though it were $2k is how the money gets wasted.

IME the thing that actually burns the small test budget isn't the budget, it's the tracking being wrong for the first two days and nobody noticing.

Fire a test conversion through the whole chain before you spend a cent. I've watched people write off a vertical on data that was double-counting.

Worth adding: budget per day matters more than total. $400 over four days beats $400 over twenty.