How to Verify a Seller's Growth Story: Four Checks, and the One That Killed Mine

A word on scope

Two sources, both free bulk downloads, neither requiring an API key. NOAA Storm Events, detail files 2019 through 2023, giving every recorded severe weather event with a county code, an event type and an estimated property damage figure. Census County Business Patterns, county files for 2021 and 2023, giving establishment counts by county and industry.

This is a null result, published as one. The hypothesis I set out to confirm does not survive its own test, and the article is the method rather than the finding. House practice is to publish tested-and-killed hypotheses, because a method that only appears when it produces a headline is not a method.

The honest limitation: this measures establishment counts, not revenue. An existing roofer could have doubled revenue without a single new business appearing anywhere in the data. What is killed here is the claim that storms create roofing businesses; whether storms raise revenue at existing ones is untested and not testable with public data. Full method at the end.

What growth story was I testing?

Somebody is going to tell you why the revenue went up. The seller will have a reason, the broker will repeat it, and it will be a good reason — the kind that makes sense the moment you hear it and that you stop examining for exactly that reason.

Here is the one I set out to confirm.

Big hailstorms make roofing companies grow. A storm comes through a county, insurance pays out, and every roof on every street gets replaced inside eighteen months. Roofers hire, buy trucks, and some crews spin out into new companies. Revenue at every roofing business in that county rises for two or three years.

Then it stops. The roofs are done.

A buyer arriving in year three sees a roofing business with a beautiful three-year trend and pays a multiple on trailing earnings that were an insurance event. Nobody lied to them. The financials are real. The growth is simply not repeatable and not coming back.

That is a genuinely useful thing for a buyer to know, if it is true. I had a prior result on file saying roofing growth roughly doubled in heavily damaged counties, with a flat placebo.

It is not true.

Check one: does the cause scale with the size of the place?

A big county has more of everything, including damage.

My first cut was the obvious one: counties with more than fifty million dollars of storm damage against counties with almost none.

Roofing ahead by four and a half points, roughly the shape the prior result described. Then I looked at the dentists: up 3.7 points on the same cut.

Dentists do not benefit from hail. So whatever was moving roofing was also moving dentistry, which meant it was not hail.

The explanation is boring and it is the most common error in this genre. A county with more buildings has more property to damage. A fifty-million-dollar threshold selects large counties almost as reliably as sorting by population, and large counties grew faster over that window in every industry.

There was a second problem. Only six counties cleared the threshold with enough roofing establishments to compute a growth rate. A single roofing company opening would have moved the result.

Both problems have the same fix: divide damage by the number of businesses in the county.

Check two: does something that shouldn't move stay still?

A control is a thing you would be embarrassed to see react.

On the corrected measure, the top ten percent of counties by damage per business against the bottom half — 249 high-exposure counties against 1,242 low.

Now the placebo is flat. Dead flat, to two decimal places. And roofing is ahead by 1.4 points.

This is where a lot of analysis stops. The design looks correct, the control behaves, and the treatment moves in the predicted direction.

Two things kept me going.

First, 1.4 points is not "roughly doubled." When a replication comes back much smaller than the original, the honest reading is that both numbers are measuring noise at different sample sizes. You have not found a smaller version of the same truth.

Second, a flat placebo only rules out one kind of error: a cause that hits everything. It does nothing about a cause that hits construction but not dentistry, in the same counties, at the same time. I could not think of one, so I went and looked instead of thinking.

Check three: what else do those places have in common?

This is the one that kills most growth stories.

I printed the list of high-exposure counties by state.

Texas 28. Mississippi 24. Tennessee 21. Iowa 20. Georgia 18. Missouri 18. Minnesota 12. Arkansas 11. Louisiana 11. Illinois 9. Nebraska 9. Oklahoma 8.

Sixty-one percent of the high-exposure counties are in the Sun Belt or Tornado Alley.

Which is not surprising — that is where severe convective weather happens. The measure is working correctly.

It is also a list of the states that absorbed the largest movement of people and businesses in America between 2021 and 2023.

So my storm measure was doing something else at the same time. It was quietly marking the counties in the part of the country everyone moved to.

Any test built on that variable will light up for anything that moved south. Construction moved south, because people moved south and needed houses. Dentistry moved south more slowly, because dental practices follow population with a longer lag and are less elastic to a building boom.

That asymmetry is exactly what produced a 1.4-point gap with a flat placebo. Not hail. Migration, hitting a construction trade harder than a health trade.

Check four: what happens when you compare like with like?

The fix is to stop comparing Texas to Vermont.

If the problem is that storm exposure encodes geography, hold geography still. I ran it again with every state as its own separate contest: a storm-hit county in Texas compared only against other Texas counties, Alabama against Alabama. Each state split at its own 75th percentile of damage per business against its own median, then the median of the state-level gaps.

Migration into Texas now affects the high and low groups equally, because both groups are in Texas.

Roofing collapses. The median gap is slightly negative and the effect appeared in 15 states out of 33 — a coin flip, which is what nothing looks like.

Note the placebos. Dentists at −0.22 with 22 of 45. Restaurants at −0.13 with 20 of 46. Both sit at approximately zero with about half the states positive, which is precisely what a null looks like in this test. That is how I know the method works rather than having broken something.

The entire national result was geography.

What survived the test?

Auto body shops. Up 1.76 points, holding in 25 of 40 states.

Twenty-five out of forty is 62% — not overwhelming, but meaningfully different from the coin flip roofing produced, and it survives the control that killed roofing.

The mechanism is obvious once the number appears. Hail destroys car panels, paintless dent repair is a genuine trade with genuine shops, and a serious hail event in a populated county produces thousands of damaged vehicles in an afternoon.

The size is right too. Real effects in establishment counts over two years are small. A result claiming roofing doubled should have prompted suspicion rather than satisfaction.

I am not writing that article yet. It deserves a hail-only exposure measure rather than hail mixed with wind and tornado, and more than one storm window. But it has the shape of a true thing: small, specific, mechanically sensible, and undisturbed by the obvious control.

Why does this keep happening?

Once you have seen this failure mode you start seeing it everywhere, because the pattern is always the same. You have a cause you want to test. You measure it. Your measure is correct. And your measure happens to also be a map of somewhere.

Storm damage per business is a real quantity, computed correctly. It is also, by accident, a map of the American South.

Anything that varies by region does this. Weather. Water rights. Union density. Housing cost. Licensing regimes. Winter. Each is a real thing worth studying, and each is also a label for a set of states that differ in a hundred other ways.

So the test becomes circular without anyone noticing. You ask whether storms grow roofers. What you actually asked was whether the South grew faster than the Northeast. It did, for reasons that have nothing to do with hail.

The tell is when the treatment group has a geography. Write down where the affected businesses are. If the list reads like a region rather than a scatter, the test has not been run yet.

The fix is cheap: comparing within states took about ten minutes once the data was loaded. The expensive part was being willing to run it after I already had a number I liked. That is the actual failure mode. Not incompetence. Momentum.

Why this matters more to a lender

A buyer gets one growth story to evaluate. A lender gets hundreds, and that changes what the mistake costs.

If a credit box treats trailing growth as a quality signal and some of that growth is a place rather than a business, the box is quietly concentrating — not by industry, which anyone would notice and which every policy already limits, but by geography and by vintage.

Picture a book of loans written in 2023, weighted toward businesses that showed strong 2022 growth. A meaningful share of that growth was migration into a handful of Southern metros. Those businesses look diversified on paper: by NAICS code, by borrower, by branch, by loan size.

They are not diversified by cause. The same thing was making all of them grow, and the same thing stopping is a correlated event across the book. It will not appear in any concentration report, because no report has a column for it.

This is the ordinary way concentration hides. Nobody writes down "we are long the Sun Belt." It accumulates one defensible credit decision at a time.

What does this mean for buyers and sellers?

For buyers, when a seller explains why revenue grew, run these seven.

1. Ask what else changed at the same time. Not what they did — what happened to them.
2. Ask whether the neighbours grew too. If every business in that county grew, the reason is the county, and you are buying an economy rather than a company.
3. Find the dentist. Name a business nearby that should not have benefitted. Check whether it grew anyway.
4. Ask if the cause is still running. Insurance money, a one-time contract, a competitor closing, a stimulus program. Those end, and the trailing average does not know it.
5. Ask for the year before the story starts. A growth story beginning exactly when the trailing window begins is a window, not a story.
6. Check whether customers are new or orders are bigger. One-time events usually show up as bigger orders from existing customers; durable growth usually shows up as more customers.
7. Ask what happens in that market over the next three years. If the answer requires another storm, another program or another boom, you are buying the last one.

For sellers, the same logic runs in reverse and it is worth getting ahead of. If a meaningful part of recent growth came from something external — a competitor closing, a one-off contract, a local boom — a competent buyer will find it, and finding it late costs more than disclosing it early. The stronger position is to separate the two yourself: here is the growth that came from the market, here is the growth that came from the business, and here is the evidence for the split. A seller who has already done that arrives with a credibility advantage that survives diligence.

Sources & method

Storms. NOAA National Centers for Environmental Information, Storm Events Database. Detail files 2019 through 2023, downloaded 9 August 2026 from www.ncei.noaa.gov/pub/data/swdi/stormevents/csvfiles/. County-level records only (CZ_TYPE = C). Property damage parsed from the coded DAMAGE_PROPERTY field. Exposure equals summed damage from hail, thunderstorm wind and tornado events during 2020 and 2021, aggregated to a five-digit county FIPS. 2,990 counties recorded non-zero damage.

Businesses. US Census Bureau, County Business Patterns, cbp21co.txt and cbp23co.txt. Free bulk files; note that the Census API itself now requires a key while the bulk files do not. Roofing contractors NAICS 238160, automotive body repair 811121, offices of dentists 621210, full-service restaurants 722511.

Filters. A county needs at least 5 establishments in the industry in 2021 and at least 200 total establishments, so a single opening cannot swing a percentage.

National test. Exposure normalised by total county establishments; top decile against bottom half. Growth computed on pooled totals within each group rather than as a median of county rates, because medians on small counts quantise badly.

Within-state test. Each state split at its own 75th percentile of damage per establishment against its own median. A state needs at least 8 usable counties, with at least 3 high and 4 low. Reported figure is the median of the state-level gaps, with the count of states in which the gap was positive.

Limitations. Establishment counts, not revenue — the effect that matters most to a buyer reading one company's P&L is invisible here. Two years is a short window. The exposure measure is blunt, mixing three event types and relying on damage estimates made by local officials, and damage reporting quality varies between weather forecast offices. State is a coarse control, since Texas contains both Houston and the Panhandle; metro or commuting-zone matching would be better. Auto body is unresolved rather than proven, and is reported because omitting it would be selective in the direction that makes the null look cleaner.

I spent a day on a story I liked and it did not survive. That is most days. It is published because the same four checks work on the growth story in the deal on your desk, and nobody else is going to run them for you.