Insights · Measurement

Incrementality Testing for Car Dealers: How to Run a Holdout and Prove Advertising Sells Cars

Nick Pascali · Sep 29, 2026 · 8 min read

Short answer: an incrementality test withholds a channel from a comparable group of ZIP codes, audiences, or time periods, runs it everywhere else, and compares sales between the two. The difference is the incremental effect of the media; media cost divided by that difference is cost per incremental sale. It is the only measurement that proves advertising caused sales rather than accompanied them, and it is now practical at dealership scale: Google runs incrementality experiments at a $5,000 minimum, Meta added incremental attribution to Ads Manager in 2025, and any channel that can be bought by ZIP (connected TV, direct mail, audio, display) can be tested by geography with registration or DMS data. A franchised store with a growth budget should be running one test at a time, all year.

Why holdouts and not attribution

Attribution assigns credit to touches that appear in a buyer's journey. It cannot tell you whether the buyer would have bought without the touch. Brand search is the classic case: it appears in nearly every journey and gets credit for nearly every sale, and the only way to learn how many of those sales it caused is to turn it off somewhere and see what happens. When advertisers do, the answer is usually a small fraction of the attributed number. The same problem, in the other direction, hides the value of reach channels like connected TV, which influence buyers who never click anything.

The Mazda case is instructive. A Meta campaign with a holdout audience and daily offline sales matched through the Conversions API reported 3.5 times the incremental return on ad spend of the standard approach and a 64% lower cost per incremental sale. Mazda did not spend more; it measured what the spend did and reallocated.

At the dealer level the published evidence is thin because the results are proprietary. Vendors publish attributed showroom visit counts with no control group, which is not lift. So the honest position is this: nobody can tell a store what connected TV or conquest social will return in its market. The store has to test it, and the test is not hard.

How to run a geographic holdout

1. Pick the channel and the question. Test one channel at a time. The question is always the same: how many incremental sales does this channel produce per dollar, in this market? Start with the largest growth line in the budget, because that is where a wrong answer costs the most.

2. Choose the outcome. Sales, not leads. For new vehicles, registrations by ZIP from S&P Global Mobility or Urban Science are the best outcome because they include the buyers who never became leads and the buyers who went to a competitor. If you cannot license registration data, use your own DMS sales by customer ZIP; it sees only your sales, but it is clean and it is free. Showroom visits from a location panel are an acceptable secondary outcome for reach channels, not a primary one.

3. Split the trade area. List every ZIP in the primary market area with its households, your trailing-twelve-month sales, and your share. Our Market Area Snapshot gives you the households and demographics by ZIP in a minute. Sort the ZIPs into pairs that look alike on sales history, share, income, and distance from the store, then assign one of each pair to test and one to control. Aim for the control group to hold at least 20 to 25% of the market's volume. Do not put your best ZIPs all on one side.

4. Size the test. The effect you are looking for has to be larger than the normal month-to-month noise in the control group. A working rule for a franchised store: the test needs enough volume that a 10% lift in the test ZIPs would be at least 25 sales over the test period. A store selling 150 units a month in a market where the test ZIPs hold 60% of sales gets there in about three months. Smaller stores need longer tests or larger lifts to read anything, and a store selling 40 units a month should test with a bigger group or pool the test with sister stores.

5. Run it long enough. Auto shoppers spend about 95 days in market, so a channel's effect keeps arriving for weeks after exposure. Run the media for at least 8 weeks and measure sales through at least 4 weeks after it ends. A 12-week window is a sensible default. Keep every other channel steady in both groups for the duration; if the store runs a tent sale in half the ZIPs mid-test, the test is over.

6. Read the result. Compare test-group sales against control-group sales, both indexed to the same pre-period, so the comparison is change against change rather than level against level. If test ZIPs rose 9% against their pre-period and control ZIPs rose 2%, the lift is about 7 points. Apply that to the test group's baseline sales to get incremental units, and divide media cost by incremental units for cost per incremental sale. Synthetic-control methods do the same thing with a weighted blend of control ZIPs and produce a tighter estimate; Haus, which partners with MNTN on connected-TV testing, reports precision up to four times better than a naive matched-market comparison.

7. Compare against gross. Presidio-NCM's Q2 2026 average is $1,840 front gross on a new unit and $1,769 in F&I, about $3,600 together. A channel delivering incremental units at $2,000 is earning its budget; at $4,500 it is not, and the money should move. Remember the fixed-ops value of a retained customer when the number is close.

What to do with the number

A cost per incremental sale turns budget conversations into arithmetic. If connected TV produces incremental units at $2,100 and conquest social at $3,900, the next dollar goes to CTV until its cost rises, and social gets rebuilt or cut. If a channel cannot be tested, it stays at maintenance level. Over a year, testing one channel per quarter gives a store a priced growth stack instead of a mix inherited from the last agency.

The number also changes the conversation with the dealer principal. "We spent $42,000 on CTV and got 4.2 million impressions" is a report about the media. "We spent $42,000 on CTV, the test ZIPs outsold the control ZIPs by 21 units against their baseline, and each incremental unit cost $2,000 against $3,600 in gross" is a report about the business. The second one gets the budget renewed.

Two examples from our own work

A Volkswagen store measured an 80% relative lift in market share during a connected-TV program, with cost per measured showroom visit falling as the program matured. A luxury store in the same group on the same program gained showroom traffic but no share. Same media, different answer, and the difference told us where to look: the luxury store had a conversion problem on the floor, not a reach problem in the market. Without a share measurement, both stores would have received the same "traffic is up" report, and the wrong one would have gotten more budget.

The second lesson from that program was the gap it exposed. The Volkswagen store had 319 measured showroom visits that had never been matched against its sales file, so the share lift could not yet be tied to a close rate. The next step in that engagement is a ZIP-level holdout so the share result can be read as causal, and a match-back so the visits become a close rate. That is the sequence we recommend to any store: measure share, test with a holdout, match visits to sales.

Platform tests: Google and Meta

Geographic holdouts work for any channel. For the two largest digital platforms there are built-in options. Google's incrementality experiments, now available at a $5,000 minimum spend, split traffic or geography inside the account and report the conversion lift, and they accept offline conversions from the DMS. Meta's conversion-lift and incremental attribution tools hold out a random audience slice and compare it against the exposed group, and they work best when the store feeds sales back through the Conversions API. Both are worth running on brand search and retargeting specifically, because those are the lines most likely to be taking credit for sales the store already had.

What can go wrong

The test group and control group are not alike, and the difference, not the media, explains the result. Something else changes mid-test: a factory incentive, a competitor's grand opening, a weather event. The window is too short to catch a 95-day shopping cycle. The outcome is leads, and the test measures form-fills instead of cars. Or the store cannot bring itself to withhold media from a quarter of its market for twelve weeks, so the test never runs. The last one is the most common. The cost of the holdout is a few sales in the control ZIPs. The cost of never testing is a budget allocated on habit, forever.

Questions dealers ask

What is incrementality testing in automotive advertising?

A controlled comparison between a group that receives a channel and a matched group that does not, measured on sales or registrations. The difference is the incremental effect of the channel. It answers the question attribution cannot: would these sales have happened without the media.

How long should a dealership run a holdout test?

At least 8 weeks of media plus 4 weeks of post-period measurement, because the average buyer spends about 95 days in market. Twelve weeks is a practical default; shorter tests undercount reach channels.

Can a small dealership run an incrementality test?

Yes, with a larger test group or a longer window, or by pooling with sister stores under the same owner. Google's experiments now run at $5,000 minimum spend. The limiting factor is sales volume, not budget: the test needs enough units for a real lift to stand out from monthly noise.

What is a good cost per incremental sale for a car dealership?

One that is comfortably below gross per unit. With average front and back gross around $3,600 on a new unit in Q2 2026 (Presidio-NCM), a growth channel delivering incremental units below about $2,500 is working, and one above gross is not. The right target depends on the store's own gross and its share goal.

Sources

Next step

Let's put your market on the board.

Tell us about your store and your market. We'll come back with a modeled market share projection and a custom strategy to grow your local share, before a dollar is spent.

Start Your 90-Day Pilot

More insights

Budget