When eCommerce Merchants Should Test For Incremental Sales

Incremental Sales Sales generated above the amount expected to occur without a promotion, campaign, or other intervention. Merchants should run incrementality tests whenever they need to know whether a marketing action creates net new demand or merely shifts existing purchases.

Testing is not always necessary. Small, low-cost promotions to highly loyal customers may be executed without formal holdouts if the operational cost of testing exceeds the expected benefit. However, when media spend is large, inventory commitments are significant, or fulfillment capacity is constrained, incrementality testing becomes a necessary discipline to avoid wasting budget and operational resources.

Signs You Need An Incrementality Test

Look for specific triggers that indicate testing will change decisions. These include large media spends, unexpected spikes in orders, repeated promotions on the same SKUs, disagreements between marketing and operations on the cause of demand changes, and strategic shifts like expanding into new channels or regions.

• High Media Investment: If paid channels represent a substantial portion of budget, test to confirm they produce net new sales.
• Inventory Risk: When promotions require bulk inventory buys or vendor commitments, verify sales are incremental before ramping supply.
• Capacity Constraints: If fulfillment labor or shipping capacity is tight, test to avoid being overwhelmed by non-incremental spikes.
• New Channel Or Geography: Pilot with holdouts to prevent cross-market contamination and understand true audience lift.

How To Prioritize Tests Across Campaigns

Not every campaign needs a full randomized trial. Prioritize tests where the decision impact is largest: big-ticket promotions, new customer acquisition efforts, and experiments that could change long-term strategy. Use lighter-weight approaches — matched-market or quasi-experimental models — for smaller campaigns. Maintain a testing roadmap that balances rigor with speed.

• Priority 1: High-cost media buys and national promotions affecting multiple fulfillment centers.
• Priority 2: New customer acquisition channels and pricing experiments with long-term LTV implications.
• Priority 3: Routine targeted offers where operational impact is small and testing cost exceeds value.

Design Considerations For Operational Teams

Coordinate with operations early. Define which SKUs and fulfillment centers are included, determine how differences in handling (e.g., gift wrapping, expedited shipping) will be applied, and establish reporting requirements for pick/pack/ship metrics. Ensure the control group is insulated from promotional inventory flows to avoid skewed warehouse KPIs.

• SKU-Level Visibility: Track lift at the SKU level to detect cannibalization and allocate stock correctly.
• Fulfillment Mapping: Map tests to specific distribution centers so labor planning matches expected incremental volume.
• Return Windows: Align test duration with typical return cycles to capture net sales effects.

Case Example: Black Friday Email Strategy

A mid-size merchant plans a Black Friday email blitz offering a sitewide coupon. Marketing expects a 30% uptick in orders; operations worry about picking capacity. The team holds out 15% of the email list and runs the campaign for early access a week before the main sale. Results show a 25% increase in orders among recipients, but SKU-level analysis reveals 40% of those orders cannibalized planned purchases from the core December window. After accounting for baseline shifts and incremental fulfilment costs, net incremental revenue is modest. The merchant uses this finding to modify future Black Friday tactics: staggered offers and targeted segments rather than sitewide discounts that pressure fulfillment without generating proportionate net revenue.

Tips For Integrating Incrementality Into Ongoing Planning

• Institutionalize Experiments: Build standard test templates for common campaign types so results are comparable over time.
• Share Results Cross-Functionally: Marketing, finance, and operations should review lift findings together to align spend and capacity decisions.
• Continuously Improve Baselines: Use historical tests to refine modeling approaches and reduce uncertainty in future estimates.

In short, the Incremental Sales question should be asked when marketing actions could meaningfully affect budget, inventory, or fulfillment. Prioritize tests where decisions matter most, design experiments with operations in mind, and use SKU-level lift data to translate marketing success into sustainable, profitable growth.