AI Inference Costs: How SaaS Can Protect Margins

Inference is eating AI software margins in plain sight.

ICONIQ’s 2026 State of AI snapshot (cited by SaaStr) puts a hard number on what many U.S. SaaS leaders have been feeling in their AWS bills: at scaling-stage AI companies, model inference averages 23% of total AI product costs—nearly as much as talent (26%), and more than infrastructure/cloud (17%). If you’re building AI-powered digital services in the United States, that 23% isn’t a rounding error. It’s a design constraint.

Here’s the part most teams miss: inference doesn’t get cheaper as you scale. Pre-launch companies see ~20% of costs go to inference; at scale it’s ~23%. Translation: usage growth and “better product” pressure often outpace efficiency gains. If your competitors are spending aggressively to produce faster answers, richer outputs, and more automation, you can’t just optimize your way out of the problem. You need a business model that can carry it.

This post is part of our series on How AI Is Powering Technology and Digital Services in the United States, and it’s focused on a question every U.S. software operator now has to answer: How will you pay for inference—without wrecking gross margin or slowing growth?

The 23% inference “tax” is now a baseline

Answer first: For many AI-first B2B products, you should plan for inference to consume roughly a quarter of AI product economics unless you redesign your architecture and pricing around it.

A lot of SaaS operators still treat inference like an early-stage experimentation line item. That mindset breaks the moment customers adopt AI features broadly: summaries get generated in bulk, agents run continuously, copilots expand into more workflows, and tokens spike.

What makes inference uniquely painful compared to traditional SaaS costs is that it’s both:

• Variable with usage (more customers + more workflows = more spend)
• Tightly coupled to perceived product quality (faster + smarter outputs generally cost more)

In classic SaaS, you can sometimes slow spend growth with multi-tenant efficiencies. With AI, the product is literally “doing work” every time a user asks for something. That’s closer to a cloud services business than it is to 2015-era SaaS.

A simple margin gut-check (use this in board decks)

If you want a quick way to explain the stakes internally, use a back-of-the-envelope model:

• Suppose you run a B2B AI SaaS product at $5M ARR.
• If inference is 23% of revenue (the benchmark SaaStr lands on as a practical baseline), that’s $1.15M/year in inference spend.

That $1.15M comes out of the same pool you need for:

• Sales hires and pipeline generation
• Customer success and support
• Security/compliance for enterprise deals
• Product and data investments

If your pricing doesn’t explicitly account for AI usage, you’ll feel “profitable” in the P&L until adoption kicks in—then margins fall off a cliff.

Why inference costs don’t fall with scale (and what does)

Answer first: Model prices may trend down, but product behavior pushes costs up: users ask more, agents run longer, and teams ship features that consume more tokens.

The SaaStr summary makes a key point: talent as a percentage of costs drops from pre-launch to scaling (32% → 26%), while inference rises slightly (20% → 23%). In practice, three forces keep inference stubbornly high:

1. Feature creep increases token consumption. “Just add citations,” “make it remember,” and “run this across the whole knowledge base” are all expensive.
2. Latency expectations tighten. Customers want near-instant responses. That can mean larger models, higher throughput, and more replicas.
3. Competitive pressure becomes an arms race. If another vendor ships an agent that completes the workflow with fewer clicks, customers notice—even if it’s costly to run.

Meanwhile, some things really do get more efficient:

• Internal dev tooling improves
• Teams learn what not to build
• AI assists replace certain roles or reduce hiring growth

That’s why headcount share can fall. But it doesn’t automatically solve inference economics. It just shifts where the bill shows up.

Five realistic ways U.S. SaaS companies pay for inference

Answer first: You’ll fund inference through a mix of headcount discipline, pricing changes, efficiency engineering, and—only for a subset—venture capital.

Jason Lemkin outlines five options. I agree with the list, but I’ll add the operator’s nuance: most companies need a blended approach, and the sequencing matters.

1) Smaller teams (headcount discipline that actually sticks)

If you’re building AI-powered digital services, you’re probably already doing this: keeping hiring flat while revenue grows. Shopify’s widely discussed headcount discipline is a clear example of the broader trend: AI features replace certain categories of manual work, which lets companies fund inference with avoided payroll.

What works in practice:

• Set a rule: every new AI feature must include an “hours saved” estimate and a plan for where that time goes (support, sales ops, onboarding).
• Keep the savings real: don’t “save hours” and then refill them with new busywork.

Hard truth: headcount discipline buys time, not a business model. Eventually, pricing still has to match usage.

2) Treat inference as the new marketing budget (but be honest about the bar)

Lemkin’s framing is sharp: if inference makes the product so good that it sells itself, you can spend less on traditional marketing and sales.

That’s real—for a narrow set of products:

• PLG motion
• Clear “wow” output
• Low implementation overhead
• Fast time-to-value

If your product requires heavy change management, security reviews, or deep integrations, inference won’t replace enterprise marketing and sales. It may reduce sales cycle friction, but it won’t eliminate it.

A more achievable version:

• Shift budget from top-of-funnel volume to activation and retention, where AI assistance lowers churn and increases expansion.

3) Better pricing (usage and outcomes are winning for a reason)

SaaStr notes a crucial data point: 37% of companies plan to change their AI pricing model in the next 12 months, and the pricing mix is shifting quickly—outcome-based pricing rose from 2% to 18% in six months, and usage-based from 19% to 35%.

That’s not a fad. It’s a correction.

If your COGS scale with tokens, but your revenue is flat per seat, you’ve built a margin trap. Usage-based pricing is the cleanest fix, but only if you implement it without customer revolt.

Here’s what I’ve found works:

• Bundle a baseline (so customers aren’t scared to try features)
• Add clear overage tiers (so finance teams can forecast)
• Align the meter to value (documents processed, tickets resolved, minutes saved)

Outcome-based pricing can be powerful, but it’s harder to instrument and defend. If you can’t measure the outcome cleanly, you’ll end up in discount negotiations.

4) Model routing and efficiency (table stakes, not a silver bullet)

Routing—sending most requests to cheaper models and escalating only hard cases—is one of the few universally good ideas in AI product engineering.

But teams overestimate how far it gets them, because:

• Users quickly expand usage when the feature is helpful
• New features (agents, tool-use, long context) increase cost per interaction

A practical “efficiency stack” that actually moves the needle:

1. Route by intent (classification step → cheapest viable model)
2. Constrain outputs (shorter answers, structured JSON, fewer retries)
3. Cache aggressively (especially for repeated prompts and shared contexts)
4. Use retrieval well (better RAG reduces hallucinations and reduces “try again” loops)
5. Measure cost per workflow (not cost per token) so product teams feel it

Snippet-worthy rule: If engineers don’t see inference cost next to latency and error rate, it won’t get fixed.

5) Venture funding (fine—if growth is elite)

Venture can subsidize inference while you race to distribution and category leadership. But Lemkin’s point stands: it only works if growth is exceptional.

If you’re not in the top decile of growth, “we’ll fund inference with VC” turns into deferred pain:

• You train customers to expect premium AI
• You delay pricing changes
• You end up forced into abrupt packaging changes later

Operators should treat venture-funded inference as a temporary go-to-market weapon, not the default operating plan.

A margin-first operating plan for AI-powered digital services

Answer first: To protect gross margin, tie AI product design to unit economics: measure cost per workflow, price to value, and enforce engineering guardrails.

If you want something concrete to bring to your leadership team next week, use this checklist.

Step 1: Define your “cost per outcome” metric

Examples that map to how customers think:

• Cost per support ticket resolved
• Cost per sales email drafted and approved
• Cost per contract reviewed
• Cost per onboarding completed

Cost per token is an engineering metric. Cost per outcome is a business metric.

Step 2: Create a pricing fence before customers abuse the system

Pricing fences aren’t evil; they’re clarity.

• Include monthly AI credits in each tier
• Charge for premium workflows (agents, bulk processing, long-context analysis)
• Offer enterprise plans with committed usage (discounted) and overages (transparent)

The goal is to avoid the worst-case scenario: your biggest customers become your least profitable.

Step 3: Engineer for “good enough” by default

Most tasks don’t need frontier models.

Design your product so:

• The default experience uses a smaller/cheaper model
• The user can request “high accuracy mode” for critical work
• The system auto-escalates only when confidence is low

This keeps quality high where it matters and prevents silent cost creep.

Step 4: Treat inference budget like cloud budget—owned and audited

The companies that win in U.S. SaaS don’t just optimize models. They operationalize cost control:

• Weekly cost reviews (not quarterly)
• Per-feature inference budgets
• Alerts on cost anomalies
• “Kill switches” for runaway agent loops

Where this lands for U.S. SaaS leaders in 2026

Inference is now a permanent line item in AI product strategy, not an experiment. The benchmark number—23%—is useful because it forces a decision: either you build and price for that reality, or you pretend it’s temporary and watch margins compress as adoption grows.

The bigger narrative for our series—How AI Is Powering Technology and Digital Services in the United States—is that AI is absolutely driving new products, faster service, and more automation. But it’s also pushing U.S. software companies to mature faster: pricing has to reflect usage, product design has to reflect unit economics, and engineering has to treat inference as a first-class reliability metric.

If you’re building or buying AI-powered SaaS this quarter, the forward-looking question to ask is simple: Which workflows will your customers run 10x more often once they trust the AI—and can your margins survive that success?