AI Gross Margin: Why Yours Isn't 80%

If you run an AI company, plan on a gross margin somewhere around 50 to 60% in 2026, not the 80 to 90% Bessemer cites for classic SaaS.¹ ICONIQ's survey has the average AI product at 45% in 2025 and 53% projected for 2026.² The reason is simple: every query your customer runs costs you real compute.

I get asked about this a lot, because AI shows up in all four industries I go real deep in: technology (AI and SaaS), healthcare and biotech, pharma, and pharmacy. A founder builds a board deck, the margin line says 85%, and it looks great. Then someone on the board asks what is sitting inside cost of revenue, and the answer is hosting and payment fees. That is when the number falls apart, usually in the room, which is the worst place for it. By the end of this you should be able to explain the why behind your margin to your board in a couple of sentences.

What gross margin should my AI company actually have in 2026?

Gross margin is the share of each revenue dollar left after the direct costs of delivering your product. Gross profit is "the accounting result obtained after deducting the cost of goods sold and sales returns/allowances from total sales revenue," and you divide that by revenue to get the percentage.³ Everything interesting is in what you count as a direct cost.

The benchmarks do not all agree, so look at who said what. Bessemer's February 2026 pricing playbook puts AI companies at 50 to 60% and SaaS at 80 to 90%, and it does not give a sample or a method.¹ ICONIQ's survey of software companies building AI products is the better dataset, and it starts lower: 45% in 2025, projected to reach 53% in 2026.² Its January snapshot had the 2026 figure at about 52%.⁴

The application layer, meaning companies building products on top of someone else's model, runs lower still. In the ICONIQ data as Upstarts Media reported it in January, application-layer products ran 38% in 2025 with 45% projected for 2026.⁵ Bessemer's State of AI 2025 found the same spread from a different angle: the ten fast growing Supernovas it surveyed averaged only 25% gross margins, while its Shooting Stars ran about 60%.⁶ That is a small, hand picked group, so I read it as a picture of the range.

And before you beat yourself up over 80%, classic software does not hit it as cleanly as the headline suggests. Jamin Ball's public software set showed a median gross margin of 76% in February 2026,⁷ and Tomasz Tunguz found a 76.1% median across 69 public B2B software companies in January.⁸ So the honest comparison, back of the napkin, is an AI company somewhere between 45 and 60% against classic software at roughly 75 to 90%, depending on whose set you read.

Why does every AI query show up in cost of revenue?

Because your customer caused it. In classic SaaS, serving one more customer costs almost nothing. In an AI product, every answer, every summary and every agent step runs a model, and somebody sends you a bill for that. Bessemer's playbook says it plainly: for AI companies, cost of goods sold matters again.¹

You can see how light the old cost base was in SaaS Capital's 2026 survey of more than 1,000 private B2B SaaS companies. The median company spent 5% of ARR on hosting, 4% on DevOps, 5% on professional services cost of revenue and 3% on other cost of revenue.⁹ Add those medians up and you get about 17%, so roughly an 83% margin. That is my arithmetic, not a figure SaaS Capital publishes.

Datadog's first quarter of 2025 shows cost of revenue of $157.6 million against $761.6 million of revenue, a 79.3% gross margin by my math.¹⁰ A platform that runs on someone else's compute looks different: Snowflake's 10-Q for the quarter ended April 30, 2025 says approximately 68% of its cost of product revenue was third-party cloud infrastructure.¹¹ An AI product adds a second layer, because the model provider bills you for every call.

The rules push in the same direction. Regulation S-X tells public companies to state the cost of tangible goods sold and the cost of services separately from other expenses.¹² SEC staff also push back when gross profit leaves out costs of revenue. One example comment in Deloitte's roadmap reads: "You present cost of sales exclusive of amortization expense and a subtotal for gross profit."¹³ It was about amortization, not AI, but the principle carries over.

Here is how I think about it, and it is the same test I use on any line: if a cost exists only because a customer used the product, it belongs above the gross profit line. Model API fees and the GPUs or cloud instances serving customers are cost of revenue in my book, and compute used to train or improve a model is R&D. KPMG points out that infrastructure costs fall under several different standards, including Topic 360 for purchased equipment and Topic 842 for identified leases, depending on the arrangement.¹⁴ So model it and run it past your CPA before the number goes into a board deck or a data room.

Which three lines do I pull from my P&L to find my real margin?

Four lines carry most of the difference between a SaaS margin and an AI margin, and three of them usually sit somewhere else on the P&L. Here is each one side by side.

Nothing in the SaaS column moves with how much a customer uses the product on a given day. Everything in the AI column does, so the same product can make money on one customer and lose it on the next. Let me walk through a quick example with illustrative numbers (not a client). An AI company books $100,000 of revenue in a month. The books show $12,000 of cost of revenue: $9,000 of cloud hosting and $3,000 of payment fees. That is an 88% gross margin, and on the board deck it looks like a SaaS company.

Now go find the three lines that live somewhere else. There is $24,000 of model API fees paid to providers, coded as software subscriptions. There is $6,000 of GPU rental for a fine tuned model that serves customers, coded as R&D because the engineers set it up. And there is $4,000 of the support team's time spent fixing wrong answers, sitting in operating expenses.

Pull those three up and cost of revenue is $46,000, gross profit is $54,000 and the gross margin is 54%. Same company, same month, a 34 point difference. The 54% is the one a board or a buyer will accept, because nobody can poke a hole in it. Here are the 3 cost lines to pull from your P&L to find your true AI margin:

1. Model provider and inference fees. Every dollar paid to an API provider to answer your customers, wherever the invoice got coded.
2. GPU and cloud compute that serves customers. Instances and reserved capacity running production, kept separate from the compute you use to train or test.
3. People who deliver or correct the product. Support, human review, labeling and implementation time tied to customer usage.

Getting the timing right matters as much as getting the line right. Model providers bill after the usage, so the month's inference cost has to be accrued from your usage console before the invoice shows up. I worked closely with a pharmaceutical client as an outside contractor on its accounting team, providing accrual accounting and supporting the monthly close as the company scaled toward a public IPO. The accrual workpapers alone ran 40 hours or more in the first week of every month.

The lesson carries straight over to AI: a cost you owe before you have been billed still belongs in the month you incurred it. Skip it and your margin jumps around with the invoice dates. Our guide to reading a SaaS P&L covers where each of these lines should sit, and the month-end close is where the accrual actually gets booked.

What does my board read into a 50% margin, and when should I worry?

Most investors read a 50% margin as a price you are paying for growth, not a verdict on the business. Bessemer describes its 25% margin group as fast growing companies trading distribution for profit in the short term.⁶ Bain Capital Ventures goes further: "For AI apps, gross margin today is not indicative of terminal gross margin."¹⁵ Tunguz notes that publicly traded software companies have gross margins of 71-72% and AI companies run lower, and he watches gross profit per token instead.¹⁶

So the number by itself rarely sinks a raise. What sinks it is a number you cannot explain. When a founder can walk the board line by line through cost of revenue, the conversation turns to how the margin gets better. Four signs tell me a low margin has become a real warning:

1. Margin falls as revenue grows. A young AI company at 45% should be moving toward the 53% ICONIQ projects,² not away from it.
2. Heavy users cost more than they pay. GitHub Copilot launched at $10 a month and, per a Wall Street Journal report relayed by Tom's Hardware, was losing more than $20 per user per month on average in early 2023, with some users costing as much as $80.¹⁷ That came from one insider, not a company disclosure.
3. Cheaper tokens do not lower your bill. As Jason Lemkin of SaaStr puts it, "Per-token costs are falling. But total costs per task are rising."¹⁸
4. You cannot break the lines out. If a diligence team asks for inference cost by customer and it takes you a week to answer, the margin is a guess.

Our piece on the SaaS metrics investors ask for before a raise shows where gross margin sits, and unit economics shows how a weak margin stretches out CAC payback.

Has anyone public actually reset their margin over AI costs?

Yes, and the cleanest record I found is Duolingo's, because it wrote the numbers down in shareholder letters filed with the SEC. It is a consumer app, not a startup, but the pattern is worth studying. In its fourth quarter 2024 letter, Duolingo said its "gross margin decreased by approximately 120 basis points year over year to 71.9% due to lower subscription margins from increased generative AI costs."¹⁹ In the first quarter of 2025 the total was 71.1% against 73.0% a year earlier, and the letter pointed to "increased generative AI costs related to the expansion of our Duolingo Max tier."²⁰

Then it made a call, on purpose, and said so ahead of time. In February 2026 it guided to "approximately 71% in Q1 and roughly 69% for the rest of the year, driven primarily by expanding access to AI-powered features for all users."²¹ The first quarter of 2026 then came in at 73.0%: "Gross margin expanded 190 basis points year over year to 73.0%, driven primarily by continued reductions in per-unit AI costs."²²

Duolingo started above 70%, so a reset to the high 60s left it in healthy territory, and a startup sitting at 45% does not have that cushion. And every step came with the margin, the driver and the guide, in numbers a board could check. That is the standard I would hold your finance team to, and it starts with a clean monthly close.

How do I get my AI margin moving up?

Four levers do most of the work, and the first one is only partly yours to pull. The price of a given level of model performance keeps falling. Epoch AI's September 22, 2026 analysis finds "the price for a given level of performance has fallen about 47% per quarter, or 13× per year."²³ Andreessen Horowitz measured a 10 times annual decline in 2024.²⁴

The catch is what your customers want. As one analysis puts it, "the price of top-end models has stayed steady or even gone up."²⁵ If your product needs the frontier model to be any good, falling prices on older models will not bail you out.

The second lever is routing. ICONIQ reports that many companies send most of their workloads to smaller or fine tuned models and only escalate the high complexity tasks to frontier models, and it ties that directly to margin.⁴ The third is caching. Anthropic's documentation says "Cache read tokens are 0.1 times the base input tokens price," which matters most when your product sends the same long instructions or documents with every request.²⁶

The fourth lever is how you price. Flat plans break when one customer uses 10 times what another does. ICONIQ found consumption based pricing rose from 35% to 42% of companies in six months, and outcome based pricing from 18% to 23%.² Cursor made that change in public in July 2025, explaining that "new models can spend more tokens per request on longer-horizon tasks."²⁷ Whichever levers you pick, put a number and an owner next to each one.

If you run a biotech, pharma, pharmacy or AI company, what changes?

For an AI or technology company, pull the three lines, accrue the inference bill every month, and be ready to show margin by customer, not just in total. Our technology practice builds that view for AI and SaaS founders.

For a healthcare or biotech company that sells software or an AI service, the extra line is people. If clinicians, pharmacists or compliance staff review model output before it reaches a customer, that time is a cost of delivering the product. It belongs in cost of revenue, even though the salaries sit in a clinical or quality department.

For a pharmacy adding an AI tool, keep the tool's cost on its own line. A pharmacy's gross margin is dispensing revenue less the cost of the drugs, and lenders and buyers read it that way. Bury a software cost inside it and you muddy the one number they trust most. In every case the fix is a chart of accounts that keeps the lines separate and a close that ties each one to a document.