AI Compute Costs: COGS or R&D?

Cost of revenue is what it costs to deliver your product to a customer, so the compute a customer's request triggers goes there, and the compute you spend building or improving the model goes in R&D. Split a shared GPU bill by a usage driver you can show, write the policy down and follow it every month.¹²

Picture the board meeting. Your deck shows a 70% gross margin on $400,000 of revenue. A director asks where the training runs sit, and you say R&D. Then she asks to see the bill. I would want to poke a hole in your split before she does, because on the illustrative numbers below (not a client) the same $186,000 compute bill shows a 53.5% gross margin or a 70.0% one. I work with founders in the four industries I go real deep in: technology (AI and SaaS), healthcare and biotech, pharma, and pharmacy. By the end you will be able to split one month's compute bill, book the entries and defend the policy. I am not a CPA, so your auditor or accountant makes the final call on your classification.

Where does an AI compute bill go, cost of revenue or R&D?

Here is how I explain it over coffee. Cost of revenue is the cost of delivering what you sold. One practitioner guide on SaaS gross margin puts it plainly: "any cost incurred to deliver the product to a customer should be included in COGS."¹ R&D is the work you do to build or improve the product, and KPMG's handbook says R&D costs in the scope of ASC Subtopic 730-10 are expensed as incurred.³ So the line between the two is not about the machine or the vendor. It is about what the spend is doing for you in that month.

Now the part that surprises founders: no rule tells you where a GPU goes. Deloitte's overview of ASC 705, the cost of sales topic, says the Topic only provides links to guidance in other Subtopics.⁴ For companies that file with the SEC, Regulation S-X Rule 5-03 tells them to state the cost of tangible goods sold separately, and the text I read says nothing about compute.⁵ Two companies can read the same bill and book it differently, and both can be within the rules. That is why I tell founders the policy is yours to set. The job is to set it, write it down and follow it in every period.

The test I use is the customer. CloudZero, a cloud cost vendor, puts it this way: "when product usage triggers the cost, it belongs in cost of revenue, exactly like hosting."² Its flip side is the one I find most useful: when your own people trigger the cost, through internal tools and experiments, it is not serving a customer, so it is not cost of revenue. That is a vendor's framework, not a standard, so I treat it as a starting point.

I learned why this matters at a specialty pharmacy that a private equity group had laughed out of the room because its books were a mess. My first job was to go three years backwards and close the books under GAAP so an audit could be done, and it took about 9 months. What slowed us down was not the math. It was costs sitting in places nobody could explain. A classification you cannot explain is one you will redo, and a redo costs more than the original decision.

Which compute costs go in which bucket?

Short answer: What a customer's request triggers is cost of revenue. What your own team triggers to build or test is R&D.

Start with the table. It is my policy for a typical AI startup, and your auditor may adjust it.

Training is the row founders argue about. Deloitte's piece on developing generative AI software products says training intended to keep an AI application current would most likely be considered maintenance, while training that creates new functionality might be an upgrade or enhancement.⁶ It adds that, "Unless the costs are separately capitalizable as an intangible asset, such costs would be expensed as incurred."⁶ A second Deloitte piece, from August 2026, says fees paid to third parties for the use of AI do not automatically create a new category of capitalizable cost.⁷

My take for an early-stage company is to expense training as incurred and put it in R&D. Capitalizing it needs a documented policy and a judgment about which stage the work is in, and most startups do not have either. FASB's ASU 2025-06 removes the old development stages from the internal-use software guidance for annual periods beginning after December 15, 2027, so your auditor will have a view.⁸

The bill itself helps you. Model APIs charge per token, and the price lists are public. OpenAI lists GPT-5.6 Sol at $5.00 per 1 million input tokens and $30.00 per 1 million output tokens, and Anthropic lists Claude Sonnet 5.5 at $2 and $10, both with a 50% batch discount.⁹¹⁰ Because the charge follows usage, you can pull usage and cost data programmatically and tie the bill to the workloads behind it.¹¹ Those are list prices as I read them on October 7, 2026.

How do I split a shared GPU bill?

Short answer: Book it to a clearing account, then split it on a driver you can show, like GPU hours by job tag.

Do it with a driver, not a guess. Say you reserve one cluster and it serves customers in the day and runs experiments at night. On Google Cloud, a single 8-GPU H100 machine (a3-highgpu-8g) lists at $88.49 an hour on demand, which is about $64,600 for a 730 hour month, my arithmetic from that listed rate.¹² One machine, one invoice line, two uses. I book that line to a clearing account and then allocate it, because the invoice cannot do the split for you. Microsoft says cost allocation does not affect your billing invoice.¹³

Pick the driver you can evidence. GPU hours by job tag is the cleanest. Tokens served works when one model serves everyone. AWS lets you activate cost allocation tags and group costs with Cost Categories.¹⁴¹⁵ The FinOps Foundation defines allocation as apportioning costs to those responsible for each component, and notes that tagging and labeling is possible for many cloud services.¹⁶¹⁷ The rule I hold myself to is simple: tag every job from day one, and untagged spend defaults to cost of revenue until someone proves otherwise. It is the conservative call, and nobody has to defend it. Then write down the driver, apply the same driver every month and keep the report that supports it.

What do the journal entries look like for one month?

Short answer: Two entries. One accrues the bill and parks shared compute in a clearing account, and one allocates it.

Here is the back of the napkin version, on illustrative numbers (not a client). An AI startup bills $400,000 in the month and its compute and platform bill comes to $186,000. It is month end and the invoices have not arrived, so I would accrue from the usage consoles. The bill has 6 pieces:

1. $62,000 of model API fees on customer requests.
2. $38,000 of self-hosted GPUs serving production inference.
3. $8,000 of vector database and monitoring for the live product.
4. $44,000 of training and fine-tuning runs.
5. $14,000 of evaluations and experiments.
6. $20,000 for one shared cluster, where 6,000 of its 10,000 GPU hours served production.

The first entry accrues the bill and parks the shared cluster in a clearing account:

1. Debit cost of revenue, model API fees, $62,000.
2. Debit cost of revenue, hosting and inference, $46,000 (the $38,000 of production GPUs plus the $8,000 of monitoring and storage).
3. Debit R&D, model training and evaluation compute, $58,000 (the $44,000 plus the $14,000).
4. Debit cloud compute clearing, $20,000.
5. Credit accrued expenses, $186,000.

The second entry allocates the clearing account on the driver, which is 60% production (6,000 of 10,000 GPU hours):

1. Debit cost of revenue, hosting and inference, $12,000.
2. Debit R&D, model training and evaluation compute, $8,000.
3. Credit cloud compute clearing, $20,000.

Now tie it out. Cost of revenue is $62,000 plus $46,000 plus $12,000, which is $120,000. R&D is $58,000 plus $8,000, which is $66,000. Together that is the whole $186,000, and the clearing account is back to zero. That zero is the check I look at first. When the invoice lands, you clear accrued expenses against it and book any difference to the same accounts. If you have not set up accounts for this yet, I walk through the structure in how to set up a startup chart of accounts.

What does the split do to my gross margin?

Short answer: On the example below, 53.5% or 70.0%, depending only on where the bill lands.

This is the why behind the number. On $400,000 of revenue, put the whole $186,000 in cost of revenue and your gross margin is $214,000 divided by $400,000, or 53.5%. Use the split above and it is $280,000 divided by $400,000, or 70.0%. Push even more out, so that only the model API fees and the production GPUs count, and cost of revenue is $100,000 and the margin is 75.0%. Same bill, three stories, and the third is the one I would not defend, because the vector database and the shared cluster do serve customers.

I would rather show you 70% and be able to defend it than show 75% and spend the diligence call on a footnote. Benchmarks count differently too. Andreessen Horowitz wrote in 2020 that AI companies often have lower gross margins than software because of heavy cloud use, at 50 to 60% versus 60 to 80% or more for SaaS.¹⁸ Bessemer's State of AI 2025 says its AI Supernova companies average only about 25% gross margins, and its Shooting Star companies about 60%.¹⁹ Bessemer's piece on scaling an AI Supernova sums it up: "COGS (cost of goods sold) is the new CAC."²⁰ Those are survey and playbook numbers with different definitions, not a rule of thumb for your company.

Public companies show the same pressure. Duolingo's fourth quarter 2024 shareholder letter cites lower subscription margins from increased generative AI costs.²¹ In August 2025 Newcomer reported, citing sources familiar with the figures, that Cursor had negative gross margins, driven by compute bills it pays to Anthropic, while Sacra says that as of April 2026 its large-enterprise accounts are gross-margin positive and individual developer accounts remain loss-making.²²²³ The compute is real, and the classification decides which line shows it. If you want the longer version of this story, I wrote about it in why your AI gross margin is not 80%, and I explain how to read the whole income statement in how to read a SaaS P&L.

What will my auditor or an investor challenge?

Short answer: Consistency, evidence and motive.

Three things, in the order I see them. First, consistency. If your policy changes between periods, I would expect a question and a recast, and I tell founders to disclose any change in policy and show both versions for the periods affected. Second, evidence. A driver you cannot support with a tagged report is a guess, and a diligence team will treat it as one. Third, motive. Anybody can see that moving cost from cost of revenue to R&D lifts gross margin, so the move needs a reason that is not the margin.

That third one has a documented failure behind it. In October 2015 the SEC charged former executives of OCZ Technology Group, and alleged that the former CFO's accounting policies included "reclassifying costs of goods sold as research and development expenses without sufficient basis for doing so." The SEC also alleged a scheme to inflate OCZ's revenues and gross margins from 2010 to 2012.²⁴²⁵ Those are allegations, not findings, and OCZ made hardware, not AI. But it is the exact boundary this article is about.

Regulators are already asking AI companies about this. In March 2026 Doximity answered an SEC staff comment about AI infrastructure costs. It said its new AI product had raised its generative AI platform and inference costs, called them not material for the periods in question, and committed to enhanced disclosure about cost of revenue and gross profit starting with its 2026 Form 10-K.²⁶ You are not a public company, but I expect the same question to reach you through a lender or an investor.

Tax is a separate question. Section 174A and Rev. Proc. 2025-28 cover research expenditures, so keep your GAAP classification apart from tax treatment and ask your tax partner.²⁷

What changes if I run a biotech, pharma, pharmacy or AI company?

Short answer: The habit does not change. Building goes in R&D and delivering goes in cost of revenue.

For an AI company, nothing above changes. For a health tech or biotech company that uses AI to develop a product, costs of building go in R&D and costs of delivering what you sold go in cost of revenue. Your auditor decides, and the policy memo is yours.

The habit carries across all four industries, and I picked it up on the pharma side. Through a High Rock Accounting engagement, I served as the sole accountant on a clinical-stage biopharmaceutical client's month-end close, owning its preclinical, clinical, manufacturing and general accruals through its Nasdaq IPO. Classification there came down to the same two questions: what work was done this month, and what has not been billed yet. A compute bill is no different, and that is the line I use to explain it to a board.

If you only do 4 things after reading this, do these:

1. Write the policy. One page: what goes in cost of revenue, what goes in R&D and why.
2. Tag every job. Production, training and experiments get different tags from day one.
3. Accrue from the usage console. Do not wait for the invoice at month end.
4. Show both margins. Put the margin on your split next to the all-in number, so nobody finds the gap for you.