The Real Cost of AI

When does a subscription make sense, and when should a company bring AI inside?

I use AI every day.

Like many people, I pay a monthly subscription. I open ChatGPT, ask a question, upload a document, analyse an idea, write something, challenge something, perhaps ask another ten questions, and I rarely think about what all of this is actually costing computationally. The experience makes AI feel almost unlimited; I pay my subscription and AI is simply there.

But recently I started thinking about this from a company perspective, and I realised that the economics are quite different.

What happens when I am no longer asking AI a question?

What happens when I give AI a task?

What happens when 1,000 employees are using it?

What happens when an AI system reads 50,000 maintenance reports, reviews every incoming purchase order, monitors contracts, checks invoices, searches internal databases, prepares reports and interacts with other systems?

At that point, I am no longer really talking about ChatGPT; I am talking about compute, and compute has a cost.

That led me to a question that I think many executives are going to have to answer:

At what point does it make sense to stop buying AI as a service and start running some of it ourselves?

The answer turned out to be much more interesting than I expected.

First, a subscription and an AI service are not the same thing; this is probably the most important distinction to understand.

When I buy a personal ChatGPT subscription, I am buying access to an application. As of September 2026, ChatGPT Plus costs $20 per month. It provides higher usage limits and access to additional capabilities, but it does not include API usage. API billing is separate.

The same principle applies in business. ChatGPT Business currently starts at $20 per user per month when billed annually, or $25 on monthly billing for a standard seat. Those seats provide the interface, workspace, administration, security controls and included usage allowances. That model makes enormous sense when I am buying AI for people.

An employee wants help preparing a presentation. Someone wants to analyse a spreadsheet. A manager wants to summarise a report. I want to challenge a strategy or improve an email.

The human is driving the interaction.

But software does not usually consume AI that way. If I connect AI to an ERP system, a document repository, a customer portal or a maintenance database, there may be no human sitting in front of a chat window at all. My application calls an AI model through an API.

Now I am paying for consumption, and consumption is normally measured largely in tokens.

So what exactly is a token?

Tokens sound technical, but the concept is actually simple.

AI does not read text exactly as we read words. It breaks text into smaller pieces called tokens.

For English, OpenAI gives a useful approximation: one token is roughly four characters or about three quarters of a word. Around 1,500 English words therefore correspond to approximately 2,000 tokens, although the exact number varies by language, model and content.

There are several types of tokens. Input tokens are what I send to the model and Output tokens are what the model generates.

Cached tokens are pieces of previously processed context that can sometimes be reused more cheaply.

Reasoning models can also consume reasoning tokens while working through a problem.

That difference matters because input and output are normally priced differently.

As of 9 September 2026, for example, OpenAI lists GPT 5.6 Sol at $4 per million input tokens and $20 per million output tokens. GPT 5.6 Luna is dramatically cheaper at $0.20 per million input tokens and $1.20 per million output tokens. OpenAI states that the current Sol price is promotional through at least 21 November 2026.

Anthropic currently lists Claude Sonnet 5 at $2 per million input tokens and $10 per million output tokens.

Google lists Gemini 3.1 Pro Preview at $2 per million input tokens and $12 per million output tokens for prompts up to 200,000 tokens. Longer prompts cost more.

I am not presenting these figures to declare a winner. These models are not identical in capability, speed or behaviour.

I am showing them because the range is extraordinary.

The choice of model can change the cost of exactly the same workload by ten times, twenty times or even more.

That means the question “How expensive is AI?” does not really have an answer.

The correct question is: Which AI, doing what, how often?

A document can cost almost nothing. Let me make this tangible.

Suppose I have a 50 page contract containing around 25,000 English words.

That is approximately 33,000 input tokens.

Suppose I ask a model to analyse it and produce a 2,000 token assessment.

Using the current GPT 5.6 Sol API rates, that single pass costs approximately:

Input, 33,000 tokens: about $0.13

Output, 2,000 tokens: about $0.04

Total: around $0.17.

Using Claude Sonnet 5, the same token quantities would cost approximately $0.09.

Using GPT 5.6 Luna, approximately one cent.

Those calculations use published token prices and deliberately simple assumptions. They do not include additional tool calls or repeated reasoning passes.

This is where I had my first surprise. The raw cost of AI can be incredibly small.

If my company analyses 100 contracts per month, buying a server because I am worried about token costs would make very little economic sense.

But there is another side to this. A task is not necessarily one AI call, this is where things become more interesting.

Imagine that instead of saying: “Read this contract and summarise the risks.” I say:

“Review this contract, identify the obligations, compare them with our standard terms, check the referenced regulations, identify deviations, classify the risks, produce a management summary and prepare suggested amendments.”

That AI may have to perform several operations.

It may search documents.

It may retrieve company policies.

It may make several model calls.

It may call external tools.

It may examine the result of those tools.

It may discover that something is missing and try again.

It may carry its previous context into the next reasoning step.

This is what increasingly happens with AI agents.

MLCommons describes agentic inference as a sequence of dependent turns. A coding agent, for example, may inspect files, execute commands, observe errors, edit code and repeat the process. As the task progresses, previous context can travel with it.

OpenAI's own business rate card makes the same point very clearly. Its ChatGPT Work, Codex and Workspace Agent products consume tokens according to the actual work performed. OpenAI currently says that a typical GPT 5.6 Sol Codex task may consume roughly 5 to 30 credits, while a typical Workspace Agent run may consume roughly 5 to 25 credits. The company explicitly states that task cost varies with input size, output length, caching and complexity.

So the phrase “one AI task” tells me almost nothing about cost.

I need to know what happens inside that task.

Let us scale the problem.

Now imagine a company with 1,000 employees.

Suppose each employee makes 20 requests per working day to a corporate AI assistant.

Suppose each request sends approximately 4,000 input tokens, including instructions and retrieved company information, and produces 500 output tokens.

That means every working day the system processes approximately 80 million input tokens, 10 million output tokens

Using current public API prices, my approximate daily model cost would be:

GPT 5.6 Sol: $520

Claude Sonnet 5: $260

Gemini 3.1 Pro: $280

GPT 5.6 Luna: $28

Across 22 working days, the approximate monthly model cost becomes:

GPT 5.6 Sol: $11,440

Claude Sonnet 5: $5,720

Gemini 3.1 Pro: $6,160

GPT 5.6 Luna: $616

Again, these are not forecasts. They are arithmetic examples using clearly stated assumptions and current published prices. They exclude other possible charges for search, storage, databases, networking and tools.

But look at the difference.

The exact same workload could cost hundreds of dollars per month or more than $10,000 depending primarily on the model I choose.

That is why I believe AI architecture is becoming a financial management issue as much as a technology issue.

Now imagine 50,000 maintenance reports; here is another example that surprised me.

Suppose an industrial company has 50,000 maintenance reports and wants AI to classify them and extract risks.

I will assume each report contains about 1,000 English words, approximately 1,333 tokens, and that AI generates an average of 150 output tokens per report.

That gives me approximately 66.7 million input tokens and 7.5 million output tokens.

Using GPT 5.6 Sol, the total model cost would be approximately $417.

Using Claude Sonnet 5, around $208.

Using Gemini 3.1 Pro, around $223.

Using GPT 5.6 Luna, around $22.

Fifty thousand reports.

Potentially less than the cost of taking a few people to dinner.

This is why I would be very careful when somebody tells me that a company needs to build its own AI infrastructure simply because it has “a lot of data”.

Data volume is not enough. I need to understand frequency.

If I perform that analysis once, renting intelligence is extremely attractive.

If I process a similar volume every hour, every day, for years, the economics change completely.

Agents are where the bill can start becoming significant; consider 10,000 automated tasks per day.

I will assume, simply for illustration, that each task consumes 10,000 input tokens and 2,000 output tokens across its workflow.

That means 100 million input tokens and 20 million output tokens every day.

At today's rates, approximately:

GPT 5.6 Sol: $800 per day, or about $24,000 over 30 days

Claude Sonnet 5: $400 per day, or about $12,000 over 30 days

Gemini 3.1 Pro: $440 per day, or about $13,200 over 30 days

GPT 5.6 Luna: $44 per day, or about $1,320 over 30 days

Now I am interested.

At $1,320 a month, I probably still do not want to own infrastructure purely to save token costs.

At $24,000 every month, with a stable and predictable workload, I would absolutely start modelling alternatives.

But I would still not buy a GPU based on that figure alone.

Because owning the AI is not free; the expression “run the AI internally” sounds wonderfully simple. It is not.

First, I need a model that I can actually run.

Fortunately, this part of the market has changed dramatically.

OpenAI's gpt oss 120b open weight model can run on a single 80 GB GPU. Its smaller gpt oss 20b model can operate with as little as 16 GB of memory. The model weights are available under an Apache 2.0 licence.

Mistral has demonstrated another approach. Its 24 billion parameter Mistral Small 3.1 can run on a single RTX 4090 or a Mac with 32 GB of memory and is also released under Apache 2.0.

So yes, a company genuinely can operate useful generative AI on hardware it controls, but then I inherit responsibility for that AI.

I need hardware.

I need storage.

I need model serving software.

I need monitoring.

I need access control.

I need patching.

I need backups.

I need security.

I need evaluation.

I need someone who understands why performance suddenly changed after an update.

If the system matters operationally, I also need to think about resilience and redundant capacity.

If I want the AI to answer questions using company knowledge, I may need document ingestion, embeddings, retrieval, a vector database and the surrounding application architecture.

MLCommons makes an important point here. A modern retrieval augmented generation system is not simply one language model. It can involve document parsing, chunking, embedding, indexing, retrieval and multiple models working together.

That is an AI system.

The model is only one component.

Then there is electricity: an NVIDIA H100 SXM has 80 GB of GPU memory and a configurable thermal design power of up to 700 watts.

If that GPU operated continuously at its maximum rated power for 730 hours in a month, the GPU alone would consume approximately 511 kWh. That does not include the CPUs, memory, storage, networking or cooling supporting it.

This does not mean local AI is energy inefficient. Quite the opposite can be true when hardware is heavily utilised.

NVIDIA currently cites an April 2026 SemiAnalysis benchmark estimating inference on GPT OSS 120B using an H100 and vLLM at approximately $0.09 per million tokens at 66 tokens per second per user. That figure illustrates how cheap optimised self hosted inference can become when hardware is properly utilised.

But I would never compare that $0.09 directly with the price of GPT 5.6 Sol and conclude that local AI is forty times cheaper.

They are different models.

Their capabilities are different.

The workloads are different.

The benchmark assumptions are different, and my company's total cost is not simply the GPU inference cost.

That is exactly the kind of comparison that produces attractive PowerPoint slides and bad investment decisions.

Utilisation is the word I would watch.

A GPU that I rent through an API effectively disappears when I stop using it. A GPU I buy continues to exist whether anyone needs AI or not.

This is a fundamental economic difference.

Google Cloud currently lists a Flex Start A3 instance containing one H100 GPU, 26 virtual CPUs, 234 GB of memory and 750 GiB of local SSD at $4.79 per hour. Running that continuously for 730 hours would be roughly $3,497 for the month. I use this only as a current reference point for rented GPU capacity, not as a production hosting quotation.

If my workload needs a powerful GPU for three hours today and nothing tomorrow, an API is extremely efficient.

If my workload can keep that GPU productive throughout the day, every day, the mathematics becomes much more interesting.

This is one of the reasons AI serving technology focuses so heavily on batching, memory management and concurrency. Research behind the vLLM serving engine found that better memory management could improve serving throughput by two to four times under the evaluated workloads.

The economics of internal AI therefore depend heavily on how effectively I can use the equipment I own.

An idle GPU is not cheap AI. It is expensive furniture.

What about confidentiality?

This is another area where I think the discussion sometimes becomes too simplistic.

I often hear: “We cannot use cloud AI because our data is confidential.” Sometimes that is absolutely correct.

There are environments where information simply cannot leave controlled infrastructure. Defence, highly sensitive intellectual property, certain regulated workloads, isolated industrial networks and air gapped environments can create a very strong case for internal AI.

But confidentiality does not automatically mean self hosting. Enterprise AI providers have changed considerably.

OpenAI states that business and API customer data is not used to train its models by default, and it now offers regional data residency options for eligible customers, including in the United Arab Emirates.

Microsoft similarly states that prompts, completions, embeddings and training data used with models sold through Azure Foundry are not made available to model providers and are not used to train foundation models without the customer's permission. Depending on deployment configuration, processing can also be constrained geographically.

So when I assess this question, I would not ask: “Is the data confidential?” I would ask: “What are the actual data handling, sovereignty, retention, regulatory and security requirements?”

Those are governance questions, and the answer may be public cloud, private cloud, regional cloud, internal infrastructure or some combination of them.

There is another complication: AI is getting cheaper very quickly, This is the part that makes buying hardware particularly interesting.

Stanford's AI Index found that the inference price for a system reaching approximately GPT 3.5 level performance on one benchmark dropped from $20 per million tokens in November 2022 to $0.07 by October 2024, a reduction of more than 280 times.

That does not mean every AI workload becomes 280 times cheaper every two years. It does tell me something important.

This market moves extremely fast.; the server that looks economically attractive today is competing not only with today's API prices. It is competing with next year's API prices, next year's smaller models, next year's hardware and next year's optimisation techniques.

I therefore think the hurdle for buying internal AI hardware should be reasonably high.

I am locking capital into a technology cycle that is moving extraordinarily quickly.

So when would I use a monthly AI subscription?

For normal knowledge work, I think this is the easiest decision.

If I want employees to research, draft, analyse, brainstorm, work with spreadsheets, prepare presentations, interrogate documents and generally improve their productivity, a business subscription is extraordinarily difficult to beat for simplicity.

I pay per user.

The interface already exists.

The models are maintained for me.

The infrastructure scales for me.

Security administration is provided.

New model capabilities arrive without me replacing hardware.

For human led, variable and unpredictable usage, I see little economic reason to build a company AI platform simply to avoid subscription fees.

When would I use an API?

If I am embedding AI into a business process, the API becomes much more interesting.

Document classification.

Contract extraction.

Customer support.

Report generation.

Data analysis.

Internal knowledge systems.

Quality checking.

Maintenance analysis.

Procurement review.

Workflow automation.

Agents.

This is where I stop buying seats and start buying units of intelligence.

I can measure the workload.

I can route simple tasks to cheaper models.

I can use expensive models only when the problem actually requires them.

I can cache repeated context.

I can monitor spend.

I can scale almost instantly.

And most importantly, I do not own idle compute.

For many companies, I suspect this middle ground will remain the economically rational architecture for a surprisingly long time.

When would I seriously consider hosting AI internally?

I would start investigating it when several conditions begin to occur together.

The workload is large.

The workload is predictable.

The hardware can remain highly utilised.

The company has a repetitive set of tasks.

An open weight model performs sufficiently well on those specific tasks. Latency matters. Offline operation matters.

Data control requirements strongly favour internal processing.

The company has enough technical capability to operate the environment properly, and the total cost of ownership, not merely the GPU cost, beats the managed alternative over a sensible investment horizon.

Notice something important here. I did not say: “When the company becomes large.”

A multinational company performing occasional high complexity AI tasks may be much better served by an API.

A comparatively small company processing millions of repetitive AI transactions every day may have an excellent case for its own inference infrastructure.

Company size is not the deciding variable. Workload shape is.

I think the most sensible architecture will often be hybrid The more I looked at the economics, the less convinced I became that companies should make a single choice.

I can easily imagine an organisation where employees use an enterprise AI subscription for everyday work.

At the same time, business applications use APIs for variable workloads. A smaller internal model handles repetitive, sensitive and predictable activities. A powerful external model is called only when a problem exceeds the capability of the internal model.

Highly sensitive data stays inside.

General reasoning goes outside under controlled enterprise agreements.

That architecture lets me buy cheap intelligence when cheap intelligence is sufficient and expensive intelligence only when expensive intelligence adds value.

To me, that feels much closer to how mature enterprise AI will actually develop.

If I were discussing this with a board or executive committee, I would not begin with GPUs.

I would begin with five numbers:

How many AI tasks are we performing?

How many tokens does an average task actually consume?

What models are genuinely required to achieve acceptable quality?

How predictable is the workload?

What is our fully loaded cost per successful task?

That final word matters. Successful.

The cheapest model is not cheap if a human has to repair half of its work.

The cheapest infrastructure is not cheap if it fails when the business needs it.

And the most powerful model is unnecessarily expensive if a much smaller model performs the task perfectly well.

I would therefore measure cost per successful business outcome rather than cost per token.

That is the number I think executives should care about.

My conclusion. When I started looking into this, I expected to find a fairly simple crossover point where cloud AI becomes expensive and owning AI becomes cheaper.

I did not find one, but I found something more useful.

For individual users and ordinary employee productivity, subscriptions make enormous sense.

For automated business processes, APIs can remain surprisingly inexpensive even at volumes that initially sound enormous.

For high volume, stable and predictable workloads, self hosted AI can become economically compelling.

For workloads involving strict sovereignty, offline operation or exceptional confidentiality, internal AI may be justified even before it becomes the cheapest option.

For many mature organisations, I think the answer will eventually be a combination of all three.

What has changed most in my own thinking is this: I no longer see AI cost as the price of a subscription, and I do not see it simply as the price of a token.

I see it as the cost of completing a task successfully.

Once I look at AI that way, the conversation becomes much more familiar. It becomes a normal business question about utilisation, quality, risk, capital, operating cost and return.

Perhaps that is a sign that AI itself is maturing.

We are slowly moving beyond asking what it can do.

We are beginning to ask what it costs to do it properly.

Method and date of verification

All pricing in this article was checked against publicly available sources on 9 September 2026.

Worked examples are calculations, not vendor forecasts. I have stated the token assumptions so that the arithmetic can be independently reproduced.

Actual AI consumption will vary according to model, language, context size, reasoning effort, caching, tool calls, retries, concurrency, output length and application architecture.

Prices in this market change frequently. Anyone using these figures for an investment decision should refresh the underlying prices and benchmark the proposed models against the company's actual workload.

Sources

OpenAI, ChatGPT Plus. OpenAI ChatGPT Plus information

OpenAI, ChatGPT Business pricing. OpenAI Business pricing

OpenAI, API model pricing. OpenAI API Platform

OpenAI, understanding and counting tokens. OpenAI token guidance

OpenAI, ChatGPT Business and Enterprise rate card for chat, tasks, Work, Codex and Workspace Agents. OpenAI ChatGPT rate card

Anthropic, Claude API pricing. Anthropic Claude pricing

Google, Gemini Developer API pricing. Google Gemini API pricing

MLCommons, Agentic Inference for MLPerf Inference. MLCommons agentic inference benchmark

MLCommons, End to End Retrieval Augmented Generation Inference Benchmark. MLCommons RAG benchmark

OpenAI, Introducing gpt oss. OpenAI gpt oss announcement

OpenAI, gpt oss model card. OpenAI gpt oss model card

Mistral AI, Mistral Small 3.1. Mistral Small 3.1 technical information

NVIDIA, H100 specifications and inference economics. NVIDIA H100 product information

Google Cloud, Dynamic Workload Scheduler GPU pricing. Google Cloud GPU pricing

Stanford Institute for Human Centered Artificial Intelligence, AI Index, inference cost research. Stanford AI Index research and development data

OpenAI, business data privacy and security. OpenAI business data information

Microsoft, data privacy and security for models sold through Azure Foundry. Microsoft Foundry data privacy documentation

NIST, Artificial Intelligence Risk Management Framework, Generative Artificial Intelligence Profile. NIST Generative AI Risk Management Profile

This article is also published on Medium as part of my public research and writing.