A one-page primer on understanding and controlling cloud & AI costs.
01What is FinOps?
FinOps (Cloud Financial Management) is the practice of getting the most
value out of cloud spend — through visibility, optimization,
and making cost a shared responsibility between engineering and finance.
It is not about spending less; it's about spending wisely.
The mindset shift: every engineer's design decision is also
a spending decision. FinOps makes those costs visible so teams can make
informed trade-offs.
02Every cloud bill = 4 categories
No matter the provider (AWS, Azure, GCP), all spend maps to these:
🖥️Compute
The machines running your code — VMs, containers, functions, and GPUs.
💾Storage
Where data lives — disks, object storage, backups, and snapshots.
🌐Data transfer
Moving bytes between places — the internet, regions, and zones (egress).
⚙️Managed services
Things the provider runs for you — databases, queues, AI/ML APIs, logging.
03Why AI changes the game
Traditional FinOps assumes cost scales with the servers you provision. AI
breaks that: cost now scales with usage — tokens, requests,
and GPU-hours. A feature that goes viral can multiply the bill overnight.
AI spend shows up on three surfaces:
Inference — running the model to serve requests. Usually the biggest, fastest-growing AI cost.
Training / fine-tuning — big, bursty GPU jobs. Expensive per run, but occasional.
Supporting footprint — vector databases, embeddings, pipelines, and storage.
04The levers that actually work
Model routing — send easy requests to a cheaper model; reserve the expensive one for hard tasks. Highest-leverage move.
Prompt & output discipline — you pay per token, so trim context and cap output length.
Caching — don't pay twice for the same or near-identical request.
Turn off idle resources — non-prod environments and idle GPUs are pure waste.
Right-size & autoscale — match resource size and count to real demand.
Commit to the baseline — buy discounts (Savings Plans / CUDs) for stable usage only.
05The one metric to build first
Cost per request (or cost per active user).
cost per request = total inference cost ÷ number of requests
It normalizes away volume growth, so it tells you whether your
efficiency is improving even as usage climbs. If a feature costs
more to run than it earns, that's a unit-economics problem — and scaling it
makes it worse.
06Mini glossary
Token
The unit LLMs process text in — roughly ¾ of a word. Hosted APIs bill per input and output token.
GPU-hour
One GPU running for one hour. An idle GPU-hour costs the same as a busy one — so utilization is everything.
Egress
Data transferred out of the cloud. Frequently billed and frequently a surprise.
Rack rate vs. effective cost
Sticker price vs. what you actually paid after discounts and commitments.
Unit economics
The cost to deliver one unit of value (per request, per user). The most important thing FinOps produces.