The real cost of running an LLM on-premise in 2026: VRAM, GPU or Mac Studio, and electricity

"Self-host AI on-premise so data never leaves the organisation" is an increasingly popular direction. But before buying hardware, the first question has to be: how much does it cost? This article breaks down the three real costs of running a large language model (LLM) yourself: VRAM (how much GPU memory a model needs), electricity (what a GPU running 24/7 costs at official Vietnam rates), and hardware (GPUs or Apple Silicon machines such as Mac Studio) — then compares them against calling a cloud API for the same model. Every figure is sourced from public data, checked on 18 July 2026; the Mac Studio / Mac mini sections were updated on 24 September 2026.

How much VRAM does a model need?

A model needs a minimum amount of GPU memory (VRAM) equal to its parameter count times the bytes stored per parameter. At FP16/BF16 precision each parameter takes 2 bytes; quantizing down to INT8 makes it 1 byte, INT4 makes it 0.5 byte. So a 72.7-billion-parameter model (Qwen2.5-72B, per its HuggingFace model card) needs ~145 GB of VRAM at FP16, but only ~36 GB at INT4 — enough to fit one 80 GB GPU. That is why quantization is nearly mandatory for self-hosting.

Important caveat: the numbers in the table below are weights only. In practice you must add roughly 20–30% for KV-cache, activations and runtime overhead — so treat these as a floor, not a recommended configuration. (GB here uses billions of bytes, to compare cleanly against GPU capacities.)

The takeaway: choose the model size first, then the GPU. A quantized 7–32B model can run on a single 24 GB consumer card; but the 70B-and-up tier already needs an 80 GB data-centre GPU or multiple cards stacked — a large jump in cost.

Apple Silicon machines solve this differently: unified memory — the CPU and GPU share one pool of RAM. By default macOS lets the GPU use about 75% of that memory, so by Namtech's estimate a 64GB Mac Studio can dedicate ~48 GB to the model — enough to load the 70–72B tier at INT4 on a single machine, without stacking cards. This is an estimate from memory capacity, not a speed measurement.

Electricity: what does a GPU running 24/7 cost?

The monthly electricity of a GPU running continuously is TDP (kW) × 720 hours × electricity price. At Vietnam's average retail price of VND 2,204.0655/kWh (excl. VAT, effective 10 May 2025 per the Ministry of Industry and Trade), an H100 rated at 700W running 24/7 draws ~504 kWh and costs about VND 1.11 million/month — and that is the electricity of the GPU alone.

We use the average price as a transparent, verifiable anchor. In reality, a business's bill also depends on voltage level and time-of-use band (off-peak/normal/peak) under the tariff in Decision 1279/QĐ-BCT, plus VAT — and, importantly, total system electricity (CPU, RAM, fans) plus room cooling is typically 1.5–2× the bare GPU draw. In other words, roughly double the figures below to estimate a real bill.

*Electricity for the GPU alone, at the average price excl. VAT. A real bill is higher due to CPU/RAM/cooling (typically ×1.5–2), VAT, and voltage/time-of-use tariffs.

How much electricity do Mac Studio and Mac mini use?

Same formula, but for Apple machines we use Apple's published figures for the whole machine (CPU, GPU, memory, SSD), not just the GPU as in Table 2. Per Apple, Mac Studio's maximum measured consumption is about 200W with M5 Max and 385W with M5 Ultra; the maximum continuous power rating is 480W for Mac Studio and 155W for Mac mini (Apple spec pages).

*For the whole machine at maximum, at the average price excl. VAT — the machine does not always run flat out, so this is a ceiling. Apple measured 385W on the M5 Ultra 36-core CPU/80-core GPU; the AI Enterprise package uses the 30-core CPU/64-core GPU version, so we treat 385W as the ceiling. A real bill still depends on VAT and voltage/time-of-use tariffs.

To compare fairly: Table 2 is the GPU alone, Table 2b is the whole machine. Even so, a Mac Studio M5 Max at maximum (~200W for the whole machine) is below the GPU-only draw of every GPU in Table 2; the M5 Ultra (~385W) is in the range of a single A100 on its own. And as a desktop machine (operating temperature 10–35°C per Apple), it does not require a dedicated server room the way a GPU rack does.

The surprise for many: GPU electricity is usually NOT the biggest line. A data-centre GPU costs VND 0.5–1.1 million/month in power — significant, but small next to the capital to buy that same GPU. That is why a TCO analysis cannot skip hardware.

The three components of on-premise TCO

The total cost of ownership (TCO) of self-hosting an LLM has three parts, and the largest is hardware capital — usually far more than electricity and operations combined. The table below ranks the three by magnitude and volatility, so a business knows which variable to lock down before deciding.

Self-hosting versus renting a cloud API

At low-to-moderate usage, renting a cloud API for the same model is usually cheaper than self-hosting, because you avoid hardware capital. Take Qwen2.5-72B itself: the reference cloud price (provider median, per Artificial Analysis) is $0.475 per 1M input tokens and $0.495 per 1M output tokens. Converting at ~VND 26,440/USD (Vietcombank, 17 Jul 2026):

Read the table correctly: self-host electricity is nearly fixed per running hour, while cloud cost scales with tokens. So cloud is cheaper at low usage; only above a high, continuous volume threshold does self-hosting's electricity + hardware depreciation become cheaper per token. But the deciding figure is hardware capital. Data-centre GPU prices swing sharply, so this article does not price them; a business must get real quotes and amortise them.

Mac Studio has public prices, so the division can be done. The AI Pro machine (64GB, enough to load Qwen2.5-72B at INT4 per the Table 1 estimate) costs 111,150,000₫ VAT included. Assuming 36-month depreciation, that is ~VND 3.09 million/month, plus at most ~VND 0.32 million electricity (Table 2b) — on par with the API cost at 200M input + 100M output tokens (VND 3.82 million). This is only an illustrative division: the article has not verified whether one machine can keep up with that volume, because speed depends on the model, prompt length and number of concurrent users — measure on real workloads before concluding.

What it means for Vietnamese businesses

For most Vietnamese businesses, the sensible choice is not "all self-hosted" or "all cloud", but segmenting by data and volume. For the self-hosted part, Namtech deploys on Apple Silicon (Mac Studio), extending our roadmap to build internal AI and our piece on AI data sovereignty. The reasons: large unified memory in one machine, public pricing, low power draw, and it sits right in the office.

Choosing a configuration by need

*Namtech estimates to guide machine choice, not benchmark measurements: usable memory ≈ RAM × 0.75 (the share macOS lets the GPU use by default); a 4-bit quantized model ≈ 0.5 byte/parameter, with 20% reserved for context. Prices cover hardware only; software and deployment are quoted separately. Power: Apple's measured maximum (see Table 2b).

Models run on common Mac tools: MLX, Ollama or LM Studio, with open models such as Qwen, SEA-LION and Gemma — pick a size from Table 1, then check it against the "Max model size" column above. See each machine on the Mac Studio page, or compare the three configurations side by side.

If you only need to experiment or for personal use, Mac mini is cheaper: the M6 32GB costs 41,040,000₫ (estimated models up to ~38B parameters) and the M5 Pro 64GB costs 86,070,000₫ (~76B). The trade-off is lower memory bandwidth (170 GB/s and 307 GB/s, versus 460–614 GB/s for M5 Max per Apple) — and bandwidth is the main factor in text-generation speed — so it will be slower than a Mac Studio with the same memory. Put machines side by side on the Mac comparison page.

When should you still use GPUs?

Mac Studio does not win every case. NVIDIA GPUs remain the right choice when:

• Many concurrent users or batch processing: data-centre GPUs with dedicated serving software (such as vLLM) batch many requests at once more efficiently than a desktop machine.
• Very long prompts, RAG over large document sets: reading the prompt depends on raw compute, where GPUs are stronger; the Mac's advantage is large memory in one machine, not long-prompt processing speed.
• Training / fine-tuning models, or software that only runs on NVIDIA CUDA.
• You already have a server room, racks and a GPU operations team — the marginal cost of adding GPUs is then lower.

The selection principles stay the same, GPU or Mac:

• Start with one machine, sized by model: a quantized 7–32B need → AI Box; the 70–72B tier → AI Pro; larger models or the whole company → AI Enterprise, then AI Cluster beyond one machine.
• Sensitive data → self-host; routine tasks → consider the API: if the use case requires keeping data on-site (compliance with the Personal Data Protection Law No. 91/2025/QH15, customer contracts), self-hosting is a requirement — the cost is the price of sovereignty, not a way to save money.
• Compute TCO with your own numbers: take your real token volume + real hardware prices + real electricity bills, then compare cloud vs self-host. Do not decide on the gut feeling that "on-premise must be cheaper".

Self-hosting an LLM on-premise is not cheaper by default — it becomes cheaper when usage is high and continuous, and it is always worth it when data must stay inside the organisation.

Frequently asked questions

Note: This article is compiled from public sources as of 18 July 2026; the Mac Studio / Mac mini sections were updated on 24 September 2026. VRAM figures are estimates from the params × bytes formula (weights only, before 20–30% overhead); model sizes on Mac are estimates from memory capacity, not benchmarks; electricity is computed from official power figures (NVIDIA, Apple) and the official average electricity price, excl. VAT; Mac prices are Namtech selling prices including VAT as of 24 September 2026 and follow Apple's list prices; cloud API prices and exchange rates may change. Informational only — not technical, investment or legal advice.