Local Inference Economics (Jetson Thor)

TL;DR

At 120W, running an 80B(3B) MoE inference node on Jetson Thor across EU. The revenue from Code Generation at 68% load is ~€7.21/day. Looks good, but you still need to sell it to someone (lol) at market prices

At 6% load, you'll start to be profitable (by electricity).

You'll never be profitable with AWS A100 & the same model 😊

As I was already saying:

A lot of people believe the future of software development is going toward autonomous agents. The future IT revenues are not coming from hiring hundreds of developers and upselling them with margin locked in a Seychelles account, but from paying for electricity and running self-hosted models that build features for the next AI-first startup.

This post is about what that actually costs—three chapters: benchmarks, architecture, and electricity.

I. The Benchmarks

Over the past three months, we ran three benchmark series on Jetson Thor at 120W: text LLMs, vision-language models, and an 80B coding agent. The model family across all three: Qwen3, in MoE configurations with W4A16 or NVFP4 quantization.

Qwen3-30B-A3B beat Nemotron-3-Nano by 1.6× on TTFT and 1.8× on throughput. Qwen3-VL-30B beat Nemotron-Nano-12B-VL by 2.9× on TTFT — 42ms median, which is faster than human perception of delay. Qwen3-Coder-Next at 80B scored 79.26% MMLU-PRO with NVFP4 quantization — a 3.44% gap versus FP16, on a model twice the size of anything the previous generation of edge hardware could load.

The throughput numbers across all workloads show a clear pattern:

• Autocomplete at C=32 generates 116.6 million tokens per day.
• Hardware cost is €3,075.
• Payback is 55 days.
• After that, the unit generates €56/day on €0.84 in electricity.

The Operational Picture

Taking all three series together and running the workload profiles at full sustained load — 120W, Netherlands household electricity at €0.29/kWh, €0.8352/day.

The four panels cover daily token capacity by mode, daily profit, days to recover hardware costs, and a load-scenario breakdown for Autocomplete. The numbers in full:

Revenue at €0.4907/MTok blended rate. Electricity cost: €0.8352/day.

The autocomplete scenario makes it funnier: at C=32, the device generates 116 million tokens per day. Hardware cost is €3,075. Payback is 55 days. The electricity cost is 1.46% of revenue. After day 55, the unit generates €56/day on €0.84 in electricity.

The more typical scenario — a development team on Code Gen at C=4 running Qwen3-Coder-Next ($0.12/M input · $0.75/M output on OpenRouter) — pays back in 282 days and returns €3,986 in its second year of operation.

II. The Architecture

Why 120W is possible, and why it changes the economics

A reasonable question at this point: why is a device with a Blackwell GPU, 128GB of memory, and 30B-parameter inference running at 120 watts? An A100 has a 300–400W TDP alone. Total system under load: 450–700W before cooling.

W4A16 on Thor outperformed FP8 Nemotron on the same device because Blackwell's memory subsystem handles packed low-precision weights more efficiently, saturating its bandwidth.

The efficiency comparison in concrete terms:

7.6× more tokens per joule. (Yet, better quality, but guys, please, you don't need FP16 for NextJS vibe-coding.

III. The Plug

On electricity prices across Europe, and why Amsterdam is close to the average

The map below shows live day-ahead wholesale electricity prices across European countries, sourced from ENTSO-E:

thingler.io/map — updated daily

Four representative countries across the EU electricity price spectrum, Code Gen at C=4 (340 tok/s):

Household rates, Eurostat H1 2025 incl. taxes. Code Gen C=4, 340 tok/s.

Amsterdam at €0.29/kWh sits almost exactly at the EU household average. It is a representative number, not an outlier.

What does change: break-even load — the minimum utilization needed to cover electricity. From Amsterdam:

"What if I want to dump a market?"

Do you know what an actual price is now?

Conclusion

If you have a sustained and predictable inference workload — a team, a robot, a pipeline — the economics of running it on your own hardware are better than cloud and significantly better at the high end. Plus, it's kinda more green if you use less energy.

The architecture chapter explains how 120W is possible: unified memory, no PCIe overhead, an ARM CPU that adds almost nothing to the power budget, and a Blackwell GPU designed for this operating point. The efficiency advantage — 7.6× tokens per joule versus a cloud A100.

The three benchmark series described a device capable of 340 tok/s on a 30B text model, 42ms TTFT on a 30B vision model, and 79.26% MMLU-PRO on an 80B coding model — all at 120W, all from the same hardware.

The electricity chapter shows that from Amsterdam, the daily operating cost is €0.84, break-even utilization is 6%, and the annual profit delta versus the cheapest country in Europe is €316. Geography is not a variable that changes the decision.

What changes the decision is whether you have a sustained workload to run. If you do, the hardware pays back in 55–232 days. After that, you are paying €0.84/day to run inference that costs €10–20/day at current API rates.

Sources:

• Benchmark data: LLM Benchmark · VLM Benchmark Part 2 · Qwen3 Coder Next — 79.26% MMLU-PRO
• Electricity prices: Eurostat nrg_pc_204, H1 2025, household incl. taxes
• Live EU price map: thingler.io/map via ENTSO-E
• NVIDIA Jetson Thor

All calculations: Jetson Thor @ 120W sustained, Netherlands household 0.2724/kWh, March 2026.

Hope you have a good day,

Vsevolod Prudius, Sobaka Team