Run AI on your own machine

Which local model
should you run?

One rule decides everything: the model has to fit in your memory. Let's match your exact hardware to the right model size — fast.

The rule that matters

Model size has to fit your memory

There are only two numbers you need to know about your machine.

Windows / Linux

Your GPU VRAM

The memory on your graphics card. A 4090 has 24GB, a 5090 has 32GB. That's your hard ceiling.

Apple Silicon

Your unified memory

The RAM in your Mac is shared with the GPU. A 64GB MacBook can load far bigger models than a 24GB PC GPU.

Apple vs GPU

How much memory can you actually use?

Mac · unified memory~75–80% usable

model

macOS

GPU · VRAM100% usable — but capped

model (4090 = 24GB · 5090 = 32GB)

~½

the throughput of a same-tier GPU. Tokens/sec follow bandwidth — M4 Max ≈ 546 GB/s vs RTX 4090 ≈ 1008 GB/s.

Mac = bigger models · GPU = faster models

Rough memory needs

What each model size actually costs

Memorize these four. Everything else is interpolation.

7B

~8 GB

14B

~12–16 GB

32B

~24 GB

70B

~64 GB

Leave headroom for your OS, your editor, and the context window — don't fill memory to the brim.

The cheat sheet · screenshot this

Hardware → model tier table

Your hardware	Chat / edit model	Autocomplete model
8GB VRAM / 16GB Apple	`qwen2.5-coder:7b`	`qwen2.5-coder:1.5b`
12–16GB VRAM / 24–32GB Apple	`qwen2.5-coder:14b` · `qwen3-coder:30b` (32GB Apple)	`qwen2.5-coder:1.5b`
24GB+ VRAM / 48–64GB Apple	`qwen3-coder:30b`	`qwen2.5-coder:1.5b or 7b`
64GB+ Apple / dual-GPU	`qwen3-coder-next` (80B-A3B)	`qwen2.5-coder:7b`
CPU only / under 8GB	`qwen2.5-coder:1.5b–3b`	`same — expect it slow`

Qwen3-Coder is the 2026 default — MoE, so only ~3B params run at a time (fast). DeepSeek-Coder-V2 stays a solid alt.

Find yours

Exact hardware recommendations

Your machine	Chat / edit model	Autocomplete
MacBook Air M-series · 16GB	`qwen2.5-coder:7b`	`qwen2.5-coder:1.5b`
M4 Pro · 24–48GB	`qwen2.5-coder:14b` → `qwen3-coder:30b`	`qwen2.5-coder:1.5b`
M5 / M4 Max · 64GB	`qwen3-coder-next` 80B-A3B	`qwen2.5-coder:7b`
M5 / M4 Max · 128GB	`qwen3-coder-next` (480B on 256GB+)	`qwen2.5-coder:7b`
Older Intel MacBook	`qwen2.5-coder:1.5b–3b`	`same — expect it slow`
RTX 3060 / 4060 · 8GB	`qwen2.5-coder:7b`	`qwen2.5-coder:1.5b`
RTX 4070 / 4080 · 12–16GB	`qwen2.5-coder:14b` / `deepseek-coder-v2:16b`	`qwen2.5-coder:1.5b`
RTX 4090 · 24GB	`qwen3-coder:30b`	`qwen2.5-coder:1.5b or 7b`
RTX 5090 · 32GB	`qwen3-coder:30b`	`qwen2.5-coder:7b`
Dual-GPU · 48GB+ VRAM	`qwen3-coder-next` 80B-A3B	`qwen2.5-coder:7b`
CPU only / under 8GB	`qwen2.5-coder:1.5b–3b`	`same — expect it slow`

Why two models?

Autocomplete and chat have different jobs

⚡
Autocomplete fires on every keystroke — it needs a tiny, fast model (1.5B) or it feels laggy.
🧠
Chat / edit you trigger manually — it can be big and smart because you'll wait a second or two for it.
📏
Size down if your machine is smaller. A slow model isn't a broken setup — it's just the wrong size for your hardware.

Reality check

CPU-only works — but it's 10–30 sec per response.

A 7B model on a GPU or Apple Silicon is the real entry point. If you're on a laptop with no dedicated GPU, you can still try it — just set your expectations. The moment you have a GPU or Apple Silicon chip, the experience changes completely.

My rig: M5 Max, 64GB — running qwen3-coder-next comfortably (MoE, only ~3B active so it's fast).

Which local modelshould you run?