Skip to content
PocketWebToolsPocketWebTools
Get the Mac app
All tools

Can I run AI on my computer?

Probably, if the model fits the memory your GPU can use: a 16 GB Mac lets it use about 12.7 GB, enough for Qwen3.5 9B (about 6.9 GB at 8,192 tokens, replying at 19 tokens per second measured on a 16 GB M1 Pro). An 8 GB graphics card keeps about 7.5 GB, enough for Llama 3.1 8B (6.7 GB); detect your machine below to check any model.

Your machine

Reading your graphics chip from the browser…

Your computer

What it can run

Confirm your machine above and every model is graded on it here: what fits, how fast it should reply, and why.

How the checker works

  1. We read your graphics chip from the browser. Chrome, Edge and Opera name the exact model; Safari always says Apple GPU and Firefox reports a stand-in, so there you pick it from the list.
  2. You confirm your memory. No browser tells a page how much video memory a card has, and Chrome, Edge and Opera, the only ones that report system memory, cap it at 32 GB on desktop, so we always ask.
  3. We grade every model on that machine: the memory it needs at your context length against what the GPU can use, and the reply speed, measured where a source measured it and estimated everywhere else.

Detection runs in your browser and nothing is stored on a server. Our anonymous usage counts note the kind of computer (Mac, PC or unified-memory PC), the memory size you picked, which app or tool button you clicked and your country, never your graphics chip's name. This browser remembers the last machine you confirmed, so the checker opens on it next time.

What decides whether a model runs

Memory comes first. A model needs its weights (the real file size of the quant you pick), context memory (the KV cache, computed from the model's own config at your context length) and room for the runtime: 0.7 GB, or 2% of the weights for files big enough that this is more.

For example, Llama 3.1 8B at Q4_K_M with 8,192 tokens of context needs 4.9 GB of weights, 1.1 GB of context memory and 0.7 GB for the runtime: 6.7 GB in all.

Context length matters. Qwen3 30B‑A3B holds 0.8 GB of context memory at 8,192 tokens and 3.2 GB at 32,768, so the context selector re-grades every row.

Speed follows memory bandwidth, because every new token reads the model's active weights once. Our formula was fitted to 53 llama.cpp and MLX runs: its median error on them is 6% and every one is within 20%. Speeds marked measured come from a published run on that machine instead.

Macs: the GPU can use only part of the memory

macOS caps how much of a Mac's unified memory the GPU may hold. On macOS 26 and later a 16 GB Mac gives it about 12.7 GB, a 32 GB Mac 26.8 GB and a 128 GB Mac 115.4 GB; macOS 15 and earlier give the same or less. Apple publishes no formula, so the figures come from public llama.cpp, Ollama and MLX logs, except 512 GB, which the table marks as assumed.

What the GPU can use by default, by Mac memory size
Mac memorymacOS 26 and latermacOS 15 or earlierEvidence
8 GB5.7 GB5.7 GBMedium
16 GB12.7 GB11.5 GBStrong
18 GB14.3 GB12.9 GBMedium
24 GB19.1 GB17.2 GBStrong
32 GB26.8 GB22.9 GBStrong
36 GB30.2 GB29.0 GBMedium
48 GB40.2 GB38.7 GBStrong
64 GB55.7 GB51.5 GBStrong
96 GB83.5 GB77.3 GBWeak
128 GB115.4 GB103.1 GBStrong
192 GB173.2 GB154.6 GBWeak
256 GB239.1 GB206.2 GBWeak
512 GB478.3 GB412.3 GBAssumed, no log yet

Evidence is for the macOS 26 and later figure; How we estimate lists the logs behind it.

If a model just misses, you can raise the limit until the next restart. The value is in MiB; leave several GB for macOS itself. When it would help, a model's reasons in the checker give the exact command.

Raise the GPU memory limit until restart
sudo sysctl iogpu.wired_limit_mb=<MiB>

What your Mac can run, by memory walks through each memory size.

PCs: video memory first, system memory as overflow

We set aside 0.5 GB of a card's video memory for the display, so an 8 GB card keeps about 7.5 GB for a model. llama.cpp-style runtimes can put the rest in system memory, and we allow up to 60% of it, but that part runs far slower, so any spill grades Tight. Both figures are our assumptions, not measurements.

Some cards are sold in two memory sizes under one name, so the checker asks which you have: GeForce RTX 5060 Ti (8 or 16 GB), GeForce RTX 5070 Laptop GPU (8 or 12 GB), GeForce RTX 4060 Ti (8 or 16 GB), GeForce RTX 3080 (10 or 12 GB), GeForce RTX 3060 (8 or 12 GB), GeForce RTX 3050 (6 or 8 GB), GeForce RTX 3080 Laptop GPU (8 or 16 GB), GeForce RTX 3050 Laptop GPU (4 or 6 GB), GeForce RTX 2060 (6 or 12 GB), Radeon RX 9060 XT (8 or 16 GB), Radeon RX 9050 (4 or 8 GB) and Arc A770 (8 or 16 GB).

Unified-memory PCs share one pool. AMD documents up to 96 GB of a 128 GB Ryzen AI Max+ 395 for the GPU. NVIDIA publishes no GPU-only limit for the DGX Spark, so we assume 84%, about 107.5 GB of its 128 GB; it reads memory at 273 GB/s.

What GPU do I have? helps you find your card.

Speech, voice and image models

These are graded on memory alone, with a speed shown only where a source measured one.

Three run in your browser here today: Audio Transcription with Whisper base or large-v3 turbo, Text to Speech with Kokoro 82M, and Voice Cloning with Chatterbox Turbo on WebGPU or Pocket TTS without it.

For image models on a Mac the checker points to Draw Things. Free for local generation. Draw Things+ costs $8.99 a month.

How we estimate

Show the formulas, constants and sources

Memory a model needs

Memory = weights + context memory + runtime. Weights are the real download size of the file you pick, read from Hugging Face. When a size is unknown we use parameters x bits per weight / 8, with typical averages per quant type (Q2_K 2.96, Q3_K_M 3.91, Q4_K_M 4.8, UD-Q4_K_XL 4.9, MXFP4 4.25, NVFP4 4.5, Q5_K_M 5.7, Q6_K 6.56, Q8_0 8.5, BF16 16). Runtime is 0.7 GB, or 2% of the weights when that is more, for compute buffers and allocator slack: measured buffers stay near that size whatever the model. Against our own Mac app's measured memory for six models, and Apple's published totals for two models at 65,536 tokens, this lands within 15%. Where someone measured the memory of the exact file, we use that run instead and say so. Checked 2026-10-19.

Context memory (KV cache)

Each token of context stores keys and values: 2 x layers x KV heads x head size x 2 bytes (16‑bit cache, the default in llama.cpp and MLX), read from each model's config.json. Sliding-window layers keep at most their window (2,048 tokens on Muse Spark Open, for example), so they add a fixed amount instead of growing. MLX is confirmed to cap them on Muse Spark Open; whether llama.cpp does for that model is not verified, so its GGUF figures may run low at long context. Models with compressed attention (MLA) store one small latent per layer. Linear-attention layers keep a small fixed state, counted in the runtime share. In llama.cpp you can store the cache at lower precision with --cache-type-k and --cache-type-v (-ctk and -ctv): per element that is q8_0 1.0625 bytes, q4_0 0.5625 bytes, from the size of each type's data blocks. A lower-precision V cache needs flash attention, which llama.cpp turns on by itself unless you switch it off. Measured runs are taken to use the 16‑bit default. Checked 2026-10-19.

How much memory a Mac's GPU can use

macOS only lets the GPU use part of a Mac's memory. Apple publishes no formula, so we read the default from public llama.cpp, Ollama and MLX logs where one exists. On macOS 26 and later: 8 GB: 2/3, about 5.7 GB (medium evidence); 16 GB: 0.74, about 12.7 GB (strong evidence); 18 GB: 0.74, about 14.3 GB (medium evidence); 24 GB: 0.74, about 19.1 GB (strong evidence); 32 GB: 0.78, about 26.8 GB (strong evidence); 36 GB: 0.78, about 30.2 GB (medium evidence); 48 GB: 0.78, about 40.2 GB (strong evidence); 64 GB: 0.81, about 55.7 GB (strong evidence); 96 GB: 0.81, about 83.5 GB (weak evidence); 128 GB: 0.84, about 115.4 GB (strong evidence); 192 GB: 0.84, about 173.2 GB (weak evidence); 256 GB: 0.87, about 239.1 GB (weak evidence); 512 GB: 0.87 (assumed), about 478.3 GB (unconfirmed evidence). Why the weaker rows are weaker: 96 GB: One log, from 2026-09-30. 192 GB: One M2 Ultra log, from 2025-10-30. 256 GB: Two M3 Ultra logs, possibly from the same person. 512 GB: No confirmed default. Logs show 464 GiB (about 0.906), but the same value appears on a macOS 15 Mac, so it is probably a manual override. We use 0.87, the 256 GB ratio, which may understate a 512 GB Mac by about 20 GB. On macOS 15 or earlier it was 2/3 up to 32 GB and 3/4 above. The limit can be raised with sudo sysctl iogpu.wired_limit_mb=<MiB>; it resets on restart and macOS needs memory of its own, so leave headroom. The setting cannot go past a second limit, the kernel's cap on all locked memory, which Apple's open-source macOS kernel sets by memory size (16 GB: 13.1 GB; 32 GB: 27.1 GB; 64 GB: 56.3 GB; 128 GB: 116.8 GB; 256 GB: 241.9 GB; 512 GB: 515.4 GB) and two measured Macs match exactly. That cap sits a little above the default at every size, so the default is what a model gets, and we suggest the setting only when a model fits under the cap. It may still fail. Checked 2026-10-19.

Several Macs linked together

For a group of Macs with the same memory, we add up what each one's GPU can use: two 256 GB Macs on macOS 26 and later come to about 478.3 GB. Assumes the model splits evenly across Macs of the same memory size (as exo and MLX distributed do) and that linking them costs no memory on any one Mac. Neither is measured. We give no estimated speed for a group; only published runs, each with its number of Macs. Checked 2026-10-19.

How much memory a PC can use

On a graphics card we take its video memory minus 0.5 GB for the display. Runtimes like llama.cpp, LM Studio and Ollama can put layers that do not fit into system memory; we allow up to 60% of it, and any model that needs it is graded Tight. Both numbers are our assumptions, not measurements. AMD documents up to 96 GB of GPU memory on a 128 GB Ryzen AI Max machine; on smaller ones we assume the same 75% share. NVIDIA publishes no GPU limit for DGX Spark, so we assume 84% of its memory, the share macOS gives a 128 GB Mac. Checked 2026-10-19.

Reply speed

Writing each token reads the model's active weights once, so speed follows memory bandwidth: seconds per token = active GB / (bandwidth x 0.85) + a fixed cost per token of 2 ms on Apple for standard models, 5 ms for hybrid linear-attention models (Qwen3.5 and 3.8), 6 ms for mixture of experts; 1.3 ms on NVIDIA and DGX Spark for standard models and 2 ms for mixture of experts. Active GB comes from the file's tensor table where we computed it, else from a similar file of the same model whose table we read, scaled by file size, else from the active parameters at the file's own bits per weight. When a PC spills into system memory, that part reads at 60 GB/s x 0.6 (assumed dual-channel DDR5). Fitted to 53 published llama.cpp and MLX runs, it had a median error of 6% and every point within 20%; that fit is in-sample. AMD and Intel cards borrow the NVIDIA values and are marked rough. Speeds are at short context; long chats run slower. Checked 2026-10-19.

Why speeds are a range

The top of the range is the formula above. The bottom is 65% of it on M1 and M2 chips, where our own Mac app ran about a third under the formula on two mid-size models, and 85% everywhere else. The M1 and M2 rule rests on those two points only. Checked 2026-10-19.

When we do not estimate speed

We show no estimated speed for weights under 2 bits each, which are limited by computation rather than bandwidth (a 1‑bit 27B model ran at about a quarter of the bandwidth limit), or for mixture-of-experts models above 120B total parameters, where the formula ran 40% fast on Muse Spark Open against a measured 18 tokens per second. Those rows show measured speeds where someone published them, and "not measured" otherwise, and a model with no speed figure is graded Tight at best. Checked 2026-10-19.

Measured beats estimated

When someone published a speed for the same model, quant and chip, we show that number, labelled measured, with its source and caveats, instead of the formula. A run on another runtime (Meta's ExecuTorch numbers for a GGUF, for example) or one that does not say which GPU-core version of a chip it used is shown for comparison only. Our own measurements come from PocketWebTools for Mac on a 16 GB M1 Pro and a 32 GB Mac mini M4. A speed measured at a context more than 4 times shorter than the one you picked (or at an unstated one, which usually means a short test) is labelled measured at short context, since long chats run slower. Measurements of several Macs linked together never stand in for one Mac, and each is shown with its number of Macs. Model data checked 2026-10-19.

Grades

Won't fit: the model needs more than the GPU can use (plus the allowed system memory on a PC). Tight: it fits but needs more than 90% of that memory, spills into system memory, replies under 8 tokens per second, or has no speed figure for that machine. Runs well: everything else. Every grade states the quant, the context length (8,192 tokens unless you change it) and the memory figure it assumed. Speech, voice and image models are graded on memory only. Checked 2026-10-19.

Time to the first word

Reading your prompt is limited by computation, not bandwidth. We scale each chip's published llama.cpp prompt speed on a 7B model by 6.6B / the model's active parameters, x 0.75 for mixture of experts. It matched measured runs within 8% for standard models. Checked 2026-10-19.

Questions

Can I run an LLM on my PC?
Yes, if the model fits your graphics card's memory, or your system memory at a much slower speed. An 8 GB card keeps about 7.5 GB for a model, enough for Llama 3.1 8B at Q4_K_M (6.7 GB at 8,192 tokens).
Do I need a GPU to run a local LLM?
No, but it is much faster with one. Speed follows memory bandwidth: dual-channel DDR5 system memory manages about 60 GB/s (our assumption) against 272 GB/s on a GeForce RTX 4060. On a Mac the built-in GPU shares the unified memory, so no separate card is needed.
How much VRAM do I need for Ollama?
It depends on the model, not on Ollama: roughly the file size, plus context memory, plus about 0.7 GB for the runtime. Llama 3.1 8B at Q4_K_M needs about 6.7 GB at 8,192 tokens, gpt-oss-20b about 13.0 GB.
Can Ollama run without a GPU?
Yes. Ollama's macOS page lists Intel Macs as “x86 (CPU only)”, and llama.cpp-based runtimes can run on the processor alone, just much more slowly.
Is my GPU NVIDIA or AMD?
The checker reads it from your browser: Chrome, Edge and Opera name the exact card, while Safari and Firefox hide it, so there you pick yours from the list. What GPU do I have? covers finding it on any computer.
Is running a local LLM worth it?
If privacy or cost matters to you, yes: your prompts and files never leave your device, and once a model is downloaded it is free to use as much as you like, with no account.

Where to go next

Updated Added Meta's measured memory for Muse Glimmer 30B in LM Studio, the local Ollama tag for Qwen3-Coder-Next, and the Mac count on every multi-Mac run.