Can you cluster Mac Studios for AI?

A cluster is two or more Macs joined by Thunderbolt cables, each holding part of one model. Apple's TN3205 says RDMA over Thunderbolt "is available starting with macOS 26.2 on Macs with Apple silicon with Thunderbolt 5". Each link is point to point, so for the lowest latency every Mac connects to every other Mac. A ring can join more Macs than a Mac has ports, but then the software has to forward data from Mac to Mac.

Apple's Mac Studio page promises "up to 3x faster distributed AI inference", measured as the time to summarize a long prompt on four preproduction 512 GB M5 Ultras in a full mesh, against one. The same page tells you to "use exo to cluster multiple AI workstations".

The link is the bottleneck. Apple rates Thunderbolt 5 at up to 120 Gb/s, and Jeff Geerling measured about 50 to 60 Gb/s in practice (his test). 60 Gb/s is 7.5 GB/s, against 1.2 TB/s of memory bandwidth inside one M5 Ultra, which is why splitting a model across Macs never scales perfectly.

• Pipeline parallel: each Mac holds a block of layers and they take turns on every token. It adds memory, not generation speed. Apple did see prefill, the wait before the first token, up to about 1.8x faster at 65,536 tokens of context with --prefill-step-size 8192, because the Macs overlap work on chunks of a long prompt.
• Tensor parallel: every Mac works on every layer at the same time. It adds speed, and it needs the low latency of RDMA: the MLX docs call their RDMA backend "Necessary for things like tensor parallelism".

Two Macs in pipeline mode stayed within 2% of one Mac's decode speed in every run. Tensor mode was 1.7x to 1.8x faster on two Macs and 2.6x to 3.1x faster on four. exo claims "up to 1.8x speedup on 2 devices and 3.2x speedup on 4 devices" for its own tensor sharding (a vendor figure).

For Muse Spark Open, Meta's open-weight model, which needs 544.6 GB at 4‑bit, two Macs ran at 15.74 tokens per second in pipeline mode and 24.99 in tensor mode, and four ran at 36.73. MLX LM's server and chat use tensor parallel by default; --pipeline switches to pipeline mode.

Raw RAM is not what the GPU can use. On macOS 26 and later, one 256 GB Mac gives the GPU about 239.1 GB, and we assume about 478.3 GB for a 512 GB Mac. Four 512 GB Macs have 2,048 GB of RAM but about 1,913.2 GB of GPU memory, and that model is not on sale yet.

You do not always need a cluster for a big model. Muse Spark Open's Q2_K build (a 304.1 GB file) needs about 311.1 GB to run, which fits one 512 GB Mac; Hugging Face, a launch partner, measured 18 tokens per second on a 512 GB M3 Ultra. DeepSeek V3.1 at Q4_K_M needs 414.1 GB: too big for one 256 GB Mac, but within a 512 GB one once Apple sells it. More on that Mac in the 512 GB section.

Disk adds up too. Each Mac loads the model from its own disk (Meta's cookbook), so a four-Mac Muse Spark Open setup stores the 533 GB NVFP4 checkpoint four times.

Two 256 GB Macs give the GPU about 478.3 GB for $18,998, the same as one 512 GB Mac's assumed 478.3 GB, whose price Apple has not published.

One 256 GB M5 Ultra costs less than either group, gives the GPU more memory and reads it faster, with no cables or cluster software. Groups of smaller Macs only make sense if you already own them.

• Thunderbolt 5, so RDMA works on macOS 26.2 or later: Mac mini (M5 Pro), Mac Studio (M5 Max, M5 Ultra), MacBook Pro 14-inch (M5 Pro, M5 Max), MacBook Pro 16-inch (M5 Pro, M5 Max).
• Thunderbolt 4, so no RDMA: Mac mini (M6), MacBook Air 13-inch (M5), MacBook Air 15-inch (M5), MacBook Pro 14-inch (M5), iMac 24-inch (M4).
• No Thunderbolt: MacBook Neo (A18 Pro).

exo adds two rules of its own: every Mac must run exactly the same macOS version, beta numbers included, and on a Mac Studio the Thunderbolt 5 port next to the Ethernet port cannot be used. exo wrote the second rule for earlier Mac Studios. Apple's own four-Mac M5 Ultra mesh uses all 6 ports, so it does not apply there; for the M5 Max Mac Studio, which has the same back ports as the M4 Max, it is unconfirmed.

Only the M5 Pro Mac mini has Thunderbolt 5, so only it can use RDMA. With 3 ports each, up to 4 can form a full mesh, which matches TN3205's own example of four Mac minis. The M6 Mac mini has Thunderbolt 4 and can join only MLX's ring backend, which runs over ordinary TCP networking; we know of no published speed for that setup.

Four 64 GB M5 Pro minis give the GPU about 222.7 GB for $10,796, less than one 256 GB Mac Studio's 239.1 GB for $9,499, and each reads its memory at 307 GB/s. The setup steps are the same as below.

• MLX LM, from Apple: what Apple measured the cluster speeds with, and what Meta's Mac cookbook uses. Its distributed mode has two backends, ring over TCP and JACCL over RDMA, started with mlx.launch. Current versions: MLX 0.32.4, MLX LM 0.32.2 (PyPI).
• exo (Apache 2.0): a Mac app that finds the other Macs and splits the model for you, on top of MLX. It needs macOS 26.2 or later, installs with brew install --cask exo, and serves a dashboard and an API at localhost:52415 with OpenAI, Claude Messages and Ollama style endpoints. Its last packaged release is v1.0.71 from April 23, 2026, though the code was still updated on October 13, 2026. The PyPI package named exo is unrelated: do not pip install it.
• llama.cpp RPC: its own README calls it a "proof-of-concept" that is "fragile and insecure", so never run the RPC server on an open network. On macOS it can use RDMA over Thunderbolt, and v0.7.0 added tensor parallel RPC (-sm tensor).
• An independent test: Jeff Geerling linked four M3 Ultra Mac Studios and found exo got faster as he added Macs, while llama.cpp RPC got slower. He also wrote that he has "some underlying trust issues with Exo, since the developers went AWOL for a while".
• PocketWebTools for Mac does not link Macs and serves no local API, so it is not a cluster tool.

1. Cable every Mac to every other Mac: Use Macs with Thunderbolt 5 on macOS 26.2 or later and run a Thunderbolt 5 cable between every pair. MLX's RDMA backend supports only fully connected groups.
2. Turn off the Thunderbolt bridge: On each Mac open System Settings, Network, Thunderbolt Bridge and choose Make Inactive, so frames cannot loop between the cables.
3. Enable RDMA in macOS Recovery: Start each Mac in macOS Recovery, open Utilities, Terminal, run rdma_ctl enable and restart. It cannot be done remotely. Back in macOS, ibv_devices should list the RDMA interfaces.
4. Set up passwordless SSH: Turn on Remote Login (System Settings, General, Sharing) on every Mac. From the first Mac, create a key with ssh-keygen and copy it with ssh-copy-id to every Mac in the group, the first one included: mlx.launch starts each part of the job over SSH, so ssh mac1.local has to work without a password even on mac1. Below, mac1.local to mac4.local stand for each Mac's local hostname, shown under Sharing.
5. Install MLX LM on every Mac: Run pip install -U mlx-lm on each Mac (MLX LM 0.32.2 with MLX 0.32.4 today), into the same Python path on every Mac, such as a virtual environment at the same location: mlx.launch runs that exact Python binary on each one, and mlx.launch --print-python shows which.
6. Write the host file: On the first Mac, run mlx.distributed_config over Thunderbolt with the jaccl backend. With --auto-setup it also configures the links, which needs passwordless sudo on each Mac.
7. Copy the model to every Mac: Each Mac loads the model from its own disk. Download it on the first Mac with hf download, then mlx_lm.share copies it to the others over the cluster.
8. Serve the model across the Macs: Start mlx_lm.server (or mlx_lm.chat in the terminal) through mlx.launch with the jaccl backend. It splits the model tensor parallel by default; add --pipeline for pipeline mode. Do not use mlx_lm.generate: it has no multi-Mac code and every Mac would load the whole model.

rdma_ctl enable

ibv_devices

ssh-keygen -t ed25519
ssh-copy-id mac1.local
ssh-copy-id mac2.local
ssh-copy-id mac3.local
ssh-copy-id mac4.local

pip install -U mlx-lm

mlx.distributed_config --verbose --hosts mac1.local,mac2.local,mac3.local,mac4.local --over thunderbolt --backend jaccl --auto-setup --output hostfile.json

hf download Qwen/Qwen3.8-27B
mlx_lm.share --model Qwen/Qwen3.8-27B --hostfile hostfile.json

mlx.launch --hostfile hostfile.json --backend jaccl -- mlx_lm.server --model Qwen/Qwen3.8-27B --port 8080

mlx.launch --hostfile hostfile.json --backend jaccl -- mlx_lm.chat --model Qwen/Qwen3.8-27B

The example model is the Qwen3.8 27B checkpoint Apple measured; swap in any MLX model that is too big or too slow on one Mac. Muse Spark Open needs a gated download and MLX LM 0.32.2 or later: see how to run Muse Spark Open on a Mac. If you would rather not use Terminal, exo does the discovery and splitting for you.

Cluster only if the model you need does not fit one Mac, or if you will run it tensor parallel for speed. Two Macs in pipeline mode buy memory, not faster generation.

For anything up to about 239.1 GB, one 256 GB M5 Ultra ($9,499) is cheaper and simpler than a group of smaller Macs.

For 4‑bit frontier models such as Muse Spark Open (544.6 GB) and Kimi K2.6 (596.0 GB), four 256 GB M5 Ultras ($37,996, about 956.6 GB of GPU memory) are the setup you can order today; two 512 GB Macs give the same once Apple sells them.