Summary
Apple says its MLX software can run the 816-billion-parameter Muse Spark Open model across Mac Studio computers linked over Thunderbolt. In its benchmark, four M5 Ultra systems generated 36.73 tokens per second at an 8,192-token context.
Apple says its MLX framework can run Muse Spark Open, an 816-billion-parameter language model, across multiple Mac Studio computers connected with Thunderbolt. In the company’s benchmark, four Mac Studios with M5 Ultra chips generated 36.73 tokens per second at an 8,192-token context. The result comes from a defined inference test, rather than a measurement of a complete AI coding or agent workflow.
How MLX splits the model across Macs
Muse Spark Open is a mixture-of-experts (MoE) model: it has 816 billion parameters in total, but Apple lists 42 billion as active during inference. Apple evaluated it in NVFP4, a four-bit numerical format that reduces the model’s memory footprint to about 493GB. That is still too large for a single Mac Studio once the additional memory needed during inference is included.
MLX distributes the work in two ways. Pipeline parallelization divides a model by depth: one computer processes an initial set of layers, then passes its output to another computer to process the remaining layers. Tensor parallelization divides computation across the width of the model, so multiple computers work on parts of each layer. This can speed up computation, but it also requires frequent communication between machines.
For that communication, Apple’s setup uses a low-latency RDMA driver over Thunderbolt 5. RDMA, or remote direct memory access, lets computers exchange data with less intervention from their processors. Apple says macOS 26.2 introduced its Thunderbolt RDMA driver, while MLX version 0.30.1 and later provides communication tools for distributed workloads. The company connected the machines with two Thunderbolt cables per link to increase link bandwidth.
What Apple’s benchmark measured
Apple tested Muse Spark Open alongside two dense models, Mistral Medium 3.5 and Qwen 3.8. Its benchmarks measured time to first token—the delay before a response begins—generation speed, and peak power use. The reported generation speeds were measured over 128 output tokens.
At an 8,192-token context, Apple reports 36.73 tokens per second for Muse Spark Open using tensor parallelization across four M5 Ultra Mac Studios. The same model reached 24.99 tokens per second across two machines using tensor parallelization, and 15.74 tokens per second across two using pipeline parallelization. Apple’s results also show that adding machines can improve throughput, though communication between them is part of the calculation.
The post describes separate gains from the M5 Ultra for the smaller models that fit on one Mac Studio. In tests of Qwen 3.8 and Mistral Medium 3.5, Apple reports time-to-first-token improvements of up to 4.6 times over the M3 Ultra and generation speeds up to 1.6 times faster. The company attributes the first-token improvement to GPU Neural Accelerators that speed up matrix multiplication, and the generation gain to the M5 Ultra’s 1.2TB/s of unified memory bandwidth, compared with 819GB/s on the M3 Ultra.
Apple presents local MLX inference as a way for researchers and developers to run models on their own hardware, including for agentic software. Its example starts a local model server that can be connected to a coding tool. The demonstration shows how distributed inference can bring a model too large for one Mac onto a small cluster; its quoted throughput describes Apple’s benchmark conditions, not the performance of every application using that server.