Rapid-MLX: Faster Local LLMs on Apple Silicon

On X a developer introduces a new tool and calls it, with the usual hedge, possibly the fastest local inference engine on Mac. Rapid-MLX is open source, runs only on Apple Silicon, and aims to be a drop-in replacement for Ollama at two to four times the speed. The code sits on GitHub under the Apache 2.0 license.

We already wrote in late July that on the Mac Studio the software layer decides the speed, not the hardware alone. Rapid-MLX is a fresh, nameable instance of exactly that: an engine at version 0.11.1 whose documentation carries the date 28 July 2026, and which puts concrete numbers against Ollama rather than claiming to be faster in the abstract.

What Rapid-MLX is

Rapid-MLX builds directly on MLX, Apple's own machine-learning framework, with no detour through llama.cpp and no Metal shim. That is the technical heart of the claim: pure MLX kernels, tuned for the unified memory of Apple Silicon.

For daily work the interface matters more than the architecture. Rapid-MLX speaks the OpenAI and Anthropic APIs, at addresses like /v1/chat/completions. A program that talks to a cloud model today can be pointed at a local model by changing the endpoint and nothing else. The developer names Claude Code, Cursor and Aider as tools that work this way with no adaptation. By the project's own account the engine covers more than 200 model variants, among them Qwen3.5 and 3.6 and Gemma 4.

Installation runs through a script, a one-liner in the terminal. That lowers the bar for a test, but it also shifts responsibility. Running a script from the internet blind, on a work machine with access to company data, is a decision, not a reflex.

The numbers, and who measured them

On a Mac Studio M3 Ultra with 192 GB the developer reports, among others, these figures against Ollama:

• Qwen3.5-9B: 66 versus 16 tokens per second, roughly four times.
• Qwen3.5-4B: 95 versus 38 tokens per second.
• Devstral-24B: 31 versus 13 tokens per second.

There is a second number that often matters more in tool work than raw throughput: time to first token. On repeated requests within the same conversation, a mechanism called DeltaNet is said to cache the state and cut that wait from about 1.5 seconds to under 200 milliseconds. Cached, the project cites 0.08 seconds.

Now the honest part. These figures come from the developer, measured on the developer's own machine. In the discussion where he proposes the engine as a backend for a well-known coding tool, there is to date not one independent reply, no counter-test, no confirmation from a third party. That does not make the numbers wrong. It makes them a vendor claim, and you read a vendor claim differently from a checked measurement.

For a business, in practice: a fourfold headline does not mean your task runs four times faster. The biggest jump shows on the smaller model, in the best case. How much of it survives on your model, your context length and your hardware is a question only a test on your own machine can answer. Ollama measures on the same box in a few minutes, and that is the comparison that counts.

What actually matters for a business

Raw speed is rarely the bottleneck. Firms adopting local AI trip over operation, maintenance and reliability far more often than over tokens per second. Three points weigh heavier than throughput on a project this young.

First, maturity. Version 0.11.1 is an early number. Ollama and llama.cpp have years of bug reports, packages and guides behind them. A newer tool can be technically better and still have a rough edge at the wrong moment, one with no known fix yet.

Second, support. Behind Rapid-MLX there is visibly a single person. That is no criticism, plenty of good tools start that way. But for a firm wiring a system into its workflow, it is fair to ask who fixes bugs and keeps models current a year from now.

Third, security. The advantage of a local engine is that no request leaves the building. That only holds if the tool does what it promises. A server that opens an OpenAI-compatible endpoint on your network wants locking down, or the supposedly private setup becomes an open door inside your own network. The gain in sovereignty we describe under local AI rests on clean operation, not on a benchmark.

When switching makes sense, when it does not

Two camps, one clear recommendation.

For development and experimentation on a single Mac, Rapid-MLX is worth an afternoon. The OpenAI-compatible API makes the switch nearly free: change the endpoint, run your own tool against it, measure the same task against Ollama. If the difference is large on your machine, you have gained speed for nothing. If it is small, you learned that in an hour and you stay with what works.

For production, with colleagues or customer processes depending on it, the timing is too early. Here a setup that has run stable for months, with someone to call, is worth more than the tallest bar in a chart. Keep your current engine as a fallback until the new one has proven itself over weeks of real use.

And the underlying question stays the same as with any model story: does local AI pay off for you at all, measured against confidentiality and request volume, or is the cloud cheaper for your pattern. If you want to work through that trade-off once, cleanly, with your own numbers rather than being pushed by a benchmark, start with a pilot project. We build the setup with you and measure it against your task, not against someone else's bars.