Teams that invest in a Mac Studio for local AI often face the same surprise: benchmarks from the internet rarely match what they see in practice. A 32B model runs noticeably slower than expected after setup. Switching frameworks adds 40 percent throughput with zero hardware changes. The machine is the same. The software is not.
AI researcher rewind02 on X put it plainly in a widely shared post earlier this month. Running a head-to-head comparison of MLX, Ollama, llama.cpp, and the new vllm-mlx on the same Mac Studio M3 Ultra, they concluded: "Performance isn't about the hardware, it's about three software layers stacked on top of it." The differences were substantial, and the machine never changed.
This article breaks down what those three layers are, how each inference engine positions itself, and how European SMBs can make the right stack choice from the start.
The Investment Question
A Mac Studio M3 Ultra with 192 GB Unified Memory carries a list price of around 4,999 to 5,999 euros. For many businesses, that is a calculable one-time spend. Teams paying for cloud API access at scale can easily reach several hundred euros per month in API fees. Over two to three years, those recurring costs frequently exceed the hardware investment.
The question that matters most is not whether to buy the hardware. It is whether the software stack on top of it delivers the throughput needed to make the machine useful. Based on practitioner reports, token generation rates for a 32B model can differ by a factor of 2 to 4 on identical hardware, depending on the inference engine used. That difference decides whether five colleagues can use the system comfortably or whether it buckles under the load.
The Three Software Layers
What actually determines how fast a local LLM runs on Apple Silicon? Three layers interact:
Layer 1: The Inference Backend
This is the lowest layer, the one that executes tensor operations and talks directly to the hardware. Four frameworks currently compete here:
- MLX: Apple's own machine learning framework, which delegates operations like matrix multiplication directly to the GPU and Neural Engine of the M-series chip. Community benchmarks report the highest raw throughput for single requests on Apple Silicon.
- llama.cpp: A C++ implementation with strong support for quantisation and CPU offloading. Relevant when a model is too large to fit entirely in Unified Memory.
- vllm-mlx: A newer project that pairs vLLM's Continuous Batching algorithm with the MLX backend. According to practitioner reports, it delivers meaningfully higher aggregate throughput in multi-user scenarios by processing requests in efficient batches.
Layer 2: The API Server
Ollama is the best-known example of this layer. It wraps an inference backend in an OpenAI-compatible REST API and handles model switching, context windows, and parallel connections. Depending on configuration, Ollama uses either MLX or llama.cpp underneath. The API server layer adds convenience: a consistent interface, automatic model management, and broad client compatibility.
Layer 3: The Client Application
Open WebUI, LM Studio, direct API calls, or a custom internal application, this is the layer users interact with daily. Misconfigured prompt formats, unnecessarily large context windows, or inefficient serialisation can introduce hidden performance losses that look like hardware problems at first glance.
MLX, Ollama, llama.cpp and vllm-mlx Compared
MLX direct delivers the highest raw throughput for single requests on Apple Silicon, according to community benchmarks. Apple can route matrix operations to the Neural Engine natively. For a solo developer or researcher who wants maximum speed without API overhead, MLX direct is a strong option. The setup is more involved than Ollama.
Ollama is the standard entry point and the right choice for most SMBs. Installation takes minutes, compatibility with clients like Open WebUI or LibreChat is broad, documentation is good, and the project ships updates regularly. The slight performance overhead compared to MLX direct is acceptable for most use cases.
llama.cpp is the first choice when models exceed the available Unified Memory. 70B variants on a Mac Studio with 64 or 96 GB benefit from mature CPU offloading. Configuration is more complex, but the payoff for large-model scenarios is clear.
vllm-mlx is the newcomer attracting attention in the community. It combines vLLM's Continuous Batching scheduler, which assembles multiple incoming requests into a single optimised processing batch, with Apple's MLX framework. Practitioner reports indicate it outperforms Ollama at five to twenty concurrent users, where the batching advantage translates directly into higher overall throughput.
Saiyam Pathak captured the current state of the ecosystem on X: "The Local LLM space in 2026 is where Kubernetes was in 2017. Lots of tools. Confusing landscape. Best practices still emerging." Teams that invest in understanding the stack now are building a compounding advantage over those that wait for the dust to settle.
Choosing the Right Stack
For a solo developer or a small group with occasional usage: Ollama with the standard MLX backend. Easiest installation, widest client compatibility, sufficient performance for this scenario.
For a team of 5 to 20 people sharing the same inference endpoint: vllm-mlx as a shared team server, paired with Open WebUI or a similar frontend. The batching advantage pays off measurably here.
For models larger than 70 billion parameters on hardware with limited Unified Memory: llama.cpp with a correctly configured offloading setup. The initial configuration effort is worth it.
Framework updates ship on roughly a two-week cycle. An Ollama release can shift throughput on specific models by 20 to 30 percent. It is worth running a quick sanity check after major updates.
Privacy, Compliance and Funding
Choosing local AI in Europe is often privacy-driven: no data leaves the premises, no data processing agreement with a cloud vendor required, no exposure to changes in third-party service terms. That holds regardless of which inference framework you choose.
What the framework choice does affect is whether the investment is economically justified. A stack that delivers too little throughput means either buying more hardware for the same result, or poor user adoption because response times are too long. Both undermine the business case for local AI.
For European SMBs looking to reduce upfront cost: local AI infrastructure investments may qualify for support under regional innovation voucher schemes or national digitalisation grant programmes. Eligibility and amounts vary by country and project scope. Our pilot project programme includes an initial assessment of available funding options in your jurisdiction.
For the technical foundations of the MLX stack and a comparison with Rapid-MLX, see our article Rapid-MLX: fastest open-source inference on Apple Silicon. Teams setting up a shared inference server will find the right starting point in vLLM and SGLang as a local team server.
Ready to run a proper stack evaluation for your setup? Start a pilot project or get in touch to discuss the right architecture.