DGX Spark Clusters: Three Nodes Without a Switch, Four With

5. Aug 2026 English 17 min read Also in: Deutsch, Español

nvidia-dgx-spark local-llm hardware

A short switch-buying guide for DGX Spark clusters has been making the rounds on X since 4 August. It names three MikroTik models and assumes a switch becomes necessary beyond three units. That is the right shape of the answer, but it leaves out what a business needs first: what actually changes between two, three and four units, and what the money buys.

In late June we wired two DGX Spark units into a cluster step by step. This piece picks up where that one stopped: three units, four units, eight units, then the counter-check against a Mac Studio and a worked example with 25 concurrent users.

What NVIDIA officially supports

The binding answer is not in a forum post but in the cluster assistant in NVIDIA Sync. It covers two to a maximum of four DGX Spark units and describes three paths:

Two units: one cable, port to port.
Three units: three cables as a ring, each unit connected to the other two.
Four units: one cable per unit through a switch. Four units require a switch.

There is one rule that is easy to skip past: do not mix. Either connect every unit directly or run every unit through the switch. Two units cabled directly plus a third hanging off a switch is not a supported layout.

Going beyond four units leaves the ground this tool covers. Nothing technically forbids it, it is still ordinary Ethernet with RoCE, but the guided setup ends there, and so does the easy way back when something misbehaves.

Three units is the quiet spot

Three DGX Spark units give you 384 GB of shared memory, need three cables and add no extra box to the rack. No switch means no second firmware, no second power supply, no additional part that can fail at three in the morning. For a model that does not fit in 128 GB but does fit in 384 GB, this is the simplest shape available.

Know the arithmetic behind it anyway. One unit has roughly 200 Gb per second in total, spread across two PCIe Gen5 x4 paths. In a ring, two neighbours share that budget. Between two directly connected units it belongs to a single link, in a ring it does not. So plan for roughly half the per-neighbour bandwidth of the two-unit pair. NVIDIA publishes no throughput figures for the ring, which leaves your own measurement as the only honest source.

The 200 Gb no single stream will ever see

This is where most of the time gets burned in practice, and it deserves more than half a sentence.

ServeTheHome measured the setup and calls it genuinely unusual for a 200GbE link: behind the QSFP cages sit two 100 Gb MACs on two separate PCIe Gen5 x4 paths. They measured 92 to 98 Gb per second per path with RoCE, roughly 190 combined. With jumbo frames and 60 to 64 parallel streams the result was 160 to 198 Gb per second depending on direction. The number on the box is reachable, but it is a sum, not something a single connection will ever show you.

A field report on the NVIDIA developer forum shows the same effect from the other side. A four-unit cluster there runs on a MikroTik CRS812 with breakout cables. At first a single TCP stream reached about 30 Gb per second and parallel streams around 106, even though the link had negotiated at 200GbE. Only when the operator realised that the one physical port appears as two logical interfaces and both have to be driven, then set MTU 9000 and disabled IPv6, did the pair settle at roughly 196 to 198 Gb per second. An NCCL run then reported about 23.76 GB per second of bus bandwidth. Along the way he found a stale NCCLIBDISABLE=1 in the config that had been blocking RDMA outright.

Four practical points follow.

A single stream ends near 100 Gb per second. That is not a misconfiguration, that is the shape of the hardware. Measure with iperf3 and no parallelism and you will see half of your capacity, then spend days hunting a fault that does not exist.

The advertised figure assumes parallelism. Both logical interfaces have to carry traffic, otherwise one of the two PCIe paths sits idle. For distributed inference NCCL handles that when it is set up correctly. For any hand-rolled data transfer, nobody does.

One more cable does not raise a unit's budget. The second cage only splits it differently. That is why the second cage exists for topology rather than for speed, and why a ring halves the bandwidth per neighbour.

Measure three numbers before a model ever touches the machine. First, iperf3 with a single stream, that is your floor. Second, iperf3 with several parallel streams and MTU 9000, that is your ceiling. Third, an NCCL run across all nodes, because only that shows whether RDMA is actually live. A link that says 200 on the datasheet and delivers 30 in the first test is the normal state before configuration, not a defect.

From four units: which switch

Here the guide on X is useful, because it places the three models correctly. We checked the claims against the vendor pages.

MikroTik CRS504-4XQ-IN, four 100 Gb ports, street prices roughly 640 to 780 US dollars. Four ports means four units, which is exactly the ceiling of the official cluster assistant. The guide cites user reports of about 98 percent of the performance of far more expensive switches. That figure is unverified, but it is plausible: if a single stream ends at 100 Gb anyway, a 100 Gb port costs less performance than the spec sheet suggests.

MikroTik CRS804-4DDQ-hRM, four 400 Gb ports in QSFP56-DD form, list price 1,295 US dollars. It has no native 200 Gb ports. You split each 400 Gb port into two 200 Gb links with breakout cables, four times two gives eight units. This is the cleanest route if eight is the target.

MikroTik CRS812-8DS-2DQ-2DDQ-RM, also 1,295 US dollars, with two 400 Gb ports, two native 200 Gb ports and eight 50 Gb ports. Two breakouts give four 200 Gb links, plus the two native ports that is six units at full width. For eight you split one of the 400 Gb ports into four times 100 Gb instead, which leaves four units at 200 Gb and four at 100 Gb. This is the exact switch in the field report linked above, so the path has been walked.

One correction of our own: our June guide described the CRS504 as a 200GbE switch for four to eight units. Both parts were wrong. It has four 100 Gb ports, which caps it at four units. We have fixed the passage.

What a cluster actually buys: memory, not speed

The most important number is not on any switch datasheet. StorageReview benchmarked a two-unit cluster running distributed inference and reports roughly 464 to 505 tokens per second on GPT-OSS-120B depending on the vendor of the unit. Their conclusion is blunt: clustering mainly buys memory capacity, not throughput, and the fabric between the units stays the dominant bottleneck.

The reason lies in how the work is split. Under pipeline parallelism, the usual arrangement, each unit holds a slice of the layers. Every token then travels through every unit, one after another. The interconnect sits in the path of each individual token, not just in the model load. Large batches hide part of that cost because the pipeline stays full, but they do not remove it. Two units double the memory, not the throughput.

Mixture-of-experts models add a second wrinkle that is easy to miss. MoE saves compute, not memory. Gemma 4 26B activates only four billion parameters per token, but all 26 billion have to sit in memory, because the router does not know in advance which of the 128 experts the next token will need. Google states exactly that in the model overview. And with many concurrent users the compute advantage shrinks too: every user hits different experts, so across a batch of 25 requests the union of activated experts is far larger than for a single one. Memory traffic therefore approaches the full model precisely when many people are working at once.

That changes the purchase decision. A DGX Spark has listed at 4,699 US dollars since February 2026, up from 3,999. Three units are therefore over 14,000 US dollars plus cables before a single request has been answered. Spending that to make a model faster that already runs in 128 GB is spending past the goal. The first question is always whether the model you actually need fits into one unit. If it does, the second box is misplaced money. If it does not, the cluster is not luxury, it is the precondition for running locally at all.

Counter-check: Mac Studio Max and Ultra

For local inference the Mac Studio has been the obvious alternative, and on paper it has a strong argument. An M4 Max sits at roughly 546 GB per second of memory bandwidth, an M3 Ultra at roughly 819. The DGX Spark comes in at about 273. For a single user waiting on an answer, that number translates directly into tokens per second. We covered this in July in a head-to-head of the two machines, and nothing has changed technically.

What has changed is the market. Because of the shortage in memory chips, Apple has removed the large configurations: the 512 GB option in March 2026, then 128 and 256 GB as well. Today a Mac Studio with M3 Ultra can only be ordered with 96 GB, from 3,999 US dollars, and Macworld reports a 13 to 14 week lead time with no stock at the large retailers. A DGX Spark with 128 GB ships. That is not a technical argument, but it is the argument that decides a project in the third quarter.

For a Mac cluster there is also the interconnect question. Apple links its machines over Thunderbolt 5, with RDMA through JACCL since macOS Tahoe 26.2, which in our own coverage and community measurements lands at 50 to 60 Gb per second. Three Mac Studios at 96 GB each do add up to 288 GB, but on roughly a third of the fabric bandwidth two DGX Spark units have between them. And since a cluster mainly buys memory anyway, that is exactly where it hurts.

Then there is the software. For a single user, MLX is mature and fast. For many concurrent users, the MLX stack still lacks an equivalent to PagedAttention and to the request scheduling in vLLM. A paper on inference at scale on Apple Silicon measures a factor of 2.6 between one and 16 concurrent requests for Qwen3-8B on an M4 Max with 128 GB, and names memory bandwidth saturation as the ceiling for larger models. That is respectable, but it is not the scaling a CUDA stack with continuous batching delivers.

Short version: one user, long answers, no fine-tuning, that is Mac territory, provided you can wait for delivery. Many users, long inputs, fine-tuning or a Linux production environment, that is Spark territory.

Worked example: Gemma 4 26B MoE for 25 concurrent users

Let us make it concrete, because abstract advice is worth nothing at this point.

Gemma 4 26B A4B has 26 billion parameters, activates four billion per token across 128 experts, and works with a 256K context. Google officially puts the weights at 57.7 GB in BF16, 28.8 GB at 8 bit and 14.4 GB at 4 bit.

That settles the cluster question on its own. In BF16 the model occupies a good 58 of the 128 GB on a single DGX Spark. Roughly 70 GB remain for KV cache, activations and the operating system, and with 25 sessions the KV cache is the line item that matters. For this model and this user count you need no second box, no third, and certainly no switch. At 8 bit it fits with room to spare, which leaves headroom for longer contexts or a second model alongside.

On a Mac Studio with the 96 GB you can order today, BF16 also fits, but the margin is thin: macOS reserves part of the shared pool, and 25 sessions with long contexts eat precisely what is left. Here 8 bit is not an option, it is the sensible setting.

For the 25 users, three things then decide the outcome, and bandwidth is only one of them.

Prompt processing. 25 users each sending a page of context or a RAG excerpt generate compute load, not memory traffic. That is the discipline where the Spark's Blackwell GPU holds its lead and where the Mac's bandwidth helps least.

KV cache management. 25 sessions mean 25 growing caches of different length and lifetime. PagedAttention exists for exactly that. Without an equivalent, the failure mode is not bad numbers, it is rejected requests once memory fragments.

The expert effect above. At batch size 25 memory traffic approaches the full model and the benefit of four active billions shrinks. Sizing your deployment on the single-user figure means sizing it too tight.

The recommendation for this case is therefore: one DGX Spark, weights at 8 bit, vLLM with continuous batching, and an acceptance test at 25 parallel requests rather than one. And if one unit cannot carry the load, the answer is still not a cluster, it is a second copy beside it. The next section explains why.

One honest limitation: the memory figures are official, the conclusion drawn from them is arithmetic. We are not aware of a published benchmark of exactly this combination, Gemma 4 26B on a DGX Spark at 25 concurrent users. What your prompts, your context lengths and your response patterns make of it will only show in a test run with your real traffic.

Three machines side by side instead of in a cluster

When the model fits into one unit and the load still grows, the right answer is not a cluster, it is a replica. These are two shapes for two different problems, and they get confused all the time.

A cluster with pipeline parallelism solves: the model does not fit into one unit. Memory is pooled, every token travels through every node in sequence, throughput does not improve, and if one node fails the whole thing stops.

Replicas behind a router solve: the model fits, but the load does not. Each unit holds the complete model and answers requests independently of the others. Throughput grows almost linearly with the number of units, a failure costs a third of your capacity instead of all of it, and updates can be rolled one machine at a time without stopping the service.

For Gemma 4 26B with 25 users, the second shape is the right one as soon as a single unit runs short. It needs no 100 or 400 Gb switch, no QSFP cable and no NCCL. Only requests and responses travel between replicas, never intermediate results, so the ordinary office network carries it.

LiteLLM is the obvious layer for this: one OpenAI-compatible endpoint in front, three vLLM instances behind it, routing strategies such as least-busy or latency-based, automatic fallback chains when an instance stops answering, plus keys, budgets and cost attribution per team. That is the infrastructure a business needs for local models anyway, regardless of how many units it runs.

One detail here quietly burns a lot of compute, and it appears on no cable list: round-robin destroys prefix cache reuse. When 25 users send the same long system prompt or the same RAG context, the same session should keep landing on the same unit, otherwise the identical prefix gets processed again on every machine. The router in the vLLM production stack does exactly this, session-based, prefix-aware and KV-cache-aware. With short, varied requests it makes no difference. With long shared prefixes it is the difference between processing them once and processing them twenty-five times.

The second machine is often not a twin anyway but carries a different model, embeddings, a reranker or a coding model. Then LiteLLM routes by model name rather than by load, and the cluster was never the question.

Seats are not concurrent requests

Before anything gets ordered, one number has to be settled, and it gets confused constantly: 25 seats and 25 requests in flight at the same time are two entirely different things.

As a rule of thumb, three to eight percent of seats are being served simultaneously at peak. 25 seats therefore produce one or two parallel requests, which one unit covers with room to spare. 500 seats produce roughly 15 to 40, so 25 concurrent requests is a realistic design point for a workforce that size. That case needs different planning.

Three requirements appear at 500 seats that do not exist at 25. Peaks: Monday morning, a company-wide email, a batch job someone kicked off overnight. Size for the average and you produce the queue exactly when everyone is watching. Availability: 500 people on one machine is a single point of failure. N plus one is not a nicety here, it is the reason you buy more than one machine in the first place. Workload variety: 500 people bring coding, long RAG contexts and summarisation, and context length drives the KV cache requirement harder than user count does.

When you split across replicas, a mixture-of-experts model adds an effect that runs against the usual intuition. Gemma 4 picks two of 128 experts per token. Across a batch of B concurrent requests, a given expert is hit with a probability of one minus (one minus 2 over 128) to the power of B:

Batch 8: roughly 12 percent of the experts per layer
Batch 25: roughly 33 percent
Batch 50: roughly 55 percent

Three replicas at eight concurrent requests each therefore read considerably fewer weights per step than one unit at 25. This is back-of-envelope arithmetic assuming uniformly distributed experts, and in practice similar prompts correlate, so the real figure comes out lower. The direction holds regardless: for mixture-of-experts models, several small batches are gentler on memory bandwidth than one large one.

Which leads to the uncomfortable question at this load level: are three DGX Spark units the right purchase at all? At 273 GB per second of memory bandwidth the Spark is built as a development machine, not as a server card. An RTX PRO 6000 Blackwell carries 96 GB of GDDR7 at 1,792 GB per second, about six and a half times as much, and Gemma 4 26B at 8 bit fits comfortably into 28.8 GB of it with plenty left for the KV cache. Since June 2026 it sits at roughly 13,250 US dollars, which is about what three Sparks cost.

So the honest comparison for 25 concurrent requests looks like this:

Three DGX Spark units, roughly 14,100 US dollars. Eight concurrent requests per unit, N plus one at no extra cost, no rack and no server cooling required, and each unit doubles as a development machine. In exchange, the lowest bandwidth per request.
One RTX PRO 6000 Blackwell in a server, roughly 13,250 US dollars plus the host. By far the best response time per request, but a single point of failure for 500 people and 600 watts in the rack. For N plus one, double the figure.

For 500 seats we plan with three replicas and decide between Spark and server card on the response speed you have promised. If reading speed is enough for the user, three Sparks are the calmer and more resilient build. If short waits on long inputs are part of the deal, in a coding tool for instance, the server card is the more honest choice. What helps in neither case is the cluster.

What this means for a business

Cables and switch are the smallest line item in this build, and they still decide whether the investment shows its performance. A cluster running on the wrong MTU with a single stream delivers a third of what was paid for, and nobody notices, because everything works.

The rest is the familiar calculation. When inference runs in house, no request leaves the network, which is the core of what we describe under local AI and, in regulated industries, usually the real reason for this hardware in the first place. Whether that takes one unit or three depends on the model that solves your task, not on a recommendation in a thread.

If you would rather run that comparison on your own numbers than derive it from a cable list, a pilot project is the short path. We measure your workload on one unit before anyone talks about the second.

Sources

Frequently asked questions

How many DGX Spark units can I connect without a switch?

Up to three. Two units take a single cable point to point, three units form a ring with three cables where each unit connects to the other two. From the fourth unit on, NVIDIA's own cluster assistant requires a switch. Mixing is not supported, either every unit is wired directly or every unit goes through the switch.

Which switch fits a DGX Spark cluster?

For up to four units a MikroTik CRS504 with four 100 Gb ports is enough, street prices run roughly 640 to 780 US dollars. If eight units are the goal, take the CRS804 with four 400 Gb ports and split each one into two 200 Gb links with breakout cables. The CRS812 is the fallback when the CRS804 is out of stock, both list at about 1,295 US dollars.

Does Gemma 4 26B MoE need a cluster for 25 concurrent users?

No. The model has 26 billion parameters in total and activates four billion per token. Google puts the weights at 57.7 GB in BF16, 28.8 GB at 8 bit and 14.4 GB at 4 bit. All three fit into a single 128 GB unit with room left for the KV cache of 25 sessions. If one unit cannot carry the load, you put a second copy beside it and balance with LiteLLM rather than building a cluster. Replicas add throughput, cluster nodes do not.

Mac Studio or DGX Spark for many concurrent users?

For a single user the Mac Studio wins, because its 546 to 819 GB per second of memory bandwidth against 273 on the Spark translates directly into tokens per second. At 25 concurrent users the balance shifts to prompt processing and KV cache scheduling, and there the CUDA stack with vLLM is ahead of the MLX stack. On top of that comes the market situation, the Mac Studio is currently capped at 96 GB with a 13 to 14 week lead time.