Run Your Coding Assistant Locally: Source Code Stays In

A coding assistant sees more of a company than almost any other tool. It reads the source code, the comments, the config files, sometimes a credential that slipped into the repository by accident. Run that assistant in the cloud and you are shipping exactly that to a third party. For a lot of teams, this is the point where they hesitate.

The good news: the assistant no longer has to let the code out. Last week we described how to stand up a local AI for many concurrent users with vLLM or Ollama. That was the server side. This time we look at one concrete use for it, programming, and at whether a locally run assistant is actually enough day to day. What is new this summer is mainly that the open coder models took a jump and the tooling around them settled down.

Why source code is a special case

With a customer-service chatbot you can argue about how sensitive the inputs are. With source code the case is clearer. It is often the company's core asset, it holds the business logic, and many client contracts state in plain terms that it must not go to third parties. A law firm, a payment processor, or a supplier holding someone else's design knowledge simply cannot afford it.

Then there is data protection. Test data, error messages and comments often carry real personal data. Send them to a provider outside the EU and the transfer becomes a matter you would have to justify. A model running locally sidesteps the question, because the content never leaves the machine.

What runs locally today

The setup has two parts: a model and a tool that wires it into your work.

For code, one family has taken the lead. Qwen3-Coder-Next, an open model from Alibaba's Qwen team, works with roughly 80 billion parameters according to the published specs, of which only about 3 billion are active per token. That design lets it fit on a single workstation while still coming, by reported accounts, close to the far larger 480 billion parameter variant. With less hardware you reach for smaller coder models in the 7 to 32 billion range.

For the tool there are a few paths, depending on how you work:

• Ollama as a local model server with an OpenAI-compatible endpoint. The model sits ready as a service, and a call like aider --model ollama/qwen2.5-coder:32b is enough for a fully local terminal agent.
• Continue as an editor extension. It points at any OpenAI-compatible server, local or self-hosted, and brings completion and chat straight into VS Code or a JetBrains IDE.
• Tabby as its own self-hosted assistant server. It indexes the repository, runs in VS Code, JetBrains and Vim, and comes with admin controls, which makes it interesting for teams.

None of the three needs a cloud when you run it purely locally. The choice comes down to whether you prefer the terminal, the editor, or one central server for the whole team.

How good is it really

Honestly: very good for the daily small stuff, not quite at cloud level for the very largest tasks, but the gap is shrinking.

Autocomplete, explaining unfamiliar code, writing tests, smaller refactors, open coder models are reported to handle these well today. On big agentic tasks, meaning a change that spans many files and plans several steps on its own, the biggest cloud models still lead a little. As a rough marker, the community cites around 70 percent on the SWE-Bench Verified test for Qwen3-Coder-Next, a figure to read as a signpost, not a guarantee.

On speed: a small 7 billion parameter model runs even on a plain CPU with 8 gigabytes of memory according to community measurements, there at about 10 to 15 tokens per second. Enough for completion, sluggish for longer answers. With a GPU or Apple Silicon carrying plenty of unified memory it turns fluid.

The trap in the word local

Precision pays here, because local is not automatically private. Several of the tools above have introduced optional, sometimes paid cloud tiers. With those, individual features do route through the vendor's servers, even though the tool is known as a local one.

In practice that means the brand does not decide confidentiality, the specific feature you switched on does. A model sitting on your machine can be private while an add-on feature of the same program sends your request outward. Before you trust a setup with confidential repositories, check for each active feature whether it computes locally. A simple test helps: cut the network and see what still works. Whatever keeps running offline is running locally.

Who this pays off for

Clearest for teams whose code cannot leave for contractual or legal reasons, so finance, critical infrastructure, law firms, and anywhere someone else's operating knowledge sits in the repository. For them the local assistant is not a compromise, it is the one option that never puts confidentiality at risk in the first place.

Getting started is smaller than it sounds. One model, one tool, one workstation, and a scoped repository as the first test. In a few days you see whether quality and speed carry your work, without a single snippet leaving the building. For how a local setup looks with data sovereignty in mind, see our note on data sovereignty.

Want to know whether a local coding assistant fits your team? In a pilot project we set up model, tool and data flow on a real repository and measure quality and speed before you commit.