"Which customers haven't placed an order in the last 90 days?" Sounds simple. In practice, answering it means either writing SQL yourself, asking a developer to pull the number, or waiting for the quarterly BI report. Practitioners on X have been demonstrating a fourth option with increasing frequency: a local LLM running on-premises, translating natural language into SQL and executing it directly against the ERP database — with no data ever leaving the company network.
How Text-to-SQL Works with a Local LLM
The workflow is straightforward. A business user types a question in plain language. A locally-running model converts it into SQL. The query runs against the ERP database. The result comes back, formatted as a readable summary or a structured table.
Four steps, all local:
- Natural language input: "Show me all open invoices above €5,000 from this quarter."
- SQL generation: The LLM produces a
SELECTquery tailored to your specific database schema. - Database execution: The query runs against the local ERP backend — SAP Business One, Odoo, or a DATEV export file.
- Result formatting: The model renders the output in readable language or as a structured table.
No API keys, no cloud subscription, no data transmission — the entire pipeline runs on hardware the company owns.
Which Models Work Best?
Not every local model is equally capable at structured data queries. The local AI community has converged on several strong options:
SQLCoder by Defog.ai is an open-source model fine-tuned specifically for text-to-SQL. It runs via Ollama and, as reported by community practitioners, significantly outperforms general-purpose models of similar parameter counts on typical ERP query patterns — multi-table joins, aggregations, date-range filters. On Apple Silicon hardware with 192 GB unified memory, community benchmarks report 20–40 tokens per second for the 15B variant, which is sufficient for interactive use.
Qwen 2.5 32B and above delivers strong SQL reasoning even for complex queries involving many tables or nested conditions. Based on reported community measurements, 14B models handle basic reporting tasks reliably — revenue by period, top customers, open receivables — while 32B+ is recommended for complex analytical workloads.
Llama 3.3 70B is worth considering if your team queries in multiple languages. This is relevant for international SMBs running German, Spanish, and English queries against the same ERP instance.
ERP Compatibility: What's Possible Today
The three most common ERP platforms in European SMBs have different technical access points:
SAP Business One
SAP B1 runs on either HANA or Microsoft SQL Server as its database backend. Both are accessible via standard ODBC connectors or direct SQL clients. The primary setup task is writing a schema description: documenting your key tables and columns so the LLM understands what it's querying. In practice, this is a one-time investment of one to two days of work.
Odoo
Odoo uses PostgreSQL — one of the most extensively documented relational databases available. Because Odoo's data model is publicly documented, building the schema context for a local LLM is faster here than with proprietary systems. SQLCoder-based setups work well with Odoo from the start.
DATEV and Accounting Exports
For companies using DATEV (prevalent in German-speaking markets) or similar accounting platforms without direct database access, the practical approach is to export reports as CSV or XLSX on a scheduled basis, load them into a local SQLite database, and point the LLM at that. Zero cloud contact — financial data stays within the company's own environment.
Why GDPR Makes This More Than a Convenience Feature
ERP systems hold some of the most sensitive business data there is: customer payment histories, employee compensation records, strategic supplier pricing, confidential inventory levels. When you send any of this to a cloud-based AI system — even as part of a prompt — you're transmitting it to third-party infrastructure governed by that provider's terms of service and the law of wherever their servers sit.
Based on our reading of GDPR — particularly Article 5(1)(f) on the principle of integrity and confidentiality, and Article 32 on appropriate technical and organisational measures — that transmission is precisely where legal risk accumulates. Third-party terms, potential training data use, and extraterritorial jurisdiction claims do not apply when processing stays entirely on-premises.
A local LLM removes these risks structurally. If nothing leaves your network, there is nothing to intercept. What is processed on your own hardware is subject to no external jurisdiction. This is not a marketing claim — it is the technical reality of local AI infrastructure.
We explore the topic of data sovereignty in depth at /data-sovereignty.html.
A Practical Setup Overview
A production-ready text-to-SQL implementation for an SMB typically involves a small set of components:
- Ollama as the local model server (runs on Windows, macOS, Linux)
- SQLCoder or Qwen 2.5 as the LLM (served via Ollama)
- Open WebUI as the chat interface, with a database connector plugin
- Schema description as system context (written once, updated as the schema evolves)
- SQL connector to SAP B1, Odoo, or a local SQLite file for accounting exports
The standard implementation approach: start with a pilot covering two to three of your most frequently queried tables and 10–15 representative test queries. Evaluate accuracy before expanding to the full schema. This controlled rollout typically takes two to four weeks.
What a pilot process covers in detail — including evaluation criteria and quality thresholds — is outlined at /local-ai.html.
What Text-to-SQL Replaces — and What It Doesn't
This approach is not a replacement for full BI tooling like Power BI or Tableau. It is a complement — and in many cases a substitute for the informal "can you pull that number for me?" requests that tie up developers and analysts every day.
The business users who benefit most are the ones with time-sensitive, ad-hoc information needs: the sales manager who wants to know which accounts are going cold before a meeting, the operations lead who needs a quick breakdown of supplier delivery times by product category, the executive who wants to see the last three purchase orders before a vendor call.
For these use cases, local text-to-SQL is often exactly the right tool — precisely because it requires no dashboard-building project, no BI specialist on call, and no IT approval process. A question typed in plain language, answered in seconds, on hardware that stays in the building.
If you'd like to explore what an ERP-connected local AI setup could look like for your business, let's talk about a pilot project.