Running an AI agent that actually learns your workflows, remembers past sessions, and improves on its own — without sending a single token to a cloud provider — has moved from theoretical to practical with Ollama 0.21. As the official Ollama account posted on X: "Ollama 0.21 includes support for Hermes Agent, the self-improving AI agent built by @NousResearch."
This is a meaningful shift. Until now, running capable AI agents on local hardware meant stitching together inference servers, memory layers, and tool-calling wrappers — each requiring separate maintenance. Ollama 0.21 and Hermes collapse that stack into a single, manageable setup that any technically literate team can operate.
What Hermes Agent Actually Does
Hermes Agent is an open-source AI assistant built by Nous Research around a closed learning loop. The core idea: every completed task generates a reusable skill — a Python script the agent saves and refines over time. Unlike a static chatbot, Hermes gets more useful the more you use it.
Key capabilities according to the project documentation:
- Agent-curated memory: Hermes maintains full-text search over past conversations, with LLM-based summarization for cross-session recall. You don't need to re-brief the agent each morning.
- Autonomous skill creation: After a complex task, Hermes automatically writes reusable scripts — a template parser for your invoice format, a summary template for your meeting notes, a data cleaner for your CSV exports.
- Parallel subagents: Large tasks can be split across isolated sub-agents running simultaneously, reducing latency on multi-step workflows.
- Multi-platform access: Hermes connects to Telegram, Slack, WhatsApp, Discord, Signal, and email. Your team can interact with the agent through whatever they already use.
- Flexible deployment: Local machine, Docker, SSH, or serverless options (Daytona, Modal) — the same Hermes instance can run on a Mac Studio in your office or a small VPS.
All API calls go exclusively to the LLM provider you configure. With a local Ollama instance as the backend, no data leaves your hardware.
How the Ollama 0.21 Integration Works
The integration is technically straightforward. Hermes points to Ollama's local endpoint (http://127.0.0.1:11434/v1), which exposes an OpenAI-compatible API. From Hermes's perspective, your local Ollama server looks identical to any cloud LLM provider — except everything stays on-premises.
Ollama 0.21 standardizes this connection: model selection, context management, and streaming are handled directly through the Ollama CLI. The Hermes installer no longer requires manual API configuration. One command launches the integration:
ollama launch hermesModel requirements: Hermes requires a model with at least 64,000 tokens of context window to maintain multi-step tool-calling workflows in memory. Models available through Ollama that meet this requirement:
- Gemma 4 27B — our current recommendation; around 60 tok/s on Mac Studio Ultra, strong multilingual performance and reliable tool-calling
- DeepSeek-V3 (quantized) — reported by community practitioners to excel at code generation and structured output tasks
On a Mac Studio M3 Ultra with 64–192 GB unified memory, Gemma 4 27B runs at production-suitable throughput. For teams with tighter budgets, Gemma 4 12B on a Mac Mini M4 Pro (24 GB) covers the majority of standard office automation tasks.
The Privacy Case: Why Local Agents Matter for European SMBs
European businesses operating under GDPR face a specific challenge with cloud AI: every document you send to a third-party LLM provider potentially triggers obligations around data processing agreements, cross-border data transfers, and consent management.
A fully local agent stack sidesteps these concerns at the architecture level. Based on our reading of GDPR requirements:
- No personal data leaves your server — no training on your data, no third-party logging of inputs or outputs
- Data Processing Agreements are simplified — when there's no external processor, Article 28 obligations toward your AI provider don't apply
- Full auditability — all inputs and outputs can be logged locally and produced on request for compliance audits
This matters most for use cases that are risky in the cloud:
- Customer communications containing personal details
- HR workflows touching employee data (payroll, onboarding, scheduling)
- Contract review and legal document summarization
- Internal knowledge bases built from project files and client records
For professional services firms — law firms, accountants, healthcare practices, engineering consultancies — local processing isn't a preference, it's increasingly a compliance baseline.
The EU AI Act adds another layer: deploying a self-improving agent in a business context may qualify as a "limited risk" or "general purpose" AI system depending on its use. Running locally, with your own models and documented audit trails, puts you in a significantly stronger compliance position than relying on a cloud agent API. This is based on our reading of current EU AI Act guidance and should not be treated as legal advice for your specific situation.
Setup Guide: Local Agent Stack in Under an Hour
The setup on macOS is now genuinely approachable:
# Update Ollama to 0.21+
brew upgrade ollama
# Install Hermes (Nous Research installer script)
curl -fsSL https://hermes-agent.nousresearch.com/install.sh | bash
# Pull your model
ollama pull qwen2.5:72b
# Start Hermes connected to your local Ollama instance
ollama launch hermesAfter initial configuration, Hermes presents a terminal UI. The optional messenger integration — Slack, WhatsApp, Telegram — is configured through Hermes's YAML config file and takes minutes to set up.
For production deployments requiring higher availability, Hermes supports Docker natively. Running Ollama and Hermes as containers on a dedicated Mac Studio or a small Linux server gives you a persistent agent endpoint the rest of your team can query.
Linux and WSL2 setups follow the same pattern. Hermes also runs on Android via Termux for teams with specific mobile requirements.
What Hermes Can Automate Today
Document processing: Hermes extracts structured data from incoming invoices, purchase orders, and contracts, then passes it to downstream systems via CSV export or direct API calls to your CRM or ERP.
Internal knowledge search: After each research session, Hermes stores a searchable summary. Teams that repeatedly look up the same internal information — product specs, pricing tables, project histories — stop paying the retrieval cost each time.
Scheduled reporting: Via cron jobs, Hermes generates regular summaries — daily email digests, weekly project status reports, monthly KPI overviews compiled from raw data sources.
Code assistance: For development teams, Hermes acts as a local code agent: reviewing pull requests, generating documentation, running debugging workflows — all without cloud API costs.
Multi-step research: Hermes can spawn parallel subagents to research multiple topics simultaneously and synthesize findings into a single structured output. Useful for competitive analysis, supplier comparisons, or technical due diligence.
Current limitations: Complex workflows require capable hardware — 32–64 GB unified memory minimum for 70B-class models. The autonomous skill creation feature is still maturing; auto-generated skills should be reviewed before production use. Multi-agent orchestration, while powerful, requires configuration time upfront.
Cost Comparison: Local Agent vs. Cloud Subscription
Cloud-based agent platforms (OpenAI Assistants API, Anthropic Claude Teams, Azure AI Studio) typically cost a 10-person team between €200 and €600 per month at moderate usage. At high volume, costs scale further with each token.
A local stack — Mac Studio M3 Ultra 64 GB (~€4,500 new, ~€3,500 used), plus electricity (~€40–60/month for sustained operation) — reaches break-even against cloud agent subscriptions in roughly 9–15 months by this comparison. Teams building on existing Mac hardware reach break-even earlier.
More importantly, local agents scale without incremental cost. Additional queries, new team members, higher automation volume — none of it changes your monthly operating cost.
For a detailed comparison of cloud vs. on-premises total cost of ownership, see our local AI for business page. For the data governance and sovereignty angle, see our guide on data sovereignty.
Getting Started with a Structured Pilot
Ollama 0.21 and Hermes represent the most accessible entry point into real local AI automation we've seen. But the path from installation to productive use involves decisions: which model fits your tasks, what hardware makes sense for your volume, which process is the right first candidate for automation.
Freshlab helps European SMBs through that decision — from pilot scoping to production integration. Start with a free consultation or review how we've structured local AI pilot projects for other businesses.