Hermes Agent on a Mac mini with Ollama

In Progress

A practical guide to running Hermes Agent on Ubuntu while using a GPU-backed Ollama server over Tailscale—without giving the agent general access to the server or local network.

Environment used for this build

  • 2014 Intel Mac mini
  • Ubuntu 26.04.1 LTS
  • Unraid server with an NVIDIA Tesla V100 32 GB
  • Ollama running in Docker on Unraid
  • Hermes Agent running under a dedicated Linux account
  • Tailscale for the private connection between the two machines
  • Nginx as an authenticated, inference-only proxy
  • nftables as the final outbound network boundary

What I wanted to build

I wanted a small, dedicated computer that could run Hermes Agent continuously while sending inference requests to the more powerful GPU in my Unraid server. The important requirement was isolation: Hermes could use the model, but it could not browse Unraid shares, reach the Docker API, access the Unraid web interface, SSH into the server, or scan other devices on my network.

The resulting design uses several independent controls:

  1. Hermes runs as a dedicated, unprivileged Linux user named hermesagent.
  2. The Mac mini reaches Unraid through Tailscale instead of exposing Ollama publicly.
  3. An Nginx proxy requires a bearer token and exposes only model listing and chat inference.
  4. Ollama’s native port remains outside the agent’s allowlist.
  5. An nftables rule restricts every process owned by hermesagent to one destination IP and TCP port.
  6. Hermes starts with a minimal tool configuration and manual approval enabled.

The important distinction is that the firewall and Linux account are the hard security boundaries. Hermes’ internal tool settings are an additional layer, not the only protection.

Architecture

Hermes Agent
  running as hermesagent on Ubuntu
          |
          | Tailscale, TCP 11435 only
          v
Authenticated Nginx proxy on Unraid
          |
          | localhost:11434
          v
Ollama container and local GGUF model

Replace the following placeholders throughout this guide:

For my model, <MODEL_NAME> was:

hf.co/unsloth/Qwen3.8-27B-GGUF:UD-Q6_K_M

1. Prepare Ubuntu on the Mac mini

I first made sure Ethernet, DNS, and package installation were working, then installed the small set of system packages used by the setup:

sudo apt update
sudo apt install -y curl ca-certificates nftables

Useful network checks are:

ip address
ip route
getent hosts tailscale.com

2. Install and connect Tailscale

I installed Tailscale on the Mac mini using the official Linux installer:

curl -fsSL https://tailscale.com/install.sh | sh
sudo tailscale up

The second command printed an authentication URL. After opening it and approving the Mac mini, I confirmed the connection:

tailscale status
tailscale ip -4

Unraid was already connected to the same tailnet. I recorded its Tailscale IPv4 address for the proxy and firewall rules.

3. Create a dedicated Hermes account

I did not install or run Hermes as my normal administrative account. Instead, I created a separate user with its own home directory:

sudo adduser --disabled-password --gecos "" hermesagent
id hermesagent

I recorded the numeric UID:

id -u hermesagent

In my installation it was 1002, but the firewall rule should always use the value returned on the system being configured.

I confirmed the account was not a member of privileged groups:

id hermesagent
sudo -l -U hermesagent

It should not belong to groups such as sudo, docker, lxd, disk, or adm.

4. Configure Ollama for agent-sized context

Hermes requires a large context window for its system prompt, tool definitions, conversation history, and tool results. A 4,096-token Ollama context was too small even for the first message, so I configured Ollama for 64,000 tokens.

In the Unraid Ollama container template, I set the following environment variables:

VariableValuePurpose
OLLAMA_CONTEXT_LENGTH64000Gives Hermes the context window it expects
OLLAMA_NUM_PARALLEL1Prevents parallel requests from multiplying context memory
OLLAMA_FLASH_ATTENTIONtrueReduces memory growth at longer contexts when supported
OLLAMA_KV_CACHE_TYPEq8_0Roughly halves KV-cache memory compared with f16
OLLAMA_MAX_LOADED_MODELS1Prevents multiple large models from competing for VRAM

One practical Unraid detail: each environment-variable key should appear only once in the template. If an advanced entry and a newly added entry use the same key, remove the duplicate and edit the original advanced entry.

After applying the template and recreating the container, I verified the live values:

docker inspect ollama \
  --format '{{range .Config.Env}}{{println .}}{{end}}' |
  grep -E '^OLLAMA_(CONTEXT_LENGTH|NUM_PARALLEL|FLASH_ATTENTION|KV_CACHE_TYPE|MAX_LOADED_MODELS)='

I then loaded the model and checked its actual allocation:

docker exec ollama ollama run \
  'hf.co/unsloth/Qwen3.8-27B-GGUF:UD-Q6_K_M' \
  'Reply with only OK.'

docker exec ollama ollama ps

The CONTEXT column should show 64000. Ideally, PROCESSOR should show 100% GPU; CPU offloading works but reduces performance.

5. Put an inference-only proxy in front of Ollama

Ollama’s normal API includes management operations in addition to inference. I did not want the token available to Hermes to authorize model pulls, deletion, copying, or model creation. I therefore placed Nginx in front of Ollama and exposed only:

First, I generated a random token on Unraid:

openssl rand -hex 32

I saved that value privately as <PROXY_TOKEN>. It should never be committed to a public repository or included in the published version of this article.

I created a directory for the proxy configuration:

mkdir -p /mnt/user/appdata/ollama-inference-proxy
nano /mnt/user/appdata/ollama-inference-proxy/nginx.conf

The Nginx configuration was:

worker_processes auto;

events {
    worker_connections 1024;
}

http {
    map_hash_bucket_size 128;

    map $http_authorization $ollama_proxy_authorized {
        default 0;
        "Bearer <PROXY_TOKEN>" 1;
    }

    server {
        listen <UNRAID_TAILSCALE_IP>:11435;
        server_name _;

        client_max_body_size 20m;

        location = /v1/models {
            if ($ollama_proxy_authorized = 0) { return 401; }

            proxy_pass http://127.0.0.1:11434;
            proxy_set_header Authorization "";
            proxy_set_header Host $host;
            proxy_http_version 1.1;
            proxy_buffering off;
            proxy_read_timeout 3600s;
        }

        location = /v1/chat/completions {
            if ($ollama_proxy_authorized = 0) { return 401; }

            proxy_pass http://127.0.0.1:11434;
            proxy_set_header Authorization "";
            proxy_set_header Host $host;
            proxy_http_version 1.1;
            proxy_buffering off;
            proxy_request_buffering off;
            proxy_read_timeout 3600s;
        }

        location / {
            return 403;
        }
    }
}

I ran the proxy in host-network mode so that it could bind specifically to the Unraid Tailscale address and reach Ollama on 127.0.0.1:11434:

docker run -d \
  --name ollama-inference-proxy \
  --restart unless-stopped \
  --network host \
  -v /mnt/user/appdata/ollama-inference-proxy/nginx.conf:/etc/nginx/nginx.conf:ro \
  nginx:alpine

I validated the proxy configuration:

docker exec ollama-inference-proxy nginx -t
docker logs --tail 30 ollama-inference-proxy

Proxy tests

From the Mac mini, a request with the token should succeed:

curl -H 'Authorization: Bearer <PROXY_TOKEN>' \
  http://<UNRAID_TAILSCALE_IP>:11435/v1/models

A request without the token should return 401 Unauthorized:

curl -i http://<UNRAID_TAILSCALE_IP>:11435/v1/models

A valid token sent to an unapproved Ollama route should return 403 Forbidden:

curl -i \
  -H 'Authorization: Bearer <PROXY_TOKEN>' \
  http://<UNRAID_TAILSCALE_IP>:11435/api/tags

6. Restrict the Hermes account with nftables

Installation order: Complete the internet-dependent Hermes installation in section 7 before enabling this firewall. Once the restriction is active, the installer and other internet downloads will be blocked for hermesagent. Return here to apply and verify the restriction before starting an agent session.

The proxy limits what can be done through port 11435, but I also wanted to prevent Hermes from bypassing it by connecting directly to Ollama, Unraid, another Tailscale device, the LAN, or the public internet.

I created an nftables rule that applies only to the UID of hermesagent. My normal administrator account remains unaffected.

sudo install -d -m 0755 /etc/nftables.d
sudo nano /etc/nftables.d/hermes-guard.nft

I added the following, replacing <HERMES_UID> and <UNRAID_TAILSCALE_IP>:

table inet hermes_guard {
    chain output {
        type filter hook output priority -50; policy accept;

        # Preserve reply traffic for connections that already exist.
        meta skuid <HERMES_UID> ct state established,related accept

        # The only new connection this account may initiate.
        meta skuid <HERMES_UID> ip daddr <UNRAID_TAILSCALE_IP> tcp dport 11435 accept

        # Reject all other IPv4 and IPv6 network traffic from the account.
        meta skuid <HERMES_UID> counter reject with icmpx type admin-prohibited
    }
}

I checked the file without applying it:

sudo nft -c -f /etc/nftables.d/hermes-guard.nft

No output means the syntax check passed.

Make the rule persistent

I used a dedicated systemd unit rather than replacing the system’s complete nftables ruleset:

sudo nano /etc/systemd/system/hermes-guard.service

[Unit]
Description=Restrict hermesagent network access
Wants=network-online.target
After=network-online.target tailscaled.service

[Service]
Type=oneshot
RemainAfterExit=yes
ExecStartPre=-/usr/sbin/nft delete table inet hermes_guard
ExecStart=/usr/sbin/nft -f /etc/nftables.d/hermes-guard.nft
ExecReload=-/usr/sbin/nft delete table inet hermes_guard
ExecReload=/usr/sbin/nft -f /etc/nftables.d/hermes-guard.nft
ExecStop=-/usr/sbin/nft delete table inet hermes_guard

[Install]
WantedBy=multi-user.target

I enabled and started it:

sudo systemctl daemon-reload
sudo systemctl enable --now hermes-guard.service
sudo systemctl status hermes-guard.service --no-pager
sudo nft list table inet hermes_guard

active (exited) is normal for this oneshot service; the nftables rule remains loaded in the kernel.

Verify the restriction

I switched into the restricted account:

sudo -iu hermesagent
whoami
id -u

The approved proxy should work:

curl -H 'Authorization: Bearer <PROXY_TOKEN>' \
  http://<UNRAID_TAILSCALE_IP>:11435/v1/models

The raw Ollama port, public internet, and normal LAN addresses should fail:

curl --connect-timeout 3 http://<UNRAID_TAILSCALE_IP>:11434/v1/models
curl --connect-timeout 3 https://example.com
curl --connect-timeout 3 http://192.168.0.1

Back in the administrator account, I confirmed the reject counter increased:

sudo nft list table inet hermes_guard

7. Install Hermes as the restricted user

I opened a shell as hermesagent and ran the official per-user installer without sudo:

sudo -iu hermesagent
curl -fsSL https://hermes-agent.nousresearch.com/install.sh | bash
source ~/.bashrc

Installing as this account placed Hermes and its configuration under /home/hermesagent/.hermes/ and the launcher under /home/hermesagent/.local/bin/.

During setup, I chose:

The local terminal backend is not a filesystem sandbox. Commands run with the same permissions as hermesagent, which is why the dedicated account is important.

8. Configure the custom model endpoint

I started the model configuration wizard:

hermes model

I selected Custom endpoint and entered:

SettingValue
API base URLhttp://<UNRAID_TAILSCALE_IP>:11435/v1
API key<PROXY_TOKEN>
API compatibilityChat Completions
Model<MODEL_NAME>
Context length64000

I also pinned the active model context explicitly:

hermes config set model.context_length 64000

Hermes stores normal model configuration in ~/.hermes/config.yaml and secrets in ~/.hermes/.env.

9. Apply conservative Hermes settings

I enabled manual approvals and denied unattended execution modes:

hermes config set approvals.mode manual
hermes config set approvals.cron_mode deny
hermes config set approvals.single_query_mode deny
hermes config set approvals.unattended_mode deny
hermes config set agent.max_turns 25

I kept only the capabilities needed for local interactive work—primarily file access, terminal commands, and local skills—and left web access, browser automation, messaging, cron, delegation, Home Assistant, and external connections disabled.

The available tools can be reviewed with:

hermes tools

I also protected the configuration files:

chmod 700 ~/.hermes
chmod 600 ~/.hermes/config.yaml ~/.hermes/.env
mkdir -p ~/workspace
chmod 700 ~/workspace

From the administrator account, I protected both users’ home directories:

sudo chmod 700 /home/hermesagent
sudo chmod 700 /home/<ADMIN_USER>

I kept SSH keys, SMB credentials, Unraid passwords, browser profiles, and other secrets out of /home/hermesagent.

10. Validate the final configuration

As hermesagent, I ran:

hermes config
hermes doctor

The important model fields were:

model:
  default: hf.co/unsloth/Qwen3.8-27B-GGUF:UD-Q6_K_M
  provider: custom
  base_url: http://<UNRAID_TAILSCALE_IP>:11435/v1
  api_mode: chat_completions
  context_length: 64000

The API key should be represented by an environment-variable reference, not published as plaintext.

I then started Hermes from its workspace:

cd ~/workspace
hermes

The full-screen interface is also available:

hermes --tui

My first test prompt was intentionally simple:

Reply with exactly: Hermes is working.

I then checked local tool execution:

Run whoami and pwd. Show the output and do nothing else.

The expected user was hermesagent, and the working directory was /home/hermesagent/workspace.

Daily operation

To start Hermes:

sudo -iu hermesagent
cd ~/workspace
hermes

Useful commands inside a session include:

/help
/tools list
/status
/context
/usage
/new
/quit

To resume the most recent session:

hermes --continue

The interactive CLI does not run continuously after /quit. I did not install the Hermes messaging gateway, so there is no always-on agent process or unattended task runner in this design.

Monitoring

On Unraid, I use the following commands to monitor GPU and container activity:

watch -n 1 nvidia-smi
docker stats ollama ollama-inference-proxy
docker exec ollama ollama ps
docker logs -f --tail 100 ollama
docker logs -f --tail 100 ollama-inference-proxy

On the Mac mini:

top -u hermesagent
hermes logs -f
hermes status --deep
hermes sessions list

Ollama may keep a model loaded in VRAM for a short period after the last request. Allocated VRAM does not mean the GPU is continuously computing; nvidia-smi should show utilization dropping near zero when generation stops.

Final security properties

With this configuration, Hermes can:

It cannot initiate connections to:

The remaining risk is local: Hermes can change files owned by hermesagent, because the local terminal backend is not sandboxed. The dedicated account, restricted home directory, manual approvals, minimal tool selection, and absence of sensitive credentials keep that risk contained.

Maintenance notes

References

This guide documents one home-lab design, not a claim of perfect isolation. Anyone reproducing it should review the rules, paths, account permissions, and network addresses for their own environment.

← All projects