Local AI Server Build
Overview
My Unraid server provides local AI inference alongside storage. Ollama runs in Docker and supplies the model backend for the Hermes Agent setup on my Intel Mac mini.
Hardware and storage
| Component | Configuration |
|---|---|
| Operating system | Unraid 7.2.7 |
| CPU | AMD Ryzen 5 5600G |
| System memory | 32 GB DDR4 |
| GPU | NVIDIA Tesla V100 PCIe, 32 GB VRAM |
| Data drives | 2 × 4 TB HDD |
| Parity drive | 1 × 4 TB HDD |
| Cache | 256 GB SSD |
GPU driver and power configuration
The V100 uses the Unraid Nvidia-Driver plugin with the legacy R580 driver branch for Volta (sm_70).
A boot script managed through the User Scripts plugin applies a 150 W GPU power limit. This keeps the configured limit part of the server’s startup process.
Cooling
The V100 uses a custom 3D-printed fan shroud in a pull configuration. The shroud directs airflow through the GPU heatsink.
Local AI services
Ollama runs in a Docker container on the Unraid server. The separate Hermes Agent project documents how the Mac mini connects to this backend over Tailscale through an authenticated Nginx proxy, with account-level outbound restrictions enforced by nftables.
Related project
- Hermes Agent on a Mac mini with Ollama — agent setup, inference access, and network restrictions.