Hindsight + Hermes — giving my assistant a real long-term memory

I wanted my assistant to remember — not just within one chat, but across weeks of work, with structure it can reason over. That is what Hindsight adds to Hermes.

What is Hermes, again

Hermes is the desktop AI assistant I run daily. It is local-first, model-agnostic, and agentic: it can call tools, drive a browser, run terminals, and orchestrate sub-agents. But like most assistants, its memory is mostly per-session — once a conversation scrolls away, the useful details fade unless I re-explain them.

That gap is exactly what a memory layer fixes.

What is Hindsight

Hindsight is a self-hosted long-term memory service for AI agents. Think of it as a small, private knowledge base that sits next to the assistant:

• it stores facts, decisions and context as structured memory entries,
• it retrieves the relevant ones when a new question arrives (semantic + keyword search),
• it reflects — it can synthesise and consolidate memories so the agent reasons over a clean picture, not a raw pile of notes.

No analytics, no cloud: the data stays on my machine.

What I installed (the stack)

The whole thing runs as a container, which keeps it isolated and easy to redeploy.

Layer	Choice	Role
Runtime	Docker	isolates Hindsight, one command to (re)start
Service	Hindsight	memory API on a local port (:8888)
Storage	local vector + JSON store	embeddings + facts, persisted on disk
Bridge	Hermes memory provider	points Hermes at the Hindsight endpoint
Models	the same local models I already use	embeddings + reflection, no extra cost

docker run -d --name hindsight -p 8888:8888 \
  -v ./hindsight-data:/data \
  hindsight/hindsight:latest

Hermes is then configured to use Hindsight as a memory provider — every answered question, every durable fact, lands there.

How the memory flows

The loop is simple and runs on every turn:

1. Capture — Hermes writes durable context (preferences, environment, decisions) to Hindsight.
2. Index — Hindsight embeds the text and stores it alongside the raw entry.
3. Recall — on a new request, Hermes asks Hindsight for relevant memories (hindsight_recall); only the matching entries are injected into context.
4. Reflect — periodically, Hindsight consolidates overlapping memories so retrieval stays sharp.

You ──▶ Hermes ──write──▶ Hindsight (embed + store)
                       │
You ──▶ Hermes ──recall─▶ Hindsight ──▶ top matches ──▶ context

Why bother coupling them

• Continuity : I no longer re-explain my setup, my constraints, or "the thing we decided last week".
• Reasoning, not just storage : reflection turns a note pile into something the agent can actually use.
• Privacy by design : everything is local; nothing leaves the box.
• Cost : it reuses models I already run, so the memory layer is nearly free.

The one gotcha

Editing .env is not enough — you must recreate the container (stop + rm + run). A plain restart keeps the stale config and silently does nothing.

That bitten me once; now it is muscle memory.

Try it

If you want the same setup, run the container above and point your agent's memory provider at :8888. For the agent itself, OpenCode (referral link) is a clean, local-first coding agent worth a look.

See also: Finding application notes fast with a targeted RAG — another piece of the same self-hosted, privacy-first toolkit.