NETGRIMOIRE

Overview

NetGrimoire's Local Intelligence system is a self-hosted personal knowledge graph, retrieval-augmented generation (RAG) pipeline, and AI proxy that gives every AI tool you use — Claude, Claude Code, ChatGPT, Gemini, Perplexity, and local Ollama models — persistent memory of your entire life and infrastructure.

The system is managed by Gremlin, a grumbling but diligent AI agent persona that runs on your homelab. Gremlin acts as the connective tissue: ingesting knowledge from all your services, storing it in a vector database, and injecting relevant context into every query before it reaches an external provider.

How It Works

1. Ingest (The Slow Work)

Gremlin continuously ingests from all connected knowledge sources on a schedule. Each source gets its own Qdrant collection, versioned for safe re-embedding when better models become available:

• Obsidian vault — Git push webhook → n8n → chunk by note → nomic-embed-text → obsidian_v1
• Paperless-ngx — REST API, nightly pull → Ollama summarize + entity extract → paperless_v1
• Nextcloud — CalDAV/files → structured event records → nextcloud_v1
• Forgejo repos — CI/CD decisions, commit messages, stack configs → repo_v1
• MailCow — IMAP, VIP senders prioritized → email_v1
• Immich — EXIF + album metadata now; full photo understanding after GPU upgrade → immich_v1

All records include embedded_with, summarized_with, and ingested_at metadata so re-processing is targeted when hardware improves.

2. Query (The Fast Work)

When you send a query through any interface, the proxy layer intercepts it before it reaches any AI provider:

# Every query follows this path
1. Query arrives at Gremlin Proxy (FastAPI)
2. Embed query with nomic-embed-text
3. Qdrant similarity search across relevant collections
4. Top-k chunks formatted as context block
5. Router selects provider (config-driven or @override)
6. Enriched prompt dispatched to provider
7. Response returned to interface
8. Write-back worker captures exchange

The embedding step (step 2–3) runs on CPU in milliseconds — it's just a vector lookup, not inference. The external provider sees a well-contextualized prompt and responds as if it has long-term memory.

3. Routing

The router selects a backend based on query classification or explicit prefix. Rules live in gremlin/config.yaml so no workflow code changes are needed to add or adjust providers:

routing:
  default: claude
  rules:
    - pattern: code|script|function|debug
      backend: claude-code
    - pattern: research|news|current|latest
      backend: perplexity
    - pattern: private|local|sensitive
      backend: ollama
  overrides:
    @claude:    claude
    @code:      claude-code
    @gemini:    gemini
    @perplexity: perplexity
    @local:     ollama      # never leaves network

4. Memory Write-Back

After every significant exchange, n8n triggers a write-back worker:

• Ollama summarizes the session (what was asked, what was decided, what changed)
• Summary appended to daily log in traveler/memory/YYYY-MM-DD.md
• Key decisions extracted to structured frontmatter
• Committed to Forgejo — fully git-versioned and auditable
• New chunks embedded and upserted into Qdrant

5. Claude Code Continuity

Claude Code sessions achieve continuity through two Gremlin-maintained files in every repo root:

• CLAUDE.md — auto-generated by Gremlin CI/CD. Stack standards, active work, recent decisions, open questions. Always current.
• CONTEXT.md — session state. Current focus, last stopping point, exact next steps. Written by Gremlin write-back after each session.

Claude Code reads both automatically at startup. Every session continues exactly where the last one left off, regardless of how much time has passed.

Hardware Tiers

The architecture does not change between tiers. Gremlin's endpoints and config.yaml routing rules are the only things that update as hardware improves.

Key Design Principles

• Deterministic before LLM — embedding and retrieval are fast and reliable. Ollama only runs for summarization and entity extraction, never for routing logic.
• Opt-in privacy — @local prefix forces any query to Ollama. It never reaches an external API. Sensitive infrastructure queries stay home.
• Versioned collections — obsidian_v1, paperless_v1 etc. Re-embedding with a better model populates _v2 alongside without downtime.
• Config-driven routing — no code changes to add a provider or adjust routing rules. Everything lives in gremlin/config.yaml.
• Auditability — every AI exchange is logged with provider, estimated cost, timestamp, and summary. Full history in Forgejo.
• Single memory store — Qdrant + MD files in traveler/memory. All interfaces read from and write to the same place.

Related Pages

• Gremlin CI/CD Pipeline — stack deployment automation
• Gremlin Morning Briefing — daily HTML dashboard
• Pocket Grimoire — travel-subset deployment
• Obsidian Vault — traveler/notes structure and sync
• Qdrant — vector database configuration

Container Consolidation Analysis

The question is whether to consolidate related services into shared containers, or keep them separate. Here's the analysis for each logical grouping.

Option: Gremlin Intelligence Stack

The proxy, write-back worker, and ingest workers are all Python services with shared library dependencies. These are strong consolidation candidates:

Recommendation: One gremlin-proxy container running FastAPI with the write-back logic as an async background task. Ingest workers stay as scripts on the host, executed by n8n. Keeps the container small and debuggable.

Option: Ollama + Open WebUI

Option: n8n + Supporting Services

Recommended Stack Layout

Two new Swarm stacks covering the new intelligence layer:

# Stack 1: gremlin-intelligence
services:
  gremlin-proxy:        # FastAPI proxy + write-back + router
    image: gremlin-proxy:latest
    # Built from traveler/services/gremlin-proxy/

  gremlin-avatar:       # Open-LLM-VTuber (future)
    image: openllm-vtuber:latest

# Stack 2: gremlin-knowledge (already partially exists)
services:
  qdrant:               # already running
  ollama:               # already running
  open-webui:           # already running

What NOT to Consolidate

• Qdrant — stateful, needs independent backup management, separate from application logic
• Ollama — shared dependency for too many services; isolating it in a bundle breaks other consumers
• n8n — orchestrator, too critical to co-locate
• Forgejo — git server, completely separate lifecycle