wiki-vault/Sources/Homelab/2026-05-15-postgres-consolidation-phase0-recon.md

---
created: '2026-05-15'
path: Sources/Homelab
project: postgres-consolidation-reflection-layer
status: active
tags:
- ob1
- mcp
- pgvector
- embeddings
- session-note
type: session-notes
updated: '2026-05-15'
worker: 2
---

# Postgres Consolidation — Worker 2, Phase 0 recon

Snapshot of the homelab Postgres instance before any consolidation
work. Recorded from `homelab-postgres` (pgvector/pgvector:pg17),
container in `/opt/projects/docker/postgres/`, port `127.0.0.1:5433`,
volume `ob1_data` (external).

## Postgres instance

- Container: `homelab-postgres` (image `pgvector/pgvector:pg17`)
- Volume: `ob1_data` (external — survives stack down)
- Network: `homelab` (alias `homelab-postgres`, also `postgres`)
- Healthy, up 40 hours.
- POSTGRES_DB default = `openbrain`.

## Databases on instance

| Name        | Owner           | Size    | pgvector | Notes |
|-------------|-----------------|---------|----------|-------|
| `herbylab`  | `herbylab_user` | 7958 kB | NO       | Holds the herby-dev artifact store |
| `openbrain` | `postgres`      | 8838 kB | YES (0.8.2) | OB1 captures live here |
| `trellis`   | `trellis_user`  | 8262 kB | NO       | Trellis MCP threads/events |
| `postgres`  | `postgres`      | 7478 kB | -        | default; not touched |

No other user databases. Plan's "instance hosts only wiki, ob1, trellis"
holds — under the actual names `herbylab` / `openbrain` / `trellis`.

## Existing roles

- `postgres` — Superuser, Create role, Create DB, Replication, Bypass RLS
- `herbylab_user` — owns `herbylab.herbylab` schema
- `trellis_user` — owns `trellis.trellis` schema

No `lovebug` role exists. No `ob1_mcp` / `vault_mcp` / `trellis_mcp`
service users yet — these come in Phase 2.

## Schemas, tables, row counts

### `herbylab` DB
- Schemas: `herbylab` (owner `herbylab_user`), `public`.
- Tables in `herbylab`:
  - `projects` — 0 rows (8192 B; one empty heap page)
  - `artifacts` — 0 rows (8192 B; one empty heap page)
  - `alembic_version` — 1 row (`9c7e3f1a2b4d`)
- Custom enums: `herbylab.project_status`, `herbylab.artifact_type`, `herbylab.project_domain`.
- Triggers / functions: `projects_notify` (NOTIFY on row change), `artifacts_set_updated_at`.
- No vector columns.

**Why empty (verified, not a missed lookup):** the schema was created
by alembic migration `9c7e3f1a2b4d` on the unmerged branch
`claude/competent-knuth-de9483` (commit `90bc6a9 Phase 1: herbylab
Postgres schema infrastructure`). That branch also has commits
`8e408c5 Phase 2: polymorphic artifact tools backed by Postgres +
vault` and `78ec094 Phase 3: Trello projection via n8n` which would
have wired writes into the schema, but **the branch is not merged to
`main`.**

The deployed `herbydev-mcp` runs `main`'s code, which has zero DB
access:
- `pyproject.toml` declares no DB driver (no psycopg / sqlalchemy /
  asyncpg).
- `mcp_server/tools/memory.py` and `mcp_server/tools/tasks.py` are
  empty stub files marked "Phase 2. Do not implement until workflow
  patterns are established."
- `mcp_server/tools/vault.py` writes via `core/git_writer.py` →
  filesystem + git commit + push to gitea. No database insertion.
- `HERBYLAB_DATABASE_URL` is plumbed through compose env but never
  read by the running code.

So the DB schema exists from an integration-test run of the unmerged
branch's migration; no code path on `main` ever populates it.

Implication for consolidation: the `wiki` schema in `petalbrain` is
genuinely empty content-wise. Worker 4's vault-indexing source is the
filesystem (`/opt/projects/wiki-vault/`), not a `wiki.notes` /
`wiki.artifacts` table. If `competent-knuth-de9483` lands later, its
artifact tooling will need to be re-pointed at `petalbrain.wiki.*`
instead of `herbylab.herbylab.*`, and the Phase 4 reflection design
should account for `wiki.artifacts` becoming a populated source over
time.

### `openbrain` DB
- Schemas: `public` only.
- Tables in `public`:
  - `thoughts` — 70 rows (live; daemons writing today)
- pgvector 0.8.2 in `public`.

`public.thoughts` shape (relevant columns):

```
id          bigint            PK (nextval thoughts_id_seq)
content     text              NOT NULL
embedding   vector(768)
metadata    jsonb             DEFAULT '{}'
user_id     text              NOT NULL DEFAULT 'travis'
created_at  timestamptz       DEFAULT CURRENT_TIMESTAMP
updated_at  timestamptz       DEFAULT CURRENT_TIMESTAMP
```

Existing indexes on `public.thoughts`:
- `thoughts_pkey` (id)
- `idx_thoughts_created_at` (created_at DESC)
- `idx_thoughts_embedding_hnsw` — **hnsw (embedding vector_cosine_ops) WITH (m=16, ef_construction=64)** — already matches the locked Phase 4 spec for `public.embeddings`.
- `idx_thoughts_fingerprint` — UNIQUE (md5(content), user_id)
- `idx_thoughts_metadata` — gin (metadata)
- `idx_thoughts_user_created` (user_id, created_at DESC)
- `idx_thoughts_user_id` (user_id)

Additional objects:
- Function `match_thoughts(query_embedding vector, match_threshold float, match_count int, filter jsonb, p_user_id text)` — bare-`vector` param (no dim), so dimension changes flow through unchanged.
- Trigger `thoughts_updated_at` -> `update_updated_at()`.

Live writer state: 70 rows total, 0 with NULL embedding, 2 still flagged `metadata->>'pending' = 'true'` (enricher backlog). Most recent row id 255 at `2026-05-16 02:07:23 UTC`. Watcher and enricher daemons confirmed live.

### `trellis` DB
- Schemas: `trellis` (owner `trellis_user`), `public`.
- Tables in `trellis`:
  - `threads` — 13 rows
  - `events` — 50 rows
  - `tags` — 42 rows
  - `thread_tags` — 55 rows
  - `alembic_version` — 1 row
- No vector columns.

## OB1 embedding model

From `ob1-mcp` container env (and `/opt/projects/homelab/ob1-deploy/compose.yml`):

- `EMBEDDING_PROVIDER=ollama`
- `OLLAMA_URL=http://ollama:11434`
- `EMBEDDING_MODEL=nomic-embed-text` (768-d output)
- `CHAT_MODEL=openai/gpt-4o-mini` (mcp side)
- `CHAT_MODEL=llama3.2:3b` (enricher side)

Migration trail:
- `init.sql` originally declared `vector(1536)` for `text-embedding-3-small`.
- `migrations/001_embedding_dim_768.sql` (applied 2026-05-14) dropped the
  HNSW index, altered the column to `vector(768)`, recreated the index.
  Required `thoughts` to be empty pre-migration.

**Conclusion:** the shared `public.embeddings` table goes in as
`vector(768)` to match `nomic-embed-text`, no model switch needed. HNSW
index spec from Travis's overrides (`vector_cosine_ops, m=16,
ef_construction=64`) matches the existing OB1 convention exactly.

## MCP server connection configurations

| Container       | Image               | DB host / DB / user | Live? |
|-----------------|--------------------|---------------------|-------|
| `herbydev-mcp`  | `herbydev-mcp`     | `homelab-postgres:5432/herbylab` as `herbylab_user` | Up 40h |
| `vault-mcp`     | `vault-mcp`        | **`vault-postgres:5432/herbylab` as `herbylab_user`** | Up 59m |
| `ob1-mcp`       | `ob1-mcp`          | `homelab-postgres:5432/openbrain` as `postgres` | Up 14h |
| `ob1-watcher`   | `ob1-watcher`      | (talks to ob1-mcp HTTP, not DB directly) | Up 26h |
| `ob1-enricher`  | `ob1-enricher`     | (uses `OLLAMA_URL`; DB password set but no direct connect string visible) | Up 14h |
| `trellis-mcp`   | `trellis-trellis-mcp` | `homelab-postgres:5432/trellis` as `trellis_user` | Up 40h healthy |

Connection strings live in:
- `vault-mcp` / `herbydev-mcp` — `/opt/projects/homelab/herby-dev/deploy/.env` (`HERBYLAB_DATABASE_URL`)
- `ob1-mcp` / `ob1-enricher` — `/opt/projects/homelab/ob1-deploy/.env` (`POSTGRES_PASSWORD` + per-service `DB_*` env in `compose.yml`)
- `trellis-mcp` — `/opt/projects/homelab/trellis-mcp/` (compose env, `DATABASE_URL`)

**Anomaly:** `vault-mcp` references hostname `vault-postgres` which
does not resolve on the `homelab` network (alias is `homelab-postgres`
+ `postgres`). This is consistent with Travis's mid-flight rename
rollout — the container is currently running OLD code per the
priming, and the connection-string update is part of the unfinished
rollout. Not blocking Phase 1, but Phase 3 must coordinate with the
rename completion.

## Backup destination

**Local-only on herbydev** (Travis decision, 2026-05-15). The NAS
reference in the plan was unintended — disregard it.

- Destination directory: `/opt/backups/postgres-consolidation/`
  (does not exist yet; Phase 1 step zero is `mkdir -p` it).
- Local disk on `/`: 14 G free of 97 G (86% used).
- Combined DB size is ~25 MB, so headroom is ample for both Phase 1
  bulk dumps and Phase 2-start fresh dumps.
- Format: `pg_dumpall` (full instance) + per-database `pg_dump` in
  custom format (`-Fc`), gzipped. Filenames include UTC timestamp.
- No NAS / Tailscale / off-host destination involved.

## Coordination state — running daemons

`ob1-watcher` and `ob1-enricher` are both live and writing to
`openbrain.public.thoughts` (latest write ~5 minutes before this
note). Per priming, the Phase 2 `ALTER TABLE public.thoughts SET
SCHEMA ob1` step is the cutover gate: daemons must be paused, table
moved + ob1.thoughts shape rewrite by Worker 3, then daemons resumed.
Worker 2 stops at the announcement of readiness for that move and
waits.

## Naming reconciliation (plan vs. reality)

Plan vocabulary `wiki / ob1 / trellis` will land as the **schema**
names in the consolidated database, per Travis's locked decisions.
Source DB → target schema mapping:

- `herbylab` DB content → `wiki` schema (owned by `vault_mcp` per
  override; was `herby_mcp` in the plan)
- `openbrain` DB content → `ob1` schema; `public.thoughts` becomes
  `ob1.thoughts` (preserving `thoughts` vocabulary per override; plan
  said `ob1.captures`)
- `trellis` DB content → `trellis` schema (owned by `trellis_mcp`)

Plus shared `public.embeddings` per Phase 4.

**Open: target database name itself.** OB1's database is currently
`openbrain`. The plan says "if OB1's database is the target and is
named appropriately, use in place. Otherwise, rename or create new."
`openbrain` becomes a misleading name once it also holds wiki +
trellis schemas. Worth a deliberate rename (e.g., `homelab`,
`petalbrain`, or similar) before Phase 2 work begins.

## Phase 1 readiness assessment

Ready:
- Source DB sizes are trivial; dumps will be small and fast.
- pgvector already present in target DB (`openbrain`); no extension
  install needed.
- Embedding model + dim are confirmed and stable (`nomic-embed-text` /
  768) — no re-embed forced.
- Existing HNSW spec on `public.thoughts` matches the locked
  `public.embeddings` HNSW spec exactly.

Not ready / needs decision:
- ~~NAS backup destination~~ **Resolved 2026-05-15**: local-only at
  `/opt/backups/postgres-consolidation/`. No NAS involved.
- ~~Target database name~~ **Resolved 2026-05-15**: `petalbrain`.
  `openbrain` renamed before Phase 2 destructive ops begin.
- **`wiki` schema content scope** — `herbylab` DB schema came from the
  unmerged `claude/competent-knuth-de9483` branch's migration
  `9c7e3f1a2b4d`; `main`'s deployed MCP doesn't open a DB connection
  at all. Wiki content is filesystem-only under
  `/opt/projects/wiki-vault/`. Worker 4's vault-indexing source must
  walk the filesystem and write into `public.embeddings` directly.
  See the herbylab-DB section above for the full evidence trail.
- **Connection-string anomaly in `vault-mcp`** (`vault-postgres`
  hostname). Tied to Travis's rename rollout, blocks Phase 3 cutover
  for that MCP only. Phase 1/2 unaffected.

Conservative call: gate before Phase 1 until the NAS destination and
target DB name decisions are made.

## Addendum — Phase 2-start fresh backup (Travis directive, 2026-05-15)

**In addition to Phase 1 bulk backups**, Phase 2 must begin with a
fresh `pg_dump` of each source database immediately before the first
destructive operation (no `CREATE SCHEMA`, no `ALTER TABLE`, no
restore — nothing destructive precedes this step).

Rationale: `ob1-watcher` is writing to `public.thoughts` continuously
(latest write ~5 min before this note). Time elapses between Phase 1
dumps and Phase 2 cutover, especially with gating. Any rows captured
in that window would be lost on rollback if Phase 1 dumps are the
only restore point.

Specifics:
- Run fresh `pg_dump` on `openbrain` first — it's the live writer and
  the most critical. Then `herbylab` and `trellis` for symmetry.
- Destination: `/opt/backups/postgres-consolidation/` (same as Phase
  1), gzipped custom format.
- Filename suffix: `-phase2-start-<UTC-timestamp>` to distinguish from
  Phase 1 backups.
- Verify each fresh dump is readable via test restore to a scratch
  database (same pattern as Phase 1).
- Only after the three fresh dumps are verified does Phase 2's first
  destructive operation run.

This is not a replacement for Phase 1 dumps — Phase 1 remains the
broad safety net (full instance `pg_dumpall` + per-DB granular
dumps). Phase 2-start dumps are the just-before-disruption snapshot
that captures any in-flight writer activity.

Operational note: this is also the natural point to pause
`ob1-watcher` and `ob1-enricher`. Sequence:
1. Pause both daemons (`docker compose stop ob1-watcher ob1-enricher`).
2. Confirm no in-flight writes (`pg_stat_activity` quiet on `openbrain`).
3. Take the fresh per-DB dumps + verify.
4. Begin Phase 2 destructive work.
5. (Worker 3 resumes daemons after `ob1.thoughts` cutover lands.)