Synergy
| As of | |
|---|---|
| Version | 2.1.0 |
| Kind | plan |
| Length | 2,101 words, about 9 min |
| Tags | planliving-planphase-3synergyrlspostgresqlapigraph |
Synergy is Phase 3: how any entity in any module links to any other. Scope: the entity index and version log, links and relation types, staleness, lists, mentions and anchors, delete and restore, and tags, built in two parts (3a the store and its API, 3b resolving and rendering links).
Milestone: Synergy is live in
packages/platform/synergy. Any entity can be linked to any other entity and tagged, links stay true when their targets change, and every module renders a link the same way. It is proven across a module boundary by two mock providers, before Taxila becomes the first real tenant.Learning payoff: graph modelling in Postgres (recursive CTEs, composite foreign keys, partial unique indexes), concurrency (advisory locks, stale-write checks, race tests with two connections), an outbox, per-package migrations, layered packages enforced by lint, and contracts shared by server and browser.
The design is settled: the Synergy PRD and spec say what Synergy does, and the Synergy system design (1.3.0) and its decisions say how. This plan turns that design into lessons.
Phase 3 has two parts. 3a builds the store, its rules and its API. 3b resolves and renders links. Only 3a is planned in lessons now: what 3b needs depends on how 3a gets built, so Checkpoint 3a plans 3b (Pratik, 2026-11-01).
Lessons (3a: the store and its API)
Lessons run one at a time, in this order, each in its own thread, on Pratik's Mac through Remote Control. A lesson starts only after the previous lesson's PRs merge. Each one ends at a check Pratik can run, and depends only on the lessons above it.
| Lesson | Brief | What Pratik learns | Status |
|---|---|---|---|
| 1 | Layer folders and lint | monorepo layers, git mv, ESLint boundaries policies | |
| 2 | Per-package migrations | Drizzle configs and migration tables, Turbo task ordering, a shared test database | |
| 3 | Entity index and version log | RLS without the tombstone filter, restore-aware triggers, advisory locks, stale-write checks | |
| 4 | Sandbox providers and the first routes | feature packages, deferrable foreign keys, ETags and 304, error codes to HTTP | |
| 5 | Outbox, status and versions | the outbox pattern, transaction ids and commit order, a read-only role | |
| 6 | Links, relation types and cycles | polymorphic references, recursive CTEs with CYCLE, idempotency keys, registries as data | |
| 7 | Staleness dots and the roll-up | snapshots, state derived on read, one query for a roll-up | |
| 8 | Lists and supersede | composite foreign keys, CHECK constraints, lexorank, confirm-then-act requests | |
| 9 | Save call | current sets vs. diffs, self-healing writes, Tiptap JSON | |
| 10 | Save call | anchors in content, passage protection, the anchor receipt | |
| 11 | Delete and restore | soft-delete batches, re-checking under a lock, restore_conflict | |
| 12 | Tags | aliases, merges, key:value checks, tagging a link | |
| 13 | Race tests | two-connection tests, measuring before optimising | |
| ✦ | Checkpoint 3a | proving a part on main, planning from what was built |
To start a lesson, open a new thread and paste this, changing the lesson number:
Coach me through Phase 3 lesson 1. Read AGENTS.md, the Phase 3 plan
(docs/04-phases/03-synergy/00-plan.md) and that lesson's
brief (the plan's Lessons table links each one) before we start.How we build
Phase 3 follows Phase 2's how we build, in the style Pratik chose from Phase 2 lesson 4 on, and the agent guide's Working with Pratik:
- Pratik writes the code. The coach gives exactly what to write for one small piece, then explains it below: what each part does, why, the concepts, the consequences and the alternatives rejected. No quiz questions. Design questions come as recommendations; Pratik decides. A brief's "good to write yourself" steps are ones worth designing before seeing the coach's version.
- Explain backend mechanisms from first principles: the concept, then the problem as a practical story, then the fix, then ask to agree.
- One piece at a time. Nothing lands in the repo until Pratik agrees to it. Pratik commits, pushes and opens the PR unless they ask otherwise.
- Tests go through the guarded API wherever a route exists (testing layers). Whether a pure function also gets a unit test is Pratik's call in each lesson; the briefs point out the candidates.
- The design can change. When Pratik's design wins in a lesson, the coach updates the Synergy overview (or the decision it touches) and bumps its version in the lesson's arkive PR, so later lessons build on what was built.
- Every lesson ends with one arkive PR: a walkthrough at
04-phases/03-synergy/00-lessons/lesson-N-<slug>/01-walkthrough.md, a learnings note at04-phases/03-synergy/00-lessons/lesson-N-<slug>/02-notes.md(publish: false) holding everything explained in the lesson, the lesson marked ✅ in the table above, and a "Handover from lesson N" section in the next brief. A fix that changes an earlier lesson's or phase's result adds a post note to that walkthrough. Verify withnpm run build(Astro and the privacy gate).
Exit criteria
From scope and phases. Integration tests through the public API only prove that:
| Criterion | Proven in |
|---|---|
| a link can carry a label and a type | lesson 6 |
| a version bump with a change note shows the red dot | lesson 7 |
| deprecate and supersede work | lessons 5 and 8 |
| deleting a referenced target is blocked | lesson 11 |
| reference cycles are rejected | lessons 6 and 11 |
key:value tags work | lesson 12 |
| disabling a module makes its links locked | lesson 6 |
| another user's entities never leak | every lesson with a table |
| search finds entities by title and metadata | 3b |
Checkpoint 3a re-proves the 3a rows on main. The phase closes with a close-out lesson at the end of 3b, like Phase 2's lesson 8.
3b: resolving and rendering links (planned at Checkpoint 3a)
Candidates from the planning handover, to be merged, split or reordered at the checkpoint. None has a brief yet.
- One React: a pnpm catalog for
react,react-domand@types/react;apps/webusescatalog:;@sidekick/uiand@sidekick/synergylist React as a peer (decision). Check:pnpm why reactresolves to one version. - Resolve and the React registry:
resolveonLinkableTypeServer(batched per provider),LinkableTypeReact, the registry assembling both maps,POST /api/synergy/links/resolve, the resolved-link zod schema frozen in/contracts. - Sandbox renderers:
resolveandrenderfor the sandbox notes and tasks. - Sneak Peek Link: the
/reactcomponent with purerender, an injectedresolveLinkstransport and labels frompackages/copy, tested in Vitest browser mode with Testing Library and MSW. - Render cache and backlinks: cache keyed by (entity,
content_seq, display type); backlinks list live links only. - Links by type across the graph:
GET /api/synergy/links?type=, cursor-paged, with a(user_id, type)index, so one list shows every open violation from any module (decision). - Search:
simplefull-text pluspg_trgmon display names,btree_ginindexes leading withuser_id,GET /api/synergy/search, and the test pinning spec story 71 (trigramword_similarity). - Playground and the RLS uuid fix: a playground page in
apps/webbehind the sandbox entitlements, built on the public API; Sidekick migrations 0002 and 0007 compare uuids directly instead of casting to text. - A test that no API response carries label text (keys, never copy), once the resolved link exists.
Decisions made while planning
Settled with Pratik on 2026-10-31 and 2026-11-01. Each can be revisited in its lesson.
- The mock providers are feature packages, not
ops. Nothing may importpackages/ops, which holds the privilegedpostgresclient, so neitherapps/webnor the registry could use mocks there. They live inpackages/features/_sandbox/notesandpackages/features/_sandbox/tasks: the underscore marks them dev-only, and retiring them deletes one folder. Two packages, not one, because a single provider and entitlement can't prove that disabling a module locks its links: revoking it would hide the note too. Provider and feature keys aresandbox-notesandsandbox-tasks. This overrides the PRD and spec (providerops, entitlementsandbox). - Routes in every lesson, not one API lesson at the end. Each lesson adds the routes for what it builds, and its tests go through them. Lesson 3 is the exception: entities are created through provider routes, which arrive in lesson 4, so lesson 3 tests the repository directly.
- Lesson size. The handover's 10 candidates for 3a became 13 lessons: relationships split into links, staleness, and lists with supersede; the save call split into mentions and anchors; the API lesson spread over the others; the outbox moved up so each later lesson writes its own events.
- The server half of the registry lands in 3a (lesson 6), because
POST /linksrefuseslockedand only a type's registration names its entitlement.addAnchorjoins it in lesson 10;resolveand the React half are 3b. - One Phase 3, with Checkpoint 3a. 3b is part of Phase 3's exit criteria, not a new phase, because Phase 4 needs its renderers and a new phase would renumber Phases 4 to 12. 3b is planned at the checkpoint.
- Delete models are chosen, not drifted into (Pratik, 2026-11-01). Lesson 3 revises row-level security and soft-delete support into explicit variants, with a guide for choosing among hard delete, soft delete with tombstones hidden (the default) and soft delete with tombstones visible. The guide lists every table and its model; each Drizzle schema file and each new migration names its model. Applied migrations are not edited.
- Cloudflare builds arkive on Node 24.
undici8 (through Astro'sunifont) needs Node 22.19 or later, and Pages built on 22.12. Cloudflare reads the version from the repo's.node-version(was22, now24), not from a dashboard setting; arkive'senginesasks for>=22.19.0. violatesandexception-toare in Phase 3 (Pratik, 2026-11-01). Dogfooding Synergy on Arkive found two gaps in the vocabulary: no directed violation and no exception. Both types are seeded in lesson 6. A violation has no status: unlinking resolves it, retyping it toexception-towaives it, and that retype takes a fresh snapshot. The graph-wide list of links by type (every open violation, from any module) is a 3b candidate; issue tracking is post-MVP (Violations and exceptions).- Deferred to Checkpoint 3a: whether the shared editor moves ahead of Taxila's passages (Phase 4) or the passages move to Phase 5, and which lessons retire the sandbox providers. Both shape Phase 4, not 3a; mock note bodies are minimal Tiptap JSON either way.
Handover from system design
Every item from the system-design session, and where it is built.
| Item | Where |
|---|---|
Layer folders: core, ui, copy into packages/foundation/; packages/platform/synergy; workspace globs; boundaries element types, policies and the unknown-files rule; live docs that name old paths; a post note in the Phase 2 walkthrough (decision, dependency flow 1.3.0) | lesson 1 |
| Per-package migrations in dependency order | lesson 2 |
provider, not module, in the schema and every API field | lessons 3, 4 |
content_seq: the stale-write check (stale_content), the ETag (decision) | lessons 3, 4 |
Lock first: pg_advisory_xact_lock(hashtextextended(user_id::text, 0)) before every Synergy write | lesson 3 |
Tombstones: RLS compares only user_id, the restore-aware update trigger (decision); revising the RLS and soft-delete templates | lesson 3 |
GET /entities/:id with If-None-Match and 304; the error-to-HTTP map; /server, /react and /contracts entry points | lesson 4 (/react stays empty until 3b) |
Outbox with xact_id, every event type, a read-only consumer role; outbox and idempotency keys hard-deleted; no pruning job until the first reader (decision) | lesson 5 (idempotency keys: lesson 6) |
violates (association) and exception-to (reference) seeded; an association-to-reference type change takes a fresh snapshot; the type filter on GET /entities/:id/links (decision) | lessons 6, 7, 11 |
List links: type NOT NULL with seeded untyped, role in the composite FK, relation types keyed by source_key and target_key, lexorank with unique (parent_id, position) | lessons 6, 8 |
Links and mentions: cited and mentioned, tombstone when both are false; the outgoing-reference roll-up (decision) | lessons 6, 7, 9 |
Supersede: superseded-by from old to each replacement, any number of successors | lesson 8 |
Anchors: to_anchor_id and quote, no from_anchor, the (to_id, to_anchor_id) index; the save call's current sets built with extractRefs; entity_anchors and entity_mentions; mock note bodies in minimal Tiptap JSON (decision) | lessons 9, 10 |
| Anchor receipt on link create | lesson 10 |
Restore checks: restore, undo and relink re-run the delete block and the cycle check, restore_conflict, deletion_batch_id | lesson 11 |
| Race tests with two connections; the cycle check on a large synthetic graph | lesson 13 |
| Keys, never copy: Synergy stores and returns keys only | every lesson; the response test in 3b |
| Testing: integration tests through the guarded API (testing layers) | every lesson |
React as a peer dependency; resolve, POST /links/resolve and the frozen resolved-link schema; the Sneak Peek Link in Vitest browser mode with MSW; render cache; simple full-text and pg_trgm; spec story 71 | 3b, planned at Checkpoint 3a |
| Editor order; the 409 auto-merge and the "Page updated" banner (built with the shared editor, not in Phase 3) | decided at Checkpoint 3a |