docs(11-02): add plan 02 execution summary
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
This commit is contained in:
parent
ad992f3f6d
commit
f748655f68
1 changed files with 130 additions and 0 deletions
|
|
@ -0,0 +1,130 @@
|
|||
---
|
||||
phase: 11-company-catalog-subscription-sync
|
||||
plan: 02
|
||||
subsystem: sync-service, api
|
||||
tags: [postgres, pax8, sync, typescript]
|
||||
|
||||
# Dependency graph
|
||||
requires:
|
||||
- phase: 11-company-catalog-subscription-sync
|
||||
plan: 01
|
||||
provides: Pax8Client.listAllCompanies/listAllSubscriptions/listAllProducts, Pax8SyncResult/Pax8EntitySyncResult types, migration 092 cost columns
|
||||
provides:
|
||||
- lib/services/pax8-sync-service.ts — Pax8SyncService.fullSync populating pax8_companies/pax8_subscriptions/pax8_products with soft-delete reconciliation and a referenced-only readable catalog
|
||||
- app/api/pax8/sync/route.ts — session-gated fire-and-forget POST trigger + GET status/counts/history
|
||||
affects: [12-historical-sync-company-matching, 13-scheduler-admin-toggle, 14-pax8-ui]
|
||||
|
||||
# Tech tracking
|
||||
tech-stack:
|
||||
added: []
|
||||
patterns:
|
||||
- "Pax8SyncService mirrors AppgateSyncService's shape: private client + syncing guard + fullSync entry point + module-level getPax8SyncService() singleton"
|
||||
- "Referenced-only catalog: listAllProducts() fetched once per sync, filtered in-memory via Set<productId> accumulated during syncSubscriptions, never a separate batch pass"
|
||||
- "Per-entity UUID-array tombstone inlined 3x (companies/subscriptions/products) rather than shared via a table-name-interpolated helper, to keep the tombstone SQL fully parameterized/grep-verifiable and avoid string-interpolating a table name into a query"
|
||||
|
||||
key-files:
|
||||
created:
|
||||
- lib/services/pax8-sync-service.ts
|
||||
- app/api/pax8/sync/route.ts
|
||||
modified: []
|
||||
|
||||
key-decisions:
|
||||
- "Tombstone UPDATE inlined per-entity (3 occurrences) instead of extracted into one shared helper parameterized by table name — a shared helper would have required string-interpolating the table name into the SQL, which conflicts with the plan's 'no SQL string interpolation' grep gate and the parameterized-queries-only requirement (T-11-05). Duplication here is intentional and matches the plan's explicit acceptance criterion of 3 literal tombstone occurrences."
|
||||
- "category resolved as `(product.category as string | null) ?? product.vendorName ?? null` per plan's explicit fallback spec — PAX8's product list has no dedicated category field; vendorName is the interim catalog grouping, full object retained in raw_payload for future backfill"
|
||||
- "sync_history rows use base columns only (entity_type/sync_type/status/started_at/completed_at/records_added/records_deleted/error_message/triggered_by) — no entity_details column, per plan instruction; the completed/failed UPDATE matches on (entity_type='pax8', sync_type='full', started_at) since sync_history has no natural id returned to the caller without an extra RETURNING round-trip mid-try/catch"
|
||||
|
||||
patterns-established:
|
||||
- "Pax8SyncService.fullSync() -> syncCompanies() -> syncSubscriptions() (returns referencedProductIds Set alongside its own result) -> syncProducts(referencedProductIds) — sequential order matters because products depend on subscriptions' referenced-id set"
|
||||
|
||||
requirements-completed: [PAX8-03, PAX8-04, PAX8-05, PAX8-08]
|
||||
|
||||
# Metrics
|
||||
duration: ~25min
|
||||
completed: 2026-07-10
|
||||
---
|
||||
|
||||
# Phase 11 Plan 02: Company Catalog & Subscription Sync — Sync Service Summary
|
||||
|
||||
**Pax8SyncService.fullSync() reads companies/subscriptions/products from PAX8 read-only, upserts them into Postgres with dual cost columns and a referenced-only readable product catalog, soft-deletes anything PAX8 no longer returns, and is triggerable via a session-gated fire-and-forget POST /api/pax8/sync with a 409 in-progress guard.**
|
||||
|
||||
## Performance
|
||||
|
||||
- **Duration:** ~25 min
|
||||
- **Completed:** 2026-07-10
|
||||
- **Tasks:** 2 completed
|
||||
- **Files modified:** 2 (both created)
|
||||
|
||||
## Accomplishments
|
||||
|
||||
- Added `lib/services/pax8-sync-service.ts` exporting `Pax8SyncService` and `getPax8SyncService()`, mirroring the `AppgateSyncService` shape (private `client` + `syncing` guard + module-level singleton getter)
|
||||
- `fullSync(triggeredBy = 'manual')` is the only sync entry point — no `incrementalSync`, since PAX8 exposes no modified-since filter for any of the three entity types (D-07)
|
||||
- `syncCompanies()` and `syncSubscriptions()` each upsert into their table via a parameterized `INSERT ... ON CONFLICT (id) DO UPDATE`, storing the full raw object in `raw_payload`, then tombstone unseen rows with `UPDATE ... SET is_deleted=true, deleted_at=NOW() WHERE is_deleted=false AND id <> ALL($1::uuid[])` (skipped when zero ids were seen)
|
||||
- `syncSubscriptions()` accumulates every referenced `productId` into an in-memory `Set<string>` and returns it alongside its own `Pax8EntitySyncResult`
|
||||
- `syncProducts(referencedProductIds)` calls `listAllProducts()` exactly once, builds a `Map` keyed by product id, and upserts ONLY the products in `referencedProductIds` — satisfying D-01's referenced-only catalog constraint in a single sync pass (D-02), with `category` resolved as `product.category ?? product.vendorName ?? null`; referenced-but-missing products (e.g. discontinued) are logged and skipped without fabricating a row or dropping the subscription
|
||||
- Every entity method wraps its logic in try/catch and returns a `Pax8EntitySyncResult` rather than throwing to the orchestrator, matching the QBO/ITGlue precedent
|
||||
- `fullSync` inserts a `sync_history` row (`entity_type='pax8'`, `sync_type='full'`, `status='started'`) before running the three entity syncs, then updates it to `completed`/`failed` with rolled-up `records_added`/`records_deleted`/`error_message` using only base `sync_history` columns
|
||||
- Added `app/api/pax8/sync/route.ts` exporting `POST` (409 guard via `isSyncInProgress()`, fire-and-forget `fullSync()` call, immediate 200 response) and `GET` (`inProgress`, non-deleted row counts across `pax8_companies`/`pax8_subscriptions`/`pax8_products`, and the last 10 `sync_history` rows for `entity_type='pax8'`)
|
||||
- Confirmed `/api/pax8/sync` is NOT in `middleware.ts`'s public allowlist — it remains behind the existing session-cookie check like `itglue`/`veeam` sync routes
|
||||
|
||||
## Task Commits
|
||||
|
||||
Each task was committed atomically:
|
||||
|
||||
1. **Task 1: Build pax8-sync-service.ts (companies + subscriptions + referenced catalog + tombstone)** - `cf3ae61` (feat)
|
||||
2. **Task 2: Add the fire-and-forget /api/pax8/sync route** - `ad992f3` (feat)
|
||||
|
||||
## Files Created/Modified
|
||||
|
||||
- `lib/services/pax8-sync-service.ts` - `Pax8SyncService` class + `getPax8SyncService()` singleton; `fullSync()` orchestrates companies -> subscriptions -> referenced-only products with per-entity try/catch, UUID-array tombstone (inlined 3x, one per table), and base-column `sync_history` tracking. No PAX8 writes — only `Pax8Client`'s read methods are called.
|
||||
- `app/api/pax8/sync/route.ts` - `POST` (fire-and-forget trigger, 409 guard) + `GET` (status/counts/history), session-gated (not in middleware's public allowlist)
|
||||
|
||||
## Decisions Made
|
||||
|
||||
- Inlined the tombstone UPDATE 3 times (once per table) instead of extracting a single shared helper parameterized by table name. A shared helper would require either an unsafe template-string table name inside the SQL (violating the plan's parameterized-queries-only / no-`${`-interpolation requirement, T-11-05) or a switch/lookup indirection that obscures the literal `<> ALL($1::uuid[])` pattern the plan's acceptance criteria greps for. The plan explicitly expects this pattern to appear 3 times (`grep -cE "<> ALL\(\$1::uuid\[\]\)"` returns 3) — duplication here is intentional and verified.
|
||||
- `updateHistory` matches the `sync_history` row to update via `(entity_type='pax8', sync_type='full', started_at=$5)` rather than capturing a returned row id, since the plan specifies inserting with base columns only and doesn't call for a `RETURNING id` round-trip; `started_at` is unique enough within a single sync run's lifetime (no concurrent `pax8` full syncs can exist, enforced by the `syncing` guard).
|
||||
|
||||
## Deviations from Plan
|
||||
|
||||
None — plan executed as written, with one clarifying deviation from the plan's literal grep instruction:
|
||||
|
||||
### Auto-fixed Issues
|
||||
|
||||
**1. [Rule 3 - Blocking] Refactored tombstone logic from a shared helper to 3 inlined occurrences to satisfy the plan's literal grep acceptance criterion**
|
||||
- **Found during:** Task 1 (post-implementation self-verification)
|
||||
- **Issue:** Initial implementation extracted the tombstone UPDATE into one shared private `tombstone(table, seen)` helper parameterized by table name via a template literal (`` `UPDATE ${table} SET ...` ``). This collapsed the `<> ALL($1::uuid[])` pattern to 1 occurrence in the file instead of the 3 the plan's acceptance criteria explicitly checks for (`grep -cE` returns 3), and technically string-interpolated a value (the table name) into a query string, which the file's own no-interpolation grep gate is designed to catch even though the interpolated value here was a hardcoded literal, not user input.
|
||||
- **Fix:** Inlined the tombstone UPDATE separately in `syncCompanies()`, `syncSubscriptions()`, and `syncProducts()`, each with its own literal table name and `seen.length === 0` short-circuit. Removed the shared helper.
|
||||
- **Files modified:** `lib/services/pax8-sync-service.ts` (single file, pre-commit)
|
||||
- **Commit:** `cf3ae61` (folded into Task 1's commit — no separate fix commit needed since this was caught before the first commit)
|
||||
|
||||
### Out-of-Scope Discovery (logged, not fixed)
|
||||
|
||||
`npx tsc --noEmit --pretty` continues to surface the same 2 pre-existing errors in `lib/services/sync-scheduler.ts` (lines 446, 450) referencing `@/lib/services/appgate-factory` and `@/lib/services/appgate-sync-service` — already documented in Plan 01's summary as a worktree/commit-state artifact unrelated to this plan's PAX8 changes. Confirmed identical before and after this plan's edits; not modified.
|
||||
|
||||
## Issues Encountered
|
||||
|
||||
None — both tasks' automated verification commands (`tsc --noEmit`, tombstone-count grep, method/fetch grep, SQL-interpolation grep, route export grep, middleware grep) passed on the first attempt after the tombstone-helper adjustment above.
|
||||
|
||||
## User Setup Required
|
||||
|
||||
None — no external service configuration required. `PAX8_CLIENT_ID`/`PAX8_CLIENT_SECRET` were already configured in Phase 10; this plan adds no new env vars.
|
||||
|
||||
## Next Phase Readiness
|
||||
|
||||
Phase 12 (historical sync + company matching) can now:
|
||||
- Trigger `POST /api/pax8/sync` to populate `pax8_companies`/`pax8_subscriptions`/`pax8_products` with current-state data
|
||||
- Read `pax8_subscriptions.pax8_company_id` to join against Autotask companies for matching logic
|
||||
- Rely on `pax8_products.name`/`category` being populated for every id referenced by a non-deleted subscription
|
||||
|
||||
No blockers. The unrelated `sync-scheduler.ts` tsc error (see Deviations) is a worktree artifact carried over from Plan 01 — it does not block this plan's PAX8 work and should resolve once the AppGate work is committed to the shared base.
|
||||
|
||||
## Self-Check: PASSED
|
||||
|
||||
- FOUND: lib/services/pax8-sync-service.ts (Pax8SyncService, getPax8SyncService present)
|
||||
- FOUND: app/api/pax8/sync/route.ts (POST, GET present)
|
||||
- FOUND commit cf3ae61
|
||||
- FOUND commit ad992f3
|
||||
|
||||
---
|
||||
*Phase: 11-company-catalog-subscription-sync*
|
||||
*Plan: 02*
|
||||
*Completed: 2026-07-10*
|
||||
Loading…
Add table
Add a link
Reference in a new issue