From c8dd64f766972ed9d855b27ef5cb0ad9a29929f7 Mon Sep 17 00:00:00 2001 From: lorentz Date: Fri, 10 Jul 2026 19:04:29 -0400 Subject: [PATCH] docs(11): create phase plan Co-Authored-By: Claude Sonnet 5 Claude-Session: https://claude.ai/code/session_01LHRgZqkzBHBbAbc3KHneuR --- .planning/ROADMAP.md | 5 +- .../11-01-PLAN.md | 240 ++++++++++++++++ .../11-02-PLAN.md | 259 ++++++++++++++++++ .../11-03-PLAN.md | 175 ++++++++++++ 4 files changed, 678 insertions(+), 1 deletion(-) create mode 100644 .planning/phases/11-company-catalog-subscription-sync/11-01-PLAN.md create mode 100644 .planning/phases/11-company-catalog-subscription-sync/11-02-PLAN.md create mode 100644 .planning/phases/11-company-catalog-subscription-sync/11-03-PLAN.md diff --git a/.planning/ROADMAP.md b/.planning/ROADMAP.md index 3377dfa..a129b91 100644 --- a/.planning/ROADMAP.md +++ b/.planning/ROADMAP.md @@ -269,7 +269,10 @@ render, including the manual-resolution workflow for flagged companies. 2. Running the sync populates a product/catalog table (SKUs, categories) and a subscriptions table (product, seat count, billing term) per company 3. A synced subscription row displays a readable product name and category by joining to the catalog table — not a bare SKU/product ID 4. No code path in the PAX8 client or this sync service issues a write (POST/PUT/PATCH/DELETE) to the PAX8 API — every call is a read, verified by inspection of the client's exposed methods -**Plans**: TBD +**Plans**: 3 plans +- [ ] 11-01-PLAN.md — Migration 092 subscription cost columns + extend pax8 types + read-only client pagination helpers (PAX8-04, PAX8-05, PAX8-08) +- [ ] 11-02-PLAN.md — pax8-sync-service.ts (companies + subscriptions + referenced-only catalog + soft-delete reconciliation) + /api/pax8/sync fire-and-forget route (PAX8-03, PAX8-04, PAX8-05, PAX8-08) +- [ ] 11-03-PLAN.md — Read-only invariant proof + live sync run DB verification checkpoint (PAX8-03, PAX8-04, PAX8-05, PAX8-08) **UI hint**: no ### Phase 12: Orders/Invoices & Company Matching diff --git a/.planning/phases/11-company-catalog-subscription-sync/11-01-PLAN.md b/.planning/phases/11-company-catalog-subscription-sync/11-01-PLAN.md new file mode 100644 index 0000000..dc3e9f3 --- /dev/null +++ b/.planning/phases/11-company-catalog-subscription-sync/11-01-PLAN.md @@ -0,0 +1,240 @@ +--- +phase: 11-company-catalog-subscription-sync +plan: 01 +type: execute +wave: 1 +depends_on: [] +files_modified: + - migrations/092_pax8_subscription_costs.sql + - lib/types/pax8.ts + - lib/services/pax8-client.ts + - lib/services/pax8-client.test.ts +autonomous: true +requirements: [PAX8-04, PAX8-05, PAX8-08] + +must_haves: + truths: + - "pax8_subscriptions has price, partner_cost, and currency columns so both the customer price and the partner cost can be stored (D-03/D-04)" + - "The PAX8 client can page through all companies, all subscriptions, and all products with read-only GET calls" + - "No PUT/PATCH/DELETE call exists in the client; the only POST is the OAuth token exchange (PAX8-08)" + artifacts: + - path: "migrations/092_pax8_subscription_costs.sql" + provides: "price + partner_cost + currency columns on pax8_subscriptions" + contains: "ALTER TABLE pax8_subscriptions" + - path: "lib/types/pax8.ts" + provides: "cost fields on Pax8Subscription, vendorName on Pax8Product, Pax8SyncResult/Pax8EntitySyncResult" + contains: "partnerCost" + - path: "lib/services/pax8-client.ts" + provides: "listAllCompanies / listAllSubscriptions / listAllProducts pagination helpers" + exports: ["Pax8Client"] + - path: "lib/services/pax8-client.test.ts" + provides: "mocked-fetch coverage for the new pagination helpers" + key_links: + - from: "lib/services/pax8-client.ts" + to: "lib/types/pax8.ts" + via: "imports Pax8Company/Pax8Subscription/Pax8Product/Pax8PageEnvelope" + pattern: "from '@/lib/types/pax8'" +--- + + +Lay down the data contract layer the Phase 11 sync service consumes: the missing +subscription cost columns (D-03/D-04), the extended TypeScript types, and the +read-only pagination helpers on the PAX8 client. + +Purpose: The sync service (Plan 02) must not have to discover the schema, the +API response shapes, or invent pagination. This plan defines all three up front +(interface-first) so Plan 02 is pure orchestration against known contracts. + +Output: migration 092 (applied to dev DB), extended `lib/types/pax8.ts`, new +read-only client methods with mocked-fetch tests. + + + +@$HOME/.claude/get-shit-done/workflows/execute-plan.md +@$HOME/.claude/get-shit-done/templates/summary.md + + + +@.planning/PROJECT.md +@.planning/ROADMAP.md +@.planning/STATE.md +@.planning/phases/11-company-catalog-subscription-sync/11-CONTEXT.md +@.planning/phases/11-company-catalog-subscription-sync/11-PATTERNS.md + + + + + + + + + + + + + + + + + + + + + + + + + + Task 1: Add subscription cost columns (migration 092) + migrations/092_pax8_subscription_costs.sql + + - migrations/091_pax8_tables.sql (existing pax8_subscriptions shape and the currency CHAR(3) DEFAULT 'USD' convention used on pax8_orders/pax8_order_items) + - CLAUDE.md "Database" + "Migrations" sections (IF NOT EXISTS convention; migrations apply on Postgres init only — an existing volume needs a manual apply) + - MEMORY note: "Postgres init applies migrations on first volume boot only" — dev DB needs manual apply via docker exec psql + + + Create migration `092_pax8_subscription_costs.sql` that adds three columns to + the existing `pax8_subscriptions` table using `ALTER TABLE pax8_subscriptions + ADD COLUMN IF NOT EXISTS`: + (1) `price NUMERIC(12,2)` — the customer/list price PAX8 returns per subscription (D-03), + (2) `partner_cost NUMERIC(12,2)` — the partner/reseller cost PAX8 returns per subscription (D-03), + (3) `currency CHAR(3) NOT NULL DEFAULT 'USD'` — matching the `currency` column convention already on pax8_orders/pax8_order_items in migration 091. + Store the raw per-billing-period amounts exactly as PAX8 returns them, paired + with the existing `billing_term` column — do NOT add any monthly-normalization + columns (D-04 defers that to a read-time concern). Add a leading comment block + citing that PAX8's Subscription object exposes both `price` and `partnerCost` + (Phase 11 research, devx.pax8.com findsubscriptions). Do not alter any other + column or table. After writing the file, apply it to the running dev database + manually (the Postgres container will not re-run migrations on an existing + volume): run the ALTER statements via `docker exec` against the pulse-postgres + container per the MEMORY caveat. + + + - `migrations/092_pax8_subscription_costs.sql` exists and contains exactly three `ADD COLUMN IF NOT EXISTS` clauses for `price`, `partner_cost`, and `currency` + - The file contains no `DROP`, no destructive statement, and touches only `pax8_subscriptions` + - `grep -cE "ADD COLUMN IF NOT EXISTS" migrations/092_pax8_subscription_costs.sql` returns 3 + - After manual apply, `\d pax8_subscriptions` in the dev DB lists `price`, `partner_cost`, and `currency` columns + + + test -f migrations/092_pax8_subscription_costs.sql && grep -cE "ADD COLUMN IF NOT EXISTS" migrations/092_pax8_subscription_costs.sql | grep -qx 3 && echo OK + + Migration file created with price/partner_cost/currency columns and applied to the dev DB. + + + + Task 2: Extend PAX8 types with cost fields and sync-result shapes + lib/types/pax8.ts + + - lib/types/pax8.ts (current Pax8Company/Pax8Subscription/Pax8Product/Pax8PageEnvelope; note the `[key: string]: unknown` escape hatch on each interface) + - lib/types/appgate.ts (AppgateSyncResult / AppgateEntity result shape to mirror for Pax8SyncResult/Pax8EntitySyncResult) + + + Extend `Pax8Subscription` with the pricing/lifecycle fields PAX8 actually + returns: `price: number | null`, `partnerCost: number | null`, + `currencyCode: string | null`, `productName: string | null`, + `endDate: string | null`, `updatedDate: string | null` (keep the existing + id/companyId/productId/quantity/billingTerm/status/startDate fields and the + escape hatch). Extend `Pax8Product` with `vendorName: string | null` and + `shortDescription: string | null` (keep existing fields; leave the existing + `category` typing in place — the sync will populate it from `category ?? + vendorName`). Add two new exported interfaces mirroring the AppGate result + shape: `Pax8EntitySyncResult` with fields `entity: string`, `success: + boolean`, `upserted: number`, `tombstoned: number`, `durationMs: number`, + `error?: string`; and `Pax8SyncResult` with fields `syncId: string`, + `status: 'completed' | 'failed'`, `startedAt: Date`, `completedAt: Date | + null`, `durationMs: number`, `entities: Pax8EntitySyncResult[]`, `errors: + string[]`. Do not remove or rename any existing exported member. + + + - `Pax8Subscription` declares `price`, `partnerCost`, and `currencyCode` members + - `Pax8Product` declares `vendorName` + - `Pax8SyncResult` and `Pax8EntitySyncResult` are exported + - `npx tsc --noEmit --pretty` passes with no new errors + - `grep -c "partnerCost" lib/types/pax8.ts` is at least 1 + + + npx tsc --noEmit --pretty && grep -q "partnerCost" lib/types/pax8.ts && grep -q "Pax8SyncResult" lib/types/pax8.ts && echo OK + + Types compile and expose the cost fields plus the sync-result shapes the service will consume. + + + + Task 3: Add read-only pagination helpers to the PAX8 client + lib/services/pax8-client.ts, lib/services/pax8-client.test.ts + + - lib/services/pax8-client.ts (existing getToken, fetchJson with 429/Retry-After backoff, single-page listCompanies — do NOT change these signatures; verify-pax8-auth.ts from Phase 10 depends on listCompanies) + - lib/services/pax8-client.test.ts (the vi.stubGlobal('fetch', ...) mocking convention keyed by URL substring, vi.unstubAllGlobals in beforeEach — follow this exact style, do not add nock/msw) + - lib/types/pax8.ts (Pax8PageEnvelope shape: { content, page: { size, totalElements, totalPages, number } }) + + + - listAllCompanies() returns every company across all pages: given a mocked 2-page response (page 0 with content + totalPages:2, page 1 with content + totalPages:2), the returned array concatenates both pages' content in order + - listAllSubscriptions() and listAllProducts() behave the same across pages + - Each helper requests size=200 and stops once page.number >= page.totalPages - 1 (no infinite loop when totalPages is 1) + - Every request the helpers issue is a GET with an `Authorization: Bearer` header (asserted via the fetch mock); no request sets method POST/PUT/PATCH/DELETE + + + Add three async methods to `Pax8Client`, each looping pages via the existing + private `fetchJson()` (so 429 backoff is inherited) and requesting + `size=200`: `listAllCompanies(): Promise`, + `listAllSubscriptions(): Promise`, and + `listAllProducts(): Promise`. Each fetches + `/companies?page=N&size=200` (respectively `/subscriptions`, `/products`), + accumulates `envelope.content`, and increments N until `envelope.page.number + >= envelope.page.totalPages - 1`, then returns the accumulated array. Keep the + existing single-page `listCompanies(page, size)` method unchanged. Do NOT add + any method that issues a mutating HTTP verb — these are read-only list reads + only (PAX8-08). Add tests in `pax8-client.test.ts` following the existing + mocked-fetch style: assert multi-page concatenation for at least + listAllSubscriptions, assert the request URLs carry `size=200`, and assert the + fetch mock only ever sees GET requests (no method override) plus the bearer + header. + + + - `Pax8Client` exposes `listAllCompanies`, `listAllSubscriptions`, `listAllProducts` + - Existing `listCompanies(page, size)` signature is unchanged + - No new method sets `method:` to POST/PUT/PATCH/DELETE + - `grep -nE "method:\s*['\"](PUT|PATCH|DELETE)" lib/services/pax8-client.ts` returns nothing (exit 1) + - `npx vitest run lib/services/pax8-client.test.ts` passes, including a multi-page concatenation test + + + npx vitest run lib/services/pax8-client.test.ts && ! grep -nE "method:[[:space:]]*['\"](PUT|PATCH|DELETE)" lib/services/pax8-client.ts && echo OK + + Client can enumerate all companies/subscriptions/products via read-only GET pagination, with passing mocked tests. + + + + + +## Trust Boundaries + +| Boundary | Description | +|----------|-------------| +| Pulse server → PAX8 API | Outbound OAuth2 client-credentials calls; client secret crosses here | +| Migration → Postgres | DDL applied to the pax8_subscriptions table | + +## STRIDE Threat Register + +| Threat ID | Category | Component | Disposition | Mitigation Plan | +|-----------|----------|-----------|-------------|-----------------| +| T-11-01 | Information Disclosure | pax8-client.ts token exchange | mitigate | Never log/interpolate `config.clientSecret`; existing getToken already errors with status text only, not the body echo of credentials — new pagination methods add no logging of secrets | +| T-11-02 | Elevation of Privilege | pax8-client.ts new read methods | mitigate | New methods issue GET only; grep gate asserts no PUT/PATCH/DELETE added; the only POST in the file remains the `/v1/token` auth handshake (not a data write) — upholds PAX8-08 at the client layer | +| T-11-03 | Tampering | migration 092 DDL | accept | Additive ADD COLUMN IF NOT EXISTS only, no destructive ops; idempotent on rerun; low risk | +| T-11-SC | Tampering | package installs | accept | This plan installs zero npm/pip/cargo packages (native fetch + existing pg) — package legitimacy gate not triggered | + + + +- `npx tsc --noEmit --pretty` passes +- `npx vitest run lib/services/pax8-client.test.ts` passes +- Migration 092 present and applied to dev DB (`\d pax8_subscriptions` shows price/partner_cost/currency) +- No PUT/PATCH/DELETE method in pax8-client.ts + + + +pax8_subscriptions carries price/partner_cost/currency columns; lib/types/pax8.ts +exposes the cost fields and sync-result shapes; the PAX8 client can page through +all three entity types with read-only GET calls covered by mocked tests. + + + +Create `.planning/phases/11-company-catalog-subscription-sync/11-01-SUMMARY.md` when done + diff --git a/.planning/phases/11-company-catalog-subscription-sync/11-02-PLAN.md b/.planning/phases/11-company-catalog-subscription-sync/11-02-PLAN.md new file mode 100644 index 0000000..4ed265a --- /dev/null +++ b/.planning/phases/11-company-catalog-subscription-sync/11-02-PLAN.md @@ -0,0 +1,259 @@ +--- +phase: 11-company-catalog-subscription-sync +plan: 02 +type: execute +wave: 2 +depends_on: [11-01] +files_modified: + - lib/services/pax8-sync-service.ts + - app/api/pax8/sync/route.ts +autonomous: true +requirements: [PAX8-03, PAX8-04, PAX8-05, PAX8-08] + +must_haves: + truths: + - "A full sync upserts every current PAX8 company into pax8_companies (PAX8-03)" + - "A full sync upserts every current PAX8 subscription with product, quantity, billing term, customer price, and partner cost (PAX8-04)" + - "The catalog table (pax8_products) holds a readable name + category for every product referenced by a subscription, populated from the full product list — not bare SKU IDs (PAX8-05, D-01)" + - "Companies/subscriptions/products no longer returned by PAX8 are soft-deleted (is_deleted=true, deleted_at set), never hard-removed (D-05/D-06)" + - "No code path in the sync service issues a PAX8 API write; it calls only the client's read methods (PAX8-08)" + - "POST /api/pax8/sync starts the sync fire-and-forget and returns immediately; a concurrent trigger returns 409" + artifacts: + - path: "lib/services/pax8-sync-service.ts" + provides: "Pax8SyncService.fullSync orchestrating companies→subscriptions→products with tombstone reconciliation" + exports: ["Pax8SyncService", "getPax8SyncService"] + - path: "app/api/pax8/sync/route.ts" + provides: "fire-and-forget POST trigger + GET status" + exports: ["POST", "GET"] + key_links: + - from: "lib/services/pax8-sync-service.ts" + to: "getPax8Client()" + via: "constructor: this.client = client ?? getPax8Client()" + pattern: "getPax8Client" + - from: "lib/services/pax8-sync-service.ts" + to: "postgresClient" + via: "parameterized upsert + UUID-array tombstone" + pattern: "postgresClient\\.query" + - from: "pax8_subscriptions.product_id" + to: "pax8_products.id" + via: "readable-name join (D-01 referenced-only catalog)" + pattern: "product_id" + - from: "app/api/pax8/sync/route.ts" + to: "getPax8SyncService().fullSync" + via: "fire-and-forget call" + pattern: "fullSync" +--- + + +Build the read-only PAX8 current-state sync: companies, subscriptions (with dual +cost), and the referenced-only product catalog — plus the fire-and-forget trigger +route. This is the phase's core deliverable. + +Purpose: Populate pax8_companies / pax8_subscriptions / pax8_products so a +subscription can be shown with a readable product name and category (not a bare +SKU), with removed entities soft-deleted and zero writes back to PAX8. + +Output: `lib/services/pax8-sync-service.ts` and `app/api/pax8/sync/route.ts`. + + + +@$HOME/.claude/get-shit-done/workflows/execute-plan.md +@$HOME/.claude/get-shit-done/templates/summary.md + + + +@.planning/PROJECT.md +@.planning/ROADMAP.md +@.planning/STATE.md +@.planning/phases/11-company-catalog-subscription-sync/11-CONTEXT.md +@.planning/phases/11-company-catalog-subscription-sync/11-PATTERNS.md +@.planning/phases/11-company-catalog-subscription-sync/11-01-SUMMARY.md + + + + + + + + + + + + + + + + + + + + + + + + + Task 1: Build pax8-sync-service.ts (companies + subscriptions + referenced catalog + tombstone) + lib/services/pax8-sync-service.ts + + - lib/services/appgate-sync-service.ts (PRIMARY analog: class + `syncing` guard + singleton getter; the UUID-keyed upsert + `id <> ALL($1::uuid[])` tombstone in syncAppliances, lines ~185-244; run()/persistHistory shape) + - lib/services/qbo-sync-service.ts (per-entity try/catch returning a result rather than throwing; tombstone query shape) + - lib/services/veeam-sync-service.ts lines 78-165 (generic sync_history insert 'started' → update completed/failed using only base columns entity_type/sync_type/status/records_added/records_deleted/error_message/triggered_by) + - lib/services/postgres-client.ts (postgresClient.query signature; note softDeleteMany casts ::bigint[] and is NOT usable for UUID ids — write the tombstone query raw) + - lib/services/pax8-client.ts + lib/services/pax8-factory.ts (getPax8Client, the new list-all methods) + - migrations/091_pax8_tables.sql + migrations/092_pax8_subscription_costs.sql (exact column names for pax8_companies, pax8_subscriptions, pax8_products) + - lib/types/pax8.ts (Pax8SyncResult/Pax8EntitySyncResult, entity field names) + + + Create `Pax8SyncService` mirroring `appgate-sync-service.ts`'s class shape: a + private `client: Pax8Client` set in the constructor via `this.client = client + ?? getPax8Client()`, a private `syncing = false` guard, an `isSyncInProgress(): + boolean`, and a module-level `getPax8SyncService()` singleton. Expose one + public entry point `async fullSync(triggeredBy = 'manual'): Promise` + — there is NO incrementalSync because no PAX8 entity supports a modified-since + filter (D-07 resolves to full-sync-only for all entity types). fullSync must: + throw if `this.syncing` is already true; set the guard; insert a + `sync_history` row with `entity_type='pax8'`, `sync_type='full'`, + `status='started'`, `triggered_by=triggeredBy` (base columns only — do not use + entity_details); then run three entity syncs IN THIS ORDER inside try/catch, + collecting a Pax8EntitySyncResult each, and finally update the sync_history row + to 'completed' (or 'failed' if any entity errored) with records_added = + total upserted and records_deleted = total tombstoned, and clear the guard in a + finally block. + + syncCompanies(): call `this.client.listAllCompanies()`; for each company upsert + into pax8_companies via a parameterized `INSERT ... ON CONFLICT (id) DO UPDATE + SET ... synced_at = NOW(), is_deleted = false, deleted_at = NULL` (mapping + name/external_id/website/status/city/state_or_province/postal_code/country and + storing the full object in raw_payload); collect seen ids; after the loop + tombstone with `UPDATE pax8_companies SET is_deleted=true, deleted_at=NOW() + WHERE is_deleted=false AND id <> ALL($1::uuid[])` (skip the tombstone UPDATE + when zero ids were seen, to avoid deleting everything on an empty/failed pull). + + syncSubscriptions(): call `this.client.listAllSubscriptions()`; for each + subscription upsert into pax8_subscriptions mapping pax8_company_id (from + companyId), product_id (from productId), quantity, billing_term, status, + start_date, and the Plan-01 columns price (from `price`), partner_cost (from + `partnerCost`), currency (from `currencyCode` ?? 'USD') plus raw_payload; + accumulate every referenced productId into an in-memory Set and return it + alongside the entity result so syncProducts can consume it; collect seen ids; + tombstone unseen subscriptions with the same `<> ALL($1::uuid[])` pattern. + + syncProducts(referencedProductIds: Set): because PAX8 exposes NO + single-product-detail endpoint, call `this.client.listAllProducts()` once, build + a Map keyed by product id, then upsert ONLY the products whose id is in + referencedProductIds (this satisfies D-01 "referenced-only catalog" — the full + list is fetched into memory but only referenced rows are STORED). Map name from + `name`, sku from `sku`, vendor_sku from `vendorSku`, and category from + `(product as any).category ?? product.vendorName ?? null` (PAX8's product list + endpoint has no dedicated category field per Phase 11 research; vendorName is + the catalog grouping — the full object is kept in raw_payload so a true category + can be backfilled later). If a referenced product id is not present in the + fetched catalog (e.g. discontinued), log it and continue — do NOT fabricate a + row and do NOT drop the subscription (the subscription's raw_payload retains + productName). The "seen" set for the products tombstone is referencedProductIds: + `UPDATE pax8_products SET is_deleted=true, deleted_at=NOW() WHERE is_deleted=false + AND id <> ALL($1::uuid[])` (skip when the set is empty) — this keeps discontinued + catalog rows soft-deleted per D-06 while never hard-deleting them. + + Every entity method wraps its own logic in try/catch and returns a + Pax8EntitySyncResult { entity, success, upserted, tombstoned, durationMs, + error? } rather than throwing to the orchestrator (qbo/itglue precedent). Use + ONLY parameterized queries ($1,$2,...) — never string-interpolate values into + SQL. The service must call only the client's read methods; it must never issue + an HTTP request itself and never call a mutating PAX8 verb (PAX8-08). + + + - Exports `Pax8SyncService` and `getPax8SyncService` + - `fullSync` exists; there is no `incrementalSync` (full-sync-only per D-07) + - Products are upserted only when referenced by a subscription (D-01) and the full catalog is fetched via listAllProducts (no /products/{id} call) + - category is populated from `category ?? vendorName` and each of the three tables' tombstone uses `id <> ALL($1::uuid[])` + - `grep -cE "<> ALL\(\\\$1::uuid\[\]\)" lib/services/pax8-sync-service.ts` returns 3 (companies, subscriptions, products) + - `grep -nE "method:[[:space:]]*['\"](POST|PUT|PATCH|DELETE)|fetch\(" lib/services/pax8-sync-service.ts` returns nothing (the service issues no HTTP itself; PAX8-08) + - No SQL string interpolation: `grep -nE "query\(\s*[\`'\"].*\\\$\{" lib/services/pax8-sync-service.ts` returns nothing + - `npx tsc --noEmit --pretty` passes + + + npx tsc --noEmit --pretty && ! grep -nE "method:[[:space:]]*['\"](POST|PUT|PATCH|DELETE)" lib/services/pax8-sync-service.ts && ! grep -nE "[^.]fetch\(" lib/services/pax8-sync-service.ts && echo OK + + Pax8SyncService.fullSync populates all three tables read-only with soft-delete reconciliation and referenced-only catalog; compiles clean; no PAX8 writes. + + + + Task 2: Add the fire-and-forget /api/pax8/sync route + app/api/pax8/sync/route.ts + + - app/api/itglue/sync/route.ts (cleanest fire-and-forget analog: 409 guard, background .catch, GET status with counts + recent history) + - app/api/veeam/sync/route.ts lines 13-46 (409 in-progress guard shape) + - lib/services/pax8-sync-service.ts (getPax8SyncService, isSyncInProgress, fullSync — built in Task 1) + - middleware.ts (public-route allowlist — confirm /api/pax8/sync is NOT added; it stays behind the session-cookie check like other sync routes, unlike webhooks) + + + Create `app/api/pax8/sync/route.ts` exporting `POST` and `GET`. POST: read the + JSON body defensively (`await req.json().catch(() => ({}))`), take + `triggeredBy = body.triggeredBy || 'manual'`, get the service via + `getPax8SyncService()`, return `NextResponse.json({ error: 'Sync already in + progress' }, { status: 409 })` if `isSyncInProgress()`, otherwise call + `svc.fullSync(triggeredBy).catch(err => console.error('[Pax8Sync] Background + sync error:', err.message))` WITHOUT awaiting (fire-and-forget) and return + `NextResponse.json({ ok: true, message: 'PAX8 sync started' })`. GET: return + `{ inProgress: svc.isSyncInProgress(), counts, history }` where counts are + non-deleted row counts from pax8_companies/pax8_subscriptions/pax8_products + (single parameterized/constant query) and history is the last 10 `sync_history` + rows WHERE `entity_type='pax8'` ordered by started_at DESC, wrapped in + try/catch returning `{ error }` with status 500 on failure. Do NOT add + `/api/pax8/sync` to middleware.ts's public-route allowlist — it must remain + behind the existing session-cookie check. + + + - Exports both `POST` and `GET` + - POST returns 409 when `isSyncInProgress()` is true and calls `fullSync` without `await` otherwise + - `grep -c "pax8/sync\|/api/pax8" middleware.ts` returns 0 (route is not made public) + - `npx tsc --noEmit --pretty` passes + + + npx tsc --noEmit --pretty && grep -q "export async function POST" app/api/pax8/sync/route.ts && grep -q "export async function GET" app/api/pax8/sync/route.ts && ! grep -qE "/api/pax8" middleware.ts && echo OK + + POST /api/pax8/sync triggers fullSync fire-and-forget with a 409 guard; GET reports status/counts/history; route stays session-gated. + + + + + +## Trust Boundaries + +| Boundary | Description | +|----------|-------------| +| Pulse server → PAX8 API | Outbound read-only GET calls via getPax8Client() | +| PAX8 response → Postgres | Untrusted external data written via bulk upsert | +| Client (browser/session) → /api/pax8/sync | Authenticated trigger; session-cookie gated by middleware | + +## STRIDE Threat Register + +| Threat ID | Category | Component | Disposition | Mitigation Plan | +|-----------|----------|-----------|-------------|-----------------| +| T-11-04 | Elevation of Privilege | pax8-sync-service.ts (ROADMAP SC#4 write-invariant) | mitigate | Service calls only client read methods; issues no HTTP itself; grep gate asserts no POST/PUT/PATCH/DELETE and no `fetch(` in the service — upholds PAX8-08 | +| T-11-05 | Tampering (SQL injection) | pax8-sync-service.ts upsert/tombstone | mitigate | All queries parameterized ($1,$2,…); UUID arrays passed as `$1::uuid[]` params, never interpolated; grep gate asserts no `${` template interpolation inside query() | +| T-11-06 | Spoofing/Repudiation | app/api/pax8/sync route | mitigate | Route NOT added to middleware public allowlist; remains behind session-cookie check (matches itglue/veeam sync routes; not a public webhook) | +| T-11-07 | Denial of Service | PAX8 1000/min rate limit during pagination | mitigate | Entities synced sequentially (companies→subscriptions→products), not fanned out concurrently; existing 429 Retry-After backoff in the client's fetchJson is inherited | +| T-11-08 | Denial of Service (self-inflicted) | concurrent sync triggers | mitigate | `syncing` guard + route 409 prevent overlapping runs that would double-hit PAX8 and race the tombstone pass | +| T-11-SC | Tampering | package installs | accept | This plan installs zero npm/pip/cargo packages (native fetch + existing pg) — package legitimacy gate not triggered | + + + +- `npx tsc --noEmit --pretty` passes +- No POST/PUT/PATCH/DELETE method and no bare `fetch(` in pax8-sync-service.ts (PAX8-08) +- Three UUID-array tombstone passes present (companies/subscriptions/products) +- /api/pax8/sync absent from middleware.ts public allowlist +- POST and GET exported from the route + + + +Pax8SyncService.fullSync reads companies/subscriptions/products from PAX8 (read-only), +upserts them into Postgres with dual cost columns and a referenced-only readable +catalog, soft-deletes anything PAX8 no longer returns, and is triggerable via a +session-gated fire-and-forget POST /api/pax8/sync with a 409 in-progress guard. + + + +Create `.planning/phases/11-company-catalog-subscription-sync/11-02-SUMMARY.md` when done + diff --git a/.planning/phases/11-company-catalog-subscription-sync/11-03-PLAN.md b/.planning/phases/11-company-catalog-subscription-sync/11-03-PLAN.md new file mode 100644 index 0000000..93b14b4 --- /dev/null +++ b/.planning/phases/11-company-catalog-subscription-sync/11-03-PLAN.md @@ -0,0 +1,175 @@ +--- +phase: 11-company-catalog-subscription-sync +plan: 03 +type: execute +wave: 3 +depends_on: [11-02] +files_modified: [] +autonomous: false +requirements: [PAX8-03, PAX8-04, PAX8-05, PAX8-08] + +user_setup: + - service: pax8 + why: "Live sync run requires real PAX8 partner credentials to populate Postgres" + env_vars: + - name: PAX8_CLIENT_ID + source: "PAX8 developer portal (devx.pax8.com) — provisioned client ID (per SEED-002)" + - name: PAX8_CLIENT_SECRET + source: "PAX8 developer portal (devx.pax8.com) — provisioned client secret" + +must_haves: + truths: + - "A real sync run populates pax8_companies with rows (SC#1)" + - "A real sync run populates pax8_products and pax8_subscriptions with rows, and subscriptions carry price + partner_cost (SC#2)" + - "A subscription joined to pax8_products shows a readable product name and category, not a bare SKU/product id (SC#3)" + - "The client and sync service issue no PAX8 write calls (SC#4 / PAX8-08)" + artifacts: [] + key_links: + - from: "POST /api/pax8/sync" + to: "pax8_companies / pax8_subscriptions / pax8_products" + via: "live sync run populates rows" + pattern: "pax8_" +--- + + +End-to-end verification of the Phase 11 sync against the live PAX8 API and the dev +database — the four ROADMAP success criteria can only be proven with real +credentials and a real run (matching the project convention that no integration +client has an automated live-API test tier). + +Purpose: Confirm the sync actually populates readable, cost-bearing data and holds +the read-only invariant before the phase is marked complete. + +Output: developer confirmation (no code changes in this plan). + + + +@$HOME/.claude/get-shit-done/workflows/execute-plan.md +@$HOME/.claude/get-shit-done/templates/summary.md + + + +@.planning/ROADMAP.md +@.planning/phases/11-company-catalog-subscription-sync/11-01-SUMMARY.md +@.planning/phases/11-company-catalog-subscription-sync/11-02-SUMMARY.md + + + + + + Task 1: Static read-only invariant proof (SC#4 / PAX8-08) + lib/services/pax8-client.ts, lib/services/pax8-sync-service.ts + + - lib/services/pax8-client.ts + - lib/services/pax8-sync-service.ts + + + Prove by inspection that no data-write call to the PAX8 API exists. Grep the + client and the sync service for mutating HTTP verbs and confirm the only POST + anywhere in the PAX8 client is the `/v1/token` OAuth handshake (an auth + exchange, not a data-resource write), and that the sync service issues no HTTP + at all (it only calls the client's read methods). Record the grep output in the + summary as the PAX8-08 evidence. + + + - `grep -nE "method:[[:space:]]*['\"](PUT|PATCH|DELETE)" lib/services/pax8-client.ts` returns nothing + - The only POST in pax8-client.ts targets `https://api.pax8.com/v1/token` + - `grep -nE "method:[[:space:]]*['\"](POST|PUT|PATCH|DELETE)" lib/services/pax8-sync-service.ts` returns nothing + + + ! grep -nE "method:[[:space:]]*['\"](PUT|PATCH|DELETE)" lib/services/pax8-client.ts && ! grep -nE "method:[[:space:]]*['\"](POST|PUT|PATCH|DELETE)" lib/services/pax8-sync-service.ts && echo OK + + Grep evidence confirms no PAX8 data-write path exists (SC#4 upheld). + + + + Task 2: Live sync run + database verification (SC#1-3) + + + - .planning/ROADMAP.md Phase 11 Success Criteria #1-3 + - .planning/phases/11-company-catalog-subscription-sync/11-02-SUMMARY.md + + + Human-verification checkpoint (no code changes). Pause and have the developer + run the live PAX8 sync against real credentials and confirm the database + outcomes described in how-to-verify below. Do not auto-approve — this gate + confirms ROADMAP Success Criteria #1-3, which cannot be proven without live + credentials and a real run. + + + Pax8SyncService.fullSync (read-only) and POST /api/pax8/sync, which sync PAX8 + companies, subscriptions (with customer price + partner cost), and a + referenced-only readable product catalog into Postgres with soft-delete + reconciliation. + + + 1. Ensure `PAX8_CLIENT_ID` and `PAX8_CLIENT_SECRET` are set in the environment + the dev server reads (`.env` / `.env.local`) and restart the app if needed. + 2. Trigger a sync: `curl -X POST http://localhost:3100/api/pax8/sync` — expect + `{ ok: true, message: "PAX8 sync started" }`. An immediate second POST while + it runs should return HTTP 409. + 3. Wait for completion, then check status: `curl http://localhost:3100/api/pax8/sync` + — expect `inProgress: false` and non-zero `counts` for companies, + subscriptions, and products, plus a recent `sync_history` entry with + status `completed`. + 4. In psql against the dev DB confirm SC#1/#2: `SELECT count(*) FROM pax8_companies WHERE is_deleted=false;` + is > 0; `SELECT count(*) FROM pax8_subscriptions WHERE is_deleted=false;` is > 0; + `SELECT count(*) FROM pax8_subscriptions WHERE price IS NOT NULL OR partner_cost IS NOT NULL;` + is > 0 (dual cost columns populated). + 5. Confirm SC#3 (readable via join, not bare SKU): + `SELECT s.id, p.name AS product_name, p.category, s.quantity, s.price, s.partner_cost + FROM pax8_subscriptions s JOIN pax8_products p ON p.id = s.product_id + WHERE s.is_deleted=false LIMIT 5;` — rows show a human-readable product name + and a category value (not a UUID/SKU). + 6. Confirm D-01 referenced-only catalog: every non-deleted product is referenced + by at least one subscription — `SELECT count(*) FROM pax8_products p WHERE p.is_deleted=false + AND NOT EXISTS (SELECT 1 FROM pax8_subscriptions s WHERE s.product_id = p.id);` + should be 0 (or explain any expected exceptions). + 7. (Optional soft-delete spot check, D-05/D-06) Re-run the sync; confirm counts + stay stable and no rows were hard-deleted (is_deleted toggling only). + + + - GET /api/pax8/sync reports non-zero counts for companies, subscriptions, and products and a completed sync_history entry + - A subscription joined to pax8_products returns a readable product name + category (SC#3) + - At least one subscription row has price and/or partner_cost populated (SC#2) + - No non-deleted product is unreferenced by any subscription (D-01), barring explained exceptions + + + Developer confirms the DB queries in how-to-verify return the expected non-zero, readable, cost-bearing results. + + All four ROADMAP Phase 11 success criteria confirmed against live data; developer typed "approved". + Type "approved" once the DB checks pass, or describe what failed. + + + + + +## Trust Boundaries + +| Boundary | Description | +|----------|-------------| +| Pulse server → PAX8 API | Live outbound read-only calls with real credentials | +| Operator → .env secrets | Real PAX8 client secret placed in a committed .env is a disclosure risk (per CLAUDE.md warning) | + +## STRIDE Threat Register + +| Threat ID | Category | Component | Disposition | Mitigation Plan | +|-----------|----------|-----------|-------------|-----------------| +| T-11-09 | Information Disclosure | PAX8_CLIENT_SECRET in committed .env | accept | CLAUDE.md documents that .env is committed; treated like every other integration secret in this repo — flag to operator, prefer .env.local if available; no code change in scope | +| T-11-04 | Elevation of Privilege | live sync run (SC#4 write-invariant) | mitigate | Task 1 grep proof plus the run itself only exercises read endpoints; upholds PAX8-08 end-to-end | + + + +- Static grep proof: no PAX8 write path (Task 1) +- Live run populates all three tables; join yields readable name + category; costs populated (Task 2) + + + +All four ROADMAP Phase 11 success criteria verified against the live PAX8 API and +the dev database, and the read-only invariant (PAX8-08 / SC#4) confirmed by +inspection and by the run exercising only read endpoints. + + + +Create `.planning/phases/11-company-catalog-subscription-sync/11-03-SUMMARY.md` when done +