diff --git a/.planning/REQUIREMENTS.md b/.planning/REQUIREMENTS.md index ba2c219..8647e2f 100644 --- a/.planning/REQUIREMENTS.md +++ b/.planning/REQUIREMENTS.md @@ -15,12 +15,12 @@ Requirements for this milestone. Each maps to a roadmap phase. ### PAX8 — Data Sync -- [ ] **PAX8-03**: Pulse syncs PAX8 companies into Postgres -- [ ] **PAX8-04**: Pulse syncs PAX8 subscriptions (product, seat count, billing term) into Postgres -- [ ] **PAX8-05**: Pulse syncs PAX8 product catalog (SKUs, categories) into Postgres, so subscriptions are human-readable +- [x] **PAX8-03**: Pulse syncs PAX8 companies into Postgres +- [x] **PAX8-04**: Pulse syncs PAX8 subscriptions (product, seat count, billing term) into Postgres +- [x] **PAX8-05**: Pulse syncs PAX8 product catalog (SKUs, categories) into Postgres, so subscriptions are human-readable - [ ] **PAX8-06**: Pulse syncs PAX8 orders/invoices (historical line items) into Postgres, enabling cost reconciliation over time, not just current-state seats - [ ] **PAX8-07**: Sync runs on a daily schedule via the existing `sync-scheduler.ts` cron pattern -- [ ] **PAX8-08**: All PAX8 sync operations are read-only — no writes back to the PAX8 API (no seat adjustments, no order placement) +- [x] **PAX8-08**: All PAX8 sync operations are read-only — no writes back to the PAX8 API (no seat adjustments, no order placement) - [ ] **PAX8-09**: PAX8 integration can be toggled on/off via `/admin/integrations`, consistent with other integrations (`integration_settings` table) ### PAX8 — Company Matching @@ -55,10 +55,10 @@ Requirements for this milestone. Each maps to a roadmap phase. |-------------|-------|--------| | PAX8-01 | Phase 10 | Complete | | PAX8-02 | Phase 10 | Complete | -| PAX8-03 | Phase 11 | Pending | -| PAX8-04 | Phase 11 | Pending | -| PAX8-05 | Phase 11 | Pending | -| PAX8-08 | Phase 11 | Pending | +| PAX8-03 | Phase 11 | Complete | +| PAX8-04 | Phase 11 | Complete | +| PAX8-05 | Phase 11 | Complete | +| PAX8-08 | Phase 11 | Complete | | PAX8-06 | Phase 12 | Pending | | PAX8-10 | Phase 12 | Pending | | PAX8-11 | Phase 12 | Pending | diff --git a/.planning/ROADMAP.md b/.planning/ROADMAP.md index 7edbe5e..f2cc17b 100644 --- a/.planning/ROADMAP.md +++ b/.planning/ROADMAP.md @@ -330,7 +330,7 @@ Phases execute in numeric order. v1.0 (Phases 1-9.1) shipped 2026-07-10. v2.0 ph | 9. User Profile & Preferences | v1.0 | 6/6 | Complete | 2026-07-10 | | 9.1. ntfy Backend Fix | v1.0 | 1/1 | Complete | 2026-07-10 | | 10. PAX8 Client & Auth Foundation | v2.0 | 3/3 | Complete | 2026-07-10 | -| 11. Company, Catalog & Subscription Sync | v2.0 | 3/3 | Complete | 2026-07-11 | +| 11. Company, Catalog & Subscription Sync | v2.0 | 3/3 | Complete | 2026-07-11 | | 12. Orders/Invoices & Company Matching | v2.0 | 0/TBD | Not started | - | | 13. Scheduler & Admin Toggle | v2.0 | 0/TBD | Not started | - | | 14. /pax8 UI Surface | v2.0 | 0/TBD | Not started | - | diff --git a/.planning/STATE.md b/.planning/STATE.md index 1dc2ffd..122d545 100644 --- a/.planning/STATE.md +++ b/.planning/STATE.md @@ -2,15 +2,15 @@ gsd_state_version: 1.0 milestone: v2.0 milestone_name: PAX8 Integration -status: executing -stopped_at: Phase 11 context gathered -last_updated: "2026-07-10T23:35:13.307Z" +status: ready_to_plan +stopped_at: Phase 11 complete (3/3) — ready to discuss Phase 12 +last_updated: 2026-07-11T01:06:17.049Z last_activity: 2026-07-10 -- Phase 11 execution started progress: total_phases: 5 completed_phases: 1 total_plans: 6 - completed_plans: 3 + completed_plans: 6 percent: 20 --- @@ -21,14 +21,14 @@ progress: See: .planning/PROJECT.md (updated 2026-07-10) **Core value:** Managers can see PAX8 subscription costs and seat counts alongside existing Autotask company data in Pulse, without manually cross-referencing PAX8's own portal. -**Current focus:** Phase 11 — company-catalog-subscription-sync +**Current focus:** Phase 12 — orders/invoices & company matching ## Current Position -Phase: 11 (company-catalog-subscription-sync) — EXECUTING -Plan: 1 of 3 -Status: Executing Phase 11 -Last activity: 2026-07-10 -- Phase 11 execution started +Phase: 12 +Plan: Not started +Status: Ready to plan +Last activity: 2026-07-11 Progress: [░░░░░░░░░░] 0% @@ -36,7 +36,7 @@ Progress: [░░░░░░░░░░] 0% **Velocity:** -- Total plans completed: 33 (all from v1.0 Mobile Shell Redesign) +- Total plans completed: 36 (all from v1.0 Mobile Shell Redesign) - Average duration: — - Total execution time: 0.0 hours (v2.0) @@ -56,7 +56,7 @@ Progress: [░░░░░░░░░░] 0% | 09 | 6 | - | - | | 09.1 | 1 | - | - | | 10 | 3 | - | - | -| 11 | TBD | - | - | +| 11 | 3 | - | - | | 12 | TBD | - | - | | 13 | TBD | - | - | | 14 | TBD | - | - | diff --git a/.planning/phases/11-company-catalog-subscription-sync/11-VERIFICATION.md b/.planning/phases/11-company-catalog-subscription-sync/11-VERIFICATION.md new file mode 100644 index 0000000..9e98d1d --- /dev/null +++ b/.planning/phases/11-company-catalog-subscription-sync/11-VERIFICATION.md @@ -0,0 +1,144 @@ +--- +phase: 11-company-catalog-subscription-sync +verified: 2026-07-10T21:10:00Z +status: passed +score: 4/4 roadmap success criteria verified (12/12 plan-level truths verified) +overrides_applied: 0 +--- + +# Phase 11: Company, Catalog & Subscription Sync Verification Report + +**Phase Goal:** PAX8 companies, the product catalog, and current subscriptions are synced into Postgres and are human-readable (not raw SKU IDs) — the "current state" half of the integration. +**Verified:** 2026-07-10T21:10:00Z +**Status:** passed +**Re-verification:** No — initial verification + +## Method + +This phase's SUMMARY.md (11-03) claims live verification against the real PAX8 +API and dev Postgres was already performed and approved by the developer. That +claim was **not** taken on trust. Independent verification performed here: + +1. Read all source files (migrations/092, lib/types/pax8.ts, lib/services/pax8-client.ts, + lib/services/pax8-sync-service.ts, app/api/pax8/sync/route.ts) line-by-line. +2. Ran `npx tsc --noEmit --pretty` — no errors in any Phase 11 file (2 pre-existing + errors remain in an unrelated untracked script, `scripts/diagnose-ticket-varchar-overflow.ts`, + confirmed unrelated to this phase). +3. Ran `npx vitest run lib/services/pax8-client.test.ts` — 9/9 passed. +4. Ran grep gates for the read-only invariant myself (not copied from SUMMARY). +5. **Connected directly to the live `pulse-postgres` container** and ran the exact + SQL queries from 11-03-PLAN.md's `` section myself, independent + of the developer's earlier session — confirming row counts, the join output, + and sync_history rows. + +## Goal Achievement + +### Observable Truths (ROADMAP Success Criteria) + +| # | Truth | Status | Evidence | +|---|-------|--------|----------| +| 1 | Running the sync populates a companies table with every PAX8 company (PAX8 ID, name, other identifying fields) | VERIFIED | `syncCompanies()` in pax8-sync-service.ts upserts id/name/external_id/website/status/city/state_or_province/postal_code/country + raw_payload. Live DB: `SELECT count(*) FROM pax8_companies WHERE is_deleted=false` → **118** (independently queried, not copied from SUMMARY) | +| 2 | Running the sync populates a product/catalog table (SKUs, categories) and a subscriptions table (product, seat count, billing term) per company | VERIFIED | `syncProducts()` upserts sku/vendor_sku/name/category; `syncSubscriptions()` upserts product_id/quantity/billing_term/price/partner_cost/currency. Live DB: pax8_products → **46** rows, pax8_subscriptions → **445** rows, all 445 have price and/or partner_cost populated (`SELECT count(*) FROM pax8_subscriptions WHERE price IS NOT NULL OR partner_cost IS NOT NULL` → 445) | +| 3 | A synced subscription row displays a readable product name and category by joining to the catalog table — not a bare SKU/product ID | VERIFIED | Ran the join myself: `SELECT s.id, p.name, p.category, s.quantity, s.price, s.partner_cost FROM pax8_subscriptions s JOIN pax8_products p ON p.id=s.product_id WHERE s.is_deleted=false LIMIT 5` returned rows like `"Exchange Online (Plan 1) [New Commerce Experience]" / "Microsoft"` — human-readable, not UUIDs/SKUs. Also confirmed D-01 (referenced-only catalog): `SELECT count(*) FROM pax8_products p WHERE p.is_deleted=false AND NOT EXISTS (SELECT 1 FROM pax8_subscriptions s WHERE s.product_id=p.id)` → **0** | +| 4 | No code path in the PAX8 client or this sync service issues a write (POST/PUT/PATCH/DELETE) to the PAX8 API — every call is a read | VERIFIED | `grep -nE "method:[[:space:]]*['\"](PUT\|PATCH\|DELETE)" lib/services/pax8-client.ts` → empty. Only `method: 'POST'` in the client targets `/v1/token` (OAuth handshake, not a data write). `grep -nE "method:[[:space:]]*['\"](POST\|PUT\|PATCH\|DELETE)" lib/services/pax8-sync-service.ts` → empty, and `grep -nE "[^.]fetch\(" lib/services/pax8-sync-service.ts` → empty (service never issues HTTP itself, only calls `Pax8Client`'s read methods) | + +**Score:** 4/4 ROADMAP success criteria verified + +### Plan-Level Must-Have Truths (supplementary, all merged/confirmed) + +| # | Truth | Status | Evidence | +|---|-------|--------|----------| +| 5 | pax8_subscriptions has price, partner_cost, currency columns (D-03/D-04) | VERIFIED | migrations/092 adds all 3 via `ADD COLUMN IF NOT EXISTS`; `\d pax8_subscriptions` confirms live schema has all 3 | +| 6 | Client can page through all companies/subscriptions/products via read-only GET | VERIFIED | `paginateAll()` private helper + `listAllCompanies/listAllSubscriptions/listAllProducts`; 9 mocked tests pass incl. multi-page concatenation | +| 7 | No PUT/PATCH/DELETE in client; only POST is OAuth token exchange | VERIFIED | Grep confirms; single `fetch('https://api.pax8.com/v1/token', {method:'POST',...})` call site | +| 8 | Referenced-but-unknown products resolved in the same single-pass sync (D-02), not fabricated | VERIFIED | `syncProducts()` builds a `Map` from one `listAllProducts()` call, filters to `referencedProductIds`, logs+skips missing ids without inserting a row or dropping the subscription | +| 9 | Companies/subscriptions/products no longer returned by PAX8 are soft-deleted, never hard-removed (D-05/D-06) | VERIFIED | All 3 tombstone UPDATEs use `is_deleted=true, deleted_at=NOW()`, no DELETE statement anywhere in the file; `grep -cE "<> ALL\(\$1::uuid\[\]\)"` → 3 | +| 10 | fullSync is the only sync mode; no incrementalSync (D-07) | VERIFIED | Only `async fullSync(...)` exists; grep for `incrementalSync` shows only comment mentions explaining its absence | +| 11 | Every fullSync run performs the complete tombstone/reconciliation pass unconditionally (D-08) | VERIFIED | Tombstone UPDATE runs after every entity loop unconditionally (skipped only when the seen-set is empty, guarding against a mass-delete on a failed/empty pull — this is the documented safety guard, not a conditional skip of reconciliation itself) | +| 12 | POST /api/pax8/sync fire-and-forget + 409 on concurrent trigger | VERIFIED | Route checks `isSyncInProgress()` → 409; otherwise calls `svc.fullSync(triggeredBy).catch(...)` without `await` and returns 200 immediately | + +**Combined score:** 12/12 truths verified + +### Required Artifacts + +| Artifact | Expected | Status | Details | +|----------|----------|--------|---------| +| `migrations/092_pax8_subscription_costs.sql` | price/partner_cost/currency columns | VERIFIED | 3x `ADD COLUMN IF NOT EXISTS`, additive only, applied to dev DB (confirmed via live `\d`) | +| `lib/types/pax8.ts` | cost fields, sync-result shapes | VERIFIED | `price`, `partnerCost`, `currencyCode` on `Pax8Subscription`; `vendorName` on `Pax8Product`; `Pax8SyncResult`/`Pax8EntitySyncResult` exported | +| `lib/services/pax8-client.ts` | read-only pagination helpers | VERIFIED | `listAllCompanies/listAllSubscriptions/listAllProducts` present, GET-only, wired through `fetchJson()` | +| `lib/services/pax8-client.test.ts` | mocked-fetch coverage | VERIFIED | 9/9 tests pass (ran directly, not trusted from SUMMARY) | +| `lib/services/pax8-sync-service.ts` | `Pax8SyncService`, `getPax8SyncService` | VERIFIED | Both exported; `fullSync` orchestrates companies→subscriptions→products with tombstone reconciliation | +| `app/api/pax8/sync/route.ts` | fire-and-forget POST + GET | VERIFIED | Both `POST` and `GET` exported; not in `middleware.ts` public allowlist (`grep -c "pax8" middleware.ts` → 0) | + +### Key Link Verification + +| From | To | Via | Status | Details | +|------|-----|-----|--------|---------| +| `pax8-client.ts` | `lib/types/pax8.ts` | imports Pax8Company/Subscription/Product/PageEnvelope | WIRED | Confirmed via `gsd-sdk query verify.key-links` | +| `pax8-sync-service.ts` | `getPax8Client()` | constructor default | WIRED | Confirmed via `gsd-sdk query verify.key-links` and manual read | +| `pax8-sync-service.ts` | `postgresClient` | parameterized upsert + tombstone | WIRED | Automated tool reported false negative (`postgresClient\\.query` regex-escaping issue); manually confirmed `grep -c "postgresClient\.query" lib/services/pax8-sync-service.ts` → 8 | +| `pax8_subscriptions.product_id` | `pax8_products.id` | readable-name join | WIRED | Not a file-path link — automated tool can't check DB-column links; **independently verified by running the live join query myself** (see truth #3 above) | +| `app/api/pax8/sync/route.ts` | `getPax8SyncService().fullSync` | fire-and-forget call | WIRED | Confirmed via `gsd-sdk query verify.key-links` and manual read (`svc.fullSync(triggeredBy).catch(...)`, no `await`) | +| `POST /api/pax8/sync` | `pax8_companies/subscriptions/products` | live sync run populates rows | WIRED | Not a file-path link — **independently confirmed via live DB query**: `sync_history` shows two `entity_type='pax8'` rows with `status='completed'`, `records_added=609`, `records_deleted=0` (idempotent re-run), matching the row counts found in the tables themselves | + +### Data-Flow Trace (Level 4) + +| Artifact | Data Variable | Source | Produces Real Data | Status | +|----------|---------------|--------|---------------------|--------| +| `pax8_companies` | `companies` from `client.listAllCompanies()` | Live PAX8 API via OAuth2 client-credentials | Yes — 118 real rows, independently queried | FLOWING | +| `pax8_subscriptions` | `subscriptions` from `client.listAllSubscriptions()` | Live PAX8 API | Yes — 445 real rows with price/partner_cost, independently queried | FLOWING | +| `pax8_products` | `products` filtered by `referencedProductIds` | Live PAX8 API via `client.listAllProducts()` | Yes — 46 real referenced-only rows, independently queried | FLOWING | + +### Behavioral Spot-Checks + +| Behavior | Command | Result | Status | +|----------|---------|--------|--------| +| No PUT/PATCH/DELETE in client | `grep -nE "method:[[:space:]]*['\"](PUT\|PATCH\|DELETE)" lib/services/pax8-client.ts` | empty | PASS | +| No mutating verb or bare `fetch(` in sync service | `grep -nE "method:...(POST\|PUT\|PATCH\|DELETE)" / `[^.]fetch\(`` on pax8-sync-service.ts | both empty | PASS | +| Type check | `npx tsc --noEmit --pretty` | no errors in Phase 11 files | PASS | +| Unit tests | `npx vitest run lib/services/pax8-client.test.ts` | 9/9 passed | PASS | +| Live row counts | psql direct query against `pulse-postgres` | 118 companies / 445 subscriptions / 46 products, all subs cost-bearing | PASS | +| Readable join | psql direct query | human-readable name + category returned, not SKU/UUID | PASS | +| Referenced-only catalog (D-01) | psql direct query | 0 unreferenced non-deleted products | PASS | + +### Probe Execution + +No `scripts/*/tests/probe-*.sh` probes declared or discovered for this phase. SKIPPED (no probes applicable — this phase's live-verification mechanism is a manual DB/API checkpoint, not a probe script). + +### Requirements Coverage + +| Requirement | Source Plan | Description | Status | Evidence | +|-------------|------------|-------------|--------|----------| +| PAX8-03 | 11-02, 11-03 | Pulse syncs PAX8 companies into Postgres | SATISFIED | `syncCompanies()` + live 118-row count | +| PAX8-04 | 11-01, 11-02, 11-03 | Pulse syncs PAX8 subscriptions (product, seat count, billing term) into Postgres | SATISFIED | `syncSubscriptions()` + live 445-row count with price/partner_cost | +| PAX8-05 | 11-01, 11-02, 11-03 | Pulse syncs PAX8 product catalog (SKUs, categories) into Postgres, so subscriptions are human-readable | SATISFIED | `syncProducts()` + live join returning readable name/category | +| PAX8-08 | 11-01, 11-02, 11-03 | All PAX8 sync operations are read-only — no writes back to the PAX8 API | SATISFIED | Grep evidence (client + sync service), confirmed by direct inspection | + +No orphaned requirements: REQUIREMENTS.md maps exactly PAX8-03, PAX8-04, PAX8-05, PAX8-08 to Phase 11 (lines 58-61), and all four appear in every plan's `requirements:` frontmatter for this phase. Note: REQUIREMENTS.md checkboxes for these four still show `[ ]` (unchecked) as of this verification — that file has not yet been updated to reflect Phase 11 completion; this is a documentation-sync item, not a functional gap (flagged for the orchestrator to update REQUIREMENTS.md checkboxes). + +### Anti-Patterns Found + +No blocker-level anti-patterns (TBD/FIXME/XXX/TODO/HACK/PLACEHOLDER) found in any of the 6 files this phase modified. The independent code review (`11-REVIEW.md`, standard depth, 2026-07-10) found 0 critical findings and 5 warnings, none of which contradict the phase's success criteria: + +| File | Line | Pattern | Severity | Impact | +|------|------|---------|----------|--------| +| `lib/services/pax8-client.ts` | 62 | `Retry-After` backoff breaks on HTTP-date-form header (`parseInt` → `NaN`) | WARNING (carried over from Phase 10, unfixed) | Reliability risk under sustained 429s; does not block the observed successful live sync (445 subscriptions synced without hitting this path) | +| `lib/services/pax8-sync-service.ts` | 113-173, 175-253, 262-325 | Whole-loop try/catch reports `upserted:0` on a mid-loop row failure even though earlier rows already committed | WARNING | Misreporting only, not data loss; live run had 0 failures | +| `app/api/pax8/sync/route.ts` | 5-20 | `POST` has no try/catch — missing PAX8 credentials throw unhandled instead of returning 503 | WARNING | Credentials were present and valid for the live run; this is an error-path gap, not a goal blocker | +| `lib/services/pax8-client.ts` | 23-50 | `getToken()` has no response-shape validation (carried over from Phase 10, unfixed) | WARNING | Same class of issue as above — didn't manifest in the live run | +| `app/api/pax8/sync/route.ts` | 5-7 | `triggeredBy` from request body is unvalidated (any type) | WARNING | Low impact, authenticated internal endpoint | + +These are legitimate code-quality gaps worth fixing but do not fail any of the four ROADMAP success criteria — the phase goal (data synced, human-readable, read-only) is independently confirmed true regardless of them. + +### Human Verification Required + +None. The phase's own plan (11-03-PLAN.md Task 2) specified a `checkpoint:human-verify` gate for the live sync run, and the developer already ran and approved it per 11-03-SUMMARY.md. This verifier did not stop at trusting that summary — it independently re-ran the exact same DB queries against the live `pulse-postgres` container and confirmed identical results (118/445/46, readable join, 0 unreferenced products, completed sync_history rows). No further human action is needed. + +### Gaps Summary + +No gaps. All 4 ROADMAP success criteria and all 12 merged plan-level truths are independently verified against the actual codebase and the live database — not merely asserted by SUMMARY.md. The five WARNING-level code-review findings (retry-after date-header bug, error-count misreporting on partial failure, missing try/catch on POST, unvalidated token-response shape, unvalidated triggeredBy) are real but do not block phase goal achievement; they are recommended follow-up hardening, not blockers. + +--- + +_Verified: 2026-07-10T21:10:00Z_ +_Verifier: Claude (gsd-verifier)_