docs(11): create phase plan
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01LHRgZqkzBHBbAbc3KHneuR
This commit is contained in:
parent
eee3c77e32
commit
c8dd64f766
4 changed files with 678 additions and 1 deletions
|
|
@ -269,7 +269,10 @@ render, including the manual-resolution workflow for flagged companies.
|
|||
2. Running the sync populates a product/catalog table (SKUs, categories) and a subscriptions table (product, seat count, billing term) per company
|
||||
3. A synced subscription row displays a readable product name and category by joining to the catalog table — not a bare SKU/product ID
|
||||
4. No code path in the PAX8 client or this sync service issues a write (POST/PUT/PATCH/DELETE) to the PAX8 API — every call is a read, verified by inspection of the client's exposed methods
|
||||
**Plans**: TBD
|
||||
**Plans**: 3 plans
|
||||
- [ ] 11-01-PLAN.md — Migration 092 subscription cost columns + extend pax8 types + read-only client pagination helpers (PAX8-04, PAX8-05, PAX8-08)
|
||||
- [ ] 11-02-PLAN.md — pax8-sync-service.ts (companies + subscriptions + referenced-only catalog + soft-delete reconciliation) + /api/pax8/sync fire-and-forget route (PAX8-03, PAX8-04, PAX8-05, PAX8-08)
|
||||
- [ ] 11-03-PLAN.md — Read-only invariant proof + live sync run DB verification checkpoint (PAX8-03, PAX8-04, PAX8-05, PAX8-08)
|
||||
**UI hint**: no
|
||||
|
||||
### Phase 12: Orders/Invoices & Company Matching
|
||||
|
|
|
|||
|
|
@ -0,0 +1,240 @@
|
|||
---
|
||||
phase: 11-company-catalog-subscription-sync
|
||||
plan: 01
|
||||
type: execute
|
||||
wave: 1
|
||||
depends_on: []
|
||||
files_modified:
|
||||
- migrations/092_pax8_subscription_costs.sql
|
||||
- lib/types/pax8.ts
|
||||
- lib/services/pax8-client.ts
|
||||
- lib/services/pax8-client.test.ts
|
||||
autonomous: true
|
||||
requirements: [PAX8-04, PAX8-05, PAX8-08]
|
||||
|
||||
must_haves:
|
||||
truths:
|
||||
- "pax8_subscriptions has price, partner_cost, and currency columns so both the customer price and the partner cost can be stored (D-03/D-04)"
|
||||
- "The PAX8 client can page through all companies, all subscriptions, and all products with read-only GET calls"
|
||||
- "No PUT/PATCH/DELETE call exists in the client; the only POST is the OAuth token exchange (PAX8-08)"
|
||||
artifacts:
|
||||
- path: "migrations/092_pax8_subscription_costs.sql"
|
||||
provides: "price + partner_cost + currency columns on pax8_subscriptions"
|
||||
contains: "ALTER TABLE pax8_subscriptions"
|
||||
- path: "lib/types/pax8.ts"
|
||||
provides: "cost fields on Pax8Subscription, vendorName on Pax8Product, Pax8SyncResult/Pax8EntitySyncResult"
|
||||
contains: "partnerCost"
|
||||
- path: "lib/services/pax8-client.ts"
|
||||
provides: "listAllCompanies / listAllSubscriptions / listAllProducts pagination helpers"
|
||||
exports: ["Pax8Client"]
|
||||
- path: "lib/services/pax8-client.test.ts"
|
||||
provides: "mocked-fetch coverage for the new pagination helpers"
|
||||
key_links:
|
||||
- from: "lib/services/pax8-client.ts"
|
||||
to: "lib/types/pax8.ts"
|
||||
via: "imports Pax8Company/Pax8Subscription/Pax8Product/Pax8PageEnvelope"
|
||||
pattern: "from '@/lib/types/pax8'"
|
||||
---
|
||||
|
||||
<objective>
|
||||
Lay down the data contract layer the Phase 11 sync service consumes: the missing
|
||||
subscription cost columns (D-03/D-04), the extended TypeScript types, and the
|
||||
read-only pagination helpers on the PAX8 client.
|
||||
|
||||
Purpose: The sync service (Plan 02) must not have to discover the schema, the
|
||||
API response shapes, or invent pagination. This plan defines all three up front
|
||||
(interface-first) so Plan 02 is pure orchestration against known contracts.
|
||||
|
||||
Output: migration 092 (applied to dev DB), extended `lib/types/pax8.ts`, new
|
||||
read-only client methods with mocked-fetch tests.
|
||||
</objective>
|
||||
|
||||
<execution_context>
|
||||
@$HOME/.claude/get-shit-done/workflows/execute-plan.md
|
||||
@$HOME/.claude/get-shit-done/templates/summary.md
|
||||
</execution_context>
|
||||
|
||||
<context>
|
||||
@.planning/PROJECT.md
|
||||
@.planning/ROADMAP.md
|
||||
@.planning/STATE.md
|
||||
@.planning/phases/11-company-catalog-subscription-sync/11-CONTEXT.md
|
||||
@.planning/phases/11-company-catalog-subscription-sync/11-PATTERNS.md
|
||||
|
||||
<interfaces>
|
||||
<!-- Verified against PAX8 devx docs during Phase 11 planning (2026-07-10). -->
|
||||
<!-- Subscription object fields (GET /subscriptions?page=&size=, max size 200): -->
|
||||
<!-- id, parentSubscriptionId, companyId, productId, vendorSubscriptionId, -->
|
||||
<!-- vendorSkuId, quantity, startDate, endDate, createdDate, updatedDate, -->
|
||||
<!-- billingStart, status, price (customer price), currencyCode, -->
|
||||
<!-- partnerCost (partner/reseller cost), productName, billingTerm, -->
|
||||
<!-- provisioningDetails, commitmentTerm -->
|
||||
<!-- Product object fields (GET /products?page=&size=, max size 200): -->
|
||||
<!-- id, name, vendorName, shortDescription, sku, vendorSku, -->
|
||||
<!-- altVendorSku (deprecated), requiresCommitment -->
|
||||
<!-- NO `category` field; NO single-product-detail (GET /products/{id}) endpoint. -->
|
||||
<!-- Company object: id, name, externalId, website, status, city, -->
|
||||
<!-- stateOrProvince, postalCode, country, updatedDate -->
|
||||
<!-- No endpoint (companies/subscriptions/products) supports a modified-since / -->
|
||||
<!-- delta query param — full sync only for every entity type (resolves D-07). -->
|
||||
<!-- All list endpoints return { content: T[], page: { size, totalElements, -->
|
||||
<!-- totalPages, number } }; page.number is 0-indexed. -->
|
||||
</interfaces>
|
||||
</context>
|
||||
|
||||
<tasks>
|
||||
|
||||
<task type="auto">
|
||||
<name>Task 1: Add subscription cost columns (migration 092)</name>
|
||||
<files>migrations/092_pax8_subscription_costs.sql</files>
|
||||
<read_first>
|
||||
- migrations/091_pax8_tables.sql (existing pax8_subscriptions shape and the currency CHAR(3) DEFAULT 'USD' convention used on pax8_orders/pax8_order_items)
|
||||
- CLAUDE.md "Database" + "Migrations" sections (IF NOT EXISTS convention; migrations apply on Postgres init only — an existing volume needs a manual apply)
|
||||
- MEMORY note: "Postgres init applies migrations on first volume boot only" — dev DB needs manual apply via docker exec psql
|
||||
</read_first>
|
||||
<action>
|
||||
Create migration `092_pax8_subscription_costs.sql` that adds three columns to
|
||||
the existing `pax8_subscriptions` table using `ALTER TABLE pax8_subscriptions
|
||||
ADD COLUMN IF NOT EXISTS`:
|
||||
(1) `price NUMERIC(12,2)` — the customer/list price PAX8 returns per subscription (D-03),
|
||||
(2) `partner_cost NUMERIC(12,2)` — the partner/reseller cost PAX8 returns per subscription (D-03),
|
||||
(3) `currency CHAR(3) NOT NULL DEFAULT 'USD'` — matching the `currency` column convention already on pax8_orders/pax8_order_items in migration 091.
|
||||
Store the raw per-billing-period amounts exactly as PAX8 returns them, paired
|
||||
with the existing `billing_term` column — do NOT add any monthly-normalization
|
||||
columns (D-04 defers that to a read-time concern). Add a leading comment block
|
||||
citing that PAX8's Subscription object exposes both `price` and `partnerCost`
|
||||
(Phase 11 research, devx.pax8.com findsubscriptions). Do not alter any other
|
||||
column or table. After writing the file, apply it to the running dev database
|
||||
manually (the Postgres container will not re-run migrations on an existing
|
||||
volume): run the ALTER statements via `docker exec` against the pulse-postgres
|
||||
container per the MEMORY caveat.
|
||||
</action>
|
||||
<acceptance_criteria>
|
||||
- `migrations/092_pax8_subscription_costs.sql` exists and contains exactly three `ADD COLUMN IF NOT EXISTS` clauses for `price`, `partner_cost`, and `currency`
|
||||
- The file contains no `DROP`, no destructive statement, and touches only `pax8_subscriptions`
|
||||
- `grep -cE "ADD COLUMN IF NOT EXISTS" migrations/092_pax8_subscription_costs.sql` returns 3
|
||||
- After manual apply, `\d pax8_subscriptions` in the dev DB lists `price`, `partner_cost`, and `currency` columns
|
||||
</acceptance_criteria>
|
||||
<verify>
|
||||
<automated>test -f migrations/092_pax8_subscription_costs.sql && grep -cE "ADD COLUMN IF NOT EXISTS" migrations/092_pax8_subscription_costs.sql | grep -qx 3 && echo OK</automated>
|
||||
</verify>
|
||||
<done>Migration file created with price/partner_cost/currency columns and applied to the dev DB.</done>
|
||||
</task>
|
||||
|
||||
<task type="auto">
|
||||
<name>Task 2: Extend PAX8 types with cost fields and sync-result shapes</name>
|
||||
<files>lib/types/pax8.ts</files>
|
||||
<read_first>
|
||||
- lib/types/pax8.ts (current Pax8Company/Pax8Subscription/Pax8Product/Pax8PageEnvelope; note the `[key: string]: unknown` escape hatch on each interface)
|
||||
- lib/types/appgate.ts (AppgateSyncResult / AppgateEntity result shape to mirror for Pax8SyncResult/Pax8EntitySyncResult)
|
||||
</read_first>
|
||||
<action>
|
||||
Extend `Pax8Subscription` with the pricing/lifecycle fields PAX8 actually
|
||||
returns: `price: number | null`, `partnerCost: number | null`,
|
||||
`currencyCode: string | null`, `productName: string | null`,
|
||||
`endDate: string | null`, `updatedDate: string | null` (keep the existing
|
||||
id/companyId/productId/quantity/billingTerm/status/startDate fields and the
|
||||
escape hatch). Extend `Pax8Product` with `vendorName: string | null` and
|
||||
`shortDescription: string | null` (keep existing fields; leave the existing
|
||||
`category` typing in place — the sync will populate it from `category ??
|
||||
vendorName`). Add two new exported interfaces mirroring the AppGate result
|
||||
shape: `Pax8EntitySyncResult` with fields `entity: string`, `success:
|
||||
boolean`, `upserted: number`, `tombstoned: number`, `durationMs: number`,
|
||||
`error?: string`; and `Pax8SyncResult` with fields `syncId: string`,
|
||||
`status: 'completed' | 'failed'`, `startedAt: Date`, `completedAt: Date |
|
||||
null`, `durationMs: number`, `entities: Pax8EntitySyncResult[]`, `errors:
|
||||
string[]`. Do not remove or rename any existing exported member.
|
||||
</action>
|
||||
<acceptance_criteria>
|
||||
- `Pax8Subscription` declares `price`, `partnerCost`, and `currencyCode` members
|
||||
- `Pax8Product` declares `vendorName`
|
||||
- `Pax8SyncResult` and `Pax8EntitySyncResult` are exported
|
||||
- `npx tsc --noEmit --pretty` passes with no new errors
|
||||
- `grep -c "partnerCost" lib/types/pax8.ts` is at least 1
|
||||
</acceptance_criteria>
|
||||
<verify>
|
||||
<automated>npx tsc --noEmit --pretty && grep -q "partnerCost" lib/types/pax8.ts && grep -q "Pax8SyncResult" lib/types/pax8.ts && echo OK</automated>
|
||||
</verify>
|
||||
<done>Types compile and expose the cost fields plus the sync-result shapes the service will consume.</done>
|
||||
</task>
|
||||
|
||||
<task type="auto" tdd="true">
|
||||
<name>Task 3: Add read-only pagination helpers to the PAX8 client</name>
|
||||
<files>lib/services/pax8-client.ts, lib/services/pax8-client.test.ts</files>
|
||||
<read_first>
|
||||
- lib/services/pax8-client.ts (existing getToken, fetchJson with 429/Retry-After backoff, single-page listCompanies — do NOT change these signatures; verify-pax8-auth.ts from Phase 10 depends on listCompanies)
|
||||
- lib/services/pax8-client.test.ts (the vi.stubGlobal('fetch', ...) mocking convention keyed by URL substring, vi.unstubAllGlobals in beforeEach — follow this exact style, do not add nock/msw)
|
||||
- lib/types/pax8.ts (Pax8PageEnvelope shape: { content, page: { size, totalElements, totalPages, number } })
|
||||
</read_first>
|
||||
<behavior>
|
||||
- listAllCompanies() returns every company across all pages: given a mocked 2-page response (page 0 with content + totalPages:2, page 1 with content + totalPages:2), the returned array concatenates both pages' content in order
|
||||
- listAllSubscriptions() and listAllProducts() behave the same across pages
|
||||
- Each helper requests size=200 and stops once page.number >= page.totalPages - 1 (no infinite loop when totalPages is 1)
|
||||
- Every request the helpers issue is a GET with an `Authorization: Bearer` header (asserted via the fetch mock); no request sets method POST/PUT/PATCH/DELETE
|
||||
</behavior>
|
||||
<action>
|
||||
Add three async methods to `Pax8Client`, each looping pages via the existing
|
||||
private `fetchJson<T>()` (so 429 backoff is inherited) and requesting
|
||||
`size=200`: `listAllCompanies(): Promise<Pax8Company[]>`,
|
||||
`listAllSubscriptions(): Promise<Pax8Subscription[]>`, and
|
||||
`listAllProducts(): Promise<Pax8Product[]>`. Each fetches
|
||||
`/companies?page=N&size=200` (respectively `/subscriptions`, `/products`),
|
||||
accumulates `envelope.content`, and increments N until `envelope.page.number
|
||||
>= envelope.page.totalPages - 1`, then returns the accumulated array. Keep the
|
||||
existing single-page `listCompanies(page, size)` method unchanged. Do NOT add
|
||||
any method that issues a mutating HTTP verb — these are read-only list reads
|
||||
only (PAX8-08). Add tests in `pax8-client.test.ts` following the existing
|
||||
mocked-fetch style: assert multi-page concatenation for at least
|
||||
listAllSubscriptions, assert the request URLs carry `size=200`, and assert the
|
||||
fetch mock only ever sees GET requests (no method override) plus the bearer
|
||||
header.
|
||||
</action>
|
||||
<acceptance_criteria>
|
||||
- `Pax8Client` exposes `listAllCompanies`, `listAllSubscriptions`, `listAllProducts`
|
||||
- Existing `listCompanies(page, size)` signature is unchanged
|
||||
- No new method sets `method:` to POST/PUT/PATCH/DELETE
|
||||
- `grep -nE "method:\s*['\"](PUT|PATCH|DELETE)" lib/services/pax8-client.ts` returns nothing (exit 1)
|
||||
- `npx vitest run lib/services/pax8-client.test.ts` passes, including a multi-page concatenation test
|
||||
</acceptance_criteria>
|
||||
<verify>
|
||||
<automated>npx vitest run lib/services/pax8-client.test.ts && ! grep -nE "method:[[:space:]]*['\"](PUT|PATCH|DELETE)" lib/services/pax8-client.ts && echo OK</automated>
|
||||
</verify>
|
||||
<done>Client can enumerate all companies/subscriptions/products via read-only GET pagination, with passing mocked tests.</done>
|
||||
</task>
|
||||
|
||||
</tasks>
|
||||
|
||||
<threat_model>
|
||||
## Trust Boundaries
|
||||
|
||||
| Boundary | Description |
|
||||
|----------|-------------|
|
||||
| Pulse server → PAX8 API | Outbound OAuth2 client-credentials calls; client secret crosses here |
|
||||
| Migration → Postgres | DDL applied to the pax8_subscriptions table |
|
||||
|
||||
## STRIDE Threat Register
|
||||
|
||||
| Threat ID | Category | Component | Disposition | Mitigation Plan |
|
||||
|-----------|----------|-----------|-------------|-----------------|
|
||||
| T-11-01 | Information Disclosure | pax8-client.ts token exchange | mitigate | Never log/interpolate `config.clientSecret`; existing getToken already errors with status text only, not the body echo of credentials — new pagination methods add no logging of secrets |
|
||||
| T-11-02 | Elevation of Privilege | pax8-client.ts new read methods | mitigate | New methods issue GET only; grep gate asserts no PUT/PATCH/DELETE added; the only POST in the file remains the `/v1/token` auth handshake (not a data write) — upholds PAX8-08 at the client layer |
|
||||
| T-11-03 | Tampering | migration 092 DDL | accept | Additive ADD COLUMN IF NOT EXISTS only, no destructive ops; idempotent on rerun; low risk |
|
||||
| T-11-SC | Tampering | package installs | accept | This plan installs zero npm/pip/cargo packages (native fetch + existing pg) — package legitimacy gate not triggered |
|
||||
</threat_model>
|
||||
|
||||
<verification>
|
||||
- `npx tsc --noEmit --pretty` passes
|
||||
- `npx vitest run lib/services/pax8-client.test.ts` passes
|
||||
- Migration 092 present and applied to dev DB (`\d pax8_subscriptions` shows price/partner_cost/currency)
|
||||
- No PUT/PATCH/DELETE method in pax8-client.ts
|
||||
</verification>
|
||||
|
||||
<success_criteria>
|
||||
pax8_subscriptions carries price/partner_cost/currency columns; lib/types/pax8.ts
|
||||
exposes the cost fields and sync-result shapes; the PAX8 client can page through
|
||||
all three entity types with read-only GET calls covered by mocked tests.
|
||||
</success_criteria>
|
||||
|
||||
<output>
|
||||
Create `.planning/phases/11-company-catalog-subscription-sync/11-01-SUMMARY.md` when done
|
||||
</output>
|
||||
|
|
@ -0,0 +1,259 @@
|
|||
---
|
||||
phase: 11-company-catalog-subscription-sync
|
||||
plan: 02
|
||||
type: execute
|
||||
wave: 2
|
||||
depends_on: [11-01]
|
||||
files_modified:
|
||||
- lib/services/pax8-sync-service.ts
|
||||
- app/api/pax8/sync/route.ts
|
||||
autonomous: true
|
||||
requirements: [PAX8-03, PAX8-04, PAX8-05, PAX8-08]
|
||||
|
||||
must_haves:
|
||||
truths:
|
||||
- "A full sync upserts every current PAX8 company into pax8_companies (PAX8-03)"
|
||||
- "A full sync upserts every current PAX8 subscription with product, quantity, billing term, customer price, and partner cost (PAX8-04)"
|
||||
- "The catalog table (pax8_products) holds a readable name + category for every product referenced by a subscription, populated from the full product list — not bare SKU IDs (PAX8-05, D-01)"
|
||||
- "Companies/subscriptions/products no longer returned by PAX8 are soft-deleted (is_deleted=true, deleted_at set), never hard-removed (D-05/D-06)"
|
||||
- "No code path in the sync service issues a PAX8 API write; it calls only the client's read methods (PAX8-08)"
|
||||
- "POST /api/pax8/sync starts the sync fire-and-forget and returns immediately; a concurrent trigger returns 409"
|
||||
artifacts:
|
||||
- path: "lib/services/pax8-sync-service.ts"
|
||||
provides: "Pax8SyncService.fullSync orchestrating companies→subscriptions→products with tombstone reconciliation"
|
||||
exports: ["Pax8SyncService", "getPax8SyncService"]
|
||||
- path: "app/api/pax8/sync/route.ts"
|
||||
provides: "fire-and-forget POST trigger + GET status"
|
||||
exports: ["POST", "GET"]
|
||||
key_links:
|
||||
- from: "lib/services/pax8-sync-service.ts"
|
||||
to: "getPax8Client()"
|
||||
via: "constructor: this.client = client ?? getPax8Client()"
|
||||
pattern: "getPax8Client"
|
||||
- from: "lib/services/pax8-sync-service.ts"
|
||||
to: "postgresClient"
|
||||
via: "parameterized upsert + UUID-array tombstone"
|
||||
pattern: "postgresClient\\.query"
|
||||
- from: "pax8_subscriptions.product_id"
|
||||
to: "pax8_products.id"
|
||||
via: "readable-name join (D-01 referenced-only catalog)"
|
||||
pattern: "product_id"
|
||||
- from: "app/api/pax8/sync/route.ts"
|
||||
to: "getPax8SyncService().fullSync"
|
||||
via: "fire-and-forget call"
|
||||
pattern: "fullSync"
|
||||
---
|
||||
|
||||
<objective>
|
||||
Build the read-only PAX8 current-state sync: companies, subscriptions (with dual
|
||||
cost), and the referenced-only product catalog — plus the fire-and-forget trigger
|
||||
route. This is the phase's core deliverable.
|
||||
|
||||
Purpose: Populate pax8_companies / pax8_subscriptions / pax8_products so a
|
||||
subscription can be shown with a readable product name and category (not a bare
|
||||
SKU), with removed entities soft-deleted and zero writes back to PAX8.
|
||||
|
||||
Output: `lib/services/pax8-sync-service.ts` and `app/api/pax8/sync/route.ts`.
|
||||
</objective>
|
||||
|
||||
<execution_context>
|
||||
@$HOME/.claude/get-shit-done/workflows/execute-plan.md
|
||||
@$HOME/.claude/get-shit-done/templates/summary.md
|
||||
</execution_context>
|
||||
|
||||
<context>
|
||||
@.planning/PROJECT.md
|
||||
@.planning/ROADMAP.md
|
||||
@.planning/STATE.md
|
||||
@.planning/phases/11-company-catalog-subscription-sync/11-CONTEXT.md
|
||||
@.planning/phases/11-company-catalog-subscription-sync/11-PATTERNS.md
|
||||
@.planning/phases/11-company-catalog-subscription-sync/11-01-SUMMARY.md
|
||||
|
||||
<interfaces>
|
||||
<!-- From Plan 01 (already built): -->
|
||||
<!-- Pax8Client.listAllCompanies(): Promise<Pax8Company[]> -->
|
||||
<!-- Pax8Client.listAllSubscriptions(): Promise<Pax8Subscription[]> -->
|
||||
<!-- Pax8Client.listAllProducts(): Promise<Pax8Product[]> -->
|
||||
<!-- getPax8Client(): Pax8Client (lib/services/pax8-factory.ts) -->
|
||||
<!-- Pax8SyncResult / Pax8EntitySyncResult (lib/types/pax8.ts) -->
|
||||
<!-- Subscription carries: price, partnerCost, currencyCode, productName, -->
|
||||
<!-- companyId, productId, quantity, billingTerm, status, startDate -->
|
||||
<!-- Product carries: name, sku, vendorSku, vendorName (NO category field) -->
|
||||
<!-- No product-detail (GET /products/{id}) endpoint exists — the catalog must -->
|
||||
<!-- be fetched as a full list and looked up in memory by id. -->
|
||||
<!-- pax8_* tables all have UUID primary keys + is_deleted/deleted_at/synced_at. -->
|
||||
<!-- Generic sync_history table (migrations/001) columns: entity_type, sync_type -->
|
||||
<!-- (must be 'full'|'incremental'|'entity-specific'), status ('started'| -->
|
||||
<!-- 'in_progress'|'completed'|'failed'), started_at, completed_at, -->
|
||||
<!-- records_added, records_updated, records_deleted, error_message, triggered_by -->
|
||||
</interfaces>
|
||||
</context>
|
||||
|
||||
<tasks>
|
||||
|
||||
<task type="auto">
|
||||
<name>Task 1: Build pax8-sync-service.ts (companies + subscriptions + referenced catalog + tombstone)</name>
|
||||
<files>lib/services/pax8-sync-service.ts</files>
|
||||
<read_first>
|
||||
- lib/services/appgate-sync-service.ts (PRIMARY analog: class + `syncing` guard + singleton getter; the UUID-keyed upsert + `id <> ALL($1::uuid[])` tombstone in syncAppliances, lines ~185-244; run()/persistHistory shape)
|
||||
- lib/services/qbo-sync-service.ts (per-entity try/catch returning a result rather than throwing; tombstone query shape)
|
||||
- lib/services/veeam-sync-service.ts lines 78-165 (generic sync_history insert 'started' → update completed/failed using only base columns entity_type/sync_type/status/records_added/records_deleted/error_message/triggered_by)
|
||||
- lib/services/postgres-client.ts (postgresClient.query signature; note softDeleteMany casts ::bigint[] and is NOT usable for UUID ids — write the tombstone query raw)
|
||||
- lib/services/pax8-client.ts + lib/services/pax8-factory.ts (getPax8Client, the new list-all methods)
|
||||
- migrations/091_pax8_tables.sql + migrations/092_pax8_subscription_costs.sql (exact column names for pax8_companies, pax8_subscriptions, pax8_products)
|
||||
- lib/types/pax8.ts (Pax8SyncResult/Pax8EntitySyncResult, entity field names)
|
||||
</read_first>
|
||||
<action>
|
||||
Create `Pax8SyncService` mirroring `appgate-sync-service.ts`'s class shape: a
|
||||
private `client: Pax8Client` set in the constructor via `this.client = client
|
||||
?? getPax8Client()`, a private `syncing = false` guard, an `isSyncInProgress():
|
||||
boolean`, and a module-level `getPax8SyncService()` singleton. Expose one
|
||||
public entry point `async fullSync(triggeredBy = 'manual'): Promise<Pax8SyncResult>`
|
||||
— there is NO incrementalSync because no PAX8 entity supports a modified-since
|
||||
filter (D-07 resolves to full-sync-only for all entity types). fullSync must:
|
||||
throw if `this.syncing` is already true; set the guard; insert a
|
||||
`sync_history` row with `entity_type='pax8'`, `sync_type='full'`,
|
||||
`status='started'`, `triggered_by=triggeredBy` (base columns only — do not use
|
||||
entity_details); then run three entity syncs IN THIS ORDER inside try/catch,
|
||||
collecting a Pax8EntitySyncResult each, and finally update the sync_history row
|
||||
to 'completed' (or 'failed' if any entity errored) with records_added =
|
||||
total upserted and records_deleted = total tombstoned, and clear the guard in a
|
||||
finally block.
|
||||
|
||||
syncCompanies(): call `this.client.listAllCompanies()`; for each company upsert
|
||||
into pax8_companies via a parameterized `INSERT ... ON CONFLICT (id) DO UPDATE
|
||||
SET ... synced_at = NOW(), is_deleted = false, deleted_at = NULL` (mapping
|
||||
name/external_id/website/status/city/state_or_province/postal_code/country and
|
||||
storing the full object in raw_payload); collect seen ids; after the loop
|
||||
tombstone with `UPDATE pax8_companies SET is_deleted=true, deleted_at=NOW()
|
||||
WHERE is_deleted=false AND id <> ALL($1::uuid[])` (skip the tombstone UPDATE
|
||||
when zero ids were seen, to avoid deleting everything on an empty/failed pull).
|
||||
|
||||
syncSubscriptions(): call `this.client.listAllSubscriptions()`; for each
|
||||
subscription upsert into pax8_subscriptions mapping pax8_company_id (from
|
||||
companyId), product_id (from productId), quantity, billing_term, status,
|
||||
start_date, and the Plan-01 columns price (from `price`), partner_cost (from
|
||||
`partnerCost`), currency (from `currencyCode` ?? 'USD') plus raw_payload;
|
||||
accumulate every referenced productId into an in-memory Set and return it
|
||||
alongside the entity result so syncProducts can consume it; collect seen ids;
|
||||
tombstone unseen subscriptions with the same `<> ALL($1::uuid[])` pattern.
|
||||
|
||||
syncProducts(referencedProductIds: Set<string>): because PAX8 exposes NO
|
||||
single-product-detail endpoint, call `this.client.listAllProducts()` once, build
|
||||
a Map keyed by product id, then upsert ONLY the products whose id is in
|
||||
referencedProductIds (this satisfies D-01 "referenced-only catalog" — the full
|
||||
list is fetched into memory but only referenced rows are STORED). Map name from
|
||||
`name`, sku from `sku`, vendor_sku from `vendorSku`, and category from
|
||||
`(product as any).category ?? product.vendorName ?? null` (PAX8's product list
|
||||
endpoint has no dedicated category field per Phase 11 research; vendorName is
|
||||
the catalog grouping — the full object is kept in raw_payload so a true category
|
||||
can be backfilled later). If a referenced product id is not present in the
|
||||
fetched catalog (e.g. discontinued), log it and continue — do NOT fabricate a
|
||||
row and do NOT drop the subscription (the subscription's raw_payload retains
|
||||
productName). The "seen" set for the products tombstone is referencedProductIds:
|
||||
`UPDATE pax8_products SET is_deleted=true, deleted_at=NOW() WHERE is_deleted=false
|
||||
AND id <> ALL($1::uuid[])` (skip when the set is empty) — this keeps discontinued
|
||||
catalog rows soft-deleted per D-06 while never hard-deleting them.
|
||||
|
||||
Every entity method wraps its own logic in try/catch and returns a
|
||||
Pax8EntitySyncResult { entity, success, upserted, tombstoned, durationMs,
|
||||
error? } rather than throwing to the orchestrator (qbo/itglue precedent). Use
|
||||
ONLY parameterized queries ($1,$2,...) — never string-interpolate values into
|
||||
SQL. The service must call only the client's read methods; it must never issue
|
||||
an HTTP request itself and never call a mutating PAX8 verb (PAX8-08).
|
||||
</action>
|
||||
<acceptance_criteria>
|
||||
- Exports `Pax8SyncService` and `getPax8SyncService`
|
||||
- `fullSync` exists; there is no `incrementalSync` (full-sync-only per D-07)
|
||||
- Products are upserted only when referenced by a subscription (D-01) and the full catalog is fetched via listAllProducts (no /products/{id} call)
|
||||
- category is populated from `category ?? vendorName` and each of the three tables' tombstone uses `id <> ALL($1::uuid[])`
|
||||
- `grep -cE "<> ALL\(\\\$1::uuid\[\]\)" lib/services/pax8-sync-service.ts` returns 3 (companies, subscriptions, products)
|
||||
- `grep -nE "method:[[:space:]]*['\"](POST|PUT|PATCH|DELETE)|fetch\(" lib/services/pax8-sync-service.ts` returns nothing (the service issues no HTTP itself; PAX8-08)
|
||||
- No SQL string interpolation: `grep -nE "query\(\s*[\`'\"].*\\\$\{" lib/services/pax8-sync-service.ts` returns nothing
|
||||
- `npx tsc --noEmit --pretty` passes
|
||||
</acceptance_criteria>
|
||||
<verify>
|
||||
<automated>npx tsc --noEmit --pretty && ! grep -nE "method:[[:space:]]*['\"](POST|PUT|PATCH|DELETE)" lib/services/pax8-sync-service.ts && ! grep -nE "[^.]fetch\(" lib/services/pax8-sync-service.ts && echo OK</automated>
|
||||
</verify>
|
||||
<done>Pax8SyncService.fullSync populates all three tables read-only with soft-delete reconciliation and referenced-only catalog; compiles clean; no PAX8 writes.</done>
|
||||
</task>
|
||||
|
||||
<task type="auto">
|
||||
<name>Task 2: Add the fire-and-forget /api/pax8/sync route</name>
|
||||
<files>app/api/pax8/sync/route.ts</files>
|
||||
<read_first>
|
||||
- app/api/itglue/sync/route.ts (cleanest fire-and-forget analog: 409 guard, background .catch, GET status with counts + recent history)
|
||||
- app/api/veeam/sync/route.ts lines 13-46 (409 in-progress guard shape)
|
||||
- lib/services/pax8-sync-service.ts (getPax8SyncService, isSyncInProgress, fullSync — built in Task 1)
|
||||
- middleware.ts (public-route allowlist — confirm /api/pax8/sync is NOT added; it stays behind the session-cookie check like other sync routes, unlike webhooks)
|
||||
</read_first>
|
||||
<action>
|
||||
Create `app/api/pax8/sync/route.ts` exporting `POST` and `GET`. POST: read the
|
||||
JSON body defensively (`await req.json().catch(() => ({}))`), take
|
||||
`triggeredBy = body.triggeredBy || 'manual'`, get the service via
|
||||
`getPax8SyncService()`, return `NextResponse.json({ error: 'Sync already in
|
||||
progress' }, { status: 409 })` if `isSyncInProgress()`, otherwise call
|
||||
`svc.fullSync(triggeredBy).catch(err => console.error('[Pax8Sync] Background
|
||||
sync error:', err.message))` WITHOUT awaiting (fire-and-forget) and return
|
||||
`NextResponse.json({ ok: true, message: 'PAX8 sync started' })`. GET: return
|
||||
`{ inProgress: svc.isSyncInProgress(), counts, history }` where counts are
|
||||
non-deleted row counts from pax8_companies/pax8_subscriptions/pax8_products
|
||||
(single parameterized/constant query) and history is the last 10 `sync_history`
|
||||
rows WHERE `entity_type='pax8'` ordered by started_at DESC, wrapped in
|
||||
try/catch returning `{ error }` with status 500 on failure. Do NOT add
|
||||
`/api/pax8/sync` to middleware.ts's public-route allowlist — it must remain
|
||||
behind the existing session-cookie check.
|
||||
</action>
|
||||
<acceptance_criteria>
|
||||
- Exports both `POST` and `GET`
|
||||
- POST returns 409 when `isSyncInProgress()` is true and calls `fullSync` without `await` otherwise
|
||||
- `grep -c "pax8/sync\|/api/pax8" middleware.ts` returns 0 (route is not made public)
|
||||
- `npx tsc --noEmit --pretty` passes
|
||||
</acceptance_criteria>
|
||||
<verify>
|
||||
<automated>npx tsc --noEmit --pretty && grep -q "export async function POST" app/api/pax8/sync/route.ts && grep -q "export async function GET" app/api/pax8/sync/route.ts && ! grep -qE "/api/pax8" middleware.ts && echo OK</automated>
|
||||
</verify>
|
||||
<done>POST /api/pax8/sync triggers fullSync fire-and-forget with a 409 guard; GET reports status/counts/history; route stays session-gated.</done>
|
||||
</task>
|
||||
|
||||
</tasks>
|
||||
|
||||
<threat_model>
|
||||
## Trust Boundaries
|
||||
|
||||
| Boundary | Description |
|
||||
|----------|-------------|
|
||||
| Pulse server → PAX8 API | Outbound read-only GET calls via getPax8Client() |
|
||||
| PAX8 response → Postgres | Untrusted external data written via bulk upsert |
|
||||
| Client (browser/session) → /api/pax8/sync | Authenticated trigger; session-cookie gated by middleware |
|
||||
|
||||
## STRIDE Threat Register
|
||||
|
||||
| Threat ID | Category | Component | Disposition | Mitigation Plan |
|
||||
|-----------|----------|-----------|-------------|-----------------|
|
||||
| T-11-04 | Elevation of Privilege | pax8-sync-service.ts (ROADMAP SC#4 write-invariant) | mitigate | Service calls only client read methods; issues no HTTP itself; grep gate asserts no POST/PUT/PATCH/DELETE and no `fetch(` in the service — upholds PAX8-08 |
|
||||
| T-11-05 | Tampering (SQL injection) | pax8-sync-service.ts upsert/tombstone | mitigate | All queries parameterized ($1,$2,…); UUID arrays passed as `$1::uuid[]` params, never interpolated; grep gate asserts no `${` template interpolation inside query() |
|
||||
| T-11-06 | Spoofing/Repudiation | app/api/pax8/sync route | mitigate | Route NOT added to middleware public allowlist; remains behind session-cookie check (matches itglue/veeam sync routes; not a public webhook) |
|
||||
| T-11-07 | Denial of Service | PAX8 1000/min rate limit during pagination | mitigate | Entities synced sequentially (companies→subscriptions→products), not fanned out concurrently; existing 429 Retry-After backoff in the client's fetchJson is inherited |
|
||||
| T-11-08 | Denial of Service (self-inflicted) | concurrent sync triggers | mitigate | `syncing` guard + route 409 prevent overlapping runs that would double-hit PAX8 and race the tombstone pass |
|
||||
| T-11-SC | Tampering | package installs | accept | This plan installs zero npm/pip/cargo packages (native fetch + existing pg) — package legitimacy gate not triggered |
|
||||
</threat_model>
|
||||
|
||||
<verification>
|
||||
- `npx tsc --noEmit --pretty` passes
|
||||
- No POST/PUT/PATCH/DELETE method and no bare `fetch(` in pax8-sync-service.ts (PAX8-08)
|
||||
- Three UUID-array tombstone passes present (companies/subscriptions/products)
|
||||
- /api/pax8/sync absent from middleware.ts public allowlist
|
||||
- POST and GET exported from the route
|
||||
</verification>
|
||||
|
||||
<success_criteria>
|
||||
Pax8SyncService.fullSync reads companies/subscriptions/products from PAX8 (read-only),
|
||||
upserts them into Postgres with dual cost columns and a referenced-only readable
|
||||
catalog, soft-deletes anything PAX8 no longer returns, and is triggerable via a
|
||||
session-gated fire-and-forget POST /api/pax8/sync with a 409 in-progress guard.
|
||||
</success_criteria>
|
||||
|
||||
<output>
|
||||
Create `.planning/phases/11-company-catalog-subscription-sync/11-02-SUMMARY.md` when done
|
||||
</output>
|
||||
|
|
@ -0,0 +1,175 @@
|
|||
---
|
||||
phase: 11-company-catalog-subscription-sync
|
||||
plan: 03
|
||||
type: execute
|
||||
wave: 3
|
||||
depends_on: [11-02]
|
||||
files_modified: []
|
||||
autonomous: false
|
||||
requirements: [PAX8-03, PAX8-04, PAX8-05, PAX8-08]
|
||||
|
||||
user_setup:
|
||||
- service: pax8
|
||||
why: "Live sync run requires real PAX8 partner credentials to populate Postgres"
|
||||
env_vars:
|
||||
- name: PAX8_CLIENT_ID
|
||||
source: "PAX8 developer portal (devx.pax8.com) — provisioned client ID (per SEED-002)"
|
||||
- name: PAX8_CLIENT_SECRET
|
||||
source: "PAX8 developer portal (devx.pax8.com) — provisioned client secret"
|
||||
|
||||
must_haves:
|
||||
truths:
|
||||
- "A real sync run populates pax8_companies with rows (SC#1)"
|
||||
- "A real sync run populates pax8_products and pax8_subscriptions with rows, and subscriptions carry price + partner_cost (SC#2)"
|
||||
- "A subscription joined to pax8_products shows a readable product name and category, not a bare SKU/product id (SC#3)"
|
||||
- "The client and sync service issue no PAX8 write calls (SC#4 / PAX8-08)"
|
||||
artifacts: []
|
||||
key_links:
|
||||
- from: "POST /api/pax8/sync"
|
||||
to: "pax8_companies / pax8_subscriptions / pax8_products"
|
||||
via: "live sync run populates rows"
|
||||
pattern: "pax8_"
|
||||
---
|
||||
|
||||
<objective>
|
||||
End-to-end verification of the Phase 11 sync against the live PAX8 API and the dev
|
||||
database — the four ROADMAP success criteria can only be proven with real
|
||||
credentials and a real run (matching the project convention that no integration
|
||||
client has an automated live-API test tier).
|
||||
|
||||
Purpose: Confirm the sync actually populates readable, cost-bearing data and holds
|
||||
the read-only invariant before the phase is marked complete.
|
||||
|
||||
Output: developer confirmation (no code changes in this plan).
|
||||
</objective>
|
||||
|
||||
<execution_context>
|
||||
@$HOME/.claude/get-shit-done/workflows/execute-plan.md
|
||||
@$HOME/.claude/get-shit-done/templates/summary.md
|
||||
</execution_context>
|
||||
|
||||
<context>
|
||||
@.planning/ROADMAP.md
|
||||
@.planning/phases/11-company-catalog-subscription-sync/11-01-SUMMARY.md
|
||||
@.planning/phases/11-company-catalog-subscription-sync/11-02-SUMMARY.md
|
||||
</context>
|
||||
|
||||
<tasks>
|
||||
|
||||
<task type="auto">
|
||||
<name>Task 1: Static read-only invariant proof (SC#4 / PAX8-08)</name>
|
||||
<files>lib/services/pax8-client.ts, lib/services/pax8-sync-service.ts</files>
|
||||
<read_first>
|
||||
- lib/services/pax8-client.ts
|
||||
- lib/services/pax8-sync-service.ts
|
||||
</read_first>
|
||||
<action>
|
||||
Prove by inspection that no data-write call to the PAX8 API exists. Grep the
|
||||
client and the sync service for mutating HTTP verbs and confirm the only POST
|
||||
anywhere in the PAX8 client is the `/v1/token` OAuth handshake (an auth
|
||||
exchange, not a data-resource write), and that the sync service issues no HTTP
|
||||
at all (it only calls the client's read methods). Record the grep output in the
|
||||
summary as the PAX8-08 evidence.
|
||||
</action>
|
||||
<acceptance_criteria>
|
||||
- `grep -nE "method:[[:space:]]*['\"](PUT|PATCH|DELETE)" lib/services/pax8-client.ts` returns nothing
|
||||
- The only POST in pax8-client.ts targets `https://api.pax8.com/v1/token`
|
||||
- `grep -nE "method:[[:space:]]*['\"](POST|PUT|PATCH|DELETE)" lib/services/pax8-sync-service.ts` returns nothing
|
||||
</acceptance_criteria>
|
||||
<verify>
|
||||
<automated>! grep -nE "method:[[:space:]]*['\"](PUT|PATCH|DELETE)" lib/services/pax8-client.ts && ! grep -nE "method:[[:space:]]*['\"](POST|PUT|PATCH|DELETE)" lib/services/pax8-sync-service.ts && echo OK</automated>
|
||||
</verify>
|
||||
<done>Grep evidence confirms no PAX8 data-write path exists (SC#4 upheld).</done>
|
||||
</task>
|
||||
|
||||
<task type="checkpoint:human-verify" gate="blocking">
|
||||
<name>Task 2: Live sync run + database verification (SC#1-3)</name>
|
||||
<files></files>
|
||||
<read_first>
|
||||
- .planning/ROADMAP.md Phase 11 Success Criteria #1-3
|
||||
- .planning/phases/11-company-catalog-subscription-sync/11-02-SUMMARY.md
|
||||
</read_first>
|
||||
<action>
|
||||
Human-verification checkpoint (no code changes). Pause and have the developer
|
||||
run the live PAX8 sync against real credentials and confirm the database
|
||||
outcomes described in how-to-verify below. Do not auto-approve — this gate
|
||||
confirms ROADMAP Success Criteria #1-3, which cannot be proven without live
|
||||
credentials and a real run.
|
||||
</action>
|
||||
<what-built>
|
||||
Pax8SyncService.fullSync (read-only) and POST /api/pax8/sync, which sync PAX8
|
||||
companies, subscriptions (with customer price + partner cost), and a
|
||||
referenced-only readable product catalog into Postgres with soft-delete
|
||||
reconciliation.
|
||||
</what-built>
|
||||
<how-to-verify>
|
||||
1. Ensure `PAX8_CLIENT_ID` and `PAX8_CLIENT_SECRET` are set in the environment
|
||||
the dev server reads (`.env` / `.env.local`) and restart the app if needed.
|
||||
2. Trigger a sync: `curl -X POST http://localhost:3100/api/pax8/sync` — expect
|
||||
`{ ok: true, message: "PAX8 sync started" }`. An immediate second POST while
|
||||
it runs should return HTTP 409.
|
||||
3. Wait for completion, then check status: `curl http://localhost:3100/api/pax8/sync`
|
||||
— expect `inProgress: false` and non-zero `counts` for companies,
|
||||
subscriptions, and products, plus a recent `sync_history` entry with
|
||||
status `completed`.
|
||||
4. In psql against the dev DB confirm SC#1/#2: `SELECT count(*) FROM pax8_companies WHERE is_deleted=false;`
|
||||
is > 0; `SELECT count(*) FROM pax8_subscriptions WHERE is_deleted=false;` is > 0;
|
||||
`SELECT count(*) FROM pax8_subscriptions WHERE price IS NOT NULL OR partner_cost IS NOT NULL;`
|
||||
is > 0 (dual cost columns populated).
|
||||
5. Confirm SC#3 (readable via join, not bare SKU):
|
||||
`SELECT s.id, p.name AS product_name, p.category, s.quantity, s.price, s.partner_cost
|
||||
FROM pax8_subscriptions s JOIN pax8_products p ON p.id = s.product_id
|
||||
WHERE s.is_deleted=false LIMIT 5;` — rows show a human-readable product name
|
||||
and a category value (not a UUID/SKU).
|
||||
6. Confirm D-01 referenced-only catalog: every non-deleted product is referenced
|
||||
by at least one subscription — `SELECT count(*) FROM pax8_products p WHERE p.is_deleted=false
|
||||
AND NOT EXISTS (SELECT 1 FROM pax8_subscriptions s WHERE s.product_id = p.id);`
|
||||
should be 0 (or explain any expected exceptions).
|
||||
7. (Optional soft-delete spot check, D-05/D-06) Re-run the sync; confirm counts
|
||||
stay stable and no rows were hard-deleted (is_deleted toggling only).
|
||||
</how-to-verify>
|
||||
<acceptance_criteria>
|
||||
- GET /api/pax8/sync reports non-zero counts for companies, subscriptions, and products and a completed sync_history entry
|
||||
- A subscription joined to pax8_products returns a readable product name + category (SC#3)
|
||||
- At least one subscription row has price and/or partner_cost populated (SC#2)
|
||||
- No non-deleted product is unreferenced by any subscription (D-01), barring explained exceptions
|
||||
</acceptance_criteria>
|
||||
<verify>
|
||||
<human-check>Developer confirms the DB queries in how-to-verify return the expected non-zero, readable, cost-bearing results.</human-check>
|
||||
</verify>
|
||||
<done>All four ROADMAP Phase 11 success criteria confirmed against live data; developer typed "approved".</done>
|
||||
<resume-signal>Type "approved" once the DB checks pass, or describe what failed.</resume-signal>
|
||||
</task>
|
||||
|
||||
</tasks>
|
||||
|
||||
<threat_model>
|
||||
## Trust Boundaries
|
||||
|
||||
| Boundary | Description |
|
||||
|----------|-------------|
|
||||
| Pulse server → PAX8 API | Live outbound read-only calls with real credentials |
|
||||
| Operator → .env secrets | Real PAX8 client secret placed in a committed .env is a disclosure risk (per CLAUDE.md warning) |
|
||||
|
||||
## STRIDE Threat Register
|
||||
|
||||
| Threat ID | Category | Component | Disposition | Mitigation Plan |
|
||||
|-----------|----------|-----------|-------------|-----------------|
|
||||
| T-11-09 | Information Disclosure | PAX8_CLIENT_SECRET in committed .env | accept | CLAUDE.md documents that .env is committed; treated like every other integration secret in this repo — flag to operator, prefer .env.local if available; no code change in scope |
|
||||
| T-11-04 | Elevation of Privilege | live sync run (SC#4 write-invariant) | mitigate | Task 1 grep proof plus the run itself only exercises read endpoints; upholds PAX8-08 end-to-end |
|
||||
</threat_model>
|
||||
|
||||
<verification>
|
||||
- Static grep proof: no PAX8 write path (Task 1)
|
||||
- Live run populates all three tables; join yields readable name + category; costs populated (Task 2)
|
||||
</verification>
|
||||
|
||||
<success_criteria>
|
||||
All four ROADMAP Phase 11 success criteria verified against the live PAX8 API and
|
||||
the dev database, and the read-only invariant (PAX8-08 / SC#4) confirmed by
|
||||
inspection and by the run exercising only read endpoints.
|
||||
</success_criteria>
|
||||
|
||||
<output>
|
||||
Create `.planning/phases/11-company-catalog-subscription-sync/11-03-SUMMARY.md` when done
|
||||
</output>
|
||||
Loading…
Add table
Add a link
Reference in a new issue