docs(12): create phase plan

This commit is contained in:
lorentz 2026-07-10 22:27:17 -04:00
parent fcc93b38f4
commit f68e1c50b6
5 changed files with 77 additions and 42 deletions

View file

@ -285,7 +285,7 @@ render, including the manual-resolution workflow for flagged companies.
3. A PAX8 company with no match, or with multiple similarly-scored Autotask candidates, is persisted with a flagged/needs-review status instead of being auto-assigned
4. Re-running the sync does not overwrite a match that has already been manually confirmed/resolved (idempotent with respect to human decisions)
**Plans**: 5 plans
- [ ] 12-01-PLAN.md — Migration 092 (pg_trgm + pax8_order_items/pax8_companies columns) + Pax8Invoice/Pax8InvoiceItem types (PAX8-06, PAX8-10, PAX8-11)
- [ ] 12-01-PLAN.md — Migration 093 (pg_trgm + pax8_order_items/pax8_companies columns) + Pax8Invoice/Pax8InvoiceItem types (PAX8-06, PAX8-10, PAX8-11)
- [ ] 12-02-PLAN.md — pax8-client listAllInvoices/listAllInvoiceItems + tests + live field-mapping spot-check (PAX8-06)
- [ ] 12-03-PLAN.md — pax8-company-matcher.ts (pg_trgm similarity, 0.90 threshold, tie/empty/idempotency policy) + tests (PAX8-10, PAX8-11)
- [ ] 12-04-PLAN.md — syncOrders + syncCompanyMatches wired into Pax8SyncService.fullSync + sync-service tests (PAX8-06, PAX8-10, PAX8-11)

View file

@ -2,14 +2,14 @@
gsd_state_version: 1.0
milestone: v2.0
milestone_name: PAX8 Integration
status: planning
status: executing
stopped_at: Phase 12 context gathered
last_updated: "2026-07-11T01:35:38.200Z"
last_activity: 2026-07-11
last_updated: "2026-07-11T02:26:37.362Z"
last_activity: 2026-07-11 -- Phase 12 planning complete
progress:
total_phases: 5
completed_phases: 2
total_plans: 6
total_plans: 11
completed_plans: 6
percent: 40
---
@ -27,8 +27,8 @@ See: .planning/PROJECT.md (updated 2026-07-10)
Phase: 12
Plan: Not started
Status: Ready to plan
Last activity: 2026-07-11
Status: Ready to execute
Last activity: 2026-07-11 -- Phase 12 planning complete
Progress: [░░░░░░░░░░] 0%

View file

@ -5,7 +5,7 @@ type: execute
wave: 1
depends_on: []
files_modified:
- migrations/092_pax8_orders_company_matching.sql
- migrations/093_pax8_orders_company_matching.sql
- lib/types/pax8.ts
autonomous: true
requirements: [PAX8-06, PAX8-10, PAX8-11]
@ -17,18 +17,18 @@ must_haves:
- "pax8_companies has auto-match columns (autotask_company_id, match_confidence, match_method, matched_at)"
- "TypeScript exposes Pax8Invoice and Pax8InvoiceItem types matching the live PAX8 /invoices field shape"
artifacts:
- path: "migrations/092_pax8_orders_company_matching.sql"
- path: "migrations/093_pax8_orders_company_matching.sql"
provides: "pg_trgm + additive columns on pax8_order_items and pax8_companies"
contains: "CREATE EXTENSION IF NOT EXISTS pg_trgm"
- path: "lib/types/pax8.ts"
provides: "Pax8Invoice / Pax8InvoiceItem API types"
contains: "Pax8InvoiceItem"
key_links:
- from: "migrations/092_pax8_orders_company_matching.sql"
- from: "migrations/093_pax8_orders_company_matching.sql"
to: "pax8_order_items"
via: "ALTER TABLE ADD COLUMN IF NOT EXISTS"
pattern: "ALTER TABLE pax8_order_items"
- from: "migrations/092_pax8_orders_company_matching.sql"
- from: "migrations/093_pax8_orders_company_matching.sql"
to: "pax8_companies"
via: "ALTER TABLE ADD COLUMN IF NOT EXISTS"
pattern: "ALTER TABLE pax8_companies"
@ -48,7 +48,7 @@ migration and enables the `pg_trgm` extension the matcher needs.
Purpose: Everything downstream (client methods, matcher, sync wiring) depends
on these columns and types existing first.
Output: migrations/092_pax8_orders_company_matching.sql applied to the dev DB;
Output: migrations/093_pax8_orders_company_matching.sql applied to the dev DB;
Pax8Invoice / Pax8InvoiceItem types in lib/types/pax8.ts.
</objective>
@ -90,8 +90,8 @@ unchanged.
<tasks>
<task type="auto">
<name>Task 1: Write migration 092 (pg_trgm + additive columns) and apply to dev DB</name>
<files>migrations/092_pax8_orders_company_matching.sql</files>
<name>Task 1: Write migration 093 (pg_trgm + additive columns) and apply to dev DB</name>
<files>migrations/093_pax8_orders_company_matching.sql</files>
<read_first>
- migrations/091_pax8_tables.sql (current pax8_order_items and pax8_companies definitions this migration ALTERs — confirm which columns already exist so ADD COLUMN IF NOT EXISTS is correct)
- migrations/069_create_analyzer_tables.sql (CREATE EXTENSION IF NOT EXISTS pgcrypto precedent — copy this exact statement shape for pg_trgm)
@ -99,7 +99,7 @@ unchanged.
- scripts/apply-migrations.sh (how migrations are applied to the running container; DB_USER default pulse_user, DB_NAME default pulse_autotask)
</read_first>
<action>
Create migrations/092_pax8_orders_company_matching.sql, additive-only, using
Create migrations/093_pax8_orders_company_matching.sql, additive-only, using
CREATE EXTENSION IF NOT EXISTS and ADD COLUMN IF NOT EXISTS / CREATE INDEX IF
NOT EXISTS throughout (never edit committed migration 091). Include a header
comment stating this is Phase 12, that it enables pg_trgm and closes the
@ -135,13 +135,13 @@ unchanged.
Then apply to the running dev database (existing volume — Postgres init does
not re-run migrations per CLAUDE.md): run
`docker exec -i pulse-postgres psql -U pulse_user -d pulse_autotask < migrations/092_pax8_orders_company_matching.sql`.
`docker exec -i pulse-postgres psql -U pulse_user -d pulse_autotask < migrations/093_pax8_orders_company_matching.sql`.
If the container name or credentials differ, read scripts/apply-migrations.sh
and adapt.
</action>
<acceptance_criteria>
- File migrations/092_pax8_orders_company_matching.sql exists and contains the literal string `CREATE EXTENSION IF NOT EXISTS pg_trgm`
- `grep -c "ADD COLUMN IF NOT EXISTS" migrations/092_pax8_orders_company_matching.sql` returns 13 (9 on pax8_order_items + 4 on pax8_companies)
- File migrations/093_pax8_orders_company_matching.sql exists and contains the literal string `CREATE EXTENSION IF NOT EXISTS pg_trgm`
- `grep -c "ADD COLUMN IF NOT EXISTS" migrations/093_pax8_orders_company_matching.sql` returns 13 (9 on pax8_order_items + 4 on pax8_companies)
- After apply, `docker exec pulse-postgres psql -U pulse_user -d pulse_autotask -tAc "SELECT 1 FROM pg_extension WHERE extname='pg_trgm'"` prints `1`
- After apply, `docker exec pulse-postgres psql -U pulse_user -d pulse_autotask -tAc "SELECT count(*) FROM information_schema.columns WHERE table_name='pax8_order_items' AND column_name IN ('pax8_company_id','subscription_id','item_type','sku','description','start_period','end_period','partner_cost','partner_cost_total')"` prints `9`
- After apply, the same query for table_name='pax8_companies' and column_name IN ('autotask_company_id','match_confidence','match_method','matched_at') prints `4`
@ -214,7 +214,7 @@ unchanged.
| Threat ID | Category | Component | Disposition | Mitigation Plan |
|-----------|----------|-----------|-------------|-----------------|
| T-12-01 | Tampering | migration 092 DDL | mitigate | Additive-only (ADD COLUMN IF NOT EXISTS, CREATE INDEX IF NOT EXISTS, CREATE EXTENSION IF NOT EXISTS); no DROP/ALTER of existing columns; re-runnable without data loss |
| T-12-01 | Tampering | migration 093 DDL | mitigate | Additive-only (ADD COLUMN IF NOT EXISTS, CREATE INDEX IF NOT EXISTS, CREATE EXTENSION IF NOT EXISTS); no DROP/ALTER of existing columns; re-runnable without data loss |
| T-12-SC | Tampering | package installs | accept | No new npm/pip/cargo packages this phase; pg_trgm is a Postgres 16 contrib extension verified live against the project's own container (12-RESEARCH.md) — not a registry package |
</threat_model>

View file

@ -49,9 +49,14 @@ Invoice → items is a nested per-parent fetch (12-RESEARCH.md Pattern 1): page
94 headers once, then for each header page its ~500-700 items. Per-company cost
lives ONLY on the item (`companyId` is always NULL on the header — Pitfall 1), so
pax8_order_items.pax8_company_id is populated from the item; pax8_orders stays
header-only. Cost column mapping was confirmed live in Plan 02
(unit_price←price, line_total←amountDue, partner_cost←cost,
partner_cost_total←costTotal) — use the mapping recorded in 12-02-SUMMARY.md.
header-only. Plan 02 Task 3 spot-checked the cost mapping live and resolved
RESEARCH.md Open Question 1; the DEFAULT mapping is
unit_price←price, line_total←amountDue, partner_cost←cost,
partner_cost_total←costTotal. **12-02-SUMMARY.md is authoritative and records a
CONFIRM or DIVERGENCE verdict *per item type* ('subscription', 'prorate',
'one-time', and any others found live). Where it records a DIVERGENCE for a type,
Task 1 branches the column mapping on `item.type` instead of using the default —
this is an explicit switch, not a prose reconciliation.**
Purpose: The integration point that makes the phase's data actually populate on a
sync run.
@ -90,7 +95,7 @@ Matcher (Plan 03): import { matchPax8Companies, Pax8CompanyMatchResult } from
pax8_orders columns (migration 091): id UUID, pax8_company_id UUID (stays NULL),
order_date TIMESTAMPTZ, total NUMERIC(12,2), status TEXT, currency CHAR(3),
raw_payload JSONB, synced_at, is_deleted, deleted_at.
pax8_order_items columns (091 + 092): id UUID, order_id UUID NOT NULL (FK CASCADE),
pax8_order_items columns (091 + 093): id UUID, order_id UUID NOT NULL (FK CASCADE),
product_id UUID, quantity INTEGER, unit_price NUMERIC(12,2), line_total
NUMERIC(12,2), currency CHAR(3), raw_payload, synced_at, is_deleted, deleted_at,
+ pax8_company_id UUID, subscription_id UUID, item_type TEXT, sku TEXT,
@ -107,8 +112,8 @@ NUMERIC(12,2), partner_cost_total NUMERIC(12,2).
<read_first>
- lib/services/pax8-sync-service.ts (syncSubscriptions lines 175-253 for the upsert+tombstone shape; syncCompanies lines 113-173 for the single-table baseline; the import block lines 12-21; the per-method try/catch error shape lines 168-172)
- lib/services/itglue-sync-service.ts (the syncModels() per-parent nested-fetch iteration referenced by 12-PATTERNS.md Pattern 1 — parent loop calling a child endpoint per id)
- .planning/phases/12-orders-invoices-company-matching/12-02-SUMMARY.md (the CONFIRMED invoice-item cost field mapping — authoritative for the item upsert)
- .planning/phases/12-orders-invoices-company-matching/12-RESEARCH.md (Pitfall 1 header companyId always NULL; Pitfall 2 field mapping; Pattern 1 nested pagination)
- .planning/phases/12-orders-invoices-company-matching/12-02-SUMMARY.md — AUTHORITATIVE for the invoice-item cost mapping. It records a CONFIRM or DIVERGENCE verdict PER item type ('subscription', 'prorate', 'one-time', and any others the live spot-check found). Read the per-type verdicts BEFORE writing the item upsert: any type marked DIVERGENCE has a corrected field mapping you MUST branch on (see the branching instruction in the action); any type marked CONFIRM uses the default mapping.
- .planning/phases/12-orders-invoices-company-matching/12-RESEARCH.md (Pitfall 1 header companyId always NULL; Pitfall 2 field mapping; Pattern 1 nested pagination; Open Question 1 — the mapping question Plan 02 resolved)
</read_first>
<action>
Extend the type import block to add Pax8Invoice, Pax8InvoiceItem from
@ -131,14 +136,35 @@ NUMERIC(12,2), partner_cost_total NUMERIC(12,2).
quantity, unit_price, line_total, currency, raw_payload, synced_at, is_deleted,
deleted_at, pax8_company_id, subscription_id, item_type, sku, description,
start_period, end_period, partner_cost, partner_cost_total) with order_id =
invoice.id, and the CONFIRMED mapping from 12-02-SUMMARY.md: unit_price←price,
line_total←amountDue, partner_cost←cost, partner_cost_total←costTotal,
pax8_company_id←companyId, subscription_id←subscriptionId, item_type←type,
sku←sku, description←description, start_period←startPeriod, end_period←endPeriod,
invoice.id. Non-cost columns map directly: pax8_company_id←companyId,
subscription_id←subscriptionId, item_type←type, sku←sku,
description←description, start_period←startPeriod, end_period←endPeriod,
product_id←productId, quantity←quantity, currency←currencyCode ?? 'USD',
raw_payload←JSON.stringify(item). ON CONFLICT (id) DO UPDATE SET all the above
+ synced_at=NOW(), is_deleted=false, deleted_at=NULL.
COST-COLUMN MAPPING — branch on item.type per 12-02-SUMMARY.md:
The four cost columns (unit_price, line_total, partner_cost,
partner_cost_total) are the ones Plan 02 Task 3 spot-checked per item type.
Before wiring them, read 12-02-SUMMARY.md's per-type verdict table and encode
it EXPLICITLY — do not reconcile a prose note against a hardcoded mapping:
1. DEFAULT mapping (applies to every item type 12-02-SUMMARY.md marks
CONFIRM): unit_price←price, line_total←amountDue, partner_cost←cost,
partner_cost_total←costTotal.
2. For any item type 12-02-SUMMARY.md marks DIVERGENCE, do NOT use the
default. Implement an explicit `switch (item.type)` (or equivalent
if/else on item.type) that, for that specific type, selects the corrected
source fields exactly as 12-02-SUMMARY.md specifies for it, and falls
through to the DEFAULT for all CONFIRM types (and unknown/new types).
3. If 12-02-SUMMARY.md marks CONFIRM for ALL types, the switch collapses to
a single default branch — that is expected and acceptable. Still write it
as a resolveCostColumns(item) helper returning { unitPrice, lineTotal,
partnerCost, partnerCostTotal } so the per-type branch point is a named,
reviewable seam rather than inline ternaries.
Add a short comment on the helper citing 12-02-SUMMARY.md as the source of
the per-type verdicts.
After all invoices processed, tombstone in child-then-parent order to respect
the FK: first UPDATE pax8_order_items SET is_deleted=true, deleted_at=NOW()
WHERE is_deleted=false AND id <> ALL($1::uuid[]) with seenItemIds; then the
@ -153,16 +179,16 @@ NUMERIC(12,2), partner_cost_total NUMERIC(12,2).
</action>
<acceptance_criteria>
- syncOrders() upserts into both `pax8_orders` and `pax8_order_items`
- The item upsert binds `pax8_company_id` from the item's `companyId` and sets `line_total` from `amountDue`, `partner_cost` from `cost`, `partner_cost_total` from `costTotal`, `unit_price` from `price` (per 12-02-SUMMARY.md)
- Cost columns are resolved through a `resolveCostColumns(item)` helper that branches on `item.type`; for every type 12-02-SUMMARY.md marks CONFIRM the helper uses the default mapping (unit_price←price, line_total←amountDue, partner_cost←cost, partner_cost_total←costTotal), and for any type it marks DIVERGENCE the helper uses that type's corrected mapping from 12-02-SUMMARY.md
- The header insert leaves pax8_orders.pax8_company_id NULL (Pitfall 1)
- Tombstone runs on pax8_order_items before pax8_orders, both using `id <> ALL($1::uuid[])`
- Returns a Pax8EntitySyncResult with `entity: 'orders'`
- `npx tsc --noEmit --pretty` passes
</acceptance_criteria>
<verify>
<automated>npx tsc --noEmit --pretty 2>&1 | tail -5; grep -c "listAllInvoiceItems" lib/services/pax8-sync-service.ts</automated>
<automated>npx tsc --noEmit --pretty 2>&1 | tail -5; grep -c "listAllInvoiceItems\|resolveCostColumns" lib/services/pax8-sync-service.ts</automated>
</verify>
<done>syncOrders performs the nested invoice→item sync with the confirmed cost mapping, per-company id on items, and child-then-parent tombstoning; type-check passes.</done>
<done>syncOrders performs the nested invoice→item sync with the per-type cost mapping resolved through resolveCostColumns() (branching on item.type where 12-02-SUMMARY.md records a divergence), per-company id on items, and child-then-parent tombstoning; type-check passes.</done>
</task>
<task type="auto">
@ -208,12 +234,14 @@ NUMERIC(12,2), partner_cost_total NUMERIC(12,2).
<behavior>
- syncOrders (via fullSync or a directly-invoked instance with a mock client): a two-invoice fixture with items produces upserts into both pax8_orders and pax8_order_items; the item upsert binds companyId into the pax8_company_id parameter position
- unseen headers/items are tombstoned via id <> ALL (the tombstone UPDATE runs)
- the cost-column resolution honors item.type: a 'subscription'-type fixture item maps line_total←amountDue (default); if 12-02-SUMMARY.md recorded a DIVERGENCE for a type, add a fixture item of that type asserting the corrected mapping
- syncCompanyMatches delegates to the mocked matchPax8Companies and returns entity 'company_matches'
- fullSync's entities array includes an 'orders' and a 'company_matches' result
</behavior>
<read_first>
- lib/services/pax8-client.test.ts (vi.fn / vi.mock discipline to mirror)
- lib/services/pax8-sync-service.ts (the constructor accepts an optional client: `constructor(client?: Pax8Client)` — inject a mock client; the query shapes to assert)
- lib/services/pax8-sync-service.ts (the constructor accepts an optional client: `constructor(client?: Pax8Client)` — inject a mock client; the query shapes to assert; the resolveCostColumns helper to exercise per type)
- .planning/phases/12-orders-invoices-company-matching/12-02-SUMMARY.md (which item types are CONFIRM vs DIVERGENCE — drive the fixture item types from this)
- .planning/phases/12-orders-invoices-company-matching/12-RESEARCH.md (Validation Architecture — Wave 0 note that no prior Pax8SyncService test exists; mock postgresClient.query)
</read_first>
<action>
@ -224,20 +252,27 @@ NUMERIC(12,2), partner_cost_total NUMERIC(12,2).
Construct the service with an injected mock Pax8Client exposing
listAllCompanies/listAllSubscriptions/listAllProducts (return []) and
listAllInvoices (two headers) + listAllInvoiceItems (items per header, one with
a companyId + subscriptionId + amountDue/cost/costTotal).
a companyId + subscriptionId + amountDue/cost/costTotal). Give at least one
fixture item type='subscription'; if 12-02-SUMMARY.md marked any type
DIVERGENCE, also add a fixture item of that type so the per-type branch is
exercised.
Assert: after fullSync (or a direct syncOrders call if you expose it via the
instance), the mocked postgresClient.query received an INSERT INTO pax8_orders
and an INSERT INTO pax8_order_items call; for the item insert, the params array
includes the fixture's companyId value (pax8_company_id) and its amountDue value
(line_total). Assert a tombstone UPDATE against pax8_order_items and one against
pax8_orders were issued. Assert the returned Pax8SyncResult.entities contains an
object with entity 'orders' and one with entity 'company_matches'. Keep DB-free
(all mocked), environment node.
(line_total) for the default-mapped ('subscription') item. If a DIVERGENCE-type
fixture item was added, assert its line_total/partner_cost params reflect the
corrected per-type mapping from 12-02-SUMMARY.md rather than the default. Assert
a tombstone UPDATE against pax8_order_items and one against pax8_orders were
issued. Assert the returned Pax8SyncResult.entities contains an object with
entity 'orders' and one with entity 'company_matches'. Keep DB-free (all
mocked), environment node.
</action>
<acceptance_criteria>
- Test asserts INSERT calls against both `pax8_orders` and `pax8_order_items`
- Test asserts the item insert params include the fixture companyId (pax8_company_id) and amountDue (line_total)
- Test asserts the item insert params include the fixture companyId (pax8_company_id) and amountDue (line_total) for a CONFIRM/default-mapped item
- If 12-02-SUMMARY.md recorded a DIVERGENCE type, a fixture item of that type asserts its corrected cost-column mapping
- Test asserts tombstone UPDATEs against both tables
- Test asserts fullSync entities include `entity: 'orders'` and `entity: 'company_matches'`
- `npx vitest run lib/services/pax8-sync-service.test.ts` passes
@ -245,7 +280,7 @@ NUMERIC(12,2), partner_cost_total NUMERIC(12,2).
<verify>
<automated>npx vitest run lib/services/pax8-sync-service.test.ts</automated>
</verify>
<done>The sync service's first unit tests prove the nested order upsert (with per-company mapping), tombstoning, and matcher delegation.</done>
<done>The sync service's first unit tests prove the nested order upsert (with per-type cost mapping), tombstoning, and matcher delegation.</done>
</task>
</tasks>
@ -276,7 +311,7 @@ NUMERIC(12,2), partner_cost_total NUMERIC(12,2).
</verification>
<success_criteria>
fullSync runs orders sync + company matching as first-class entity steps; invoice items land with per-company id, billing period, and dual-cost columns; unseen rows tombstone; matcher delegation reports through the sync result. Unit tests prove it.
fullSync runs orders sync + company matching as first-class entity steps; invoice items land with per-company id, billing period, and dual-cost columns (cost columns resolved per item.type per 12-02-SUMMARY.md); unseen rows tombstone; matcher delegation reports through the sync result. Unit tests prove it.
</success_criteria>
<output>

View file

@ -54,7 +54,7 @@ Output: a reusable verification script + a human confirmation of the results.
- Alternatively invoke getPax8SyncService().fullSync('verify') directly from a
tsx script (dotenv from ../.env.local), like scripts/verify-pax8-auth.ts.
- DB access for assertions: docker exec pulse-postgres psql -U pulse_user -d pulse_autotask.
- Migration 092 (Plan 01) must already be applied to the dev DB.
- Migration 093 (Plan 01) must already be applied to the dev DB.
</interfaces>
</context>