docs(12): create phase plan
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01LHRgZqkzBHBbAbc3KHneuR
This commit is contained in:
parent
6d8b2610bb
commit
1a9c491b03
6 changed files with 1212 additions and 1 deletions
|
|
@ -284,7 +284,12 @@ render, including the manual-resolution workflow for flagged companies.
|
|||
2. At sync time, each PAX8 company is automatically matched to an Autotask company by fuzzy name similarity when a sufficiently confident match exists, and the match is persisted
|
||||
3. A PAX8 company with no match, or with multiple similarly-scored Autotask candidates, is persisted with a flagged/needs-review status instead of being auto-assigned
|
||||
4. Re-running the sync does not overwrite a match that has already been manually confirmed/resolved (idempotent with respect to human decisions)
|
||||
**Plans**: TBD
|
||||
**Plans**: 5 plans
|
||||
- [ ] 12-01-PLAN.md — Migration 092 (pg_trgm + pax8_order_items/pax8_companies columns) + Pax8Invoice/Pax8InvoiceItem types (PAX8-06, PAX8-10, PAX8-11)
|
||||
- [ ] 12-02-PLAN.md — pax8-client listAllInvoices/listAllInvoiceItems + tests + live field-mapping spot-check (PAX8-06)
|
||||
- [ ] 12-03-PLAN.md — pax8-company-matcher.ts (pg_trgm similarity, 0.90 threshold, tie/empty/idempotency policy) + tests (PAX8-10, PAX8-11)
|
||||
- [ ] 12-04-PLAN.md — syncOrders + syncCompanyMatches wired into Pax8SyncService.fullSync + sync-service tests (PAX8-06, PAX8-10, PAX8-11)
|
||||
- [ ] 12-05-PLAN.md — Live full-sync verification of all 4 success criteria + human-verify checkpoint (PAX8-06, PAX8-10, PAX8-11)
|
||||
**UI hint**: no
|
||||
|
||||
### Phase 13: Scheduler & Admin Toggle
|
||||
|
|
|
|||
|
|
@ -0,0 +1,232 @@
|
|||
---
|
||||
phase: 12-orders-invoices-company-matching
|
||||
plan: 01
|
||||
type: execute
|
||||
wave: 1
|
||||
depends_on: []
|
||||
files_modified:
|
||||
- migrations/092_pax8_orders_company_matching.sql
|
||||
- lib/types/pax8.ts
|
||||
autonomous: true
|
||||
requirements: [PAX8-06, PAX8-10, PAX8-11]
|
||||
|
||||
must_haves:
|
||||
truths:
|
||||
- "pg_trgm extension is enabled in the dev Postgres database"
|
||||
- "pax8_order_items has per-company id, billing period, and dual-cost columns"
|
||||
- "pax8_companies has auto-match columns (autotask_company_id, match_confidence, match_method, matched_at)"
|
||||
- "TypeScript exposes Pax8Invoice and Pax8InvoiceItem types matching the live PAX8 /invoices field shape"
|
||||
artifacts:
|
||||
- path: "migrations/092_pax8_orders_company_matching.sql"
|
||||
provides: "pg_trgm + additive columns on pax8_order_items and pax8_companies"
|
||||
contains: "CREATE EXTENSION IF NOT EXISTS pg_trgm"
|
||||
- path: "lib/types/pax8.ts"
|
||||
provides: "Pax8Invoice / Pax8InvoiceItem API types"
|
||||
contains: "Pax8InvoiceItem"
|
||||
key_links:
|
||||
- from: "migrations/092_pax8_orders_company_matching.sql"
|
||||
to: "pax8_order_items"
|
||||
via: "ALTER TABLE ADD COLUMN IF NOT EXISTS"
|
||||
pattern: "ALTER TABLE pax8_order_items"
|
||||
- from: "migrations/092_pax8_orders_company_matching.sql"
|
||||
to: "pax8_companies"
|
||||
via: "ALTER TABLE ADD COLUMN IF NOT EXISTS"
|
||||
pattern: "ALTER TABLE pax8_companies"
|
||||
---
|
||||
|
||||
<objective>
|
||||
Lay the schema + type foundation for both halves of Phase 12: historical
|
||||
invoice/line-item cost storage (PAX8-06) and fuzzy company matching
|
||||
(PAX8-10, PAX8-11).
|
||||
|
||||
Live PAX8 verification (12-RESEARCH.md) proved two schema gaps in the
|
||||
already-committed migration 091: `pax8_order_items` has no per-company id and
|
||||
no billing-period columns (invoice headers carry no per-customer data —
|
||||
`companyId` is always NULL on the header), and `pax8_companies` has nowhere to
|
||||
store a confident auto-match. This plan closes both gaps with a new additive
|
||||
migration and enables the `pg_trgm` extension the matcher needs.
|
||||
|
||||
Purpose: Everything downstream (client methods, matcher, sync wiring) depends
|
||||
on these columns and types existing first.
|
||||
Output: migrations/092_pax8_orders_company_matching.sql applied to the dev DB;
|
||||
Pax8Invoice / Pax8InvoiceItem types in lib/types/pax8.ts.
|
||||
</objective>
|
||||
|
||||
<execution_context>
|
||||
@$HOME/.claude/get-shit-done/workflows/execute-plan.md
|
||||
@$HOME/.claude/get-shit-done/templates/summary.md
|
||||
</execution_context>
|
||||
|
||||
<context>
|
||||
@.planning/PROJECT.md
|
||||
@.planning/ROADMAP.md
|
||||
@.planning/STATE.md
|
||||
@.planning/phases/12-orders-invoices-company-matching/12-CONTEXT.md
|
||||
@.planning/phases/12-orders-invoices-company-matching/12-RESEARCH.md
|
||||
@.planning/phases/12-orders-invoices-company-matching/12-PATTERNS.md
|
||||
|
||||
<interfaces>
|
||||
Existing schema this migration extends (migrations/091_pax8_tables.sql):
|
||||
- pax8_order_items already has: id UUID PK, order_id UUID NOT NULL REFERENCES
|
||||
pax8_orders(id) ON DELETE CASCADE, product_id UUID, quantity INTEGER,
|
||||
unit_price NUMERIC(12,2), line_total NUMERIC(12,2), currency CHAR(3),
|
||||
raw_payload JSONB, synced_at, is_deleted, deleted_at.
|
||||
- pax8_companies already has: id UUID PK, name TEXT NOT NULL, external_id,
|
||||
website, status, city, state_or_province, postal_code, country,
|
||||
raw_payload, synced_at, is_deleted, deleted_at.
|
||||
- companies (Autotask, migrations/001): id BIGINT PK, company_name
|
||||
VARCHAR(255), is_active BOOLEAN.
|
||||
|
||||
Extension-enable precedent (migrations/069, 070): `CREATE EXTENSION IF NOT
|
||||
EXISTS pgcrypto;`
|
||||
|
||||
Existing stale stubs to replace in lib/types/pax8.ts (currently unused — no
|
||||
file imports Pax8Order/Pax8OrderItem): Pax8Order (lines 65-75),
|
||||
Pax8OrderItem (lines 79-88). Keep Pax8EntitySyncResult / Pax8SyncResult
|
||||
unchanged.
|
||||
</interfaces>
|
||||
</context>
|
||||
|
||||
<tasks>
|
||||
|
||||
<task type="auto">
|
||||
<name>Task 1: Write migration 092 (pg_trgm + additive columns) and apply to dev DB</name>
|
||||
<files>migrations/092_pax8_orders_company_matching.sql</files>
|
||||
<read_first>
|
||||
- migrations/091_pax8_tables.sql (current pax8_order_items and pax8_companies definitions this migration ALTERs — confirm which columns already exist so ADD COLUMN IF NOT EXISTS is correct)
|
||||
- migrations/069_create_analyzer_tables.sql (CREATE EXTENSION IF NOT EXISTS pgcrypto precedent — copy this exact statement shape for pg_trgm)
|
||||
- .planning/phases/12-orders-invoices-company-matching/12-RESEARCH.md (Code Examples > migration skeleton; Pitfall 1 and Pitfall 2 explain why each column exists)
|
||||
- scripts/apply-migrations.sh (how migrations are applied to the running container; DB_USER default pulse_user, DB_NAME default pulse_autotask)
|
||||
</read_first>
|
||||
<action>
|
||||
Create migrations/092_pax8_orders_company_matching.sql, additive-only, using
|
||||
CREATE EXTENSION IF NOT EXISTS and ADD COLUMN IF NOT EXISTS / CREATE INDEX IF
|
||||
NOT EXISTS throughout (never edit committed migration 091). Include a header
|
||||
comment stating this is Phase 12, that it enables pg_trgm and closes the
|
||||
per-company/period gap on pax8_order_items (invoice headers carry no
|
||||
per-customer data — see 12-RESEARCH.md Pitfall 1), and that
|
||||
pax8_orders.pax8_company_id will intentionally stay NULL for every real row.
|
||||
|
||||
Statement 1: `CREATE EXTENSION IF NOT EXISTS pg_trgm;`
|
||||
|
||||
Statement 2: ALTER TABLE pax8_order_items ADD COLUMN IF NOT EXISTS these nine
|
||||
columns: pax8_company_id UUID (soft ref to pax8_companies(id), no hard FK —
|
||||
matches the soft-ref convention on pax8_subscriptions.pax8_company_id),
|
||||
subscription_id UUID (soft ref to pax8_subscriptions(id)), item_type TEXT
|
||||
(free-form — 'subscription'/'prorate'/'one-time' and possibly others; NO
|
||||
CHECK constraint, matching pax8_subscriptions.status), sku TEXT, description
|
||||
TEXT, start_period TIMESTAMPTZ, end_period TIMESTAMPTZ, partner_cost
|
||||
NUMERIC(12,2), partner_cost_total NUMERIC(12,2).
|
||||
|
||||
Statement 3: CREATE INDEX IF NOT EXISTS idx_pax8_order_items_company ON
|
||||
pax8_order_items(pax8_company_id); and idx_pax8_order_items_period ON
|
||||
pax8_order_items(start_period, end_period).
|
||||
|
||||
Statement 4: ALTER TABLE pax8_companies ADD COLUMN IF NOT EXISTS
|
||||
autotask_company_id BIGINT (soft ref to companies(id), no hard FK),
|
||||
match_confidence NUMERIC(4,3) (raw pg_trgm score 0.000-1.000), match_method
|
||||
TEXT ('pg_trgm' | 'manual'), matched_at TIMESTAMPTZ.
|
||||
|
||||
Statement 5: CREATE INDEX IF NOT EXISTS idx_pax8_companies_autotask ON
|
||||
pax8_companies(autotask_company_id).
|
||||
|
||||
Do NOT touch pax8_orders, pax8_order_items existing columns, or
|
||||
pax8_company_match_review — they are already correctly shaped by migration 091.
|
||||
|
||||
Then apply to the running dev database (existing volume — Postgres init does
|
||||
not re-run migrations per CLAUDE.md): run
|
||||
`docker exec -i pulse-postgres psql -U pulse_user -d pulse_autotask < migrations/092_pax8_orders_company_matching.sql`.
|
||||
If the container name or credentials differ, read scripts/apply-migrations.sh
|
||||
and adapt.
|
||||
</action>
|
||||
<acceptance_criteria>
|
||||
- File migrations/092_pax8_orders_company_matching.sql exists and contains the literal string `CREATE EXTENSION IF NOT EXISTS pg_trgm`
|
||||
- `grep -c "ADD COLUMN IF NOT EXISTS" migrations/092_pax8_orders_company_matching.sql` returns 13 (9 on pax8_order_items + 4 on pax8_companies)
|
||||
- After apply, `docker exec pulse-postgres psql -U pulse_user -d pulse_autotask -tAc "SELECT 1 FROM pg_extension WHERE extname='pg_trgm'"` prints `1`
|
||||
- After apply, `docker exec pulse-postgres psql -U pulse_user -d pulse_autotask -tAc "SELECT count(*) FROM information_schema.columns WHERE table_name='pax8_order_items' AND column_name IN ('pax8_company_id','subscription_id','item_type','sku','description','start_period','end_period','partner_cost','partner_cost_total')"` prints `9`
|
||||
- After apply, the same query for table_name='pax8_companies' and column_name IN ('autotask_company_id','match_confidence','match_method','matched_at') prints `4`
|
||||
- No ALTER or DROP against pax8_orders / pax8_company_match_review appears in the file
|
||||
</acceptance_criteria>
|
||||
<verify>
|
||||
<automated>docker exec pulse-postgres psql -U pulse_user -d pulse_autotask -tAc "SELECT (SELECT count(*) FROM pg_extension WHERE extname='pg_trgm') || '/' || (SELECT count(*) FROM information_schema.columns WHERE table_name='pax8_order_items' AND column_name IN ('pax8_company_id','subscription_id','item_type','sku','description','start_period','end_period','partner_cost','partner_cost_total')) || '/' || (SELECT count(*) FROM information_schema.columns WHERE table_name='pax8_companies' AND column_name IN ('autotask_company_id','match_confidence','match_method','matched_at'))"</automated>
|
||||
Expected output: 1/9/4
|
||||
</verify>
|
||||
<done>pg_trgm enabled; all 13 additive columns and 3 indexes present in the dev DB; migration is additive-only and does not edit migration 091.</done>
|
||||
</task>
|
||||
|
||||
<task type="auto">
|
||||
<name>Task 2: Add Pax8Invoice / Pax8InvoiceItem types to lib/types/pax8.ts</name>
|
||||
<files>lib/types/pax8.ts</files>
|
||||
<read_first>
|
||||
- lib/types/pax8.ts (current Pax8Order/Pax8OrderItem stubs at lines 65-88 to replace; Pax8Company/Pax8Subscription convention with the `[key: string]: unknown` escape hatch to mirror)
|
||||
- .planning/phases/12-orders-invoices-company-matching/12-RESEARCH.md (Pitfall 2 — the live-verified invoice item JSON shape is the source of truth for field names/types)
|
||||
</read_first>
|
||||
<action>
|
||||
Replace the two stale speculative stubs Pax8Order and Pax8OrderItem (currently
|
||||
unused — grep confirms no importers) with live-verified types named
|
||||
Pax8Invoice and Pax8InvoiceItem (favor these names per 12-RESEARCH.md
|
||||
Recommended Project Structure, matching the client methods listAllInvoices /
|
||||
listAllInvoiceItems added in Plan 02). Keep the `[key: string]: unknown`
|
||||
escape hatch on both.
|
||||
|
||||
Pax8Invoice (header — the partner's consolidated monthly bill): id: string;
|
||||
companyId: string | null (add a comment: always null on the header for this
|
||||
single-tenant reseller account — per-company data lives on items); invoiceDate:
|
||||
string | null; total: number | null; status: string | null; currencyCode:
|
||||
string | null; plus the escape hatch.
|
||||
|
||||
Pax8InvoiceItem (line item — per-company, per-period cost): id: string; type:
|
||||
string | null; externalId: string | null; companyId: string | null;
|
||||
forCompanyId: string | null; companyName: string | null; startPeriod: string |
|
||||
null; endPeriod: string | null; quantity: number | null; unitOfMeasure: string
|
||||
| null; term: string | null; sku: string | null; description: string | null;
|
||||
rateType: string | null; chargeType: string | null; price: number | null;
|
||||
subTotal: number | null; cost: number | null; costTotal: number | null; total:
|
||||
number | null; amountDue: number | null; productId: string | null;
|
||||
productName: string | null; vendorName: string | null; currencyCode: string |
|
||||
null; subscriptionId: string | null; plus the escape hatch.
|
||||
|
||||
Do not change Pax8Company, Pax8Subscription, Pax8Product, Pax8PageEnvelope,
|
||||
Pax8EntitySyncResult, or Pax8SyncResult.
|
||||
</action>
|
||||
<acceptance_criteria>
|
||||
- lib/types/pax8.ts exports interfaces named exactly `Pax8Invoice` and `Pax8InvoiceItem`
|
||||
- `grep -c "Pax8Order" lib/types/pax8.ts` returns 0 (stale stubs removed)
|
||||
- Pax8InvoiceItem declares fields `amountDue`, `cost`, `costTotal`, `subscriptionId`, `startPeriod`, `endPeriod`, `companyId`, `type`
|
||||
- `npx tsc --noEmit --pretty` passes with no new errors
|
||||
</acceptance_criteria>
|
||||
<verify>
|
||||
<automated>npx tsc --noEmit --pretty 2>&1 | tail -5; grep -c "Pax8Order" lib/types/pax8.ts</automated>
|
||||
</verify>
|
||||
<done>Pax8Invoice and Pax8InvoiceItem exist with the live-verified field set; no Pax8Order/Pax8OrderItem references remain; type-check passes.</done>
|
||||
</task>
|
||||
|
||||
</tasks>
|
||||
|
||||
<threat_model>
|
||||
## Trust Boundaries
|
||||
|
||||
| Boundary | Description |
|
||||
|----------|-------------|
|
||||
| migration file → dev Postgres | DDL applied to a live database with existing PAX8 data |
|
||||
|
||||
## STRIDE Threat Register
|
||||
|
||||
| Threat ID | Category | Component | Disposition | Mitigation Plan |
|
||||
|-----------|----------|-----------|-------------|-----------------|
|
||||
| T-12-01 | Tampering | migration 092 DDL | mitigate | Additive-only (ADD COLUMN IF NOT EXISTS, CREATE INDEX IF NOT EXISTS, CREATE EXTENSION IF NOT EXISTS); no DROP/ALTER of existing columns; re-runnable without data loss |
|
||||
| T-12-SC | Tampering | package installs | accept | No new npm/pip/cargo packages this phase; pg_trgm is a Postgres 16 contrib extension verified live against the project's own container (12-RESEARCH.md) — not a registry package |
|
||||
</threat_model>
|
||||
|
||||
<verification>
|
||||
- Migration is idempotent: re-applying it against the dev DB produces no error and no duplicate columns.
|
||||
- `npx tsc --noEmit --pretty` passes.
|
||||
</verification>
|
||||
|
||||
<success_criteria>
|
||||
pg_trgm is enabled; pax8_order_items and pax8_companies carry the new columns; Pax8Invoice/Pax8InvoiceItem types match the live PAX8 field shape; type-check green.
|
||||
</success_criteria>
|
||||
|
||||
<output>
|
||||
Create `.planning/phases/12-orders-invoices-company-matching/12-01-SUMMARY.md` when done.
|
||||
</output>
|
||||
|
|
@ -0,0 +1,233 @@
|
|||
---
|
||||
phase: 12-orders-invoices-company-matching
|
||||
plan: 02
|
||||
type: execute
|
||||
wave: 2
|
||||
depends_on: ["12-01"]
|
||||
files_modified:
|
||||
- lib/services/pax8-client.ts
|
||||
- lib/services/pax8-client.test.ts
|
||||
- scripts/verify-pax8-invoice-items.ts
|
||||
autonomous: true
|
||||
requirements: [PAX8-06]
|
||||
|
||||
must_haves:
|
||||
truths:
|
||||
- "Pax8Client can page through every invoice header via listAllInvoices()"
|
||||
- "Pax8Client can page through a single invoice's line items via listAllInvoiceItems(invoiceId)"
|
||||
- "Both new methods are GET-only (no mutating verb), preserving PAX8-08"
|
||||
- "The live invoice-item cost field mapping (unit_price/line_total/partner_cost/partner_cost_total) is confirmed against real prorate + one-time + subscription items"
|
||||
artifacts:
|
||||
- path: "lib/services/pax8-client.ts"
|
||||
provides: "listAllInvoices + listAllInvoiceItems read methods"
|
||||
contains: "listAllInvoiceItems"
|
||||
- path: "lib/services/pax8-client.test.ts"
|
||||
provides: "pagination + GET-only tests for the new methods"
|
||||
contains: "listAllInvoices"
|
||||
- path: "scripts/verify-pax8-invoice-items.ts"
|
||||
provides: "live field-mapping spot-check resolving RESEARCH Open Question 1"
|
||||
key_links:
|
||||
- from: "lib/services/pax8-client.ts"
|
||||
to: "paginateAll"
|
||||
via: "reuse existing private helper for both new methods"
|
||||
pattern: "paginateAll<Pax8Invoice"
|
||||
---
|
||||
|
||||
<objective>
|
||||
Add the two read-only PAX8 client methods this phase's invoice sync needs
|
||||
(PAX8-06), and resolve the one open field-mapping question from research before
|
||||
the sync service (Plan 04) writes any cost columns.
|
||||
|
||||
`/invoices` is a flat list (94 headers, full history since 2019). Invoice items
|
||||
are a per-invoice child resource (`/invoices/{id}/items`) — there is no flat
|
||||
items endpoint (12-RESEARCH.md Pitfall 3; `/orders` is unreliable, returns 504
|
||||
— do NOT add it). Both methods reuse the existing generic `paginateAll<T>`
|
||||
helper unchanged.
|
||||
|
||||
The spot-check script closes RESEARCH Open Question 1: confirm that the billed
|
||||
line amount is `amountDue` (not `total`), and that partner cost maps from
|
||||
`cost`/`costTotal`, across all three observed item types.
|
||||
|
||||
Purpose: Correct client surface + confirmed cost mapping before Plan 04 codes
|
||||
the upsert SQL.
|
||||
Output: two new client methods, their tests, and a verification script.
|
||||
</objective>
|
||||
|
||||
<execution_context>
|
||||
@$HOME/.claude/get-shit-done/workflows/execute-plan.md
|
||||
@$HOME/.claude/get-shit-done/templates/summary.md
|
||||
</execution_context>
|
||||
|
||||
<context>
|
||||
@.planning/phases/12-orders-invoices-company-matching/12-RESEARCH.md
|
||||
@.planning/phases/12-orders-invoices-company-matching/12-PATTERNS.md
|
||||
|
||||
<interfaces>
|
||||
Existing pax8-client.ts (Plan 01 added Pax8Invoice/Pax8InvoiceItem to lib/types/pax8.ts):
|
||||
- private paginateAll<T>((page, size) => Promise<Pax8PageEnvelope<T>>): Promise<T[]>
|
||||
— size=200, loops until page.number >= page.totalPages - 1; inherits fetchJson's
|
||||
429/Retry-After backoff. GET-only.
|
||||
- Existing analog to copy: listAllProducts() (lines 117-122) returns
|
||||
paginateAll<Pax8Product>((page, size) => this.fetchJson<Pax8PageEnvelope<Pax8Product>>(`/products?page=${page}&size=${size}`)).
|
||||
- Import Pax8Invoice, Pax8InvoiceItem from '@/lib/types/pax8'.
|
||||
|
||||
Existing test helpers (pax8-client.test.ts):
|
||||
- makeMultiPageFetchMock({ resource, pages }) — matches url.includes(`/${resource}`),
|
||||
keys page body by `page=N`. Works for `/invoices` and for `/invoices/{id}/items`
|
||||
(both contain `/invoices`).
|
||||
- SECRET constant + the GET-only / secret-never-leaked assertion patterns.
|
||||
|
||||
verify script precedent: scripts/verify-pax8-auth.ts — dotenv from ../.env.local,
|
||||
getPax8Client(), prints ONLY a summary, never the token/secret. Run with npx tsx.
|
||||
</interfaces>
|
||||
</context>
|
||||
|
||||
<tasks>
|
||||
|
||||
<task type="auto">
|
||||
<name>Task 1: Add listAllInvoices() and listAllInvoiceItems(invoiceId) to Pax8Client</name>
|
||||
<files>lib/services/pax8-client.ts</files>
|
||||
<read_first>
|
||||
- lib/services/pax8-client.ts (paginateAll helper lines 86-101; listAllProducts lines 117-122 as the exact shape to copy; the import line 7 to extend)
|
||||
- lib/types/pax8.ts (Pax8Invoice / Pax8InvoiceItem added in Plan 01)
|
||||
- .planning/phases/12-orders-invoices-company-matching/12-PATTERNS.md (pax8-client.ts section — nested per-invoice method shape)
|
||||
</read_first>
|
||||
<action>
|
||||
Extend the type import on line 7 to include Pax8Invoice and Pax8InvoiceItem.
|
||||
|
||||
Add listAllInvoices(): Promise<Pax8Invoice[]> — identical shape to
|
||||
listAllProducts but hitting `/invoices?page=${page}&size=${size}`, returning
|
||||
paginateAll<Pax8Invoice>. Add a doc comment marking it read-only / GET-only
|
||||
(PAX8-08), consistent with the other listAll* comments.
|
||||
|
||||
Add listAllInvoiceItems(invoiceId: string): Promise<Pax8InvoiceItem[]> —
|
||||
paginateAll<Pax8InvoiceItem> hitting the nested child path
|
||||
`/invoices/${invoiceId}/items?page=${page}&size=${size}`. Same GET-only doc
|
||||
comment. Do NOT add any `/orders` method (12-RESEARCH.md Pitfall 3 — the
|
||||
endpoint is unreliable and not part of the design).
|
||||
|
||||
paginateAll and fetchJson need no changes — both new methods are purely
|
||||
additive call sites.
|
||||
</action>
|
||||
<acceptance_criteria>
|
||||
- pax8-client.ts declares `async listAllInvoices(): Promise<Pax8Invoice[]>` and `async listAllInvoiceItems(invoiceId: string): Promise<Pax8InvoiceItem[]>`
|
||||
- listAllInvoiceItems interpolates the invoiceId into the path segment `/invoices/${invoiceId}/items`
|
||||
- `grep -c "/orders" lib/services/pax8-client.ts` returns 0
|
||||
- No `method:` other than the default GET appears in either new method
|
||||
- `npx tsc --noEmit --pretty` passes
|
||||
</acceptance_criteria>
|
||||
<verify>
|
||||
<automated>npx tsc --noEmit --pretty 2>&1 | tail -5; grep -c "/orders" lib/services/pax8-client.ts</automated>
|
||||
</verify>
|
||||
<done>Both read-only methods exist, reuse paginateAll, and type-check passes; no /orders call added.</done>
|
||||
</task>
|
||||
|
||||
<task type="auto" tdd="true">
|
||||
<name>Task 2: Add pagination + GET-only tests for the new client methods</name>
|
||||
<files>lib/services/pax8-client.test.ts</files>
|
||||
<behavior>
|
||||
- listAllInvoices() concatenates content across all pages in order (multi-page mock, resource 'invoices'), and every request is size=200
|
||||
- listAllInvoiceItems(invoiceId) requests the nested path containing that invoiceId (e.g. `/invoices/inv-123/items`) and concatenates its pages
|
||||
- Every data request issued by both methods is a GET with an Authorization: Bearer header, none use a mutating method (extends the existing PAX8-08 assertion to the new methods)
|
||||
</behavior>
|
||||
<read_first>
|
||||
- lib/services/pax8-client.test.ts (makeMultiPageFetchMock lines 49-98; the listAllSubscriptions/listAllProducts test cases lines 165-223 to copy; the GET-only test lines 225-240)
|
||||
</read_first>
|
||||
<action>
|
||||
Add test cases mirroring the existing listAllSubscriptions/listAllProducts
|
||||
tests. For listAllInvoices use makeMultiPageFetchMock with resource: 'invoices'
|
||||
and assert the concatenated order plus size=200 on each call.
|
||||
|
||||
For listAllInvoiceItems, the shared helper's url.includes('/invoices') already
|
||||
routes the nested path, but it does not assert the invoiceId segment — add an
|
||||
explicit assertion that at least one issued data-call URL contains
|
||||
`/invoices/inv-123/items` (use invoiceId 'inv-123'). If the shared helper's
|
||||
page-keying does not cleanly disambiguate the nested path, write a small
|
||||
bespoke fetch mock for this one case following makeMultiPageFetchMock's shape.
|
||||
|
||||
Extend the existing GET-only / no-mutating-method assertion to also cover a
|
||||
listAllInvoiceItems call so PAX8-08 stays proven for the new surface.
|
||||
</action>
|
||||
<acceptance_criteria>
|
||||
- Test file contains cases titled to cover listAllInvoices and listAllInvoiceItems
|
||||
- The listAllInvoiceItems test asserts a data-call URL includes the literal substring `/invoices/inv-123/items`
|
||||
- `npx vitest run lib/services/pax8-client.test.ts` passes with the new cases included
|
||||
</acceptance_criteria>
|
||||
<verify>
|
||||
<automated>npx vitest run lib/services/pax8-client.test.ts</automated>
|
||||
</verify>
|
||||
<done>New tests pass, proving in-order pagination, the nested invoice-item path, and GET-only behavior for both methods.</done>
|
||||
</task>
|
||||
|
||||
<task type="auto">
|
||||
<name>Task 3: Live field-mapping spot-check script (resolves RESEARCH Open Question 1)</name>
|
||||
<files>scripts/verify-pax8-invoice-items.ts</files>
|
||||
<read_first>
|
||||
- scripts/verify-pax8-auth.ts (the dotenv-from-.env.local + getPax8Client() + secret-safe logging pattern to copy)
|
||||
- .planning/phases/12-orders-invoices-company-matching/12-RESEARCH.md (Open Question 1 and Pitfall 2 — the exact fields and the hypothesized mapping to confirm)
|
||||
- lib/services/pax8-client.ts (the listAllInvoices / listAllInvoiceItems methods from Task 1)
|
||||
</read_first>
|
||||
<action>
|
||||
Create scripts/verify-pax8-invoice-items.ts (run via `npx tsx`), copying the
|
||||
dotenv + getPax8Client + secret-safe logging discipline from
|
||||
verify-pax8-auth.ts (never print the token or client secret — summary output
|
||||
only).
|
||||
|
||||
The script fetches the first invoice via listAllInvoices(), then its items via
|
||||
listAllInvoiceItems(invoice.id). It locates one item of each observed type
|
||||
('subscription', 'prorate', 'one-time') where present, and for each prints:
|
||||
type, quantity, price, subTotal, cost, costTotal, total, amountDue,
|
||||
companyId (non-null?), subscriptionId, startPeriod, endPeriod. It then prints
|
||||
a one-line CONFIRM/DIVERGENCE verdict for the mapping the sync service (Plan 04)
|
||||
will use: unit_price ← price, line_total ← amountDue, partner_cost ← cost,
|
||||
partner_cost_total ← costTotal — flagging any type where amountDue is not the
|
||||
plausible billed amount so Plan 04 can adjust before writing the upsert SQL.
|
||||
|
||||
The script must run read-only (it only calls listAll* GET methods). Print a
|
||||
final summary line with the count of item types inspected.
|
||||
</action>
|
||||
<acceptance_criteria>
|
||||
- Running `npx tsx scripts/verify-pax8-invoice-items.ts` prints, for at least the 'subscription' type, the fields price/subTotal/cost/costTotal/total/amountDue and a CONFIRM or DIVERGENCE verdict line for the unit_price/line_total/partner_cost/partner_cost_total mapping
|
||||
- The script output never contains the client secret or access token
|
||||
- The script issues only GET calls (uses only listAllInvoices / listAllInvoiceItems)
|
||||
- The SUMMARY (12-02-SUMMARY.md) records the confirmed mapping (or any divergence) so Plan 04 can rely on it
|
||||
</acceptance_criteria>
|
||||
<verify>
|
||||
<automated>npx tsx scripts/verify-pax8-invoice-items.ts 2>&1 | tail -20</automated>
|
||||
Manual read of output: confirm the mapping verdict line and that no secret is printed.
|
||||
</verify>
|
||||
<done>Live field mapping confirmed (or divergence documented) for the invoice-item cost columns; result recorded in the SUMMARY for Plan 04 to consume.</done>
|
||||
</task>
|
||||
|
||||
</tasks>
|
||||
|
||||
<threat_model>
|
||||
## Trust Boundaries
|
||||
|
||||
| Boundary | Description |
|
||||
|----------|-------------|
|
||||
| PAX8 API → Pulse client | External JSON responses (untrusted) parsed into typed objects |
|
||||
| PAX8 credentials → logs/output | Client secret + bearer token must never surface in logs or script output |
|
||||
|
||||
## STRIDE Threat Register
|
||||
|
||||
| Threat ID | Category | Component | Disposition | Mitigation Plan |
|
||||
|-----------|----------|-----------|-------------|-----------------|
|
||||
| T-12-03 | Information Disclosure | verify-pax8-invoice-items.ts, new client error paths | mitigate | Copy verify-pax8-auth.ts's summary-only logging; error messages include HTTP status but never the secret (existing pax8-client.test.ts asserts `rejects.not.toThrow(new RegExp(SECRET))`) |
|
||||
| T-12-05 | Elevation of Privilege | new client methods | mitigate | Both methods are GET-only via paginateAll; no mutating verb ever set (PAX8-08); covered by the extended GET-only test |
|
||||
| T-12-SC | Tampering | package installs | accept | No new packages; methods reuse existing client internals |
|
||||
</threat_model>
|
||||
|
||||
<verification>
|
||||
- `npx vitest run lib/services/pax8-client.test.ts` green.
|
||||
- `npx tsc --noEmit --pretty` passes.
|
||||
- Spot-check script runs read-only and prints the mapping verdict with no secret leakage.
|
||||
</verification>
|
||||
|
||||
<success_criteria>
|
||||
listAllInvoices + listAllInvoiceItems exist, are GET-only, tested; the invoice-item cost field mapping is confirmed live and recorded for Plan 04.
|
||||
</success_criteria>
|
||||
|
||||
<output>
|
||||
Create `.planning/phases/12-orders-invoices-company-matching/12-02-SUMMARY.md` when done. Record the confirmed invoice-item field mapping (or any divergence) explicitly.
|
||||
</output>
|
||||
|
|
@ -0,0 +1,287 @@
|
|||
---
|
||||
phase: 12-orders-invoices-company-matching
|
||||
plan: 03
|
||||
type: execute
|
||||
wave: 2
|
||||
depends_on: ["12-01"]
|
||||
files_modified:
|
||||
- lib/services/pax8-company-matcher.ts
|
||||
- lib/services/pax8-company-matcher.test.ts
|
||||
autonomous: true
|
||||
requirements: [PAX8-10, PAX8-11]
|
||||
|
||||
must_haves:
|
||||
truths:
|
||||
- "A PAX8 company with a single candidate scoring >= 0.90 and no near-tie is auto-linked (columns written on pax8_companies)"
|
||||
- "A PAX8 company with the best score < 0.90, or a second candidate within 0.05 of the top, is flagged with the top-3 candidates (never auto-linked)"
|
||||
- "A PAX8 company with zero candidates above the 0.3 floor is flagged with an empty candidate_company_ids array (D-03)"
|
||||
- "A human-resolved match is never overwritten by re-running the matcher (D-05 / SC#4)"
|
||||
- "PAX8 company names are passed as bound parameters into similarity(), never string-interpolated"
|
||||
artifacts:
|
||||
- path: "lib/services/pax8-company-matcher.ts"
|
||||
provides: "matchPax8Companies() + candidate/decision/apply/conflict helpers"
|
||||
exports: ["matchPax8Companies"]
|
||||
min_lines: 120
|
||||
- path: "lib/services/pax8-company-matcher.test.ts"
|
||||
provides: "auto-link / review / empty-candidate / idempotency unit tests"
|
||||
contains: "matchPax8Companies"
|
||||
key_links:
|
||||
- from: "lib/services/pax8-company-matcher.ts"
|
||||
to: "companies"
|
||||
via: "similarity($1, company_name) with is_active filter"
|
||||
pattern: "similarity\\(\\$1, company_name\\)"
|
||||
- from: "lib/services/pax8-company-matcher.ts"
|
||||
to: "pax8_companies"
|
||||
via: "UPDATE auto-match columns guarded against resolved rows"
|
||||
pattern: "UPDATE pax8_companies"
|
||||
- from: "lib/services/pax8-company-matcher.ts"
|
||||
to: "pax8_company_match_review"
|
||||
via: "upsert on the open-review partial unique index"
|
||||
pattern: "INSERT INTO pax8_company_match_review"
|
||||
---
|
||||
|
||||
<objective>
|
||||
Build the fuzzy company matcher (PAX8-10, PAX8-11) as a close structural port of
|
||||
lib/services/device-link-reconciler.ts, scored with Postgres `pg_trgm`
|
||||
similarity() at a conservative, tunable 0.90 auto-link floor with a 0.05
|
||||
tie-margin — the thresholds research validated live against this project's real
|
||||
118 pax8_companies vs 242 active companies (every genuine match scored 1.00; the
|
||||
highest non-match was 0.70).
|
||||
|
||||
Match policy (locked decisions):
|
||||
- D-01: auto-link only at similarity >= 0.90 (AUTO_LINK_THRESHOLD, initial &
|
||||
tunable) with a single candidate and no near-tie.
|
||||
- D-02: never silently tie-break — a second candidate within 0.05 (TIE_MARGIN)
|
||||
of the top score forces review even at score 1.0.
|
||||
- D-03: zero candidates above a 0.3 floor still creates a review row with an
|
||||
empty candidate_company_ids array — never silently dropped.
|
||||
- D-04: ambiguous review rows carry the top 3 candidates.
|
||||
- D-05 / SC#4: only unresolved rows are (re)scored every sync; a human-resolved
|
||||
match (resolved review row, or match_method='manual') is never overwritten.
|
||||
|
||||
Purpose: The hardest problem of the phase, isolated in its own service + test
|
||||
plan (matching logic + SQL is context-heavy). Runs from Pax8SyncService in Plan 04.
|
||||
Output: lib/services/pax8-company-matcher.ts and its unit tests.
|
||||
</objective>
|
||||
|
||||
<execution_context>
|
||||
@$HOME/.claude/get-shit-done/workflows/execute-plan.md
|
||||
@$HOME/.claude/get-shit-done/templates/summary.md
|
||||
</execution_context>
|
||||
|
||||
<context>
|
||||
@.planning/phases/12-orders-invoices-company-matching/12-CONTEXT.md
|
||||
@.planning/phases/12-orders-invoices-company-matching/12-RESEARCH.md
|
||||
@.planning/phases/12-orders-invoices-company-matching/12-PATTERNS.md
|
||||
|
||||
<interfaces>
|
||||
Precedent to port (lib/services/device-link-reconciler.ts):
|
||||
- findBySerial/findByMac/findByHostnameInCompany — parameterized candidate
|
||||
queries filtered on is_deleted=false (adapt to similarity() + is_active=true).
|
||||
- pickBestCandidate — ambiguity guard (adapt to numeric TIE_MARGIN).
|
||||
- applyLink — UPDATE ... WHERE ... AND configuration_item_id IS NULL idempotency
|
||||
guard (adapt to a resolved-row guard).
|
||||
- recordConflict — INSERT ... ON CONFLICT (subject) WHERE resolved_at IS NULL
|
||||
DO UPDATE (device_link_review). pax8_company_match_review has the identical
|
||||
partial-unique-index shape.
|
||||
- reconcileUnlinkedDevices(opts?: { limit?; dryRun? }) — top-level scan loop +
|
||||
ReconcileResult rollup.
|
||||
|
||||
Schema (after Plan 01's migration 092):
|
||||
- pax8_companies: id UUID, name TEXT, is_deleted BOOLEAN, autotask_company_id
|
||||
BIGINT, match_confidence NUMERIC(4,3), match_method TEXT, matched_at TIMESTAMPTZ.
|
||||
- companies (Autotask): id BIGINT, company_name VARCHAR(255), is_active BOOLEAN.
|
||||
- pax8_company_match_review (migration 091): id UUID, pax8_company_id UUID NOT
|
||||
NULL, candidate_company_ids BIGINT[] NOT NULL, match_confidences TEXT[] NOT
|
||||
NULL, detected_at, resolved_at TIMESTAMPTZ, resolved_by_user_id TEXT,
|
||||
resolved_to_company_id BIGINT, resolution_note TEXT. Partial unique index
|
||||
uq_pax8_company_match_review_open on (pax8_company_id) WHERE resolved_at IS NULL.
|
||||
|
||||
postgresClient: default export from '@/lib/services/postgres-client';
|
||||
postgresClient.query<T>(sql, params) — parameterized only, never interpolate.
|
||||
</interfaces>
|
||||
</context>
|
||||
|
||||
<tasks>
|
||||
|
||||
<task type="auto">
|
||||
<name>Task 1: Create lib/services/pax8-company-matcher.ts</name>
|
||||
<files>lib/services/pax8-company-matcher.ts</files>
|
||||
<read_first>
|
||||
- lib/services/device-link-reconciler.ts (full file — the structural precedent: findBy*, applyLink, recordConflict, pickBestCandidate, reconcileUnlinkedDevices)
|
||||
- migrations/091_pax8_tables.sql (pax8_company_match_review shape + partial unique index)
|
||||
- .planning/phases/12-orders-invoices-company-matching/12-PATTERNS.md (pax8-company-matcher.ts section — the exact adapted findCandidates/decide/applyLink/recordConflict snippets AND the "IMPORTANT" note on the resolved_at re-scoring gate)
|
||||
- .planning/phases/12-orders-invoices-company-matching/12-RESEARCH.md (Pattern 2; Pitfall 4 case-insensitivity; Pitfall 5 is_active filter; Security Domain SQL-injection mitigation)
|
||||
- lib/types/pax8.ts (Pax8EntitySyncResult shape the matcher result maps into)
|
||||
</read_first>
|
||||
<action>
|
||||
Create lib/services/pax8-company-matcher.ts. File header comment: states this
|
||||
is the PAX8 company↔Autotask fuzzy matcher, ported from
|
||||
device-link-reconciler.ts, and that unlike the reconciler it runs from
|
||||
Pax8SyncService.fullSync() (Plan 04) — NOT a standalone cron (no scheduler
|
||||
wiring this phase). Import postgresClient default from
|
||||
'@/lib/services/postgres-client'.
|
||||
|
||||
Module constants (export them so they are documented/tunable and testable):
|
||||
AUTO_LINK_THRESHOLD = 0.90 (D-01, conservative initial/tunable value),
|
||||
TIE_MARGIN = 0.05 (D-02), CANDIDATE_FLOOR = 0.3 (keeps the candidate list
|
||||
small). Add a comment citing the live evidence: genuine matches clustered at
|
||||
1.00, highest observed non-match 0.70, so 0.90 sits in the empty gap.
|
||||
|
||||
Types: CompanyCandidate { autotask_company_id: number; score: number }. Export
|
||||
interface Pax8CompanyMatchResult { scanned: number; autoLinked: number;
|
||||
flaggedAmbiguous: number; flaggedNoCandidate: number; durationMs: number }.
|
||||
|
||||
findCandidates(pax8Name: string): parameterized query — SELECT id::text,
|
||||
similarity($1, company_name)::text AS score FROM companies WHERE is_active =
|
||||
true AND similarity($1, company_name) > CANDIDATE_FLOOR ORDER BY score DESC
|
||||
LIMIT 5. Bind pax8Name as $1 (NEVER interpolate — Security Domain). Return
|
||||
rows mapped to CompanyCandidate (Number(id), Number(score)). Rely on
|
||||
pg_trgm's built-in case-insensitivity (Pitfall 4) — do not add LOWER(); TRIM
|
||||
the input name for cleaner review display only.
|
||||
|
||||
decide(candidates): returns { kind: 'auto'; match } | { kind: 'review'; top3 }.
|
||||
Zero candidates → review with empty top3 (D-03). Else let [best, second] =
|
||||
candidates; tie = second exists AND best.score - second.score < TIE_MARGIN.
|
||||
If best.score >= AUTO_LINK_THRESHOLD AND NOT tie → auto. Otherwise review with
|
||||
candidates.slice(0, 3) (D-02/D-04).
|
||||
|
||||
applyLink(pax8CompanyId, autotaskCompanyId, score): UPDATE pax8_companies SET
|
||||
autotask_company_id=$2, match_confidence=$3, match_method='pg_trgm',
|
||||
matched_at=NOW() WHERE id=$1 AND match_method IS DISTINCT FROM 'manual' AND
|
||||
NOT EXISTS (SELECT 1 FROM pax8_company_match_review r WHERE
|
||||
r.pax8_company_id = pax8_companies.id AND r.resolved_at IS NOT NULL). This is
|
||||
the SC#4 / D-05 idempotency guard: never overwrite a manually-resolved match.
|
||||
Then close any open (never-human-touched) review row for that company: DELETE
|
||||
FROM pax8_company_match_review WHERE pax8_company_id=$1 AND resolved_at IS NULL
|
||||
(a now-confident match supersedes an open flag; human-resolved rows have
|
||||
resolved_at set and are untouched). Pass score as a NUMERIC bound param.
|
||||
|
||||
recordConflict(pax8CompanyId, top3): first, if this company previously held a
|
||||
pg_trgm auto-match that is now ambiguous/below-threshold, clear the stale
|
||||
columns conservatively — UPDATE pax8_companies SET autotask_company_id=NULL,
|
||||
match_confidence=NULL, match_method=NULL, matched_at=NULL WHERE id=$1 AND
|
||||
match_method='pg_trgm' (never clears 'manual'). Then upsert the review row:
|
||||
INSERT INTO pax8_company_match_review (pax8_company_id, candidate_company_ids,
|
||||
match_confidences) VALUES ($1, $2::bigint[], $3::text[]) ON CONFLICT
|
||||
(pax8_company_id) WHERE resolved_at IS NULL DO UPDATE SET candidate_company_ids
|
||||
= EXCLUDED.candidate_company_ids, match_confidences = EXCLUDED.match_confidences,
|
||||
detected_at = NOW(). candidate_company_ids = top3 ids (empty [] when no
|
||||
candidates, D-03); match_confidences = top3 scores formatted with toFixed(3)
|
||||
as TEXT[] (consistent precision with match_confidence NUMERIC(4,3)).
|
||||
|
||||
matchPax8Companies(opts?: { limit?: number; dryRun?: boolean }):
|
||||
Promise<Pax8CompanyMatchResult>. Select the re-scoring-eligible set (D-05):
|
||||
SELECT id::text, name FROM pax8_companies WHERE is_deleted = false AND
|
||||
match_method IS DISTINCT FROM 'manual' AND NOT EXISTS (SELECT 1 FROM
|
||||
pax8_company_match_review r WHERE r.pax8_company_id = pax8_companies.id AND
|
||||
r.resolved_at IS NOT NULL) ORDER BY name LIMIT $1 (default limit 1000). Loop:
|
||||
scanned++; candidates = findCandidates(name.trim()); d = decide(candidates);
|
||||
if kind auto → (unless dryRun) applyLink + autoLinked++; if kind review with
|
||||
empty top3 → (unless dryRun) recordConflict(id, []) + flaggedNoCandidate++;
|
||||
else review → (unless dryRun) recordConflict(id, top3) + flaggedAmbiguous++.
|
||||
Wrap in try/catch per the CLAUDE.md convention; return the result with
|
||||
durationMs. Console.log a one-line summary at the end.
|
||||
|
||||
Use error handling shape: catch (err) { const msg = err instanceof Error ?
|
||||
err.message : String(err); console.error('[Pax8Match] failed:', msg); ... }.
|
||||
</action>
|
||||
<acceptance_criteria>
|
||||
- Exports `matchPax8Companies`, `AUTO_LINK_THRESHOLD` (= 0.90), `TIE_MARGIN` (= 0.05), and interface `Pax8CompanyMatchResult`
|
||||
- The candidate query passes the PAX8 name as a bound `$1` param and filters `is_active = true`; `grep -c "company_name +" lib/services/pax8-company-matcher.ts` returns 0 (no string concatenation into SQL)
|
||||
- The applyLink UPDATE contains both `match_method IS DISTINCT FROM 'manual'` and a `resolved_at IS NOT NULL` NOT EXISTS guard
|
||||
- The recordConflict upsert targets `pax8_company_match_review` with `ON CONFLICT (pax8_company_id) WHERE resolved_at IS NULL`
|
||||
- The eligibility SELECT excludes rows where a review row has `resolved_at IS NOT NULL` and where `match_method` = 'manual'
|
||||
- `npx tsc --noEmit --pretty` passes
|
||||
</acceptance_criteria>
|
||||
<verify>
|
||||
<automated>npx tsc --noEmit --pretty 2>&1 | tail -5; grep -c "similarity(\$1, company_name)" lib/services/pax8-company-matcher.ts</automated>
|
||||
</verify>
|
||||
<done>Matcher implements the D-01..D-05 policy with parameterized queries, the resolved-row idempotency guard, and the top-3/empty-array review semantics; type-check passes.</done>
|
||||
</task>
|
||||
|
||||
<task type="auto" tdd="true">
|
||||
<name>Task 2: Create lib/services/pax8-company-matcher.test.ts</name>
|
||||
<files>lib/services/pax8-company-matcher.test.ts</files>
|
||||
<behavior>
|
||||
- auto-link: eligible company, one candidate scoring 0.95 → applyLink UPDATE against pax8_companies is issued; no INSERT into pax8_company_match_review
|
||||
- review (below threshold): best candidate 0.80 → an INSERT/UPSERT into pax8_company_match_review with that candidate; no auto-link UPDATE writing autotask_company_id
|
||||
- review (near-tie): candidates 0.95 and 0.92 (margin < 0.05) → review row with top candidates, no auto-link even though top >= 0.90 (D-02)
|
||||
- empty candidates: zero rows above the floor → review row with an empty candidate_company_ids array (D-03)
|
||||
- idempotent (D-05/SC#4): the applyLink UPDATE SQL includes the `resolved_at IS NOT NULL` guard AND `match_method IS DISTINCT FROM 'manual'`; the eligibility SELECT excludes manually-resolved rows
|
||||
- dryRun: no write queries (no UPDATE/INSERT/DELETE) are issued when dryRun is true
|
||||
</behavior>
|
||||
<read_first>
|
||||
- lib/services/pax8-client.test.ts (vi.fn / vi.stubGlobal mocking discipline to mirror, applied to postgresClient.query instead of fetch)
|
||||
- lib/services/pax8-company-matcher.ts (Task 1 — the query shapes to assert against)
|
||||
- .planning/phases/12-orders-invoices-company-matching/12-RESEARCH.md (Validation Architecture > Phase Requirements → Test Map; Wave 0 Gaps recommends mocking postgresClient.query)
|
||||
</read_first>
|
||||
<action>
|
||||
Create the matcher test with vitest. Mock the postgres client module:
|
||||
vi.mock('@/lib/services/postgres-client', () => ({ default: { query: vi.fn() } })).
|
||||
Import the mocked query and, per test, program its return sequence so the
|
||||
first call (eligibility SELECT) yields the pax8_companies rows under test and
|
||||
subsequent findCandidates calls yield the candidate score rows. Assert
|
||||
behavior by inspecting the SQL string + params of the mock's calls: for
|
||||
auto-link assert a call whose SQL matches /UPDATE pax8_companies/ ran; for
|
||||
review assert a call whose SQL matches /INSERT INTO pax8_company_match_review/
|
||||
ran (and, for the empty case, that its bound candidate id array is empty). For
|
||||
the idempotency test, assert the applyLink SQL string contains both
|
||||
'resolved_at IS NOT NULL' and "IS DISTINCT FROM 'manual'". For dryRun, assert
|
||||
no call SQL matches /UPDATE|INSERT|DELETE/ after the read-only eligibility and
|
||||
candidate SELECTs.
|
||||
|
||||
Follow the RESEARCH test-name map so the -t filters resolve: name the cases so
|
||||
'auto-link', 'review', 'empty candidates', and 'idempotent' each appear.
|
||||
similarity() runs in Postgres, so these tests never touch a real DB — the mock
|
||||
returns canned score rows (keeps the suite fast and consistent with the rest of
|
||||
lib/services/*.test.ts, all pure-mock, environment: 'node').
|
||||
</action>
|
||||
<acceptance_criteria>
|
||||
- Test titles include the substrings 'auto-link', 'review', 'empty candidates', and 'idempotent'
|
||||
- The auto-link test asserts a mock query call whose SQL matches `/UPDATE pax8_companies/`
|
||||
- The empty-candidates test asserts the review upsert's bound `candidate_company_ids` param is an empty array
|
||||
- The idempotent test asserts the applyLink SQL contains `resolved_at IS NOT NULL` and `IS DISTINCT FROM 'manual'`
|
||||
- `npx vitest run lib/services/pax8-company-matcher.test.ts` passes
|
||||
</acceptance_criteria>
|
||||
<verify>
|
||||
<automated>npx vitest run lib/services/pax8-company-matcher.test.ts</automated>
|
||||
</verify>
|
||||
<done>All five decision branches (auto, below-threshold review, near-tie review, empty-candidate review, idempotency guard) plus dryRun are proven with mocked postgresClient.query.</done>
|
||||
</task>
|
||||
|
||||
</tasks>
|
||||
|
||||
<threat_model>
|
||||
## Trust Boundaries
|
||||
|
||||
| Boundary | Description |
|
||||
|----------|-------------|
|
||||
| PAX8 company name → SQL | External string fed into a similarity() comparison — must be a bound parameter |
|
||||
| matcher writes → billing-sensitive match state | A wrong auto-link misattributes cost data between companies |
|
||||
| re-sync → human-resolved rows | Automated re-scoring must never overwrite a manual decision |
|
||||
|
||||
## STRIDE Threat Register
|
||||
|
||||
| Threat ID | Category | Component | Disposition | Mitigation Plan |
|
||||
|-----------|----------|-----------|-------------|-----------------|
|
||||
| T-12-01 | Tampering | findCandidates similarity() query | mitigate | PAX8 name bound as `$1`, never string-interpolated (parameterized query per postgresClient convention); asserted by the no-concatenation acceptance check |
|
||||
| T-12-02 | Tampering / Repudiation | decide() / applyLink() | mitigate | Conservative 0.90 threshold + 0.05 tie-margin + top-3 review; unit tests cover below-threshold, near-tie, and empty-candidate branches so wrong-company cost attribution is caught in CI |
|
||||
| T-12-04 | Tampering | applyLink() re-scoring | mitigate | UPDATE guarded with `match_method IS DISTINCT FROM 'manual'` + `resolved_at IS NOT NULL` NOT EXISTS; eligibility SELECT excludes human-resolved rows; idempotency unit test asserts the guard |
|
||||
| T-12-06 | Denial of Service | matchPax8Companies loop | accept | Bounded by LIMIT (default 1000) and CANDIDATE_FLOOR keeping candidate lists to ≤5; dataset is ~120 pax8 companies — no scaling concern |
|
||||
| T-12-SC | Tampering | package installs | accept | No new packages; pg_trgm is a Postgres contrib extension enabled in Plan 01 |
|
||||
</threat_model>
|
||||
|
||||
<verification>
|
||||
- `npx vitest run lib/services/pax8-company-matcher.test.ts` green.
|
||||
- `npx tsc --noEmit --pretty` passes.
|
||||
- No string concatenation of company names into SQL (grep gate).
|
||||
</verification>
|
||||
|
||||
<success_criteria>
|
||||
The matcher auto-links only on conservative, unambiguous, high-confidence matches; flags everything else (including zero-candidate and near-tie cases); never overwrites a human decision; all parameterized. Unit tests prove each branch.
|
||||
</success_criteria>
|
||||
|
||||
<output>
|
||||
Create `.planning/phases/12-orders-invoices-company-matching/12-03-SUMMARY.md` when done.
|
||||
</output>
|
||||
|
|
@ -0,0 +1,284 @@
|
|||
---
|
||||
phase: 12-orders-invoices-company-matching
|
||||
plan: 04
|
||||
type: execute
|
||||
wave: 3
|
||||
depends_on: ["12-02", "12-03"]
|
||||
files_modified:
|
||||
- lib/services/pax8-sync-service.ts
|
||||
- lib/services/pax8-sync-service.test.ts
|
||||
autonomous: true
|
||||
requirements: [PAX8-06, PAX8-10, PAX8-11]
|
||||
|
||||
must_haves:
|
||||
truths:
|
||||
- "fullSync() runs the invoice/line-item sync and the company matcher as two additional entity steps"
|
||||
- "syncOrders() upserts invoice headers to pax8_orders and line items to pax8_order_items with per-company id, billing period, and dual-cost columns populated"
|
||||
- "syncOrders() tombstones headers and items no longer returned by PAX8 using the id <> ALL($1) soft-delete pattern"
|
||||
- "syncCompanyMatches() delegates to matchPax8Companies() and reports a Pax8EntitySyncResult"
|
||||
artifacts:
|
||||
- path: "lib/services/pax8-sync-service.ts"
|
||||
provides: "syncOrders + syncCompanyMatches steps wired into fullSync"
|
||||
contains: "syncOrders"
|
||||
- path: "lib/services/pax8-sync-service.test.ts"
|
||||
provides: "upsert/tombstone + matcher-delegation unit tests"
|
||||
contains: "syncOrders"
|
||||
key_links:
|
||||
- from: "lib/services/pax8-sync-service.ts"
|
||||
to: "pax8-client listAllInvoices/listAllInvoiceItems"
|
||||
via: "nested per-invoice fetch loop"
|
||||
pattern: "listAllInvoiceItems"
|
||||
- from: "lib/services/pax8-sync-service.ts"
|
||||
to: "pax8-company-matcher matchPax8Companies"
|
||||
via: "syncCompanyMatches wrapper"
|
||||
pattern: "matchPax8Companies"
|
||||
- from: "lib/services/pax8-sync-service.ts"
|
||||
to: "fullSync entities array"
|
||||
via: "entities.push(ordersResult) and entities.push(matchResult)"
|
||||
pattern: "entities.push"
|
||||
---
|
||||
|
||||
<objective>
|
||||
Wire both halves of Phase 12 into the existing Pax8SyncService.fullSync()
|
||||
orchestration: a new syncOrders() step (historical invoice headers + per-company
|
||||
line items, PAX8-06) and a new syncCompanyMatches() step delegating to the Plan
|
||||
03 matcher (PAX8-10, PAX8-11). Both conform to the existing Pax8EntitySyncResult
|
||||
shape and slot into fullSync's entities.push accumulation unchanged.
|
||||
|
||||
Invoice → items is a nested per-parent fetch (12-RESEARCH.md Pattern 1): page all
|
||||
94 headers once, then for each header page its ~500-700 items. Per-company cost
|
||||
lives ONLY on the item (`companyId` is always NULL on the header — Pitfall 1), so
|
||||
pax8_order_items.pax8_company_id is populated from the item; pax8_orders stays
|
||||
header-only. Cost column mapping was confirmed live in Plan 02
|
||||
(unit_price←price, line_total←amountDue, partner_cost←cost,
|
||||
partner_cost_total←costTotal) — use the mapping recorded in 12-02-SUMMARY.md.
|
||||
|
||||
Purpose: The integration point that makes the phase's data actually populate on a
|
||||
sync run.
|
||||
Output: syncOrders + syncCompanyMatches in pax8-sync-service.ts, wired into
|
||||
fullSync, plus the service's first unit tests.
|
||||
</objective>
|
||||
|
||||
<execution_context>
|
||||
@$HOME/.claude/get-shit-done/workflows/execute-plan.md
|
||||
@$HOME/.claude/get-shit-done/templates/summary.md
|
||||
</execution_context>
|
||||
|
||||
<context>
|
||||
@.planning/phases/12-orders-invoices-company-matching/12-RESEARCH.md
|
||||
@.planning/phases/12-orders-invoices-company-matching/12-PATTERNS.md
|
||||
@.planning/phases/12-orders-invoices-company-matching/12-02-SUMMARY.md
|
||||
|
||||
<interfaces>
|
||||
Existing pax8-sync-service.ts:
|
||||
- fullSync(triggeredBy): builds entities: Pax8EntitySyncResult[]; currently pushes
|
||||
syncCompanies(), syncSubscriptions() (returns { result, referencedProductIds }),
|
||||
syncProducts(referencedProductIds). Add ordersResult then matchResult after
|
||||
productsResult, before the rollup that computes success/status/totals.
|
||||
- syncSubscriptions() (lines 175-253) is the closest upsert+tombstone analog:
|
||||
`seen: string[]`, INSERT ... ON CONFLICT (id) DO UPDATE SET ..., synced_at=NOW(),
|
||||
is_deleted=false, deleted_at=NULL; then tombstone UPDATE ... WHERE is_deleted=false
|
||||
AND id <> ALL($1::uuid[]). Per-method try/catch returning a failed
|
||||
Pax8EntitySyncResult on error.
|
||||
|
||||
New client methods (Plan 02): this.client.listAllInvoices(): Promise<Pax8Invoice[]>;
|
||||
this.client.listAllInvoiceItems(invoiceId: string): Promise<Pax8InvoiceItem[]>.
|
||||
|
||||
Matcher (Plan 03): import { matchPax8Companies, Pax8CompanyMatchResult } from
|
||||
'./pax8-company-matcher'. matchPax8Companies(opts?: { limit?; dryRun? }).
|
||||
|
||||
pax8_orders columns (migration 091): id UUID, pax8_company_id UUID (stays NULL),
|
||||
order_date TIMESTAMPTZ, total NUMERIC(12,2), status TEXT, currency CHAR(3),
|
||||
raw_payload JSONB, synced_at, is_deleted, deleted_at.
|
||||
pax8_order_items columns (091 + 092): id UUID, order_id UUID NOT NULL (FK CASCADE),
|
||||
product_id UUID, quantity INTEGER, unit_price NUMERIC(12,2), line_total
|
||||
NUMERIC(12,2), currency CHAR(3), raw_payload, synced_at, is_deleted, deleted_at,
|
||||
+ pax8_company_id UUID, subscription_id UUID, item_type TEXT, sku TEXT,
|
||||
description TEXT, start_period TIMESTAMPTZ, end_period TIMESTAMPTZ, partner_cost
|
||||
NUMERIC(12,2), partner_cost_total NUMERIC(12,2).
|
||||
</interfaces>
|
||||
</context>
|
||||
|
||||
<tasks>
|
||||
|
||||
<task type="auto">
|
||||
<name>Task 1: Add syncOrders() (nested invoice → item upsert + tombstone)</name>
|
||||
<files>lib/services/pax8-sync-service.ts</files>
|
||||
<read_first>
|
||||
- lib/services/pax8-sync-service.ts (syncSubscriptions lines 175-253 for the upsert+tombstone shape; syncCompanies lines 113-173 for the single-table baseline; the import block lines 12-21; the per-method try/catch error shape lines 168-172)
|
||||
- lib/services/itglue-sync-service.ts (the syncModels() per-parent nested-fetch iteration referenced by 12-PATTERNS.md Pattern 1 — parent loop calling a child endpoint per id)
|
||||
- .planning/phases/12-orders-invoices-company-matching/12-02-SUMMARY.md (the CONFIRMED invoice-item cost field mapping — authoritative for the item upsert)
|
||||
- .planning/phases/12-orders-invoices-company-matching/12-RESEARCH.md (Pitfall 1 header companyId always NULL; Pitfall 2 field mapping; Pattern 1 nested pagination)
|
||||
</read_first>
|
||||
<action>
|
||||
Extend the type import block to add Pax8Invoice, Pax8InvoiceItem from
|
||||
'@/lib/types/pax8'. Add a private async syncOrders(): Promise<Pax8EntitySyncResult>
|
||||
modeled on syncSubscriptions.
|
||||
|
||||
Fetch headers: const invoices = await this.client.listAllInvoices(). Track
|
||||
seenOrderIds: string[] and seenItemIds: string[]; count ordersUpserted and
|
||||
itemsUpserted. For each invoice with an id: push id to seenOrderIds; upsert
|
||||
pax8_orders — INSERT (id, pax8_company_id, order_date, total, status, currency,
|
||||
raw_payload, synced_at, is_deleted, deleted_at) VALUES ($1, NULL, $2..., NOW(),
|
||||
false, NULL) ON CONFLICT (id) DO UPDATE SET the same fields + synced_at=NOW(),
|
||||
is_deleted=false, deleted_at=NULL. Map order_date←invoice.invoiceDate,
|
||||
total←invoice.total, status←invoice.status, currency←invoice.currencyCode ??
|
||||
'USD'. Keep pax8_company_id NULL (Pitfall 1 — comment why).
|
||||
|
||||
Then fetch that invoice's items: const items = await
|
||||
this.client.listAllInvoiceItems(invoice.id). For each item with an id: push id
|
||||
to seenItemIds; upsert pax8_order_items — INSERT (id, order_id, product_id,
|
||||
quantity, unit_price, line_total, currency, raw_payload, synced_at, is_deleted,
|
||||
deleted_at, pax8_company_id, subscription_id, item_type, sku, description,
|
||||
start_period, end_period, partner_cost, partner_cost_total) with order_id =
|
||||
invoice.id, and the CONFIRMED mapping from 12-02-SUMMARY.md: unit_price←price,
|
||||
line_total←amountDue, partner_cost←cost, partner_cost_total←costTotal,
|
||||
pax8_company_id←companyId, subscription_id←subscriptionId, item_type←type,
|
||||
sku←sku, description←description, start_period←startPeriod, end_period←endPeriod,
|
||||
product_id←productId, quantity←quantity, currency←currencyCode ?? 'USD',
|
||||
raw_payload←JSON.stringify(item). ON CONFLICT (id) DO UPDATE SET all the above
|
||||
+ synced_at=NOW(), is_deleted=false, deleted_at=NULL.
|
||||
|
||||
After all invoices processed, tombstone in child-then-parent order to respect
|
||||
the FK: first UPDATE pax8_order_items SET is_deleted=true, deleted_at=NOW()
|
||||
WHERE is_deleted=false AND id <> ALL($1::uuid[]) with seenItemIds; then the
|
||||
same on pax8_orders with seenOrderIds. Guard each with the seen.length === 0 ?
|
||||
0 : (...).rowCount ?? 0 pattern from syncSubscriptions. Sum tombstoned across
|
||||
both tables.
|
||||
|
||||
Return { entity: 'orders', success: true, upserted: ordersUpserted +
|
||||
itemsUpserted, tombstoned, durationMs: Date.now() - start }. Wrap in the
|
||||
standard try/catch returning a failed result with the error message
|
||||
('[Pax8Sync] Order sync failed:').
|
||||
</action>
|
||||
<acceptance_criteria>
|
||||
- syncOrders() upserts into both `pax8_orders` and `pax8_order_items`
|
||||
- The item upsert binds `pax8_company_id` from the item's `companyId` and sets `line_total` from `amountDue`, `partner_cost` from `cost`, `partner_cost_total` from `costTotal`, `unit_price` from `price` (per 12-02-SUMMARY.md)
|
||||
- The header insert leaves pax8_orders.pax8_company_id NULL (Pitfall 1)
|
||||
- Tombstone runs on pax8_order_items before pax8_orders, both using `id <> ALL($1::uuid[])`
|
||||
- Returns a Pax8EntitySyncResult with `entity: 'orders'`
|
||||
- `npx tsc --noEmit --pretty` passes
|
||||
</acceptance_criteria>
|
||||
<verify>
|
||||
<automated>npx tsc --noEmit --pretty 2>&1 | tail -5; grep -c "listAllInvoiceItems" lib/services/pax8-sync-service.ts</automated>
|
||||
</verify>
|
||||
<done>syncOrders performs the nested invoice→item sync with the confirmed cost mapping, per-company id on items, and child-then-parent tombstoning; type-check passes.</done>
|
||||
</task>
|
||||
|
||||
<task type="auto">
|
||||
<name>Task 2: Add syncCompanyMatches() and wire both steps into fullSync()</name>
|
||||
<files>lib/services/pax8-sync-service.ts</files>
|
||||
<read_first>
|
||||
- lib/services/pax8-sync-service.ts (fullSync lines 42-109 — the entities.push accumulation and rollup)
|
||||
- lib/services/pax8-company-matcher.ts (matchPax8Companies signature + Pax8CompanyMatchResult from Plan 03)
|
||||
- lib/types/pax8.ts (Pax8EntitySyncResult)
|
||||
</read_first>
|
||||
<action>
|
||||
Import { matchPax8Companies } from './pax8-company-matcher' (and the
|
||||
Pax8CompanyMatchResult type). Add a private async syncCompanyMatches():
|
||||
Promise<Pax8EntitySyncResult> that calls matchPax8Companies() (no dryRun),
|
||||
then maps its result into a Pax8EntitySyncResult: entity: 'company_matches',
|
||||
success: true, upserted: result.autoLinked, tombstoned: result.flaggedAmbiguous
|
||||
+ result.flaggedNoCandidate (repurposed here as the flagged-for-review count —
|
||||
document this in a comment), durationMs: result.durationMs. Wrap in try/catch
|
||||
returning a failed result ('[Pax8Sync] Company match failed:').
|
||||
|
||||
In fullSync(), after `entities.push(productsResult);` add: const ordersResult =
|
||||
await this.syncOrders(); entities.push(ordersResult); then const matchResult =
|
||||
await this.syncCompanyMatches(); entities.push(matchResult); Matching runs
|
||||
AFTER orders and companies so pax8_companies is fully populated first. The
|
||||
existing rollup (success = entities.every, totals = reduce) picks both up
|
||||
unchanged.
|
||||
</action>
|
||||
<acceptance_criteria>
|
||||
- fullSync() contains `entities.push(ordersResult)` and `entities.push(matchResult)` after the products step
|
||||
- syncCompanyMatches() calls `matchPax8Companies()` and returns a Pax8EntitySyncResult with `entity: 'company_matches'`
|
||||
- The matcher step runs after syncCompanies (companies populated before matching)
|
||||
- `npx tsc --noEmit --pretty` passes
|
||||
</acceptance_criteria>
|
||||
<verify>
|
||||
<automated>npx tsc --noEmit --pretty 2>&1 | tail -5; grep -c "matchPax8Companies\|entities.push" lib/services/pax8-sync-service.ts</automated>
|
||||
</verify>
|
||||
<done>Both new steps are wired into fullSync and roll up into the sync result and sync_history counts.</done>
|
||||
</task>
|
||||
|
||||
<task type="auto" tdd="true">
|
||||
<name>Task 3: Create lib/services/pax8-sync-service.test.ts</name>
|
||||
<files>lib/services/pax8-sync-service.test.ts</files>
|
||||
<behavior>
|
||||
- syncOrders (via fullSync or a directly-invoked instance with a mock client): a two-invoice fixture with items produces upserts into both pax8_orders and pax8_order_items; the item upsert binds companyId into the pax8_company_id parameter position
|
||||
- unseen headers/items are tombstoned via id <> ALL (the tombstone UPDATE runs)
|
||||
- syncCompanyMatches delegates to the mocked matchPax8Companies and returns entity 'company_matches'
|
||||
- fullSync's entities array includes an 'orders' and a 'company_matches' result
|
||||
</behavior>
|
||||
<read_first>
|
||||
- lib/services/pax8-client.test.ts (vi.fn / vi.mock discipline to mirror)
|
||||
- lib/services/pax8-sync-service.ts (the constructor accepts an optional client: `constructor(client?: Pax8Client)` — inject a mock client; the query shapes to assert)
|
||||
- .planning/phases/12-orders-invoices-company-matching/12-RESEARCH.md (Validation Architecture — Wave 0 note that no prior Pax8SyncService test exists; mock postgresClient.query)
|
||||
</read_first>
|
||||
<action>
|
||||
Create the service's first unit test. Mock '@/lib/services/postgres-client'
|
||||
(default: { query: vi.fn() }) and mock './pax8-company-matcher' so
|
||||
matchPax8Companies returns a canned Pax8CompanyMatchResult ({ scanned: 2,
|
||||
autoLinked: 1, flaggedAmbiguous: 1, flaggedNoCandidate: 0, durationMs: 1 }).
|
||||
Construct the service with an injected mock Pax8Client exposing
|
||||
listAllCompanies/listAllSubscriptions/listAllProducts (return []) and
|
||||
listAllInvoices (two headers) + listAllInvoiceItems (items per header, one with
|
||||
a companyId + subscriptionId + amountDue/cost/costTotal).
|
||||
|
||||
Assert: after fullSync (or a direct syncOrders call if you expose it via the
|
||||
instance), the mocked postgresClient.query received an INSERT INTO pax8_orders
|
||||
and an INSERT INTO pax8_order_items call; for the item insert, the params array
|
||||
includes the fixture's companyId value (pax8_company_id) and its amountDue value
|
||||
(line_total). Assert a tombstone UPDATE against pax8_order_items and one against
|
||||
pax8_orders were issued. Assert the returned Pax8SyncResult.entities contains an
|
||||
object with entity 'orders' and one with entity 'company_matches'. Keep DB-free
|
||||
(all mocked), environment node.
|
||||
</action>
|
||||
<acceptance_criteria>
|
||||
- Test asserts INSERT calls against both `pax8_orders` and `pax8_order_items`
|
||||
- Test asserts the item insert params include the fixture companyId (pax8_company_id) and amountDue (line_total)
|
||||
- Test asserts tombstone UPDATEs against both tables
|
||||
- Test asserts fullSync entities include `entity: 'orders'` and `entity: 'company_matches'`
|
||||
- `npx vitest run lib/services/pax8-sync-service.test.ts` passes
|
||||
</acceptance_criteria>
|
||||
<verify>
|
||||
<automated>npx vitest run lib/services/pax8-sync-service.test.ts</automated>
|
||||
</verify>
|
||||
<done>The sync service's first unit tests prove the nested order upsert (with per-company mapping), tombstoning, and matcher delegation.</done>
|
||||
</task>
|
||||
|
||||
</tasks>
|
||||
|
||||
<threat_model>
|
||||
## Trust Boundaries
|
||||
|
||||
| Boundary | Description |
|
||||
|----------|-------------|
|
||||
| PAX8 API → sync service | External invoice/item JSON persisted to Postgres |
|
||||
| sync writes → Postgres | Upserts/tombstones on billing-history tables |
|
||||
|
||||
## STRIDE Threat Register
|
||||
|
||||
| Threat ID | Category | Component | Disposition | Mitigation Plan |
|
||||
|-----------|----------|-----------|-------------|-----------------|
|
||||
| T-12-01 | Tampering | syncOrders upsert SQL | mitigate | All PAX8 field values bound as `$n` params via postgresClient.query (no interpolation), matching the existing syncSubscriptions convention |
|
||||
| T-12-07 | Denial of Service | nested invoice→item fetch | accept | ~94 headers × ~500-700 items (≈56k rows), well within the client's 1000/min handling and normal Postgres sizes (12-RESEARCH.md Assumption A3); full sync only, no unbounded growth path |
|
||||
| T-12-08 | Repudiation | sync_history rollup | mitigate | orders + company_matches counts fold into the existing sync_history record via fullSync's rollup, preserving the audit trail of what each run changed |
|
||||
| T-12-05 | Elevation of Privilege | client calls | mitigate | Only read (GET) client methods are called (listAll*); PAX8-08 read-only invariant preserved |
|
||||
| T-12-SC | Tampering | package installs | accept | No new packages |
|
||||
</threat_model>
|
||||
|
||||
<verification>
|
||||
- `npx vitest run lib/services/pax8-sync-service.test.ts` and `npx vitest run lib/services/pax8-client.test.ts lib/services/pax8-company-matcher.test.ts` green.
|
||||
- `npx tsc --noEmit --pretty` passes.
|
||||
- `npm test` (full suite) green before the wave merges.
|
||||
</verification>
|
||||
|
||||
<success_criteria>
|
||||
fullSync runs orders sync + company matching as first-class entity steps; invoice items land with per-company id, billing period, and dual-cost columns; unseen rows tombstone; matcher delegation reports through the sync result. Unit tests prove it.
|
||||
</success_criteria>
|
||||
|
||||
<output>
|
||||
Create `.planning/phases/12-orders-invoices-company-matching/12-04-SUMMARY.md` when done.
|
||||
</output>
|
||||
|
|
@ -0,0 +1,170 @@
|
|||
---
|
||||
phase: 12-orders-invoices-company-matching
|
||||
plan: 05
|
||||
type: execute
|
||||
wave: 4
|
||||
depends_on: ["12-04"]
|
||||
files_modified:
|
||||
- scripts/verify-pax8-orders-matching.ts
|
||||
autonomous: false
|
||||
requirements: [PAX8-06, PAX8-10, PAX8-11]
|
||||
|
||||
must_haves:
|
||||
truths:
|
||||
- "A real full sync populates pax8_order_items with per-company id and billing period rows (SC#1)"
|
||||
- "At least one pax8_companies row carries a confident auto-match (autotask_company_id + match_confidence >= 0.90) (SC#2)"
|
||||
- "No-match and ambiguous PAX8 companies have pax8_company_match_review rows (empty array for no-match) (SC#3)"
|
||||
- "Running the sync a second time does not change already-resolved matches (SC#4 idempotency)"
|
||||
artifacts:
|
||||
- path: "scripts/verify-pax8-orders-matching.ts"
|
||||
provides: "live full-sync + DB assertion harness for the phase's 4 success criteria"
|
||||
key_links:
|
||||
- from: "scripts/verify-pax8-orders-matching.ts"
|
||||
to: "pax8-sync-service fullSync"
|
||||
via: "invokes a real sync then queries the resulting rows"
|
||||
pattern: "fullSync|/api/pax8/sync"
|
||||
---
|
||||
|
||||
<objective>
|
||||
Prove the phase's four ROADMAP success criteria against real PAX8 data and the
|
||||
real dev Postgres — the truest verification, mirroring Phase 11's live-run
|
||||
verification plan (11-03). Unit tests (Plans 02-04) prove the logic with mocks;
|
||||
this plan proves the integration end-to-end and lets the developer eyeball the
|
||||
match distribution, since conservative billing-reconciliation matching (D-01) is
|
||||
a judgment the user explicitly cares about.
|
||||
|
||||
Purpose: End-to-end confidence that orders populate, auto-matches persist,
|
||||
flags are created (never silently guessed), and a human decision survives a
|
||||
re-sync.
|
||||
Output: a reusable verification script + a human confirmation of the results.
|
||||
</objective>
|
||||
|
||||
<execution_context>
|
||||
@$HOME/.claude/get-shit-done/workflows/execute-plan.md
|
||||
@$HOME/.claude/get-shit-done/templates/summary.md
|
||||
</execution_context>
|
||||
|
||||
<context>
|
||||
@.planning/phases/12-orders-invoices-company-matching/12-RESEARCH.md
|
||||
@.planning/phases/12-orders-invoices-company-matching/12-CONTEXT.md
|
||||
|
||||
<interfaces>
|
||||
- POST /api/pax8/sync (existing route) fires a fire-and-forget fullSync;
|
||||
GET /api/pax8/sync returns inProgress + counts + history. App runs on port 3100.
|
||||
- Alternatively invoke getPax8SyncService().fullSync('verify') directly from a
|
||||
tsx script (dotenv from ../.env.local), like scripts/verify-pax8-auth.ts.
|
||||
- DB access for assertions: docker exec pulse-postgres psql -U pulse_user -d pulse_autotask.
|
||||
- Migration 092 (Plan 01) must already be applied to the dev DB.
|
||||
</interfaces>
|
||||
</context>
|
||||
|
||||
<tasks>
|
||||
|
||||
<task type="auto">
|
||||
<name>Task 1: Write scripts/verify-pax8-orders-matching.ts (live full-sync + assertions)</name>
|
||||
<files>scripts/verify-pax8-orders-matching.ts</files>
|
||||
<read_first>
|
||||
- scripts/verify-pax8-auth.ts (dotenv + getPax8Client + secret-safe logging pattern; run via npx tsx)
|
||||
- lib/services/pax8-sync-service.ts (getPax8SyncService / fullSync from Plan 04)
|
||||
- .planning/phases/12-orders-invoices-company-matching/12-RESEARCH.md (Live-verified distribution — expected ~80 exact matches at 1.00, so auto-links should be substantial; ~56k order items total)
|
||||
</read_first>
|
||||
<action>
|
||||
Create scripts/verify-pax8-orders-matching.ts (npx tsx), dotenv from
|
||||
../.env.local, secret-safe logging only. It runs getPax8SyncService().fullSync
|
||||
('verify') once, awaiting completion, then queries the dev DB (via a direct
|
||||
postgresClient.query, since the script runs in-process) and prints a report:
|
||||
|
||||
- SC#1: count of pax8_order_items WHERE is_deleted=false AND pax8_company_id
|
||||
IS NOT NULL, and count with start_period IS NOT NULL. Both must be > 0.
|
||||
- SC#2: count of pax8_companies WHERE autotask_company_id IS NOT NULL AND
|
||||
match_confidence >= 0.90 AND match_method = 'pg_trgm'. Print a sample of 5
|
||||
(pax8 name, matched autotask company_name, score). Count must be > 0.
|
||||
- SC#3: count of pax8_company_match_review rows WHERE resolved_at IS NULL; of
|
||||
those, count with candidate_company_ids = '{}' (no-match, D-03) and count
|
||||
with array_length >= 1 (ambiguous/below-threshold). Report both.
|
||||
- SC#4: capture the (pax8_company_id, autotask_company_id, match_confidence)
|
||||
for auto-matched rows into a snapshot; then run fullSync('verify') a SECOND
|
||||
time; re-read the same rows and assert none of the previously-auto-matched
|
||||
rows changed AND that no row with a resolved review (simulate none exist yet
|
||||
— Phase 14 owns manual resolution) was mutated. Print PASS/FAIL for
|
||||
idempotency (auto-match set stable across two runs).
|
||||
|
||||
Print a final verdict block: SC#1..SC#4 each PASS/FAIL with the underlying
|
||||
numbers. Exit non-zero if any SC fails so the verify command surfaces it.
|
||||
Never log the client secret or token.
|
||||
</action>
|
||||
<acceptance_criteria>
|
||||
- `npx tsx scripts/verify-pax8-orders-matching.ts` runs a real sync and prints a verdict block with SC#1, SC#2, SC#3, SC#4 each marked PASS or FAIL and their counts
|
||||
- SC#1 count of pax8_order_items with non-null pax8_company_id is > 0
|
||||
- SC#2 count of pax8_companies with autotask_company_id set and match_confidence >= 0.90 is > 0
|
||||
- SC#3 reports both the empty-candidate (no-match) review count and the ambiguous review count
|
||||
- SC#4 shows the auto-match set is identical across two consecutive full syncs (idempotency PASS)
|
||||
- Script exits 0 only when all four criteria pass; no secret/token appears in output
|
||||
</acceptance_criteria>
|
||||
<verify>
|
||||
<automated>npx tsx scripts/verify-pax8-orders-matching.ts; echo "exit=$?"</automated>
|
||||
</verify>
|
||||
<done>A live full sync populates orders/items and match state; all four success criteria print PASS with real numbers; the auto-match set is stable across two runs.</done>
|
||||
</task>
|
||||
|
||||
<task type="checkpoint:human-verify" gate="blocking">
|
||||
<name>Task 2: Human review of the auto-match sample + success-criteria verdict</name>
|
||||
<action>PAUSE for the developer. This is a human-verify checkpoint: the developer reviews the SC#1..SC#4 verdict block and the sample of auto-matches produced by Task 1's script, confirming the conservative matches are correct (D-01) and the flagged queue is genuinely ambiguous/no-match. Do not auto-approve — resume only on the developer signal below.</action>
|
||||
<what-built>
|
||||
Historical PAX8 invoice line items are synced into pax8_order_items (with
|
||||
per-company id + billing period + dual cost), PAX8 companies are auto-linked to
|
||||
Autotask companies at similarity >= 0.90 (conservative, no near-ties), and
|
||||
no-match/ambiguous companies are flagged in pax8_company_match_review rather
|
||||
than silently guessed. The verification script ran two full syncs and confirmed
|
||||
idempotency.
|
||||
</what-built>
|
||||
<how-to-verify>
|
||||
1. Review the verdict block from `npx tsx scripts/verify-pax8-orders-matching.ts`
|
||||
(SC#1..SC#4 all PASS).
|
||||
2. Eyeball the SC#2 sample of 5 auto-matches: each PAX8 company name should
|
||||
clearly be the same real company as its matched Autotask company_name.
|
||||
Conservative intent (D-01): if any sampled auto-match looks wrong, that is a
|
||||
problem — report it (the threshold may need raising).
|
||||
3. Optionally spot-check the review queue:
|
||||
`docker exec pulse-postgres psql -U pulse_user -d pulse_autotask -c "SELECT p.name, r.candidate_company_ids, r.match_confidences FROM pax8_company_match_review r JOIN pax8_companies p ON p.id=r.pax8_company_id WHERE r.resolved_at IS NULL LIMIT 10"`
|
||||
— confirm flagged companies are genuinely ambiguous or genuinely have no
|
||||
clear Autotask match.
|
||||
4. Confirm a per-company cost query returns rows:
|
||||
`docker exec pulse-postgres psql -U pulse_user -d pulse_autotask -c "SELECT pax8_company_id, count(*), sum(line_total) FROM pax8_order_items WHERE is_deleted=false GROUP BY 1 ORDER BY 2 DESC LIMIT 5"`.
|
||||
</how-to-verify>
|
||||
<resume-signal>Type "approved" if the auto-matches look correct and all SC pass, or describe any wrong match / bad flag so the threshold or mapping can be adjusted.</resume-signal>
|
||||
</task>
|
||||
|
||||
</tasks>
|
||||
|
||||
<threat_model>
|
||||
## Trust Boundaries
|
||||
|
||||
| Boundary | Description |
|
||||
|----------|-------------|
|
||||
| live PAX8 API → dev Postgres | Real historical billing data written during verification |
|
||||
| verification output → developer | Match results reviewed by a human before the phase is accepted |
|
||||
|
||||
## STRIDE Threat Register
|
||||
|
||||
| Threat ID | Category | Component | Disposition | Mitigation Plan |
|
||||
|-----------|----------|-----------|-------------|-----------------|
|
||||
| T-12-02 | Tampering / Repudiation | auto-match correctness | mitigate | Human-verify checkpoint reviews a sample of auto-matches; conservative D-01 threshold means wrong auto-links should be near-zero and any spotted one blocks approval |
|
||||
| T-12-04 | Tampering | idempotency | mitigate | SC#4 double-run asserts the auto-match set is stable and (by design) resolved rows are never overwritten |
|
||||
| T-12-03 | Information Disclosure | verification script logging | mitigate | Secret-safe logging copied from verify-pax8-auth.ts; summary/counts only |
|
||||
| T-12-SC | Tampering | package installs | accept | No new packages |
|
||||
</threat_model>
|
||||
|
||||
<verification>
|
||||
- Verification script exits 0 with SC#1..SC#4 all PASS.
|
||||
- Human confirms the auto-match sample is correct.
|
||||
- Full test suite (`npm test`) still green.
|
||||
</verification>
|
||||
|
||||
<success_criteria>
|
||||
All four ROADMAP Phase 12 success criteria proven against live data; auto-match sample human-approved; idempotency confirmed across two syncs.
|
||||
</success_criteria>
|
||||
|
||||
<output>
|
||||
Create `.planning/phases/12-orders-invoices-company-matching/12-05-SUMMARY.md` when done. Record the SC counts and the human verdict.
|
||||
</output>
|
||||
Loading…
Add table
Add a link
Reference in a new issue