diff --git a/.planning/ROADMAP.md b/.planning/ROADMAP.md index 7d9e8fe..79a5ea8 100644 --- a/.planning/ROADMAP.md +++ b/.planning/ROADMAP.md @@ -336,7 +336,7 @@ Phases execute in numeric order. v1.0 (Phases 1-9.1) shipped 2026-07-10. v2.0 ph | 9.1. ntfy Backend Fix | v1.0 | 1/1 | Complete | 2026-07-10 | | 10. PAX8 Client & Auth Foundation | v2.0 | 3/3 | Complete | 2026-07-10 | | 11. Company, Catalog & Subscription Sync | v2.0 | 3/3 | Complete | 2026-07-11 | -| 12. Orders/Invoices & Company Matching | v2.0 | 5/5 | Complete | 2026-07-11 | +| 12. Orders/Invoices & Company Matching | v2.0 | 5/5 | Complete | 2026-07-11 | | 13. Scheduler & Admin Toggle | v2.0 | 0/TBD | Not started | - | | 14. /pax8 UI Surface | v2.0 | 0/TBD | Not started | - | diff --git a/.planning/STATE.md b/.planning/STATE.md index bea9058..1d2a903 100644 --- a/.planning/STATE.md +++ b/.planning/STATE.md @@ -2,15 +2,15 @@ gsd_state_version: 1.0 milestone: v2.0 milestone_name: PAX8 Integration -status: executing -stopped_at: Phase 12 context gathered -last_updated: "2026-07-11T02:38:03.649Z" +status: ready_to_plan +stopped_at: Phase 12 complete (5/5) — ready to discuss Phase 13 +last_updated: 2026-07-11T11:11:56.891Z last_activity: 2026-07-11 -- Phase 12 execution started progress: total_phases: 5 completed_phases: 2 total_plans: 11 - completed_plans: 6 + completed_plans: 11 percent: 40 --- @@ -21,14 +21,14 @@ progress: See: .planning/PROJECT.md (updated 2026-07-10) **Core value:** Managers can see PAX8 subscription costs and seat counts alongside existing Autotask company data in Pulse, without manually cross-referencing PAX8's own portal. -**Current focus:** Phase 12 — orders-invoices-company-matching +**Current focus:** Phase 13 — scheduler & admin toggle ## Current Position -Phase: 12 (orders-invoices-company-matching) — EXECUTING -Plan: 1 of 5 -Status: Executing Phase 12 -Last activity: 2026-07-11 -- Phase 12 execution started +Phase: 13 +Plan: Not started +Status: Ready to plan +Last activity: 2026-07-11 Progress: [░░░░░░░░░░] 0% @@ -36,7 +36,7 @@ Progress: [░░░░░░░░░░] 0% **Velocity:** -- Total plans completed: 36 (all from v1.0 Mobile Shell Redesign) +- Total plans completed: 41 (all from v1.0 Mobile Shell Redesign) - Average duration: — - Total execution time: 0.0 hours (v2.0) @@ -57,7 +57,7 @@ Progress: [░░░░░░░░░░] 0% | 09.1 | 1 | - | - | | 10 | 3 | - | - | | 11 | 3 | - | - | -| 12 | TBD | - | - | +| 12 | 5 | - | - | | 13 | TBD | - | - | | 14 | TBD | - | - | diff --git a/.planning/phases/12-orders-invoices-company-matching/12-VERIFICATION.md b/.planning/phases/12-orders-invoices-company-matching/12-VERIFICATION.md new file mode 100644 index 0000000..71802fa --- /dev/null +++ b/.planning/phases/12-orders-invoices-company-matching/12-VERIFICATION.md @@ -0,0 +1,117 @@ +--- +phase: 12-orders-invoices-company-matching +verified: 2026-07-11T11:09:08Z +status: passed +score: 12/12 must-haves verified +overrides_applied: 0 +--- + +# Phase 12: Orders/Invoices & Company Matching Verification Report + +**Phase Goal:** Historical PAX8 invoice/order-item costs are stored per company with billing periods, and PAX8 companies are automatically matched to Autotask companies via fuzzy name matching (pg_trgm), with no-match/ambiguous cases flagged for manual review instead of silently guessed. +**Verified:** 2026-07-11T11:09:08Z +**Status:** passed +**Re-verification:** No — initial verification + +## Goal Achievement + +### Observable Truths + +| # | Truth | Status | Evidence | +|---|-------|--------|----------| +| 1 | (ROADMAP SC#1) Running the sync populates an orders/invoices table with historical line items, not just current-state seats | ✓ VERIFIED | Live dev DB: `pax8_order_items` has 29,311 non-deleted rows, 29,295 with `pax8_company_id` set, 29,311 with `start_period` set (queried directly against `pulse-postgres`) | +| 2 | (ROADMAP SC#2) Each PAX8 company is automatically matched to an Autotask company by fuzzy similarity when a confident match exists, and the match is persisted | ✓ VERIFIED | Live dev DB: 80 `pax8_companies` rows have `autotask_company_id IS NOT NULL AND match_confidence >= 0.90 AND match_method='pg_trgm'` | +| 3 | (ROADMAP SC#3) A PAX8 company with no match or multiple similarly-scored candidates is flagged/needs-review instead of auto-assigned | ✓ VERIFIED | Live dev DB: 38 unresolved `pax8_company_match_review` rows — 16 with empty `candidate_company_ids` (no-match, D-03) and 22 with ≥1 candidate (ambiguous/below-threshold, D-02/D-04) | +| 4 | (ROADMAP SC#4) Re-running the sync does not overwrite an already-resolved/manually-confirmed match | ✓ VERIFIED | `applyLink()` UPDATE guarded by `match_method IS DISTINCT FROM 'manual' AND NOT EXISTS (... resolved_at IS NOT NULL ...)`; eligibility SELECT in `matchPax8Companies()` excludes `match_method='manual'` and human-resolved review rows; unit test `pax8-company-matcher.test.ts` asserts the guard text; live verification (12-05-SUMMARY.md) ran two consecutive full syncs and confirmed an identical 80-row auto-match snapshot | +| 5 | pg_trgm extension enabled in dev Postgres | ✓ VERIFIED | `SELECT count(*) FROM pg_extension WHERE extname='pg_trgm'` → 1 | +| 6 | pax8_order_items has per-company id, billing period, and dual-cost columns | ✓ VERIFIED | `information_schema.columns` query confirms all 9 columns present (`pax8_company_id, subscription_id, item_type, sku, description, start_period, end_period, partner_cost, partner_cost_total`); migrations/093 committed and applied | +| 7 | pax8_companies has auto-match columns | ✓ VERIFIED | `information_schema.columns` confirms all 4 columns present (`autotask_company_id, match_confidence, match_method, matched_at`) | +| 8 | TypeScript exposes Pax8Invoice/Pax8InvoiceItem types matching the live PAX8 field shape | ✓ VERIFIED | `lib/types/pax8.ts` defines both interfaces with the full live-verified field set; stale `Pax8Order`/`Pax8OrderItem` stubs removed (`grep -c "Pax8Order"` → 0) | +| 9 | Pax8Client can page invoice headers and per-invoice line items, GET-only | ✓ VERIFIED | `lib/services/pax8-client.ts` has `listAllInvoices()` (line 138) and `listAllInvoiceItems(invoiceId)` (line 149), both via `paginateAll`; no `/orders` call added; `pax8-client.test.ts` passes (includes GET-only assertions) | +| 10 | The invoice-item cost mapping (unit_price/line_total/partner_cost/partner_cost_total) is confirmed against real data | ✓ VERIFIED | `scripts/verify-pax8-invoice-items.ts` exists and was run live; 12-02-SUMMARY.md records CONFIRM verdicts for all 3 observed item types (subscription/prorate/one-time); `resolveCostColumns()` in `pax8-sync-service.ts` implements the confirmed default mapping | +| 11 | Matcher decides auto-link vs. review correctly (D-01..D-05), parameterized, never string-interpolated | ✓ VERIFIED | `lib/services/pax8-company-matcher.ts` implements `findCandidates`/`decide`/`applyLink`/`recordConflict`; `similarity($1, company_name)` bound param (2 occurrences, `grep -F`); `grep -c "company_name +"` → 0; 6/6 unit tests pass covering auto-link, below-threshold review, near-tie review, empty-candidate review, idempotency, dryRun | +| 12 | fullSync() runs syncOrders + syncCompanyMatches as entity steps; tombstoning respects the items→orders FK | ✓ VERIFIED | `pax8-sync-service.ts`: `entities.push(ordersResult)` and `entities.push(matchResult)` after products step; `syncOrders()` tombstones `pax8_order_items` before `pax8_orders` via `id <> ALL($1::uuid[])`; `pax8-sync-service.test.ts` (part of 23/23 passing PAX8 suite) asserts this | + +**Score:** 12/12 truths verified + +### Required Artifacts + +| Artifact | Expected | Status | Details | +|----------|----------|--------|---------| +| `migrations/093_pax8_orders_company_matching.sql` | pg_trgm + additive columns | ✓ VERIFIED | Applied to dev DB (1/9/4 column-count check passes); additive-only, no edits to migration 091 | +| `migrations/094_pax8_order_items_quantity_numeric.sql` | (unplanned, auto-fixed during live verification) widen `quantity` to NUMERIC | ✓ VERIFIED | Applied; `information_schema.columns` confirms `numeric(14,4)`; documented in 12-05-SUMMARY.md as a necessary bug fix for SC#1's true full-history coverage | +| `lib/types/pax8.ts` | Pax8Invoice/Pax8InvoiceItem types | ✓ VERIFIED | Both interfaces present with full live-verified field set | +| `lib/services/pax8-client.ts` | listAllInvoices + listAllInvoiceItems | ✓ VERIFIED | Both methods present, GET-only, reuse `paginateAll` | +| `lib/services/pax8-client.test.ts` | pagination + GET-only tests | ✓ VERIFIED | Part of 23/23 passing PAX8 test run | +| `scripts/verify-pax8-invoice-items.ts` | live field-mapping spot-check | ✓ VERIFIED | Exists, referenced/run per 12-02-SUMMARY.md | +| `lib/services/pax8-company-matcher.ts` | matchPax8Companies + helpers | ✓ VERIFIED | 253 lines, exports `matchPax8Companies`, `AUTO_LINK_THRESHOLD`, `TIE_MARGIN`, `Pax8CompanyMatchResult`; well above `min_lines: 120` | +| `lib/services/pax8-company-matcher.test.ts` | decision-branch + idempotency tests | ✓ VERIFIED | 6/6 tests passing | +| `lib/services/pax8-sync-service.ts` | syncOrders + syncCompanyMatches wired into fullSync | ✓ VERIFIED | Both private methods present; wired into `fullSync()` | +| `lib/services/pax8-sync-service.test.ts` | upsert/tombstone/matcher-delegation tests | ✓ VERIFIED | Part of 23/23 passing PAX8 test run | +| `scripts/verify-pax8-orders-matching.ts` | live full-sync + SC#1-4 assertion harness | ✓ VERIFIED | Exists, implements all 4 SC checks + idempotency snapshot comparison, exits non-zero on failure | + +### Key Link Verification + +| From | To | Via | Status | Details | +|------|-----|-----|--------|---------| +| migrations/093 | pax8_order_items | `ALTER TABLE ADD COLUMN IF NOT EXISTS` | ✓ WIRED | 9 columns confirmed present in live DB | +| migrations/093 | pax8_companies | `ALTER TABLE ADD COLUMN IF NOT EXISTS` | ✓ WIRED | 4 columns confirmed present in live DB | +| pax8-client.ts | paginateAll | reused private helper | ✓ WIRED | Both new methods call `this.paginateAll<...>` | +| pax8-company-matcher.ts | companies | `similarity($1, company_name)` with `is_active` filter | ✓ WIRED | Confirmed via source read + grep; live DB shows 80 real auto-matches produced by this exact query | +| pax8-company-matcher.ts | pax8_companies | guarded UPDATE | ✓ WIRED | `applyLink`/`recordConflict` UPDATE statements confirmed in source; live DB reflects their effect | +| pax8-company-matcher.ts | pax8_company_match_review | upsert on open-review partial unique index | ✓ WIRED | `ON CONFLICT (pax8_company_id) WHERE resolved_at IS NULL` confirmed in source; live DB has 38 rows produced by this path | +| pax8-sync-service.ts | pax8-client listAllInvoices/listAllInvoiceItems | nested per-invoice fetch loop | ✓ WIRED | `syncOrders()` calls both, confirmed in source and via live sync producing 29,311 items | +| pax8-sync-service.ts | pax8-company-matcher matchPax8Companies | syncCompanyMatches wrapper | ✓ WIRED | Confirmed in source; live sync produced the matched rows | +| pax8-sync-service.ts | fullSync entities array | `entities.push(ordersResult)` / `entities.push(matchResult)` | ✓ WIRED | Confirmed in source at lines 98-104 | + +### Data-Flow Trace (Level 4) + +| Artifact | Data Variable | Source | Produces Real Data | Status | +|----------|---------------|--------|---------------------|--------| +| `pax8_order_items` rows | invoice line items | `Pax8Client.listAllInvoiceItems()` → real PAX8 API → `syncOrders()` upsert | Yes — 29,311 live rows in dev Postgres, not a stub/empty return | ✓ FLOWING | +| `pax8_companies.autotask_company_id` etc. | matcher decision | `matchPax8Companies()` → real `companies` table via `similarity()` → `applyLink()`/`recordConflict()` | Yes — 80 auto-matched + 38 flagged rows in dev Postgres | ✓ FLOWING | + +### Behavioral Spot-Checks + +| Behavior | Command | Result | Status | +|----------|---------|--------|--------| +| pg_trgm + schema columns present | `docker exec pulse-postgres psql ... SELECT ...` | `1/9/4` | ✓ PASS | +| pax8_order_items populated with company id + period | `SELECT count(*) ... WHERE pax8_company_id IS NOT NULL` / `start_period IS NOT NULL` | 29,295 / 29,311 | ✓ PASS | +| Confident auto-matches exist | `SELECT count(*) FROM pax8_companies WHERE autotask_company_id IS NOT NULL AND match_confidence >= 0.90 AND match_method='pg_trgm'` | 80 | ✓ PASS | +| No-match/ambiguous flagged | `SELECT count(*) FROM pax8_company_match_review WHERE resolved_at IS NULL` (split by empty vs non-empty candidate array) | 38 total (16 no-match, 22 ambiguous) | ✓ PASS | +| PAX8-scoped test suites | `npx vitest run lib/services/pax8-client.test.ts lib/services/pax8-company-matcher.test.ts lib/services/pax8-sync-service.test.ts` | 3 files, 23/23 passing | ✓ PASS | +| Type-check | `npx tsc --noEmit --pretty` | Only 2 pre-existing errors in an unrelated untracked script (`scripts/diagnose-ticket-varchar-overflow.ts`, not part of Phase 12) | ✓ PASS | +| Full test suite | `npm test` | 214/216 passing; 2 pre-existing failures in `lib/services/analyzer/itglue-search.test.ts`, confirmed unrelated (file untouched by Phase 12, last modified in commit `a0a6e7f`) | ✓ PASS (non-blocking, unrelated) | + +### Probe Execution + +No `scripts/*/tests/probe-*.sh`-style probes declared for this phase. Plan 05's `scripts/verify-pax8-orders-matching.ts` functions as the phase's live-data probe; it was not re-executed during this verification (would require a ~9-minute live PAX8 API sync per 12-05-SUMMARY.md) because the dev database already reflects its documented output exactly (29,295/29,311/80/38/16 all match the SUMMARY's recorded numbers), confirmed independently via direct DB queries in this verification pass. Re-running it would not add information beyond what direct DB inspection already confirmed, and risks an unnecessary long-running external API call. + +### Requirements Coverage + +| Requirement | Source Plan | Description | Status | Evidence | +|-------------|-------------|-------------|--------|----------| +| PAX8-06 | 12-01, 12-02, 12-04, 12-05 | Pulse syncs PAX8 orders/invoices (historical line items) into Postgres | ✓ SATISFIED | Live DB has 29,311 order items with per-company id, billing period, dual cost | +| PAX8-10 | 12-01, 12-03, 12-04, 12-05 | PAX8 companies automatically matched to Autotask companies by fuzzy name similarity at sync time | ✓ SATISFIED | Live DB has 80 confident (`>=0.90`) `pg_trgm` auto-matches | +| PAX8-11 | 12-01, 12-03, 12-04, 12-05 | Unmatched/ambiguous company matches flagged rather than silently guessed | ✓ SATISFIED | Live DB has 38 unresolved review rows (16 no-match, 22 ambiguous) | + +No orphaned requirements — REQUIREMENTS.md maps only PAX8-06/10/11 to Phase 12, and all three appear in every plan's `requirements` frontmatter. REQUIREMENTS.md already marks all three `Complete`. + +### Anti-Patterns Found + +None. Scanned all files created/modified by this phase's 5 plans (migrations 093/094, `lib/types/pax8.ts`, `lib/services/pax8-client.ts`(+test), `lib/services/pax8-company-matcher.ts`(+test), `lib/services/pax8-sync-service.ts`(+test), `scripts/verify-pax8-invoice-items.ts`, `scripts/verify-pax8-orders-matching.ts`) for `TBD`/`FIXME`/`XXX`/`TODO`/`HACK`/`PLACEHOLDER`/`placeholder`/`not yet implemented` — zero matches. + +The `resolveCostColumns()` single-default-branch "switch" was checked as a potential stub pattern (a switch with only a `default` case can look like an unfinished feature), but it is not a stub: it implements a fully functional, live-confirmed cost mapping (12-02-SUMMARY.md recorded CONFIRM for every observed item type), deliberately kept as a named seam per the plan's own acceptance criteria for the "all CONFIRM" case. Live DB data confirms the mapping produces real, non-null cost values. + +### Human Verification Required + +None outstanding. Plan 12-05's `checkpoint:human-verify` (Task 2) was executed during the phase and the developer responded "approved" (recorded in 12-05-SUMMARY.md: SC#1-4 verdicts reviewed, auto-match sample spot-checked, no wrong matches found). No other plan in this phase deferred a `` block to end-of-phase. + +### Gaps Summary + +No gaps. All four ROADMAP Phase 12 success criteria are independently verified against the live dev database (not just SUMMARY claims): historical order items are populated with per-company id/billing period/cost (29,311 rows), confident auto-matches are persisted (80 rows at `>=0.90`), no-match/ambiguous cases are flagged rather than guessed (38 review rows split 16/22), and the idempotency guard against overwriting resolved matches is present in code, unit-tested, and was live-confirmed stable across two consecutive full syncs per 12-05-SUMMARY.md. All artifacts exist, are substantive (well above stub thresholds), are wired into `fullSync()`, and produce real data (Level 4 data-flow trace confirmed via direct Postgres queries, not SUMMARY narrative). The one operational bug discovered mid-phase (integer `quantity` overflow on fractional PAX8 usage quantities) was caught, fixed via an additive migration (094), and re-verified before the phase closed — exactly the kind of self-correction this verification process expects to see evidence of, not just claimed. + +--- + +*Verified: 2026-07-11T11:09:08Z* +*Verifier: Claude (gsd-verifier)*