diff --git a/.planning/phases/12-orders-invoices-company-matching/12-VALIDATION.md b/.planning/phases/12-orders-invoices-company-matching/12-VALIDATION.md index 7b37ed7..44fa28e 100644 --- a/.planning/phases/12-orders-invoices-company-matching/12-VALIDATION.md +++ b/.planning/phases/12-orders-invoices-company-matching/12-VALIDATION.md @@ -1,10 +1,11 @@ --- phase: 12 slug: orders-invoices-company-matching -status: draft -nyquist_compliant: false -wave_0_complete: false +status: audited +nyquist_compliant: true +wave_0_complete: true created: 2026-07-11 +audited: 2026-07-11 --- # Phase 12 — Validation Strategy @@ -30,7 +31,24 @@ created: 2026-07-11 - **After every task commit:** Run `npx vitest run lib/services/pax8-*.test.ts` - **After every plan wave:** Run `npm test` (full suite) - **Before `/gsd:verify-work`:** Full suite must be green -- **Max feedback latency:** 30 seconds +- **Max feedback latency:** 30 seconds — applies to the **automated unit-test tier** (vitest) only. + +### Live-tier exception (intentional) + +Two verify scripts in this phase are **live / network-bound** and are an explicit, +intentional exception to the 30-second automated-test latency budget above. They +call the real PAX8 API — and one runs `fullSync()` against ~56k rows twice — so +they run well past 30 seconds by design: + +| Script | Plan / Task | Why it exceeds 30s | Gate | +|--------|-------------|--------------------|------| +| `scripts/verify-pax8-invoice-items.ts` | 12-02 Task 3 | Live PAX8 `/invoices/{id}/items` spot-check against real invoice data | Live-tier, developer-run | +| `scripts/verify-pax8-orders-matching.ts` | 12-05 Task 1 | Runs `fullSync('verify')` against the real PAX8 API **twice** (~56k rows each) to assert idempotency | Gated behind the Plan 05 `checkpoint:human-verify` | + +These scripts do **not** run in the per-commit / per-wave vitest sampling loop. +They are run deliberately during live verification (Plan 05 is gated behind a +human-verify checkpoint), so the 30s contract for the automated tier is not +silently violated by their runtime. --- @@ -38,42 +56,59 @@ created: 2026-07-11 | Task ID | Plan | Wave | Requirement | Threat Ref | Secure Behavior | Test Type | Automated Command | File Exists | Status | |---------|------|------|-------------|------------|-----------------|-----------|-------------------|-------------|--------| -| TBD | TBD | TBD | PAX8-06 | V5 | `listAllInvoices()` concatenates pages in order | unit | `npx vitest run lib/services/pax8-client.test.ts -t "listAllInvoices"` | ❌ W0 | ⬜ pending | -| TBD | TBD | TBD | PAX8-06 | V5 | `listAllInvoiceItems(invoiceId)` calls correct nested path and concatenates pages | unit | `npx vitest run lib/services/pax8-client.test.ts -t "listAllInvoiceItems"` | ❌ W0 | ⬜ pending | -| TBD | TBD | TBD | PAX8-06 | — | Sync upserts orders/items and tombstones missing rows | unit | `npx vitest run lib/services/pax8-sync-service.test.ts` | ❌ W0 | ⬜ pending | -| TBD | TBD | TBD | PAX8-10 | T-12-01 | Auto-link fires only when score ≥ 0.90 and no near-tie | unit | `npx vitest run lib/services/pax8-company-matcher.test.ts -t "auto-link"` | ❌ W0 | ⬜ pending | -| TBD | TBD | TBD | PAX8-10 | T-12-01 | Below-threshold or tied candidates produce a review row, not an auto-link | unit | `npx vitest run lib/services/pax8-company-matcher.test.ts -t "review"` | ❌ W0 | ⬜ pending | -| TBD | TBD | TBD | PAX8-11 | — | Zero-candidate case still creates a review row with empty `candidate_company_ids` | unit | `npx vitest run lib/services/pax8-company-matcher.test.ts -t "empty candidates"` | ❌ W0 | ⬜ pending | -| TBD | TBD | TBD | SC#4 | T-12-02 | Re-running matcher never overwrites a `resolved_at IS NOT NULL` row | unit | `npx vitest run lib/services/pax8-company-matcher.test.ts -t "idempotent"` | ❌ W0 | ⬜ pending | +| 12-02 T2 | 12-02 | 2 | PAX8-06 | T-12-05 | `listAllInvoices()` concatenates pages in order | unit | `npx vitest run lib/services/pax8-client.test.ts -t "listAllInvoices"` | ✅ | ✅ green | +| 12-02 T2 | 12-02 | 2 | PAX8-06 | T-12-05 | `listAllInvoiceItems(invoiceId)` calls correct nested path and concatenates pages | unit | `npx vitest run lib/services/pax8-client.test.ts -t "listAllInvoiceItems"` | ✅ | ✅ green | +| 12-04 T3 | 12-04 | 3 | PAX8-06 | T-12-01, T-12-08 | Sync upserts orders/items (binding companyId->pax8_company_id, amountDue->line_total) and tombstones missing items before orders | unit | `npx vitest run lib/services/pax8-sync-service.test.ts` | ✅ | ✅ green | +| 12-03 T2 | 12-03 | 2 | PAX8-10 | T-12-02 | Auto-link fires only when score ≥ 0.90 and no near-tie | unit | `npx vitest run lib/services/pax8-company-matcher.test.ts -t "auto-link"` | ✅ | ✅ green | +| 12-03 T2 | 12-03 | 2 | PAX8-10 | T-12-02 | Below-threshold or tied candidates produce a review row, not an auto-link | unit | `npx vitest run lib/services/pax8-company-matcher.test.ts -t "review"` | ✅ | ✅ green | +| 12-03 T2 | 12-03 | 2 | PAX8-11 | — | Zero-candidate case still creates a review row with empty `candidate_company_ids` | unit | `npx vitest run lib/services/pax8-company-matcher.test.ts -t "empty candidates"` | ✅ | ✅ green | +| 12-03 T2 | 12-03 | 2 | SC#4 | T-12-04 | Re-running matcher never overwrites a `resolved_at IS NOT NULL` row (unit-level guard) | unit | `npx vitest run lib/services/pax8-company-matcher.test.ts -t "idempotent"` | ✅ | ✅ green | +| 12-05 T1 | 12-05 | 4 | SC#4 | T-12-04 | Two consecutive live `fullSync()` runs produce an identical 80-row auto-match set | live/integration | `npx tsx scripts/verify-pax8-orders-matching.ts` | ✅ | ✅ green (live-verified 2026-07-11, human-approved) | *Status: ⬜ pending · ✅ green · ❌ red · ⚠️ flaky* +*All 23 pax8-scoped vitest tests confirmed passing as of this audit (`npx vitest run lib/services/pax8-client.test.ts lib/services/pax8-company-matcher.test.ts lib/services/pax8-sync-service.test.ts` → 3 files, 23 tests). Live-tier script re-run post-code-review-fixes (2026-07-11 07:32 UTC) reconfirmed all 4 ROADMAP success criteria PASS.* + --- ## Wave 0 Requirements -- [ ] `lib/services/pax8-company-matcher.test.ts` — covers PAX8-10, PAX8-11, SC#4. `pg_trgm`'s `similarity()` runs in Postgres — mock `postgresClient.query` (following `pax8-client.test.ts`'s `vi.fn` canned-row convention) to keep this DB-free and consistent with the rest of the suite. -- [ ] Extend `lib/services/pax8-client.test.ts` — add `listAllInvoices`/`listAllInvoiceItems` cases following the existing `makeMultiPageFetchMock` helper. -- [ ] Confirm whether `lib/services/pax8-sync-service.test.ts` exists yet (research did not find one — Phase 11 shipped with a live-verification script instead per `11-03-SUMMARY.md`). Planner decides whether Phase 12 introduces the first unit tests for `Pax8SyncService` or continues the live-verification-script pattern. +- [x] `lib/services/pax8-company-matcher.test.ts` — covers PAX8-10, PAX8-11, SC#4 (6/6 tests). `pg_trgm`'s `similarity()` mocked via `postgresClient.query` per convention. +- [x] Extended `lib/services/pax8-client.test.ts` — added `listAllInvoices`/`listAllInvoiceItems` cases (3 new tests, 12/12 total). +- [x] `lib/services/pax8-sync-service.test.ts` created — Phase 12 introduces the first unit tests for `Pax8SyncService` (5 tests), in addition to continuing the live-verification-script pattern for full-history proof (Plan 05). --- ## Manual-Only Verifications +> The two scripts below are the phase's **live-tier exception** to the 30s +> automated-latency budget (see *Live-tier exception* under Sampling Rate). They +> are developer-run / human-verify-gated, not part of the vitest sampling loop. + | Behavior | Requirement | Why Manual | Test Instructions | |----------|-------------|------------|-------------------| -| Full sync against real PAX8 account produces plausible order/item counts and match rates | PAX8-06, PAX8-10, PAX8-11 | Live external API + live production-like company data; unit tests mock both | Run full sync against dev environment, spot-check `pax8_orders`/`pax8_order_items` row counts (~94 invoices, ~500-700 items each) and `pax8_company_match_review` queue size against expectations in RESEARCH.md | -| Exact `total`/`subTotal`/`amountDue` semantics for prorated/credited invoice items | PAX8-06 | Open question in RESEARCH.md not fully closed by static research — needs a live-API spot-check | Query a known prorated invoice via `/invoices/{id}/items` and confirm field mapping before finalizing the item-persistence logic | +| Full sync against real PAX8 account produces plausible order/item counts and match rates | PAX8-06, PAX8-10, PAX8-11 | Live external API + live production-like company data; unit tests mock both. Runtime far exceeds 30s (runs `fullSync()` twice over ~56k rows). | Run `scripts/verify-pax8-orders-matching.ts` (Plan 05 Task 1) against the dev environment; spot-check `pax8_orders`/`pax8_order_items` row counts (~94 invoices, ~500-700 items each) and `pax8_company_match_review` queue size against expectations in RESEARCH.md | +| Exact `total`/`subTotal`/`amountDue` semantics for prorated/credited invoice items | PAX8-06 | Open question in RESEARCH.md not fully closed by static research — needs a live-API spot-check; live network runtime exceeds 30s | Run `scripts/verify-pax8-invoice-items.ts` (Plan 02 Task 3) to query a known invoice via `/invoices/{id}/items` and confirm the per-item-type field mapping (CONFIRM vs DIVERGENCE) before Plan 04 finalizes item-persistence logic | --- ## Validation Sign-Off -- [ ] All tasks have `` verify or Wave 0 dependencies -- [ ] Sampling continuity: no 3 consecutive tasks without automated verify -- [ ] Wave 0 covers all MISSING references -- [ ] No watch-mode flags -- [ ] Feedback latency < 30s -- [ ] `nyquist_compliant: true` set in frontmatter +- [x] All tasks have `` verify or Wave 0 dependencies — confirmed across all 5 plans' PLAN.md task blocks +- [x] Sampling continuity: no 3 consecutive tasks without automated verify +- [x] Wave 0 covers all MISSING references — all 3 Wave 0 items complete, no MISSING rows remain in the Per-Task Verification Map +- [x] No watch-mode flags +- [x] Feedback latency < 30s for the automated (vitest) tier; two live-tier scripts are a documented exception +- [x] `nyquist_compliant: true` set in frontmatter -**Approval:** pending +**Approval:** approved 2026-07-11 — 7/7 requirement rows COVERED, 0 gaps. All pax8-scoped vitest suites green (23/23); live-tier scripts (`verify-pax8-invoice-items.ts`, `verify-pax8-orders-matching.ts`) executed and human-approved during Plans 02/05, and re-confirmed passing post-code-review-fixes. + +## Validation Audit 2026-07-11 + +| Metric | Count | +|--------|-------| +| Gaps found | 0 | +| Resolved | 0 | +| Escalated | 0 | + +All 7 requirement rows from the pre-execution draft matched to real, passing tests once Plans 01-05 executed. No gap-filling agent was needed — Discovery + Cross-Reference (Step 2) found full coverage on first pass.