docs(12): research phase domain
This commit is contained in:
parent
85687bf893
commit
b99f84ffa0
1 changed files with 771 additions and 0 deletions
|
|
@ -0,0 +1,771 @@
|
|||
# Phase 12: Orders/Invoices & Company Matching - Research
|
||||
|
||||
**Researched:** 2026-07-11
|
||||
**Domain:** PAX8 invoice/line-item historical sync + Postgres fuzzy-name entity matching
|
||||
**Confidence:** HIGH (both major open questions were resolved with **live, authenticated calls** against the real PAX8 API and the real dev Postgres database — not documentation guesses)
|
||||
|
||||
## Summary
|
||||
|
||||
Two independent problems, both now de-risked with hard evidence instead of assumptions.
|
||||
|
||||
**Orders/invoices:** PAX8's `/invoices` resource is the *partner's own consolidated
|
||||
monthly bill* — one header row per billing period, **not** one per end-customer.
|
||||
`companyId` is `null` on every invoice header in this account's live data. The
|
||||
per-customer cost data lives one level down, on `/invoices/{id}/items`, where
|
||||
`companyId` is populated on **every** item. This is a genuine mismatch against
|
||||
`migrations/091_pax8_tables.sql`: `pax8_orders.pax8_company_id` will always be
|
||||
`NULL`, and `pax8_order_items` has **no `company_id` column at all** today — a
|
||||
required new migration, not optional. Full history is small and safe: 94
|
||||
invoices total since 2019-05-01, ~500-700 items each (≈50-65k rows total),
|
||||
zero pagination-limit or rate-limit risk given the client's existing 1000/min
|
||||
handling.
|
||||
|
||||
**Company matching:** Postgres `pg_trgm` (not yet enabled — only `uuid-ossp`
|
||||
and `pgcrypto` are) is the right tool: no new npm dependency, matches existing
|
||||
extension-based conventions, and a live test against the real 118
|
||||
`pax8_companies` rows vs. 242 active `companies` rows shows a **clean
|
||||
separation** — every genuine match scored 1.00 (case/whitespace-only
|
||||
differences already normalized away by `similarity()`, which is
|
||||
case-insensitive), and the single highest-scoring *non*-match in the entire
|
||||
dataset was 0.70 ("Thoroughbred Construction Company" vs "...Construction
|
||||
Group" — arguably a real non-match). Recommend an auto-link threshold of
|
||||
**0.90** (`similarity() >= 0.90`), well clear of every observed false-positive
|
||||
risk, satisfying D-01's "conservative" mandate with room to spare.
|
||||
|
||||
**Primary recommendation:** Add a new migration (092) that (a) enables
|
||||
`pg_trgm`, (b) adds `company_id`, `subscription_id`, `type`, `start_period`,
|
||||
`end_period` and dual-cost columns to `pax8_order_items`, and (c) adds
|
||||
`autotask_company_id` / `match_confidence` / `matched_at` / `match_method`
|
||||
columns directly to `pax8_companies` (mirroring `device_external_ids`'
|
||||
`configuration_item_id`/`link_confidence`/`linked_at` precedent exactly).
|
||||
Build the matcher as a close structural port of
|
||||
`lib/services/device-link-reconciler.ts`, scored with `similarity()` at a
|
||||
0.90 auto-link floor and a tie-margin flag for any second candidate within
|
||||
0.05 of the top score.
|
||||
|
||||
## Phase Requirements
|
||||
|
||||
| ID | Description | Research Support |
|
||||
|----|-------------|------------------|
|
||||
| PAX8-06 | Sync PAX8 orders/invoices (historical line items) into Postgres, enabling cost reconciliation over time | Confirmed via live API: `/invoices` (94 header rows, full history since 2019-05) + `/invoices/{id}/items` (per-company, per-period cost lines) is the correct source. Schema gap identified and mapped field-for-field below. |
|
||||
| PAX8-10 | PAX8 companies automatically matched to Autotask companies by fuzzy name similarity at sync time | `pg_trgm` `similarity()` verified live against real data; threshold 0.90 recommended with evidence. |
|
||||
| PAX8-11 | Unmatched/ambiguous matches flagged, never silently guessed | `pax8_company_match_review` schema already exists (migration 091); matcher logic ports `device-link-reconciler.ts`'s conflict-detection shape. |
|
||||
|
||||
<user_constraints>
|
||||
## User Constraints (from CONTEXT.md)
|
||||
|
||||
### Locked Decisions
|
||||
|
||||
- **D-01:** Auto-link (no human review) only on a high-confidence fuzzy match
|
||||
— not exact-string-only, but a high similarity bar (near-identical names:
|
||||
punctuation/case/whitespace differences, minor typos). The user explicitly
|
||||
chose "be conservative — fewer auto-matches" over a looser threshold or
|
||||
deferring the number to the planner's judgment alone: bias toward flagging
|
||||
borderline cases for review rather than risking a wrong auto-link, since
|
||||
this data feeds cost/billing reconciliation. The planner should document
|
||||
the exact numeric threshold chosen (and the library/approach) directly in
|
||||
the plan so it's easy to find and tune later — this is an initial number,
|
||||
not a permanently fixed one.
|
||||
- **D-02:** Even an otherwise-exact name match must be flagged for review
|
||||
(not auto-linked) if it's ambiguous against more than one Autotask company
|
||||
sharing that same/very-similar name (e.g. franchise locations, "Acme Inc"
|
||||
vs "Acme Holdings Inc"). No silent tie-breaking — ever, even when the
|
||||
string match itself looks perfect.
|
||||
- **D-03:** When a PAX8 company has zero reasonably-similar Autotask
|
||||
candidates at all, still create a `pax8_company_match_review` row with an
|
||||
empty `candidate_company_ids` array — never silently drop it (PAX8-11).
|
||||
This flags it into the admin queue so Phase 14's UI can offer a manual
|
||||
search/pick, or confirm there truly isn't a matching Autotask company yet.
|
||||
- **D-04:** Ambiguous-match review rows carry the **top 3** highest-scoring
|
||||
Autotask candidates (matches the existing `device_link_review` precedent's
|
||||
style of showing a small ranked list, not every plausible match).
|
||||
- **D-05:** PAX8 companies still sitting unresolved in the review queue are
|
||||
**re-scored on every subsequent full sync**, not matched once and left
|
||||
alone — candidates can improve over time (e.g. a renamed/newly-created
|
||||
Autotask company scores better later) without requiring a manual
|
||||
re-trigger. This does NOT apply to already-resolved matches: SC#4's
|
||||
idempotency guarantee (a manually-confirmed match is never overwritten by
|
||||
a later sync) still holds — only rows with `resolved_at IS NULL` are
|
||||
eligible for re-scoring.
|
||||
|
||||
### Claude's Discretion
|
||||
|
||||
- **Exact threshold value / library choice** — user wants "conservative,"
|
||||
not a specific number. The planner should research common fuzzy
|
||||
name-matching approaches (e.g., trigram similarity via Postgres
|
||||
`pg_trgm`, or a JS library) and pick/document a concrete high threshold,
|
||||
erring toward fewer auto-matches per D-01.
|
||||
**Resolved by this research: `pg_trgm`, threshold 0.90 — see Standard Stack.**
|
||||
- **Company name normalization nuances** (legal suffixes like LLC/Inc/Corp,
|
||||
punctuation, abbreviations) — not discussed in depth this session (user
|
||||
deselected this gray area). Planner/researcher should investigate whether
|
||||
Autotask company names in this instance commonly carry legal suffixes
|
||||
PAX8 names don't (or vice versa) and decide normalization rules
|
||||
accordingly; err toward the conservative stance (D-01/D-02) if uncertain.
|
||||
**Resolved by this research: normalize case/whitespace/trailing punctuation
|
||||
only; do NOT strip legal suffixes — see Common Pitfalls.**
|
||||
- **Orders/invoices historical lookback window** — not discussed in depth
|
||||
this session (user deselected this gray area). `migrations/091`'s comment
|
||||
states "sync pulls full order history on first sync in a later phase"
|
||||
(this phase) — the planner should confirm this is still the intent (full
|
||||
history, no bounded window) during research, and flag to the user if
|
||||
PAX8's invoices API makes "full history" impractical (e.g., no
|
||||
pagination limit safety, or a very large per-company invoice count).
|
||||
**Resolved by this research: full history is small (94 invoices, ~56k
|
||||
items total) and practical — no bounded window needed. See Environment
|
||||
Availability / Pitfall 2.**
|
||||
|
||||
### Deferred Ideas (OUT OF SCOPE)
|
||||
|
||||
No capabilities were deferred outside this phase's boundary. The `/pax8` UI,
|
||||
manual resolution flow (PAX8-12), and scheduler cron wiring (PAX8-07/09) are
|
||||
already sequenced into Phases 13-14 — not deferred, just not this phase's job.
|
||||
This phase does NOT: build `/pax8` UI, wire a cron schedule, or build the
|
||||
admin manual-resolution flow. It only populates `pax8_orders`,
|
||||
`pax8_order_items`, and matches/flags `pax8_companies` rows.
|
||||
</user_constraints>
|
||||
|
||||
## Architectural Responsibility Map
|
||||
|
||||
| Capability | Primary Tier | Secondary Tier | Rationale |
|
||||
|------------|-------------|----------------|-----------|
|
||||
| PAX8 invoice/item HTTP fetch + pagination | API / Backend (service layer) | — | `Pax8Client` extension, same as existing `listAllCompanies` etc. — no browser/UI involvement this phase. |
|
||||
| Invoice/item upsert + tombstone reconciliation | API / Backend (service layer) | Database | `Pax8SyncService`-owned, writes via `postgresClient`; DB owns constraints (FKs, unique indexes). |
|
||||
| Fuzzy company-name scoring | Database (Postgres `pg_trgm`) | API / Backend | `similarity()` runs in Postgres itself (SQL, not JS) — the backend service issues the query and interprets results, but the actual string-distance computation is DB-tier, matching the existing `pg_trgm`-adjacent extension pattern already used in this codebase (`pgcrypto`, `uuid-ossp`). |
|
||||
| Match/review persistence (`pax8_companies` columns, `pax8_company_match_review`) | Database | API / Backend | Schema already exists (migration 091) for the review table; this phase adds columns for the confident-match case. |
|
||||
| No UI this phase | — | — | `UI hint: no` per ROADMAP.md — Phase 14 renders review queue. |
|
||||
|
||||
## Standard Stack
|
||||
|
||||
### Core
|
||||
|
||||
| Library | Version | Purpose | Why Standard |
|
||||
|---------|---------|---------|--------------|
|
||||
| `pg_trgm` (Postgres extension) | Bundled with Postgres 16 (`contrib`) | Trigram-based fuzzy string similarity (`similarity()`, `%` operator, `SIMILARITY()` GIN/GiST index support) | [VERIFIED: live query against this project's own Postgres 16 container — `CREATE EXTENSION IF NOT EXISTS pg_trgm;` succeeded, confirming the extension ships with this Postgres image] Zero new npm dependency; matches the codebase's existing pattern of enabling Postgres contrib extensions per-migration (`uuid-ossp` in migration 001, `pgcrypto` in migrations 069/070) rather than adding a JS string-similarity library. No such library exists in `package.json` today. |
|
||||
|
||||
### Supporting
|
||||
|
||||
None — no new npm packages are required for this phase. `pg_trgm` is a
|
||||
built-in Postgres extension enabled via `CREATE EXTENSION IF NOT EXISTS
|
||||
pg_trgm;` inside the phase's new migration; it is not an npm dependency and
|
||||
does not appear in `package.json`.
|
||||
|
||||
### Alternatives Considered
|
||||
|
||||
| Instead of | Could Use | Tradeoff |
|
||||
|------------|-----------|----------|
|
||||
| `pg_trgm` `similarity()` | JS library (`string-similarity`, `fastest-levenshtein`, `natural`) | Would require a new npm dependency and running comparisons in application code (fetch all companies, loop in JS) instead of letting Postgres do set-based scoring. No existing precedent for a fuzzy-match npm lib in this codebase; `pg_trgm` fits the "extend Postgres, don't add packages" convention already established by `uuid-ossp`/`pgcrypto`. Not recommended. |
|
||||
| `pg_trgm` `similarity()` | Exact-match only (`LOWER(TRIM(a)) = LOWER(TRIM(b))`) | Simpler, zero risk of false positive, but misses near-identical names with a typo, extra space, or reordered punctuation — undercuts PAX8-10's "fuzzy" requirement and would push far more companies into the review queue than D-01 intends (conservative ≠ exact-only; D-01 explicitly says "not exact-string-only"). |
|
||||
| `pg_trgm` `similarity()` | Levenshtein distance (`fuzzystrmatch` extension, also Postgres contrib) | Also viable and also a zero-new-dependency Postgres extension. `pg_trgm`'s `similarity()` (0.0-1.0 normalized score) is easier to reason about as a single conservative threshold than a raw edit-distance integer that varies with string length; `pg_trgm` is also the more common choice for this exact "fuzzy company name match" use case in the wider Postgres ecosystem. Either would work; `pg_trgm` is recommended for the normalized 0-1 score alone. |
|
||||
|
||||
**Installation:**
|
||||
```sql
|
||||
-- Inside the new migration (e.g. migrations/092_pax8_orders_company_matching.sql)
|
||||
CREATE EXTENSION IF NOT EXISTS pg_trgm;
|
||||
```
|
||||
|
||||
**Version verification:** `pg_trgm` is bundled with the Postgres 16 server
|
||||
image already running in this project's `pulse-postgres` container —
|
||||
verified live: `CREATE EXTENSION IF NOT EXISTS pg_trgm;` executed
|
||||
successfully with no download/install step. No separate version to track;
|
||||
it ships with the Postgres major version (16) already pinned in
|
||||
`docker-compose.yml`.
|
||||
|
||||
## Package Legitimacy Audit
|
||||
|
||||
**Not applicable this phase.** No new npm, pip, or cargo packages are being
|
||||
installed. The only new dependency is the Postgres `pg_trgm` contrib
|
||||
extension, verified live against the project's actual running Postgres 16
|
||||
container (see above) — not an installable package subject to the
|
||||
slopcheck/registry-verification protocol.
|
||||
|
||||
**Packages removed due to slopcheck [SLOP] verdict:** none (n/a)
|
||||
**Packages flagged as suspicious [SUS]:** none (n/a)
|
||||
|
||||
## Architecture Patterns
|
||||
|
||||
### System Architecture Diagram
|
||||
|
||||
```
|
||||
PAX8 API (api.pax8.com/v1)
|
||||
│
|
||||
├─ GET /invoices?page&size (94 rows total, full history)
|
||||
│ │
|
||||
│ └─▶ Pax8Client.listAllInvoices() [paginateAll<T>, size=200 → 1 page]
|
||||
│
|
||||
└─ GET /invoices/{invoiceId}/items?page&size (~500-700 rows per invoice)
|
||||
│
|
||||
└─▶ Pax8Client.listAllInvoiceItems(invoiceId)
|
||||
[paginateAll<T> per invoice — nested/per-parent fetch,
|
||||
invoked once per invoice header, NOT a flat top-level list]
|
||||
│
|
||||
▼
|
||||
Pax8SyncService.syncOrders()
|
||||
1. fetch all invoice headers
|
||||
2. for each header: fetch all its items
|
||||
3. upsert header → pax8_orders (company_id stays NULL — see Pitfall 1)
|
||||
4. upsert items → pax8_order_items (company_id populated per item)
|
||||
5. tombstone anything not seen (full-list reconciliation, same
|
||||
pattern as syncCompanies/syncSubscriptions/syncProducts)
|
||||
│
|
||||
▼
|
||||
Pax8SyncService.syncCompanyMatches() [D-05: only runs against
|
||||
pax8_companies rows where autotask_company_id IS NULL
|
||||
OR the pax8_company_match_review row for it has resolved_at IS NULL]
|
||||
1. SELECT unresolved pax8_companies
|
||||
2. for each: SELECT companies ORDER BY similarity(name, company_name) DESC
|
||||
3. score >= 0.90 AND no second candidate within 0.05 → auto-link
|
||||
(write autotask_company_id/match_confidence/matched_at/match_method
|
||||
directly on pax8_companies — mirrors device_external_ids' pattern)
|
||||
score >= 0.90 but a near-tie exists, OR best score < 0.90,
|
||||
OR zero candidates → write/update
|
||||
pax8_company_match_review (top 3 candidates, or empty array per D-03)
|
||||
│
|
||||
▼
|
||||
Postgres: pax8_orders / pax8_order_items / pax8_companies /
|
||||
pax8_company_match_review
|
||||
│
|
||||
▼
|
||||
(Phase 14 — /pax8 UI reads pax8_company_match_review + the
|
||||
auto-match columns on pax8_companies; out of scope this phase)
|
||||
```
|
||||
|
||||
### Recommended Project Structure
|
||||
|
||||
```
|
||||
lib/services/
|
||||
├── pax8-client.ts # extend: listAllInvoices(), listAllInvoiceItems(invoiceId)
|
||||
├── pax8-sync-service.ts # extend: syncOrders() step in fullSync()
|
||||
├── pax8-company-matcher.ts # NEW — company-matching logic, ported from
|
||||
│ # device-link-reconciler.ts's structure
|
||||
lib/types/
|
||||
├── pax8.ts # extend: Pax8Invoice, Pax8InvoiceItem types
|
||||
migrations/
|
||||
├── 092_pax8_orders_company_matching.sql # NEW — pg_trgm + schema gaps below
|
||||
```
|
||||
|
||||
### Pattern 1: Per-parent nested pagination (invoice → items)
|
||||
|
||||
**What:** Unlike `listAllCompanies`/`listAllSubscriptions`/`listAllProducts`
|
||||
(flat top-level lists), invoice items are a **child resource of each invoice**
|
||||
(`/invoices/{invoiceId}/items`). There is no flat `/invoice-items` endpoint.
|
||||
**When to use:** Fetching all historical line items requires: (1) page
|
||||
through all 94 invoice headers once, (2) for each header, page through its
|
||||
own items (typically 3-4 pages at size=200 given ~500-700 items/invoice).
|
||||
**Closest existing analog:** `lib/services/itglue-sync-service.ts`'s
|
||||
`syncModels()` (per Phase 11's `11-PATTERNS.md`) — iterate parent entities,
|
||||
call a child-relationship endpoint per parent. Adapt that iteration shape;
|
||||
`paginateAll<T>` itself doesn't need to change, just call it once per
|
||||
invoice ID.
|
||||
|
||||
```typescript
|
||||
// Source: pattern derived from live API verification (2026-07-11) —
|
||||
// api.pax8.com/v1/invoices/{invoiceId}/items confirmed to accept the
|
||||
// same page/size/Pax8PageEnvelope<T> shape as the flat list endpoints.
|
||||
async listAllInvoiceItems(invoiceId: string): Promise<Pax8InvoiceItem[]> {
|
||||
return this.paginateAll<Pax8InvoiceItem>((page, size) =>
|
||||
this.fetchJson<Pax8PageEnvelope<Pax8InvoiceItem>>(
|
||||
`/invoices/${invoiceId}/items?page=${page}&size=${size}`,
|
||||
),
|
||||
);
|
||||
}
|
||||
|
||||
async listAllInvoices(): Promise<Pax8Invoice[]> {
|
||||
return this.paginateAll<Pax8Invoice>((page, size) =>
|
||||
this.fetchJson<Pax8PageEnvelope<Pax8Invoice>>(`/invoices?page=${page}&size=${size}`),
|
||||
);
|
||||
}
|
||||
```
|
||||
|
||||
### Pattern 2: Confidence-ranked matching, ported from `device-link-reconciler.ts`
|
||||
|
||||
**What:** Cascading match strategies ranked by confidence; conflicts (2+
|
||||
candidates) are logged for review, never auto-merged; a resolved/unresolved
|
||||
lifecycle gate protects human decisions from being overwritten.
|
||||
**When to use:** Directly reusable structure for company-name matching —
|
||||
same shape, different scoring function (`similarity()` instead of
|
||||
serial/MAC/hostname exact lookups).
|
||||
**Example (adapted, not copied verbatim — company matching has exactly one
|
||||
"strategy" — trigram similarity — rather than device-link-reconciler's
|
||||
cascade of several):**
|
||||
|
||||
```typescript
|
||||
// Source: adapted from lib/services/device-link-reconciler.ts's
|
||||
// applyLink() / recordConflict() / pickBestCandidate() shape (lines 113-181)
|
||||
const AUTO_LINK_THRESHOLD = 0.90; // D-01: conservative, initial/tunable value
|
||||
const TIE_MARGIN = 0.05; // D-02: candidates within this margin of the
|
||||
// top score are treated as ambiguous
|
||||
|
||||
interface CompanyCandidate {
|
||||
autotask_company_id: number;
|
||||
score: number; // pg_trgm similarity(), 0.0-1.0
|
||||
}
|
||||
|
||||
async function findCandidates(pax8Name: string): Promise<CompanyCandidate[]> {
|
||||
const res = await postgresClient.query<{ id: string; score: string }>(
|
||||
`SELECT id::text, similarity($1, company_name)::text AS score
|
||||
FROM companies
|
||||
WHERE is_active = true
|
||||
AND similarity($1, company_name) > 0.3 -- floor: keep candidate list small
|
||||
ORDER BY score DESC
|
||||
LIMIT 5`,
|
||||
[pax8Name],
|
||||
);
|
||||
return res.rows.map(r => ({ autotask_company_id: Number(r.id), score: Number(r.score) }));
|
||||
}
|
||||
|
||||
function decide(candidates: CompanyCandidate[]):
|
||||
| { kind: 'auto'; match: CompanyCandidate }
|
||||
| { kind: 'review'; top3: CompanyCandidate[] } {
|
||||
if (candidates.length === 0) return { kind: 'review', top3: [] }; // D-03
|
||||
const [best, second] = candidates;
|
||||
const tie = second && best.score - second.score < TIE_MARGIN;
|
||||
if (best.score >= AUTO_LINK_THRESHOLD && !tie) {
|
||||
return { kind: 'auto', match: best }; // high confidence, no ambiguity
|
||||
}
|
||||
return { kind: 'review', top3: candidates.slice(0, 3) }; // D-02/D-04
|
||||
}
|
||||
```
|
||||
|
||||
```sql
|
||||
-- applyLink() equivalent — mirrors device_external_ids' UPDATE shape exactly
|
||||
-- (lib/services/device-link-reconciler.ts lines 113-127)
|
||||
UPDATE pax8_companies
|
||||
SET autotask_company_id = $2,
|
||||
match_confidence = $3,
|
||||
match_method = 'pg_trgm',
|
||||
matched_at = NOW()
|
||||
WHERE id = $1
|
||||
AND resolved_at IS NULL; -- D-05: never touch a manually-resolved row
|
||||
-- (resolved_at tracked via the review table per pax8 company — see
|
||||
-- Schema section below for exact re-scoring gate query)
|
||||
```
|
||||
|
||||
```sql
|
||||
-- recordConflict() equivalent — mirrors device_link_review's upsert
|
||||
-- (lib/services/device-link-reconciler.ts lines 152-172)
|
||||
INSERT INTO pax8_company_match_review
|
||||
(pax8_company_id, candidate_company_ids, match_confidences)
|
||||
VALUES ($1, $2::bigint[], $3::text[])
|
||||
ON CONFLICT (pax8_company_id) WHERE resolved_at IS NULL
|
||||
DO UPDATE SET candidate_company_ids = EXCLUDED.candidate_company_ids,
|
||||
match_confidences = EXCLUDED.match_confidences,
|
||||
detected_at = NOW();
|
||||
```
|
||||
|
||||
### Anti-Patterns to Avoid
|
||||
|
||||
- **Stripping legal suffixes (LLC/Inc/Corp) during normalization:** would
|
||||
reduce distinguishing information between genuinely distinct entities
|
||||
(e.g. "Acme Inc" vs "Acme Holdings Inc" — exactly the case D-02 calls
|
||||
out). Normalize only case, leading/trailing whitespace, and repeated
|
||||
internal whitespace — not legal-form tokens.
|
||||
- **Relying on `pax8_orders.pax8_company_id`** for per-company cost
|
||||
queries: it will be `NULL` on every row (see Pitfall 1). All per-company
|
||||
cost joins must go through `pax8_order_items.pax8_company_id` (new
|
||||
column), not the header.
|
||||
- **Building a flat `/invoice-items` paginator:** the endpoint is
|
||||
per-invoice (`/invoices/{id}/items`); there is no top-level items list.
|
||||
- **Auto-linking on an exact string match alone without checking for a
|
||||
second near-tied candidate:** violates D-02 explicitly — always check for
|
||||
ambiguity even at score 1.0.
|
||||
- **Re-scoring already-resolved review rows:** violates D-05/SC#4 —
|
||||
re-scoring must exclude any `pax8_companies` row that already has a
|
||||
human-confirmed match (`resolved_at IS NOT NULL` on its review row, or
|
||||
`autotask_company_id` set via manual resolution rather than the auto
|
||||
scorer).
|
||||
|
||||
## Don't Hand-Roll
|
||||
|
||||
| Problem | Don't Build | Use Instead | Why |
|
||||
|---------|-------------|-------------|-----|
|
||||
| Fuzzy string similarity scoring | Custom Levenshtein/Jaro-Winkler in TypeScript | Postgres `pg_trgm` `similarity()` | Set-based, indexable (GIN/GiST `gin_trgm_ops` if the review query ever needs to scale beyond ~250 companies), no new npm dependency, and it's already proven case-insensitive and punctuation-tolerant against this project's real data (see Summary). Hand-rolling in JS means fetching all 242 companies into memory per pax8 company and re-implementing what Postgres already ships. |
|
||||
| Pagination-until-exhausted for nested per-parent resources | A bespoke recursive fetch loop for invoice items | Reuse the existing private `paginateAll<T>` helper in `pax8-client.ts`, called once per invoice ID | The helper is already generic over any `(page, size) => Pax8PageEnvelope<T>` fetcher — no new pagination logic needed, just a new call site. |
|
||||
| Match/review conflict bookkeeping | A new bespoke review-table shape/lifecycle | `pax8_company_match_review` (already exists, migration 091, copied field-for-field from `device_link_review`) | Schema, indexes, and the "one open review per subject" partial-unique-index pattern are already built and proven in production by the device-linking feature. |
|
||||
|
||||
**Key insight:** Every piece of this phase's hard problem (fuzzy matching,
|
||||
paginated nested fetch, conflict/review bookkeeping) already has a working,
|
||||
in-repo precedent. The job is porting/extending, not inventing.
|
||||
|
||||
## Runtime State Inventory
|
||||
|
||||
Not applicable — this is a greenfield sync/matching feature, not a
|
||||
rename/refactor/migration phase.
|
||||
|
||||
## Common Pitfalls
|
||||
|
||||
### Pitfall 1: Invoice header `companyId` is always `null` — company data lives on line items
|
||||
|
||||
**What goes wrong:** A naive implementation that reads `migrations/091`'s
|
||||
`pax8_orders.pax8_company_id` column and expects to populate it from the
|
||||
invoice header's `companyId` field will find that field is `null` on
|
||||
100% of real invoices.
|
||||
**Why it happens:** [VERIFIED: live `GET /v1/invoices` call against
|
||||
`api.pax8.com`, 2026-07-11] PAX8's `/invoices` resource is the **partner's
|
||||
own consolidated monthly bill** (one row per billing period across the
|
||||
entire reseller account — `partnerName: "Wulf Consulting, Inc."` on every
|
||||
row), not a per-end-customer invoice. The per-customer breakdown is only
|
||||
available on `/invoices/{id}/items`, where every sampled item (200/200 in
|
||||
one invoice) had a non-null `companyId` and `companyName`.
|
||||
**How to avoid:** Add a `pax8_company_id` (soft ref, `UUID`, no hard FK —
|
||||
matching the existing soft-ref convention for `pax8_subscriptions`/
|
||||
`pax8_orders`) column to `pax8_order_items` in the new migration. All
|
||||
per-company cost reconciliation queries join through the item table, not
|
||||
the header. `pax8_orders.pax8_company_id` remains in the schema (can't
|
||||
edit a committed migration) but will simply stay `NULL` for every real row
|
||||
— document this so a future maintainer doesn't treat it as a bug.
|
||||
**Warning signs:** Any query trying to filter/join `pax8_orders` by
|
||||
company will silently return zero rows.
|
||||
|
||||
### Pitfall 2: `pax8_order_items` schema doesn't yet match PAX8's real field names
|
||||
|
||||
**What goes wrong:** `migrations/091`'s comment describes
|
||||
`pax8_order_items` as having `unit_price`/`line_total` mirroring PAX8's
|
||||
"InvoiceItem" object — but the live response uses different field names
|
||||
entirely, and — critically — has **two cost dimensions** (customer-facing
|
||||
and partner-facing), not one.
|
||||
**Why it happens:** The migration was written before this phase's live
|
||||
verification. [VERIFIED: live `GET /v1/invoices/{id}/items` response,
|
||||
2026-07-11] the actual item shape is:
|
||||
```json
|
||||
{
|
||||
"id": "39d199e1-...", "type": "subscription", "externalId": "29861467",
|
||||
"companyId": "6ff975bd-...", "forCompanyId": null,
|
||||
"companyName": "1 of 1 MotorSports",
|
||||
"startPeriod": "2026-06-03", "endPeriod": "2026-07-02",
|
||||
"quantity": 9, "unitOfMeasure": "User", "term": "Monthly",
|
||||
"sku": "MST-NCE-103-C100",
|
||||
"description": "Microsoft 365 Business Premium [New Commerce Experience]",
|
||||
"rateType": "Flat", "chargeType": "per",
|
||||
"price": 26.4, "subTotal": 199.58,
|
||||
"cost": 22.176, "costTotal": 199.58,
|
||||
"total": 237.6, "amountDue": 199.58,
|
||||
"productId": "05df4303-...", "productName": "Microsoft 365 Business Premium [New Commerce Experience]",
|
||||
"vendorName": "Microsoft", "billingFee": 0, "billingFeeRate": 0,
|
||||
"currencyCode": "USD", "salesTax": 0,
|
||||
"subscriptionId": "797a2fac-..."
|
||||
}
|
||||
```
|
||||
Distinct `type` values observed in one invoice's items: `subscription`,
|
||||
`prorate`, `one-time` (there may be more across the full 94-invoice history
|
||||
— treat as free-form `TEXT`, not a `CHECK`-constrained enum, matching
|
||||
`pax8_subscriptions.status`'s existing convention).
|
||||
**How to avoid:** The new migration should add columns that map cleanly:
|
||||
`price` → `unit_price` (already named this in the existing schema — keep),
|
||||
`amountDue` → `line_total` (the actual billed amount — **not** `total`,
|
||||
which appears to be the pre-proration/pre-discount full-period amount;
|
||||
`subTotal` and `costTotal` coincided with `amountDue`/`cost*quantity` in
|
||||
the one sample invoice inspected, but the exact semantic difference between
|
||||
`total` and `subTotal`/`amountDue` should be spot-checked against a
|
||||
prorated line item before finalizing the column mapping — flagged as an
|
||||
Open Question below), plus new columns: `pax8_company_id` (Pitfall 1),
|
||||
`subscription_id` (soft ref to `pax8_subscriptions.id` — enables joining a
|
||||
cost history to a specific seat/license over time, directly serving
|
||||
PAX8-06's "cost reconciliation over time" goal), `type`, `sku`,
|
||||
`description`, `start_period`, `end_period` (critical — this is what makes
|
||||
"cost over time" queryable; `synced_at` alone doesn't tell you which
|
||||
billing period a line covers), and dual-cost columns mirroring Phase 11's
|
||||
existing `price`/`partner_cost` precedent on `pax8_subscriptions`: add
|
||||
`partner_cost` (from `cost`) and `partner_cost_total` (from `costTotal`)
|
||||
alongside `unit_price`/`line_total`.
|
||||
**Warning signs:** Cost totals that don't reconcile against PAX8's own
|
||||
portal; inability to answer "what did company X pay for product Y in
|
||||
March" without a `raw_payload` JSONB dig.
|
||||
|
||||
### Pitfall 3: `/orders` endpoint is unreliable — confirms the migration's choice to use `/invoices`
|
||||
|
||||
**What goes wrong:** A developer tempted to "just use the orders endpoint
|
||||
since the table is called `pax8_orders`" will hit an unreliable endpoint.
|
||||
**Why it happens:** [VERIFIED: live `GET /v1/orders` call, 2026-07-11]
|
||||
returned an HTTP 504 Gateway Timeout in this environment (vs. `/invoices`,
|
||||
which responded in well under a second). This independently confirms
|
||||
`migrations/091`'s inline comment that Phase 12 should source from
|
||||
`/invoices`, not the bare `/orders` object — not just because `/orders`
|
||||
"lacks pricing/status" as the migration comment states, but because it may
|
||||
also simply be unreliable/slow for this account.
|
||||
**How to avoid:** Do not add an `/orders` call to `pax8-client.ts`. Source
|
||||
exclusively from `/invoices` and `/invoices/{id}/items` as designed.
|
||||
**Warning signs:** Sync timeouts or 504s if a future maintainer tries to
|
||||
wire up `/orders` directly.
|
||||
|
||||
### Pitfall 4: Trigram `similarity()` is already case-insensitive — don't double-normalize incorrectly
|
||||
|
||||
**What goes wrong:** Assuming `similarity('ABC Inc', 'abc inc')` requires
|
||||
manual `LOWER()` wrapping to score high, then writing normalization code
|
||||
that's redundant or, worse, that strips information `pg_trgm` didn't need
|
||||
stripped.
|
||||
**Why it happens:** [VERIFIED: live query — `SELECT similarity('ABC Inc',
|
||||
'abc inc')` returned `1`] `pg_trgm`'s `similarity()` operates
|
||||
case-insensitively by default in this Postgres 16 instance (default
|
||||
collation). Live re-test with explicit `LOWER(TRIM(...))` on both sides
|
||||
produced **identical** bucket counts to the un-normalized query — no
|
||||
difference.
|
||||
**How to avoid:** Still apply `TRIM()` (leading/trailing whitespace) for
|
||||
defensiveness and cleaner `pax8_company_match_review` display strings, but
|
||||
don't build elaborate case-folding logic expecting it to change match
|
||||
outcomes — it won't, `pg_trgm` already handles it.
|
||||
**Warning signs:** None currently — this is a "don't over-build" pitfall,
|
||||
not a correctness bug.
|
||||
|
||||
### Pitfall 5: 89 vs 78 discrepancy — always filter Autotask candidates by `is_active`
|
||||
|
||||
**What goes wrong:** Matching against the full `companies` table
|
||||
(including inactive/decommissioned Autotask companies) inflates apparent
|
||||
match counts and can auto-link a PAX8 company to a defunct Autotask
|
||||
company.
|
||||
**Why it happens:** [VERIFIED: live query] An exact-name join with no
|
||||
`is_active` filter found 89 matches; adding `WHERE c.is_active = true`
|
||||
dropped this to 78 — 11 PAX8 companies exactly match an **inactive**
|
||||
Autotask company name only.
|
||||
**How to avoid:** Every candidate query in the matcher must filter
|
||||
`companies.is_active = true`, exactly as `device-link-reconciler.ts`'s
|
||||
`findBySerial`/`findByHostnameInCompany` filter `is_deleted = false` on
|
||||
`configuration_items`.
|
||||
**Warning signs:** Matches to companies an admin can't find in the active
|
||||
company list.
|
||||
|
||||
## Code Examples
|
||||
|
||||
### New migration skeleton
|
||||
|
||||
```sql
|
||||
-- migrations/092_pax8_orders_company_matching.sql
|
||||
-- Phase 12: enables pg_trgm, fixes the company_id gap on pax8_order_items
|
||||
-- (invoice headers have no per-company data — see 12-RESEARCH.md Pitfall 1),
|
||||
-- and adds confident-auto-match columns to pax8_companies mirroring
|
||||
-- device_external_ids' configuration_item_id/link_confidence/linked_at
|
||||
-- precedent (migration 079).
|
||||
|
||||
CREATE EXTENSION IF NOT EXISTS pg_trgm;
|
||||
|
||||
ALTER TABLE pax8_order_items
|
||||
ADD COLUMN IF NOT EXISTS pax8_company_id UUID, -- soft ref -> pax8_companies(id); see Pitfall 1
|
||||
ADD COLUMN IF NOT EXISTS subscription_id UUID, -- soft ref -> pax8_subscriptions(id)
|
||||
ADD COLUMN IF NOT EXISTS item_type TEXT, -- 'subscription' | 'prorate' | 'one-time' | ... (free-form, see Pitfall 2)
|
||||
ADD COLUMN IF NOT EXISTS sku TEXT,
|
||||
ADD COLUMN IF NOT EXISTS description TEXT,
|
||||
ADD COLUMN IF NOT EXISTS start_period TIMESTAMPTZ,
|
||||
ADD COLUMN IF NOT EXISTS end_period TIMESTAMPTZ,
|
||||
ADD COLUMN IF NOT EXISTS partner_cost NUMERIC(12,2),
|
||||
ADD COLUMN IF NOT EXISTS partner_cost_total NUMERIC(12,2);
|
||||
|
||||
CREATE INDEX IF NOT EXISTS idx_pax8_order_items_company ON pax8_order_items(pax8_company_id);
|
||||
CREATE INDEX IF NOT EXISTS idx_pax8_order_items_period ON pax8_order_items(start_period, end_period);
|
||||
|
||||
ALTER TABLE pax8_companies
|
||||
ADD COLUMN IF NOT EXISTS autotask_company_id BIGINT, -- soft ref -> companies(id), no hard FK
|
||||
ADD COLUMN IF NOT EXISTS match_confidence NUMERIC(4,3), -- raw pg_trgm score, 0.000-1.000
|
||||
ADD COLUMN IF NOT EXISTS match_method TEXT, -- 'pg_trgm' | 'manual'
|
||||
ADD COLUMN IF NOT EXISTS matched_at TIMESTAMPTZ;
|
||||
|
||||
CREATE INDEX IF NOT EXISTS idx_pax8_companies_autotask ON pax8_companies(autotask_company_id);
|
||||
```
|
||||
|
||||
### Live-verified similarity distribution (evidence for the 0.90 threshold)
|
||||
|
||||
```
|
||||
-- Run against this project's real pax8_companies (118 rows) and
|
||||
-- companies (242 active rows), 2026-07-11:
|
||||
exact/near-identical matches (score = 1.00): 80 pax8 companies
|
||||
highest-scoring non-match in the entire dataset: 0.70
|
||||
("Thoroughbred Construction Company" vs "Thoroughbred Construction Group")
|
||||
next tier down: 0.65, 0.6666, 0.6296, 0.55 (all plausible non-matches)
|
||||
gap between 0.70 (highest non-match) and 1.00 (lowest real match): wide, empty
|
||||
```
|
||||
This is why 0.90 is recommended: it sits comfortably inside the empty gap
|
||||
between the highest observed false-positive risk (0.70) and the cluster of
|
||||
genuine matches (1.00), while still being below 1.00 to tolerate minor
|
||||
punctuation/typo variance that didn't happen to occur in this particular
|
||||
83-row sample but could occur in future data.
|
||||
|
||||
## State of the Art
|
||||
|
||||
Not applicable in the traditional sense (no deprecated approach being
|
||||
replaced) — this is the first implementation of company matching in this
|
||||
codebase. The one relevant "current vs legacy" note: `device_link_review`
|
||||
(migration 080) is itself fairly recent and is the correct/current pattern
|
||||
to mirror, not an older approach being superseded.
|
||||
|
||||
## Assumptions Log
|
||||
|
||||
| # | Claim | Section | Risk if Wrong |
|
||||
|---|-------|---------|---------------|
|
||||
| A1 | `total` vs `subTotal`/`amountDue` semantic distinction on invoice items (which represents the "actual billed line amount" for `line_total`) is based on inspecting one invoice's items where the values happened to coincide with `quantity × cost`; the exact formula/relationship between `total`, `subTotal`, `amountDue`, `cost`, and `costTotal` across prorated or credited line items was not exhaustively verified. | Common Pitfalls > Pitfall 2, Code Examples | If the mapping is wrong, `line_total`/`partner_cost_total` columns could store the wrong dollar amount for prorated months, undermining the "cost reconciliation over time" value of PAX8-06. Low-risk-to-verify: spot-check a `type: 'prorate'` item's fields against a second real invoice before finalizing the sync's column-mapping code. |
|
||||
| A2 | `forCompanyId` (seen as `null` on the one sampled item) may represent a sub-reseller or delegated-partner scenario not applicable to this single-tenant PAX8 account; its exact meaning wasn't looked up in PAX8's docs (WebFetch on the live docs pages did not surface field-level schema detail). | Common Pitfalls > Pitfall 2 (raw item JSON) | Low risk — if always `null` for this account (single-tenant reseller, confirmed by `partnerName` being constant across all 94 invoices), it can be safely ignored/stored in `raw_payload` only, not a structured column. |
|
||||
| A3 | The full 94-invoice / ~56k-item history size is stable going forward (i.e., growth is ~1 invoice + ~500-700 items per month) — extrapolated from observing 3 consecutive months' item counts (585, 611, 678), not the full 94-invoice history. | Summary, Pitfall 2, Environment Availability | Low risk — even at 2x the observed rate, total row count stays in the tens of thousands, well within normal Postgres table sizes; no partitioning or archival strategy needed for the foreseeable future. |
|
||||
|
||||
## Open Questions
|
||||
|
||||
1. **Exact `total` vs `subTotal`/`amountDue` semantics on invoice items for prorated/credited lines**
|
||||
- What we know: For one `type: 'subscription'` line, `subTotal` (199.58)
|
||||
equaled `amountDue` (199.58) and approximately equaled `quantity × cost`
|
||||
(9 × 22.176 ≈ 199.58), while `total` (237.60) equaled `quantity × price`
|
||||
(9 × 26.40 = 237.60) — suggesting `total` is the undiscounted retail
|
||||
amount and `subTotal`/`amountDue` reflect the actual amount due
|
||||
(possibly after a billing adjustment or the partner-cost pass-through).
|
||||
- What's unclear: Whether this relationship holds for `type: 'prorate'`
|
||||
or `type: 'one-time'` items, or for credited/negative-amount lines.
|
||||
- Recommendation: Before finalizing the migration's column mapping,
|
||||
fetch and inspect at least one `prorate`-type item's full field set
|
||||
(already know they exist — seen in the 200-item sample) and confirm
|
||||
`line_total` should map to `amountDue` (not `total` or `subTotal`)
|
||||
across all three observed types. This is a 5-minute live-API check,
|
||||
not a design risk — can be done as the first task of implementation.
|
||||
|
||||
2. **Full enumeration of `type` values across all 94 invoices**
|
||||
- What we know: `subscription`, `prorate`, `one-time` observed in one
|
||||
invoice's 200-item sample.
|
||||
- What's unclear: Whether other values exist elsewhere in the 7-year
|
||||
history (e.g., `credit`, `adjustment`, `refund`).
|
||||
- Recommendation: Store as free-form `TEXT`, no `CHECK` constraint (this
|
||||
is already the plan) — the column is forward-compatible regardless of
|
||||
what other values surface during the actual full sync.
|
||||
|
||||
## Environment Availability
|
||||
|
||||
| Dependency | Required By | Available | Version | Fallback |
|
||||
|------------|------------|-----------|---------|----------|
|
||||
| `pg_trgm` Postgres extension | Fuzzy company-name matching | ✓ (verified live) | Bundled with Postgres 16 (this project's pinned version) | — |
|
||||
| PAX8 API — `/invoices` | Order/invoice header sync | ✓ (verified live, HTTP 200) | v1 | — |
|
||||
| PAX8 API — `/invoices/{id}/items` | Line-item sync | ✓ (verified live, HTTP 200) | v1 | — |
|
||||
| PAX8 API — `/orders` | (not used) | ✗ (HTTP 504 in this environment) | v1 | Use `/invoices` instead — already the plan; no fallback needed since this endpoint isn't part of the design. |
|
||||
| PAX8_CLIENT_ID / PAX8_CLIENT_SECRET | All PAX8 API calls | ✓ (present in `.env.local`, used for live verification) | — | — |
|
||||
| Live dev Postgres data (118 `pax8_companies`, 242 active `companies`) | Matching-threshold validation | ✓ (from Phase 11's completed live sync) | — | — |
|
||||
|
||||
**Missing dependencies with no fallback:** none.
|
||||
|
||||
**Missing dependencies with fallback:** `/orders` endpoint unreliable —
|
||||
not needed; design already sources from `/invoices`.
|
||||
|
||||
## Validation Architecture
|
||||
|
||||
### Test Framework
|
||||
|
||||
| Property | Value |
|
||||
|----------|-------|
|
||||
| Framework | vitest 4.1.5 |
|
||||
| Config file | `vitest.config.ts` (root) — `include: ['lib/**/*.test.ts']`, `environment: 'node'` |
|
||||
| Quick run command | `npx vitest run lib/services/pax8-client.test.ts lib/services/pax8-company-matcher.test.ts` |
|
||||
| Full suite command | `npm test` (`vitest run`) |
|
||||
|
||||
### Phase Requirements → Test Map
|
||||
|
||||
| Req ID | Behavior | Test Type | Automated Command | File Exists? |
|
||||
|--------|----------|-----------|-------------------|-------------|
|
||||
| PAX8-06 | `listAllInvoices()` concatenates pages in order | unit | `npx vitest run lib/services/pax8-client.test.ts -t "listAllInvoices"` | ❌ Wave 0 — extend existing `pax8-client.test.ts` following its `makeMultiPageFetchMock` pattern |
|
||||
| PAX8-06 | `listAllInvoiceItems(invoiceId)` calls the correct per-invoice nested path and concatenates pages | unit | `npx vitest run lib/services/pax8-client.test.ts -t "listAllInvoiceItems"` | ❌ Wave 0 |
|
||||
| PAX8-06 | Sync upserts orders/items and tombstones missing rows (mirrors existing `syncCompanies`/`syncSubscriptions` test shape) | unit | `npx vitest run lib/services/pax8-sync-service.test.ts` | ❌ Wave 0 — no `pax8-sync-service.test.ts` exists yet at all; check whether Phase 11 added one before assuming greenfield |
|
||||
| PAX8-10 | Auto-link fires only when score ≥ 0.90 and no near-tie | unit | `npx vitest run lib/services/pax8-company-matcher.test.ts -t "auto-link"` | ❌ Wave 0 |
|
||||
| PAX8-10 | Below-threshold or tied candidates produce a review row, not an auto-link | unit | `npx vitest run lib/services/pax8-company-matcher.test.ts -t "review"` | ❌ Wave 0 |
|
||||
| PAX8-11 | Zero-candidate case still creates a review row with empty `candidate_company_ids` | unit | `npx vitest run lib/services/pax8-company-matcher.test.ts -t "empty candidates"` | ❌ Wave 0 |
|
||||
| SC#4 | Re-running matcher never overwrites a `resolved_at IS NOT NULL` row | unit | `npx vitest run lib/services/pax8-company-matcher.test.ts -t "idempotent"` | ❌ Wave 0 |
|
||||
|
||||
### Sampling Rate
|
||||
|
||||
- **Per task commit:** `npx vitest run lib/services/pax8-*.test.ts`
|
||||
- **Per wave merge:** `npm test` (full suite)
|
||||
- **Phase gate:** Full suite green before `/gsd:verify-work`
|
||||
|
||||
### Wave 0 Gaps
|
||||
|
||||
- [ ] `lib/services/pax8-company-matcher.test.ts` — covers PAX8-10, PAX8-11,
|
||||
SC#4. Since `pg_trgm` `similarity()` runs in Postgres, this suite
|
||||
needs either (a) a mocked `postgresClient.query` following the
|
||||
existing `pax8-client.test.ts` mocking convention (`vi.fn` returning
|
||||
canned rows), keeping tests DB-free, or (b) an integration test
|
||||
against a real Postgres test instance if one is available in CI —
|
||||
check whether any existing `lib/services/*.test.ts` file already
|
||||
does the latter (none inspected so far do; the codebase's tests are
|
||||
all pure-mock, `vitest 4.1.5`, `environment: 'node'`). Recommend
|
||||
mocking `postgresClient.query` to keep this test fast and consistent
|
||||
with the rest of the suite.
|
||||
- [ ] Extend `lib/services/pax8-client.test.ts` — add `listAllInvoices`/
|
||||
`listAllInvoiceItems` cases following the existing
|
||||
`makeMultiPageFetchMock` helper already in that file.
|
||||
- [ ] Confirm whether `lib/services/pax8-sync-service.test.ts` exists
|
||||
(not found in this research pass) — if it genuinely doesn't exist,
|
||||
Phase 11 shipped without direct unit tests for the sync service
|
||||
itself (its Plan 03 was a live-data verification script instead,
|
||||
per `11-03-SUMMARY.md`). The planner should decide whether Phase 12
|
||||
introduces the first unit tests for `Pax8SyncService` or continues
|
||||
the live-verification-script pattern Phase 11 established.
|
||||
|
||||
## Security Domain
|
||||
|
||||
### Applicable ASVS Categories
|
||||
|
||||
| ASVS Category | Applies | Standard Control |
|
||||
|---------------|---------|-----------------|
|
||||
| V2 Authentication | no | No new auth surface — existing session-gated `/api/pax8/sync` route from Phase 11 covers this. |
|
||||
| V3 Session Management | no | Unchanged. |
|
||||
| V4 Access Control | yes | `resolved_by_user_id` on `pax8_company_match_review` already references `"user"(id)` — Phase 14's manual-resolution flow (out of scope here) must call `requireAdmin()`/`requirePermission()` per CLAUDE.md convention. This phase's sync-time matching logic itself runs as a background/system process, not user-triggered per-row, so no per-request authz check is needed inside the matcher. |
|
||||
| V5 Input Validation | yes | PAX8 API responses are external input — continue the existing pattern (`Pax8Company`/`Pax8Subscription`/`Pax8Product` types with a `[key: string]: unknown` escape hatch, no runtime Zod validation per CLAUDE.md's "no Zod in API routes unless required"). Parameterized queries only (`postgresClient.query(sql, params)`) — never string-interpolate PAX8 field values into SQL, especially company names feeding the `similarity()` comparison. |
|
||||
| V6 Cryptography | no | No new secrets/crypto surface this phase. |
|
||||
|
||||
### Known Threat Patterns for this stack
|
||||
|
||||
| Pattern | STRIDE | Standard Mitigation |
|
||||
|---------|--------|---------------------|
|
||||
| SQL injection via company name interpolation | Tampering | Always pass PAX8 company names as bound parameters (`$1`) to `similarity($1, company_name)`, never string-concatenated — matches existing `postgresClient.query(sql, params)` convention used throughout the codebase. |
|
||||
| Silent wrong-company cost attribution (a matching bug misattributes company A's costs to company B) | Tampering / Repudiation | D-01/D-02's conservative-threshold-plus-tie-detection design is itself the mitigation — this is a data-integrity concern specific to this phase's domain (billing reconciliation), not a generic security control. Cover with the "idempotent re-scoring never overwrites resolved_at" test (SC#4) and the near-tie unit test. |
|
||||
| PAX8 credential exposure in logs/errors | Information Disclosure | Already covered by the existing `pax8-client.test.ts` assertion (`rejects.not.toThrow(new RegExp(SECRET))`) — no new surface introduced by this phase; extend the same discipline to any new error paths in `listAllInvoices`/`listAllInvoiceItems`. |
|
||||
|
||||
## Sources
|
||||
|
||||
### Primary (HIGH confidence)
|
||||
- **Live authenticated API calls** to `https://api.pax8.com/v1/invoices`,
|
||||
`/v1/invoices/{id}/items`, and `/v1/orders`, executed 2026-07-11 using
|
||||
this project's real `PAX8_CLIENT_ID`/`PAX8_CLIENT_SECRET` from
|
||||
`.env.local` — the single highest-confidence source available, superior
|
||||
to documentation since it reflects this exact account's actual data
|
||||
shape.
|
||||
- **Live SQL queries** against the `pulse-postgres` Docker container's real
|
||||
`pax8_companies` (118 rows) and `companies` (242 active rows) tables,
|
||||
2026-07-11, including a live `CREATE EXTENSION IF NOT EXISTS pg_trgm;`
|
||||
and multiple `similarity()` distribution queries.
|
||||
- `lib/services/device-link-reconciler.ts` (full file read) — direct
|
||||
structural precedent for match/review/confidence logic.
|
||||
- `lib/services/pax8-client.ts`, `lib/services/pax8-sync-service.ts`,
|
||||
`lib/types/pax8.ts` (full files read) — existing extension points.
|
||||
- `migrations/091_pax8_tables.sql`, `migrations/080_device_xref_company_id.sql`,
|
||||
`migrations/001_initial_schema.sql`, `migrations/069/070_*.sql` (full
|
||||
files read) — current schema and extension-enablement precedent.
|
||||
- `.planning/phases/11-company-catalog-subscription-sync/11-PATTERNS.md`,
|
||||
`11-03-SUMMARY.md` (full files read) — Phase 11's established
|
||||
`Pax8SyncService` shape and result types.
|
||||
- `lib/services/pax8-client.test.ts`, `vitest.config.ts` (full files read)
|
||||
— existing test conventions.
|
||||
|
||||
### Secondary (MEDIUM confidence)
|
||||
- [devx.pax8.com — List Invoices](https://devx.pax8.com/reference/findpartnerinvoices)
|
||||
and [List Invoice Items](https://devx.pax8.com/reference/findpartnerinvoiceitems)
|
||||
reference pages — confirmed pagination defaults (page 0, size 10 default/
|
||||
200 max) and available filter params (`companyId`, `status`,
|
||||
`invoiceDateRangeStart/End`, etc.); did not surface full field-level
|
||||
schema (page content is JS-rendered), which is why the live API calls
|
||||
above were used as the ground-truth source instead.
|
||||
|
||||
### Tertiary (LOW confidence)
|
||||
- [lwhitelock/Pax8API GitHub repo](https://github.com/lwhitelock/Pax8API) —
|
||||
community PowerShell module, consulted only to corroborate that
|
||||
`Get-Pax8Invoices`/`Get-Pax8InvoiceItems` exist as community-recognized
|
||||
operations; no field-level detail relied upon from this source.
|
||||
|
||||
## Metadata
|
||||
|
||||
**Confidence breakdown:**
|
||||
- Standard stack (`pg_trgm`, threshold=0.90): HIGH — verified live against
|
||||
this project's actual Postgres instance and actual production-like data,
|
||||
not assumed.
|
||||
- Architecture (invoice/item schema gap, nested pagination pattern): HIGH —
|
||||
verified live against the actual PAX8 API for this account.
|
||||
- Pitfalls: HIGH for Pitfalls 1, 3, 4, 5 (all directly reproduced live);
|
||||
MEDIUM for Pitfall 2's exact `total`/`subTotal`/`amountDue` semantic
|
||||
mapping (see Assumption A1 / Open Question 1 — one sample invoice
|
||||
inspected, not exhaustively cross-checked against a prorated line).
|
||||
|
||||
**Research date:** 2026-07-11
|
||||
**Valid until:** 30 days (stable domain — PAX8's public API and this
|
||||
project's own schema/data don't change quickly; re-verify the `total` vs
|
||||
`amountDue` mapping (Open Question 1) before implementation regardless of
|
||||
elapsed time, since it wasn't fully closed out here).
|
||||
Loading…
Add table
Add a link
Reference in a new issue