docs(11): capture phase context
This commit is contained in:
parent
72ffcc230c
commit
e557e7f4a6
2 changed files with 289 additions and 0 deletions
|
|
@ -0,0 +1,190 @@
|
||||||
|
# Phase 11: Company, Catalog & Subscription Sync - Context
|
||||||
|
|
||||||
|
**Gathered:** 2026-07-10
|
||||||
|
**Status:** Ready for planning
|
||||||
|
|
||||||
|
<domain>
|
||||||
|
## Phase Boundary
|
||||||
|
|
||||||
|
PAX8 companies, the product/SKU catalog, and current subscriptions are synced
|
||||||
|
into Postgres, human-readable (joined to catalog, not bare SKU IDs) — the
|
||||||
|
"current state" half of the integration. No orders/invoices (Phase 12), no
|
||||||
|
company-matching to Autotask (Phase 12), no scheduler wiring (Phase 13), no
|
||||||
|
`/pax8` UI (Phase 14). This phase is sync-service logic only, writing into the
|
||||||
|
tables `10-01`/`10-02` already created (`pax8_companies`, `pax8_products`,
|
||||||
|
`pax8_subscriptions`).
|
||||||
|
|
||||||
|
</domain>
|
||||||
|
|
||||||
|
<decisions>
|
||||||
|
## Implementation Decisions
|
||||||
|
|
||||||
|
### Product Catalog Scope
|
||||||
|
- **D-01:** Lazy/referenced-only catalog sync — only fetch and store
|
||||||
|
`pax8_products` entries for SKUs that actually appear in at least one
|
||||||
|
synced subscription. Do not sync PAX8's full catalog independent of what's
|
||||||
|
referenced.
|
||||||
|
- **D-02:** When a subscription references a product ID not yet in the local
|
||||||
|
`pax8_products` table, fetch it inline during the same sync run (call
|
||||||
|
PAX8's product-detail endpoint for that SKU, upsert, then continue) — not a
|
||||||
|
separate two-pass batch step.
|
||||||
|
|
||||||
|
### Subscription Cost Fields
|
||||||
|
- **D-03:** Store both PAX8's list/retail price and the actual partner/
|
||||||
|
reseller cost as separate columns on `pax8_subscriptions`, if PAX8's API
|
||||||
|
exposes both per-subscription. This directly serves the milestone's stated
|
||||||
|
margin/profitability reconciliation goal (see SEED-002). If research
|
||||||
|
reveals PAX8 only exposes one of the two at the subscription level, note
|
||||||
|
the gap rather than fabricating the missing figure.
|
||||||
|
- **D-04:** Cost columns store the raw per-billing-period amount exactly as
|
||||||
|
PAX8 returns it (paired with the existing `billing_term` field) — no
|
||||||
|
monthly-equivalent normalization at sync time. Any "normalize to monthly"
|
||||||
|
math is a read-time/API-layer concern for a later phase (Phase 14's
|
||||||
|
`/pax8` page or SEED-003's data assistant), not this sync service.
|
||||||
|
|
||||||
|
### Stale/Removed Entity Handling
|
||||||
|
- **D-05:** Soft-delete convention — when a company, subscription, or
|
||||||
|
catalog entry that existed in a prior sync no longer comes back from PAX8,
|
||||||
|
mark it `is_deleted = true`, `deleted_at = now()` rather than removing the
|
||||||
|
row. Matches the audit-column convention already used elsewhere in Pulse
|
||||||
|
(Autotask tables, per `CLAUDE.md`'s "Audit columns convention").
|
||||||
|
- **D-06:** Apply the soft-delete pattern consistently across all three
|
||||||
|
tables touched by this phase — `pax8_companies`, `pax8_subscriptions`, AND
|
||||||
|
`pax8_products` — not just companies/subscriptions. A cancelled
|
||||||
|
subscription still needs to display its (possibly now-discontinued)
|
||||||
|
product name, so catalog rows are never hard-deleted either.
|
||||||
|
|
||||||
|
### Sync Strategy
|
||||||
|
- **D-07:** Incremental sync where PAX8's API supports a modified-since /
|
||||||
|
delta mechanism, following the `entity-sync.ts` pattern already used for
|
||||||
|
Autotask (read last-sync timestamp, request only changed records). If
|
||||||
|
research during planning finds PAX8 has no such filter for a given entity
|
||||||
|
type, fall back to full sync for that entity type specifically — this is
|
||||||
|
a per-entity-type decision, not all-or-nothing across companies/
|
||||||
|
subscriptions/catalog.
|
||||||
|
- **D-08:** Reconcile incremental sync with reliable removal detection via a
|
||||||
|
periodic full reconciliation pass — incremental syncs handle routine
|
||||||
|
updates, but the sync service must also periodically (e.g., once per
|
||||||
|
sync run, or on a longer cadence — planner's call on frequency) fetch the
|
||||||
|
full list of company/subscription/product IDs currently in PAX8, diff
|
||||||
|
against what's `is_deleted = false` in Postgres, and soft-delete anything
|
||||||
|
no longer present. This directly satisfies the D-05/D-06 soft-delete
|
||||||
|
decisions, which an incremental-only pull cannot detect on its own.
|
||||||
|
|
||||||
|
### Claude's Discretion
|
||||||
|
- **Reconciliation-pass cadence** — whether the full-ID reconciliation pass
|
||||||
|
(D-08) runs on every sync invocation or on a separate, less-frequent
|
||||||
|
schedule is left to the planner, informed by whatever sync-trigger
|
||||||
|
mechanism this phase builds (Phase 13 owns the actual cron wiring, but
|
||||||
|
this phase's sync service should expose whatever hooks that eventual
|
||||||
|
schedule needs).
|
||||||
|
- **Missing partner-cost field fallback** — if research shows PAX8's
|
||||||
|
subscription API doesn't expose reseller/partner cost distinctly from list
|
||||||
|
price, the planner should decide the fallback column shape (e.g., a single
|
||||||
|
`cost` column vs `list_price` + nullable `partner_cost`) rather than
|
||||||
|
blocking on this ambiguity.
|
||||||
|
|
||||||
|
</decisions>
|
||||||
|
|
||||||
|
<canonical_refs>
|
||||||
|
## Canonical References
|
||||||
|
|
||||||
|
**Downstream agents MUST read these before planning or implementing.**
|
||||||
|
|
||||||
|
### Project scope & requirements
|
||||||
|
- `.planning/PROJECT.md` — Current Milestone: v2.0 PAX8 Integration section
|
||||||
|
- `.planning/REQUIREMENTS.md` — PAX8-03, PAX8-04, PAX8-05, PAX8-08 (this
|
||||||
|
phase's requirement IDs)
|
||||||
|
- `.planning/seeds/SEED-002-pax8-integration.md` — original exploration:
|
||||||
|
"cost reconciliation" and "margin/profitability reporting" as explicit
|
||||||
|
goals (informs D-03); "build the Postgres schema with [SEED-003] in mind —
|
||||||
|
clean, well-typed tables, avoid PAX8-API-shaped blobs"
|
||||||
|
|
||||||
|
### Prior phase (10) foundation this phase builds on
|
||||||
|
- `.planning/phases/10-pax8-client-auth-foundation/10-CONTEXT.md` — records
|
||||||
|
that product catalog scope (this phase's D-01/D-02) and raw_payload
|
||||||
|
columns across all PAX8 tables were explicitly deferred here from Phase 10
|
||||||
|
- `lib/services/pax8-client.ts` / `lib/services/pax8-factory.ts` — the
|
||||||
|
OAuth2 client this phase's sync service must call (`getPax8Client()`)
|
||||||
|
- `migrations/091_pax8_tables.sql` — existing schema for `pax8_companies`,
|
||||||
|
`pax8_products`, `pax8_subscriptions`, `pax8_orders`, `pax8_order_items`,
|
||||||
|
`pax8_company_match_review`; all four non-order tables already have
|
||||||
|
`raw_payload JSONB` per the Phase 10 discretion decision — this phase
|
||||||
|
populates them, does not alter schema unless a genuine gap is found
|
||||||
|
- `.planning/phases/10-pax8-client-auth-foundation/10-RESEARCH.md` — Phase
|
||||||
|
10's PAX8 API research; check first before re-researching auth/base-URL
|
||||||
|
basics in Phase 11's own research pass
|
||||||
|
|
||||||
|
### Existing patterns to follow
|
||||||
|
- `lib/services/entity-sync.ts` — canonical "last sync timestamp +
|
||||||
|
incremental pull + batch upsert" pattern (D-07 follows this shape)
|
||||||
|
- `lib/services/itglue-sync-service.ts`, `lib/services/veeam-sync-service.ts`
|
||||||
|
— sibling sync-service examples for structure/conventions
|
||||||
|
- `postgresClient.bulkUpsert()` — batch upsert method, per
|
||||||
|
`lib/services/postgres-client.ts`
|
||||||
|
- Audit columns convention (`created_at`, `updated_at`, `synced_at`,
|
||||||
|
`is_deleted`, `deleted_at`) — per `CLAUDE.md`; D-05/D-06 rely on this
|
||||||
|
already-established convention, columns already exist on the Phase 10
|
||||||
|
schema
|
||||||
|
- `/api/<name>/sync` fire-and-forget POST pattern (e.g., `/api/veeam/sync`,
|
||||||
|
`/api/itglue/sync`) — likely shape for this phase's sync trigger endpoint,
|
||||||
|
though scheduler cron wiring itself is Phase 13's scope, not this phase's
|
||||||
|
|
||||||
|
</canonical_refs>
|
||||||
|
|
||||||
|
<code_context>
|
||||||
|
## Existing Code Insights
|
||||||
|
|
||||||
|
### Reusable Assets
|
||||||
|
- `lib/services/entity-sync.ts` — direct template for incremental sync
|
||||||
|
orchestration (D-07/D-08)
|
||||||
|
- `lib/services/msgraph-client.ts` pattern (already followed by
|
||||||
|
`pax8-client.ts`) — no new auth work needed, `getPax8Client()` is ready
|
||||||
|
- `postgresClient.bulkUpsert()` — batch write mechanism for all three
|
||||||
|
entity types
|
||||||
|
|
||||||
|
### Established Patterns
|
||||||
|
- Sync services live at `lib/services/<name>-sync-service.ts`; a
|
||||||
|
`pax8-sync-service.ts` (or similarly named) is the expected new file per
|
||||||
|
this convention
|
||||||
|
- Fire-and-forget sync trigger: `POST /api/<name>/sync` returns immediately,
|
||||||
|
sync runs async — matches Veeam/IT Glue/SentinelOne/Zoom/QBO precedent
|
||||||
|
- Soft-delete via `is_deleted`/`deleted_at` columns already exists on the
|
||||||
|
Phase 10 migration for all relevant tables — no new migration needed for
|
||||||
|
D-05/D-06 unless research finds a gap
|
||||||
|
|
||||||
|
### Integration Points
|
||||||
|
- New file: `lib/services/pax8-sync-service.ts` (or equivalent name per
|
||||||
|
planner)
|
||||||
|
- New route: `/api/pax8/sync` (POST, fire-and-forget) — trigger only, no
|
||||||
|
scheduler registration yet (Phase 13)
|
||||||
|
- No route/nav/UI integration in this phase (`UI hint: no` per ROADMAP.md)
|
||||||
|
|
||||||
|
</code_context>
|
||||||
|
|
||||||
|
<specifics>
|
||||||
|
## Specific Ideas
|
||||||
|
|
||||||
|
No specific UI or behavioral references were given — this phase is sync
|
||||||
|
service logic only. The eight numbered decisions above (D-01 through D-08)
|
||||||
|
are the concrete specifics: lazy catalog sync with inline SKU fetch,
|
||||||
|
dual cost columns (list + partner) stored per-billing-period as-is,
|
||||||
|
consistent soft-delete across all three tables, and incremental sync with a
|
||||||
|
periodic full-reconciliation pass for removal detection.
|
||||||
|
|
||||||
|
</specifics>
|
||||||
|
|
||||||
|
<deferred>
|
||||||
|
## Deferred Ideas
|
||||||
|
|
||||||
|
None — discussion stayed within phase scope. (Orders/invoices, company
|
||||||
|
matching, scheduler cron wiring, and the `/pax8` UI are already sequenced
|
||||||
|
into Phases 12-14 per ROADMAP.md and REQUIREMENTS.md — not deferred from
|
||||||
|
this discussion, just out of this phase's boundary.)
|
||||||
|
|
||||||
|
</deferred>
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
*Phase: 11-company-catalog-subscription-sync*
|
||||||
|
*Context gathered: 2026-07-10*
|
||||||
|
|
@ -0,0 +1,99 @@
|
||||||
|
# Phase 11: Company, Catalog & Subscription Sync - Discussion Log
|
||||||
|
|
||||||
|
> **Audit trail only.** Do not use as input to planning, research, or execution agents.
|
||||||
|
> Decisions are captured in CONTEXT.md — this log preserves the alternatives considered.
|
||||||
|
|
||||||
|
**Date:** 2026-07-10
|
||||||
|
**Phase:** 11-company-catalog-subscription-sync
|
||||||
|
**Areas discussed:** Product catalog scope, Subscription cost fields, Stale/removed entity handling, Sync strategy (full vs incremental)
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Product Catalog Scope
|
||||||
|
|
||||||
|
| Option | Description | Selected |
|
||||||
|
|--------|-------------|----------|
|
||||||
|
| Full catalog sync | Fetch and store PAX8's complete product catalog every sync, independent of subscriptions | |
|
||||||
|
| Lazy/referenced-only | Only fetch/store catalog entries for SKUs referenced by a synced subscription | ✓ |
|
||||||
|
| You decide | Let planner choose based on PAX8's actual catalog API shape | |
|
||||||
|
|
||||||
|
**User's choice:** Lazy/referenced-only
|
||||||
|
**Notes:** Directly carries forward the gray area left open in Phase 10's CONTEXT.md discretion section.
|
||||||
|
|
||||||
|
| Option | Description | Selected |
|
||||||
|
|--------|-------------|----------|
|
||||||
|
| Fetch it inline during the same sync run | Call PAX8's product-detail endpoint immediately when an unrecognized SKU is hit | ✓ |
|
||||||
|
| Two-pass sync | First pass collects referenced product IDs, second pass batch-fetches them | |
|
||||||
|
|
||||||
|
**User's choice:** Fetch it inline during the same sync run
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Subscription Cost Fields
|
||||||
|
|
||||||
|
| Option | Description | Selected |
|
||||||
|
|--------|-------------|----------|
|
||||||
|
| Your partner/reseller cost only | Store only what PAX8 bills the reseller | |
|
||||||
|
| List price + partner cost, both | Store both as separate columns, if PAX8 exposes both | ✓ |
|
||||||
|
| Whatever PAX8's subscription endpoint returns, typed generically | Don't presuppose the pricing model | |
|
||||||
|
|
||||||
|
**User's choice:** List price + partner cost, both
|
||||||
|
**Notes:** Ties directly to SEED-002's stated margin/profitability reconciliation goal.
|
||||||
|
|
||||||
|
| Option | Description | Selected |
|
||||||
|
|--------|-------------|----------|
|
||||||
|
| Store as-is, normalize at read time | Keep raw per-billing-period amount; monthly-equivalent math happens later | ✓ |
|
||||||
|
| Normalize to monthly at sync time | Compute and store a monthly-equivalent column at sync time | |
|
||||||
|
|
||||||
|
**User's choice:** Store as-is, normalize at read time
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Stale/Removed Entity Handling
|
||||||
|
|
||||||
|
| Option | Description | Selected |
|
||||||
|
|--------|-------------|----------|
|
||||||
|
| Soft-delete (is_deleted flag) | Mark is_deleted=true, deleted_at=now() instead of removing | ✓ |
|
||||||
|
| Leave rows as-is (no tracking) | Don't track removal this phase | |
|
||||||
|
|
||||||
|
**User's choice:** Soft-delete (is_deleted flag)
|
||||||
|
**Notes:** Matches existing Pulse audit-column convention.
|
||||||
|
|
||||||
|
| Option | Description | Selected |
|
||||||
|
|--------|-------------|----------|
|
||||||
|
| Companies + subscriptions only | Catalog stays as historical reference, never soft-deleted | |
|
||||||
|
| All three tables consistently | Apply is_deleted/deleted_at to pax8_products too | ✓ |
|
||||||
|
|
||||||
|
**User's choice:** All three tables consistently
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Sync Strategy (Full vs Incremental)
|
||||||
|
|
||||||
|
| Option | Description | Selected |
|
||||||
|
|--------|-------------|----------|
|
||||||
|
| Full resync every run | Fetch and upsert everything each sync, no last-sync tracking | |
|
||||||
|
| Incremental where PAX8 supports it | Follow entity-sync.ts pattern; modified-since filter, fall back to full if unsupported | ✓ |
|
||||||
|
| You decide | Let planner/researcher decide after seeing PAX8's actual API | |
|
||||||
|
|
||||||
|
**User's choice:** Incremental where PAX8 supports it
|
||||||
|
|
||||||
|
**Follow-up tension surfaced:** Incremental sync alone can't detect entities PAX8 stops returning (no signal that something is now missing), which conflicts with the soft-delete decision above.
|
||||||
|
|
||||||
|
| Option | Description | Selected |
|
||||||
|
|--------|-------------|----------|
|
||||||
|
| Periodic full reconciliation pass | Incremental for routine updates + periodic full ID-listing diff to catch removals | ✓ |
|
||||||
|
| Full sync only for existence, incremental for details | Always fetch full ID list, but incremental-fetch only changed record details | |
|
||||||
|
|
||||||
|
**User's choice:** Periodic full reconciliation pass
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Claude's Discretion
|
||||||
|
|
||||||
|
- **Reconciliation-pass cadence** — whether the full-ID reconciliation pass runs every sync invocation or on a separate schedule, informed by whatever trigger mechanism this phase builds (Phase 13 owns actual cron wiring).
|
||||||
|
- **Missing partner-cost field fallback** — if PAX8's API doesn't expose partner cost distinctly from list price, planner decides the fallback column shape.
|
||||||
|
|
||||||
|
## Deferred Ideas
|
||||||
|
|
||||||
|
None — discussion stayed within phase scope.
|
||||||
Loading…
Add table
Add a link
Reference in a new issue