docs(11): capture phase context
This commit is contained in:
parent
72ffcc230c
commit
e557e7f4a6
2 changed files with 289 additions and 0 deletions
|
|
@ -0,0 +1,190 @@
|
|||
# Phase 11: Company, Catalog & Subscription Sync - Context
|
||||
|
||||
**Gathered:** 2026-07-10
|
||||
**Status:** Ready for planning
|
||||
|
||||
<domain>
|
||||
## Phase Boundary
|
||||
|
||||
PAX8 companies, the product/SKU catalog, and current subscriptions are synced
|
||||
into Postgres, human-readable (joined to catalog, not bare SKU IDs) — the
|
||||
"current state" half of the integration. No orders/invoices (Phase 12), no
|
||||
company-matching to Autotask (Phase 12), no scheduler wiring (Phase 13), no
|
||||
`/pax8` UI (Phase 14). This phase is sync-service logic only, writing into the
|
||||
tables `10-01`/`10-02` already created (`pax8_companies`, `pax8_products`,
|
||||
`pax8_subscriptions`).
|
||||
|
||||
</domain>
|
||||
|
||||
<decisions>
|
||||
## Implementation Decisions
|
||||
|
||||
### Product Catalog Scope
|
||||
- **D-01:** Lazy/referenced-only catalog sync — only fetch and store
|
||||
`pax8_products` entries for SKUs that actually appear in at least one
|
||||
synced subscription. Do not sync PAX8's full catalog independent of what's
|
||||
referenced.
|
||||
- **D-02:** When a subscription references a product ID not yet in the local
|
||||
`pax8_products` table, fetch it inline during the same sync run (call
|
||||
PAX8's product-detail endpoint for that SKU, upsert, then continue) — not a
|
||||
separate two-pass batch step.
|
||||
|
||||
### Subscription Cost Fields
|
||||
- **D-03:** Store both PAX8's list/retail price and the actual partner/
|
||||
reseller cost as separate columns on `pax8_subscriptions`, if PAX8's API
|
||||
exposes both per-subscription. This directly serves the milestone's stated
|
||||
margin/profitability reconciliation goal (see SEED-002). If research
|
||||
reveals PAX8 only exposes one of the two at the subscription level, note
|
||||
the gap rather than fabricating the missing figure.
|
||||
- **D-04:** Cost columns store the raw per-billing-period amount exactly as
|
||||
PAX8 returns it (paired with the existing `billing_term` field) — no
|
||||
monthly-equivalent normalization at sync time. Any "normalize to monthly"
|
||||
math is a read-time/API-layer concern for a later phase (Phase 14's
|
||||
`/pax8` page or SEED-003's data assistant), not this sync service.
|
||||
|
||||
### Stale/Removed Entity Handling
|
||||
- **D-05:** Soft-delete convention — when a company, subscription, or
|
||||
catalog entry that existed in a prior sync no longer comes back from PAX8,
|
||||
mark it `is_deleted = true`, `deleted_at = now()` rather than removing the
|
||||
row. Matches the audit-column convention already used elsewhere in Pulse
|
||||
(Autotask tables, per `CLAUDE.md`'s "Audit columns convention").
|
||||
- **D-06:** Apply the soft-delete pattern consistently across all three
|
||||
tables touched by this phase — `pax8_companies`, `pax8_subscriptions`, AND
|
||||
`pax8_products` — not just companies/subscriptions. A cancelled
|
||||
subscription still needs to display its (possibly now-discontinued)
|
||||
product name, so catalog rows are never hard-deleted either.
|
||||
|
||||
### Sync Strategy
|
||||
- **D-07:** Incremental sync where PAX8's API supports a modified-since /
|
||||
delta mechanism, following the `entity-sync.ts` pattern already used for
|
||||
Autotask (read last-sync timestamp, request only changed records). If
|
||||
research during planning finds PAX8 has no such filter for a given entity
|
||||
type, fall back to full sync for that entity type specifically — this is
|
||||
a per-entity-type decision, not all-or-nothing across companies/
|
||||
subscriptions/catalog.
|
||||
- **D-08:** Reconcile incremental sync with reliable removal detection via a
|
||||
periodic full reconciliation pass — incremental syncs handle routine
|
||||
updates, but the sync service must also periodically (e.g., once per
|
||||
sync run, or on a longer cadence — planner's call on frequency) fetch the
|
||||
full list of company/subscription/product IDs currently in PAX8, diff
|
||||
against what's `is_deleted = false` in Postgres, and soft-delete anything
|
||||
no longer present. This directly satisfies the D-05/D-06 soft-delete
|
||||
decisions, which an incremental-only pull cannot detect on its own.
|
||||
|
||||
### Claude's Discretion
|
||||
- **Reconciliation-pass cadence** — whether the full-ID reconciliation pass
|
||||
(D-08) runs on every sync invocation or on a separate, less-frequent
|
||||
schedule is left to the planner, informed by whatever sync-trigger
|
||||
mechanism this phase builds (Phase 13 owns the actual cron wiring, but
|
||||
this phase's sync service should expose whatever hooks that eventual
|
||||
schedule needs).
|
||||
- **Missing partner-cost field fallback** — if research shows PAX8's
|
||||
subscription API doesn't expose reseller/partner cost distinctly from list
|
||||
price, the planner should decide the fallback column shape (e.g., a single
|
||||
`cost` column vs `list_price` + nullable `partner_cost`) rather than
|
||||
blocking on this ambiguity.
|
||||
|
||||
</decisions>
|
||||
|
||||
<canonical_refs>
|
||||
## Canonical References
|
||||
|
||||
**Downstream agents MUST read these before planning or implementing.**
|
||||
|
||||
### Project scope & requirements
|
||||
- `.planning/PROJECT.md` — Current Milestone: v2.0 PAX8 Integration section
|
||||
- `.planning/REQUIREMENTS.md` — PAX8-03, PAX8-04, PAX8-05, PAX8-08 (this
|
||||
phase's requirement IDs)
|
||||
- `.planning/seeds/SEED-002-pax8-integration.md` — original exploration:
|
||||
"cost reconciliation" and "margin/profitability reporting" as explicit
|
||||
goals (informs D-03); "build the Postgres schema with [SEED-003] in mind —
|
||||
clean, well-typed tables, avoid PAX8-API-shaped blobs"
|
||||
|
||||
### Prior phase (10) foundation this phase builds on
|
||||
- `.planning/phases/10-pax8-client-auth-foundation/10-CONTEXT.md` — records
|
||||
that product catalog scope (this phase's D-01/D-02) and raw_payload
|
||||
columns across all PAX8 tables were explicitly deferred here from Phase 10
|
||||
- `lib/services/pax8-client.ts` / `lib/services/pax8-factory.ts` — the
|
||||
OAuth2 client this phase's sync service must call (`getPax8Client()`)
|
||||
- `migrations/091_pax8_tables.sql` — existing schema for `pax8_companies`,
|
||||
`pax8_products`, `pax8_subscriptions`, `pax8_orders`, `pax8_order_items`,
|
||||
`pax8_company_match_review`; all four non-order tables already have
|
||||
`raw_payload JSONB` per the Phase 10 discretion decision — this phase
|
||||
populates them, does not alter schema unless a genuine gap is found
|
||||
- `.planning/phases/10-pax8-client-auth-foundation/10-RESEARCH.md` — Phase
|
||||
10's PAX8 API research; check first before re-researching auth/base-URL
|
||||
basics in Phase 11's own research pass
|
||||
|
||||
### Existing patterns to follow
|
||||
- `lib/services/entity-sync.ts` — canonical "last sync timestamp +
|
||||
incremental pull + batch upsert" pattern (D-07 follows this shape)
|
||||
- `lib/services/itglue-sync-service.ts`, `lib/services/veeam-sync-service.ts`
|
||||
— sibling sync-service examples for structure/conventions
|
||||
- `postgresClient.bulkUpsert()` — batch upsert method, per
|
||||
`lib/services/postgres-client.ts`
|
||||
- Audit columns convention (`created_at`, `updated_at`, `synced_at`,
|
||||
`is_deleted`, `deleted_at`) — per `CLAUDE.md`; D-05/D-06 rely on this
|
||||
already-established convention, columns already exist on the Phase 10
|
||||
schema
|
||||
- `/api/<name>/sync` fire-and-forget POST pattern (e.g., `/api/veeam/sync`,
|
||||
`/api/itglue/sync`) — likely shape for this phase's sync trigger endpoint,
|
||||
though scheduler cron wiring itself is Phase 13's scope, not this phase's
|
||||
|
||||
</canonical_refs>
|
||||
|
||||
<code_context>
|
||||
## Existing Code Insights
|
||||
|
||||
### Reusable Assets
|
||||
- `lib/services/entity-sync.ts` — direct template for incremental sync
|
||||
orchestration (D-07/D-08)
|
||||
- `lib/services/msgraph-client.ts` pattern (already followed by
|
||||
`pax8-client.ts`) — no new auth work needed, `getPax8Client()` is ready
|
||||
- `postgresClient.bulkUpsert()` — batch write mechanism for all three
|
||||
entity types
|
||||
|
||||
### Established Patterns
|
||||
- Sync services live at `lib/services/<name>-sync-service.ts`; a
|
||||
`pax8-sync-service.ts` (or similarly named) is the expected new file per
|
||||
this convention
|
||||
- Fire-and-forget sync trigger: `POST /api/<name>/sync` returns immediately,
|
||||
sync runs async — matches Veeam/IT Glue/SentinelOne/Zoom/QBO precedent
|
||||
- Soft-delete via `is_deleted`/`deleted_at` columns already exists on the
|
||||
Phase 10 migration for all relevant tables — no new migration needed for
|
||||
D-05/D-06 unless research finds a gap
|
||||
|
||||
### Integration Points
|
||||
- New file: `lib/services/pax8-sync-service.ts` (or equivalent name per
|
||||
planner)
|
||||
- New route: `/api/pax8/sync` (POST, fire-and-forget) — trigger only, no
|
||||
scheduler registration yet (Phase 13)
|
||||
- No route/nav/UI integration in this phase (`UI hint: no` per ROADMAP.md)
|
||||
|
||||
</code_context>
|
||||
|
||||
<specifics>
|
||||
## Specific Ideas
|
||||
|
||||
No specific UI or behavioral references were given — this phase is sync
|
||||
service logic only. The eight numbered decisions above (D-01 through D-08)
|
||||
are the concrete specifics: lazy catalog sync with inline SKU fetch,
|
||||
dual cost columns (list + partner) stored per-billing-period as-is,
|
||||
consistent soft-delete across all three tables, and incremental sync with a
|
||||
periodic full-reconciliation pass for removal detection.
|
||||
|
||||
</specifics>
|
||||
|
||||
<deferred>
|
||||
## Deferred Ideas
|
||||
|
||||
None — discussion stayed within phase scope. (Orders/invoices, company
|
||||
matching, scheduler cron wiring, and the `/pax8` UI are already sequenced
|
||||
into Phases 12-14 per ROADMAP.md and REQUIREMENTS.md — not deferred from
|
||||
this discussion, just out of this phase's boundary.)
|
||||
|
||||
</deferred>
|
||||
|
||||
---
|
||||
|
||||
*Phase: 11-company-catalog-subscription-sync*
|
||||
*Context gathered: 2026-07-10*
|
||||
|
|
@ -0,0 +1,99 @@
|
|||
# Phase 11: Company, Catalog & Subscription Sync - Discussion Log
|
||||
|
||||
> **Audit trail only.** Do not use as input to planning, research, or execution agents.
|
||||
> Decisions are captured in CONTEXT.md — this log preserves the alternatives considered.
|
||||
|
||||
**Date:** 2026-07-10
|
||||
**Phase:** 11-company-catalog-subscription-sync
|
||||
**Areas discussed:** Product catalog scope, Subscription cost fields, Stale/removed entity handling, Sync strategy (full vs incremental)
|
||||
|
||||
---
|
||||
|
||||
## Product Catalog Scope
|
||||
|
||||
| Option | Description | Selected |
|
||||
|--------|-------------|----------|
|
||||
| Full catalog sync | Fetch and store PAX8's complete product catalog every sync, independent of subscriptions | |
|
||||
| Lazy/referenced-only | Only fetch/store catalog entries for SKUs referenced by a synced subscription | ✓ |
|
||||
| You decide | Let planner choose based on PAX8's actual catalog API shape | |
|
||||
|
||||
**User's choice:** Lazy/referenced-only
|
||||
**Notes:** Directly carries forward the gray area left open in Phase 10's CONTEXT.md discretion section.
|
||||
|
||||
| Option | Description | Selected |
|
||||
|--------|-------------|----------|
|
||||
| Fetch it inline during the same sync run | Call PAX8's product-detail endpoint immediately when an unrecognized SKU is hit | ✓ |
|
||||
| Two-pass sync | First pass collects referenced product IDs, second pass batch-fetches them | |
|
||||
|
||||
**User's choice:** Fetch it inline during the same sync run
|
||||
|
||||
---
|
||||
|
||||
## Subscription Cost Fields
|
||||
|
||||
| Option | Description | Selected |
|
||||
|--------|-------------|----------|
|
||||
| Your partner/reseller cost only | Store only what PAX8 bills the reseller | |
|
||||
| List price + partner cost, both | Store both as separate columns, if PAX8 exposes both | ✓ |
|
||||
| Whatever PAX8's subscription endpoint returns, typed generically | Don't presuppose the pricing model | |
|
||||
|
||||
**User's choice:** List price + partner cost, both
|
||||
**Notes:** Ties directly to SEED-002's stated margin/profitability reconciliation goal.
|
||||
|
||||
| Option | Description | Selected |
|
||||
|--------|-------------|----------|
|
||||
| Store as-is, normalize at read time | Keep raw per-billing-period amount; monthly-equivalent math happens later | ✓ |
|
||||
| Normalize to monthly at sync time | Compute and store a monthly-equivalent column at sync time | |
|
||||
|
||||
**User's choice:** Store as-is, normalize at read time
|
||||
|
||||
---
|
||||
|
||||
## Stale/Removed Entity Handling
|
||||
|
||||
| Option | Description | Selected |
|
||||
|--------|-------------|----------|
|
||||
| Soft-delete (is_deleted flag) | Mark is_deleted=true, deleted_at=now() instead of removing | ✓ |
|
||||
| Leave rows as-is (no tracking) | Don't track removal this phase | |
|
||||
|
||||
**User's choice:** Soft-delete (is_deleted flag)
|
||||
**Notes:** Matches existing Pulse audit-column convention.
|
||||
|
||||
| Option | Description | Selected |
|
||||
|--------|-------------|----------|
|
||||
| Companies + subscriptions only | Catalog stays as historical reference, never soft-deleted | |
|
||||
| All three tables consistently | Apply is_deleted/deleted_at to pax8_products too | ✓ |
|
||||
|
||||
**User's choice:** All three tables consistently
|
||||
|
||||
---
|
||||
|
||||
## Sync Strategy (Full vs Incremental)
|
||||
|
||||
| Option | Description | Selected |
|
||||
|--------|-------------|----------|
|
||||
| Full resync every run | Fetch and upsert everything each sync, no last-sync tracking | |
|
||||
| Incremental where PAX8 supports it | Follow entity-sync.ts pattern; modified-since filter, fall back to full if unsupported | ✓ |
|
||||
| You decide | Let planner/researcher decide after seeing PAX8's actual API | |
|
||||
|
||||
**User's choice:** Incremental where PAX8 supports it
|
||||
|
||||
**Follow-up tension surfaced:** Incremental sync alone can't detect entities PAX8 stops returning (no signal that something is now missing), which conflicts with the soft-delete decision above.
|
||||
|
||||
| Option | Description | Selected |
|
||||
|--------|-------------|----------|
|
||||
| Periodic full reconciliation pass | Incremental for routine updates + periodic full ID-listing diff to catch removals | ✓ |
|
||||
| Full sync only for existence, incremental for details | Always fetch full ID list, but incremental-fetch only changed record details | |
|
||||
|
||||
**User's choice:** Periodic full reconciliation pass
|
||||
|
||||
---
|
||||
|
||||
## Claude's Discretion
|
||||
|
||||
- **Reconciliation-pass cadence** — whether the full-ID reconciliation pass runs every sync invocation or on a separate schedule, informed by whatever trigger mechanism this phase builds (Phase 13 owns actual cron wiring).
|
||||
- **Missing partner-cost field fallback** — if PAX8's API doesn't expose partner cost distinctly from list price, planner decides the fallback column shape.
|
||||
|
||||
## Deferred Ideas
|
||||
|
||||
None — discussion stayed within phase scope.
|
||||
Loading…
Add table
Add a link
Reference in a new issue