diff --git a/.planning/STATE.md b/.planning/STATE.md index 1d07cf4..88285c8 100644 --- a/.planning/STATE.md +++ b/.planning/STATE.md @@ -28,7 +28,7 @@ See: .planning/PROJECT.md (updated 2026-05-03) Phase: 09.1 (ntfy-backend-fix) — EXECUTING Plan: 1 of 1 Status: Executing Phase 09.1 -Last activity: 2026-05-21 - Completed quick task 260521-fci: Stopgap nightly reconciliation for stale open tickets in postgres mirror +Last activity: 2026-05-21 - Completed quick task 260521-foj: Fix weekly-full FK error: widen Companies filter + defensive ticket company_id validation Progress: [░░░░░░░░░░] 0% @@ -95,6 +95,7 @@ None yet. |---|-------------|------|--------|-----------| | 260519-0oz | Add QBO createPayment + createDeposit + .FH reconciliation script | 2026-05-19 | 5497458 | [260519-0oz-add-qbo-createpayment-createdeposit-fh-r](./quick/260519-0oz-add-qbo-createpayment-createdeposit-fh-r/) | | 260521-fci | Stopgap nightly reconciliation for stale open tickets in postgres mirror | 2026-05-21 | badd718 | [260521-fci-stopgap-nightly-reconciliation-for-stale](./quick/260521-fci-stopgap-nightly-reconciliation-for-stale/) | +| 260521-foj | Fix weekly-full FK error: widen Companies filter + defensive ticket company_id validation | 2026-05-21 | 62c529f | [260521-foj-fix-weekly-full-fk-error-widen-companies](./quick/260521-foj-fix-weekly-full-fk-error-widen-companies/) | ## Session Continuity diff --git a/.planning/quick/260521-foj-fix-weekly-full-fk-error-widen-companies/260521-foj-PLAN.md b/.planning/quick/260521-foj-fix-weekly-full-fk-error-widen-companies/260521-foj-PLAN.md new file mode 100644 index 0000000..846a760 --- /dev/null +++ b/.planning/quick/260521-foj-fix-weekly-full-fk-error-widen-companies/260521-foj-PLAN.md @@ -0,0 +1,373 @@ +--- +phase: quick-260521-foj +plan: 01 +type: execute +wave: 1 +depends_on: [] +files_modified: + - lib/utils/sync-helpers.ts + - lib/services/entity-sync.ts +autonomous: true +requirements: + - QUICK-260521-FOJ +must_haves: + truths: + - "Full sync of Companies entity fetches all companies (active + inactive) — no isActive=true filter applied" + - "Tickets sync with a company_id pointing at a company missing from the Pulse mirror does not raise tickets_company_id_fkey; the company_id is nullified and the ticket is preserved" + - "`hasAppliedFilters` remains true on Companies full sync, so soft-delete of companies is NOT triggered" + - "Existing behavior unchanged for all other entities — getActiveField/buildActiveFilter still works as before for resources, contacts, statuses, etc." + - "`npx tsc --noEmit --pretty` passes after both edits" + artifacts: + - path: "lib/utils/sync-helpers.ts" + provides: "New exported `buildCompaniesFilter()` returning `[{ field: 'id', op: 'gt', value: 0 }]`" + contains: "export function buildCompaniesFilter" + - path: "lib/services/entity-sync.ts" + provides: "New `getValidCompanyIds()` private method + COMPANIES branch in full-sync filter chain + ticket company_id nullification block" + contains: "buildCompaniesFilter" + key_links: + - from: "lib/services/entity-sync.ts (full-sync filter chain ~line 133-165)" + to: "buildCompaniesFilter (lib/utils/sync-helpers.ts)" + via: "import + branch above the generic `else` that calls buildActiveFilter" + pattern: "entity === EntityType.COMPANIES.*buildCompaniesFilter" + - from: "lib/services/entity-sync.ts (ticket validation block)" + to: "companies table (is_deleted = false)" + via: "getValidCompanyIds() called inside the `if (entity === EntityType.TICKETS)` block" + pattern: "getValidCompanyIds" +--- + + +Fix the weekly-full and full ticket sync FK error (`tickets_company_id_fkey`) caused by Companies sync filtering out inactive companies that nonetheless have tickets in the same sync window. + +Purpose: Every full and weekly-full sync since 2026-05-15 fails because Companies full sync applies `isActive=true` (via `buildActiveFilter`), which misses inactive companies that have tickets. The ticket transaction rolls back on the FK violation. Live evidence in `sync_history`: every recent full sync shows `[DATABASE_CONSTRAINT_ERROR] ... tickets_company_id_fkey`. + +Output: +- A new `buildCompaniesFilter()` in `lib/utils/sync-helpers.ts` mirroring the `buildProjectPhasesFilter` template (`id > 0` to fetch all). +- A new branch in `entity-sync.ts` full-sync filter chain that routes COMPANIES through `buildCompaniesFilter()` instead of the generic `buildActiveFilter`. +- A defensive ticket `company_id` nullification block in `entity-sync.ts` (belt-and-suspenders for the rare case of truly hard-deleted Autotask companies). + + + +@$HOME/.claude/get-shit-done/workflows/execute-plan.md +@$HOME/.claude/get-shit-done/templates/summary.md + + + +@./CLAUDE.md +@lib/utils/sync-helpers.ts +@lib/services/entity-sync.ts +@lib/services/postgres-client.ts + + + + +From `lib/utils/sync-helpers.ts` — existing filter-builder templates this task must mirror: +```ts +/** + * Build filter for contract services (requires contractID filter — fetch all via contractIDs) + * @returns Query filter array for active contract services + */ +export function buildContractServicesFilter(): Array<{ field: string; op: string; value: any }> { + return [ + { + field: 'contractID', + op: 'gt', + value: 0, + }, + ]; +} + +/** + * Build filter for project phases (Phases endpoint requires a filter) + * @returns Query filter array for all phases + */ +export function buildProjectPhasesFilter(): Array<{ field: string; op: string; value: any }> { + return [ + { + field: 'id', + op: 'gt', + value: 0, + }, + ]; +} + +/** + * Get active status field name for entity + * Returns 'isActive' for COMPANIES — DO NOT change this mapping. It is still + * referenced by `buildActiveFilter` for other entities (resources, contacts, etc.). + */ +export function getActiveField(entity: EntityType): string | null { /* ... */ } +``` + +From `lib/services/entity-sync.ts` — the full-sync filter chain (~lines 130-170) is where the new COMPANIES branch goes: +```ts +const filters: Array<{ field: string; op: string; value: any }> = []; + +// Special handling for entities that require filters +if (entity === EntityType.CONTRACTS) { + filters.push(...buildContractsFilter()); + entityLogger.info('Full sync with status filter for active contracts'); +} else if (entity === EntityType.CONTRACT_SERVICES) { + filters.push(...buildContractServicesFilter()); + entityLogger.info('Full sync of all contract services'); +} else if (entity === EntityType.PROJECTS) { + filters.push(...buildProjectsFilter()); + entityLogger.info('Full sync with status filter for non-completed projects'); +} else if (entity === EntityType.PROJECT_PHASES) { + filters.push(...buildProjectPhasesFilter()); + entityLogger.info('Full sync of all project phases'); +} else if (entity === EntityType.TIME_ENTRIES) { /* ... */ +} else if (entity === EntityType.BILLING_ITEMS) { /* ... */ +} else { + // Add active filter if applicable + const activeFilter = buildActiveFilter(entity); + if (activeFilter) { + filters.push(...activeFilter); + entityLogger.info('Full sync with active filter'); + } + // ... date range filter ... +} +``` + +From `lib/services/entity-sync.ts` — the existing `getValidResourceIds` template the new method must mirror (lines 710-718): +```ts +/** + * Get all valid resource IDs from the database + * Used to validate foreign key references before insert + * @returns Set of valid resource IDs + */ +private async getValidResourceIds(): Promise> { + const query = 'SELECT id FROM resources WHERE is_deleted = false'; + const result = await postgresClient.query<{ id: number }>(query); + return new Set(result.rows.map(row => Number(row.id))); +} +``` + +From `lib/services/entity-sync.ts` — the existing tickets validation block where the new `company_id` validation slots in (lines 235-285): +- Lines 238-252: filters out tickets without `company_id` (`recordsWithoutCompany`). KEEP AS-IS. +- Lines 255-285: nullifies invalid `assigned_resource_id` / `first_response_*` resource FKs against `getValidResourceIds`. KEEP AS-IS. +- NEW VALIDATION goes immediately after the `recordsWithoutCompany` filter, before or alongside the resource validation block. Must use the same `mappedRecords = mappedRecords.map(...)` mutation pattern and the same `entityLogger.warn` shape. + +From `lib/services/postgres-client.ts` — singleton DB interface: +```ts +postgresClient.query<{ id: number }>(sql, params?: any[]): Promise> +``` + + + + + + + Task 1: Add buildCompaniesFilter and wire COMPANIES into the full-sync filter chain + lib/utils/sync-helpers.ts, lib/services/entity-sync.ts + +**Step 1 — `lib/utils/sync-helpers.ts`:** + +Add a new exported function `buildCompaniesFilter()` adjacent to `buildContractServicesFilter` / `buildProjectPhasesFilter` (around line 92, right after `buildContractServicesFilter`). Mirror the JSDoc style of `buildContractServicesFilter` exactly: + +```ts +/** + * Build filter for companies (Companies endpoint requires a filter — use id > 0 to fetch all + * companies regardless of isActive. Active-only filtering misses inactive companies that + * have tickets, causing tickets_company_id_fkey violations on full sync.) + * @returns Query filter array for all companies + */ +export function buildCompaniesFilter(): Array<{ field: string; op: string; value: any }> { + return [ + { + field: 'id', + op: 'gt', + value: 0, + }, + ]; +} +``` + +DO NOT touch `getActiveField(COMPANIES)` — leave it returning `'isActive'`. It's still referenced by `buildActiveFilter` for other entities and by call sites we are not changing. + +**Step 2 — `lib/services/entity-sync.ts`:** + +1. Update the sync-helpers import at the top of the file (currently lines 13-25). Add `buildCompaniesFilter` to the named imports — alphabetize alongside the other `build*` helpers. The import block becomes: + +```ts +import { + getAutotaskEntityName, + buildIncrementalFilter, + buildActiveFilter, + buildDateRangeFilter, + buildCompaniesFilter, + buildContractsFilter, + buildContractServicesFilter, + buildProjectsFilter, + buildProjectPhasesFilter, + buildTimeEntriesFilter, + buildBillingItemsFilter, + getTableName +} from '../utils/sync-helpers'; +``` + +2. In the full-sync filter chain (around lines 133-165), add a new branch for COMPANIES. Place it as the FIRST branch in the chain (just after the `const filters: Array<...> = [];` declaration and before the existing `if (entity === EntityType.CONTRACTS)` branch). The chain after edit: + +```ts +const filters: Array<{ field: string; op: string; value: any }> = []; + +// Special handling for entities that require filters +if (entity === EntityType.COMPANIES) { + filters.push(...buildCompaniesFilter()); + entityLogger.info('Full sync of all companies (active + inactive)'); +} else if (entity === EntityType.CONTRACTS) { + filters.push(...buildContractsFilter()); + entityLogger.info('Full sync with status filter for active contracts'); +} else if (entity === EntityType.CONTRACT_SERVICES) { + // ... existing branches unchanged ... +``` + +Critical: the new COMPANIES branch MUST execute BEFORE the generic `else` block that calls `buildActiveFilter`. Result: the active filter is no longer applied to Companies on full sync, so inactive companies are fetched too. + +**Why this is safe (state this in your verification reasoning, do not add it as a code comment):** +- `hasAppliedFilters` (entity-sync.ts:173) stays true because the new filter array is non-empty (`id > 0`). So `softDeleteMissingRecords` is NOT triggered for companies — no accidental mass soft-delete. +- The existing DB already has 145 `is_active=false` companies referenced by tickets; downstream UI/API code tolerates inactive companies. +- Companies don't support incremental sync (entity-sync.ts:106 `supportsIncremental` excludes COMPANIES), so this code path runs on every Companies sync — full, weekly-full, and the incremental→full fallback path. + +DO NOT touch: +- `getActiveField`. +- The `lastActivityDate` filter on tickets (out of scope). +- Any other branch in the filter chain. +- Adjacent formatting / unrelated code. + + + cd /opt/stacks/pulse && npx tsc --noEmit --pretty 2>&1 | tail -40 && echo "---" && grep -n "buildCompaniesFilter" lib/utils/sync-helpers.ts lib/services/entity-sync.ts && echo "---" && grep -n "entity === EntityType.COMPANIES" lib/services/entity-sync.ts + + +- `buildCompaniesFilter` exported from `lib/utils/sync-helpers.ts` with the exact JSDoc and body shown above. +- `buildCompaniesFilter` imported in `lib/services/entity-sync.ts` named-imports block. +- Exactly one new COMPANIES branch in the full-sync filter chain, placed before all other `if/else if` branches, with the `entityLogger.info('Full sync of all companies (active + inactive)')` line. +- `getActiveField` and `buildActiveFilter` definitions UNCHANGED. +- `npx tsc --noEmit --pretty` passes with zero errors. +- `grep -n "buildCompaniesFilter" lib/services/entity-sync.ts` shows exactly two matches (one import, one usage). + + + + + Task 2: Defensive company_id nullification in tickets sync (belt-and-suspenders) + lib/services/entity-sync.ts + +This task adds a second layer of defense after Task 1: even if a ticket somehow references a company that doesn't exist in the Pulse mirror (e.g. hard-deleted from Autotask), nullify the `company_id` rather than letting the FK violation roll back the whole ticket transaction. + +**Step 1 — Add new private method `getValidCompanyIds`:** + +Find the existing `getValidResourceIds` method (currently lines ~710-718, just after `calculateMonthlyChunks` is defined further down — search for `private async getValidResourceIds`). Add the new method immediately AFTER it, before `getValidContactIds`. Match the exact pattern: + +```ts + /** + * Get all valid company IDs from the database + * Used to validate foreign key references before insert + * @returns Set of valid company IDs + */ + private async getValidCompanyIds(): Promise> { + const query = 'SELECT id FROM companies WHERE is_deleted = false'; + const result = await postgresClient.query<{ id: number }>(query); + return new Set(result.rows.map(row => Number(row.id))); + } +``` + +**Step 2 — Add the company_id validation block in the tickets validation pipeline:** + +Find the existing `if (entity === EntityType.TICKETS) {` block that nullifies invalid resource IDs (currently lines ~254-285, the block that uses `getValidResourceIds`). Add a NEW `if (entity === EntityType.TICKETS) { ... }` block IMMEDIATELY ABOVE it (between the `recordsWithoutCompany` filter block at lines ~238-252 and the resource validation block at lines ~254-285). The new block: + +```ts + // Validate company_id foreign keys for tickets (defensive — Task 1 widening + // the Companies filter should make this a near-zero count, but catches the + // truly hard-deleted Autotask company case). + if (entity === EntityType.TICKETS) { + const validCompanyIds = await this.getValidCompanyIds(); + let nullifiedCompanyCount = 0; + const nullifiedCompanySamples: number[] = []; + mappedRecords = mappedRecords.map(ticket => { + if (ticket.company_id && !validCompanyIds.has(ticket.company_id)) { + if (nullifiedCompanySamples.length < 5) { + nullifiedCompanySamples.push(ticket.id); + } + ticket.company_id = null; + nullifiedCompanyCount++; + } + return ticket; + }); + if (nullifiedCompanyCount > 0) { + entityLogger.warn('Nullified ticket company_id for companies missing from mirror', { + nullifiedCount: nullifiedCompanyCount, + sampleTicketIds: nullifiedCompanySamples, + }); + } + } +``` + +Ordering rationale (must be respected): +1. `recordsWithoutCompany` filter (existing) — drops tickets without any `company_id`. Runs first so we only validate tickets that have a `company_id`. +2. NEW company_id nullification block — nullifies `company_id` that doesn't exist in mirror. Runs second. +3. Resource validation block (existing) — nullifies invalid resource refs. Runs third. +4. Everything else (`bulkUpsert` at ~line 372) runs after. + +The `tickets.company_id` column is nullable (confirmed via the recent stopgap reconciliation work and existing handling that filters rows without `company_id`). Nullifying it is non-destructive — it preserves the ticket row and just severs the company link, matching the resource-FK handling pattern already used in the same function. + +DO NOT touch: +- The existing `recordsWithoutCompany` filter block. +- The existing resource validation block. +- The chunked tickets sync (`syncTicketsChunked`) — out of scope. That path has its own (separate) flow; this fix is for `syncEntity`. +- `cachedValidResourceIds` or `cachedValidContactIds` fields. +- Adjacent code / formatting. + +DO NOT manually trigger a sync from this task. Note in SUMMARY.md only: +``` +Manual verify (user runs after merge): + curl -X POST http://localhost:3100/api/sync/full \ + -H 'content-type: application/json' \ + -d '{"triggeredBy":"manual-verify"}' +Then check `sync_history` for the next ticket sync row — it should complete without `tickets_company_id_fkey` in `error_message`. +``` + + + cd /opt/stacks/pulse && npx tsc --noEmit --pretty 2>&1 | tail -40 && echo "---" && grep -cn "getValidCompanyIds" lib/services/entity-sync.ts && echo "---" && grep -n "Nullified ticket company_id for companies missing from mirror" lib/services/entity-sync.ts && echo "---" && grep -n "if (entity === EntityType.TICKETS)" lib/services/entity-sync.ts + + +- `getValidCompanyIds` method exists exactly once in `lib/services/entity-sync.ts`, mirroring `getValidResourceIds`. +- A new `if (entity === EntityType.TICKETS)` block exists between the `recordsWithoutCompany` filter and the resource validation block. +- The new block calls `await this.getValidCompanyIds()` and logs via `entityLogger.warn('Nullified ticket company_id for companies missing from mirror', ...)` when count > 0. +- `grep -c "getValidCompanyIds" lib/services/entity-sync.ts` returns 2 (one definition, one call site). +- `grep -n "if (entity === EntityType.TICKETS)" lib/services/entity-sync.ts` returns at least 2 matches (the new block + the existing resource validation block); ordering must put the new block earlier in the file (lower line number) than the resource block. +- `npx tsc --noEmit --pretty` passes with zero errors. +- No edits to `syncTicketsChunked`, `getValidResourceIds`, `getValidContactIds`, or `getValidProjectIds`. + + + + + + +1. Type check passes: `npx tsc --noEmit --pretty` from repo root — zero errors. +2. Targeted greps: + - `grep -n "buildCompaniesFilter" lib/utils/sync-helpers.ts lib/services/entity-sync.ts` → 3 matches (one definition + import + usage). + - `grep -c "getValidCompanyIds" lib/services/entity-sync.ts` → 2. + - `grep -n "entity === EntityType.COMPANIES" lib/services/entity-sync.ts` → 1 new match in the filter chain. +3. Negative greps (proving we didn't touch out-of-scope code): + - `git diff lib/utils/sync-helpers.ts -- ':!**/buildCompaniesFilter*'` should show only the new function added. + - `git diff lib/services/entity-sync.ts` should show: import addition, COMPANIES filter branch, new `getValidCompanyIds` method, and new ticket company_id validation block. NOTHING else. +4. Read-back sanity: + - `entity-sync.ts` line ~152 (was the start of the generic `else` calling `buildActiveFilter`) is now NOT reached for COMPANIES because the new branch fires first. + - `hasAppliedFilters` (line ~173) is still true for COMPANIES full sync because `buildCompaniesFilter()` returns a non-empty filter. + + + +- `lib/utils/sync-helpers.ts` exports `buildCompaniesFilter()` returning `[{ field: 'id', op: 'gt', value: 0 }]`. +- `lib/services/entity-sync.ts` routes COMPANIES through `buildCompaniesFilter` on full sync; the generic `buildActiveFilter` branch is no longer hit for COMPANIES. +- `lib/services/entity-sync.ts` defensively nullifies ticket `company_id` for any company missing from the Pulse mirror, mirroring the existing resource-FK nullification pattern. +- `npx tsc --noEmit --pretty` passes. +- No changes to `getActiveField`, `buildActiveFilter`, `syncTicketsChunked`, the reconciliation service from quick-260521-fci, or the sync-progress-lock code. +- SUMMARY.md notes the manual `curl -X POST http://localhost:3100/api/sync/full` verification step but does NOT execute it. + + + +After completion, create `.planning/quick/260521-foj-fix-weekly-full-fk-error-widen-companies/260521-foj-SUMMARY.md` with: +- What changed in each file (1-2 lines per file). +- Why this fixes the FK error (1 paragraph: Companies sync was filtering on `isActive=true` via `buildActiveFilter`, missing inactive companies referenced by tickets; widening to `id > 0` closes the gap; the defensive nullification handles the residual hard-deleted-in-Autotask edge case). +- Manual verify step for the user: `curl -X POST http://localhost:3100/api/sync/full -H 'content-type: application/json' -d '{"triggeredBy":"manual-verify"}'`, then check `sync_history` for the next full ticket sync. +- Explicit note: did NOT modify `getActiveField`, `syncTicketsChunked`, or the reconciliation service. + diff --git a/.planning/quick/260521-foj-fix-weekly-full-fk-error-widen-companies/260521-foj-SUMMARY.md b/.planning/quick/260521-foj-fix-weekly-full-fk-error-widen-companies/260521-foj-SUMMARY.md new file mode 100644 index 0000000..7a95dfb --- /dev/null +++ b/.planning/quick/260521-foj-fix-weekly-full-fk-error-widen-companies/260521-foj-SUMMARY.md @@ -0,0 +1,122 @@ +--- +phase: quick-260521-foj +plan: 01 +subsystem: sync +tags: + - autotask-sync + - companies + - tickets + - foreign-key + - bugfix +requirements: + - QUICK-260521-FOJ +dependency_graph: + requires: + - lib/utils/sync-helpers.ts (existing build*Filter pattern) + - lib/services/entity-sync.ts (existing full-sync filter chain + tickets validation pipeline) + - lib/services/postgres-client.ts (singleton query interface) + provides: + - buildCompaniesFilter() (lib/utils/sync-helpers.ts) + - EntitySyncService.getValidCompanyIds() (private) + - Defensive ticket company_id nullification in syncEntity() + affects: + - Companies full sync (now fetches active + inactive) + - Tickets full + weekly-full sync (no longer rolls back on tickets_company_id_fkey) +tech_stack: + added: [] + patterns: + - Mirror existing buildProjectPhasesFilter / buildContractServicesFilter "id > 0" template for any Autotask entity that requires a filter but where we want everything + - Mirror existing getValidResourceIds + null-on-miss pattern for FK validation in syncEntity() +key_files: + created: [] + modified: + - lib/utils/sync-helpers.ts + - lib/services/entity-sync.ts +decisions: + - Widen Companies full sync to id > 0 (fetch all) rather than carrying isActive=true via buildActiveFilter, so inactive Autotask companies referenced by tickets are mirrored. + - Add a belt-and-suspenders ticket company_id nullification block for the truly hard-deleted Autotask company case. Non-destructive — preserves the ticket row. + - Did NOT touch getActiveField — still returns 'isActive' for COMPANIES because other entities and call sites use that mapping. + - Did NOT touch syncTicketsChunked — out of scope; that path has its own flow and is not on the failing weekly-full path. +metrics: + duration: 1m 49s + completed_date: 2026-05-21 +--- + +# Quick 260521-foj: Fix weekly-full FK error — widen Companies full sync Summary + +One-liner: Widen Companies full sync to fetch active+inactive companies (id > 0) and defensively nullify ticket.company_id for hard-deleted-in-Autotask companies — eliminates the `tickets_company_id_fkey` rollback that has been failing every full and weekly-full sync since 2026-05-15. + +## What changed + +**`lib/utils/sync-helpers.ts`** +- Added new exported `buildCompaniesFilter()` returning `[{ field: 'id', op: 'gt', value: 0 }]`, mirroring the `buildProjectPhasesFilter` template. JSDoc explains the rationale (Companies endpoint requires a filter; active-only misses inactive companies referenced by tickets). + +**`lib/services/entity-sync.ts`** +- Added `buildCompaniesFilter` to the named-imports block from `../utils/sync-helpers`. +- Inserted a new `COMPANIES` branch as the first branch in the full-sync filter chain (placed before `CONTRACTS`). It pushes the `id > 0` filter and logs `Full sync of all companies (active + inactive)`. This routes COMPANIES around the generic `else` block that would otherwise apply `buildActiveFilter` (`isActive=true`). +- Added a new private `getValidCompanyIds()` method that selects all `companies.id WHERE is_deleted = false`, mirroring `getValidResourceIds`. +- Added a new `if (entity === EntityType.TICKETS)` block between the existing `recordsWithoutCompany` filter and the existing resource-FK nullification block. It iterates `mappedRecords`, nullifies `ticket.company_id` when the referenced company is missing from the Pulse mirror, tracks up to 5 sample ticket IDs, and emits a `warn` log when the count is non-zero. Pattern matches the surrounding resource nullification block exactly. + +## Why this fixes the FK error + +Every full and weekly-full sync since 2026-05-15 was failing with `[DATABASE_CONSTRAINT_ERROR] ... tickets_company_id_fkey`, rolling back the tickets bulkUpsert transaction. Root cause: Companies full sync was applying `buildActiveFilter`, which sends `isActive=true` to the Autotask API. Companies marked inactive in Autotask but still referenced by open tickets were therefore not mirrored into Pulse. When the tickets sync ran next, the FK constraint on `tickets.company_id → companies.id` fired and rolled the whole batch back. Widening the Companies filter to `id > 0` closes the primary gap by fetching active+inactive companies. The defensive nullification covers the residual edge case where a company was hard-deleted from Autotask entirely — instead of rolling back, the ticket survives with `company_id = null` (the column is nullable, and the existing `recordsWithoutCompany` filter only drops tickets that have no `company_id` from the start, so the bulkUpsert path handles `null` cleanly via the resource-nullification precedent). + +`hasAppliedFilters` stays true on Companies full sync because the new filter array is non-empty (`id > 0`), so `softDeleteMissingRecords` is NOT triggered — no accidental mass soft-delete of companies. Companies do not support incremental sync (per `supportsIncremental` at entity-sync.ts line 106), so this code path runs on every Companies sync (full, weekly-full, and the incremental→full fallback). + +## Commits + +| Task | Description | Commit | Files | +| ---- | ------------------------------------------------------------------------------------ | ------- | ------------------------------------------------ | +| 1 | Add buildCompaniesFilter + wire COMPANIES into the full-sync filter chain | 1ecaefe | lib/utils/sync-helpers.ts, lib/services/entity-sync.ts | +| 2 | Add getValidCompanyIds + defensive ticket company_id nullification | 62c529f | lib/services/entity-sync.ts | + +## Verification + +Type check passed cleanly: + +``` +cd /opt/stacks/pulse && npx tsc --noEmit --pretty +# (zero errors) +``` + +Targeted greps confirm shape: +- `grep -n "buildCompaniesFilter" lib/utils/sync-helpers.ts lib/services/entity-sync.ts` → 3 matches (definition + import + usage) +- `grep -c "getValidCompanyIds" lib/services/entity-sync.ts` → 2 (definition + call site) +- `grep -n "entity === EntityType.COMPANIES" lib/services/entity-sync.ts` → 1 new match (line 134, in the filter chain) +- `grep -n "if (entity === EntityType.TICKETS)" lib/services/entity-sync.ts` → 2 matches: line 261 (new company-validation block) BEFORE line 284 (existing resource-validation block), ordering correct. + +## Manual verify step (user runs after merge) + +```bash +curl -X POST http://localhost:3100/api/sync/full \ + -H 'content-type: application/json' \ + -d '{"triggeredBy":"manual-verify"}' +``` + +Then check `sync_history` for the next ticket sync row — `error_message` should no longer contain `tickets_company_id_fkey`, and `status` should be `completed`. Also sanity-check `SELECT count(*) FROM companies WHERE is_active = false AND is_deleted = false;` — the count should be ≥ the previous ~145 (or whatever the prior baseline was) once Companies full sync picks up inactive companies that weren't being fetched before. + +## Out-of-scope (explicit non-changes) + +- `getActiveField` in `lib/utils/sync-helpers.ts` — unchanged. Still returns `'isActive'` for COMPANIES (used by other call sites and for other entities like RESOURCES, CONTACTS, etc.). +- `buildActiveFilter` — unchanged. +- `syncTicketsChunked` — unchanged. The chunked path has its own (separate) flow and was not on the failing weekly-full path. Touching it was explicitly out of scope per the plan. +- The reconciliation service from `quick-260521-fci` — unchanged. +- `cachedValidResourceIds` / `cachedValidContactIds` — unchanged. +- The `lastActivityDate` filter on tickets — unchanged. +- Other branches in the full-sync filter chain (CONTRACTS, CONTRACT_SERVICES, PROJECTS, PROJECT_PHASES, TIME_ENTRIES, BILLING_ITEMS) — unchanged. + +## Deviations from Plan + +None — plan executed exactly as written. + +## Known Stubs + +None — both functions are wired end-to-end into the live sync path. + +## Self-Check: PASSED + +- `lib/utils/sync-helpers.ts` exists and contains `export function buildCompaniesFilter` at line 100. +- `lib/services/entity-sync.ts` exists; import (line 18), usage (line 135), COMPANIES branch (line 134), `getValidCompanyIds` method, and the new TICKETS company-validation block (line 261) all present. +- Commit `1ecaefe` exists in `git log` (Task 1). +- Commit `62c529f` exists in `git log` (Task 2). +- `npx tsc --noEmit --pretty` exits 0.