diff --git a/.planning/phases/15-data-model-detection-ticket-evidence/15-03-SUMMARY.md b/.planning/phases/15-data-model-detection-ticket-evidence/15-03-SUMMARY.md new file mode 100644 index 0000000..6feb979 --- /dev/null +++ b/.planning/phases/15-data-model-detection-ticket-evidence/15-03-SUMMARY.md @@ -0,0 +1,103 @@ +--- +phase: 15-data-model-detection-ticket-evidence +plan: 03 +subsystem: services +tags: [phishing-triage, webhook, cron, scheduler, detection] + +# Dependency graph +requires: + - phase: 15-02 + provides: "detectPhishingTicket(ticket) shared detection core (phishing-detector.ts)" +provides: + - "lib/services/phishing-sweep-service.ts — sweepPhishingTickets() bounded cron reconciliation" + - "webhook-service.ts triggerPhishingDetection() — fire-and-forget detection on ticket.created" + - "sync-scheduler.ts phishing-sweep sync_type + dispatch branch + defaultSchedules entry" + - "migrations/098_phishing_sweep_schedule.sql — phishing-sweep schedule seed for existing installs" +affects: [16-message-parsing, 17-mimecast-blast-radius] + +# Tech tracking +tech-stack: + added: [] + patterns: + - "Fire-and-forget webhook hook-in mirroring triggerWorkflowEngine (.catch(err => console.error(...)), never awaited in the request path" + - "Bounded cron sweep mirroring reconcileStaleTickets (LIMIT + recent-activity window + per-row try/catch, never rethrow)" + - "Dynamic await import() in every sync-scheduler dispatch branch (no eager worker import per CLAUDE.md)" + +key-files: + created: [lib/services/phishing-sweep-service.ts, migrations/098_phishing_sweep_schedule.sql] + modified: [lib/services/webhook-service.ts, lib/services/sync-scheduler.ts] + +key-decisions: + - "Reused ticket-reconciliation-service.ts's structure verbatim for the sweep (SELECT-with-LIMIT, per-row try/catch, createSyncLogger, aggregate result object) rather than inventing a new shape" + - "triggerPhishingDetection reads payload.entity.createdByContactID (not creatorContactID, which does not exist) into created_by_contact_id — verified by grep since a typo would compile cleanly (payload.entity is Record) but silently degrade EVID-01 requester capture at runtime" + - "phishing-sweep schedule is registered disabled-by-default (is_enabled: false), matching the tickets-reconcile precedent — an admin must opt in via /admin" + +requirements-completed: [DETECT-01, DETECT-02] + +# Metrics +duration: 9min +completed: 2026-07-15 +--- + +# Phase 15 Plan 03: Webhook + Cron Sweep Wiring Summary + +**Wired the Plan 02 `detectPhishingTicket` core into both of Pulse's established scan triggers — a fire-and-forget hook on the ticket.created webhook and a bounded daily cron sweep (`sweepPhishingTickets`, LIMIT 500 / 7-day window) — plus migration 098 to seed the disabled-by-default `phishing-sweep` schedule row for existing installs.** + +## Performance + +- **Duration:** 9 min +- **Started:** 2026-07-15T11:49:00Z +- **Completed:** 2026-07-15T11:58:00Z +- **Tasks:** 3 completed +- **Files modified:** 4 (2 created, 2 modified) + +## Accomplishments +- Created `lib/services/phishing-sweep-service.ts` exporting `sweepPhishingTickets()`: queries non-deleted tickets with `last_activity_date` in the last 7 days (bounded `LIMIT 500`), calls the shared `detectPhishingTicket` per row inside a try/catch (no rethrow — one bad ticket never aborts the sweep), and returns a `{ scanned, flagged, skippedUnchanged, errors }` aggregate, logged via `createSyncLogger`. +- Wired `webhook-service.ts`: added a `triggerPhishingDetection` private method that builds a `DetectableTicket` from `payload.entity` (preferring the inline entity, falling back to `payload.entityId` with null fields), reading the correct `createdByContactID` Autotask field into `created_by_contact_id`. Called it as a second fire-and-forget alongside the existing `triggerWorkflowEngine` call in the `ticket.created` handler, without awaiting it in the request path. +- Registered the sweep in `sync-scheduler.ts`: extended the `sync_type` union with `'phishing-sweep'`, added a `defaultSchedules` entry (`is_enabled: false`, daily `0 5 * * *` cron), and added a dispatch branch that dynamically imports and calls `sweepPhishingTickets`, logging the scanned/flagged/skippedUnchanged/errors summary. +- Created `migrations/098_phishing_sweep_schedule.sql`, seeding the `phishing-sweep` row via `ON CONFLICT (id) DO NOTHING` (idempotent, disabled by default) for existing installs whose `sync_schedules` table predates this migration. + +## Task Commits + +1. **Task 1: phishing-sweep-service.ts** - `dbd2ebe` (feat) +2. **Task 2: webhook hook-in** - `194b58b` (feat) +3. **Task 3: scheduler branch + migration 098** - `b199d99` (feat) + +**Plan metadata:** (this SUMMARY.md commit) + +## Files Created/Modified +- `lib/services/phishing-sweep-service.ts` - Bounded reconciliation sweep calling the shared `detectPhishingTicket` core (no duplicated match/hash logic) +- `lib/services/webhook-service.ts` - Added `detectPhishingTicket` import, `triggerPhishingDetection` method, and a fire-and-forget call in the `ticket.created` handler alongside the existing workflow-engine trigger +- `lib/services/sync-scheduler.ts` - Extended `sync_type` union, added `phishing-sweep` `defaultSchedules` entry, added a dispatch branch dynamically importing `sweepPhishingTickets` +- `migrations/098_phishing_sweep_schedule.sql` - Seeds the `phishing-sweep` schedule row for existing installs (idempotent, disabled by default) + +## Decisions Made +- Mirrored `ticket-reconciliation-service.ts`'s structure for the sweep verbatim (module constants for window/limit, `createSyncLogger`, per-row try/catch with `logger.warn` and no rethrow) rather than introducing a different shape, since the plan explicitly called this out as the closest analog. +- Used the exact `createdByContactID` field name confirmed by the plan's interface notes and `lib/utils/entity-mapper.ts:211`; verified via grep that no `creatorContactID` typo was introduced (a typo would compile cleanly since `AutotaskWebhookPayload.entity` is `Record`, but would silently produce `undefined` at runtime). +- Kept the `phishing-sweep` schedule disabled by default (`is_enabled: false`), matching the `tickets-reconcile` precedent, so the sweep does nothing until an admin explicitly enables it via `/admin`. + +## Deviations from Plan +None — plan executed exactly as written. All three tasks matched their `` specs, and every acceptance criterion (grep checks + `npx tsc --noEmit --pretty`) passed on first attempt. + +## Issues Encountered +None. + +## User Setup Required +None for this plan. The `phishing-sweep` schedule is seeded disabled; an admin can enable it later at `/admin` once ready to run reconciliation sweeps in production. No new env vars or credentials introduced (reuses existing `AUTOTASK_*` config via the Plan 02 detector). + +## Next Phase Readiness +- Both DETECT-01 scan triggers (webhook + cron) are now wired to the same shared `detectPhishingTicket` core, so DETECT-02 idempotency holds identically on either path. +- The `phishing-sweep` schedule exists in the `defaultSchedules` seed path (fresh installs) and migration 098 (existing installs), both idempotent and disabled by default. +- No blockers for Phase 16 (message parsing) or Phase 17 (Mimecast blast-radius), which build on this detection/evidence foundation. + +--- +*Phase: 15-data-model-detection-ticket-evidence* +*Completed: 2026-07-15* + +## Self-Check: PASSED +- FOUND: lib/services/phishing-sweep-service.ts +- FOUND: migrations/098_phishing_sweep_schedule.sql +- FOUND: .planning/phases/15-data-model-detection-ticket-evidence/15-03-SUMMARY.md +- FOUND: commit dbd2ebe +- FOUND: commit 194b58b +- FOUND: commit b199d99 diff --git a/lib/services/phishing-sweep-service.ts b/lib/services/phishing-sweep-service.ts new file mode 100644 index 0000000..b598daa --- /dev/null +++ b/lib/services/phishing-sweep-service.ts @@ -0,0 +1,101 @@ +/** + * Phishing Sweep Service + * + * Bounded cron reconciliation sweep over recently-modified tickets, mirroring + * ticket-reconciliation-service.ts's structure. The webhook path (Plan 03, + * webhook-service.ts) is the primary near-real-time trigger; this sweep + * catches anything the webhook missed (dropped events, backfilled tickets, + * tickets whose content changed after the webhook already fired) by + * re-running the same shared detectPhishingTicket core. + * + * No duplicated pattern-matching/hashing logic here — this file only + * queries candidate tickets and delegates detection to phishing-detector.ts. + */ + +import { postgresClient } from './postgres-client'; +import { detectPhishingTicket, type DetectableTicket } from './phishing-detector'; +import { createSyncLogger } from '../utils/sync-logger'; + +export interface PhishingSweepResult { + scanned: number; + flagged: number; + skippedUnchanged: number; + errors: number; +} + +const SWEEP_WINDOW_DAYS = 7; +const SCAN_LIMIT = 500; + +/** + * Re-scan up to SCAN_LIMIT recently-modified tickets through the shared + * phishing detector. Bounded and idempotent — safe to run on a recurring + * schedule alongside the webhook trigger. + */ +export async function sweepPhishingTickets(): Promise { + const logger = createSyncLogger({ component: 'PhishingSweep' }); + const startedAt = Date.now(); + const result: PhishingSweepResult = { + scanned: 0, + flagged: 0, + skippedUnchanged: 0, + errors: 0, + }; + + const candidatesQuery = ` + SELECT id, ticket_number, title, description, company_id, contact_id, created_by_contact_id + FROM tickets + WHERE is_deleted = false + AND last_activity_date > NOW() - INTERVAL '${SWEEP_WINDOW_DAYS} days' + ORDER BY last_activity_date DESC + LIMIT $1 + `; + const candidates = await postgresClient.query<{ + id: string; + ticket_number: string | null; + title: string | null; + description: string | null; + company_id: number | null; + contact_id: number | null; + created_by_contact_id: number | null; + }>(candidatesQuery, [SCAN_LIMIT]); + + logger.info( + `Sweeping ${candidates.rows.length} recently-modified tickets (>${SWEEP_WINDOW_DAYS}d, limit=${SCAN_LIMIT})` + ); + + for (const row of candidates.rows) { + result.scanned += 1; + const ticket: DetectableTicket = { + id: Number(row.id), + ticket_number: row.ticket_number, + title: row.title, + description: row.description, + company_id: row.company_id, + contact_id: row.contact_id, + created_by_contact_id: row.created_by_contact_id, + }; + + try { + const detection = await detectPhishingTicket(ticket); + if (detection.skippedUnchanged) { + result.skippedUnchanged += 1; + } else if (detection.flagged) { + result.flagged += 1; + } + } catch (err) { + result.errors += 1; + logger.warn( + `Phishing detection failed for ticket ${ticket.id}`, + { ticketId: ticket.id }, + err instanceof Error ? err : new Error(String(err)) + ); + } + } + + logger.info( + `Phishing sweep complete: scanned=${result.scanned} flagged=${result.flagged} skippedUnchanged=${result.skippedUnchanged} errors=${result.errors}`, + { duration: Date.now() - startedAt } + ); + + return result; +} diff --git a/lib/services/sync-scheduler.ts b/lib/services/sync-scheduler.ts index cd0dac2..48a0048 100644 --- a/lib/services/sync-scheduler.ts +++ b/lib/services/sync-scheduler.ts @@ -22,7 +22,7 @@ export interface ScheduleConfig { name: string; description: string; cron_expression: string; - sync_type: 'incremental' | 'full' | 'veeam-incremental' | 'veeam-full' | 'veeam-rpo-check' | 'contract-services' | 'engagement-daily' | 'zoom-daily' | 'morning-summary' | 'ticket-digest-daily' | 'ticket-digest-weekly' | 'ticket-digest-monthly' | 'device-link-reconcile' | 'integration-health' | 'qbo' | 'appgate-sessions' | 'appgate-daily' | 'tickets-reconcile' | 'pax8-daily'; + sync_type: 'incremental' | 'full' | 'veeam-incremental' | 'veeam-full' | 'veeam-rpo-check' | 'contract-services' | 'engagement-daily' | 'zoom-daily' | 'morning-summary' | 'ticket-digest-daily' | 'ticket-digest-weekly' | 'ticket-digest-monthly' | 'device-link-reconcile' | 'integration-health' | 'qbo' | 'appgate-sessions' | 'appgate-daily' | 'tickets-reconcile' | 'pax8-daily' | 'phishing-sweep'; years_back?: number; is_enabled: boolean; last_run?: Date; @@ -299,6 +299,14 @@ class SyncScheduler { sync_type: 'tickets-reconcile', is_enabled: false, }, + { + id: 'phishing-sweep', + name: 'Phishing Detection Sweep', + description: 'Reconciles recently-modified tickets through the phishing detector to catch reports missed by the webhook path (bounded to 500 tickets, idempotent on content hash).', + cron_expression: '0 5 * * *', + sync_type: 'phishing-sweep', + is_enabled: false, + }, ]; for (const schedule of defaultSchedules) { @@ -461,6 +469,12 @@ class SyncScheduler { console.log( `[SCHEDULER] tickets-reconcile: scanned=${result.scanned} updated=${result.updated} flippedComplete=${result.statusFlippedToComplete} softDeleted=${result.softDeleted} errors=${result.errors}` ); + } else if (config.sync_type === 'phishing-sweep') { + const { sweepPhishingTickets } = await import('@/lib/services/phishing-sweep-service'); + const result = await sweepPhishingTickets(); + console.log( + `[SCHEDULER] phishing-sweep: scanned=${result.scanned} flagged=${result.flagged} skippedUnchanged=${result.skippedUnchanged} errors=${result.errors}` + ); } else if (config.sync_type === 'pax8-daily') { const { isPax8Configured } = await import('@/lib/services/pax8-factory'); if (!isPax8Configured()) { diff --git a/lib/services/webhook-service.ts b/lib/services/webhook-service.ts index f2dc4ec..d650ba9 100644 --- a/lib/services/webhook-service.ts +++ b/lib/services/webhook-service.ts @@ -14,6 +14,7 @@ import { workflowEngine } from './workflow-engine'; import { ticketWorkflowEngine } from './ticket-workflow-engine'; import '../services/workflow-steps'; // Register all workflow step executors import { WorkflowEvent, TicketData } from '../types/workflow'; +import { detectPhishingTicket, DetectableTicket } from './phishing-detector'; export class WebhookService { private _autotaskClient: AutotaskClient | null = null; @@ -114,6 +115,9 @@ export class WebhookService { this.triggerWorkflowEngine(payload).catch(err => console.error('[WEBHOOK] Workflow engine error:', err) ); + this.triggerPhishingDetection(payload).catch(err => + console.error('[WEBHOOK] Phishing detection error:', err) + ); } @@ -436,6 +440,45 @@ export class WebhookService { // DEPRECATED: old workflow engine (will be removed after testing period) // await workflowEngine.process(event); } + + /** + * Trigger phishing detection for a new ticket. + * Runs asynchronously — does not block webhook response. Mirrors + * triggerWorkflowEngine's "prefer inline payload.entity, else payload.entityId" shape. + */ + private async triggerPhishingDetection(payload: AutotaskWebhookPayload): Promise { + let ticket: DetectableTicket; + + if (payload.entity) { + // NOTE: Autotask's requester field is `createdByContactID` (mapped to + // created_by_contact_id at lib/utils/entity-mapper.ts:211). There is no + // similarly-named alternative field — since payload.entity is typed + // Record, a wrong field name would compile cleanly but + // silently produce undefined at runtime. + ticket = { + id: payload.entityId, + ticket_number: payload.entity.ticketNumber || null, + title: payload.entity.title || null, + description: payload.entity.description || null, + company_id: payload.entity.companyID || null, + contact_id: payload.entity.contactID || null, + created_by_contact_id: payload.entity.createdByContactID || null, + }; + } else { + ticket = { + id: payload.entityId, + ticket_number: null, + title: null, + description: null, + company_id: null, + contact_id: null, + created_by_contact_id: null, + }; + } + + console.log(`[WEBHOOK] Triggering phishing detection for ticket ${payload.entityId}`); + await detectPhishingTicket(ticket); + } } // Export singleton instance diff --git a/migrations/098_phishing_sweep_schedule.sql b/migrations/098_phishing_sweep_schedule.sql new file mode 100644 index 0000000..845c3a8 --- /dev/null +++ b/migrations/098_phishing_sweep_schedule.sql @@ -0,0 +1,25 @@ +-- Migration 098: Seed the phishing-sweep sync schedule. +-- +-- The sync_scheduler.createDefaultSchedules() path only seeds defaults on a +-- virgin sync_schedules table; this migration covers existing installs. +-- Idempotent via ON CONFLICT (id) DO NOTHING. Disabled by default +-- (is_enabled=false) until an admin opts in via /admin. + +INSERT INTO sync_schedules ( + id, + name, + description, + cron_expression, + sync_type, + years_back, + is_enabled +) VALUES ( + 'phishing-sweep', + 'Phishing Detection Sweep', + 'Reconciles recently-modified tickets through the phishing detector to catch reports missed by the webhook path (bounded to 500 tickets, idempotent on content hash).', + '0 5 * * *', + 'phishing-sweep', + NULL, + false +) +ON CONFLICT (id) DO NOTHING;