diff --git a/.planning/STATE.md b/.planning/STATE.md index ff7ebd1..0159c7d 100644 --- a/.planning/STATE.md +++ b/.planning/STATE.md @@ -2,14 +2,14 @@ gsd_state_version: 1.0 milestone: v3.0 milestone_name: Phishing Triage Automation -status: planning +status: executing stopped_at: Phase 15 context gathered -last_updated: "2026-07-15T10:56:47.034Z" -last_activity: 2026-07-14 — v3.0 ROADMAP.md created, 26/26 requirements mapped across Phases 15-21 +last_updated: "2026-07-15T11:30:38.677Z" +last_activity: 2026-07-15 -- Phase 15 planning complete progress: total_phases: 7 completed_phases: 0 - total_plans: 0 + total_plans: 3 completed_plans: 0 percent: 0 --- @@ -27,8 +27,8 @@ See: .planning/PROJECT.md (updated 2026-07-14) Phase: 15 of 21 (Data Model, Detection & Ticket Evidence) Plan: — (not yet planned) -Status: Roadmap approved, ready to plan Phase 15 -Last activity: 2026-07-14 — v3.0 ROADMAP.md created, 26/26 requirements mapped across Phases 15-21 +Status: Ready to execute +Last activity: 2026-07-15 -- Phase 15 planning complete Progress: [░░░░░░░░░░] 0% diff --git a/.planning/phases/15-data-model-detection-ticket-evidence/15-01-PLAN.md b/.planning/phases/15-data-model-detection-ticket-evidence/15-01-PLAN.md index 2331e8a..ebd30b6 100644 --- a/.planning/phases/15-data-model-detection-ticket-evidence/15-01-PLAN.md +++ b/.planning/phases/15-data-model-detection-ticket-evidence/15-01-PLAN.md @@ -67,6 +67,10 @@ migrations/091_pax8_tables.sql (schema-only, multi-table, IF NOT EXISTS + traili CREATE INDEX IF NOT EXISTS per table, "future phases populate this" header comment). Postgres 16 has built-in gen_random_uuid() — no extension needed for UUID PKs. + +Dev DB connection (docker-compose.yml): container `pulse-postgres`, POSTGRES_USER +defaults to `pulse_user`, POSTGRES_DB defaults to `pulse_autotask`. These env vars are +NOT exported to a bare shell, so verify commands must hardcode these as the fallback. @@ -149,21 +153,22 @@ Postgres 16 has built-in gen_random_uuid() — no extension needed for UUID PKs. Task 2: Apply migration 097 to the dev database and verify tables exist migrations/097_phishing_triage_schema.sql - - docker-compose.yml (Postgres container name + POSTGRES_USER/POSTGRES_DB values for the psql apply command) + - docker-compose.yml (Postgres container name `pulse-postgres`; POSTGRES_USER=pulse_user, POSTGRES_DB=pulse_autotask for the psql apply command) - CLAUDE.md (Migrations section: "Postgres init applies migrations on first boot only — for an existing DB, run via scripts/apply-migrations; check first, behavior varies") - MEMORY note: existing DB volumes do NOT auto-apply new migrations; apply manually via `docker exec pulse-postgres psql ...` Apply migration 097 to the running dev Postgres. First check for a project apply helper (`ls scripts/apply-migrations*`); if one exists and matches convention, use it. - Otherwise pipe the file into the container's psql: read the Postgres container name - and POSTGRES_USER / POSTGRES_DB from docker-compose.yml, then run the migration with - `docker exec -i psql -U -d ` fed from + Otherwise pipe the file into the container's psql: the Postgres container is + `pulse-postgres`, the user is `pulse_user`, and the database is `pulse_autotask` + (per docker-compose.yml). Run the migration with + `docker exec -i pulse-postgres psql -U pulse_user -d pulse_autotask` fed from migrations/097_phishing_triage_schema.sql. Because the migration is fully IF NOT EXISTS, re-applying it must be safe. Do NOT drop or recreate any existing table. - docker exec -i "$(grep -oiE 'container_name:\s*\S*postgres\S*' docker-compose.yml | head -1 | awk '{print $2}')" psql -U "${POSTGRES_USER:-postgres}" -d "${POSTGRES_DB:-pulse}" -tAc "SELECT count(*) FROM information_schema.tables WHERE table_name IN ('campaigns','reports','messages','indicators','classifications','remediation_actions','audit_events');" | grep -qx 7 && echo "all 7 tables present" + docker exec -i pulse-postgres psql -U "${POSTGRES_USER:-pulse_user}" -d "${POSTGRES_DB:-pulse_autotask}" -tAc "SELECT count(*) FROM information_schema.tables WHERE table_name IN ('campaigns','reports','messages','indicators','classifications','remediation_actions','audit_events');" | grep -qx 7 && echo "all 7 tables present" - Querying `information_schema.tables` returns all 7 phishing-triage table names present in the dev DB diff --git a/.planning/phases/15-data-model-detection-ticket-evidence/15-02-PLAN.md b/.planning/phases/15-data-model-detection-ticket-evidence/15-02-PLAN.md index 75358bd..bc15c88 100644 --- a/.planning/phases/15-data-model-detection-ticket-evidence/15-02-PLAN.md +++ b/.planning/phases/15-data-model-detection-ticket-evidence/15-02-PLAN.md @@ -191,6 +191,9 @@ The 8 locked DETECT-01 patterns (case-insensitive substrings — all must be cov created_by_contact_id, title, description, matched_patterns, content_hash, evidence) VALUES ($1..$11) ON CONFLICT (ticket_id) DO UPDATE SET ... , updated_at = NOW() RETURNING id. matched_patterns and evidence bind as JSON (JSON.stringify or ::jsonb cast). + Column bindings from the DetectableTicket: `requester_contact_id` binds from `ticket.contact_id` + (the Autotask requester/reporter of the ticket), and `created_by_contact_id` binds from + `ticket.created_by_contact_id`. company_name comes from the gathered evidence's company_name. Return { flagged: true, reportId }. Do not duplicate the matcher/hash logic — call the Task 1 functions. Use console.error with a @@ -203,6 +206,7 @@ The 8 locked DETECT-01 patterns (case-insensitive substrings — all must be cov - `detectPhishingTicket` is exported and returns `{ flagged, reportId?, skippedUnchanged? }` - Idempotency branch present: a SELECT of reports by ticket_id compares stored `content_hash` to the freshly computed hash and returns `skippedUnchanged: true` without writing when equal - The reports upsert uses `ON CONFLICT (ticket_id) DO UPDATE ... updated_at = NOW()` and RETURNING id + - `requester_contact_id` is bound from `ticket.contact_id` and `created_by_contact_id` from `ticket.created_by_contact_id` in the INSERT - Evidence gathering reads ticket_notes and time_entries via parameterized `$1` queries (no string interpolation of ticket_id) and stores only attachment metadata (fullPath/title/contentType), never base64 `data` - The Autotask getAttachments call is wrapped so a failure yields an empty attachments array instead of throwing - `npx tsc --noEmit --pretty` passes and the Task 1 vitest suite still passes diff --git a/.planning/phases/15-data-model-detection-ticket-evidence/15-03-PLAN.md b/.planning/phases/15-data-model-detection-ticket-evidence/15-03-PLAN.md index d573593..1377aa3 100644 --- a/.planning/phases/15-data-model-detection-ticket-evidence/15-03-PLAN.md +++ b/.planning/phases/15-data-model-detection-ticket-evidence/15-03-PLAN.md @@ -71,9 +71,15 @@ webhook-service.ts (existing precedent to copy verbatim): - Fire-and-forget block, lines 112-117: on payload.entityType === WebhookEntityType.TICKETS && payload.eventType === WebhookEventType.CREATE, `this.triggerWorkflowEngine(payload).catch(err => console.error(...))`. - triggerWorkflowEngine (lines 398-420): builds typed ticket data from payload.entity when present, - else uses payload.entityId — copy this "prefer inline entity, else id" shape. + else uses payload.entityId — copy this "prefer inline entity, else id" shape. Note the Autotask + entity field names it reads: payload.entity.title, .description, .ticketNumber, .companyID, .contactID. - Imports at top, lines 13-16. +Autotask ticket field → DB column mapping (lib/utils/entity-mapper.ts): +- payload.entity.createdByContactID → created_by_contact_id (entity-mapper.ts:211). The field is + `createdByContactID`. There is NO `creatorContactID` field — using it compiles cleanly (entity is + Record) but silently yields undefined at runtime. + sync-scheduler.ts (existing precedent): - ScheduleConfig.sync_type union, line 25 (currently ends `... | 'tickets-reconcile' | 'pax8-daily'`). - defaultSchedules array — tickets-reconcile entry at lines 294-301 (id, name, description, @@ -145,6 +151,7 @@ sync_schedules.id is a text PK (values like 'tickets-reconcile'). lib/services/webhook-service.ts - lib/services/webhook-service.ts (full file — imports at 13-16, ticket.created fire-and-forget block at 112-117, triggerWorkflowEngine payload-shaping at 398-420) + - lib/utils/entity-mapper.ts (line 211 — createdByContactID → created_by_contact_id mapping) - lib/services/phishing-detector.ts (detectPhishingTicket + DetectableTicket shape) @@ -156,10 +163,17 @@ sync_schedules.id is a text PK (values like 'tickets-reconcile'). - Add a private method `triggerPhishingDetection(payload: AutotaskWebhookPayload): Promise` that builds a DetectableTicket the same "prefer inline payload.entity, else use payload.entityId" way triggerWorkflowEngine does (lines 398-420): if payload.entity is present, map - title/description/ticketNumber/companyID/contactID/creatorContactID from it; otherwise pass a - minimal ticket with `id: payload.entityId` and null fields (the detector's evidence/hash still - works, and the cron sweep reconciles anything the webhook payload lacked). Call - `await detectPhishingTicket(ticket)`. + title←payload.entity.title, description←payload.entity.description, + ticket_number←payload.entity.ticketNumber, company_id←payload.entity.companyID, + contact_id←payload.entity.contactID, and created_by_contact_id←payload.entity.createdByContactID. + IMPORTANT: the Autotask ticket field is `createdByContactID` (mapped to `created_by_contact_id` + at lib/utils/entity-mapper.ts:211). Do NOT use `creatorContactID` — that field does not exist, + and because AutotaskWebhookPayload.entity is typed `Record` a typo would compile + cleanly but silently produce undefined at runtime, degrading EVID-01 requester/reporter capture + on the primary (webhook) detection path. + Otherwise pass a minimal ticket with `id: payload.entityId` and null fields (the detector's + evidence/hash still works, and the cron sweep reconciles anything the webhook payload lacked). + Call `await detectPhishingTicket(ticket)`. - In the existing ticket.created fire-and-forget block (lines 112-117), add a second identically-shaped call immediately after the triggerWorkflowEngine call: `this.triggerPhishingDetection(payload).catch(err => console.error('[WEBHOOK] Phishing detection error:', err));`. @@ -167,16 +181,17 @@ sync_schedules.id is a text PK (values like 'tickets-reconcile'). workflow-engine precedent. Do not change the workflow-engine call or any other webhook behavior. - npx tsc --noEmit --pretty && grep -q "triggerPhishingDetection" lib/services/webhook-service.ts && grep -q "Phishing detection error" lib/services/webhook-service.ts && echo "hook OK" + npx tsc --noEmit --pretty && grep -q "triggerPhishingDetection" lib/services/webhook-service.ts && grep -q "createdByContactID" lib/services/webhook-service.ts && grep -q "Phishing detection error" lib/services/webhook-service.ts && echo "hook OK" - `webhook-service.ts` imports `detectPhishingTicket` from `./phishing-detector` - A `triggerPhishingDetection` method exists and builds ticket data using the "prefer payload.entity, else payload.entityId" shape + - The method reads `payload.entity.createdByContactID` (NOT `creatorContactID`) into `created_by_contact_id` — grep confirms `createdByContactID` is present and `creatorContactID` is absent - The ticket.created handler calls `this.triggerPhishingDetection(payload).catch(...)` as fire-and-forget (not awaited in the request path), immediately alongside the existing `triggerWorkflowEngine` call - The existing workflow-engine trigger and all other webhook behavior are unchanged (grep still finds `triggerWorkflowEngine`) - `npx tsc --noEmit --pretty` passes - New ticket webhooks fire phishing detection without blocking the webhook response, mirroring the workflow-engine trigger precedent. + New ticket webhooks fire phishing detection without blocking the webhook response, mirroring the workflow-engine trigger precedent, with the requester/reporter contact read from the correct `createdByContactID` field. @@ -252,6 +267,7 @@ sync_schedules.id is a text PK (values like 'tickets-reconcile'). - `npx tsc --noEmit --pretty` clean across all three modified/created files - `grep "'phishing-sweep'" lib/services/sync-scheduler.ts` finds union member, default entry, and dispatch branch - `grep "triggerPhishingDetection" lib/services/webhook-service.ts` finds the fire-and-forget hook +- `grep "createdByContactID" lib/services/webhook-service.ts` confirms the correct requester field (and `creatorContactID` is absent) - `grep "ON CONFLICT (id) DO NOTHING" migrations/098_phishing_sweep_schedule.sql` matches - Sweep query bounded by LIMIT (DoS guard)