docs(phase-15): create phase plan

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012wWroM6FXkQJiH3JgYcony
This commit is contained in:
lorentz 2026-07-15 07:17:10 -04:00
parent 7fa94cb6a5
commit 3f8e8dc826
4 changed files with 735 additions and 2 deletions

View file

@ -374,7 +374,10 @@ summarizes classification, blast radius, and recommended/approved remediation st
2. Running the ticket scanner against Autotask/Pulse tickets flags candidates matching the known title/body patterns ("Phishing Report", "Spam Alert", "Phishing Alert - Email Security Report", "KnowBe4 Phish Alert Report", "Source: KnowBe4 Phish Alert Button", "userSubmissionsReportMessage", "reported message destinations", "Microsoft directly") and persists a `reports` row per candidate
3. Re-scanning tickets that haven't changed since last processed does not reprocess or duplicate their `reports` rows; a ticket whose Autotask data changed since last processed IS reprocessed (idempotent on ticket state, not just ticket ID)
4. Each flagged ticket's stored evidence includes ticket ID/number, company, requester/reporter, title, description, notes, relevant time entries, and attachment metadata (EVID-01)
**Plans**: TBD
**Plans**: 3 plans
- [ ] 15-01-PLAN.md — Migration 097: 7-table phishing-triage schema (reports fully designed, others stubbed) (DETECT-01, DETECT-02, EVID-01)
- [ ] 15-02-PLAN.md — phishing-detector.ts core: pattern matcher + content-hash idempotency + EVID-01 evidence capture + reports upsert (DETECT-01, DETECT-02, EVID-01)
- [ ] 15-03-PLAN.md — Wiring: webhook fire-and-forget hook + bounded cron sweep service + scheduler branch + migration 098 seed (DETECT-01, DETECT-02)
**UI hint**: no
### Phase 16: EML/MIME Evidence Parser
@ -474,7 +477,7 @@ Phases execute in numeric order. v1.0 (Phases 1-9.1) shipped 2026-07-10. v2.0 (P
| 12. Orders/Invoices & Company Matching | v2.0 | 5/5 | Complete | 2026-07-11 |
| 13. Scheduler & Admin Toggle | v2.0 | 3/3 | Complete | 2026-07-11 |
| 14. /pax8 UI Surface | v2.0 | 6/6 | Complete | 2026-07-12 |
| 15. Data Model, Detection & Ticket Evidence | v3.0 | 0/TBD | Not started | - |
| 15. Data Model, Detection & Ticket Evidence | v3.0 | 0/3 | Not started | - |
| 16. EML/MIME Evidence Parser | v3.0 | 0/TBD | Not started | - |
| 17. Mimecast Blast Radius Lookup | v3.0 | 0/TBD | Not started | - |
| 18. Campaign Grouping & Phishing Analysis API | v3.0 | 0/TBD | Not started | - |

View file

@ -0,0 +1,209 @@
---
phase: 15-data-model-detection-ticket-evidence
plan: 01
type: execute
wave: 1
depends_on: []
files_modified:
- migrations/097_phishing_triage_schema.sql
autonomous: true
requirements: [DETECT-01, DETECT-02, EVID-01]
must_haves:
truths:
- "The 7 phishing-triage tables exist in Postgres and can be queried"
- "A reports row can store a ticket's content_hash, matched patterns, and EVID-01 evidence"
- "Re-running the migration is a no-op (IF NOT EXISTS on every table/index)"
artifacts:
- path: "migrations/097_phishing_triage_schema.sql"
provides: "campaigns, reports, messages, indicators, classifications, remediation_actions, audit_events tables"
contains: "CREATE TABLE IF NOT EXISTS reports"
key_links:
- from: "reports.ticket_id"
to: "tickets.id"
via: "foreign key"
pattern: "REFERENCES tickets\\(id\\)"
- from: "reports.content_hash"
to: "phishing-detector idempotency (Plan 02)"
via: "unique key on ticket_id + stored hash"
pattern: "content_hash"
---
<objective>
Create migration `097_phishing_triage_schema.sql` — the durable phishing-triage
schema (7 tables) that every v3.0 phase reads and writes. Phase 15 only populates
`reports` (via the Plan 02 detector); `campaigns`, `messages`, `indicators`,
`classifications`, `remediation_actions`, and `audit_events` are laid down now as
stubs so later phases (16-21) have their schema ready and never need a second
foundation migration.
Purpose: Land the schema before any service writes to it (STATE.md decision:
"durable schema lands in Phase 15, before any service that writes to it").
Output: One numbered, idempotent, schema-only migration file.
</objective>
<execution_context>
@$HOME/.claude/get-shit-done/workflows/execute-plan.md
@$HOME/.claude/get-shit-done/templates/summary.md
</execution_context>
<context>
@.planning/PROJECT.md
@.planning/ROADMAP.md
@.planning/STATE.md
@.planning/phases/15-data-model-detection-ticket-evidence/15-CONTEXT.md
@.planning/phases/15-data-model-detection-ticket-evidence/15-PATTERNS.md
<interfaces>
<!-- Existing columns the reports FKs and evidence columns reference. Do not re-derive. -->
tickets (migrations/001_initial_schema.sql): id BIGINT PK, company_id BIGINT NOT NULL,
ticket_number VARCHAR(100), title VARCHAR(255), description TEXT, contact_id BIGINT,
created_by_contact_id BIGINT, assigned_resource_id BIGINT, last_activity_date TIMESTAMP.
companies (migrations/001_initial_schema.sql): id BIGINT PK, company_name VARCHAR(255).
Reference migration for header framing + table/index conventions:
migrations/091_pax8_tables.sql (schema-only, multi-table, IF NOT EXISTS + trailing
CREATE INDEX IF NOT EXISTS per table, "future phases populate this" header comment).
Postgres 16 has built-in gen_random_uuid() — no extension needed for UUID PKs.
</interfaces>
</context>
<tasks>
<task type="auto">
<name>Task 1: Write migration 097 — 7-table phishing triage schema</name>
<files>migrations/097_phishing_triage_schema.sql</files>
<read_first>
- migrations/091_pax8_tables.sql (header-comment framing + IF NOT EXISTS + per-table CREATE INDEX convention to copy)
- migrations/090_ticket_reconcile_schedule.sql (ON CONFLICT idempotency convention used elsewhere)
- migrations/001_initial_schema.sql (confirm exact tickets/companies column names the FKs reference)
- .planning/phases/15-data-model-detection-ticket-evidence/15-PATTERNS.md (section on migrations/097 — reports column requirements)
- CLAUDE.md (Database + Migrations sections: snake_case columns, audit-column convention, IF NOT EXISTS, next sequential number)
</read_first>
<action>
Create `migrations/097_phishing_triage_schema.sql`. Open with a header comment
(mirror migrations/091_pax8_tables.sql lines 1-18) stating this is a schema-only
migration for v3.0 phishing triage, that Phase 15 populates only `reports`, and
that `campaigns`/`messages`/`indicators`/`classifications`/`remediation_actions`/
`audit_events` are stubs populated by Phases 16-21.
Create tables in this order so FKs resolve (every table `CREATE TABLE IF NOT EXISTS`,
every UUID PK `id UUID PRIMARY KEY DEFAULT gen_random_uuid()`, snake_case columns):
1. campaigns — id, campaign_key TEXT, group_method TEXT, first_seen_at TIMESTAMPTZ,
last_seen_at TIMESTAMPTZ, report_count INTEGER NOT NULL DEFAULT 0, status TEXT NOT NULL DEFAULT 'open',
created_at TIMESTAMPTZ NOT NULL DEFAULT NOW(), updated_at TIMESTAMPTZ NOT NULL DEFAULT NOW().
2. reports (the only table Phase 15 populates — design fully): id;
ticket_id BIGINT NOT NULL REFERENCES tickets(id); ticket_number VARCHAR(100);
company_id BIGINT; company_name VARCHAR(255); requester_contact_id BIGINT;
created_by_contact_id BIGINT; title VARCHAR(255); description TEXT;
matched_patterns JSONB NOT NULL DEFAULT '[]'::jsonb (the DETECT-01 pattern strings that matched);
content_hash TEXT NOT NULL (D-04 idempotency key over title+description);
evidence JSONB NOT NULL DEFAULT '{}'::jsonb (EVID-01 notes/time_entries/attachments capture);
campaign_id UUID REFERENCES campaigns(id) (nullable — Phase 18 links it);
created_at TIMESTAMPTZ NOT NULL DEFAULT NOW(); updated_at TIMESTAMPTZ NOT NULL DEFAULT NOW().
Add `CONSTRAINT uq_reports_ticket_id UNIQUE (ticket_id)` so the Plan 02 detector
can upsert one report per ticket (ON CONFLICT (ticket_id)) and enforce D-04.
3. messages (stub for Phase 16) — id, report_id UUID REFERENCES reports(id),
message_id TEXT, headers JSONB, urls JSONB, attachments JSONB, body_preview TEXT,
raw_ref TEXT, created_at TIMESTAMPTZ NOT NULL DEFAULT NOW().
4. indicators (stub for Phase 16) — id, message_id UUID REFERENCES messages(id),
indicator_type TEXT NOT NULL, value TEXT NOT NULL, created_at TIMESTAMPTZ NOT NULL DEFAULT NOW().
5. classifications (stub for Phase 19) — id, campaign_id UUID REFERENCES campaigns(id),
verdict TEXT, confidence NUMERIC, summary TEXT, reasons JSONB, recommended_actions JSONB,
requires_approval BOOLEAN NOT NULL DEFAULT false, created_at TIMESTAMPTZ NOT NULL DEFAULT NOW().
6. remediation_actions (stub for Phase 20) — id, campaign_id UUID REFERENCES campaigns(id),
action_type TEXT, status TEXT NOT NULL DEFAULT 'proposed', params JSONB,
approved_by TEXT, approved_at TIMESTAMPTZ, created_at TIMESTAMPTZ NOT NULL DEFAULT NOW().
7. audit_events (stub for Phase 20) — id, campaign_id UUID, actor TEXT, event_type TEXT NOT NULL,
payload JSONB, created_at TIMESTAMPTZ NOT NULL DEFAULT NOW().
After each CREATE TABLE add `CREATE INDEX IF NOT EXISTS` statements: reports on
(ticket_id), (content_hash), (campaign_id); messages on (report_id), (message_id);
indicators on (message_id); classifications on (campaign_id); remediation_actions on
(campaign_id); campaigns on (campaign_key). Match the trailing-index style in
migrations/091_pax8_tables.sql. Do NOT edit any committed migration; this is a new file only.
</action>
<verify>
<automated>test -f migrations/097_phishing_triage_schema.sql && grep -c "CREATE TABLE IF NOT EXISTS" migrations/097_phishing_triage_schema.sql | grep -qx 7 && echo "7 tables OK"</automated>
</verify>
<acceptance_criteria>
- `migrations/097_phishing_triage_schema.sql` exists
- File contains exactly 7 `CREATE TABLE IF NOT EXISTS` statements: campaigns, reports, messages, indicators, classifications, remediation_actions, audit_events
- `reports` table declares `ticket_id BIGINT NOT NULL REFERENCES tickets(id)`, `content_hash TEXT NOT NULL`, `matched_patterns JSONB`, `evidence JSONB`, and `CONSTRAINT uq_reports_ticket_id UNIQUE (ticket_id)`
- Every index statement uses `CREATE INDEX IF NOT EXISTS`
- No `DROP`, `ALTER ... DROP`, or `TRUNCATE` appears anywhere in the file (grep returns no matches)
</acceptance_criteria>
<done>The migration file lays down all 7 tables idempotently with the reports schema needed by Plan 02, and contains no destructive statements.</done>
</task>
<task type="auto">
<name>Task 2: Apply migration 097 to the dev database and verify tables exist</name>
<files>migrations/097_phishing_triage_schema.sql</files>
<read_first>
- docker-compose.yml (Postgres container name + POSTGRES_USER/POSTGRES_DB values for the psql apply command)
- CLAUDE.md (Migrations section: "Postgres init applies migrations on first boot only — for an existing DB, run via scripts/apply-migrations; check first, behavior varies")
- MEMORY note: existing DB volumes do NOT auto-apply new migrations; apply manually via `docker exec pulse-postgres psql ...`
</read_first>
<action>
Apply migration 097 to the running dev Postgres. First check for a project apply
helper (`ls scripts/apply-migrations*`); if one exists and matches convention, use it.
Otherwise pipe the file into the container's psql: read the Postgres container name
and POSTGRES_USER / POSTGRES_DB from docker-compose.yml, then run the migration with
`docker exec -i <pg-container> psql -U <user> -d <db>` fed from
migrations/097_phishing_triage_schema.sql. Because the migration is fully IF NOT EXISTS,
re-applying it must be safe. Do NOT drop or recreate any existing table.
</action>
<verify>
<automated>docker exec -i "$(grep -oiE 'container_name:\s*\S*postgres\S*' docker-compose.yml | head -1 | awk '{print $2}')" psql -U "${POSTGRES_USER:-postgres}" -d "${POSTGRES_DB:-pulse}" -tAc "SELECT count(*) FROM information_schema.tables WHERE table_name IN ('campaigns','reports','messages','indicators','classifications','remediation_actions','audit_events');" | grep -qx 7 && echo "all 7 tables present"</automated>
</verify>
<acceptance_criteria>
- Querying `information_schema.tables` returns all 7 phishing-triage table names present in the dev DB
- `\d reports` (or an information_schema.columns query) shows columns `ticket_id`, `content_hash`, `matched_patterns`, `evidence`, and the unique constraint on `ticket_id`
- Re-running the migration produces no error (IF NOT EXISTS makes it idempotent)
</acceptance_criteria>
<done>All 7 tables exist in the dev database; the reports table has the columns and unique constraint Plan 02 depends on.</done>
</task>
</tasks>
<threat_model>
## Trust Boundaries
| Boundary | Description |
|----------|-------------|
| migration → Postgres DDL | Schema definition applied to the system-of-record DB |
## STRIDE Threat Register
| Threat ID | Category | Component | Disposition | Mitigation Plan |
|-----------|----------|-----------|-------------|-----------------|
| T-15-01 | Tampering | migration DDL against existing tables | mitigate | Schema-only, additive, every statement IF NOT EXISTS; no DROP/ALTER-DROP/TRUNCATE (enforced by acceptance criterion grep). New FKs reference existing tickets/companies only |
| T-15-02 | Denial of Service | re-applying migration on an existing volume | mitigate | Idempotent IF NOT EXISTS on tables and indexes — safe to re-run, no data loss |
| T-15-03 | Information Disclosure | reports/evidence tables hold ticket content | accept | No new external access path in this phase; standard DB access controls apply; the /api/phishing/* auth gate (ACCESS-01) lands in Phase 18 before any read surface exists |
| T-15-SC | Tampering | package installs | accept | No npm/pip/cargo installs in this plan — SQL-only migration, no new dependencies |
</threat_model>
<verification>
- `grep -c "CREATE TABLE IF NOT EXISTS" migrations/097_phishing_triage_schema.sql` returns 7
- No destructive DDL: `grep -iE "DROP|TRUNCATE" migrations/097_phishing_triage_schema.sql` returns nothing
- Dev DB shows all 7 tables + reports unique constraint on ticket_id
</verification>
<success_criteria>
Migration 097 creates all 7 phishing-triage tables with IF NOT EXISTS (ROADMAP SC#1),
the reports table is fully designed to store content_hash + matched patterns + EVID-01
evidence, and the migration is applied and verified in the dev database.
</success_criteria>
<output>
Create `.planning/phases/15-data-model-detection-ticket-evidence/15-01-SUMMARY.md` when done.
</output>

View file

@ -0,0 +1,253 @@
---
phase: 15-data-model-detection-ticket-evidence
plan: 02
type: execute
wave: 2
depends_on: ["15-01"]
files_modified:
- lib/services/phishing-detector.ts
- lib/services/phishing-detector.test.ts
autonomous: true
requirements: [DETECT-01, DETECT-02, EVID-01]
must_haves:
truths:
- "Given a ticket whose title/description contains any of the 8 known patterns, the matcher flags it and reports which patterns matched"
- "Given a ticket with none of the patterns, the matcher does not flag it"
- "The content hash is stable for identical title+description and changes when either changes"
- "Detecting a flagged ticket persists exactly one reports row with EVID-01 evidence"
- "Re-detecting an unchanged ticket does not create or duplicate a reports row; a ticket whose title/description changed IS reprocessed"
artifacts:
- path: "lib/services/phishing-detector.ts"
provides: "KNOWN_PHISHING_PATTERNS, matchesPhishingPatterns, computePhishingContentHash, gatherTicketEvidence, detectPhishingTicket"
min_lines: 120
- path: "lib/services/phishing-detector.test.ts"
provides: "unit tests for matcher + content-hash covering all 8 patterns, negative case, and hash stability/change"
key_links:
- from: "detectPhishingTicket"
to: "reports table (Plan 01)"
via: "ON CONFLICT (ticket_id) upsert guarded by content_hash comparison"
pattern: "ON CONFLICT \\(ticket_id\\)"
- from: "computePhishingContentHash"
to: "reports.content_hash"
via: "sha256 over title+description (D-04)"
pattern: "createHash\\('sha256'\\)"
---
<objective>
Build `lib/services/phishing-detector.ts` — the shared detection core called by both
the webhook path and the cron sweep (wired in Plan 03). It matches a ticket's
title+description against the 8 locked DETECT-01 patterns, computes a content hash for
D-04 idempotency, gathers EVID-01 evidence, and upserts a single `reports` row per
candidate ticket — reprocessing only when the content hash changed.
Purpose: One deterministic, testable detector with no duplicated matching logic
(CONTEXT.md discretion: "both call the same underlying logic").
Output: The detector service + a unit test file for its pure logic.
</objective>
<execution_context>
@$HOME/.claude/get-shit-done/workflows/execute-plan.md
@$HOME/.claude/get-shit-done/templates/summary.md
</execution_context>
<context>
@.planning/PROJECT.md
@.planning/ROADMAP.md
@.planning/STATE.md
@.planning/phases/15-data-model-detection-ticket-evidence/15-CONTEXT.md
@.planning/phases/15-data-model-detection-ticket-evidence/15-PATTERNS.md
@.planning/phases/15-data-model-detection-ticket-evidence/15-01-SUMMARY.md
<interfaces>
<!-- Exact contracts the detector uses. Do not re-explore. -->
reports table (Plan 01): id UUID, ticket_id BIGINT NOT NULL UNIQUE, ticket_number,
company_id, company_name, requester_contact_id, created_by_contact_id, title, description,
matched_patterns JSONB, content_hash TEXT NOT NULL, evidence JSONB, campaign_id UUID (null here),
created_at, updated_at. Upsert target: ON CONFLICT (ticket_id).
postgresClient singleton (lib/services/postgres-client.ts): postgresClient.query(sql, params)
returns { rows, rowCount }. Always use parameterized $1/$2 placeholders.
Evidence source tables (already synced, join by ticket_id):
- ticket_notes (migration 025): id, ticket_id BIGINT, title, description, note_type INTEGER, creator_resource_id, created_at
- time_entries (migration 006): id, resource_id, ticket_id, entry_date, hours_worked, start_date_time, end_date_time
- companies: id, company_name (join tickets.company_id → companies.id for company_name)
AutotaskClient.getAttachments(entityName, entityId): Promise<Attachment[]>
(lib/services/autotask-client.ts:424) — Attachment metadata fields: fullPath, title, contentType.
Persist metadata only (fullPath, title, contentType) — NOT `data` (base64); content fetch is Phase 16.
Instantiate the client the same way ticket-reconciliation-service.ts:34-45 does (env-var config,
lazy singleton), or reuse an existing factory if one is present.
Pattern-matching analog: lib/services/robotic-classifier.ts:206-219 (evaluateContains —
case-insensitive substring via .toLowerCase() + .includes(); NO regex, NO eval).
Content-hash analog: lib/services/analyzer/preprocessor.ts:251-266 (computeContentHash —
createHash('sha256') over canonical JSON). Import `createHash` from node 'crypto'.
The 8 locked DETECT-01 patterns (case-insensitive substrings — all must be covered):
"Phishing Report", "Spam Alert", "Phishing Alert - Email Security Report",
"KnowBe4 Phish Alert Report", "Source: KnowBe4 Phish Alert Button",
"userSubmissionsReportMessage", "reported message destinations", "Microsoft directly".
</interfaces>
</context>
<tasks>
<task type="auto" tdd="true">
<name>Task 1: Pure detection logic — pattern matcher + content hash (TDD)</name>
<files>lib/services/phishing-detector.ts, lib/services/phishing-detector.test.ts</files>
<behavior>
- matchesPhishingPatterns("Fwd: Phishing Report", null) → { flagged: true, matched: ["Phishing Report"] }
- matchesPhishingPatterns("Re: order confirmation", "please review invoice") → { flagged: false, matched: [] }
- Matching is case-insensitive: matchesPhishingPatterns("SPAM ALERT from user", null).flagged === true
- Each of the 8 locked patterns individually triggers a flag when present in title OR description
- matched[] contains exactly the patterns that were present (subset of KNOWN_PHISHING_PATTERNS)
- computePhishingContentHash("t","d") === computePhishingContentHash("t","d") (stable)
- computePhishingContentHash("t","d") !== computePhishingContentHash("t2","d") (title change)
- computePhishingContentHash("t","d") !== computePhishingContentHash("t","d2") (description change)
- computePhishingContentHash("t", null) is defined and stable (null description normalizes)
</behavior>
<read_first>
- lib/services/robotic-classifier.ts (evaluateContains, lines 206-219 — case-insensitive substring shape to copy)
- lib/services/analyzer/preprocessor.ts (computeContentHash, lines 233-266 — sha256 shape to mirror, hashing only title+description)
- vitest.config.ts (confirm include glob `lib/**/*.test.ts` picks up the new test file)
- .planning/phases/15-data-model-detection-ticket-evidence/15-PATTERNS.md (detector section — matcher + content-hash requirements)
</read_first>
<action>
Write the tests in `lib/services/phishing-detector.test.ts` FIRST (RED), covering
every behavior listed above — one assertion per locked pattern (all 8), the negative
case, case-insensitivity, and hash stability/change/null-normalization. Run vitest and
confirm the suite fails because the functions don't exist yet.
Then create `lib/services/phishing-detector.ts` and implement (GREEN):
- `export const KNOWN_PHISHING_PATTERNS: readonly string[]` — the 8 locked strings
exactly as given in the interfaces block above.
- `export function matchesPhishingPatterns(title: string | null, description: string | null):
{ flagged: boolean; matched: string[] }` — build a combined `${title ?? ''} ${description ?? ''}`
lowercased haystack, return matched = patterns whose lowercased form is a substring,
flagged = matched.length > 0. Use `.toLowerCase()` + `.includes()` only — NO regex, NO eval
(matches robotic-classifier.evaluateContains).
- `export function computePhishingContentHash(title: string | null, description: string | null):
string` — `createHash('sha256').update(JSON.stringify({ title: title ?? '', description: description ?? '' })).digest('hex')`.
Do NOT include last_activity_date, status, or any bump-prone field (D-04).
Run vitest again and confirm GREEN. Do not add DB access in this task — pure functions only.
</action>
<verify>
<automated>npx vitest run lib/services/phishing-detector.test.ts && npx tsc --noEmit --pretty</automated>
</verify>
<acceptance_criteria>
- `lib/services/phishing-detector.test.ts` contains at least 8 assertions covering each locked pattern by name, plus the negative case, case-insensitivity, and hash stability/change
- `npx vitest run lib/services/phishing-detector.test.ts` passes with 0 failures
- `KNOWN_PHISHING_PATTERNS` array has exactly 8 entries matching the locked strings verbatim
- `matchesPhishingPatterns` uses `.toLowerCase()` + `.includes()` (no `RegExp`, no `eval`) — grep confirms no `RegExp(`/`eval(` in the file
- `computePhishingContentHash` uses `createHash('sha256')` and hashes only title+description
- `npx tsc --noEmit --pretty` passes
</acceptance_criteria>
<done>Matcher and content-hash are implemented as pure functions, all 8 patterns are covered by passing tests, and type-check is clean.</done>
</task>
<task type="auto">
<name>Task 2: Evidence capture + detectPhishingTicket orchestration with D-04 idempotency</name>
<files>lib/services/phishing-detector.ts</files>
<read_first>
- lib/services/phishing-detector.ts (current state after Task 1 — extend, don't rewrite)
- lib/services/ticket-reconciliation-service.ts (lazy AutotaskClient singleton pattern lines 34-45; per-row try/catch + result-shape convention)
- lib/services/analyzer/persistence.ts (findExistingAnalysisByContentHash, lines 83-102 — check-before-write idempotency shape to mirror)
- lib/services/autotask-client.ts (getAttachments signature, lines 424-436; Attachment metadata fields)
- lib/services/postgres-client.ts (query/upsert API + parameterized-query convention)
</read_first>
<action>
Extend `lib/services/phishing-detector.ts` with evidence capture and the shared
orchestration entry point. Add:
- `interface DetectableTicket { id: number; ticket_number: string | null; title: string | null;
description: string | null; company_id: number | null; contact_id?: number | null;
created_by_contact_id?: number | null }` — the minimal input both callers pass.
- `async function gatherTicketEvidence(ticket: DetectableTicket): Promise<EvidencePayload>`
capture EVID-01 evidence into a JSON object:
* company_name: SELECT company_name FROM companies WHERE id = ticket.company_id (null-safe)
* notes: SELECT id, title, description, note_type, creator_resource_id, created_at FROM ticket_notes
WHERE ticket_id = $1 ORDER BY created_at (parameterized)
* time_entries: SELECT id, resource_id, entry_date, hours_worked, start_date_time, end_date_time
FROM time_entries WHERE ticket_id = $1 ORDER BY entry_date (parameterized)
* attachments: call AutotaskClient.getAttachments('Tickets', ticket.id), map to metadata only
({ fullPath, title, contentType }) — DROP `data`/base64. Wrap the Autotask call in try/catch
so an attachment-API failure degrades to an empty attachments array and logs, never throws out
of detection. Define EvidencePayload with fields: company_name, notes[], time_entries[], attachments[].
- `export async function detectPhishingTicket(ticket: DetectableTicket):
Promise<{ flagged: boolean; reportId?: string; skippedUnchanged?: boolean }>`:
1. const { flagged, matched } = matchesPhishingPatterns(ticket.title, ticket.description);
if (!flagged) return { flagged: false }.
2. const contentHash = computePhishingContentHash(ticket.title, ticket.description).
3. SELECT id, content_hash FROM reports WHERE ticket_id = $1 (parameterized). If a row exists and
its content_hash === contentHash, return { flagged: true, skippedUnchanged: true } WITHOUT
re-gathering evidence or writing (D-04 — status/assignee/last_activity bumps don't change the hash).
4. Otherwise gather evidence, then upsert:
INSERT INTO reports (ticket_id, ticket_number, company_id, company_name, requester_contact_id,
created_by_contact_id, title, description, matched_patterns, content_hash, evidence)
VALUES ($1..$11) ON CONFLICT (ticket_id) DO UPDATE SET ... , updated_at = NOW()
RETURNING id. matched_patterns and evidence bind as JSON (JSON.stringify or ::jsonb cast).
Return { flagged: true, reportId }.
Do not duplicate the matcher/hash logic — call the Task 1 functions. Use console.error with a
`[PHISHING-DETECT]` prefix for caught errors, matching the project logging convention.
</action>
<verify>
<automated>npx tsc --noEmit --pretty && npx vitest run lib/services/phishing-detector.test.ts && grep -q "ON CONFLICT (ticket_id)" lib/services/phishing-detector.ts && echo "orchestration OK"</automated>
</verify>
<acceptance_criteria>
- `detectPhishingTicket` is exported and returns `{ flagged, reportId?, skippedUnchanged? }`
- Idempotency branch present: a SELECT of reports by ticket_id compares stored `content_hash` to the freshly computed hash and returns `skippedUnchanged: true` without writing when equal
- The reports upsert uses `ON CONFLICT (ticket_id) DO UPDATE ... updated_at = NOW()` and RETURNING id
- Evidence gathering reads ticket_notes and time_entries via parameterized `$1` queries (no string interpolation of ticket_id) and stores only attachment metadata (fullPath/title/contentType), never base64 `data`
- The Autotask getAttachments call is wrapped so a failure yields an empty attachments array instead of throwing
- `npx tsc --noEmit --pretty` passes and the Task 1 vitest suite still passes
</acceptance_criteria>
<done>detectPhishingTicket matches, hashes, checks the content-hash idempotency guard, gathers EVID-01 evidence, and upserts one reports row per candidate — reprocessing only on hash change.</done>
</task>
</tasks>
<threat_model>
## Trust Boundaries
| Boundary | Description |
|----------|-------------|
| ticket title/description → matcher + hash | Untrusted ticket content flows into substring match and sha256 |
| ticket content → Postgres reports row | Untrusted content written to DB |
| Autotask Attachments API → evidence | External API metadata captured into evidence JSON |
## STRIDE Threat Register
| Threat ID | Category | Component | Disposition | Mitigation Plan |
|-----------|----------|-----------|-------------|-----------------|
| T-15-04 | Tampering (SQL injection) | reports upsert + evidence SELECTs from ticket content | mitigate | All DB access via postgresClient parameterized `$1/$2` placeholders — ticket content is never string-interpolated into SQL (acceptance criterion enforces this) |
| T-15-05 | Elevation / code exec via pattern input | matchesPhishingPatterns over attacker-controlled title | mitigate | Matching is `.toLowerCase()` + `.includes()` only — no `RegExp` construction (no ReDoS), no `eval` (grep-enforced) |
| T-15-06 | Denial of Service | crafted content re-triggering repeated processing | mitigate | D-04 content-hash guard: unchanged content skips before any evidence gathering or write; hash excludes bump-prone fields so status/assignee churn cannot force reprocessing |
| T-15-07 | Server-Side Request Forgery / unsafe fetch | attachment handling | mitigate | Only attachment metadata (fullPath/title/contentType) is stored; the detector never fetches attachment `data`, expands, or executes any URL. Autotask call wrapped in try/catch so it degrades, never blocks |
| T-15-08 | Information Disclosure | evidence JSON holds ticket notes/time entries | accept | No new external read surface in this phase (ACCESS-01 auth gate lands in Phase 18); data already resident in Postgres |
| T-15-SC | Tampering | package installs | accept | No new npm packages — uses built-in `crypto`, existing `pg` client and AutotaskClient. No install task |
</threat_model>
<verification>
- `npx vitest run lib/services/phishing-detector.test.ts` — all 8 pattern cases + negative + hash tests pass
- `npx tsc --noEmit --pretty` clean
- `grep -E "RegExp\(|eval\(" lib/services/phishing-detector.ts` returns nothing
- `grep "ON CONFLICT (ticket_id)" lib/services/phishing-detector.ts` matches
- All reports/notes/time_entries SQL uses `$1`/`$2` parameters (no interpolated ticket_id)
</verification>
<success_criteria>
The detector flags tickets matching the 8 known patterns and persists a reports row
(ROADMAP SC#2), reprocesses only when the ticket's title/description content hash changed
(SC#3, D-04), and each reports row stores EVID-01 evidence — company, requester/reporter,
title, description, notes, time entries, and attachment metadata (SC#4).
</success_criteria>
<output>
Create `.planning/phases/15-data-model-detection-ticket-evidence/15-02-SUMMARY.md` when done.
</output>

View file

@ -0,0 +1,268 @@
---
phase: 15-data-model-detection-ticket-evidence
plan: 03
type: execute
wave: 3
depends_on: ["15-02"]
files_modified:
- lib/services/phishing-sweep-service.ts
- lib/services/webhook-service.ts
- lib/services/sync-scheduler.ts
- migrations/098_phishing_sweep_schedule.sql
autonomous: true
requirements: [DETECT-01, DETECT-02]
must_haves:
truths:
- "A newly created ticket webhook fires phishing detection without blocking the webhook response"
- "A scheduled cron sweep re-scans recently-modified tickets through the same detector core"
- "The sweep is bounded (row limit) and reports a scanned/flagged/skipped/errors summary"
- "A phishing-sweep schedule row exists for both fresh installs and existing installs"
artifacts:
- path: "lib/services/phishing-sweep-service.ts"
provides: "sweepPhishingTickets() — bounded reconciliation loop over recently-modified tickets"
min_lines: 40
- path: "migrations/098_phishing_sweep_schedule.sql"
provides: "phishing-sweep sync_schedules seed row (ON CONFLICT DO NOTHING)"
contains: "phishing-sweep"
key_links:
- from: "webhook-service.ts ticket.created handler"
to: "detectPhishingTicket (Plan 02)"
via: "fire-and-forget .catch()"
pattern: "detectPhishingTicket|triggerPhishingDetection"
- from: "sync-scheduler.ts dispatch"
to: "sweepPhishingTickets"
via: "else-if branch on sync_type === 'phishing-sweep' with dynamic import"
pattern: "phishing-sweep"
---
<objective>
Wire the Plan 02 detector into Pulse's two established scan triggers (D-01):
(1) the ticket webhook path (near-real-time, fire-and-forget) and (2) a scheduled
cron reconciliation sweep over recently-modified tickets — plus the schedule seed
row for existing installs. This is what makes DETECT-01 scanning and DETECT-02
idempotency actually run in production.
Purpose: Mirror the "webhook primary, cron reconciles" pattern already used for
ticket_notes and the pax8-daily / tickets-reconcile schedule registration.
Output: A sweep service, a webhook hook-in, a scheduler branch, and migration 098.
</objective>
<execution_context>
@$HOME/.claude/get-shit-done/workflows/execute-plan.md
@$HOME/.claude/get-shit-done/templates/summary.md
</execution_context>
<context>
@.planning/PROJECT.md
@.planning/ROADMAP.md
@.planning/STATE.md
@.planning/phases/15-data-model-detection-ticket-evidence/15-CONTEXT.md
@.planning/phases/15-data-model-detection-ticket-evidence/15-PATTERNS.md
@.planning/phases/15-data-model-detection-ticket-evidence/15-02-SUMMARY.md
<interfaces>
<!-- Exact contracts to wire against. Do not re-explore. -->
Plan 02 export (lib/services/phishing-detector.ts):
detectPhishingTicket(ticket: DetectableTicket): Promise<{ flagged: boolean; reportId?: string; skippedUnchanged?: boolean }>
DetectableTicket = { id, ticket_number, title, description, company_id, contact_id?, created_by_contact_id? }
webhook-service.ts (existing precedent to copy verbatim):
- Fire-and-forget block, lines 112-117: on payload.entityType === WebhookEntityType.TICKETS &&
payload.eventType === WebhookEventType.CREATE, `this.triggerWorkflowEngine(payload).catch(err => console.error(...))`.
- triggerWorkflowEngine (lines 398-420): builds typed ticket data from payload.entity when present,
else uses payload.entityId — copy this "prefer inline entity, else id" shape.
- Imports at top, lines 13-16.
sync-scheduler.ts (existing precedent):
- ScheduleConfig.sync_type union, line 25 (currently ends `... | 'tickets-reconcile' | 'pax8-daily'`).
- defaultSchedules array — tickets-reconcile entry at lines 294-301 (id, name, description,
cron_expression, sync_type, is_enabled: false). Seeded via ON CONFLICT (id) DO NOTHING (lines 304-318).
- Dispatch chain — tickets-reconcile branch at lines 458-463: `} else if (config.sync_type === 'tickets-reconcile') {
const { reconcileStaleTickets } = await import('@/lib/services/ticket-reconciliation-service');
const result = await reconcileStaleTickets(); console.log('[SCHEDULER] tickets-reconcile: ...'); }`.
Every branch uses dynamic `await import()` — required per CLAUDE.md (no eager worker/scheduler imports).
sweep source (ticket-reconciliation-service.ts:51-151): bounded SELECT (LIMIT), per-row try/catch,
aggregate result object, createSyncLogger. tickets columns: id, ticket_number, title, description,
company_id, contact_id, created_by_contact_id, last_activity_date.
migration seed precedent: migrations/090_ticket_reconcile_schedule.sql (INSERT INTO sync_schedules
(id, name, description, cron_expression, sync_type, years_back, is_enabled) VALUES (...) ON CONFLICT (id) DO NOTHING).
sync_schedules.id is a text PK (values like 'tickets-reconcile').
</interfaces>
</context>
<tasks>
<task type="auto">
<name>Task 1: phishing-sweep-service.ts — bounded reconciliation sweep</name>
<files>lib/services/phishing-sweep-service.ts</files>
<read_first>
- lib/services/ticket-reconciliation-service.ts (full file — SELECT-with-LIMIT, per-row try/catch, ReconcileResult aggregate shape, createSyncLogger usage to mirror)
- lib/services/phishing-detector.ts (detectPhishingTicket signature + DetectableTicket shape from Plan 02)
- lib/services/postgres-client.ts (query API + parameterized-query convention)
</read_first>
<action>
Create `lib/services/phishing-sweep-service.ts`. Export
`interface PhishingSweepResult { scanned: number; flagged: number; skippedUnchanged: number; errors: number }`
and `export async function sweepPhishingTickets(): Promise<PhishingSweepResult>`.
Implementation (mirror reconcileStaleTickets structure):
- Use `createSyncLogger({ component: 'PhishingSweep' })`.
- Define module constants `SWEEP_WINDOW_DAYS = 7` and `SCAN_LIMIT = 500` (bounded — DoS guard).
- Query recently-modified, non-deleted tickets:
`SELECT id, ticket_number, title, description, company_id, contact_id, created_by_contact_id
FROM tickets WHERE is_deleted = false AND last_activity_date > NOW() - INTERVAL '7 days'
ORDER BY last_activity_date DESC LIMIT $1` with `[SCAN_LIMIT]` (parameterized; the interval
may be inlined as a literal since it is a constant, or bound as a parameter).
- For each row, call `detectPhishingTicket({ id: Number(row.id), ticket_number, title, description,
company_id, contact_id, created_by_contact_id })` inside try/catch. Increment
result.scanned always; result.flagged when `res.flagged && !res.skippedUnchanged`;
result.skippedUnchanged when `res.skippedUnchanged`; result.errors on caught exception
(logger.warn with the ticket id, never rethrow — one bad ticket must not abort the sweep).
- Log a final summary line and return the result.
Do NOT re-implement matching/hashing — call the shared detectPhishingTicket only (CONTEXT.md:
"no duplicated pattern-matching logic"). Do NOT add a side-effect self-init or eager import.
</action>
<verify>
<automated>npx tsc --noEmit --pretty && grep -q "detectPhishingTicket" lib/services/phishing-sweep-service.ts && grep -q "LIMIT" lib/services/phishing-sweep-service.ts && echo "sweep OK"</automated>
</verify>
<acceptance_criteria>
- `lib/services/phishing-sweep-service.ts` exports `sweepPhishingTickets` returning `{ scanned, flagged, skippedUnchanged, errors }`
- The ticket query is bounded by `LIMIT` (<= 500) and filters `is_deleted = false` and a recent `last_activity_date` window
- Each ticket is processed by calling `detectPhishingTicket` (the file imports it from `./phishing-detector`) — no duplicated match/hash logic
- A per-row try/catch increments `errors` and logs without aborting the loop (no rethrow)
- No module-level side-effect / self-init / eager worker import
- `npx tsc --noEmit --pretty` passes
</acceptance_criteria>
<done>sweepPhishingTickets scans a bounded set of recently-modified tickets through the shared detector and returns an aggregate summary.</done>
</task>
<task type="auto">
<name>Task 2: Webhook hook-in — fire-and-forget phishing detection on ticket.created</name>
<files>lib/services/webhook-service.ts</files>
<read_first>
- lib/services/webhook-service.ts (full file — imports at 13-16, ticket.created fire-and-forget block at 112-117, triggerWorkflowEngine payload-shaping at 398-420)
- lib/services/phishing-detector.ts (detectPhishingTicket + DetectableTicket shape)
</read_first>
<action>
Modify `lib/services/webhook-service.ts` to trigger phishing detection on ticket
creation, exactly mirroring the existing workflow-engine fire-and-forget precedent.
- Add an import for `detectPhishingTicket` from `./phishing-detector` alongside the
existing service imports at the top of the file (lines 13-16).
- Add a private method `triggerPhishingDetection(payload: AutotaskWebhookPayload): Promise<void>`
that builds a DetectableTicket the same "prefer inline payload.entity, else use payload.entityId"
way triggerWorkflowEngine does (lines 398-420): if payload.entity is present, map
title/description/ticketNumber/companyID/contactID/creatorContactID from it; otherwise pass a
minimal ticket with `id: payload.entityId` and null fields (the detector's evidence/hash still
works, and the cron sweep reconciles anything the webhook payload lacked). Call
`await detectPhishingTicket(ticket)`.
- In the existing ticket.created fire-and-forget block (lines 112-117), add a second
identically-shaped call immediately after the triggerWorkflowEngine call:
`this.triggerPhishingDetection(payload).catch(err => console.error('[WEBHOOK] Phishing detection error:', err));`.
Do NOT `await` it in the request path — detection runs after the webhook response, matching the
workflow-engine precedent. Do not change the workflow-engine call or any other webhook behavior.
</action>
<verify>
<automated>npx tsc --noEmit --pretty && grep -q "triggerPhishingDetection" lib/services/webhook-service.ts && grep -q "Phishing detection error" lib/services/webhook-service.ts && echo "hook OK"</automated>
</verify>
<acceptance_criteria>
- `webhook-service.ts` imports `detectPhishingTicket` from `./phishing-detector`
- A `triggerPhishingDetection` method exists and builds ticket data using the "prefer payload.entity, else payload.entityId" shape
- The ticket.created handler calls `this.triggerPhishingDetection(payload).catch(...)` as fire-and-forget (not awaited in the request path), immediately alongside the existing `triggerWorkflowEngine` call
- The existing workflow-engine trigger and all other webhook behavior are unchanged (grep still finds `triggerWorkflowEngine`)
- `npx tsc --noEmit --pretty` passes
</acceptance_criteria>
<done>New ticket webhooks fire phishing detection without blocking the webhook response, mirroring the workflow-engine trigger precedent.</done>
</task>
<task type="auto">
<name>Task 3: Scheduler branch + defaultSchedules entry + migration 098 seed</name>
<files>lib/services/sync-scheduler.ts, migrations/098_phishing_sweep_schedule.sql</files>
<read_first>
- lib/services/sync-scheduler.ts (sync_type union line 25; defaultSchedules tickets-reconcile entry lines 294-301 + seeding loop 304-318; dispatch chain tickets-reconcile branch 458-463)
- migrations/090_ticket_reconcile_schedule.sql (seed-row shape + ON CONFLICT (id) DO NOTHING)
- lib/services/phishing-sweep-service.ts (sweepPhishingTickets export from Task 1)
</read_first>
<action>
Register the phishing reconciliation sweep as a scheduled sync, using the
`tickets-reconcile` entry as the closest analog (no external-integration config gate needed).
In `lib/services/sync-scheduler.ts`:
- Extend the `ScheduleConfig.sync_type` union (line 25) by adding `| 'phishing-sweep'`.
- Add a new entry to the `defaultSchedules` array (copy the tickets-reconcile shape at
lines 294-301): id `'phishing-sweep'`, name `'Phishing Detection Sweep'`, a description stating
it reconciles recently-modified tickets through the phishing detector to catch reports missed by
the webhook path (bounded to 500 tickets, idempotent on content hash), cron_expression a daily
time distinct from other jobs (e.g. `'0 5 * * *'`), sync_type `'phishing-sweep'`,
is_enabled: false (disabled-by-default, matching tickets-reconcile precedent).
- Add a dispatch branch mirroring the tickets-reconcile branch (lines 458-463):
`} else if (config.sync_type === 'phishing-sweep') {
const { sweepPhishingTickets } = await import('@/lib/services/phishing-sweep-service');
const result = await sweepPhishingTickets();
console.log('[SCHEDULER] phishing-sweep: scanned=... flagged=... skippedUnchanged=... errors=...'); }`
using dynamic `await import` (required — no eager import).
Create `migrations/098_phishing_sweep_schedule.sql` (copy migrations/090's shape): a header
comment noting createDefaultSchedules only seeds a virgin table so this covers existing installs,
then `INSERT INTO sync_schedules (id, name, description, cron_expression, sync_type, years_back,
is_enabled) VALUES ('phishing-sweep', 'Phishing Detection Sweep', '<same description>', '0 5 * * *',
'phishing-sweep', NULL, false) ON CONFLICT (id) DO NOTHING;`. Keep the description text identical to
the defaultSchedules entry. Do not edit any committed migration.
</action>
<verify>
<automated>npx tsc --noEmit --pretty && grep -q "'phishing-sweep'" lib/services/sync-scheduler.ts && grep -q "sweepPhishingTickets" lib/services/sync-scheduler.ts && test -f migrations/098_phishing_sweep_schedule.sql && grep -q "ON CONFLICT (id) DO NOTHING" migrations/098_phishing_sweep_schedule.sql && echo "scheduler+migration OK"</automated>
</verify>
<acceptance_criteria>
- `sync_type` union in sync-scheduler.ts includes `'phishing-sweep'`
- `defaultSchedules` contains a `phishing-sweep` entry with `is_enabled: false` and a valid cron expression
- A dispatch branch `else if (config.sync_type === 'phishing-sweep')` dynamically imports and calls `sweepPhishingTickets` and logs a scanned/flagged/skipped/errors summary
- `migrations/098_phishing_sweep_schedule.sql` exists, inserts the `phishing-sweep` row, and uses `ON CONFLICT (id) DO NOTHING`
- The migration description text matches the defaultSchedules entry description
- `npx tsc --noEmit --pretty` passes
</acceptance_criteria>
<done>The phishing sweep is registered in the scheduler (disabled by default) and seeded for existing installs via migration 098, invoking the shared sweep service on each tick.</done>
</task>
</tasks>
<threat_model>
## Trust Boundaries
| Boundary | Description |
|----------|-------------|
| Autotask webhook → detection trigger | External webhook event initiates a background code path |
| cron scheduler → sweep | Time-triggered bulk processing over local tickets |
## STRIDE Threat Register
| Threat ID | Category | Component | Disposition | Mitigation Plan |
|-----------|----------|-----------|-------------|-----------------|
| T-15-09 | Denial of Service | repeated/replayed ticket.created webhooks | mitigate | Detection is fire-and-forget (never blocks the webhook response) AND idempotent via the Plan 02 content-hash guard — replays of an unchanged ticket skip before any write. Webhook HMAC verification (existing middleware/webhook-service) still gates the endpoint |
| T-15-10 | Denial of Service | unbounded cron sweep | mitigate | sweepPhishingTickets is bounded by `LIMIT 500` and a 7-day recent-activity window; per-row try/catch isolates failures so one ticket cannot abort the run |
| T-15-11 | Tampering | new schedule row on existing installs | mitigate | migration 098 is `ON CONFLICT (id) DO NOTHING` (idempotent) and additive; disabled-by-default (is_enabled=false) so it does nothing until an admin opts in |
| T-15-12 | Repudiation | sweep/detection produces no trace | accept | Both paths log via createSyncLogger / console.error with `[PHISHING-DETECT]`/`[SCHEDULER]` prefixes; full audit_events wiring is Phase 20 scope |
| T-15-SC | Tampering | package installs | accept | No new npm packages — reuses postgresClient, AutotaskClient, node-cron scheduler, and the Plan 02 detector. No install task |
</threat_model>
<verification>
- `npx tsc --noEmit --pretty` clean across all three modified/created files
- `grep "'phishing-sweep'" lib/services/sync-scheduler.ts` finds union member, default entry, and dispatch branch
- `grep "triggerPhishingDetection" lib/services/webhook-service.ts` finds the fire-and-forget hook
- `grep "ON CONFLICT (id) DO NOTHING" migrations/098_phishing_sweep_schedule.sql` matches
- Sweep query bounded by LIMIT (DoS guard)
</verification>
<success_criteria>
The detector runs on both triggers per D-01: near-real-time via the webhook ticket.created
path (fire-and-forget, ROADMAP SC#2) and via a bounded scheduled reconciliation sweep, both
calling the same shared detectPhishingTicket so idempotency (SC#3) holds on either path. The
phishing-sweep schedule is registered for fresh and existing installs.
</success_criteria>
<output>
Create `.planning/phases/15-data-model-detection-ticket-evidence/15-03-SUMMARY.md` when done.
</output>