chore: merge executor worktree (worktree-agent-aa19352a908fc6624)
This commit is contained in:
commit
6787308c04
2 changed files with 255 additions and 0 deletions
|
|
@ -0,0 +1,98 @@
|
|||
---
|
||||
phase: 15-data-model-detection-ticket-evidence
|
||||
plan: 01
|
||||
subsystem: database
|
||||
tags: [postgres, migration, schema, phishing-triage]
|
||||
|
||||
# Dependency graph
|
||||
requires: []
|
||||
provides:
|
||||
- "migrations/097_phishing_triage_schema.sql applied to dev DB"
|
||||
- "7-table phishing-triage schema: campaigns, reports, messages, indicators, classifications, remediation_actions, audit_events"
|
||||
- "reports table fully designed with ticket_id FK, content_hash idempotency key, matched_patterns/evidence JSONB columns, uq_reports_ticket_id unique constraint"
|
||||
affects: [15-02-phishing-detector, 16-message-parsing, 18-campaign-grouping, 19-classification, 20-remediation]
|
||||
|
||||
# Tech tracking
|
||||
tech-stack:
|
||||
added: []
|
||||
patterns:
|
||||
- "Schema-only foundation migration lands before any service writes to it (mirrors migration 091 pax8 pattern)"
|
||||
- "UUID PKs via gen_random_uuid() (no extension needed, Postgres 16 built-in)"
|
||||
- "Stub tables for future-phase entities created now with hard FKs where insert order is guaranteed within the same migration"
|
||||
|
||||
key-files:
|
||||
created: [migrations/097_phishing_triage_schema.sql]
|
||||
modified: []
|
||||
|
||||
key-decisions:
|
||||
- "reports.ticket_id is a hard FK to tickets(id) with UNIQUE constraint (uq_reports_ticket_id) so the Plan 02 detector can upsert one report per ticket via ON CONFLICT (ticket_id)"
|
||||
- "campaigns/messages/indicators/classifications/remediation_actions/audit_events created as stubs now (per STATE.md decision: durable schema lands in Phase 15, before any service that writes to it) so Phases 16-21 never need a second foundation migration"
|
||||
- "Applied migration directly via docker exec psql (not scripts/apply-migrations.sh) because that script hardcodes the main-repo path /opt/stacks/pulse/migrations, which does not yet contain this file while running inside a worktree"
|
||||
|
||||
patterns-established:
|
||||
- "Pattern: schema-only migrations for future phases mirror migrations/091_pax8_tables.sql header framing (numbered list of tables, explicit note on what's populated now vs. stubbed)"
|
||||
|
||||
requirements-completed: [DETECT-01, DETECT-02, EVID-01]
|
||||
|
||||
# Metrics
|
||||
duration: 12min
|
||||
completed: 2026-07-15
|
||||
---
|
||||
|
||||
# Phase 15 Plan 01: Phishing Triage Schema Foundation Summary
|
||||
|
||||
**Migration 097 lays down the full 7-table phishing-triage schema in Postgres — reports table fully designed for ticket_id, content_hash idempotency, matched_patterns, and EVID-01 evidence; six other tables stubbed for Phases 16-21 — applied and verified in the dev DB.**
|
||||
|
||||
## Performance
|
||||
|
||||
- **Duration:** 12 min
|
||||
- **Started:** 2026-07-15T11:23:00Z
|
||||
- **Completed:** 2026-07-15T11:35:46Z
|
||||
- **Tasks:** 2 completed
|
||||
- **Files modified:** 1
|
||||
|
||||
## Accomplishments
|
||||
- Created `migrations/097_phishing_triage_schema.sql` with all 7 phishing-triage tables (campaigns, reports, messages, indicators, classifications, remediation_actions, audit_events), each `CREATE TABLE IF NOT EXISTS`, snake_case columns, UUID PKs via `gen_random_uuid()`.
|
||||
- `reports` table fully designed: `ticket_id BIGINT NOT NULL REFERENCES tickets(id)`, `content_hash TEXT NOT NULL` (D-04 idempotency), `matched_patterns JSONB NOT NULL DEFAULT '[]'`, `evidence JSONB NOT NULL DEFAULT '{}'`, `campaign_id UUID REFERENCES campaigns(id)` (nullable), and `CONSTRAINT uq_reports_ticket_id UNIQUE (ticket_id)`.
|
||||
- Applied migration to the running dev Postgres (`pulse-postgres` container) via `docker exec -i pulse-postgres psql`.
|
||||
- Verified all 7 tables exist in `information_schema.tables`, confirmed `reports` columns/constraint via `\d reports`, and re-ran the migration to confirm full idempotency (all statements returned `NOTICE: ... already exists, skipping`, zero errors).
|
||||
|
||||
## Task Commits
|
||||
|
||||
1. **Task 1: Write migration 097 — 7-table phishing triage schema** - `84a37e2` (feat)
|
||||
2. **Task 2: Apply migration 097 to the dev database and verify tables exist** - no commit (DB-only verification step; no file changes produced — the migration file was already committed in Task 1)
|
||||
|
||||
**Plan metadata:** (this SUMMARY.md commit)
|
||||
|
||||
## Files Created/Modified
|
||||
- `migrations/097_phishing_triage_schema.sql` - 7-table phishing-triage schema: campaigns, reports (fully designed for Plan 02's detector), messages, indicators, classifications, remediation_actions, audit_events (all stubs except reports)
|
||||
|
||||
## Decisions Made
|
||||
- Hard FK + UNIQUE constraint on `reports.ticket_id` (rather than a soft ref like PAX8's company/product tables) because report inserts always happen one-at-a-time against an already-synced ticket — no batch-insert-order ambiguity to avoid, unlike PAX8's multi-entity sync passes.
|
||||
- Applied the migration directly via `docker exec -i pulse-postgres psql -U pulse_user -d pulse_autotask < migrations/097_phishing_triage_schema.sql` instead of `scripts/apply-migrations.sh`, because that script hardcodes `MIGRATIONS_DIR="/opt/stacks/pulse/migrations"` (the main repo checkout) — inside this worktree the new file doesn't exist at that path yet, so the script would silently fail to find it. Documented here for the orchestrator/user; no change made to the script itself (out of scope for this plan).
|
||||
|
||||
## Deviations from Plan
|
||||
|
||||
None - plan executed exactly as written. Task 2's verification uncovered a worktree-path caveat with `scripts/apply-migrations.sh` (documented above under Decisions Made) but this did not require any code change — the plan's own fallback instruction ("Otherwise pipe the file into the container's psql") was used as designed.
|
||||
|
||||
## Issues Encountered
|
||||
None.
|
||||
|
||||
## User Setup Required
|
||||
None - no external service configuration required. Migration applied directly to the existing dev Postgres container; no new env vars or credentials needed.
|
||||
|
||||
## Next Phase Readiness
|
||||
- All 7 phishing-triage tables exist in the dev database and are idempotently re-appliable.
|
||||
- `reports` table is fully ready for Plan 02's phishing-detector service to upsert into via `ON CONFLICT (ticket_id)`.
|
||||
- `campaigns`, `messages`, `indicators`, `classifications`, `remediation_actions`, `audit_events` are schema-ready stubs for Phases 16-21 — no second foundation migration will be needed.
|
||||
- No blockers.
|
||||
|
||||
---
|
||||
*Phase: 15-data-model-detection-ticket-evidence*
|
||||
*Completed: 2026-07-15*
|
||||
|
||||
## Self-Check: PASSED
|
||||
- FOUND: migrations/097_phishing_triage_schema.sql
|
||||
- FOUND: .planning/phases/15-data-model-detection-ticket-evidence/15-01-SUMMARY.md
|
||||
- FOUND: commit 84a37e2
|
||||
- FOUND: commit 15d8a69
|
||||
157
migrations/097_phishing_triage_schema.sql
Normal file
157
migrations/097_phishing_triage_schema.sql
Normal file
|
|
@ -0,0 +1,157 @@
|
|||
-- Phishing Triage — Postgres schema (v3.0).
|
||||
--
|
||||
-- Schema-only migration (Phase 15) — lays down the full durable schema every
|
||||
-- v3.0 phase reads and writes, before any service writes to it.
|
||||
--
|
||||
-- • Campaign grouping (dedupe multiple reports of the same lure) -> campaigns
|
||||
-- • One row per reported/triaged ticket (Phase 15 populates this) -> reports
|
||||
-- • Parsed email content per report -> messages (stub, Phase 16)
|
||||
-- • Extracted URLs/senders/hashes per message -> indicators (stub, Phase 16)
|
||||
-- • Campaign verdict + recommended actions -> classifications (stub, Phase 19)
|
||||
-- • Proposed/approved remediation actions per campaign -> remediation_actions (stub, Phase 20)
|
||||
-- • Audit trail of triage/remediation events -> audit_events (stub, Phase 20)
|
||||
--
|
||||
-- Phase 15 only populates `reports` (via the Plan 02 phishing-detector
|
||||
-- service). `campaigns`, `messages`, `indicators`, `classifications`,
|
||||
-- `remediation_actions`, and `audit_events` are laid down now as stubs so
|
||||
-- Phases 16-21 have their schema ready and never need a second foundation
|
||||
-- migration. Postgres 16 has built-in gen_random_uuid() — no extension needed.
|
||||
|
||||
-- ---------------------------------------------------------------------------
|
||||
-- 1. campaigns — groups multiple reports of the same phishing lure
|
||||
-- ---------------------------------------------------------------------------
|
||||
-- Populated starting Phase 18 (campaign grouping). reports.campaign_id is
|
||||
-- nullable until then.
|
||||
|
||||
CREATE TABLE IF NOT EXISTS campaigns (
|
||||
id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
|
||||
campaign_key TEXT,
|
||||
group_method TEXT,
|
||||
first_seen_at TIMESTAMPTZ,
|
||||
last_seen_at TIMESTAMPTZ,
|
||||
report_count INTEGER NOT NULL DEFAULT 0,
|
||||
status TEXT NOT NULL DEFAULT 'open',
|
||||
created_at TIMESTAMPTZ NOT NULL DEFAULT NOW(),
|
||||
updated_at TIMESTAMPTZ NOT NULL DEFAULT NOW()
|
||||
);
|
||||
|
||||
CREATE INDEX IF NOT EXISTS idx_campaigns_campaign_key ON campaigns(campaign_key);
|
||||
|
||||
-- ---------------------------------------------------------------------------
|
||||
-- 2. reports — one row per reported/triaged ticket (the only table Phase 15
|
||||
-- populates; fully designed here)
|
||||
-- ---------------------------------------------------------------------------
|
||||
-- ticket_id is a hard FK to tickets(id) — every report originates from an
|
||||
-- Autotask ticket already synced into Postgres. content_hash is the D-04
|
||||
-- idempotency key (hash over title+description) the Plan 02 detector uses to
|
||||
-- avoid re-processing an unchanged ticket. matched_patterns captures the
|
||||
-- DETECT-01 pattern strings that matched. evidence captures the EVID-01
|
||||
-- notes/time_entries/attachments snapshot at detection time. campaign_id is
|
||||
-- nullable — Phase 18's campaign-grouping logic links it later.
|
||||
-- uq_reports_ticket_id lets the Plan 02 detector upsert with
|
||||
-- ON CONFLICT (ticket_id) — one report per ticket.
|
||||
|
||||
CREATE TABLE IF NOT EXISTS reports (
|
||||
id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
|
||||
ticket_id BIGINT NOT NULL REFERENCES tickets(id),
|
||||
ticket_number VARCHAR(100),
|
||||
company_id BIGINT,
|
||||
company_name VARCHAR(255),
|
||||
requester_contact_id BIGINT,
|
||||
created_by_contact_id BIGINT,
|
||||
title VARCHAR(255),
|
||||
description TEXT,
|
||||
matched_patterns JSONB NOT NULL DEFAULT '[]'::jsonb,
|
||||
content_hash TEXT NOT NULL,
|
||||
evidence JSONB NOT NULL DEFAULT '{}'::jsonb,
|
||||
campaign_id UUID REFERENCES campaigns(id),
|
||||
created_at TIMESTAMPTZ NOT NULL DEFAULT NOW(),
|
||||
updated_at TIMESTAMPTZ NOT NULL DEFAULT NOW(),
|
||||
CONSTRAINT uq_reports_ticket_id UNIQUE (ticket_id)
|
||||
);
|
||||
|
||||
CREATE INDEX IF NOT EXISTS idx_reports_ticket_id ON reports(ticket_id);
|
||||
CREATE INDEX IF NOT EXISTS idx_reports_content_hash ON reports(content_hash);
|
||||
CREATE INDEX IF NOT EXISTS idx_reports_campaign_id ON reports(campaign_id);
|
||||
|
||||
-- ---------------------------------------------------------------------------
|
||||
-- 3. messages — parsed email content per report (stub for Phase 16)
|
||||
-- ---------------------------------------------------------------------------
|
||||
|
||||
CREATE TABLE IF NOT EXISTS messages (
|
||||
id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
|
||||
report_id UUID REFERENCES reports(id),
|
||||
message_id TEXT,
|
||||
headers JSONB,
|
||||
urls JSONB,
|
||||
attachments JSONB,
|
||||
body_preview TEXT,
|
||||
raw_ref TEXT,
|
||||
created_at TIMESTAMPTZ NOT NULL DEFAULT NOW()
|
||||
);
|
||||
|
||||
CREATE INDEX IF NOT EXISTS idx_messages_report_id ON messages(report_id);
|
||||
CREATE INDEX IF NOT EXISTS idx_messages_message_id ON messages(message_id);
|
||||
|
||||
-- ---------------------------------------------------------------------------
|
||||
-- 4. indicators — extracted URLs/senders/hashes per message (stub for Phase 16)
|
||||
-- ---------------------------------------------------------------------------
|
||||
|
||||
CREATE TABLE IF NOT EXISTS indicators (
|
||||
id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
|
||||
message_id UUID REFERENCES messages(id),
|
||||
indicator_type TEXT NOT NULL,
|
||||
value TEXT NOT NULL,
|
||||
created_at TIMESTAMPTZ NOT NULL DEFAULT NOW()
|
||||
);
|
||||
|
||||
CREATE INDEX IF NOT EXISTS idx_indicators_message_id ON indicators(message_id);
|
||||
|
||||
-- ---------------------------------------------------------------------------
|
||||
-- 5. classifications — campaign verdict + recommended actions (stub for Phase 19)
|
||||
-- ---------------------------------------------------------------------------
|
||||
|
||||
CREATE TABLE IF NOT EXISTS classifications (
|
||||
id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
|
||||
campaign_id UUID REFERENCES campaigns(id),
|
||||
verdict TEXT,
|
||||
confidence NUMERIC,
|
||||
summary TEXT,
|
||||
reasons JSONB,
|
||||
recommended_actions JSONB,
|
||||
requires_approval BOOLEAN NOT NULL DEFAULT false,
|
||||
created_at TIMESTAMPTZ NOT NULL DEFAULT NOW()
|
||||
);
|
||||
|
||||
CREATE INDEX IF NOT EXISTS idx_classifications_campaign_id ON classifications(campaign_id);
|
||||
|
||||
-- ---------------------------------------------------------------------------
|
||||
-- 6. remediation_actions — proposed/approved remediation per campaign (stub
|
||||
-- for Phase 20)
|
||||
-- ---------------------------------------------------------------------------
|
||||
|
||||
CREATE TABLE IF NOT EXISTS remediation_actions (
|
||||
id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
|
||||
campaign_id UUID REFERENCES campaigns(id),
|
||||
action_type TEXT,
|
||||
status TEXT NOT NULL DEFAULT 'proposed',
|
||||
params JSONB,
|
||||
approved_by TEXT,
|
||||
approved_at TIMESTAMPTZ,
|
||||
created_at TIMESTAMPTZ NOT NULL DEFAULT NOW()
|
||||
);
|
||||
|
||||
CREATE INDEX IF NOT EXISTS idx_remediation_actions_campaign_id ON remediation_actions(campaign_id);
|
||||
|
||||
-- ---------------------------------------------------------------------------
|
||||
-- 7. audit_events — audit trail of triage/remediation events (stub for Phase 20)
|
||||
-- ---------------------------------------------------------------------------
|
||||
|
||||
CREATE TABLE IF NOT EXISTS audit_events (
|
||||
id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
|
||||
campaign_id UUID,
|
||||
actor TEXT,
|
||||
event_type TEXT NOT NULL,
|
||||
payload JSONB,
|
||||
created_at TIMESTAMPTZ NOT NULL DEFAULT NOW()
|
||||
);
|
||||
Loading…
Add table
Add a link
Reference in a new issue