diff --git a/.planning/ROADMAP.md b/.planning/ROADMAP.md index a4a7c06..484d51f 100644 --- a/.planning/ROADMAP.md +++ b/.planning/ROADMAP.md @@ -9,6 +9,7 @@ ## Phases **Phase Numbering:** + - Integer phases (1, 2, 3): Planned milestone work - Decimal phases (2.1, 2.2): Urgent insertions (marked with INSERTED) @@ -40,145 +41,192 @@ under `/mobile/*` — no `/mobile-v2`, no parallel routes. - [x] **Phase 9.1: ntfy Backend Fix (INSERTED — urgent)** — Personal ntfy channels target the company ntfy server with bearer auth + `pulse-me-` prefix (UAT gap closure) ### Phase 1: PWA Scaffolding + **Goal**: A manager who taps "Add to Home Screen" gets a standalone Pulse icon that opens to the mobile shell with content respecting the device safe areas. **Depends on**: Nothing (first phase) **Requirements**: PWA-01, PWA-02, PWA-03, PWA-04 **Success Criteria** (what must be TRUE): + 1. Visiting `/manifest.json` returns valid JSON with `name: "Pulse"`, `display: "standalone"`, `start_url: "/mobile"`, and theme/background colors matching the app shells 2. The root `app/layout.tsx` references the manifest via `` and the viewport meta includes `viewport-fit=cover` 3. A safe-area utility (Tailwind arbitrary values or shared class) is available so any sticky top/bottom bar can opt into `env(safe-area-inset-top)` / `env(safe-area-inset-bottom)` padding 4. Installing Pulse to a phone home screen launches a chromeless app pointed at `/mobile` (no service worker, no offline) + **Plans**: 2 plans + - [x] 01-01-PLAN.md — Web App Manifest + viewport-fit=cover (PWA-01, PWA-02, PWA-03) - [x] 01-02-PLAN.md — Safe-area `pt-safe` / `pb-safe` @utility blocks in brand.css (PWA-04, gap closure) + **UI hint**: no ### Phase 2: Mobile Shell + More Drawer + **Goal**: Every `/mobile/*` page renders inside a new layout — sticky header (Wulf mark + Bell placeholder + avatar), scrollable content, and a 5-cell bottom nav whose fifth control opens a Sheet drawer that fully replaces `/mobile/nav`. **Depends on**: Phase 1 **Requirements**: SHELL-01, SHELL-02, SHELL-03, SHELL-04, SHELL-05, SHELL-06, NAV-01, NAV-02, NAV-03, DRAWER-01, DRAWER-02, DRAWER-03, DRAWER-04, DRAWER-05, DRAWER-06 **Success Criteria** (what must be TRUE): + 1. On any `/mobile/*` route the user sees a sticky header with the Wulf wordmark linking to `/mobile/dashboard`, a Bell icon button (keyboard-focusable, no menu), and a compact avatar — no page title in the header 2. A fixed bottom bar exposes four primary tabs (Dashboard, Tickets, Finance, Analyzer) plus a More cell; tapping a tab routes to its page and the active tab uses `text-primary` based on `pathname.startsWith(href)` 3. Tapping More (or the header avatar) opens a single Sheet drawer with three sections — Mobile sections (Engagement), Full site (Quotes, Configuration Items, Backup Status, Ticket Digest, Admin/Sync — each with an `ExternalLink` hint), and Account (current user read-only + Sign out) 4. Tapping Sign out in the drawer signs the user out and lands them on `/auth/sign-in` 5. `app/mobile/nav/page.tsx` no longer exists; visiting `/mobile/nav` does not render the old standalone nav page 6. Page content scrolls under the sticky header and is not hidden behind the bottom nav (bottom padding accounts for nav height + safe-area inset) + **Plans**: 2 plans + - [x] 02-01-PLAN.md — Build mobile shell components (HeaderBar, BottomNav, MoreDrawer) + analyzer placeholder (SHELL-02..04, SHELL-06, NAV-01..03, DRAWER-01..05) - [x] 02-02-PLAN.md — Wire new components into app/mobile/layout.tsx, delete app/mobile/nav/page.tsx (SHELL-01, SHELL-05, DRAWER-06) + **UI hint**: yes ### Phase 3: Dashboard Restyle + **Goal**: A manager opening `/mobile/dashboard` sees the state of the business at a glance — four KPIs, items needing attention, and a worker/backup status row — with no charts. **Depends on**: Phase 2 **Requirements**: DASH-01, DASH-02, DASH-03, DASH-04 **Success Criteria** (what must be TRUE): + 1. Dashboard renders a 2×2 grid of four primary KPI cards drawn from desktop hero stats (no 1×4 row, no charts) 2. Below the grid, a "Needs Attention" horizontally-scrollable strip surfaces overdue tickets, failed backups, and stalled workflows; tapping a card opens its detail view 3. A compact status row shows analyzer worker, RMM worker, and backup-success-rate; tapping any element opens the corresponding desktop admin page 4. The page contains no recharts/chart components on phone widths + **Plans**: 2 plans + - [x] 03-01-PLAN.md — /api/mobile/dashboard reshape + KpiCardMobile/NeedsAttentionStrip/WorkerStatusRow components (DASH-01, DASH-02, DASH-03) - [x] 03-02-PLAN.md — Replace /mobile/dashboard page body with 3-section layout, no charts (DASH-01, DASH-02, DASH-03, DASH-04) + **UI hint**: yes ### Phase 4: Tickets Restyle + **Goal**: A manager triages tickets on a phone with a collapsible filter bar that deep-links via URL, priority-coloured rows, and infinite scroll — and the detail page header matches the new shell. **Depends on**: Phase 2 **Requirements**: TICK-01, TICK-02, TICK-03, TICK-04, TICK-05, TICK-06, TICK-07 **Success Criteria** (what must be TRUE): + 1. The Tickets page opens with the filter strip collapsed; expanding it reveals status, priority, queue, and an assigned-to-me toggle, and changing any filter updates the URL query string (deep link works on reload) 2. Each list row has a left-edge stripe matching priority (Critical/High/Medium/Low → red/orange/amber/slate) and shows ticket #, title, company, age, and assignee 3. Single-tapping a row navigates to `/mobile/tickets/[id]` 4. Scrolling to the bottom of the list automatically loads the next ~25 rows (no Next button); a "Load more" fallback button is also visible/focusable for accessibility 5. The detail page header uses the new shell styling (Wulf mark, breadcrumb back) while the body remains largely unchanged + **Plans**: 3 plans + - [x] 04-01-PLAN.md — /api/mobile/tickets cursor rewrite + TicketFilterStrip + TicketRowSkeleton components (TICK-01, TICK-02, TICK-05) - [x] 04-02-PLAN.md — Replace app/mobile/tickets/page.tsx with URL-synced filters, priority-stripe rows, IntersectionObserver infinite scroll (TICK-01..TICK-06) - [x] 04-03-PLAN.md — Reskin in-page header of app/mobile/tickets/[id]/page.tsx (back chevron + breadcrumb + ExternalLink) (TICK-07) + **UI hint**: yes ### Phase 5: Finance Restyle + **Goal**: A manager reading AR / invoice / payment status on a phone sees properly spaced cards and stacked lists instead of squished wide tables — same data, new shell. **Depends on**: Phase 2 **Requirements**: FIN-01, FIN-02 **Success Criteria** (what must be TRUE): + 1. `/mobile/finance` adopts the new Card and typography scale — no horizontal overflow, spacing legible on small phones 2. Sections that previously rendered wide tables on phone widths now render as stacked lists (no new sections, no new data sources) + **Plans**: 2 plans + - [x] 05-01-PLAN.md — FinanceRow + FinanceSkeleton helper components (FIN-01, FIN-02) - [x] 05-02-PLAN.md — Rewrite app/mobile/finance/page.tsx to KPI grid + stacked lists + shadcn Collapsibles (FIN-01, FIN-02) + **UI hint**: yes ### Phase 6: Analyzer Feed (NEW) + **Goal**: A manager taps the Analyzer tab and skims a most-recent-first stream of AI ticket analyses, opening any one to a phone-friendly summary view that links out to desktop for full details. **Depends on**: Phase 2 **Requirements**: ANL-01, ANL-02, ANL-03, ANL-04, ANL-05, ANL-06 **Success Criteria** (what must be TRUE): + 1. Tapping the Analyzer tab in the bottom nav lands on `/mobile/analyzer` and shows a most-recent-first list of AI ticket analyses 2. Each row shows ticket #, title, the analyzer's one-line summary, a confidence badge, and a stage indicator (Triage → Analyze → Deep Review) 3. Tapping a row opens a mobile summary view rendering Summary, Next Step, and Next Step Rationale, with a "View full analysis" link out to the desktop analyzer page 4. The mobile feed never exposes editing, re-run, or prompt-tuning controls (read-only by design) 5. The list reads from `analyzer_analyses` via `/api/mobile/analyzer/feed` (or a reused list endpoint that already returns the right shape) + **Plans**: 3 plans + - [x] 06-01-PLAN.md — /api/mobile/analyzer/feed endpoint with cursor pagination + kiosk_settings scoping (ANL-01, ANL-02, ANL-06) - [x] 06-02-PLAN.md — AnalyzerFeedRow/StagePips/ConfidenceBadge/RowSkeleton components + replace /mobile/analyzer placeholder with feed list page (ANL-01, ANL-02, ANL-05, ANL-06) - [x] 06-03-PLAN.md — /mobile/analyzer/[id] detail page reading existing /api/analyzer/analyses/[id] (ANL-03, ANL-04, ANL-05) + **UI hint**: yes ### Phase 7: Engagement Overview (NEW) + **Goal**: A manager reaches Engagement from the More drawer and sees a phone-first overview — period chips, stacked summary cards, a sortable per-employee list, and one compact sparkline. **Depends on**: Phase 2 **Requirements**: ENG-01, ENG-02, ENG-03, ENG-04, ENG-05, ENG-09 **Success Criteria** (what must be TRUE): + 1. The Mobile sections row in the More drawer links to `/mobile/engagement`; the Analyzer is on the bottom bar but Engagement is not 2. The overview page shows a period selector (today / 7d / 30d) sticky just below the H1, with active period clearly indicated 3. Summary cards (active users, total Graph hours, total Autotask hours, hours-per-active-user) render single-column stacked — no 4-up grid on phone widths 4. The per-employee list renders as stacked rows (avatar/initials, name, role, hours bar) with a search input and a sort control above (sort by hours, name, utilization) 5. A single compact "hours trend" sparkline renders at the top of the list, scoped to the selected period — no multi-series chart + **Plans**: 3 plans + - [x] 07-01-PLAN.md — /api/mobile/engagement/summary + /api/mobile/engagement/trend endpoints with period whitelist + requireAuth (ENG-03, ENG-05) - [x] 07-02-PLAN.md — Engagement* mobile components (PeriodChips, SummaryCard, HoursSparkline, SortChips, SearchInput, UserRow, UserRowSkeleton + getInitials utility) (ENG-02, ENG-03, ENG-04, ENG-05) - [x] 07-03-PLAN.md — app/mobile/engagement/page.tsx orchestration (period/sort state, IntersectionObserver, empty/error/not-configured states) (ENG-01, ENG-02, ENG-03, ENG-04, ENG-05, ENG-09) + **UI hint**: yes ### Phase 7.1: User Timezone Fix (INSERTED — urgent) + **Goal**: A user opening Pulse sees dashboards, filters, and "today/this week" date math computed in their own IANA timezone — not server UTC — so reports stop showing yesterday's data as today (and vice versa). Persistence layer remains UTC; only the read/display path changes. **Depends on**: Nothing structural (Better Auth users table extension + read-path changes) **Requirements**: TZ-01, TZ-02, TZ-03, TZ-04 **Success Criteria** (what must be TRUE): + 1. Each user has an IANA timezone (e.g. `America/New_York`) persisted server-side; default = `process.env.DEFAULT_TIMEZONE || 'UTC'` for users with no value yet 2. Mobile and desktop dashboards, ticket filters, finance views, and engagement period selectors compute day/week boundaries against the viewer's timezone — not UTC and not the browser's local zone (browser zone may differ from the user's chosen zone, e.g. travel) 3. Authenticated `GET /api/me/timezone` returns the user's tz; `PUT /api/me/timezone` accepts an IANA string and rejects anything not in `Intl.supportedValuesOf('timeZone')` 4. A shared client hook (`useUserTimezone()`) reads the value from `useSession()` so all components use a single source of truth — no per-page `Intl` calls scattered around 5. Existing UTC-stored data stays untouched (no destructive migration); only formatting and range-bucketing change + **Plans**: 6 plans + - [x] 07.1-01-PLAN.md — Add timezone column to user table + Better Auth additionalField (TZ-01) - [x] 07.1-02-PLAN.md — /api/me/timezone GET + PUT with IANA validation (TZ-03) - [x] 07.1-03-PLAN.md — Server-side read paths use user.timezone for day/week/month boundaries; auth-gates /api/mobile/finance; migrates /api/dashboard/trends (TZ-02) - [x] 07.1-04-PLAN.md — useUserTimezone() client hook + reported-bug-surface mobile page migration + codebase-wide audit (TZ-04, TZ-02 client portion) - [x] 07.1-05-PLAN.md — Codebase-wide useUserTimezone() adoption per the Plan 04 audit (TZ-04 SC#4 single-source-of-truth at codebase scale) + **UI hint**: no (this is a data/plumbing phase; the picker UI is part of Phase 9) ### Phase 8: Engagement User Profile (NEW) + **Goal**: From the Engagement overview, a manager taps an employee row and arrives at a real, shareable profile page — single-column phone-first — and the device back gesture returns them to the overview. **Depends on**: Phase 7 **Requirements**: ENG-06, ENG-07, ENG-08 **Success Criteria** (what must be TRUE): + 1. Tapping a row in the per-employee list navigates to `/mobile/engagement/[userId]` (segment form, shareable URL) 2. The profile is a real page (not a modal) — the device/browser back gesture returns to the overview at the same scroll position 3. The profile renders single-column: identity header → period selector → key metrics (compact) → activity breakdown list → recent items, sourced from the existing engagement profile data endpoints (no new data) + **Plans**: 2 plans + - [x] 08-01-PLAN.md — MS Graph user-photo proxy at /api/mobile/engagement/user/[userId]/photo (ENG-06; D-25, D-26) - [x] 08-02-PLAN.md — Mobile profile page at /mobile/engagement/[userId] + 6 EngagementProfile* components (ENG-06, ENG-07, ENG-08) + **UI hint**: yes ### Phase 9: User Profile & Preferences (NEW) + **Goal**: A logged-in user reaches a profile/settings page from the More drawer and can configure timezone (chooser UI for TZ-01), theme (light/dark/system, persisted server-side for cross-device consistency), mobile push notifications (per-event toggles, delivered via the ntfy phone app per the no-SW constraint), and personal notification channels (Teams webhook URL, Pulse-minted ntfy topic). Changes persist per-user and the existing notify pipeline routes through these per-user channels for events the user is subscribed to. **Depends on**: Phase 7.1 (timezone schema), Phase 2 (More drawer) **Requirements**: PROF-01, PROF-02, PROF-03, PROF-04, TZ-CHOOSER-01, TZ-CHOOSER-02, THEME-01, THEME-02, THEME-03, THEME-04, THEME-05, CHAN-01, CHAN-02, CHAN-03, CHAN-04, CHAN-05, CHAN-06, CHAN-07, SUB-01, SUB-02, SUB-03, SUB-04, ROUTE-01, ROUTE-02, ROUTE-03, ROUTE-04, ROUTE-05, ROUTE-06, ROUTE-07 **Canonical refs:** + - `docs/superpowers/specs/2026-05-03-mobile-shell-design.md` §4, §5.1, §7 (no-SW constraint, drawer Account section, deferred items) - `lib/services/pipeline-steps/notify.ts` (existing channel-only notify; Phase 9 adds `route_to_user`) - `lib/auth.ts` `additionalFields` (Phase 7.1 precedent for `theme` column exposure) @@ -187,34 +235,44 @@ under `/mobile/*` — no `/mobile-v2`, no parallel routes. - `app/api/me/timezone/route.ts` (per-user API conventions to mirror for `/api/me/theme`, `/api/me/channels`, `/api/me/notification-subscriptions`) - `components/mobile/MoreDrawer.tsx` (Account section gains "Profile & preferences" link) - `components/theme-toggle.tsx` + `components/theme-provider.tsx` (next-themes write-through path) + **Success Criteria** (what must be TRUE): + 1. Tapping Profile/Account in the More drawer routes to `/mobile/profile` (real page, not modal); the page renders four sections in order — Timezone, Theme, Notifications, Channels — each gated by `requireAuth()` and saving per-user (PROF-01..04) 2. Theme persists server-side and applies on sign-in across devices via the `theme` column on `user` (Better Auth additionalFields), with next-themes still handling FOUC and the desktop `ThemeToggle` writing through to the server (THEME-01..05) 3. Each user can configure one Teams webhook URL and one Pulse-minted ntfy topic; both are test-sent on save and admins have full read+edit access via `/admin/workflow/channels` (CHAN-01..07) 4. The Notifications section renders a per-event × per-channel matrix sourced from `notify_event_keys`; defaults to enabled (opt-out model); writes via `/api/me/notification-subscriptions` (SUB-01..04) 5. `lib/services/pipeline-steps/notify.ts` honors an optional `route_to_user` block on each notify step — resolving the user via a registered resolver, checking the subscription matrix, sending via the personal channel, and falling back to the step's `channel_id` on no-channel/send-failure (recorded as `user_route_fallback`) but skipping silently when the user has the toggle muted (ROUTE-01..07) + **Plans**: 6 plans + - [x] 09-01-PLAN.md — Schema foundation: theme column, owner_user_id, notify_event_keys, user_event_subscriptions (THEME-01, THEME-05, CHAN-01, SUB-01, SUB-02) - [x] 09-02-PLAN.md — /api/me/* endpoints: theme, channels (Teams + ntfy), notification-subscriptions matrix (THEME-02, CHAN-02..05, CHAN-07, SUB-04) - [x] 09-03-PLAN.md — notify.ts route_to_user branch + resolver registry + fallback semantics (ROUTE-01..06) - [x] 09-04-PLAN.md — /mobile/profile UI part 1: page shell + drawer link + Timezone/Theme/Notifications Cards + Channels placeholder (PROF-01..04, TZ-CHOOSER-01..02, THEME-03, SUB-03) - [x] 09-05-PLAN.md — /mobile/profile UI part 2: real Channels Card (Teams + ntfy + QR) + ThemeSessionBridge + ThemeToggle write-through (CHAN-02..05, CHAN-07, THEME-04) - [x] 09-06-PLAN.md — Admin surfaces: channels Owner column + filter, event-keys CRUD page, NEW /admin/workflow/executions with fallback filter (CHAN-06, SUB-01, ROUTE-07) + **UI hint**: yes ### Phase 9.1: ntfy Backend Fix (INSERTED — urgent) + **Goal**: A logged-in user enabling mobile push from `/mobile/profile` gets a topic published to `https://ntfy.wulfconsulting.cloud` (not the public `ntfy.sh`) with bearer auth via `NTFY_PULSE_TOKEN`, using the `pulse-me-` reserved prefix so personal channels never collide with the `noc-*` / `soc-*` namespaces reserved for NOC/SOC operations. **Depends on**: Phase 9 (personal channels feature must exist) **Requirements**: CHAN-03, CHAN-05, CHAN-07, ROUTE-04 (gap closure — re-targeting the existing implementation) **Source**: `.planning/phases/09-user-profile-preferences-new/09-HUMAN-UAT.md` Test 1 — diagnosed gap **Success Criteria** (what must be TRUE): + 1. `mintNtfyTopic()` returns `pulse-me-XXXXXXXX`; `NTFY_TOPIC_RE` enforces `^pulse-me-[A-Za-z0-9-]{6,64}$`; custom topics matching `pulse-`, `noc-`, `soc-`, or arbitrary names are rejected 2. All four ntfy publish paths used for personal channels (`sendChannelTest`, `pipeline-steps/notify.ts sendNtfy`, `pipeline-steps/approval.ts` ntfy branch, `ticket-digest-service.ts deliver()` ntfy branch) target `${NTFY_BASE_URL || 'https://ntfy.wulfconsulting.cloud'}` and send `Authorization: Bearer ${NTFY_PULSE_TOKEN}` when `channel.owner_user_id` is set 3. Global / admin ntfy channels (`owner_user_id IS NULL`) preserve their existing `channel.config.server_url` / `channel.config.auth_token` behavior — out-of-scope per gap diagnosis 4. `/mobile/profile` QR code and subscribe link target `process.env.NEXT_PUBLIC_NTFY_BASE_URL || 'https://ntfy.wulfconsulting.cloud'`; help line under custom-topic Input reads "Topic must start with `pulse-me-`" 5. `npx tsc --noEmit --pretty` and `npx vitest run lib/services/pipeline-steps/notify.test.ts` both pass (mute semantics intact) + **Plans**: 1 plan + - [x] 09.1-01-PLAN.md — Personal-channels regex/prefix/bearer + propagate to notify/approval/digest send paths + ProfileChannelsSection QR & copy + **UI hint**: no (backend-heavy; one component edit for QR/link target) @@ -246,83 +304,108 @@ render, including the manual-resolution workflow for flagged companies. - [x] **Phase 14: /pax8 UI Surface** — New page listing companies/subscriptions/cost breakdown, plus manual resolution of flagged company matches (completed 2026-07-12) ### Phase 10: PAX8 Client & Auth Foundation + **Goal**: Pulse can authenticate to the PAX8 API via OAuth2 client-credentials, and the Postgres schema for all four PAX8 entities exists — proving the integration pattern before any sync logic is built on top of it. **Depends on**: Nothing (first phase of v2.0) **Requirements**: PAX8-01, PAX8-02 **Success Criteria** (what must be TRUE): + 1. `lib/services/pax8-factory.ts` exports `isPax8Configured()`, returning `true` only when the PAX8 client ID and secret env vars are both set, `false` otherwise 2. `getPax8Client()` performs an OAuth2 client-credentials token exchange against `api.pax8.com/v1` and successfully calls a read-only endpoint (e.g., list companies) using the resulting bearer token 3. Calling the client with missing/invalid credentials throws a clear, typed error rather than failing silently or crashing the process — matching the existing `isConfigured()` + throw-if-missing pattern used by other integrations 4. A new numbered migration creates the PAX8 tables (companies, subscriptions, products/catalog, orders, and a company-match/review table) using `IF NOT EXISTS`, ready for Phase 11+ to populate + **Plans**: 3 plans + - [x] 10-01-PLAN.md — PAX8 types + OAuth2 client (token exchange, audience, cache) + factory (isPax8Configured/getPax8Client) + mocked tests (PAX8-01, PAX8-02) - [x] 10-02-PLAN.md — migrations/091_pax8_tables.sql (6 PAX8 tables, IF NOT EXISTS) + apply to dev DB (PAX8-01, PAX8-02) - [x] 10-03-PLAN.md — verify-pax8-auth.ts live auth-proof (SC#2) + CLAUDE.md/INTEGRATIONS.md docs (PAX8-01, PAX8-02) + **UI hint**: no ### Phase 11: Company, Catalog & Subscription Sync + **Goal**: PAX8 companies, the product catalog, and current subscriptions are synced into Postgres and are human-readable (not raw SKU IDs) — the "current state" half of the integration. **Depends on**: Phase 10 **Requirements**: PAX8-03, PAX8-04, PAX8-05, PAX8-08 **Success Criteria** (what must be TRUE): + 1. Running the sync populates a companies table with every PAX8 company (PAX8 ID, name, and other identifying fields) 2. Running the sync populates a product/catalog table (SKUs, categories) and a subscriptions table (product, seat count, billing term) per company 3. A synced subscription row displays a readable product name and category by joining to the catalog table — not a bare SKU/product ID 4. No code path in the PAX8 client or this sync service issues a write (POST/PUT/PATCH/DELETE) to the PAX8 API — every call is a read, verified by inspection of the client's exposed methods + **Plans**: 3 plans + - [x] 11-01-PLAN.md — Migration 092 subscription cost columns + extend pax8 types + read-only client pagination helpers (PAX8-04, PAX8-05, PAX8-08) - [x] 11-02-PLAN.md — pax8-sync-service.ts (companies + subscriptions + referenced-only catalog + soft-delete reconciliation) + /api/pax8/sync fire-and-forget route (PAX8-03, PAX8-04, PAX8-05, PAX8-08) - [x] 11-03-PLAN.md — Read-only invariant proof + live sync run DB verification checkpoint (PAX8-03, PAX8-04, PAX8-05, PAX8-08) + **UI hint**: no ### Phase 12: Orders/Invoices & Company Matching + **Goal**: Pulse has historical PAX8 cost data for reconciliation over time, and every PAX8 company is automatically linked to its Autotask counterpart or explicitly flagged for review — never silently guessed. **Depends on**: Phase 11 **Requirements**: PAX8-06, PAX8-10, PAX8-11 **Success Criteria** (what must be TRUE): + 1. Running the sync populates an orders/invoices table with historical line items (not just current-state seat counts), enabling cost-over-time comparisons 2. At sync time, each PAX8 company is automatically matched to an Autotask company by fuzzy name similarity when a sufficiently confident match exists, and the match is persisted 3. A PAX8 company with no match, or with multiple similarly-scored Autotask candidates, is persisted with a flagged/needs-review status instead of being auto-assigned 4. Re-running the sync does not overwrite a match that has already been manually confirmed/resolved (idempotent with respect to human decisions) + **Plans**: 5 plans + - [x] 12-01-PLAN.md — Migration 093 (pg_trgm + pax8_order_items/pax8_companies columns) + Pax8Invoice/Pax8InvoiceItem types (PAX8-06, PAX8-10, PAX8-11) - [x] 12-02-PLAN.md — pax8-client listAllInvoices/listAllInvoiceItems + tests + live field-mapping spot-check (PAX8-06) - [x] 12-03-PLAN.md — pax8-company-matcher.ts (pg_trgm similarity, 0.90 threshold, tie/empty/idempotency policy) + tests (PAX8-10, PAX8-11) - [x] 12-04-PLAN.md — syncOrders + syncCompanyMatches wired into Pax8SyncService.fullSync + sync-service tests (PAX8-06, PAX8-10, PAX8-11) - [x] 12-05-PLAN.md — Live full-sync verification of all 4 success criteria + human-verify checkpoint (PAX8-06, PAX8-10, PAX8-11) + **UI hint**: no ### Phase 13: Scheduler & Admin Toggle + **Goal**: PAX8 sync runs automatically once a day like every other Pulse integration, and can be turned on or off from `/admin/integrations` without a container restart. **Depends on**: Phase 12 **Requirements**: PAX8-07, PAX8-09 **Success Criteria** (what must be TRUE): + 1. A `pax8-daily` (or equivalently named) entry exists in the sync scheduler and fires once per day, running the full companies + catalog + subscriptions + orders sync in sequence 2. PAX8 appears as a toggleable row on `/admin/integrations`, backed by the `integration_settings` table like every other integration 3. Disabling PAX8 from that UI stops future scheduled sync runs (respecting the existing health-cache window, or immediately per the PATCH-clears-cache convention) and records `disabled_by`, `disabled_at`, and an optional `disabled_reason` 4. Re-enabling PAX8 resumes scheduled sync at the next cron tick with no code deploy or container restart required + **Plans**: 3 plans + - [x] 13-01-PLAN.md — Migration 096 pax8-daily seed + dual-guarded scheduler branch + CLAUDE.md precedent note (PAX8-07, PAX8-09) - [x] 13-02-PLAN.md — checkConfigOnly('pax8') admin-integrations row + POST /api/pax8/sync 403 disabled-gate (PAX8-09) - [x] 13-03-PLAN.md — Live verification checkpoint of Phase 13 SC#1-4 (PAX8-07, PAX8-09) + **UI hint**: no ### Phase 14: /pax8 UI Surface + **Goal**: A manager can open `/pax8` and see PAX8 companies with their subscriptions and a cost breakdown, and an admin can resolve any flagged/ambiguous company match directly from that page — no psql required. **Depends on**: Phase 12 **Requirements**: PAX8-12, PAX8-13, PAX8-14 **Success Criteria** (what must be TRUE): + 1. `/pax8` lists PAX8 companies together with their current subscriptions 2. Each company shows a cost breakdown (e.g., by subscription/product) built from the synced subscription and order/invoice data 3. Flagged/ambiguous company matches appear in a distinct, clearly-labeled review section on `/pax8` rather than being mixed silently into the main list 4. From that review section, an admin can pick the correct Autotask company for a flagged PAX8 company; the resolution persists and is respected (not overwritten) by future syncs + **Plans**: 6 plans + - [x] 14-01-PLAN.md — GET /api/pax8/companies list + /api/pax8/companies/[id] cost-breakdown (requireAuth) (PAX8-13) - [x] 14-02-PLAN.md — /api/pax8/company-matches queue + admin-gated resolve route + extracted resolver service & test (PAX8-12, PAX8-14) - [x] 14-03-PLAN.md — DetailModal additive extension: kind prop + PAX8_COMPANY_GROUPS + subscriptions cost-breakdown section (PAX8-13) - [x] 14-04-PLAN.md — /pax8 page shell + Companies tab (DataTable + DetailModal drill-down) + top-level nav entry (PAX8-13) - [x] 14-05-PLAN.md — Needs Review tab (review cards, candidate + manual-search resolve, count badge) + companies-list auth hardening (PAX8-14, PAX8-12) - [x] 14-06-PLAN.md — Automated gates + human verification of all 4 SCs and the view/resolve permission split (PAX8-12, PAX8-13, PAX8-14) + **UI hint**: yes @@ -373,105 +456,140 @@ of a functional dependency on Phase 21. ## Phase Details ### Phase 15: Data Model, Detection & Ticket Evidence + **Goal**: The durable phishing-triage schema exists in Postgres, and Pulse can scan Autotask/Pulse tickets for known phishing/spam-report patterns idempotently, capturing base ticket-level evidence for each candidate. **Depends on**: Nothing (first phase of v3.0) **Requirements**: DETECT-01, DETECT-02, EVID-01 **Success Criteria** (what must be TRUE): + 1. A new migration (`migrations/097_*.sql` or next available number) creates `campaigns`, `reports`, `messages`, `indicators`, `classifications`, `remediation_actions`, and `audit_events` tables with `IF NOT EXISTS`, ready for every later phase to read/write 2. Running the ticket scanner against Autotask/Pulse tickets flags candidates matching the known title/body patterns ("Phishing Report", "Spam Alert", "Phishing Alert - Email Security Report", "KnowBe4 Phish Alert Report", "Source: KnowBe4 Phish Alert Button", "userSubmissionsReportMessage", "reported message destinations", "Microsoft directly") and persists a `reports` row per candidate 3. Re-scanning tickets that haven't changed since last processed does not reprocess or duplicate their `reports` rows; a ticket whose Autotask data changed since last processed IS reprocessed (idempotent on ticket state, not just ticket ID) 4. Each flagged ticket's stored evidence includes ticket ID/number, company, requester/reporter, title, description, notes, relevant time entries, and attachment metadata (EVID-01) + **Plans**: 3 plans + - [x] 15-01-PLAN.md — Migration 097: 7-table phishing-triage schema (reports fully designed, others stubbed) (DETECT-01, DETECT-02, EVID-01) - [x] 15-02-PLAN.md — phishing-detector.ts core: pattern matcher + content-hash idempotency + EVID-01 evidence capture + reports upsert (DETECT-01, DETECT-02, EVID-01) - [x] 15-03-PLAN.md — Wiring: webhook fire-and-forget hook + bounded cron sweep service + scheduler branch + migration 098 seed (DETECT-01, DETECT-02) + **UI hint**: no ### Phase 16: EML/MIME Evidence Parser + **Goal**: Given a ticket's attachments, Pulse selects the correct original reported message and parses its RFC822/MIME structure into normalized, actionable evidence — without ever executing or fetching anything from the message. **Depends on**: Phase 15 (messages/indicators tables to persist output into) **Requirements**: EVID-02, EVID-03, EVID-04 **Success Criteria** (what must be TRUE): + 1. Given synthetic fixtures with both `rfc.eml` and `OriginatingEmail.eml` present, the selection logic picks `rfc.eml` as the original reported message, matching case-insensitively and by `message/rfc822` content-type — not filename alone 2. Parsing a synthetic `.eml` fixture produces normalized headers (From, display name, sender email/domain, Reply-To, Return-Path, To, Cc, Subject, Date, Message-ID, Received chain, SPF/DKIM/DMARC results), a list of extracted URLs, and attachment metadata (name, content-type, size, hash) 3. The parser never executes or fetches any URL found in a message — verified by tests asserting no outbound network calls happen during parsing 4. Parsed output includes a sanitized/truncated body preview stored alongside the raw evidence, distinct from the full raw body 5. `npx vitest run` for the new parser test file passes using synthetic fixtures only (no real customer email) + **Plans**: 3 plans + - [x] 16-01-PLAN.md — Deps (mailparser + linkify-it) + pure EML parser: 3-tier selection, RFC822/MIME normalization, structured SPF/DKIM/DMARC verdicts, sanitized preview, no-network + size-guard tests (EVID-02, EVID-03, EVID-04) - [x] 16-02-PLAN.md — Supporting infra: AutotaskClient.getAttachmentContent (items[0]), b2 EML_OBJECT_KEY_REGEX + parameterized key validation, migration 099 indicators.metadata JSONB (EVID-03, EVID-04; D-05, D-07) - [x] 16-03-PLAN.md — phishing-eml-service orchestration: list→select→fetch→B2 (gated)→parse→persist messages/indicators, end-to-end no-network + graceful-degrade tests (EVID-03, EVID-04; D-05, D-06, D-07) + **UI hint**: no ### Phase 17: Mimecast Blast Radius Lookup + **Goal**: Pulse can ask "how far did this message spread" via a Mimecast blast-radius abstraction when Mimecast is configured, and gets a clean `unavailable` signal — never a crash or a block — when it isn't. **Depends on**: Phase 15 (schema to store lookup results against) **Requirements**: BLAST-01, BLAST-02 **Success Criteria** (what must be TRUE): + 1. When Mimecast is configured, querying the blast-radius abstraction for a message (keyed on message ID, sender, recipient/reporter, subject, and date window) returns normalized delivery data — matched/delivered/held/rejected/clicked counts and per-recipient status 2. When Mimecast is not configured, the same lookup call returns `status: unavailable` synchronously rather than throwing, timing out, or blocking the caller 3. The lookup follows the existing `lib/services/` factory convention (`getMimecastClient()` + `isMimecastConfigured()`-equivalent) so Phase 19's classifier can call it without knowing whether Mimecast is present + **Plans**: 1 plan + - [x] 17-01-PLAN.md — isMimecastConfigured() gate + mimecast-blast-radius.ts fan-out/merge/cache orchestration + tests (BLAST-01, BLAST-02) + **UI hint**: no ### Phase 18: Campaign Grouping & Phishing Analysis API + **Goal**: Duplicate reports of the same phishing/spam campaign are automatically grouped and accumulate over time, and an operator can trigger analysis of a specific ticket or browse campaigns through a properly access-controlled `/api/phishing/*` surface. **Depends on**: Phase 16 (parsed Message-ID/indicators to key grouping on) **Requirements**: CAMP-01, CAMP-02, CAMP-03, DETECT-03, ACCESS-01 **Success Criteria** (what must be TRUE): + 1. Two reports sharing the same original Message-ID are grouped into the same campaign; absent a shared Message-ID, reports sharing attachment-hash/URL-domain + subject + sender within a time window are grouped instead; absent that too, sender + normalized subject + client + time-window groups them as the final fallback 2. A campaign accumulates additional linked ticket reports and recipients as new duplicate reports arrive over time, without ever creating a second campaign for the same underlying report 3. `POST /api/phishing/tickets/{ticket_id}/analyze` runs detection + evidence extraction + campaign grouping for one specific ticket on demand and returns the resulting campaign linkage, instead of waiting for the next scheduled scan 4. `GET /api/phishing/campaigns` lists campaigns and `GET /api/phishing/campaigns/{id}` returns full detail (linked reports, messages, indicators, classification history) 5. Every `/api/phishing/*` route introduced in this phase calls `requireAuth()` (or `requirePermission()`) and rejects an unauthenticated/unauthorized request with 401/403 — establishing the auth convention every later phishing endpoint (Phases 19-21) must also follow + **Plans**: 3 plans (2 waves) + - [x] 18-01-PLAN.md — Campaign grouping service (tiered match + transactional find-or-create) + tests + phishing permission resource (CAMP-01, CAMP-02, ACCESS-01) - [x] 18-02-PLAN.md — POST /api/phishing/tickets/{id}/analyze + wire groupReportIntoCampaign into webhook + cron sweep automatic paths (DETECT-03, CAMP-01, CAMP-02, ACCESS-01) - [x] 18-03-PLAN.md — GET /api/phishing/campaigns list + GET /api/phishing/campaigns/{id} nested detail (CAMP-03, ACCESS-01) + **UI hint**: no ### Phase 19: Classification Engine + **Goal**: Every campaign gets a deterministic SPAM/UNWANTED/THREAT verdict, built from bounded structured evidence (never raw unbounded email), that correctly flags destructive-action recommendations for approval and doesn't cry wolf on routine KnowBe4 simulations. **Depends on**: Phase 17 (blast-radius input), Phase 18 (campaign data input + auth convention) **Requirements**: CLASSIFY-01, CLASSIFY-02, CLASSIFY-03, CLASSIFY-04, CLASSIFY-05, CLASSIFY-06 **Success Criteria** (what must be TRUE): + 1. Classifying a campaign returns exactly one of `SPAM` / `UNWANTED` / `THREAT` with confidence, a short summary, evidence-backed reasons, recommended actions, and a `requires_approval` flag 2. A classification whose recommended actions include any destructive action (purge/block/delete/reset) always has `requires_approval: true` — proven by a test asserting the invariant can't be produced any other way 3. Classifying a campaign with incomplete evidence (no Mimecast data, no `.eml`) lowers confidence and names the specific missing evidence in the reasons 4. A synthetic KnowBe4 security-awareness-simulation fixture is not classified as `THREAT` absent contrary evidence 5. `POST /api/phishing/campaigns/{id}/classify` (re-)triggers classification, enforces the Phase 18 auth convention, and the classifier only ever receives structured, size-bounded evidence — long bodies are redacted/truncated before reaching any AI layer, and IT Glue-sourced evidence (if referenced) goes through the existing redacted `lib/services/analyzer/itglue-search.ts` path + **Plans**: 2 plans (2 waves) + - [x] 19-01-PLAN.md — campaign-classifier.ts deterministic rule engine (evidence gather + D-03/D-04/D-06 rules + D-05 confidence + D-08 actions + append-only INSERT) + vitest suite + synthetic KnowBe4/BSN fixtures (CLASSIFY-01, CLASSIFY-02, CLASSIFY-03, CLASSIFY-04, CLASSIFY-06) - [x] 19-02-PLAN.md — POST /api/phishing/campaigns/[id]/classify route (requirePermission analyze + UUID guard + classifyCampaign delegation) (CLASSIFY-05) + **UI hint**: no ### Phase 20: Remediation, Approval & Audit Safety + **Goal**: Remediation actions are proposed, never auto-executed, and every approve/remediate/mark-false-positive action is gated by elevated permission, idempotent on re-run, and fully audited. **Depends on**: Phase 18 (campaigns to act against), Phase 19 (classifications to approve/act on) **Requirements**: REMED-01, REMED-02, REMED-03, REMED-04, REMED-05, REMED-06 **Success Criteria** (what must be TRUE): + 1. Recommended remediation actions are persisted with status `proposed`, and no code path in this milestone executes one automatically 2. `POST /api/phishing/campaigns/{id}/approve` records approver, timestamp, and the exact approved action parameters, and is gated behind a permission level above plain read access (beyond the Phase 18 baseline) 3. `POST /api/phishing/campaigns/{id}/remediate` proceeds only for already-approved actions against a configured, non-destructive-by-default provider path; otherwise it returns `not_implemented`/an explicit failure and never silently succeeds without taking or logging an action 4. Re-running remediation against an already-completed action does not duplicate the destructive effect — proven by a test that calls remediate twice and asserts a single effect/log entry 5. `POST /api/phishing/campaigns/{id}/mark-false-positive` exists, and every state-changing action (classify, approve, remediate, mark-false-positive) writes an `audit_events` row recording actor, event type, and payload + **Plans**: 2 plans (2 waves) + - [x] 20-01-PLAN.md — phishing-audit.ts writeAuditEvent + remediation-service.ts approve/remediate/mark-false-positive orchestrators (idempotent, audited, D-04 guard) + vitest suite (REMED-01..06) - [x] 20-02-PLAN.md — lib/permissions.ts approve/remediate grant (D-02) + approve/remediate/mark-false-positive routes + classify audit wiring (REMED-02, REMED-03, REMED-04, REMED-05, REMED-06) + **UI hint**: no ### Phase 21: Autotask Triage Note + **Goal**: Once a campaign is classified, a human-readable, sanitized internal triage note either gets posted to the Autotask ticket (if a safe write path already exists) or is returned via API for manual use — never a raw/unsanitized dump, never a silent no-op. **Depends on**: Phase 19 (classification content to summarize), Phase 20 (recommended/approved remediation state to include) **Requirements**: NOTE-01 **Success Criteria** (what must be TRUE): + 1. If Pulse has a safe existing Autotask note-writing method, triggering note generation for a classified campaign posts an internal triage note summarizing classification, evidence, blast radius, and recommended actions to the originating ticket 2. The posted (or returned) note text is sanitized — no raw secrets/tokens/full malicious URL query strings appear in it 3. If no safe note-writing path exists, the same note content is returned via the API response instead of attempting any Autotask write, and no partial/unsanitized write is ever attempted as a fallback + **Plans**: 2 plans + - [x] 21-01-PLAN.md — Pure text layer: triage-note-sanitize (URL query/secret stripping) + triage-note-format (TriageNoteEvidence + formatTriageNote) with Vitest coverage (NOTE-01) - [x] 21-02-PLAN.md — triage-note-service (evidence gather + per-ticket TicketNotes post loop + partial-failure result) + POST /api/phishing/campaigns/[id]/triage-note route (NOTE-01) + **UI hint**: no ## Progress @@ -507,23 +625,28 @@ Phases execute in numeric order. v1.0 (Phases 1-9.1) shipped 2026-07-10. v2.0 (P | 22. Approval UI (LiveLink) | v3.0 | 6/6 | Complete | 2026-07-16 | ### Phase 22: Approval UI (LiveLink) + **Goal**: A security operator opens an Autotask ticket, clicks a LiveLink button, and lands on a Pulse page scoped to that ticket showing the campaign's timeline, evidence, and classification — with approve/remediate/mark-false-positive actions right there, so no one is calling the Phase 20 APIs by hand. **Depends on**: Phase 19 (classification + recommended action to display), Phase 20 (approve/remediate/mark-false-positive APIs the page calls) **Requirements**: REVIEW-01, REVIEW-02, REVIEW-03, REVIEW-04, REVIEW-05, REVIEW-06 **Success Criteria** (what must be TRUE): + 1. A stable, ticket-ID-addressable Pulse route (e.g. `/phishing/tickets/{ticketId}`) resolves the ticket to its campaign and renders that campaign's review page — suitable as an Autotask LiveLink target (LiveLink supplies the ticket ID as dynamic content; it does not know the internal campaign UUID), using the existing Better Auth session with no separate token/query-param auth 2. The page shows the campaign's timeline — linked reports, classification history, and audit events (classify/approve/remediate/mark-false-positive) — in chronological order 3. The page shows the gathered evidence — parsed EML headers/URLs/attachments (Phase 16), sanitized body preview, and Mimecast blast-radius data (Phase 17, including an explicit `unavailable` state when Mimecast isn't configured) — never rendering a raw/unsanitized body or unredacted secrets 4. The page shows the current classification (SPAM/UNWANTED/THREAT), confidence, reasons, and recommended remediation action(s) from Phase 19 5. Approve, remediate, and mark-false-positive buttons call the Phase 20 APIs directly from the page and reflect the resulting state (e.g. a remediated campaign shows as remediated, not re-offered for approval) 6. An operator without the elevated permission REMED-02/ACCESS-01 already require sees the approve/remediate actions disabled or hidden rather than a failed request; the page never uses a relaxed or separate permission check from the underlying APIs + **Plans**: 6 plans + - [x] 22-01-PLAN.md — Pure testable logic: ticket->campaign resolver, 7-action default-params, timeline merge (REVIEW-01, REVIEW-02, REVIEW-04) - [x] 22-02-PLAN.md — Backend routes: new ticket->campaign resolver + extend campaign-detail (evidence/timeline/classification/blast radius) + list firstReportTicketId (REVIEW-01..04) - [x] 22-03-PLAN.md — Evidence display: shadcn tooltip + inert UrlList (D-09) + tabbed EvidenceCard (REVIEW-03) - [x] 22-04-PLAN.md — ClassificationCard + TimelineCard (REVIEW-02, REVIEW-04) - [x] 22-05-PLAN.md — ActionAreaCard: approve/remediate/mark-false-positive with server-identical permission gating (REVIEW-05, REVIEW-06) - [x] 22-06-PLAN.md — Review page + campaigns list page + nav entry (REVIEW-01, REVIEW-05, REVIEW-06) + **UI hint**: yes ### Phase 23: Classification Disposition + Per-Client Automation Gate @@ -534,9 +657,14 @@ Phases execute in numeric order. v1.0 (Phases 1-9.1) shipped 2026-07-10. v2.0 (P **Plans:** 5 plans (2 waves) Plans: +**Wave 1** + - [ ] 23-01-PLAN.md — USER_AWARENESS verdict + acknowledge_user action + customer-visible note writer (noteType 18) (CLASSDISP-01, CLASSDISP-02) - [ ] 23-02-PLAN.md — Review UI: USER_AWARENESS badge + acknowledge_user manual action (CLASSDISP-03) - [ ] 23-03-PLAN.md — Migration 100 phishing_automation_gate + admin GET/PATCH/DELETE API (AUTOGATE-01) + +**Wave 2** *(blocked on Wave 1 completion)* + - [ ] 23-04-PLAN.md — /admin/phishing-automation page (3-toggle company table) + admin index tile (AUTOGATE-02) - [ ] 23-05-PLAN.md — Gate reader + gated parse->classify->acknowledge webhook chain (D-04 carve-out) (AUTOGATE-03) diff --git a/.planning/STATE.md b/.planning/STATE.md index 0beb047..476232f 100644 --- a/.planning/STATE.md +++ b/.planning/STATE.md @@ -2,16 +2,16 @@ gsd_state_version: 1.0 milestone: v3.0 milestone_name: Phishing Triage Automation -status: completed +status: executing stopped_at: Phase 23 context gathered -last_updated: "2026-07-16T22:37:09.984Z" -last_activity: 2026-07-16 -- Phase 22 marked complete +last_updated: "2026-07-16T23:31:43.767Z" +last_activity: 2026-07-16 -- Phase 23 planning complete progress: total_phases: 9 completed_phases: 8 - total_plans: 24 + total_plans: 29 completed_plans: 24 - percent: 89 + percent: 83 --- # Project State @@ -27,8 +27,8 @@ See: .planning/PROJECT.md (updated 2026-07-14) Phase: 22 — COMPLETE Plan: 1 of 6 -Status: Phase 22 complete -Last activity: 2026-07-16 -- Phase 22 marked complete +Status: Ready to execute +Last activity: 2026-07-16 -- Phase 23 planning complete Progress: [░░░░░░░░░░] 0% @@ -116,6 +116,8 @@ None yet. - 2026-07-15 — Phase 18 gap closure (18-04): decision-coverage gate flagged D-02/D-03/D-04 (grouping-parameter decisions from original discuss-phase) as not literally cited in any plan's `must_haves`/`truths`. Overridden and proceeded — these decisions were already implemented in 18-01 (24h window, subject normalization, no-merge behavior) and independently confirmed correct by both 18-REVIEW.md and 18-VERIFICATION.md. Citation gap only, not an implementation gap. +- 2026-07-16 — Phase 23 planning: decision-coverage gate flagged D-01/D-02/D-03/D-04/D-05/D-06/D-08 (7 of 8 decisions) as not literally cited by `D-NN:` prefix in any plan's `must_haves`/`truths`. Overridden and proceeded — spot-checked plan 23-01's `must_haves.truths` directly and confirmed it substantively describes D-01 through D-04's content (USER_AWARENESS verdict, acknowledge_user action, noteType 18 customer-visible note, manual-path real posting) without the literal citation prefix; the plan-checker's second-pass review (after the blocker-revision cycle) independently confirmed via live source reads that all 8 decisions map to implementing tasks with no contradictions. Same citation-format gap as the Phase 18 precedent above, not an implementation gap. + ### Quick Tasks Completed | # | Description | Date | Commit | Directory |