From 09ff5b9bb83523dff8525255f16c1512688b21bf Mon Sep 17 00:00:00 2001 From: lorentz Date: Wed, 15 Jul 2026 13:13:31 -0400 Subject: [PATCH] docs(17): capture phase context --- .../17-CONTEXT.md | 181 ++++++++++++++++++ .../17-DISCUSSION-LOG.md | 84 ++++++++ 2 files changed, 265 insertions(+) create mode 100644 .planning/phases/17-mimecast-blast-radius-lookup/17-CONTEXT.md create mode 100644 .planning/phases/17-mimecast-blast-radius-lookup/17-DISCUSSION-LOG.md diff --git a/.planning/phases/17-mimecast-blast-radius-lookup/17-CONTEXT.md b/.planning/phases/17-mimecast-blast-radius-lookup/17-CONTEXT.md new file mode 100644 index 0000000..dc16d48 --- /dev/null +++ b/.planning/phases/17-mimecast-blast-radius-lookup/17-CONTEXT.md @@ -0,0 +1,181 @@ +# Phase 17: Mimecast Blast Radius Lookup - Context + +**Gathered:** 2026-07-15 +**Status:** Ready for planning + + +## Phase Boundary + +Pulse gains a Mimecast "blast radius" lookup abstraction: given a reported +message's identity (Message-ID, sender, recipient/reporter, subject, date +window), the abstraction returns normalized delivery data — matched/ +delivered/held/rejected/clicked counts and per-recipient status — when +Mimecast is configured, and a clean `status: unavailable` signal (never a +throw, timeout, or block) when it isn't. This is a pure lookup/abstraction +layer; it does not classify, remediate, or otherwise act on the data. Phase +19's classifier is the consumer and must be able to call it without knowing +whether Mimecast is present. + + + + +## Implementation Decisions + +### Query Strategy +- **D-01:** Try `getMessageInfo(messageId)` first for an exact Message-ID + match. If that misses (no match, or Message-ID unknown/altered by + intermediate relays), fan out to `searchDeliveredMessages` + + `getHeldMessages` + `getThreatEvents` keyed on sender/subject/date-window + and merge results into one normalized blast-radius shape (matched/ + delivered/held/rejected/clicked counts + per-recipient status). This is the + most complete picture available from the existing `MimecastClient` methods, + at the cost of more API calls on the fallback path. + +### Clicked Count +- **D-02:** Mimecast's currently-wrapped API surface (`lib/services/ + mimecast-client.ts`) has no URL-click-tracking method. Derive `clicked` + best-effort from `getThreatEvents()` — where Mimecast's TTP URL Protect + logs a click as a threat-event subtype. If no matching click-type event is + found, `clicked` is `0` (not `unavailable`) — this is a best-effort signal, + documented as such in code, not a guaranteed complete count. Do not add a + new dedicated click-log API method in this phase. + +### Persistence +- **D-03:** The blast-radius lookup is ephemeral — a pure function call that + returns normalized data, nothing more. No new migration, no new table/ + column in this phase. If Phase 19's classifier wants to persist a result, + it does so via its own `classifications.reasons` JSONB column (already + defined in migration 097) — that is Phase 19's concern, not Phase 17's. + +### Caching +- **D-04:** Repeat lookups for the same message are cached in Redis via the + existing `lib/services/redis-client.ts`, keyed on message identity + (Message-ID when known, else a composite of sender+subject+date-window), + with a **5-minute TTL** — matching the existing cache pattern in + `lib/services/integration-health.ts`. This protects against Mimecast API + hammering when Phase 19 re-classifies a campaign multiple times in quick + succession, while staying short enough that data doesn't go stale across a + single classification session. + +### Claude's Discretion (explicitly deferred to research + planner) +- **`isMimecastConfigured()` helper.** `lib/services/mimecast-client.ts` + currently has `getMimecastClient()` (throws if `MIMECAST_CLIENT_ID`/ + `MIMECAST_CLIENT_SECRET` are missing) but no `isConfigured()` helper + matching the project's factory convention (see CLAUDE.md). Add one, + checking the same two env vars `lib/services/integration-health.ts` + already checks via `checkConfigOnly('mimecast', ...)`. Exact location + (alongside `getMimecastClient` in `mimecast-client.ts`, vs. a new + `mimecast-factory.ts` mirroring `veeam-factory.ts`) is planner's call — + follow whichever existing precedent is cleaner given the file is already + 1000+ lines. +- **New module name/location for the blast-radius abstraction itself** + (e.g. `lib/services/mimecast-blast-radius.ts`) — orchestration logic that + composes multiple `MimecastClient` calls into one normalized shape belongs + in a new file, not bolted onto `mimecast-client.ts` directly, but exact + naming/exports are planner's call. +- **Exact normalized output TypeScript shape** (field names, per-recipient + status enum values) — derive from BLAST-01's requirement text + (matched/delivered/held/rejected/clicked + per-recipient status) plus + whatever shapes `MimecastHeldMessage`/`MimecastDeliveredMessage`/ + `MimecastThreatEvent` already provide — planner/researcher's call. +- **Date-window sizing** for the fan-out fallback query (e.g. ±48h around + report creation time) — implementation detail, not a user preference call. +- **Redis cache key exact format** — planner's call, following whatever key + format `redis-client.ts` callers already use elsewhere. + + + + +## Canonical References + +**Downstream agents MUST read these before planning or implementing.** + +### Mimecast integration (existing) +- `lib/services/mimecast-client.ts` — existing `MimecastClient` class: + `getMessageInfo`, `searchDeliveredMessages`, `getHeldMessages`, + `getThreatEvents`, `getMimecastClient()` (throws if unconfigured, no + `isMimecastConfigured()` yet). This phase builds on top of it, does not + replace it. +- `lib/services/mimecast-sync-service.ts` — existing sync service consuming + `MimecastClient` — shows established call patterns/error handling to + follow. +- `lib/services/integration-health.ts` (`checkConfigOnly('mimecast', ...)`, + around line 338) — the exact env vars (`MIMECAST_CLIENT_ID`, + `MIMECAST_CLIENT_SECRET`) an `isMimecastConfigured()` helper must check, + and the existing 5-minute health-cache pattern this phase's Redis caching + should mirror. +- `docs/mimecast-api-guide.md` — Mimecast API reference already in the repo. + +### Factory pattern precedent +- `lib/services/veeam-factory.ts` — canonical `isConfigured()` + + `getClient()` singleton factory shape to follow if a new + `mimecast-factory.ts` is created. + +### Schema (read-only for this phase) +- `migrations/097_phishing_triage_schema.sql` — `classifications` table + (`reasons` JSONB) is where Phase 19 would persist a blast-radius result if + it chooses to; Phase 17 does not touch this schema. + +### Prior phase decisions (for consistency) +- `.planning/phases/16-eml-mime-evidence-parser/16-CONTEXT.md` (D-05, D-06, + D-07) — shows the established pattern of reusing existing service + infrastructure (B2 client, migration stubs) rather than inventing new + infrastructure; this phase follows the same instinct by reusing + `MimecastClient` + `redis-client.ts` rather than building fresh. + + + + +## Existing Code Insights + +### Reusable Assets +- `MimecastClient.getMessageInfo()`, `.searchDeliveredMessages()`, + `.getHeldMessages()`, `.getThreatEvents()` — the four existing methods this + phase's abstraction composes; no new Mimecast API surface needed beyond + possibly deriving clicks from threat events. +- `lib/services/redis-client.ts` — existing Redis client for the caching + decision (D-04). + +### Established Patterns +- Factory convention: `getClient()` + `isConfigured()`, credentials + from env vars, throws if missing (see `veeam-factory.ts`). `mimecast-client.ts` + only has half of this today. +- `checkConfigOnly()` pattern in `integration-health.ts` for config-only + health checks with a short TTL cache — the caching precedent this phase's + Redis TTL should match. + +### Integration Points +- Phase 19's classifier (not yet built) is the sole consumer of this + abstraction — it must be callable without the caller knowing whether + Mimecast is configured. + + + + +## Specific Ideas + +No specific UI/output-format requirements beyond what's captured in the +Decisions section above — this is a backend abstraction with no UI surface. + + + + +## Deferred Ideas + +- **Dedicated Mimecast click-log API method** — deferred per D-02. If + best-effort click derivation from threat events proves insufficient later, + a real `getUrlClickLogs()`-style method against Mimecast's TTP URL Protect + logs endpoint would be a future, separate addition (not blocking this + phase or Phase 19). +- **Persisting blast-radius results to a new table/column** — deferred per + D-03; left to Phase 19 (or a later phase) if audit/idempotency needs + surface once the classifier is built. + +None — discussion stayed within phase scope otherwise. + + + +--- + +*Phase: 17-mimecast-blast-radius-lookup* +*Context gathered: 2026-07-15* diff --git a/.planning/phases/17-mimecast-blast-radius-lookup/17-DISCUSSION-LOG.md b/.planning/phases/17-mimecast-blast-radius-lookup/17-DISCUSSION-LOG.md new file mode 100644 index 0000000..07fcd31 --- /dev/null +++ b/.planning/phases/17-mimecast-blast-radius-lookup/17-DISCUSSION-LOG.md @@ -0,0 +1,84 @@ +# Phase 17: Mimecast Blast Radius Lookup - Discussion Log + +> **Audit trail only.** Do not use as input to planning, research, or execution agents. +> Decisions are captured in CONTEXT.md — this log preserves the alternatives considered. + +**Date:** 2026-07-15 +**Phase:** 17-mimecast-blast-radius-lookup +**Areas discussed:** Query strategy, Clicked count, Persistence, Caching + +--- + +## Query Strategy + +| Option | Description | Selected | +|--------|-------------|----------| +| Try Message-ID first, then fan out | Call `getMessageInfo(messageId)` first for an exact match; if that misses, fan out to `searchDeliveredMessages` + `getHeldMessages` + `getThreatEvents` keyed on sender/subject/date-window and merge into one normalized count. | ✓ | +| Message-ID only, fail to unavailable | Only call `getMessageInfo` by Message-ID; return unavailable if no exact match. | | +| Always fan out (no Message-ID short-circuit) | Skip `getMessageInfo` entirely; always query the three fallback methods. | | + +**User's choice:** Try Message-ID first, then fan out (recommended option). +**Notes:** None. + +--- + +## Clicked Count + +| Option | Description | Selected | +|--------|-------------|----------| +| Best-effort from threat events | Derive `clicked` from `getThreatEvents()` where Mimecast's TTP URL Protect logs a click as a threat-event subtype; 0 (not unavailable) if no match. | ✓ | +| Always 0 / not implemented | Hardcode `clicked: 0` with a comment noting future work. | | +| Add a new MimecastClient method for click logs | Research and add a dedicated `getUrlClickLogs()`-style method now. | | + +**User's choice:** Best-effort from threat events (recommended option). +**Notes:** None. + +--- + +## Persistence + +| Option | Description | Selected | +|--------|-------------|----------| +| Ephemeral only | Pure function call, no new table/column; Phase 19's classifier decides whether to persist via `classifications.reasons`. | ✓ | +| Persist to a new column/table now | Add a migration now (e.g. `reports.blast_radius` JSONB or a dedicated table). | | + +**User's choice:** Ephemeral only (recommended option). +**Notes:** None. + +--- + +## Caching + +| Option | Description | Selected | +|--------|-------------|----------| +| No caching in this phase | Keep Phase 17 scoped to the lookup abstraction; add caching later if needed. | | +| Redis-cache by message identity | Use `lib/services/redis-client.ts` to cache blast-radius results keyed on message identity, short TTL. | ✓ | + +**User's choice:** Redis-cache by message identity. +**Notes:** Follow-up question asked for TTL specifics. + +### Cache TTL (follow-up) + +| Option | Description | Selected | +|--------|-------------|----------| +| 5 minutes | Matches the existing `integration-health.ts` cache pattern. | ✓ | +| 1 hour | Longer TTL, fewer API calls for legitimately-repeated lookups days later. | | +| No expiry — manual invalidation only | Cache indefinitely, needs explicit invalidation path. | | + +**User's choice:** 5 minutes (recommended option). +**Notes:** None. + +--- + +## Claude's Discretion + +- `isMimecastConfigured()` helper location (in `mimecast-client.ts` vs. a new `mimecast-factory.ts`) +- New module name/location for the blast-radius abstraction itself +- Exact normalized output TypeScript shape (field names, per-recipient status enum values) +- Date-window sizing for the fan-out fallback query +- Redis cache key exact format + +## Deferred Ideas + +- Dedicated Mimecast click-log API method (`getUrlClickLogs()`-style) — deferred, future work if best-effort click derivation proves insufficient +- Persisting blast-radius results to a new table/column — deferred to Phase 19 or later if audit/idempotency needs surface