diff --git a/.planning/phases/17-mimecast-blast-radius-lookup/17-RESEARCH.md b/.planning/phases/17-mimecast-blast-radius-lookup/17-RESEARCH.md new file mode 100644 index 0000000..0dcb585 --- /dev/null +++ b/.planning/phases/17-mimecast-blast-radius-lookup/17-RESEARCH.md @@ -0,0 +1,680 @@ +# Phase 17: Mimecast Blast Radius Lookup - Research + +**Researched:** 2026-07-15 +**Domain:** Internal service-layer abstraction over an existing Mimecast API client (TypeScript, Next.js `lib/services/`) +**Confidence:** HIGH (all core findings verified by reading the actual source files in this repo; one area — Mimecast's live click-event field values — is LOW/MEDIUM, see Open Questions) + + +## User Constraints (from CONTEXT.md) + +### Locked Decisions + +**D-01 (Query Strategy):** Try `getMessageInfo(messageId)` first for an exact Message-ID +match. If that misses (no match, or Message-ID unknown/altered by intermediate relays), +fan out to `searchDeliveredMessages` + `getHeldMessages` + `getThreatEvents` keyed on +sender/subject/date-window and merge results into one normalized blast-radius shape +(matched/delivered/held/rejected/clicked counts + per-recipient status). This is the most +complete picture available from the existing `MimecastClient` methods, at the cost of more +API calls on the fallback path. + +**D-02 (Clicked Count):** Mimecast's currently-wrapped API surface (`lib/services/mimecast-client.ts`) +has no URL-click-tracking method. Derive `clicked` best-effort from `getThreatEvents()` — +where Mimecast's TTP URL Protect logs a click as a threat-event subtype. If no matching +click-type event is found, `clicked` is `0` (not `unavailable`) — this is a best-effort +signal, documented as such in code, not a guaranteed complete count. Do not add a new +dedicated click-log API method in this phase. + +**D-03 (Persistence):** The blast-radius lookup is ephemeral — a pure function call that +returns normalized data, nothing more. No new migration, no new table/column in this phase. +If Phase 19's classifier wants to persist a result, it does so via its own +`classifications.reasons` JSONB column (already defined in migration 097) — that is Phase +19's concern, not Phase 17's. + +**D-04 (Caching):** Repeat lookups for the same message are cached in Redis via the existing +`lib/services/redis-client.ts`, keyed on message identity (Message-ID when known, else a +composite of sender+subject+date-window), with a **5-minute TTL** — matching the existing +cache pattern in `lib/services/integration-health.ts`. This protects against Mimecast API +hammering when Phase 19 re-classifies a campaign multiple times in quick succession, while +staying short enough that data doesn't go stale across a single classification session. + +### Claude's Discretion + +- **`isMimecastConfigured()` helper.** `lib/services/mimecast-client.ts` currently has + `getMimecastClient()` (throws if `MIMECAST_CLIENT_ID`/`MIMECAST_CLIENT_SECRET` are missing) + but no `isConfigured()` helper matching the project's factory convention. Add one, + checking the same two env vars `lib/services/integration-health.ts` already checks via + `checkConfigOnly('mimecast', ...)`. Exact location (alongside `getMimecastClient` in + `mimecast-client.ts`, vs. a new `mimecast-factory.ts` mirroring `veeam-factory.ts`) is + planner's call — follow whichever existing precedent is cleaner given the file is already + 1000+ lines. *(Research note: the file is actually ~676 lines as of this research — see + Architecture Patterns below for a concrete recommendation.)* +- **New module name/location for the blast-radius abstraction itself** (e.g. + `lib/services/mimecast-blast-radius.ts`) — orchestration logic that composes multiple + `MimecastClient` calls into one normalized shape belongs in a new file, not bolted onto + `mimecast-client.ts` directly, but exact naming/exports are planner's call. +- **Exact normalized output TypeScript shape** (field names, per-recipient status enum + values) — derive from BLAST-01's requirement text (matched/delivered/held/rejected/clicked + + per-recipient status) plus whatever shapes `MimecastHeldMessage`/`MimecastDeliveredMessage`/ + `MimecastThreatEvent` already provide — planner/researcher's call. +- **Date-window sizing** for the fan-out fallback query (e.g. ±48h around report creation + time) — implementation detail, not a user preference call. +- **Redis cache key exact format** — planner's call, following whatever key format + `redis-client.ts` callers already use elsewhere. + +### Deferred Ideas (OUT OF SCOPE) + +- **Dedicated Mimecast click-log API method** — deferred per D-02. If best-effort click + derivation from threat events proves insufficient later, a real `getUrlClickLogs()`-style + method against Mimecast's TTP URL Protect logs endpoint would be a future, separate + addition (not blocking this phase or Phase 19). +- **Persisting blast-radius results to a new table/column** — deferred per D-03; left to + Phase 19 (or a later phase) if audit/idempotency needs surface once the classifier is built. + + + +## Phase Requirements + +| ID | Description | Research Support | +|----|-------------|------------------| +| BLAST-01 | Query a Mimecast blast-radius abstraction for message delivery data (matched/delivered/held/rejected/clicked counts, per-recipient status) when Mimecast is configured, keyed on message ID, sender, recipient/reporter, subject, date window | See Architecture Patterns (exact fan-out call shapes with real field names) and Common Pitfalls #1 (getMessageInfo does NOT provide status/counts — fan-out is mandatory for this requirement, not just a fallback) | +| BLAST-02 | When Mimecast is not configured, record `status: unavailable` — never block on missing config | See Code Examples (`isMimecastConfigured()` + wrapping pattern) and existing failure-mode evidence in `app/api/mimecast/status/route.ts` | + + +## Summary + +This phase builds a pure orchestration layer on top of an already-complete, already-deployed +`MimecastClient` (`lib/services/mimecast-client.ts`, 676 lines). No new external package, no +new Mimecast API surface (beyond composing 4 existing methods), and no new database schema +are required. The work is: (1) add a missing `isMimecastConfigured()` helper matching the +project's `getClient()` + `isConfigured()` convention, (2) write a new +`lib/services/mimecast-blast-radius.ts` module that calls `getMessageInfo`, +`searchDeliveredMessages`, `getHeldMessages`, and `getThreatEvents`, merges their outputs into +one normalized shape, and wraps the whole thing in a Redis-cached, never-throwing function, +and (3) write tests using the same `vi.mock()` factory-mocking discipline established in +`pax8-factory.test.ts` and `phishing-eml-service.test.ts`. + +Two findings materially affect implementation and are surfaced prominently below: + +1. **`getMessageInfo()` cannot satisfy BLAST-01's data requirement on its own.** It returns + only `{ messageId, bodyText, bodyHtml, headers }` — no status, no delivery/held/rejected + classification, no per-recipient data at all. D-01's "try exact match first, fan out only + on miss" framing needs a small correction: the fan-out (`searchDeliveredMessages` + + `getHeldMessages` + `getThreatEvents`) is the *only* source of the matched/delivered/held/ + rejected/clicked counts and per-recipient status this phase must return — it should run + unconditionally, not merely as a fallback. `getMessageInfo` remains useful as a + supplementary existence/body check, not as a data source for the counts. +2. **There is an existing per-company multi-tenant Mimecast setup** (`mimecast_tenants` table, + migration 062, consumed via `getMimecastClientForTenant()` in the existing `/api/mimecast/held` + and `/api/mimecast/delivered` routes) that CONTEXT.md's decisions do not address. The bulk + sync service (`mimecast-sync-service.ts`) and the health check both use only the single + env-var-configured client (`getMimecastClient()`), but the two routes that most resemble + this phase's "look up one message" use case use per-tenant clients keyed to `company_id`. + This phase's success criteria and locked decisions only mention the single env-var client — + see Open Questions for the recommendation and the risk if this is wrong. + +**Primary recommendation:** Build `lib/services/mimecast-blast-radius.ts` around the single +env-var-configured `getMimecastClient()` (matching `isMimecastConfigured()`'s env-var check +and the phase's stated scope), always run the fan-out for counts, use `getMessageInfo` only +as a supplementary body/header fetch, cache via `getCachedData`/`setCachedData` from +`redis-client.ts` (already supports arbitrary TTL, default 300s), and flag the multi-tenant +question to the user/planner before implementation rather than silently picking a tenant +resolution strategy. + +## Architectural Responsibility Map + +| Capability | Primary Tier | Secondary Tier | Rationale | +|------------|-------------|----------------|-----------| +| Mimecast OAuth2 token acquisition/caching | API/Backend (`MimecastClient` internal) | — | Already implemented in `mimecast-client.ts`; this phase does not touch it | +| Message delivery data fan-out + merge | API/Backend (`lib/services/mimecast-blast-radius.ts`, new) | — | Pure orchestration function; no HTTP route in this phase | +| Configuration gating (`isMimecastConfigured`) | API/Backend (`lib/services/mimecast-client.ts` or new `mimecast-factory.ts`) | — | Mirrors existing `isConfigured()` factory convention | +| Result caching | API/Backend (Redis via `lib/services/redis-client.ts`) | — | Existing singleton client with graceful no-Redis fallback (`getRedisClient()` returns `null`, callers already no-op safely) | +| Consumption by classifier | API/Backend (Phase 19, not built yet) | — | Out of scope for this phase; this phase only needs to expose a stable, typed, non-throwing function signature | + +## Standard Stack + +### Core + +No new libraries. This phase is 100% composition of existing internal modules. + +| Module | Status | Purpose | Why reuse | +|--------|--------|---------|-----------| +| `lib/services/mimecast-client.ts` (`MimecastClient`, `getMimecastClient()`) | Existing, unmodified except adding `isMimecastConfigured()` | OAuth2 client + 4 methods this phase composes | Already deployed, already used by sync service + 7 existing API routes | +| `lib/services/redis-client.ts` (`getCachedData`, `setCachedData`) | Existing, unmodified | Generic get/set-with-TTL cache, already defaults to 300s (5 min) | Exact match for D-04's caching requirement; no raw `ioredis` calls needed | +| `ioredis` 5.9.0 `[VERIFIED: npm registry]` | Existing dependency (see `package.json`, already installed) | Underlying Redis client used by `redis-client.ts` | Already in `package.json`; not a new install | + +### Supporting + +None required. + +### Alternatives Considered + +| Instead of | Could Use | Tradeoff | +|------------|-----------|----------| +| Reusing `redis-client.ts`'s `getCachedData`/`setCachedData` | Writing bespoke `ioredis` get/set/expire calls in the new module | No advantage — `redis-client.ts` already handles the "Redis not configured" no-op case (`getRedisClient()` returns `null` when `REDIS_URL` unset) exactly the way this phase needs for graceful caching-optional behavior. Hand-rolling would duplicate that null-check logic. | +| A new `mimecast-blast-radius.ts` orchestration file | Adding the fan-out logic directly inside `mimecast-client.ts` | CONTEXT.md already decided against this (new file, not bolted on) — noted here only for completeness, not re-litigated. | + +**Installation:** None. No `npm install` needed for this phase. + +## Package Legitimacy Audit + +Not applicable — this phase introduces zero new external packages. All work reuses +`mimecast-client.ts`, `redis-client.ts`, and standard TypeScript. Skip the slopcheck/registry +verification gate for this phase; there is nothing to audit. + +## Architecture Patterns + +### System Architecture Diagram + +``` +Phase 19 classifier (future) + │ + │ getBlastRadius({ messageId?, sender, recipient, subject, dateWindow }) + ▼ +┌───────────────────────────────────────────────────────────────┐ +│ lib/services/mimecast-blast-radius.ts (NEW — this phase) │ +│ │ +│ 1. isMimecastConfigured()? ──No──► return { status:'unavailable' } (sync, no throw) +│ │ Yes │ +│ 2. Build cache key (Message-ID, else sender+subject+window) │ +│ 3. getCachedData(key) hit? ──Yes──► return cached result │ +│ │ No │ +│ 4. (optional) getMessageInfo(messageId) — body/headers only, │ +│ NOT status/counts │ +│ 5. Fan out (always, for counts): │ +│ ├─ searchDeliveredMessages({ to, from, subject, start, end }) │ +│ ├─ getHeldMessages({ recipient }) │ +│ └─ getThreatEvents() (then filter by sender/subject/window) │ +│ 6. Merge → normalized shape: matched/delivered/held/rejected/ │ +│ clicked counts + perRecipient[] status array │ +│ 7. setCachedData(key, result, 300) │ +│ 8. return result │ +└───────────────────────────────────────────────────────────────┘ + │ │ + ▼ ▼ +lib/services/mimecast-client.ts lib/services/redis-client.ts +(existing, OAuth2 + 4 methods) (existing, getCachedData/setCachedData) + │ + ▼ +https://api.services.mimecast.com (external, only if configured) +``` + +### Recommended Project Structure + +``` +lib/services/ +├── mimecast-client.ts # EXISTING — unmodified except: add isMimecastConfigured() +├── mimecast-client.test.ts # NEW — tests isMimecastConfigured() (no test file exists today) +├── mimecast-blast-radius.ts # NEW — the orchestration abstraction (this phase's deliverable) +└── mimecast-blast-radius.test.ts # NEW — orchestration tests, mocking mimecast-client +``` + +### Pattern 1: `isMimecastConfigured()` placement + +**What:** A small env-var check function matching `isVeeamConfigured()` / `isPax8Configured()`. +**Where:** Add directly inside `mimecast-client.ts`, next to `getMimecastClient()` — do NOT +create a separate `mimecast-factory.ts`. Rationale: unlike Veeam (`veeam-client.ts` + +`veeam-factory.ts` are already two separate files), `mimecast-client.ts` already contains +*both* the class and its factory function (`getMimecastClient()` is defined at the bottom of +`mimecast-client.ts`, not in a separate file). Splitting just `isMimecastConfigured()` into a +new file while `getMimecastClient()` stays put would fragment one concern across two files +for no benefit. The file is ~676 lines today (not 1000+ as CONTEXT.md's discretion note +estimated) — well within normal size for this codebase's service files. + +```typescript +// Source: existing pattern in lib/services/veeam-factory.ts, adapted for mimecast-client.ts +// Add near the bottom of mimecast-client.ts, above or below getMimecastClient(): + +export function isMimecastConfigured(): boolean { + return !!(process.env.MIMECAST_CLIENT_ID && process.env.MIMECAST_CLIENT_SECRET); +} +``` + +This matches exactly what `checkConfigOnly('mimecast', 'Mimecast', 'mail', ['MIMECAST_CLIENT_ID', 'MIMECAST_CLIENT_SECRET'])` +in `lib/services/integration-health.ts` (line 338-339) already checks — confirmed by reading +that file directly. + +### Pattern 2: Never-throw wrapper (BLAST-02) + +**What:** The public entry point must never throw, timeout unbounded, or block — it must +return `{ status: 'unavailable' }` synchronously when not configured, and should catch any +unexpected error from the fan-out calls and degrade to a partial/unavailable result rather +than propagating. + +**Existing precedent for the "unconfigured throws" failure mode this phase wraps:** +```typescript +// Source: app/api/mimecast/status/route.ts (existing, unmodified by this phase) +try { + const client = getMimecastClient(); // throws if MIMECAST_CLIENT_ID/SECRET unset + const connection = await client.testConnection(); + return NextResponse.json({ configured: true, connected: connection.ok, ... }); +} catch (err: any) { + return NextResponse.json({ configured: false, connected: false, error: err.message, ... }); +} +``` +This confirms the exact throw message this phase's `isMimecastConfigured()` check must +pre-empt: `'MIMECAST_CLIENT_ID and MIMECAST_CLIENT_SECRET must be set'` (from +`getMimecastClient()`, line ~652 of `mimecast-client.ts`). + +**Recommended shape for the new module:** +```typescript +// lib/services/mimecast-blast-radius.ts (illustrative — planner finalizes exact shape) +export interface BlastRadiusInput { + messageId?: string; + sender: string; + recipient: string; // reporter/reported-to address + subject: string; + dateWindow: { start: Date; end: Date }; +} + +export type BlastRadiusResult = + | { status: 'unavailable'; reason: 'not_configured' | 'lookup_failed'; error?: string } + | { + status: 'ok'; + matched: number; + delivered: number; + held: number; + rejected: number; + clicked: number; + perRecipient: Array<{ recipient: string; status: 'delivered' | 'held' | 'rejected' | 'unknown' }>; + source: 'message-info' | 'fan-out'; // which path produced the result + }; + +export async function getBlastRadius(input: BlastRadiusInput): Promise { + if (!isMimecastConfigured()) { + return { status: 'unavailable', reason: 'not_configured' }; + } + try { + // cache check, fan-out, merge — see diagram above + } catch (err: any) { + // BLAST-02 requires this to degrade, not throw, even on unexpected API failure + return { status: 'unavailable', reason: 'lookup_failed', error: err.message }; + } +} +``` + +### Pattern 3: Real field names for the fan-out merge (BLAST-01) + +These are the **actual** exported types from `mimecast-client.ts` — verified by reading the +file directly, not training-data recall: + +```typescript +// searchDeliveredMessages() returns: +export interface MimecastDeliveredMessage { + id: string; status: string; subject: string; from: string; fromEnv: string; + to: string; toDisplay: string; received: string; senderIP: string; + spamScore: number; detectionLevel: string; attachments: boolean; route: string; info: string; +} +// Call shape: +async searchDeliveredMessages(options: { + to?: string; from?: string; subject?: string; + startHours?: number; start?: string; end?: string; route?: string; +}): Promise<{ messages: MimecastDeliveredMessage[]; error?: string }> + +// getHeldMessages() returns: +export interface MimecastHeldMessage { + id: string; subject: string; from: string; fromDisplay: string; to: string; + toDisplay: string; dateReceived: string; reason: string; reasonCode: string; + policyInfo: string; route: string; hasAttachments: boolean; size: number; +} +// Call shape (paginates internally, caps at maxMessages, default 100): +async getHeldMessages(options: { recipient?: string; maxMessages?: number }) + : Promise<{ messages: MimecastHeldMessage[]; totalCount: number }> + +// getThreatEvents() returns: +export interface MimecastThreatEvent { + id: string; messageId?: string; eventType: string; threatLevel?: string; + url?: string; fileName?: string; verdict?: string; actorEmail?: string; + eventDateTime?: string; analysis: string[]; source: string[]; direction: string[]; status: string[]; +} +// Call shape (no sender/subject filter param — this method takes only pagination options; +// filtering by sender/subject/date-window must happen client-side after fetching): +async getThreatEvents(options?: { cursor?: string; pageSize?: number }) + : Promise<{ items: MimecastThreatEvent[]; nextCursor: string | null }> +// eventType is derived from analysis[0] (e.g. 'malware' | 'phishing' | 'spam' | 'unknown'); +// threatLevel is a derived heuristic ('high' for malware/phishing, 'medium' for spam, else 'info') +// — NOT a raw Mimecast field. See Common Pitfalls #2 re: click-event derivation. + +// getMessageInfo() returns (body/headers ONLY — no status, no counts, no recipients): +export interface MimecastMessageInfo { + messageId: string; bodyText?: string; bodyHtml?: string; headers?: Record; +} +async getMessageInfo(messageId: string): Promise +``` + +**Per-recipient merge strategy:** Both `MimecastDeliveredMessage.to` and `MimecastHeldMessage.to` +are single strings (one recipient per row), consistent with Mimecast's message-tracking model +of one log entry per recipient-message pair. Build `perRecipient[]` by grouping all rows +returned across `searchDeliveredMessages` + `getHeldMessages` (keyed by `to`), classifying +each recipient as `delivered` (from the delivered-search result set), `held` (from the held +result set), or `rejected` (delivered-search rows whose `status` field indicates rejection — +exact string values not yet confirmed against a live tenant, see Open Questions). Recipients +appearing in neither result set are `unknown`, not silently dropped. + +### Anti-Patterns to Avoid + +- **Treating `getMessageInfo()`'s non-null return as "delivered: true".** Its data comes from + `data.data?.[0]?.deliveredMessage` internally, which *suggests* a bias toward delivered + messages, but the method's only documented behavior is "returns body/headers or null" — + it has no explicit status field. Do not infer delivery status from whether this call + succeeds; use it only for body/header evidence. +- **Skipping the fan-out when `getMessageInfo` hits.** As covered in Summary/Pitfall #1, this + would silently produce a normalized result with `matched`/`delivered`/`held`/`rejected` + hardcoded to some default rather than real counts. BLAST-01 requires the fan-out regardless. +- **Calling `getThreatEvents()` with sender/subject filters.** The method's actual signature + only accepts `{ cursor?, pageSize? }` — no server-side filter params exist for this endpoint + in the current client. Fetch and filter client-side by `analysis`/`source`/`eventDateTime` + matching the input's date window and sender. +- **Writing new raw `ioredis` cache code.** `redis-client.ts`'s `getCachedData`/`setCachedData` + already do this with graceful no-Redis fallback; re-implementing it duplicates logic and + risks missing the null-check-when-`REDIS_URL`-unset behavior. + +## Don't Hand-Roll + +| Problem | Don't Build | Use Instead | Why | +|---------|-------------|-------------|-----| +| Redis get/set-with-TTL | Custom `ioredis` wrapper in the new module | `getCachedData()` / `setCachedData()` from `lib/services/redis-client.ts` | Already handles `REDIS_URL` unset (returns `null`/no-ops), already has retry/backoff config, default TTL is already 300s matching D-04 | +| OAuth2 token management for Mimecast | New token-fetch/cache logic | `MimecastClient`'s internal `getToken()` (private, already handles 60s-early refresh) | Already correct and tested in production via the existing sync service + 7 API routes | +| Config-presence checking | Ad-hoc `process.env` checks scattered in the new module | One `isMimecastConfigured()` function, called once at the top of the public entry point | Matches project-wide `isConfigured()` convention; single source of truth | + +**Key insight:** Nearly everything this phase needs already exists somewhere in the codebase. +The actual new code surface is small: one boolean-returning function, one orchestration +function with a merge algorithm, and tests. Resist the urge to add configuration options, +pagination controls, or additional Mimecast endpoints beyond what BLAST-01/02 require — +Phase 19 is the only consumer and its needs are fully described by the phase's success criteria. + +## Common Pitfalls + +### Pitfall 1: `getMessageInfo()` does not provide delivery status +**What goes wrong:** Implementer treats a non-null `getMessageInfo()` result as satisfying +BLAST-01's required counts, skips the fan-out, and returns a normalized result with fabricated +or all-zero delivered/held/rejected/clicked counts. +**Why it happens:** D-01's phrasing ("try exact match first ... if that misses, fan out") +reads naturally as an either/or, and `getMessageInfo`'s name and internal `deliveredMessage` +key structure both suggest it carries delivery information. +**How to avoid:** Read `MimecastMessageInfo`'s actual shape (verified above: `messageId`, +`bodyText`, `bodyHtml`, `headers` only) before writing the merge logic. Treat `getMessageInfo` +as a supplementary body/header fetch only; always run the fan-out to build the counts. +**Warning signs:** Tests pass with `getMessageInfo` mocked to return non-null but the +normalized result has no way to distinguish delivered vs. held vs. rejected recipients. + +### Pitfall 2: Assuming `getThreatEvents()` reliably carries click events +**What goes wrong:** D-02's best-effort click derivation is implemented assuming +`getThreatEvents()`'s `analysis`/`eventType` field set includes a click-type value, and tests +are written against a fabricated fixture with `analysis: ['click']` that has never been +validated against how Mimecast's `/threats/v1/events` endpoint actually categorizes events. +**Why it happens:** The endpoint's name ("threat events") and this client's existing +`normalizeThreatEvent()` heuristic (`analysis.includes('malware') ? 'high' : ...`) only handle +`malware`/`phishing`/`spam` — there is no existing code path or fixture proving a click +subtype appears here at all. +**Why it matters (research finding):** Per Mimecast's public API documentation (verified via +WebFetch against `integrations.mimecast.com`), URL-click data (`userEmailAddress`, `url`, +`category`, `action`, `date` of the click) is documented as living in a **separate** endpoint — +`POST /api/ttp/url/get-logs` ("Get TTP URL Logs") — not in `/threats/v1/events`, which this +client's `getThreatEvents()` wraps. This means D-02's best-effort approach may realistically +return `clicked: 0` for the vast majority of lookups, regardless of whether a click actually +occurred, because the wrapped endpoint isn't the one Mimecast uses to track clicks. This does +not block the phase (D-02 explicitly accepts `0` as a valid, documented best-effort value, +and explicitly defers building a dedicated click-log method) — but it should be documented +prominently in code comments so a future reader doesn't mistake `clicked: 0` for "confirmed no +one clicked." +**How to avoid:** In the merge function, add a code comment stating this limitation. Do not +silently treat `getThreatEvents()`'s absence of a click-type entry as proof of zero clicks in +any UI or classifier-facing text — CLASSIFY-03 (lowering confidence for incomplete evidence) +in the broader roadmap may need this signal. +**Confidence:** MEDIUM — based on official Mimecast documentation (integrations.mimecast.com), +not the actual raw response of this account's `/threats/v1/events` calls (no live tenant +access during this research pass). + +### Pitfall 3: Multi-tenant Mimecast credentials are not addressed by this phase's scope +**What goes wrong:** The new `getBlastRadius()` function is built using only the single +env-var-configured `getMimecastClient()`, but this MSP's phishing reports come from many +different client companies, some of which may have their own Mimecast tenant registered in +the `mimecast_tenants` table (migration 062) rather than being covered by the global +credentials. If Wulf Consulting's own tenant is the only one configured via env vars, lookups +for reports belonging to companies with their own `mimecast_tenants` row will silently miss +data that the existing `/api/mimecast/held` and `/api/mimecast/delivered` routes (which use +`getMimecastClientForTenant()`, keyed to `company_id`) would have found. +**Why it happens:** CONTEXT.md's locked decisions and success criteria only reference the +single env-var client and `checkConfigOnly('mimecast', ...)`'s health check — the multi-tenant +table isn't mentioned anywhere in the phase's discussion. +**How to avoid:** Flag this to the user/planner explicitly before implementation (see Open +Questions) rather than silently choosing single-tenant or silently adding tenant resolution +that wasn't discussed. `reports.company_id` (migration 097) is available if per-company +resolution turns out to be needed. +**Warning signs:** Manual QA against a report from a company that has a `mimecast_tenants` row +but isn't the account behind the global `MIMECAST_CLIENT_ID` returns `unavailable` or all-zero +counts even though Mimecast *is* configured for that company. + +## Code Examples + +### Test mocking discipline to follow (verified from this repo's most recent phase) + +```typescript +// Source: lib/services/pax8-factory.test.ts (existing, verified by reading the file) +import { describe, it, expect, beforeEach } from 'vitest'; +import { isPax8Configured, getPax8Client, _resetPax8Client } from './pax8-factory'; + +beforeEach(() => { + delete process.env.PAX8_CLIENT_ID; + delete process.env.PAX8_CLIENT_SECRET; + _resetPax8Client(); +}); + +describe('isPax8Configured', () => { + it('returns false when neither env var is set', () => { + expect(isPax8Configured()).toBe(false); + }); + it('returns true when both env vars are set', () => { + process.env.PAX8_CLIENT_ID = 'id1'; + process.env.PAX8_CLIENT_SECRET = 'secret1'; + expect(isPax8Configured()).toBe(true); + }); +}); +``` +Apply the same pattern for `isMimecastConfigured()` in a new `lib/services/mimecast-client.test.ts` +(no such file exists yet — this phase creates it). + +```typescript +// Source: lib/services/phishing-eml-service.test.ts (existing, verified by reading the file) +// Pattern for mocking an internal factory dependency in the new orchestration test: +vi.mock('./mimecast-client', () => ({ + isMimecastConfigured: () => true, + getMimecastClient: () => ({ + getMessageInfo: vi.fn(), + searchDeliveredMessages: vi.fn(), + getHeldMessages: vi.fn(), + getThreatEvents: vi.fn(), + }), +})); +``` + +### Module header/return-type convention to follow + +```typescript +// Source: lib/services/phishing-eml-service.ts (existing, verified by reading the file) +/** + * . + * + * . + */ +export interface FooInput { /* ... */ } +export interface FooResult { /* discriminated union, no throw on expected non-error outcomes */ } + +export async function doTheThing(input: FooInput): Promise { + try { + // ... + } catch (err) { + console.error('[MIMECAST-BLAST-RADIUS] ...', err); + return { status: 'unavailable', reason: 'lookup_failed' }; + } +} +``` + +## State of the Art + +Not applicable in the traditional sense — this is a small internal composition task, not a +library/framework adoption. One relevant note: the repo's own `docs/mimecast-api-guide.md` +documents a *different* set of Mimecast endpoints (`/api/audit/get-siem-logs`, +`/api/ttp/attachment/get-logs`, `/api/ttp/url/get-managed-url`, `/api/audit/get-audit-events`) +than the ones `mimecast-client.ts` actually implements (`/api/message-finder/search`, +`/api/message-finder/get-message-info`, `/api/gateway/get-hold-message-list`, +`/threats/v1/events`, `/siem/v1/batch/events/cg`). The doc appears to be a generic +"threat dashboard" reference guide, not documentation of this specific client's implementation +— treat `mimecast-client.ts`'s source code as the source of truth for what this phase can +actually call, not `docs/mimecast-api-guide.md`. + +**Deprecated/outdated:** N/A. + +## Assumptions Log + +| # | Claim | Section | Risk if Wrong | +|---|-------|---------|---------------| +| A1 | Mimecast's `/threats/v1/events` endpoint (wrapped by `getThreatEvents()`) does not carry URL-click event data — that lives in a separate, currently-unwrapped `/api/ttp/url/get-logs` endpoint | Common Pitfalls #2 | If wrong, D-02's best-effort click derivation could actually work reliably rather than mostly returning 0 — low risk either way since D-02 explicitly treats 0 as an acceptable best-effort value, but code comments should not overstate confidence in either direction until validated against a live tenant | +| A2 | This phase should use only the single env-var-configured `getMimecastClient()`, not the per-company `mimecast_tenants` table / `getMimecastClientForTenant()` | Common Pitfalls #3, Open Questions | If wrong, blast-radius lookups for companies with their own Mimecast tenant (not the global account) will silently return `unavailable` or incomplete data despite Mimecast genuinely being configured for that company — this could materially affect Phase 19's classification confidence for those companies | +| A3 | The exact string values of `MimecastDeliveredMessage.status` (used to distinguish delivered vs. rejected) have not been confirmed against a live/sandbox Mimecast tenant response — only the TypeScript field name and type (`string`) are confirmed from the client code | Architecture Patterns (Pattern 3) | If the assumed status strings (e.g. "Delivered", "Rejected") don't match Mimecast's actual values, the delivered/rejected split in the normalized shape will misclassify recipients; implementer should log raw values during first real-tenant test rather than hardcoding a guessed enum | + +## Open Questions + +1. **Should this phase's abstraction resolve a per-company Mimecast tenant, or only use the + single global env-var client?** + - What we know: `mimecast_tenants` (migration 062) exists and is actively used by + `/api/mimecast/held` and `/api/mimecast/delivered` (the two existing routes closest in + purpose to this phase's lookup). `reports.company_id` (migration 097) is available to key + off of. CONTEXT.md's decisions and BLAST-01/02 requirement text only mention the single + env-var client and `checkConfigOnly`'s env-var-only health check. + - What's unclear: Whether Phase 19's classifier will ever be classifying reports from + companies whose Mimecast is only registered per-tenant (not covered by the global + `MIMECAST_CLIENT_ID`), and whether that gap is acceptable for v1. + - Recommendation: Default to the single env-var client for this phase (matches locked + scope and keeps the abstraction simple), but flag this explicitly as a known limitation + in code comments and in the phase's done-report, so it's a conscious tradeoff rather than + a silent gap. If per-company resolution turns out to be needed, it's a small, mechanical + addition later (swap `getMimecastClient()` for a tenant lookup + `getMimecastClientForTenant()`, + using the same fan-out logic). + +2. **What are the real string values Mimecast returns for `status` in message-tracking search + results (delivered vs. rejected classification)?** + - What we know: The TypeScript field exists (`MimecastDeliveredMessage.status: string`) and + is populated from `e.status` in the raw API response, per `searchDeliveredMessages()`'s + implementation. + - What's unclear: The actual enum of values (no live/sandbox Mimecast tenant was queried + during this research pass — this repo's existing `.env` may have real Mimecast credentials, + see CLAUDE.md's warning to treat `.env` as potentially real, but exercising a live call was + out of scope for a research pass with no test tenant confirmed safe to query). + - Recommendation: During implementation/testing, log raw `status` values seen from a real + or sandbox account before finalizing the delivered/rejected classification logic. Do not + hardcode a guessed enum as if it were confirmed. + +## Environment Availability + +| Dependency | Required By | Available | Version | Fallback | +|------------|------------|-----------|---------|----------| +| `MIMECAST_CLIENT_ID` / `MIMECAST_CLIENT_SECRET` env vars | BLAST-01 (live Mimecast calls) | Unknown at research time — depends on target deploy env | — | `isMimecastConfigured()` returns `false` → `status: 'unavailable'` (BLAST-02, by design) | +| Redis (`REDIS_URL`) | D-04 caching | Optional — `redis-client.ts` already handles absence gracefully | ioredis 5.9.0 (installed) | `getCachedData`/`setCachedData` no-op when `REDIS_URL` unset; lookups just aren't cached, no functional break | +| Mimecast API reachability (`https://api.services.mimecast.com`) | BLAST-01 live calls | Not tested (would require real credentials + network egress) | — | Wrapped in try/catch per Pattern 2 — network failure degrades to `status: 'unavailable'`, never throws to caller | + +**Missing dependencies with no fallback:** None — every dependency in this phase has an +explicit graceful-degradation path by design (that's the point of BLAST-02). + +**Missing dependencies with fallback:** Redis absence (caching becomes a no-op, not a failure); +Mimecast credential absence (`unavailable` status, not a crash). + +## Validation Architecture + +### Test Framework + +| Property | Value | +|----------|-------| +| Framework | vitest 4.1.5 (`vitest.config.ts`) | +| Config file | `vitest.config.ts` — `include: ['lib/**/*.test.ts']`, `environment: 'node'` | +| Quick run command | `npx vitest run lib/services/mimecast-blast-radius.test.ts lib/services/mimecast-client.test.ts` | +| Full suite command | `npm test` (= `vitest run`) | + +**Important correction to CLAUDE.md's stated test coverage:** CLAUDE.md states `npm test` +"covers `lib/services/analyzer/**`, `lib/services/rmm/**`, `lib/services/b2/**`, and +`lib/services/analyzer/link-discovery.test.ts`. Other parts of the codebase have no tests." +This is now **out of date** — `vitest.config.ts`'s actual `include` glob is `lib/**/*.test.ts` +(no subdirectory restriction), and the repo already has passing test files well outside those +three directories: `lib/permissions.test.ts`, `lib/services/autotask-client.test.ts`, +`lib/services/pax8-*.test.ts` (4 files), `lib/services/phishing-detector.test.ts`, +`lib/services/phishing-eml-service.test.ts`, `lib/services/eml-parser.test.ts`. **Confirmed:** +a new `lib/services/mimecast-blast-radius.test.ts` and `lib/services/mimecast-client.test.ts` +WILL run under `npm test` — no special invocation or config change needed. + +### Phase Requirements → Test Map + +| Req ID | Behavior | Test Type | Automated Command | File Exists? | +|--------|----------|-----------|-------------------|-------------| +| BLAST-01 | Configured + fan-out returns normalized matched/delivered/held/rejected/clicked + perRecipient | unit | `npx vitest run lib/services/mimecast-blast-radius.test.ts` | ❌ Wave 0 | +| BLAST-01 | `isMimecastConfigured()` reflects env var presence | unit | `npx vitest run lib/services/mimecast-client.test.ts` | ❌ Wave 0 | +| BLAST-02 | Not configured → synchronous `status: 'unavailable'`, no throw | unit | `npx vitest run lib/services/mimecast-blast-radius.test.ts` | ❌ Wave 0 | +| BLAST-02 | Unexpected fan-out error (e.g. mocked rejection) degrades to `unavailable`, does not throw | unit | `npx vitest run lib/services/mimecast-blast-radius.test.ts` | ❌ Wave 0 | +| D-04 | Cached lookup returns without re-calling `MimecastClient` methods | unit | `npx vitest run lib/services/mimecast-blast-radius.test.ts` | ❌ Wave 0 | + +### Sampling Rate + +- **Per task commit:** `npx vitest run lib/services/mimecast-blast-radius.test.ts lib/services/mimecast-client.test.ts` +- **Per wave merge:** `npm test` (full suite — fast, no real network/DB in this phase's tests since everything is mocked per the `pax8-factory.test.ts` / `phishing-eml-service.test.ts` discipline) +- **Phase gate:** Full suite green before `/gsd:verify-work` + +### Wave 0 Gaps + +- [ ] `lib/services/mimecast-client.test.ts` — new file, covers `isMimecastConfigured()` (BLAST-01/02 config gate) +- [ ] `lib/services/mimecast-blast-radius.test.ts` — new file, covers BLAST-01/02 and D-01 through D-04 +- [ ] No shared fixture file needed — synthetic inline mock objects (matching `MimecastDeliveredMessage`/`MimecastHeldMessage`/`MimecastThreatEvent` shapes above) are sufficient; no `.eml`-scale fixture complexity here +- [ ] Framework install: none — vitest already installed and configured + +## Security Domain + +`security_enforcement` is not present in `.planning/config.json` — treated as enabled per +default. + +### Applicable ASVS Categories + +| ASVS Category | Applies | Standard Control | +|---------------|---------|-------------------| +| V2 Authentication | No | This phase has no HTTP endpoint, no session handling — it's an internal function called by a future in-process consumer (Phase 19) | +| V3 Session Management | No | Same as above | +| V4 Access Control | No | No new API route in this phase; ACCESS-01 (auth on `/api/phishing/*`) is correctly mapped to Phase 18, not this one | +| V5 Input Validation | Yes | Input shape (`messageId?`, `sender`, `recipient`, `subject`, `dateWindow`) should be validated at the type level (TypeScript) before being passed into Mimecast query params; no raw user-supplied string should be interpolated into request bodies beyond what `MimecastClient.request()` already JSON-serializes safely | +| V6 Cryptography | No | Reuses `MimecastClient`'s existing OAuth2 token handling; no new crypto in this phase | + +### Known Threat Patterns for this stack + +| Pattern | STRIDE | Standard Mitigation | +|---------|--------|---------------------| +| Overly broad Mimecast fan-out query (e.g. missing subject/sender bounds) returning another company's unrelated messages | Information Disclosure | Always require sender + subject + date-window (or Message-ID) — never call `searchDeliveredMessages`/`getHeldMessages` with only a date range; this phase's own input type already requires these fields | +| Logging full Mimecast response bodies (which may include other recipients' email content/subjects) in error paths | Information Disclosure | Follow the existing pattern in `mimecast-sync-service.ts` / API routes: log `error.message`, not full response bodies, in `console.error` calls | +| Redis cache key collision across companies (if per-company resolution is added later per Open Question #1) if key doesn't include a company/tenant discriminator | Tampering / Information Disclosure | If tenant resolution is added, the cache key MUST include the tenant/company identifier, not just message identity, to avoid one company's cached blast-radius data being served for another company's identically-subjected report | + +## Sources + +### Primary (HIGH confidence — read directly from this repo) +- `lib/services/mimecast-client.ts` — full file read; all type shapes and method signatures in this document are verified against this source, not recalled from training data +- `lib/services/mimecast-sync-service.ts` — confirmed `getMimecastClient()` (not per-tenant) usage in the bulk sync path +- `lib/services/veeam-factory.ts` — `isConfigured()` + `getClient()` factory precedent +- `lib/services/redis-client.ts` — full file read; `getCachedData`/`setCachedData`/default TTL confirmed +- `lib/services/integration-health.ts` (lines 1-80, 300-360) — `checkConfigOnly()` implementation and the exact `mimecast` entry (env vars, category) +- `app/api/mimecast/status/route.ts`, `held/route.ts`, `delivered/route.ts`, `threats/route.ts` — confirmed existing call patterns, per-tenant vs. singleton client usage, and the exact throw-then-catch failure mode this phase must pre-empt +- `migrations/062_create_mimecast_tenants.sql`, `migrations/097_phishing_triage_schema.sql` — confirmed multi-tenant table schema and `classifications.reasons`/`reports.company_id` schema +- `vitest.config.ts` — confirmed `include: ['lib/**/*.test.ts']` (no subdirectory restriction) +- `lib/services/pax8-factory.test.ts`, `lib/services/phishing-eml-service.ts` — confirmed test-mocking discipline and module-header/return-type conventions to follow +- `.planning/phases/17-mimecast-blast-radius-lookup/17-CONTEXT.md`, `.planning/REQUIREMENTS.md`, `.planning/STATE.md` — phase scope and locked decisions + +### Secondary (MEDIUM confidence — official Mimecast docs, cross-checked against this repo's code) +- [Get TTP URL Logs — Mimecast integrations docs](https://integrations.mimecast.com/documentation/endpoint-reference/logs-and-statistics/get-ttp-url-logs/) — confirms click-log data (`userEmailAddress`, `url`, `category`, `action`, `date`) lives at `POST /api/ttp/url/get-logs`, a different endpoint than `/threats/v1/events` +- `docs/mimecast-api-guide.md` (in-repo) — cross-checked against actual client implementation; found to document a *different* endpoint set than what `mimecast-client.ts` implements (see State of the Art) + +### Tertiary (LOW confidence — not independently verified) +- Exact runtime string values of Mimecast's message-tracking `status` field (delivered/rejected classification) — not verified against a live tenant during this research pass; flagged in Assumptions Log (A3) and Open Questions (#2) +- Whether `/threats/v1/events` ever surfaces a click-type `analysis` value in practice (vs. the documented-separate `/api/ttp/url/get-logs` endpoint) — flagged in Assumptions Log (A1) and Common Pitfalls #2 + +## Metadata + +**Confidence breakdown:** +- Standard stack: HIGH — no new dependencies, all reused modules read directly +- Architecture: HIGH — patterns and field names verified against actual source files, not assumed +- Pitfalls: HIGH for #1 and #3 (both derived directly from reading source code); MEDIUM for #2 (derived from official Mimecast docs, not this account's live API responses) + +**Research date:** 2026-07-15 +**Valid until:** 30 days (internal composition of existing stable modules; re-verify if `mimecast-client.ts` or `redis-client.ts` change before this phase is implemented)