docs(17): add pattern map

This commit is contained in:
lorentz 2026-07-15 13:39:21 -04:00
parent c702cb9b03
commit 78ff39fa48

View file

@ -0,0 +1,605 @@
# Phase 17: Mimecast Blast Radius Lookup - Pattern Map
**Mapped:** 2026-07-15
**Phase directory:** `.planning/phases/17-mimecast-blast-radius-lookup`
This document maps each file this phase touches to its closest existing
analog in the codebase, with concrete excerpts to copy the shape of (not the
literal content). All analogs below were re-read directly from source during
this mapping pass — line numbers and code are current as of 2026-07-15.
---
## File 1: `lib/services/mimecast-client.ts` (MODIFIED — add `isMimecastConfigured()`)
**Role:** Existing 676-line API client class + factory function. This phase
adds exactly one new export; it does not touch `MimecastClient` internals,
`getMimecastClient()`, or `getMimecastClientForTenant()`.
**Data flow role:** Config-gate helper. Called first by the new
`mimecast-blast-radius.ts` orchestrator (Pattern 2 below) to decide whether to
proceed at all — never itself calls the network.
### Closest analog: `lib/services/pax8-factory.ts` (verified, full file, 26 lines)
```typescript
import { Pax8Client } from './pax8-client';
let _client: Pax8Client | null = null;
export function isPax8Configured(): boolean {
return Boolean(process.env.PAX8_CLIENT_ID && process.env.PAX8_CLIENT_SECRET);
}
export function getPax8Client(): Pax8Client {
if (_client) return _client;
if (!isPax8Configured()) {
throw new Error('PAX8 is not configured — set PAX8_CLIENT_ID and PAX8_CLIENT_SECRET');
}
_client = new Pax8Client({
clientId: process.env.PAX8_CLIENT_ID!,
clientSecret: process.env.PAX8_CLIENT_SECRET!,
});
console.log('[PAX8] Client initialized');
return _client;
}
// Test seam — reset the cached client (e.g. after rotating credentials).
export function _resetPax8Client(): void {
_client = null;
}
```
`lib/services/veeam-factory.ts` (analogous, separate-file variant — NOT the
pattern to follow here since `mimecast-client.ts` already contains its own
factory function inline, unlike Veeam which splits `veeam-client.ts` /
`veeam-factory.ts`):
```typescript
export function isVeeamConfigured(): boolean {
return !!(process.env.VEEAM_VSPC_URL && process.env.VEEAM_VSPC_API_KEY);
}
```
### What's already in `mimecast-client.ts` (verified, exact lines)
```typescript
// lines 645-662 — existing, DO NOT modify behavior, only add isMimecastConfigured() near it
let _client: MimecastClient | null = null;
export function getMimecastClient(): MimecastClient {
if (!_client) {
const clientId = process.env.MIMECAST_CLIENT_ID;
const clientSecret = process.env.MIMECAST_CLIENT_SECRET;
if (!clientId || !clientSecret) {
throw new Error('MIMECAST_CLIENT_ID and MIMECAST_CLIENT_SECRET must be set');
}
_client = new MimecastClient({
clientId,
clientSecret,
baseUrl: process.env.MIMECAST_BASE_URL ?? 'https://api.services.mimecast.com',
accountCode: process.env.MIMECAST_ACCOUNT_CODE ?? '',
});
}
return _client;
}
```
The exact env vars are already confirmed by `checkConfigOnly` in
`lib/services/integration-health.ts` (line 338-339):
```typescript
Promise.resolve(checkConfigOnly('mimecast', 'Mimecast', 'mail',
['MIMECAST_CLIENT_ID', 'MIMECAST_CLIENT_SECRET'])),
```
### Concrete addition to make
```typescript
// Add near getMimecastClient(), e.g. directly above it (~line 645):
export function isMimecastConfigured(): boolean {
return !!(process.env.MIMECAST_CLIENT_ID && process.env.MIMECAST_CLIENT_SECRET);
}
```
**Gap vs. pax8/veeam precedent — flag for planner:** neither
`_client` reset helper nor a test seam exists today in `mimecast-client.ts`
(no `_resetMimecastClient()`). `mimecast-client.test.ts` (Wave 0, new file)
needs the module-level `_client` singleton to be resettable between tests
the same way `_resetPax8Client()` works — add a matching
`_resetMimecastClient(): void { _client = null; }` export alongside
`isMimecastConfigured()` so tests can isolate env-var state per `it()` block
without cross-test pollution (mirrors `pax8-factory.test.ts`'s `beforeEach`
exactly — see File 3 below).
---
## File 2: `lib/services/mimecast-blast-radius.ts` (NEW)
**Role:** Pure orchestration/composition layer. No HTTP route, no DB write,
no new external call surface — calls existing `MimecastClient` methods and
merges their output into one normalized shape. Never throws (BLAST-02).
**Data flow role:**
```
Phase 19 classifier (future, not built) ──► getBlastRadius(input)
├─ isMimecastConfigured() false ──► return {status:'unavailable'} (sync)
├─ cache hit (Redis) ──► return cached result
└─ fan-out (always, for counts):
getMessageInfo() (supplementary body/header only)
searchDeliveredMessages() ┐
getHeldMessages() ├─► merge → normalized shape → cache → return
getThreatEvents() ┘
```
### Closest analog: `lib/services/phishing-eml-service.ts` (verified, full file read, 225 lines)
This is the best structural analog in the repo: a service-layer orchestrator
that composes several existing clients (`getAutotaskClient()`, B2, `eml-parser`),
degrades gracefully on partial failure (never throws for expected "nothing
found" outcomes), and reserves `throw` only for genuinely unexpected errors.
Module header convention to copy:
```typescript
/**
* Phishing EML/MIME evidence orchestration service (Phase 16, Plan 03).
*
* Turns a detected phishing report into persisted, normalized message
* evidence: ...
*
* Hard invariant (SC#3 / T-16-03): this service never fetches or executes
* anything found in the parsed message. ...
*/
import { postgresClient } from './postgres-client';
import { getAutotaskClient } from './autotask-factory';
import { presignUpload, isB2Configured, EML_OBJECT_KEY_REGEX } from './b2/client';
import { parseEml, selectOriginalMessage, MAX_EML_BYTES, type NormalizedMessage } from './eml-parser';
export interface ParseAndStoreInput {
reportId: string;
ticketId: number;
}
export interface ParseAndStoreResult {
stored: boolean;
messageId?: string;
reason?: string;
}
```
Graceful-degrade pattern to copy (B2 upload failure does not abort the whole
operation — same posture this phase needs for Mimecast fan-out failures):
```typescript
let rawRef: string | null = null;
if (isB2Configured()) {
const objectKey = `phishing/${reportId}/${selected.id}.eml`;
try {
const uploadUrl = presignUpload(objectKey, 1800, undefined, EML_OBJECT_KEY_REGEX);
const putResponse = await fetch(uploadUrl, { method: 'PUT', body: rawEmlBuffer });
if (!putResponse.ok) {
throw new Error(`B2 PUT ${objectKey} failed: ${putResponse.status}`);
}
rawRef = objectKey;
} catch (error) {
// Graceful degrade (RESEARCH.md Pitfall 3) — a failed/absent B2 PUT
// must not abort parsing/persistence.
console.error('[PHISHING-EML] Failed to upload raw .eml to B2 for report', reportId, error);
rawRef = null;
}
} else {
console.log('[PHISHING-EML] B2 not configured; skipping raw .eml upload for report', reportId);
}
```
Top-level try/catch + logging convention (note: `phishing-eml-service.ts`
*rethrows* here because its caller needs fail-vs-degrade control — this
phase's top-level function must NOT rethrow, per BLAST-02; catch and return
`{status:'unavailable', reason:'lookup_failed'}` instead):
```typescript
} catch (error) {
console.error(
'[PHISHING-EML] Failed to parse/store message for report',
reportId, 'ticket', ticketId, error
);
throw error; // <-- this phase's equivalent must NOT do this; return degraded result instead
}
```
### Redis caching analog: `app/api/addigy-devices/route.ts` (verified, lines 1-45)
This is the only place in the codebase that actually *calls*
`getCachedData`/`setCachedData` from `redis-client.ts` end-to-end — confirms
the exact cache-key string format and TTL-call convention to follow:
```typescript
import { getCachedData, setCachedData } from '@/lib/services/redis-client';
// Create cache key based on query parameters
const cacheKey = `addigy:devices:${policyId || 'all'}:${online || 'all'}`;
// Try to get cached data
const cachedDevices = await getCachedData<any[]>(cacheKey);
if (cachedDevices) {
console.log(`Cache hit for key: ${cacheKey}`);
return NextResponse.json({ success: true, data: cachedDevices, count: cachedDevices.length, cached: true });
}
console.log(`Cache miss for key: ${cacheKey}, fetching from API`);
// ... fetch ...
// Cache the result for 5 minutes
await setCachedData(cacheKey, devices, 300);
```
**Cache key format for this phase** (following `<service>:<resource>:<discriminators>`):
```typescript
// Message-ID known:
const cacheKey = `mimecast:blast-radius:msgid:${messageId}`;
// Message-ID unknown — composite of sender+subject+date-window per D-04:
const cacheKey = `mimecast:blast-radius:composite:${sender}:${subject}:${dateWindow.start.toISOString()}:${dateWindow.end.toISOString()}`;
```
Note: raw `subject` may contain `:` or other characters that don't break
Redis keys (Redis keys are opaque byte strings) — no escaping required, but
consider a stable hash (e.g. simple string concatenation as above, matching
`addigy-devices`'s unescaped-param-interpolation precedent) rather than
inventing a new hashing scheme.
### `redis-client.ts` exports being reused (verified, full file, lines 45-73)
```typescript
export async function getCachedData<T>(key: string): Promise<T | null> {
const client = getRedisClient();
if (!client) return null; // REDIS_URL unset → silent no-op, not an error
try {
const data = await client.get(key);
if (data) return JSON.parse(data);
} catch (error) {
console.error(`Error getting cached data for key ${key}:`, error);
}
return null;
}
export async function setCachedData<T>(
key: string,
data: T,
ttlSeconds: number = 300 // Default 5 minutes — matches D-04's TTL exactly, no override needed
): Promise<void> {
const client = getRedisClient();
if (!client) return;
try {
await client.set(key, JSON.stringify(data), 'EX', ttlSeconds);
} catch (error) {
console.error(`Error setting cached data for key ${key}:`, error);
}
}
```
Because the default TTL is already 300s (5 min), `setCachedData(cacheKey, result)`
can omit the third argument entirely and still satisfy D-04 — pass it
explicitly (`300`) anyway for readability, matching the `addigy-devices`
precedent's explicit `300`.
### `MimecastClient` methods being composed (verified, `lib/services/mimecast-client.ts`)
```typescript
// getMessageInfo — supplementary body/header ONLY, no status/counts (Pitfall 1)
export interface MimecastMessageInfo {
messageId: string;
bodyText?: string;
bodyHtml?: string;
headers?: Record<string, string>;
}
async getMessageInfo(messageId: string): Promise<MimecastMessageInfo | null>
// searchDeliveredMessages — primary source of delivered/rejected counts
export interface MimecastDeliveredMessage {
id: string; status: string; subject: string; from: string; fromEnv: string;
to: string; toDisplay: string; received: string; senderIP: string;
spamScore: number; detectionLevel: string; attachments: boolean; route: string; info: string;
}
async searchDeliveredMessages(options: {
to?: string; from?: string; subject?: string;
startHours?: number; start?: string; end?: string; route?: string;
}): Promise<{ messages: MimecastDeliveredMessage[]; error?: string }>
// NEVER throws — internally try/catch, returns { messages: [], error: err.message } on failure
// getHeldMessages — primary source of held counts
export interface MimecastHeldMessage {
id: string; subject: string; from: string; fromDisplay: string; to: string;
toDisplay: string; dateReceived: string; reason: string; reasonCode: string;
policyInfo: string; route: string; hasAttachments: boolean; size: number;
}
async getHeldMessages(options: { recipient?: string; maxMessages?: number })
: Promise<{ messages: MimecastHeldMessage[]; totalCount: number }>
// CAN throw (paginated fetch loop, no top-level try/catch around the do/while) — orchestrator must wrap in try/catch
// getThreatEvents — source of clicked (best-effort, D-02) via analysis[] subtype
export interface MimecastThreatEvent {
id: string; messageId?: string; eventType: string; threatLevel?: string;
url?: string; fileName?: string; verdict?: string; actorEmail?: string;
eventDateTime?: string; analysis?: string[]; source?: string[]; direction?: string[]; status?: string[];
}
async getThreatEvents(options: { cursor?: string; pageSize?: number } = {})
: Promise<PaginatedResult<MimecastThreatEvent>>
// NEVER throws — internally try/catch, returns { items: [], nextCursor: null } on failure
// No server-side sender/subject/date filter param exists — filter client-side after fetch
```
**Per-recipient merge note (verified from source, `searchDeliveredMessages`
line 589 / `getHeldMessages` line 528):** both `MimecastDeliveredMessage.to`
and `MimecastHeldMessage.to` are populated from a single-recipient field
(`e.to?.[0]?.emailAddress` / `m.to?.emailAddress` respectively) — one row per
recipient-message pair, confirming RESEARCH.md's per-recipient merge
strategy is safe to implement as described (group by `to`, classify
delivered/held/unknown).
### Never-throw wrapper precedent: `app/api/mimecast/status/route.ts`
Confirms the exact throw message this new module's `isMimecastConfigured()`
gate pre-empts, and the try/catch-to-JSON pattern used elsewhere for
Mimecast-specific failure handling:
```typescript
try {
const client = getMimecastClient(); // throws 'MIMECAST_CLIENT_ID and MIMECAST_CLIENT_SECRET must be set'
const connection = await client.testConnection();
return NextResponse.json({ configured: true, connected: connection.ok, ... });
} catch (err: any) {
return NextResponse.json({ configured: false, connected: false, error: err.message, ... });
}
```
### Recommended module shape (from RESEARCH.md, cross-checked against analogs above)
```typescript
/**
* Mimecast blast-radius lookup abstraction (Phase 17).
*
* Given a reported message's identity (Message-ID, sender, recipient,
* subject, date window), returns normalized delivery data — matched/
* delivered/held/rejected/clicked counts and per-recipient status — when
* Mimecast is configured, or a clean `status: 'unavailable'` signal (never a
* throw) when it isn't or when the underlying lookup fails unexpectedly.
*
* `clicked` is best-effort, derived from getThreatEvents()'s analysis[]
* subtype — Mimecast's actual click-tracking data lives in a separate,
* currently-unwrapped /api/ttp/url/get-logs endpoint (see 17-RESEARCH.md
* Pitfall 2). A value of 0 means "no click-type threat event found," not
* "confirmed zero clicks."
*
* KNOWN LIMITATION (D-05): uses only the single env-var-configured
* getMimecastClient(), not the per-company mimecast_tenants table. Reports
* from companies with their own Mimecast tenant will return `unavailable`
* even though Mimecast is technically configured for that company.
*/
import { isMimecastConfigured, getMimecastClient } from './mimecast-client';
import { getCachedData, setCachedData } from './redis-client';
export interface BlastRadiusInput {
messageId?: string;
sender: string;
recipient: string;
subject: string;
dateWindow: { start: Date; end: Date };
}
export type BlastRadiusResult =
| { status: 'unavailable'; reason: 'not_configured' | 'lookup_failed'; error?: string }
| {
status: 'ok';
matched: number;
delivered: number;
held: number;
rejected: number;
clicked: number;
perRecipient: Array<{ recipient: string; status: 'delivered' | 'held' | 'rejected' | 'unknown' }>;
source: 'fan-out';
};
export async function getBlastRadius(input: BlastRadiusInput): Promise<BlastRadiusResult> {
if (!isMimecastConfigured()) {
return { status: 'unavailable', reason: 'not_configured' };
}
const cacheKey = input.messageId
? `mimecast:blast-radius:msgid:${input.messageId}`
: `mimecast:blast-radius:composite:${input.sender}:${input.subject}:${input.dateWindow.start.toISOString()}:${input.dateWindow.end.toISOString()}`;
const cached = await getCachedData<BlastRadiusResult>(cacheKey);
if (cached) return cached;
try {
const client = getMimecastClient();
// Supplementary only — never gates the fan-out (Pitfall 1)
if (input.messageId) {
await client.getMessageInfo(input.messageId); // body/headers, not used for counts
}
const [delivered, held, threats] = await Promise.all([
client.searchDeliveredMessages({
to: input.recipient,
from: input.sender,
subject: input.subject,
start: input.dateWindow.start.toISOString().replace(/\.\d{3}Z$/, '+0000'),
end: input.dateWindow.end.toISOString().replace(/\.\d{3}Z$/, '+0000'),
}),
client.getHeldMessages({ recipient: input.recipient }),
client.getThreatEvents(),
]);
// merge → perRecipient, matched/delivered/held/rejected/clicked
// (see RESEARCH.md Pattern 3 for the exact merge algorithm)
const result: BlastRadiusResult = { status: 'ok', /* ...merged fields... */ source: 'fan-out' } as BlastRadiusResult;
await setCachedData(cacheKey, result, 300);
return result;
} catch (err: any) {
console.error('[MIMECAST-BLAST-RADIUS] lookup failed', err);
return { status: 'unavailable', reason: 'lookup_failed', error: err.message };
}
}
```
---
## File 3: `lib/services/mimecast-client.test.ts` (NEW)
**Role:** Unit tests for the new `isMimecastConfigured()` export only (does
not test `MimecastClient`'s existing 20+ methods — out of scope for this
phase).
### Exact analog to copy: `lib/services/pax8-factory.test.ts` (verified, full file, 61 lines)
```typescript
import { describe, it, expect, beforeEach } from 'vitest';
import { isPax8Configured, getPax8Client, _resetPax8Client } from './pax8-factory';
beforeEach(() => {
delete process.env.PAX8_CLIENT_ID;
delete process.env.PAX8_CLIENT_SECRET;
_resetPax8Client();
});
describe('isPax8Configured', () => {
it('returns false when neither env var is set', () => {
expect(isPax8Configured()).toBe(false);
});
it('returns false when only PAX8_CLIENT_ID is set', () => {
process.env.PAX8_CLIENT_ID = 'id1';
expect(isPax8Configured()).toBe(false);
});
it('returns false when only PAX8_CLIENT_SECRET is set', () => {
process.env.PAX8_CLIENT_SECRET = 'secret1';
expect(isPax8Configured()).toBe(false);
});
it('returns true when both env vars are set', () => {
process.env.PAX8_CLIENT_ID = 'id1';
process.env.PAX8_CLIENT_SECRET = 'secret1';
expect(isPax8Configured()).toBe(true);
});
});
```
### Adapted shape for `mimecast-client.test.ts`
```typescript
import { describe, it, expect, beforeEach } from 'vitest';
import { isMimecastConfigured, getMimecastClient, _resetMimecastClient } from './mimecast-client';
beforeEach(() => {
delete process.env.MIMECAST_CLIENT_ID;
delete process.env.MIMECAST_CLIENT_SECRET;
_resetMimecastClient();
});
describe('isMimecastConfigured', () => {
it('returns false when neither env var is set', () => {
expect(isMimecastConfigured()).toBe(false);
});
it('returns false when only MIMECAST_CLIENT_ID is set', () => {
process.env.MIMECAST_CLIENT_ID = 'id1';
expect(isMimecastConfigured()).toBe(false);
});
it('returns false when only MIMECAST_CLIENT_SECRET is set', () => {
process.env.MIMECAST_CLIENT_SECRET = 'secret1';
expect(isMimecastConfigured()).toBe(false);
});
it('returns true when both env vars are set', () => {
process.env.MIMECAST_CLIENT_ID = 'id1';
process.env.MIMECAST_CLIENT_SECRET = 'secret1';
expect(isMimecastConfigured()).toBe(true);
});
});
describe('getMimecastClient', () => {
it('throws the exact configuration error when not configured', () => {
expect(() => getMimecastClient()).toThrow(
'MIMECAST_CLIENT_ID and MIMECAST_CLIENT_SECRET must be set'
);
});
it('returns the same cached instance on repeated calls', () => {
process.env.MIMECAST_CLIENT_ID = 'id1';
process.env.MIMECAST_CLIENT_SECRET = 'secret1';
const first = getMimecastClient();
const second = getMimecastClient();
expect(second).toBe(first);
});
});
```
Requires the `_resetMimecastClient()` test seam flagged in File 1 above to be
added alongside `isMimecastConfigured()`.
---
## File 4: `lib/services/mimecast-blast-radius.test.ts` (NEW)
**Role:** Unit tests for `getBlastRadius()` covering BLAST-01, BLAST-02, and
D-01 through D-04 (config gate, fan-out merge, never-throw on error, cache
hit/miss).
### Exact analog for factory-mocking discipline: `lib/services/phishing-eml-service.test.ts` (verified, lines 1-90)
```typescript
import { describe, it, expect, vi, beforeEach, afterEach } from 'vitest';
vi.mock('./postgres-client', () => ({ /* ... */ }));
vi.mock('./autotask-factory', () => ({ /* ... */ }));
vi.mock('./b2/client', () => ({ /* ... */ }));
// Import AFTER the mocks are declared so vi.mock hoisting takes effect.
```
RESEARCH.md's concrete recommended mock shape for this phase's test (verified
consistent with `mimecast-client.ts`'s actual export names):
```typescript
vi.mock('./mimecast-client', () => ({
isMimecastConfigured: vi.fn(() => true),
getMimecastClient: vi.fn(() => ({
getMessageInfo: vi.fn(),
searchDeliveredMessages: vi.fn(),
getHeldMessages: vi.fn(),
getThreatEvents: vi.fn(),
})),
}));
vi.mock('./redis-client', () => ({
getCachedData: vi.fn(() => Promise.resolve(null)),
setCachedData: vi.fn(() => Promise.resolve()),
}));
```
### Test cases to cover (from RESEARCH.md's Phase Requirements → Test Map)
| Behavior | Mock setup |
|---|---|
| Not configured → sync `{status:'unavailable', reason:'not_configured'}`, no Mimecast calls | `isMimecastConfigured` mocked `false`; assert `getMimecastClient` not called |
| Configured, cache hit → returns cached result, no fan-out calls | `getCachedData` mocked to resolve a fixture `BlastRadiusResult`; assert `searchDeliveredMessages` etc. not called |
| Configured, cache miss, fan-out succeeds → merges delivered/held/threat-event fixtures into `matched/delivered/held/rejected/clicked` + `perRecipient[]`, then calls `setCachedData` | Mock all four `MimecastClient` methods with synthetic fixtures matching `MimecastDeliveredMessage`/`MimecastHeldMessage`/`MimecastThreatEvent` shapes above |
| Fan-out call throws (e.g. `getHeldMessages` rejects) → degrades to `{status:'unavailable', reason:'lookup_failed', error}`, never throws to caller | Mock `getHeldMessages: vi.fn().mockRejectedValue(new Error('boom'))` |
| `clicked` derived from a threat event whose `analysis` includes a click-type value; `0` when none found | Two fixture variants of `getThreatEvents()`'s return |
No `.eml`-scale fixture file needed — inline synthetic objects matching the
verified interfaces are sufficient (per RESEARCH.md Wave 0 Gaps).
---
## Summary Table
| File | Status | Primary Analog | Key Precedent Extracted |
|---|---|---|---|
| `lib/services/mimecast-client.ts` | Modified (add 1 export + 1 test seam) | `lib/services/pax8-factory.ts` | `is<Name>Configured()` one-liner boolean check; needs a new `_resetMimecastClient()` seam not present today |
| `lib/services/mimecast-blast-radius.ts` | New | `lib/services/phishing-eml-service.ts` (orchestration/degrade shape) + `app/api/addigy-devices/route.ts` (cache key format) | Module header doc-comment convention; try/catch-degrade-not-throw; `service:resource:discriminators` cache key; `getCachedData`/`setCachedData` default 300s TTL |
| `lib/services/mimecast-client.test.ts` | New | `lib/services/pax8-factory.test.ts` | `beforeEach` env-var deletion + reset seam; one `describe` per exported function |
| `lib/services/mimecast-blast-radius.test.ts` | New | `lib/services/phishing-eml-service.test.ts` (`vi.mock` factory-mocking discipline) | Mock `./mimecast-client` and `./redis-client` entirely; assert call counts to prove short-circuit paths (not-configured, cache-hit) skip the fan-out |
---
*Pattern map: 2026-07-15*