---
phase: 16-eml-mime-evidence-parser
plan: 03
type: execute
wave: 2
depends_on: [16-01, 16-02]
files_modified:
  - lib/services/phishing-eml-service.ts
  - lib/services/phishing-eml-service.test.ts
autonomous: true
requirements: [EVID-03, EVID-04]
user_setup: []

must_haves:
  truths:
    - "Given a report + ticket id, the service lists attachments, selects the original message, fetches its full content, parses it, and writes one messages row linked by report_id"
    - "The parsed structured auth verdicts (D-06), normalized headers, URLs, and attachment metadata are persisted into messages.headers/urls/attachments/body_preview"
    - "When B2 is configured, raw .eml bytes are PUT to B2 and messages.raw_ref stores the B2 object key (D-05); when B2 is unconfigured the report's evidence still parses and persists without throwing"
    - "One indicators row is written per attachment-hash / URL / sender, each carrying its context in the metadata JSONB column (D-07)"
    - "No outbound network call is made to any URL found in the message — end to end"
    - "When no .eml attachment is found, the service returns a no-op result without throwing"
  artifacts:
    - path: "lib/services/phishing-eml-service.ts"
      provides: "parseAndStoreMessage orchestration: list→select→fetch→size-guard→B2 upload→parse→persist messages/indicators"
      min_lines: 90
    - path: "lib/services/phishing-eml-service.test.ts"
      provides: "orchestration coverage with mocked autotask/b2/postgres incl. no-network + B2-gated + no-eml no-op"
      min_lines: 70
  key_links:
    - from: "lib/services/phishing-eml-service.ts"
      to: "lib/services/eml-parser.ts"
      via: "parseEml + selectOriginalMessage imports"
      pattern: "from './eml-parser'"
    - from: "lib/services/phishing-eml-service.ts"
      to: "AutotaskClient.getAttachmentContent"
      via: "getAutotaskClient().getAttachmentContent(...)"
      pattern: "getAttachmentContent"
    - from: "lib/services/phishing-eml-service.ts"
      to: "messages / indicators tables"
      via: "postgresClient INSERT ... RETURNING id::text"
      pattern: "INSERT INTO messages"
    - from: "lib/services/phishing-eml-service.ts"
      to: "B2 (presignUpload + EML_OBJECT_KEY_REGEX)"
      via: "isB2Configured gate then self-PUT"
      pattern: "isB2Configured"
---

<objective>
Build the orchestration service that turns a detected phishing report into
persisted, normalized message evidence: list a ticket's attachments, select the
original reported message (Plan 01), fetch its full base64 content (Plan 02),
size-guard it, store the raw bytes in B2 under the dedicated `.eml` key (D-05,
Plan 02 regex), parse it (Plan 01), and persist a `messages` row plus
`indicators` rows (with the D-07 metadata JSONB) — never fetching or executing
anything found in the message.

Purpose: This is the consumer that makes D-05 (B2 raw storage), D-06 (structured
verdicts persisted), and D-07 (indicators.metadata) real. It is callable and
fully tested here; the live on-demand trigger (`POST /api/phishing/tickets/{id}/analyze`)
arrives in Phase 18 (DETECT-03).
Output: `lib/services/phishing-eml-service.ts` + `lib/services/phishing-eml-service.test.ts`.
</objective>

<execution_context>
@$HOME/.claude/get-shit-done/workflows/execute-plan.md
@$HOME/.claude/get-shit-done/templates/summary.md
</execution_context>

<context>
@.planning/PROJECT.md
@.planning/ROADMAP.md
@.planning/STATE.md
@.planning/phases/16-eml-mime-evidence-parser/16-RESEARCH.md
@.planning/phases/16-eml-mime-evidence-parser/16-PATTERNS.md
@lib/services/phishing-detector.ts
@lib/services/eml-parser.ts
@lib/services/b2/client.ts
@migrations/097_phishing_triage_schema.sql

<interfaces>
Consumes from Plan 01 (lib/services/eml-parser.ts):
- selectOriginalMessage(attachments: Attachment[]): Attachment | null
- async parseEml(rawEmlBuffer: Buffer): Promise<NormalizedMessage>
- const MAX_EML_BYTES: number
- NormalizedMessage { from{displayName,email,domain}, replyTo, returnPath, to[], cc[], subject, date, messageId, receivedChain[], authResults{spf,dkim,dmarc}, authResultsOriginal, urls[], attachments[{filename,contentType,size,checksum,related}], bodyPreview }

Consumes from Plan 02:
- AutotaskClient.getAttachmentContent(entityName, entityId, attachmentId): Promise<Attachment | null>  (returns attachment with base64 `data`)
- b2/client.ts: presignUpload(objectKey, ttl, cfg, keyRegex), isB2Configured(), EML_OBJECT_KEY_REGEX
- migrations/099: indicators.metadata JSONB column

Existing (lib/services/autotask-client.ts): getAttachments('Tickets', ticketId): Promise<Attachment[]>  (full list WITH id; note: reports.evidence only stores fullPath/title/contentType, NOT id — so this service must list live to get attachment ids)

Existing (lib/services/autotask-factory.ts): getAutotaskClient()
Existing (lib/services/postgres-client.ts): postgresClient.query<T>(sql, params)

Target schema (migrations/097 + 099):
- messages( id UUID PK, report_id UUID, message_id TEXT, headers JSONB, urls JSONB, attachments JSONB, body_preview TEXT, raw_ref TEXT, created_at )
- indicators( id UUID PK, message_id UUID, indicator_type TEXT, value TEXT, metadata JSONB, created_at )
</interfaces>
</context>

<tasks>

<task type="auto" tdd="true">
  <name>Task 1: phishing-eml-service.ts orchestration (list→select→fetch→B2→parse→persist)</name>
  <files>lib/services/phishing-eml-service.ts</files>
  <read_first>
    - lib/services/phishing-detector.ts lines 112-248 (gatherTicketEvidence fetch-then-transform pattern, detectPhishingTicket top-level try/catch + [TAG] console.error + rethrow, ON CONFLICT ... RETURNING id::text upsert shape)
    - lib/services/eml-parser.ts (the exact exported signatures + NormalizedMessage shape produced in Plan 01)
    - lib/services/b2/client.ts (isB2Configured, presignUpload signature incl. new keyRegex param, EML_OBJECT_KEY_REGEX, MAX_DOWNLOAD_BYTES)
    - .planning/phases/16-eml-mime-evidence-parser/16-RESEARCH.md (Pattern 4 self-PUT to B2, Pitfall 3 B2 unconfigured in dev, Open Questions 2-3 inline-attachment + size handling)
    - .planning/phases/16-eml-mime-evidence-parser/16-PATTERNS.md (phishing-eml-service.ts section: imports, graceful-degrade try/catch, INSERT ... RETURNING id::text convention, camelCase→snake_case boundary)
    - migrations/097_phishing_triage_schema.sql lines 77-108 (exact messages + indicators column lists)
  </read_first>
  <behavior>
    - parseAndStoreMessage({ reportId, ticketId }) lists attachments via getAutotaskClient().getAttachments('Tickets', ticketId), calls selectOriginalMessage on the full list (needs live list for attachment ids — reports.evidence lacks them)
    - If selectOriginalMessage returns null → returns a no-op result (e.g. { stored: false, reason: 'no-eml-attachment' }) without throwing and without writing any row
    - Otherwise fetches full content via getAttachmentContent('Tickets', ticketId, selected.id), base64-decodes `data` into a Buffer
    - Rejects/flags the report (no throw that crashes the caller) if the decoded buffer exceeds MAX_EML_BYTES — size guard before parseEml
    - When isB2Configured(): builds objectKey `phishing/{reportId}/{attachmentId}.eml`, presignUpload(objectKey, 1800, undefined, EML_OBJECT_KEY_REGEX), PUTs the raw buffer, sets rawRef = objectKey; when NOT configured: logs + skips B2, rawRef stays null, parsing/persisting still proceeds
    - Parses via parseEml, writes ONE messages row (report_id, message_id, headers JSONB = full normalized header block incl. authResults/authResultsOriginal per D-06, urls JSONB, attachments JSONB, body_preview, raw_ref) returning its id
    - Writes indicators rows: one per attachment checksum (indicator_type 'attachment_hash', value=checksum, metadata={filename,contentType,size,related}), one per URL (indicator_type 'url', value=url, metadata={part}), one for the sender (indicator_type 'sender', value=from.email, metadata={displayName,domain}) — each metadata written into the D-07 JSONB column
    - Never fetches any extracted URL (no fetch of message content beyond the Autotask attachment + B2 PUT)
  </behavior>
  <action>
    Create `lib/services/phishing-eml-service.ts` as a named-function module (no class) with a top doc-comment stating provenance and the never-fetch invariant. Imports: postgresClient from './postgres-client', getAutotaskClient from './autotask-factory', { presignUpload, isB2Configured, EML_OBJECT_KEY_REGEX } from './b2/client', { parseEml, selectOriginalMessage, MAX_EML_BYTES } from './eml-parser'. Export `async function parseAndStoreMessage(input: { reportId: string; ticketId: number }): Promise<{ stored: boolean; messageId?: string; reason?: string }>`. Implement the flow in the behavior block. Follow phishing-detector.ts conventions exactly: graceful-degrade try/catch around the Autotask list + getAttachmentContent + B2 PUT (log with `[PHISHING-EML]` tag, context values after the message string), the `INSERT INTO ... RETURNING id::text AS id` shape for both the messages insert and each indicators insert, and manual camelCase(TS)→snake_case(SQL) mapping at the query-building step (no ORM). Persist the full NormalizedMessage header block (including authResults and authResultsOriginal, D-06) into messages.headers JSONB via `$n::jsonb`. Gate the B2 PUT behind isB2Configured() (Pitfall 3 — dev has no B2 creds); on B2 PUT failure, log and continue with rawRef=null rather than aborting the whole parse. Guard buffer size against MAX_EML_BYTES before parseEml. Wrap the whole body in a top-level try/catch that logs `[PHISHING-EML]` + rethrows so a future caller (Phase 18) decides fail-vs-degrade. Decide inline/`related` attachment handling deliberately (Open Question 2): persist them but keep the `related` flag in indicator metadata so Phase 19 can weight them.
  </action>
  <verify>
    <automated>cd /opt/stacks/pulse && npx tsc --noEmit --pretty 2>&1 | (grep -i phishing-eml && exit 1 || echo tsc-ok)</automated>
  </verify>
  <done>phishing-eml-service.ts exports parseAndStoreMessage wiring list→select→fetch→(B2 gated)→parseEml→messages+indicators persistence, with graceful degrade, size guard, [PHISHING-EML] logging, and RETURNING id::text; tsc clean.</done>
  <acceptance_criteria>
    - `grep -n "selectOriginalMessage\|parseEml\|getAttachmentContent\|isB2Configured\|EML_OBJECT_KEY_REGEX" lib/services/phishing-eml-service.ts` matches all five
    - `grep -n "INSERT INTO messages" lib/services/phishing-eml-service.ts` and `INSERT INTO indicators` both present, each with `RETURNING id::text`
    - messages.headers is written from the NormalizedMessage header block including authResults (D-06) — grep for `authResults` reaching the persisted payload
    - indicators inserts write the metadata JSONB column (D-07) — grep for `metadata` in the indicators INSERT column list
    - B2 PUT is inside an `isB2Configured()` guard; a failed/absent B2 path does not abort persistence
    - No `fetch(` call targets a URL derived from parsed message content (only the Autotask attachment fetch + the B2 presigned PUT)
    - `npx tsc --noEmit` reports no errors for phishing-eml-service.ts
  </acceptance_criteria>
</task>

<task type="auto" tdd="true">
  <name>Task 2: phishing-eml-service.test.ts — orchestration coverage (mocked I/O)</name>
  <files>lib/services/phishing-eml-service.test.ts</files>
  <read_first>
    - lib/services/phishing-eml-service.ts (the module under test, from Task 1)
    - lib/services/phishing-detector.test.ts (mocking style for postgresClient + autotask factory in this repo)
    - lib/services/eml-parser.fixtures.ts (reuse the synthetic .eml buffers + attachment lists created in Plan 01)
    - .planning/phases/16-eml-mime-evidence-parser/16-RESEARCH.md (Pitfall 3 — tests must mock B2, never require real creds; SC#5 synthetic-only)
  </read_first>
  <behavior>
    - Happy path: mock getAutotaskClient to return a 3-tier fixture attachment list + getAttachmentContent returning a synthetic base64 `.eml`; mock postgresClient.query to capture inserts; assert exactly one messages insert and the expected indicators inserts, and that messages.headers payload contains the structured spf/dkim/dmarc verdicts
    - No-eml no-op: attachment list with no `.eml` → parseAndStoreMessage returns { stored: false } and postgresClient.query is never called with an INSERT INTO messages
    - B2 unconfigured: with isB2Configured()===false (env unset / mocked), the flow still parses and persists, raw_ref persisted as null, no presignUpload/PUT attempted
    - B2 configured: with isB2Configured()===true (mocked) and presignUpload/fetch mocked, raw_ref is set to `phishing/{reportId}/{attachmentId}.eml` and the PUT is issued once
    - No-network invariant: a global.fetch spy records no call to any URL contained in the fixture message body (only the mocked B2 PUT URL, if configured)
    - metadata JSONB: assert an attachment_hash indicator insert carries {filename,contentType,size,related} in its metadata param
  </behavior>
  <action>
    Create `lib/services/phishing-eml-service.test.ts` mirroring phishing-detector.test.ts's mocking approach (vi.mock the postgres-client and autotask-factory modules; vi.spyOn/stub isB2Configured + global.fetch as needed). Reuse the synthetic fixtures from eml-parser.fixtures.ts (import them) — do NOT introduce real email content. Cover every case in the behavior list. For the no-network assertion, spy on global.fetch and assert it is never called with any URL string that appears in the fixture body's extracted URLs (assert against the specific fixture URL, not just call count, since a configured-B2 test legitimately PUTs to the presigned B2 host). Use `describe('parseAndStoreMessage', ...)` with plain-language `it` titles matching the requirement wording.
  </action>
  <verify>
    <automated>cd /opt/stacks/pulse && npx vitest run lib/services/phishing-eml-service.test.ts && npx tsc --noEmit --pretty 2>&1 | (grep -i phishing-eml && exit 1 || echo tsc-ok)</automated>
  </verify>
  <done>phishing-eml-service.test.ts passes all cases: happy-path single messages row + indicators with metadata, no-eml no-op, B2-configured raw_ref set + single PUT, B2-unconfigured raw_ref null + no PUT, and the no-network-to-message-URLs invariant.</done>
  <acceptance_criteria>
    - `npx vitest run lib/services/phishing-eml-service.test.ts` exits 0
    - A test asserts no INSERT INTO messages occurs on the no-`.eml` path (stored:false)
    - A test asserts raw_ref === null when isB2Configured() is false, and no presignUpload/PUT happens
    - A test asserts raw_ref === `phishing/{reportId}/{attachmentId}.eml` and exactly one PUT when B2 is configured (mocked)
    - A test asserts global.fetch is never called with a message-body URL from the fixture
    - A test asserts the attachment_hash indicator's metadata JSONB carries filename/contentType/size/related
    - All fixtures are synthetic (imported from eml-parser.fixtures.ts) — no real customer email
  </acceptance_criteria>
</task>

</tasks>

<threat_model>
## Trust Boundaries

| Boundary | Description |
|----------|-------------|
| Autotask attachment content → Pulse server | Attacker-controlled `.eml` bytes fetched and processed server-side |
| Message body content → Pulse outbound network | URLs/HTML in the reported email must never be dereferenced by Pulse |
| Raw `.eml` bytes → storage | Potentially malware-laced raw email must not land somewhere browser-reachable |

## STRIDE Threat Register

| Threat ID | Category | Component | Disposition | Mitigation Plan |
|-----------|----------|-----------|-------------|-----------------|
| T-16-03 | Tampering / Info Disclosure (SSRF-adjacent) | orchestration around parseEml/extractUrls | mitigate | Hard invariant, test-enforced: the service never fetches any URL found in the message; the only outbound calls are the Autotask attachment GET and the B2 presigned PUT. A fetch spy asserts no call to any message-body URL |
| T-16-01 | Denial of Service | oversized fetched attachment | mitigate | Size guard against MAX_EML_BYTES on the decoded buffer BEFORE parseEml; degrade (log + skip) rather than crash |
| T-16-07 | Information Disclosure / EoP | raw `.eml` byte storage | mitigate | D-05 — raw bytes go to B2 (never Postgres, never local fs) under the path-traversal-safe EML_OBJECT_KEY_REGEX key; only the sanitized truncated preview lands in messages.body_preview |
| T-16-08 | Repudiation | parse/persist failures swallowed silently | accept | Top-level try/catch logs `[PHISHING-EML]` with ticket/report context and rethrows so the future Phase 18 caller records the failure; graceful B2 degrade is logged, not silent |
</threat_model>

<verification>
- `npx vitest run lib/services/phishing-eml-service.test.ts` — green
- `npx tsc --noEmit --pretty` — no new errors
- Grep confirms messages + indicators inserts, D-06 verdicts in headers payload, D-07 metadata in indicators, isB2Configured gate, and no fetch of message-body URLs
</verification>

<success_criteria>
- EVID-03: full pipeline persists normalized headers (incl. structured auth verdicts, D-06), URLs, and attachment metadata into a messages row + indicators rows
- EVID-04 / D-05: raw bytes stored in B2 (raw_ref = object key) when configured; sanitized truncated preview stored distinct from raw; nothing in the message is ever fetched/executed
- D-07: indicators.metadata JSONB carries per-indicator context
- Graceful degradation when B2 is unconfigured or no `.eml` attachment exists
</success_criteria>

<output>
Create `.planning/phases/16-eml-mime-evidence-parser/16-03-SUMMARY.md` when done
</output>
