---
phase: 16-eml-mime-evidence-parser
plan: 03
subsystem: api
tags: [autotask, backblaze-b2, postgres, mailparser, phishing, vitest]

# Dependency graph
requires:
  - phase: 16-eml-mime-evidence-parser (Plan 01)
    provides: "lib/services/eml-parser.ts — selectOriginalMessage, parseEml, MAX_EML_BYTES, NormalizedMessage"
  - phase: 16-eml-mime-evidence-parser (Plan 02)
    provides: "AutotaskClient.getAttachmentContent, B2 EML_OBJECT_KEY_REGEX + parameterized presignUpload, migrations/099 indicators.metadata"
provides:
  - "lib/services/phishing-eml-service.ts — parseAndStoreMessage(reportId, ticketId) orchestration: list -> select -> fetch -> (B2 gated) -> parse -> persist messages+indicators"
affects: [18-campaign-grouping-api, 19-classification]

# Tech tracking
tech-stack:
  added: []
  patterns:
    - "Orchestration service (no class, single exported async function) mirroring phishing-detector.ts's fetch->transform->persist shape"
    - "Graceful-degrade try/catch around the B2 self-PUT step, gated behind isB2Configured(), so an unconfigured/failed B2 upload never aborts parsing/persistence"
    - "Size guard on the decoded buffer BEFORE calling parseEml, returning a no-op result rather than letting parseEml's internal MAX_EML_BYTES guard throw"

key-files:
  created:
    - lib/services/phishing-eml-service.ts
    - lib/services/phishing-eml-service.test.ts
  modified: []

key-decisions:
  - "Task ordering followed the plan literally: Task 1 built the implementation file, Task 2 built the full test suite against it (not a strict RED-then-GREEN cycle within a single task) — both tasks were tagged tdd=\"true\" in the plan but structured as separate files/commits rather than interleaved test-then-feat commits within one task, matching how the plan's own read_first/action/verify blocks were written per task"
  - "indicators inserts are plain INSERT (not upsert) — migration 097's indicators table has no natural-key unique constraint, and each parseAndStoreMessage call creates a fresh messages row, so there is nothing to conflict against"
  - "reason codes ('no-eml-attachment', 'no-attachment-content', 'oversized-attachment') added to the { stored: false } result beyond the plan's single named example, to make the three distinct no-op paths distinguishable to a future caller (Phase 18) without inspecting logs"

patterns-established:
  - "Self-PUT to B2 pattern (first instance of Pulse's own server code PUTting bytes to B2 itself, vs. handing a presigned URL to an external collector) — presignUpload(objectKey, 1800, undefined, EML_OBJECT_KEY_REGEX) then fetch(url, { method: 'PUT', body })"

requirements-completed: [EVID-03, EVID-04]

# Metrics
duration: ~22min
completed: 2026-07-15
---

# Phase 16 Plan 03: EML/MIME Evidence Orchestration Service Summary

**`parseAndStoreMessage(reportId, ticketId)` orchestrates the full evidence pipeline — list Autotask attachments, select the original reported message, fetch its content, size-guard it, self-PUT the raw bytes to B2 when configured, parse it, and persist one `messages` row plus per-indicator `indicators` rows (attachment_hash/url/sender) with D-07 metadata — never fetching anything found in the message.**

## Performance

- **Duration:** ~22 min
- **Started:** 2026-07-15T14:18:00Z
- **Completed:** 2026-07-15T14:40:23Z
- **Tasks:** 2 completed
- **Files modified:** 2 (both new)

## Accomplishments
- `lib/services/phishing-eml-service.ts` exports `parseAndStoreMessage({ reportId, ticketId })`, wiring `getAutotaskClient().getAttachments` → Plan 01's `selectOriginalMessage` → `getAttachmentContent` → a `MAX_EML_BYTES` size guard → an `isB2Configured()`-gated B2 self-PUT (`phishing/{reportId}/{attachmentId}.eml`) → Plan 01's `parseEml` → one `messages` INSERT (headers incl. D-06 structured SPF/DKIM/DMARC verdicts, urls, attachments, body_preview, raw_ref) → per-attachment-hash/per-URL/sender `indicators` INSERTs carrying D-07 `metadata` JSONB.
- Returns `{ stored: false, reason }` without throwing and without writing any row when there is no `.eml` attachment, no attachment content, or an oversized decoded buffer; a failed or absent B2 PUT degrades to `raw_ref: null` rather than aborting persistence.
- 6/6 new vitest tests pass, covering the happy path (one messages row + D-06 verdicts reaching `headers`, indicators written), the no-`.eml` no-op, B2-unconfigured (`raw_ref` null, no PUT), B2-configured (`raw_ref` set, exactly one PUT), the no-network-to-message-body-URL invariant, and the attachment-hash indicator's `metadata` payload.
- `npx tsc --noEmit --pretty` is fully clean (zero errors anywhere in the repo, not just this file).

## Task Commits

1. **Task 1: phishing-eml-service.ts orchestration (list→select→fetch→B2→parse→persist)** - `5088d8d` (feat)
2. **Task 2: phishing-eml-service.test.ts — orchestration coverage (mocked I/O)** - `ce16e67` (test)

_Note: both tasks were tagged `tdd="true"` in the plan, but the plan itself structured them as implementation-first (Task 1) then full test coverage (Task 2) rather than an interleaved RED/GREEN pair within a single task — followed literally as written._

## Files Created/Modified
- `lib/services/phishing-eml-service.ts` (224 lines) - `parseAndStoreMessage`, `ParseAndStoreInput`/`ParseAndStoreResult` types
- `lib/services/phishing-eml-service.test.ts` (195 lines) - 6 tests: happy path, no-eml no-op, B2-unconfigured, B2-configured, no-network invariant, indicator metadata

## Decisions Made
- **Plain `INSERT` for `indicators`, not upsert.** Migration 097's `indicators` table has no natural-key unique constraint (unlike `reports.uq_reports_ticket_id`), and each `parseAndStoreMessage` call always creates a fresh `messages` row first — there's nothing to conflict against, so `ON CONFLICT` would be dead code.
- **Three distinct no-op `reason` codes** (`no-eml-attachment`, `no-attachment-content`, `oversized-attachment`) instead of one generic no-op shape — makes the three degrade paths distinguishable to Phase 18's future caller without grepping logs, while still satisfying the plan's single named example (`no-eml-attachment`).
- **Task 1/Task 2 split followed literally**, not force-fit into a same-task RED→GREEN commit pair, since the plan's own per-task `read_first`/`action`/`verify` blocks were already split that way (Task 1's verify is `tsc` only; Task 2's verify is `vitest` + `tsc`).

## Deviations from Plan

None - plan executed exactly as written. Both tasks' acceptance criteria (grep checks for `selectOriginalMessage`/`parseEml`/`getAttachmentContent`/`isB2Configured`/`EML_OBJECT_KEY_REGEX`, `INSERT INTO messages`/`INSERT INTO indicators` each with `RETURNING id::text`, `authResults` reaching the headers payload, `metadata` in the indicators INSERT column list, B2 PUT gated behind `isB2Configured()`, no stray `fetch(` targeting message-derived content) were verified via grep and vitest before each commit and all passed on the first attempt.

## Issues Encountered
None.

## User Setup Required
None - no external service configuration required. B2 and Autotask credentials are only exercised via mocks in this plan's tests; live credentials are Phase 18's concern when the on-demand trigger route is built.

## Next Phase Readiness
- `parseAndStoreMessage` is a stable, fully-tested export ready for Phase 18's `POST /api/phishing/tickets/{id}/analyze` route to call directly.
- No blockers identified. `messages`/`indicators` schema (migrations 097 + 099) is fully exercised by this service's insert shape.

## Self-Check: PASSED

- `lib/services/phishing-eml-service.ts` — FOUND
- `lib/services/phishing-eml-service.test.ts` — FOUND
- Commit `5088d8d` — FOUND in `git log`
- Commit `ce16e67` — FOUND in `git log`

---
*Phase: 16-eml-mime-evidence-parser*
*Completed: 2026-07-15*
