# Phase 16: EML/MIME Evidence Parser - Context

**Gathered:** 2026-07-15
**Status:** Ready for planning

<domain>
## Phase Boundary

Given a ticket's attachments, Pulse selects the correct original reported message
(`rfc.eml` preferred over `OriginatingEmail.eml`) and parses its RFC822/MIME
structure into normalized, actionable evidence (headers, auth results, URLs,
attachment metadata, sanitized body preview) — without ever executing or
fetching anything from the message. Covers EVID-02, EVID-03, EVID-04.
Does NOT cover Mimecast (Phase 17), campaign grouping or the `/api/phishing/*`
API surface (Phase 18), classification (Phase 19), remediation (Phase 20), or
the Autotask note (Phase 21).

</domain>

<decisions>
## Implementation Decisions

### Raw .eml Storage
- **D-05:** Raw `.eml` bytes are stored in Backblaze B2, reusing the existing
  LogLift evidence-storage pattern (`lib/services/b2/client.ts`). The
  `messages.raw_ref` column (already in migration 097's stub schema) stores
  the B2 object key, not raw bytes or a bare hash. Rationale: keeps
  potentially-malicious raw email bytes out of Postgres, matches the
  established precedent for evidence blobs in this codebase, and gives a
  natural place to enforce the same size/path-traversal guards B2 client
  already has (`lib/services/b2/client.ts`'s object-key validation).

### Authentication-Results Depth
- **D-06:** SPF/DKIM/DMARC are parsed into structured verdicts (pass/fail/
  none/etc — not just raw header text) from the `Authentication-Results`
  header (and `Received-SPF` as fallback where present). This is deliberately
  more than "capture the raw header" because Phase 19's classifier
  (CLASSIFY-01..06) needs these as structured evidence to reason about
  spoofing/impersonation, not raw text it would have to re-parse itself.

### Indicators Schema
- **D-07:** Add a `metadata` JSONB column to the `indicators` table stub
  (migration 097) via a small Phase 16 migration (`ALTER TABLE indicators ADD
  COLUMN metadata JSONB`). This lets an attachment-hash indicator carry its
  filename/content-type/size alongside the hash, a URL indicator carry which
  message part (text/html) it came from, etc., without duplicating that
  detail into the parent `messages` row.

### Claude's Discretion (explicitly deferred to research + planner)
- **MIME parsing library or approach.** No precedent exists in this codebase
  (no `mailparser`/similar dependency, no hand-rolled parser). This is a
  genuine technical unknown — resolve via research before planning, not a
  user preference call.
- **How to actually fetch full `.eml` attachment content from Autotask.**
  `AutotaskClient.getAttachments()` (existing, from Phase 15) returns
  attachment *metadata* only — `AUTOTASK_API_GUIDE.md` documents attachment
  *upload* but not attachment *download*, and the `Attachment.data` field is
  typed optional with no confirmed behavior for whether the list call or a
  per-ID call actually populates base64 content. This needs to be confirmed
  empirically (real Autotask credentials exist in `.env`) or via Autotask API
  docs before planning locks in an approach.
- **URL extraction scope** (dedup strategy, normalization, which MIME parts to
  scan) — implementation detail, not user-relevant preference.
- Exact migration file number for the `indicators` ALTER (next available
  after 098 — confirm at plan time).

</decisions>

<canonical_refs>
## Canonical References

**Downstream agents MUST read these before planning or implementing.**

### Project conventions
- `CLAUDE.md` — migration numbering, no-new-dependency caution (npm deps are
  fine when a real gap exists — no ORM/server-actions/state-lib restriction
  applies to a MIME parser), auth conventions
- `AUTOTASK_API_GUIDE.md` — documents attachment *upload* only; attachment
  *download*/full-content-fetch mechanics are NOT documented here and must be
  confirmed via research or empirical testing against the real API
- `migrations/097_phishing_triage_schema.sql` — the `messages` and
  `indicators` stub tables this phase populates for real (`messages.headers`,
  `messages.urls`, `messages.attachments`, `messages.body_preview`,
  `messages.raw_ref`; `indicators.indicator_type` + `indicators.value`, plus
  the new `metadata` column per D-07)
- `INTEGRATIONS.md` — confirms `lib/services/b2/client.ts` is the existing B2
  evidence-storage client (LogLift precedent), with object-key format
  validation and a 25 MB max download already established

### Reference implementations for this phase
- `lib/services/b2/client.ts` — B2 upload/download pattern to reuse for raw
  `.eml` storage per D-05 (presigned URLs, object-key validation)
- `lib/services/phishing-detector.ts` (Phase 15) — `DetectableTicket`,
  `EvidencePayload` shapes and the `gatherTicketEvidence` function this
  phase's parser output plugs into (via the `reports` → `messages` link)
- `lib/services/autotask-client.ts` `getAttachments()` (~line 424) — existing
  attachment metadata fetch; this phase needs to extend or add to this for
  full content retrieval

</canonical_refs>

<code_context>
## Existing Code Insights

### Reusable Assets
- `lib/services/b2/client.ts` — presigned URL upload/download, object-key
  path-traversal guard, 25 MB max download — directly reusable for raw `.eml`
  storage per D-05
- `migrations/097_phishing_triage_schema.sql`'s `messages`/`indicators` stub
  tables already shaped for this phase's output (see canonical_refs)

### Established Patterns
- No existing MIME/RFC822 parsing code or dependency anywhere in this
  codebase — this phase introduces a genuinely new capability, unlike Phase
  15 which reused several existing patterns
- B2 is the established "large/sensitive blob evidence" storage location in
  this codebase (LogLift), not Postgres bytea and not local filesystem

### Integration Points
- This phase reads `reports.evidence` (Phase 15's attachment metadata) to
  find which ticket attachments exist, selects the right one (`rfc.eml` over
  `OriginatingEmail.eml`), fetches its full content, parses it, and writes a
  `messages` row (+ `indicators` rows) linked via `messages.report_id`

</code_context>

<specifics>
## Specific Ideas

The exact `.eml` selection logic is locked via REQUIREMENTS.md EVID-02:
prefer `rfc.eml` (case-insensitive name match) over `OriginatingEmail.eml`,
also matching by `message/rfc822` content-type when filename alone is
ambiguous.

</specifics>

<deferred>
## Deferred Ideas

None — discussion stayed within phase scope. MIME library choice and
Autotask attachment-download mechanics were explicitly routed to research
rather than deferred to a future phase (they're needed now, just not
user-decidable).

</deferred>

---

*Phase: 16-eml-mime-evidence-parser*
*Context gathered: 2026-07-15*
