---
phase: 16-eml-mime-evidence-parser
plan: 01
subsystem: api
tags: [mailparser, linkify-it, mime, rfc822, spf, dkim, dmarc, phishing, vitest]

# Dependency graph
requires: []
provides:
  - "lib/services/eml-parser.ts — selectOriginalMessage, parseEml, parseAuthResults, extractUrls, buildBodyPreview, NormalizedMessage type"
  - "mailparser + linkify-it npm dependencies"
affects: [16-02, 16-03, 18-campaign-grouping]

# Tech tracking
tech-stack:
  added: ["mailparser@3.9.14", "linkify-it@6.0.0", "@types/mailparser@3.4.6 (devDependency)"]
  patterns:
    - "Pure I/O-free transform module (no class, named exports, top-of-file doc-comment stating a hard no-network invariant) — mirrors lib/services/b2/client.ts's module shape"
    - "Hand-rolled RFC 8601 Authentication-Results tokenizer instead of a DNS/HTTP-verifying library (mailauth rejected)"
    - "describe-per-export vitest structure with synthetic-fixture-only test data"

key-files:
  created:
    - lib/services/eml-parser.ts
    - lib/services/eml-parser.fixtures.ts
    - lib/services/eml-parser.test.ts
    - .planning/phases/16-eml-mime-evidence-parser/deferred-items.md
  modified:
    - package.json
    - package-lock.json

key-decisions:
  - "Used a small hand-rolled HTML-to-text stripper for buildBodyPreview's fallback path instead of importing mailparser's undeclared transitive dependency html-to-text directly (avoids depending on an unversioned, un-pinned package.json entry that could silently disappear on a future mailparser bump)"
  - "linkify-it constructed with { fuzzyLink: true } to catch scheme-less www.-prefixed phishing URLs (off by default in the library)"
  - "Authentication-Results / Authentication-Results-Original collected via mail.headerLines (first occurrence each), not the headers Map, since the Map only exposes the last/one occurrence of a repeated header (Pitfall 4)"

patterns-established:
  - "Pattern: Buffer-in, structured-object-out parsing module with a byte-size guard enforced BEFORE handing untrusted content to a third-party parser (DoS mitigation, T-16-01)"

requirements-completed: [EVID-02, EVID-03, EVID-04]

# Metrics
duration: 12min
completed: 2026-07-15
---

# Phase 16 Plan 01: EML/MIME Evidence Parser Core Summary

**Pure, I/O-free `lib/services/eml-parser.ts` built on `mailparser` + `linkify-it`, implementing three-tier `.eml` attachment selection, RFC822/MIME normalization with hand-rolled structured SPF/DKIM/DMARC verdicts, deduped URL extraction, and a sanitized truncated body preview — all test-enforced to never trigger a network call.**

## Performance

- **Duration:** ~12 min
- **Started:** 2026-07-15T14:19:00Z (worktree setup + npm install)
- **Completed:** 2026-07-15T14:30:18Z
- **Tasks:** 3 completed (1 auto, 2 TDD)
- **Files modified:** 5 (2 dependency files + 3 new source files) plus 1 new deferred-items tracking doc

## Accomplishments
- Installed `mailparser` 3.9.14 + `linkify-it` 6.0.0 (+ `@types/mailparser` devDependency) with a reviewed `git diff package.json` gate confirming no unrelated dependency churn
- Implemented `selectOriginalMessage` — the three-tier `.eml` attachment selection algorithm (exact `rfc.eml` → single non-`OriginatingEmail.eml` `message/rfc822` candidate → `OriginatingEmail.eml` fallback → null), validated against all real-world shapes found in Phase 16 research (Microsoft Report Message, KnowBe4 PhishER, single-attachment legacy tickets, and both ambiguous edge cases)
- Implemented `parseEml` — normalizes From/Reply-To/Return-Path/To/Cc/Subject/Date/Message-ID/Received-chain, structured `authResults`/`authResultsOriginal` verdicts, deduped URLs, and per-attachment metadata (filename/content-type/size/sha256 checksum/`related` flag for inline-CID parts), guarded by `MAX_EML_BYTES` (10 MB, below B2's 25 MB cap) enforced before `simpleParser` is ever called
- Implemented `parseAuthResults` (hand-rolled RFC 8601 tokenizer), `extractUrls` (linkify-it with fuzzy `www.` matching), and `buildBodyPreview` (truncated to 500 chars, HTML-stripped fallback)
- 26/26 vitest tests pass, `tsc --noEmit` clean for all eml-parser files, and a `global.fetch` spy confirms zero network calls across every synthetic fixture (EVID-04 hard invariant)

## Task Commits

1. **Task 1: Install mailparser + linkify-it (deliberate, reviewed)** - `0f4dc1f` (chore)
2. **Task 2: Attachment selection (EVID-02, three tiers) — RED** - `2fde115` (test)
2. **Task 2: Attachment selection (EVID-02, three tiers) — GREEN** - `e4718ae` (feat)
3. **Task 3: parseEml + auth-results + URLs + body preview + size guard — RED** - `4df4816` (test)
3. **Task 3: parseEml + auth-results + URLs + body preview + size guard — GREEN** - `654e624` (feat)

_Note: this is a `type: tdd` plan — Tasks 2 and 3 each have a RED (test) commit followed by a GREEN (feat) commit, no refactor commit was needed._

## Files Created/Modified
- `lib/services/eml-parser.ts` (282 lines) - `selectOriginalMessage`, `parseEml`, `parseAuthResults`, `extractUrls`, `buildBodyPreview`, `MAX_EML_BYTES`, `NormalizedMessage`/`AuthResults`/`AttachmentMeta`/`AuthVerdict` types
- `lib/services/eml-parser.fixtures.ts` (227 lines) - synthetic attachment-list fixtures (3 selection tiers + ambiguous/no-eml/empty edge cases) and synthetic raw `.eml` buffers (rich multipart, auth-results-original, inline/CID attachment, long-body, fuzzy-URL, oversized-buffer generator)
- `lib/services/eml-parser.test.ts` (219 lines) - 26 tests across `describe('selectOriginalMessage')`, `describe('parseAuthResults')`, `describe('extractUrls')`, `describe('buildBodyPreview')`, `describe('parseEml')`
- `package.json` / `package-lock.json` - added `mailparser`, `linkify-it`, `@types/mailparser`
- `.planning/phases/16-eml-mime-evidence-parser/deferred-items.md` - logs 2 pre-existing, unrelated `itglue-search.test.ts` failures found via `npm test` (out of scope, not fixed)

## Decisions Made
- **HTML-to-text fallback implemented by hand, not via `html-to-text`.** RESEARCH.md/the plan suggested using `html-to-text` (already a `mailparser` transitive dependency) for `buildBodyPreview`'s HTML fallback. It ships no bundled TypeScript types and isn't a declared direct dependency — importing it would depend on an un-pinned, unversioned transitive package that could silently disappear on a future `mailparser` version bump. Wrote a small (10-line) regex-based tag/entity stripper instead, fully covered by tests. This was within Claude's explicit discretion per CONTEXT.md ("URL extraction scope... implementation detail, not user-relevant preference").
- **`linkify-it` constructed with `{ fuzzyLink: true }`.** Verified empirically that linkify-it's default options do NOT enable scheme-less `www.`-prefixed URL matching (`fuzzyLink` defaults to `false`) — without this option, a phishing URL like `www.evil-example.com/login` (no `http://` prefix) would silently not be extracted, undermining EVID-03's URL-extraction requirement.
- **Authentication-Results / Authentication-Results-Original parsed from `mail.headerLines`, not `mail.headers.get(...)`.** Confirmed via research and this session's own live testing that `mail.headers` (a Map) collapses repeated header occurrences; `headerLines` (an ordered array) is the only way to reliably locate both headers when present (Pitfall 4).

## Deviations from Plan

### Auto-fixed Issues

**1. [Rule 1 - Test discoverability] Renamed a test title to match the plan's required `-t` filter**
- **Found during:** Task 3 verification
- **Issue:** The plan's acceptance criteria require `npx vitest run lib/services/eml-parser.test.ts -t "no network"` to pass. The initial test title ("never makes a network call while parsing any fixture") did not contain the literal substring "no network", so the `-t` filter matched zero tests.
- **Fix:** Renamed the test to "makes no network call (no network fetch) while parsing any fixture".
- **Files modified:** lib/services/eml-parser.test.ts
- **Verification:** `npx vitest run lib/services/eml-parser.test.ts -t "no network"` now passes (1 test).
- **Committed in:** `654e624` (part of Task 3 GREEN commit)

---

**Total deviations:** 1 auto-fixed (test-naming correctness, no behavior change)
**Impact on plan:** Cosmetic only — no scope creep, no behavior change. All three EVID-02/03/04 requirements implemented exactly as specified.

## Issues Encountered
- This worktree had no `node_modules` at all on start (fresh git worktree checkout) — ran a full `npm install` from the existing `package-lock.json` before the plan's own Task 1 dependency install, so `mailparser`/`linkify-it` could resolve correctly. Not a plan deviation; this is normal worktree bootstrap, not tracked as a commit.
- `npm test` (full suite) surfaced 2 pre-existing failures in `lib/services/analyzer/itglue-search.test.ts`, unrelated to any file this plan touched (last commit on that file predates this plan). Logged to `deferred-items.md`, not fixed, per the executor's scope-boundary rule.

## User Setup Required

None - no external service configuration required. `mailparser`/`linkify-it` need no credentials; B2 storage (D-05) and the Autotask attachment-content fetch are Plan 16-03's concern, not this plan's.

## Next Phase Readiness

`lib/services/eml-parser.ts`'s exported contract (`selectOriginalMessage`, `parseEml`, `NormalizedMessage`, `AuthResults`, `AttachmentMeta`, `MAX_EML_BYTES`) is stable and ready for Plan 16-03's orchestration layer (fetch attachment content from Autotask → upload to B2 → `parseEml` → persist `messages`/`indicators` rows). No blockers identified for 16-02 or 16-03.

---
*Phase: 16-eml-mime-evidence-parser*
*Completed: 2026-07-15*

## Self-Check: PASSED
