---
phase: 16-eml-mime-evidence-parser
plan: 02
subsystem: api
tags: [autotask, backblaze-b2, postgres, vitest, tdd]

# Dependency graph
requires:
  - phase: 15-data-model-detection-ticket-evidence
    provides: indicators table (migration 097), phishing-detector core, EVID-01 ticket evidence capture
provides:
  - AutotaskClient.getAttachmentContent() — fetches full base64 `.eml` attachment content via the items[0] convention
  - EML_OBJECT_KEY_REGEX + parameterized presignDownload/presignUpload/downloadToBuffer for storing raw `.eml` bytes in B2 under a path-traversal-safe key shape
  - migrations/099_indicators_metadata.sql — indicators.metadata JSONB column, applied to dev DB
affects: [16-03-eml-mime-parser-orchestrator, 17-mimecast-blast-radius]

# Tech tracking
tech-stack:
  added: []
  patterns:
    - "AutotaskClient per-attachment-ID GET reads response.items?.[0] (list-shaped), NOT response.item"
    - "B2 client: parallel object-key regex per use-case, never loosen an existing regex; keyRegex is an optional trailing param defaulting to the original"

key-files:
  created:
    - lib/services/autotask-client.test.ts
    - migrations/099_indicators_metadata.sql
  modified:
    - lib/services/autotask-client.ts
    - lib/services/b2/client.ts
    - lib/services/b2/client.test.ts

key-decisions:
  - "Migration 099 applied to the dev DB via direct docker exec (not scripts/apply-migrations.sh), because that script hardcodes MIGRATIONS_DIR=/opt/stacks/pulse/migrations — the main repo's absolute path, not the worktree's — so it could not see the new file from inside this worktree checkout. Used the same real credentials the script itself defaults to (pulse_user/pulse_autotask), not hand-rolled wrong defaults."

patterns-established:
  - "First AutotaskClient unit test file (autotask-client.test.ts) — mocks global.fetch, asserts items vs item envelope shapes"

requirements-completed: [EVID-03, EVID-04]

# Metrics
duration: 25min
completed: 2026-07-15
---

# Phase 16 Plan 02: Autotask Attachment Content, B2 EML Key Regex, Indicators Metadata Column Summary

**AutotaskClient.getAttachmentContent() (items[0] convention), a parallel EML_OBJECT_KEY_REGEX + parameterized B2 key validation, and migration 099 (indicators.metadata JSONB) — the three independent supporting pieces the Plan 03 EML/MIME orchestrator depends on.**

## Performance

- **Duration:** ~25 min
- **Started:** 2026-07-15T14:17:00Z
- **Completed:** 2026-07-15T14:24:00Z
- **Tasks:** 3 completed
- **Files modified:** 5 (2 new, 3 modified)

## Accomplishments
- `AutotaskClient.getAttachmentContent(entityName, entityId, attachmentId)` fetches a single attachment's full base64 content, correctly reading the live `{items:[...]}` response shape (not `{item:...}`) — verified by a new unit test that also proves an `{item:...}`-shaped response yields `null`, guarding against a future refactor copying the wrong sibling convention.
- `EML_OBJECT_KEY_REGEX` added to `lib/services/b2/client.ts` alongside the existing (untouched) `OBJECT_KEY_REGEX`, enforcing `phishing/<reportId>/<attachmentId>.eml` and rejecting path traversal. `presignDownload`, `presignUpload`, and `downloadToBuffer` now accept an optional trailing `keyRegex` parameter (defaulting to `OBJECT_KEY_REGEX`), so the existing LogLift call site (`rmm/executor.ts:337`, `presignUpload(objectKey, 1800)`) compiles and behaves identically.
- `migrations/099_indicators_metadata.sql` adds a nullable `indicators.metadata JSONB` column (D-07), applied live to the dev DB — confirmed via `information_schema.columns`.

## Task Commits

Each task was committed atomically (TDD tasks have separate test → feat commits):

1. **Task 1: AutotaskClient.getAttachmentContent()**
   - `8be10db` (test) — failing test for items[0] convention + item-shaped-yields-null guard
   - `9b65de7` (feat) — implementation
2. **Task 2: EML_OBJECT_KEY_REGEX + parameterized B2 key validation**
   - `6de92a5` (test) — failing tests for new regex + keyRegex param
   - `8630fd5` (feat) — implementation
3. **Task 3: Migration 099 — indicators.metadata JSONB**
   - `0bf37ce` (feat) — migration file + applied to dev DB

_RED confirmed for both TDD tasks by temporarily removing the implementation and re-running the test suite before restoring it and committing test-then-feat._

## Files Created/Modified
- `lib/services/autotask-client.ts` - added `getAttachmentContent()` after `getAttachments()`
- `lib/services/autotask-client.test.ts` - new; first AutotaskClient unit coverage (3 tests)
- `lib/services/b2/client.ts` - added `EML_OBJECT_KEY_REGEX`; parameterized `keyRegex` on presign*/downloadToBuffer
- `lib/services/b2/client.test.ts` - extended with EML_OBJECT_KEY_REGEX + custom-keyRegex presignUpload tests (5 new tests)
- `migrations/099_indicators_metadata.sql` - new; nullable `indicators.metadata JSONB` + COMMENT

## Decisions Made
- Applied migration 099 to the dev DB via direct `docker exec pulse-postgres psql -U pulse_user -d pulse_autotask` rather than `scripts/apply-migrations.sh`, because that script's `MIGRATIONS_DIR` is hardcoded to the main repo's absolute path and cannot see files that only exist in this worktree checkout. Used the same real project credentials (`pulse_user`/`pulse_autotask`) the script itself would use — not the script's local-non-Docker fallback defaults.

## Deviations from Plan

### Auto-fixed Issues

**1. [Rule 3 - Blocking] scripts/apply-migrations.sh could not see the new migration file from inside the worktree**
- **Found during:** Task 3 (migration 099 apply step)
- **Issue:** The script hardcodes `MIGRATIONS_DIR="/opt/stacks/pulse/migrations"` — the main repo's path, not this worktree's `.claude/worktrees/agent-a2b7d8cafac0f4270/migrations/`. Running it reported "Migration file 099_indicators_metadata.sql not found" even though the file existed in the worktree.
- **Fix:** Applied the migration directly with `docker exec -i pulse-postgres psql -U pulse_user -d pulse_autotask < migrations/099_indicators_metadata.sql`, using the identical real credentials the script defaults to for Docker mode (not the script's local/non-Docker psql fallback defaults, which are wrong for this project).
- **Files modified:** none (operational step only)
- **Verification:** `information_schema.columns` confirms `indicators.metadata` is `jsonb` on the dev DB.
- **Committed in:** 0bf37ce (Task 3 commit, migration file only — no code changed by this deviation)

---

**Total deviations:** 1 auto-fixed (1 blocking)
**Impact on plan:** No scope creep — same migration content and same live-DB outcome the plan specified; only the apply mechanism differed due to a pre-existing worktree/script path mismatch.

## Issues Encountered
None beyond the deviation above.

## User Setup Required
None - no external service configuration required.

## Next Phase Readiness
- `getAttachmentContent()`, `EML_OBJECT_KEY_REGEX`, and `indicators.metadata` are all available for Plan 03's EML/MIME parser orchestrator to consume.
- `rmm/executor.ts`'s existing `presignUpload(objectKey, 1800)` call site still type-checks unchanged (verified via `npx tsc --noEmit`).
- No blockers.

## Self-Check: PASSED

All created/modified files verified present on disk; all task and metadata commit hashes verified present in git log.

---
*Phase: 16-eml-mime-evidence-parser*
*Completed: 2026-07-15*
