docs(19): research classification engine phase domain
This commit is contained in:
parent
b09b00a08a
commit
e1c7eff088
1 changed files with 870 additions and 0 deletions
870
.planning/phases/19-classification-engine/19-RESEARCH.md
Normal file
870
.planning/phases/19-classification-engine/19-RESEARCH.md
Normal file
|
|
@ -0,0 +1,870 @@
|
|||
# Phase 19: Classification Engine - Research
|
||||
|
||||
**Researched:** 2026-07-16
|
||||
**Domain:** Deterministic rule-based security classification over structured phishing-triage evidence (Postgres-resident, no LLM)
|
||||
**Confidence:** HIGH (architecture/schema/permissions — all read directly from shipped code) / MEDIUM (confidence-weight numbers, action-recommendation mapping — reasoned recommendations, not locked decisions) / MEDIUM (BSN/KnowBe4 domain findings — grounded in real ticket data, but domain lists are inherently incomplete/rotating)
|
||||
|
||||
<user_constraints>
|
||||
## User Constraints (from CONTEXT.md)
|
||||
|
||||
### Locked Decisions
|
||||
|
||||
**Architecture: Deterministic Rules Only**
|
||||
- **D-01:** The classifier is a pure deterministic rule engine — no Anthropic/OpenRouter/LLM
|
||||
calls in this phase. Same style as `lib/services/robotic-classifier.ts` (evidence-in →
|
||||
rule-eval → verdict-out). This matches the ROADMAP goal and success criteria literally —
|
||||
none of them describe an AI call. CLASSIFY-06's "before reaching any AI layer" language is
|
||||
satisfied by shaping the evidence payload as bounded/structured now (see D-08), so a future
|
||||
phase could bolt on an LLM-assist stage later without reworking the evidence shape — but this
|
||||
phase does not build one.
|
||||
|
||||
**Trigger: On-Demand Only**
|
||||
- **D-02:** Classification runs strictly via `POST /api/phishing/campaigns/{id}/classify` — no
|
||||
automatic classification on campaign creation/update. Each classify call inserts a new
|
||||
`classifications` row (append-only history) — "current" verdict is the most recent row by
|
||||
`created_at`.
|
||||
|
||||
**Verdict Rules: THREAT Tier**
|
||||
- **D-03:** THREAT requires BOTH (a) evidence the message actually reached someone — blast-radius
|
||||
shows `delivered > 0` or `clicked > 0` via `getBlastRadius()` (Phase 17) — AND (b) a malicious
|
||||
signal: a hard SPF/DKIM/DMARC **fail** (not `none`/unchecked), or a known-bad indicator
|
||||
(attachment-hash/URL match from the `indicators` table). Either signal alone stays at
|
||||
UNWANTED — contained blast radius means it isn't a realized threat yet.
|
||||
|
||||
**Verdict Rules: SPAM vs. UNWANTED**
|
||||
- **D-04:** SPAM = generic bulk/commercial signals — no spoofing, no malicious indicator matches,
|
||||
sender not impersonating anyone. UNWANTED = has a suspicious signal but doesn't clear the
|
||||
THREAT bar — the "legitimately suspicious but not spreading or spoofed" middle tier.
|
||||
|
||||
**Confidence Scoring**
|
||||
- **D-05:** Point-deduction from a full-evidence baseline of `1.0`. Subtract a fixed amount per
|
||||
missing evidence source: no `.eml`/`messages` row parsed, no Mimecast data (`status:
|
||||
'unavailable'`), no attachment/URL indicators found. Each deduction is named explicitly in
|
||||
`reasons`. Exact deduction weights are a planner/researcher call — not a user preference, as
|
||||
long as the additive-from-1.0 shape and per-source naming are preserved.
|
||||
|
||||
**KnowBe4 / Breach Secure Now Simulation Detection**
|
||||
- **D-06:** Detection is a sender-domain allowlist, config-driven (not hardcoded inline), checked
|
||||
against the parsed original message's `From`/`Return-Path` domain (Phase 16's
|
||||
`NormalizedMessage`) — NOT against the report-pattern match alone. Phase 15's detector patterns
|
||||
(`KNOWN_PHISHING_PATTERNS`) only prove the REPORT arrived via a KnowBe4-button-flavored flow;
|
||||
they do not prove the underlying reported message is itself a simulation. If the allowlist
|
||||
matches, force verdict to SPAM/UNWANTED and skip THREAT regardless of other signals (D-03/D-04
|
||||
still decide which of the two).
|
||||
- **D-07 (added mid-discussion):** Research must search existing Autotask ticket history for real
|
||||
BSN-originated report examples and fold a BSN sender-domain entry into this phase's allowlist if
|
||||
enough signal exists. The allowlist structure (D-06) must support multiple vendors from day one.
|
||||
|
||||
**Recommended Actions Vocabulary**
|
||||
- **D-08:** Seven action types, split into two tiers:
|
||||
- **Non-destructive:** `no_action` (SPAM), `warn_user` (UNWANTED), `disable_forwarding_rule`
|
||||
(easily reversible — does NOT force approval on its own).
|
||||
- **Destructive** (always `requires_approval: true`): `block_sender`, `purge_message`,
|
||||
`reset_password`, `isolate_endpoint`.
|
||||
This vocabulary becomes Phase 20's `remediation_actions.action_type` enum. The flag is OR'd
|
||||
across all recommended actions, not per-action — recommending `disable_forwarding_rule` alone
|
||||
must NOT force `requires_approval: true`, while recommending it alongside any destructive
|
||||
action still does.
|
||||
|
||||
### Claude's Discretion
|
||||
|
||||
- Exact point-deduction weights for confidence scoring (D-05) — as long as the shape holds.
|
||||
- Exact JSON shape of `reasons` / `recommended_actions` in the API response and `classifications`
|
||||
row — follow camelCase API convention; both are JSONB columns in migration 097.
|
||||
- Whether the KnowBe4/BSN sender-domain allowlist lives as a TypeScript constant (mirroring
|
||||
`KNOWN_PHISHING_PATTERNS`) or a small config/DB table — must support multiple vendors and be
|
||||
trivially extendable.
|
||||
- Exact evaluation order/precedence when multiple rules could apply — the allowlist match (D-06)
|
||||
short-circuits before D-03/D-04 tier evaluation; finer ordering is planner's call.
|
||||
- How `isolate_endpoint`'s "configured provider path" question interacts with Phase 20 — Phase 19
|
||||
only needs to recommend it and mark it destructive.
|
||||
|
||||
### Deferred Ideas (OUT OF SCOPE)
|
||||
|
||||
- **BSN sender-domain confirmation** — if research finds no usable BSN pattern in existing ticket
|
||||
history, ship this phase with a KnowBe4-only allowlist and treat adding BSN as a fast follow-up.
|
||||
(RESOLVED BY THIS RESEARCH — see below: a real BSN pattern WAS found.)
|
||||
- **`isolate_endpoint` provider wiring** — Phase 19 only recommends/flags destructive; Phase 20
|
||||
owns execution.
|
||||
- **Non-BSN/KnowBe4 discussion scope creep** — none surfaced.
|
||||
</user_constraints>
|
||||
|
||||
<phase_requirements>
|
||||
## Phase Requirements
|
||||
|
||||
| ID | Description | Research Support |
|
||||
|----|-------------|------------------|
|
||||
| CLASSIFY-01 | Classify a campaign as exactly one of SPAM/UNWANTED/THREAT with confidence, summary, evidence-backed reasons, recommended actions, `requires_approval` | See "Verdict Decision Tree" and "Code Examples" — concrete rule-evaluation order and `classifications` row shape confirmed against migration 097 |
|
||||
| CLASSIFY-02 | Any classification recommending a destructive action always sets `requires_approval: true` | See "Recommended Actions Mapping" — `DESTRUCTIVE_ACTIONS` set + OR-across-actions computation, with `disable_forwarding_rule`-alone counter-case explicitly covered |
|
||||
| CLASSIFY-03 | Incomplete evidence lowers confidence and names the missing evidence in reasons | See "Confidence Scoring Weights" — concrete 0.4/0.3/0.2 deduction proposal with rationale, floors naturally at 0.1 |
|
||||
| CLASSIFY-04 | Known KnowBe4 simulations not classified THREAT absent contrary evidence | See "D-07 Findings: BSN Ticket Evidence" — confirmed real `it-support.care` (KnowBe4) and `breachsecurenow.com` (BSN) sender-domain patterns from live Autotask ticket history |
|
||||
| CLASSIFY-05 | Operator can (re-)trigger classification via `POST /api/phishing/campaigns/{id}/classify` | See "Auth/Route Pattern" — exact `requirePermission('phishing', 'analyze')` call-and-return shape confirmed from `analyze/route.ts` |
|
||||
| CLASSIFY-06 | Classifier accepts structured, size-bounded evidence, not raw unbounded email | See "Evidence Assembly" — Phase 16 already truncates body previews to 500 chars; recommends capping indicator/report arrays in the persisted evidence too |
|
||||
</phase_requirements>
|
||||
|
||||
## Summary
|
||||
|
||||
This phase adds one new pure-function module (`lib/services/campaign-classifier.ts`, following
|
||||
the naming convention of `campaign-grouping-service.ts` and `mimecast-blast-radius.ts`) and one
|
||||
new route (`POST /api/phishing/campaigns/{id}/classify`), following the exact
|
||||
`requirePermission('phishing', 'analyze')` pattern already used by
|
||||
`app/api/phishing/tickets/[ticket_id]/analyze/route.ts`. No new npm packages are required — the
|
||||
classifier consumes types and functions already exported by Phases 16-18
|
||||
(`NormalizedMessage`/`AuthResults` from `eml-parser.ts`, `BlastRadiusResult` from
|
||||
`mimecast-blast-radius.ts`, the `campaigns`/`reports`/`messages`/`indicators` schema from
|
||||
migration 097) and writes one row per call to the already-existing `classifications` table.
|
||||
|
||||
The highest-value finding from this research is empirical, not architectural: **live Autotask
|
||||
ticket history confirms a real, historically consistent KnowBe4 phishing-simulation sending
|
||||
domain (`it-support.care`, 219 tickets across 3+ years, ~37 distinct spoofed personas sharing one
|
||||
domain) and a real Breach Secure Now sending domain (`breachsecurenow.com`, 11+ tickets)** — see
|
||||
the dedicated D-07 section below. A second, equally load-bearing finding is a genuine pitfall
|
||||
discovered in the same ticket data: **forwarded/reported simulation emails routinely show a
|
||||
`fail` verdict on the primary `Authentication-Results` header while `Authentication-Results-Original`
|
||||
(pre-forwarding) shows `pass`** — a classifier that checks only `authResults` (not
|
||||
`authResultsOriginal`) risks misclassifying a genuine KnowBe4/BSN simulation as THREAT purely
|
||||
because Outlook's "Report Message" forwarding step invalidates the original DKIM signature. Phase
|
||||
16 anticipated exactly this in its own Open-Questions log ("Phase 19's classifier will care about
|
||||
pre-remediation verdicts") — this research confirms the concern is real, using real ticket data,
|
||||
not just theoretical.
|
||||
|
||||
**Primary recommendation:** Build `lib/services/campaign-classifier.ts` as a single exported
|
||||
`classifyCampaign(campaignId): Promise<ClassificationResult>` function (pure evidence-gathering +
|
||||
rule-evaluation, no class, mirroring `campaign-grouping-service.ts`), preferring
|
||||
`authResultsOriginal` over `authResults` for the D-03 hard-fail check, using a TypeScript-constant
|
||||
sender-domain allowlist (`KNOWN_SIMULATION_SENDERS`) seeded with `it-support.care` (KnowBe4) and
|
||||
`breachsecurenow.com` (BSN), and computing "known-bad indicator match" as **an attachment-hash or
|
||||
URL value shared across ≥2 distinct reports in the same campaign** (there is no external
|
||||
threat-intel/reputation source anywhere in this codebase's dependency chain — this is the only
|
||||
structurally available "match" signal).
|
||||
|
||||
## Architectural Responsibility Map
|
||||
|
||||
| Capability | Primary Tier | Secondary Tier | Rationale |
|
||||
|------------|-------------|----------------|-----------|
|
||||
| Evidence assembly (campaign → reports → messages → indicators + blast-radius call) | API/Backend (service layer) | Database | Pure data-gathering, no UI; lives in `lib/services/`, reads Postgres + calls Phase 17's abstraction |
|
||||
| Verdict rule evaluation (D-03/D-04/D-06) | API/Backend (service layer) | — | Deterministic in-process logic, no I/O beyond the evidence already gathered |
|
||||
| Confidence scoring (D-05) | API/Backend (service layer) | — | Pure arithmetic over the assembled evidence's completeness flags |
|
||||
| `classifications` row persistence | Database | API/Backend | Single INSERT, append-only, no update/upsert |
|
||||
| `POST /classify` trigger + auth | API/Backend (route handler) | — | `requirePermission('phishing','analyze')` early-return, matches Phase 18 convention |
|
||||
| KnowBe4/BSN sender-domain allowlist | API/Backend (TS constant) | — | Mirrors `KNOWN_PHISHING_PATTERNS` precedent; no DB/UI needed this phase |
|
||||
|
||||
No browser/client-tier or CDN/static-tier work exists in this phase — `UI hint: no` in ROADMAP.md
|
||||
is consistent with the codebase reality (no `/admin/phishing` UI exists yet in any of Phases
|
||||
15-18).
|
||||
|
||||
## Standard Stack
|
||||
|
||||
### Core
|
||||
|
||||
No new packages. This phase is 100% composed of existing, already-installed dependencies:
|
||||
|
||||
| Library | Version | Purpose | Why Standard |
|
||||
|---------|---------|---------|--------------|
|
||||
| `pg` (via `postgresClient`) | 8.11.0 (existing) | Read campaign/report/message/indicator rows, INSERT `classifications` row | Already the project's only DB access path — no ORM |
|
||||
| (none — mailparser/linkify-it already consumed upstream by Phase 16, not directly by this phase) | — | — | — |
|
||||
|
||||
**Version verification:** N/A — no new packages requested. Verified via `package.json` (already
|
||||
read as part of project context) that `pg@8.11.0` is the pinned version in use.
|
||||
|
||||
### Alternatives Considered
|
||||
|
||||
| Instead of | Could Use | Tradeoff |
|
||||
|------------|-----------|----------|
|
||||
| TS-constant sender-domain allowlist | `classification_rules`-style DB table (mirroring `robotic-classifier.ts`'s DB-driven rule cache) | DB table would let ops add a vendor domain without a deploy, matching the *existing* robotic-classifier precedent exactly — but that precedent is for ticket-triage rules edited by non-engineers via `/admin/workflow`, a UI that doesn't exist for phishing yet (`UI hint: no` this phase). A DB-backed allowlist with no admin UI is a bare table an operator would have to `psql` into directly — worse ergonomics than a reviewed TS constant, and higher risk (a compromised/mistaken raw SQL edit could silently suppress a real THREAT verdict for an attacker-controlled domain). Recommendation: TS constant now; revisit as a DB table only if/when an admin UI is scoped. |
|
||||
|
||||
**Installation:** N/A — no `npm install` needed this phase.
|
||||
|
||||
## Package Legitimacy Audit
|
||||
|
||||
**N/A this phase.** No new external packages are installed — the classifier consumes only
|
||||
already-shipped project code (`eml-parser.ts`, `mimecast-blast-radius.ts`, `postgres-client.ts`)
|
||||
and Node/TypeScript built-ins. The Package Legitimacy Gate protocol was not run because there is
|
||||
nothing to audit; if the planner introduces any new dependency during implementation, this gate
|
||||
must be run at that time.
|
||||
|
||||
## Architecture Patterns
|
||||
|
||||
### System Architecture Diagram
|
||||
|
||||
```
|
||||
POST /api/phishing/campaigns/{id}/classify
|
||||
│
|
||||
▼
|
||||
requirePermission('phishing','analyze') ──(fail)──► 403/401 response
|
||||
│ (pass)
|
||||
▼
|
||||
classifyCampaign(campaignId) [lib/services/campaign-classifier.ts]
|
||||
│
|
||||
├─► gatherCampaignEvidence(campaignId)
|
||||
│ │
|
||||
│ ├─► SELECT campaign row (campaigns)
|
||||
│ ├─► SELECT linked reports (reports, ordered by created_at ASC)
|
||||
│ ├─► SELECT linked messages (messages, keyed by report_id IN (...))
|
||||
│ ├─► SELECT linked indicators (indicators, keyed by message_id IN (...))
|
||||
│ └─► getBlastRadius({sender, recipient, subject, dateWindow}) [Phase 17, ephemeral]
|
||||
│
|
||||
▼
|
||||
EvidencePayload (bounded: counts + capped sample arrays, never raw body text beyond
|
||||
Phase 16's existing 500-char bodyPreview cap)
|
||||
│
|
||||
▼
|
||||
evaluateSimulationAllowlist(evidence) ──(match)──► forced SPAM/UNWANTED path (D-06)
|
||||
│ (no match)
|
||||
▼
|
||||
evaluateThreatTier(evidence) (D-03: delivered/clicked > 0 AND (hard auth-fail OR known-bad indicator match))
|
||||
│ (no)
|
||||
▼
|
||||
evaluateSpamVsUnwanted(evidence) (D-04)
|
||||
│
|
||||
▼
|
||||
computeConfidence(evidence) (D-05: 1.0 minus per-missing-source deductions, each named in reasons)
|
||||
│
|
||||
▼
|
||||
mapVerdictToActions(verdict, evidence) (D-08 vocabulary; requires_approval = OR of destructive flags)
|
||||
│
|
||||
▼
|
||||
INSERT INTO classifications (campaign_id, verdict, confidence, summary, reasons, recommended_actions, requires_approval)
|
||||
│
|
||||
▼
|
||||
NextResponse.json({ id, campaignId, verdict, confidence, summary, reasons, recommendedActions, requiresApproval })
|
||||
```
|
||||
|
||||
A reader can trace one classify call end-to-end: HTTP → permission check → evidence gather (4
|
||||
DB queries + 1 ephemeral Mimecast call) → 4 pure rule-evaluation stages in a fixed order → 1
|
||||
INSERT → JSON response.
|
||||
|
||||
### Recommended Project Structure
|
||||
|
||||
```
|
||||
lib/services/
|
||||
├── campaign-classifier.ts # NEW — classifyCampaign(), gatherCampaignEvidence(), pure rule fns
|
||||
├── campaign-classifier.test.ts # NEW — vitest, mocks postgresClient + getBlastRadius
|
||||
├── campaign-grouping-service.ts # existing (Phase 18) — read, not modified
|
||||
├── mimecast-blast-radius.ts # existing (Phase 17) — read, not modified
|
||||
├── eml-parser.ts # existing (Phase 16) — read, not modified
|
||||
├── phishing-eml-service.ts # existing (Phase 16) — read, not modified
|
||||
└── phishing-detector.ts # existing (Phase 15) — read, not modified (KNOWN_PHISHING_PATTERNS
|
||||
# is the REPORT-pattern list; do not confuse with the NEW
|
||||
# sender-domain allowlist this phase adds)
|
||||
|
||||
app/api/phishing/campaigns/[id]/
|
||||
└── classify/
|
||||
└── route.ts # NEW — POST handler, requirePermission('phishing','analyze')
|
||||
```
|
||||
|
||||
### Pattern 1: Pure evidence-in / rule-eval / verdict-out (D-01's architectural analog)
|
||||
|
||||
**What:** `robotic-classifier.ts`'s `evaluateContains()`/`evaluateRule()` style — small, pure,
|
||||
deterministic, no regex/eval, string-only matching. This phase's rule functions should follow the
|
||||
same style but WITHOUT `robotic-classifier.ts`'s DB-driven rule-cache layer (`loadRules()`,
|
||||
`rulesCache`) — there is no admin UI or `classification_rules`-equivalent table for phishing this
|
||||
phase, so the D-03/D-04/D-06 rules are simple hardcoded TypeScript conditionals, not
|
||||
DB-configurable rows.
|
||||
|
||||
**When to use:** Every verdict-decision function in this phase.
|
||||
|
||||
**Example (based on `robotic-classifier.ts` conventions, adapted):**
|
||||
```typescript
|
||||
// Source: lib/services/robotic-classifier.ts (existing pattern), adapted for Phase 19
|
||||
function isSimulationSender(fromDomain: string | null, returnPathDomain: string | null): boolean {
|
||||
const domains = [fromDomain, returnPathDomain].filter((d): d is string => d !== null);
|
||||
return domains.some((d) =>
|
||||
KNOWN_SIMULATION_SENDERS.some((entry) =>
|
||||
entry.domains.some((allowed) => d.toLowerCase() === allowed || d.toLowerCase().endsWith(`.${allowed}`))
|
||||
)
|
||||
);
|
||||
}
|
||||
```
|
||||
|
||||
### Pattern 2: Shared-core service, one exported orchestrator function (D-01/D-02, mirrors Phase 18)
|
||||
|
||||
**What:** `campaign-grouping-service.ts` and `phishing-detector.ts` both export ONE orchestrator
|
||||
function (`groupReportIntoCampaign`, `detectPhishingTicket`) that the route handler calls
|
||||
directly — no class, no singleton instance. `campaign-classifier.ts` should follow the identical
|
||||
shape: `export async function classifyCampaign(campaignId: string): Promise<ClassifyResult>`.
|
||||
|
||||
**When to use:** The single entry point the new route calls.
|
||||
|
||||
### Pattern 3: Auth/route early-return (Phase 18 D-06 convention, reused verbatim)
|
||||
|
||||
**What:** Exact shape confirmed from `app/api/phishing/tickets/[ticket_id]/analyze/route.ts`:
|
||||
|
||||
```typescript
|
||||
// Source: app/api/phishing/tickets/[ticket_id]/analyze/route.ts (lines 18-23)
|
||||
export async function POST(
|
||||
request: NextRequest,
|
||||
{ params }: { params: Promise<{ id: string }> }
|
||||
) {
|
||||
const { error } = await requirePermission('phishing', 'analyze');
|
||||
if (error) return error;
|
||||
|
||||
const { id } = await params;
|
||||
// ... UUID validation (see campaigns/[id]/route.ts's UUID_RE guard), then classifyCampaign(id)
|
||||
}
|
||||
```
|
||||
|
||||
Note: the SAME `'analyze'` action is reused, not a new permission — confirmed in `lib/permissions.ts`
|
||||
(`phishing: ["read", "analyze", "approve", "remediate"]`) and Phase 18's canonical_refs, which
|
||||
explicitly calls out that `/classify` should use `'analyze'`, not introduce a new action.
|
||||
|
||||
### Anti-Patterns to Avoid
|
||||
|
||||
- **Do not gate the THREAT-tier auth check on `authResults` alone.** See "Common Pitfalls" below —
|
||||
this is empirically confirmed to misfire on real BSN/KnowBe4-forwarded tickets in this org's own
|
||||
history.
|
||||
- **Do not treat `KNOWN_PHISHING_PATTERNS` (report-button pattern match) as proof of "this is a
|
||||
simulation."** D-06 already forbids this; confirmed independently by ticket 698321 in this
|
||||
research (a real/ambiguous phishing report that matched "Source: KnowBe4 Phish Alert Button"
|
||||
purely because it was reported via that plugin — the underlying message was a suspicious
|
||||
Mimecast-digest-styled email, not a KnowBe4 simulation).
|
||||
- **Do not build a `classification_rules` DB table / admin UI for this phase.** No such
|
||||
requirement exists in CLASSIFY-01..06, and `robotic-classifier.ts`'s DB-driven pattern is
|
||||
explicitly NOT what D-01 asks for (D-01 cites `evaluateContains()`'s pure-function style, not
|
||||
the DB-rule-cache wrapper around it).
|
||||
|
||||
## Don't Hand-Roll
|
||||
|
||||
| Problem | Don't Build | Use Instead | Why |
|
||||
|---------|-------------|-------------|-----|
|
||||
| Delivery/click evidence | A new Mimecast query | `getBlastRadius()` (Phase 17) | Already handles config-gating, caching, graceful `unavailable` degrade, and the fan-out merge logic — re-implementing any part of this violates the Phase 17 canonical-read contract |
|
||||
| SPF/DKIM/DMARC verdict parsing | A new Authentication-Results parser | `NormalizedMessage.authResults`/`authResultsOriginal` (Phase 16) | Already hand-rolled once, deliberately (to avoid `mailauth`'s live-DNS-verification side effect) — do not re-parse raw headers in this phase |
|
||||
| Attachment/URL indicator extraction | A new indicator scan over `messages.headers`/`.urls`/`.attachments` JSONB | The already-persisted `indicators` table rows | Phase 16 already normalized these into typed rows (`attachment_hash`/`url`/`sender`) — read them, don't re-derive from raw JSONB |
|
||||
| Report-pattern matching | A second KnowBe4/BSN detector | `phishing-detector.ts`'s `KNOWN_PHISHING_PATTERNS` (already applied upstream in Phase 15) | This phase's allowlist is a DIFFERENT signal (sender domain of the original message, not the report-button pattern) — do not duplicate or extend the existing pattern list for this purpose |
|
||||
| Threat-intel/reputation lookup for "known-bad" indicators | A URL/hash reputation API call (VirusTotal, URLhaus, etc.) | Cross-report indicator correlation within the campaign (see "Known-Bad Indicator" pitfall below) | No such integration exists anywhere in this codebase (confirmed via grep across `lib/`, `CLAUDE.md`, `ARCHITECTURE.md` — zero matches) and none is listed as a phase dependency; introducing one would be scope creep and a new external service this phase was never scoped to configure |
|
||||
|
||||
**Key insight:** Every evidence source this phase needs already exists as a typed, tested
|
||||
abstraction from Phases 15-18. The entire job of `campaign-classifier.ts` is to READ those
|
||||
abstractions and apply the D-03/D-04/D-05/D-06/D-08 rules — there is no new I/O to hand-roll.
|
||||
|
||||
## Common Pitfalls
|
||||
|
||||
### Pitfall 1: Forwarding-induced auth-verdict inversion (empirically confirmed, HIGH severity)
|
||||
|
||||
**What goes wrong:** A genuine KnowBe4/BSN security-awareness email, reported via Outlook's
|
||||
"Report Message" add-in (which forwards/wraps the original message), shows `spf=fail`,
|
||||
`dkim=fail`, `dmarc=fail` on the PRIMARY `Authentication-Results` header — even though the
|
||||
message legitimately passed all three at the point of original delivery. If the classifier's D-03
|
||||
hard-fail check reads only `NormalizedMessage.authResults` (not `authResultsOriginal`), it will
|
||||
find a "hard fail" and can push a genuine simulation into THREAT-tier evaluation.
|
||||
|
||||
**Why it happens:** Forwarding an email (or the internal M365 "report to Microsoft" pipeline
|
||||
re-wrapping it) changes the envelope sender and can invalidate the original DKIM signature: the
|
||||
receiving/reporting hop's own `Authentication-Results` header reflects THIS re-transmission, not
|
||||
the original delivery. Microsoft (and Mimecast, when in the path) preserve the original,
|
||||
pre-forwarding verdict in a separate `Authentication-Results-Original` header — which Phase 16
|
||||
already parses into `NormalizedMessage.authResultsOriginal` for exactly this reason (see Phase
|
||||
16's own Open-Questions log: *"parse both if present... Phase 19's classifier will care about
|
||||
pre-remediation verdicts for Mimecast-protected tenants"*).
|
||||
|
||||
**Empirical evidence (this research, ticket #610787, `T20260716.0291`):** A real
|
||||
`breachsecurenow.com`-originated "Welcome to Your Security Training Program" ticket shows:
|
||||
```
|
||||
authentication-results: spf=fail ... dkim=fail (body hash did not verify) ...dmarc=fail action=quarantine ...
|
||||
authentication-results-original: dkim=pass header.d=breachsecurenow.com ...; dmarc=pass (policy=quarantine) ...; spf=pass ...
|
||||
```
|
||||
A second ticket (#640079) shows the SAME domain's SAME legitimate training-notification email
|
||||
misclassified by an automated security-alert tool as a "High severity" phishing/spoofing threat,
|
||||
purely because it only looked at the primary (post-forward) auth verdict.
|
||||
|
||||
**How to avoid:** When evaluating D-03's "hard SPF/DKIM/DMARC fail" condition, check
|
||||
`authResultsOriginal` FIRST if present on a given message; only fall back to `authResults` when
|
||||
`authResultsOriginal` is null. Do this per-message (a campaign can have multiple linked messages).
|
||||
|
||||
**Warning signs:** A THREAT verdict where the ONLY malicious signal is an auth failure, on a
|
||||
message whose `authResultsOriginal` (if present) shows all-pass — this is very likely this exact
|
||||
pitfall, not a real threat.
|
||||
|
||||
### Pitfall 2: "Known-bad indicator match" has no defined data source (ambiguity, MEDIUM severity)
|
||||
|
||||
**What goes wrong:** D-03 says THREAT requires "...OR a known-bad indicator (attachment-hash/URL
|
||||
match from Phase 16's `indicators` table)." Taken literally, "match" implies matching against
|
||||
some external known-bad reference (a reputation/threat-intel list) — but no such integration
|
||||
exists anywhere in this codebase's dependency chain (confirmed: zero references to VirusTotal,
|
||||
URLhaus, AbuseIPDB, or any hash/URL reputation service in `lib/`, `CLAUDE.md`, `ARCHITECTURE.md`,
|
||||
or the v3.0 phase list). `MimecastThreatEvent` (in `mimecast-client.ts`) DOES carry a `verdict`
|
||||
and `threatLevel` field internally, but `getBlastRadius()` (the only contract this phase is
|
||||
allowed to read per canonical_refs) does not surface either field in `BlastRadiusResult` — it
|
||||
only derives a `clicked` count from threat events, discarding the rest.
|
||||
|
||||
**Why it happens:** The phrase "match" was written assuming a matching *mechanism* exists, but
|
||||
none of Phases 15-18 built one — `indicators` rows are extracted-and-stored values, not
|
||||
reputation-scored ones.
|
||||
|
||||
**How to avoid:** Interpret "known-bad indicator match" as **the SAME attachment-hash or URL value
|
||||
appearing on ≥2 distinct reports/messages within the same campaign** — i.e., independent
|
||||
confirmation that multiple different recipients received the identical artifact, which is itself
|
||||
a real signal of a deliberate, spreading campaign rather than one person's one-off spam. This is
|
||||
structurally cheap to compute (`GROUP BY value HAVING COUNT(DISTINCT message_id) >= 2` over the
|
||||
campaign's indicators) and requires no new external dependency. Flag this interpretation clearly
|
||||
to the user/planner as a reasoned substitution for an ambiguous decision, not a locked fact.
|
||||
|
||||
**Warning signs:** If the planner instead tries to wire in a live reputation API call, that is
|
||||
scope creation beyond CLASSIFY-01..06 and beyond this milestone's dependency list — flag for
|
||||
discussion before building it.
|
||||
|
||||
### Pitfall 3: `From:` header may lack a visible email address on some real report tickets
|
||||
|
||||
**What goes wrong:** Not every BSN/KnowBe4-forwarded message has a `<user@domain>` address on its
|
||||
`From:` header when it reaches the ticket/evidence pipeline — ticket #610787's raw header dump
|
||||
shows `From: "Wulf Consulting, Inc."` with NO address at all (display-name only). If
|
||||
`NormalizedMessage.from.email`/`.domain` comes back null for a genuine simulation message, the D-06
|
||||
sender-domain allowlist check (which is specified against `From`/`Return-Path` domain) will
|
||||
silently fail to match on the `From` side.
|
||||
|
||||
**Why it happens:** Some display-name-customization or forwarding paths (this appears specific to
|
||||
how BSN/M365 renders certain forwarded messages) produce a `From:` header with no bracketed
|
||||
address. The `Return-Path`/envelope-sender (`smtp.mailfrom=`) is far more reliably present and, in
|
||||
the same tickets, DOES carry the real sending domain (e.g.,
|
||||
`bounces11489798-daa2-wchapman=seubert.com@em8721.breachsecurenow.com`).
|
||||
|
||||
**How to avoid:** D-06 already says to check BOTH `From` and `Return-Path` domain — this research
|
||||
confirms that's not just belt-and-suspenders, it's load-bearing: implement the allowlist check as
|
||||
"`from.domain` matches OR `returnPath`'s domain-part matches," never `from.domain` alone.
|
||||
|
||||
**Warning signs:** A KnowBe4/BSN-domain allowlist that only ever matches on synthetic test fixtures
|
||||
(which always have complete `From:` addresses) but silently fails on some fraction of real
|
||||
campaigns.
|
||||
|
||||
### Pitfall 4: KnowBe4/BSN sender-domain lists are inherently incomplete and rotate over time
|
||||
|
||||
**What goes wrong:** KnowBe4's own documentation confirms organizations are issued a CSV of ~53
|
||||
"phish link domains" (system + custom), and this research's own ticket-history scan surfaced at
|
||||
least 7 KnowBe4-flavored domains beyond `it-support.care` (`customer-portal.info`,
|
||||
`member-services.info`, `cloud-service-care.com`, `logineverification.com`, `secureaccess.biz`,
|
||||
`packagetrackingportal.com`, and others) that were NOT independently confirmed as KnowBe4-owned in
|
||||
this research pass (only `it-support.care` was confirmed with high confidence via the
|
||||
multi-persona/single-domain pattern — see D-07 section below). A hardcoded allowlist seeded from
|
||||
today's ticket history will miss future/rotated domains.
|
||||
|
||||
**How to avoid:** Ship the allowlist as an easily-extendable, clearly-commented TS constant
|
||||
(structured per-vendor, e.g. `{ vendor: 'knowbe4', domains: [...] }`), and explicitly document in
|
||||
the code comment that this list is NOT exhaustive and should be refreshed periodically from new
|
||||
ticket evidence or a vendor-provided domain export — do not present it as a complete/permanent
|
||||
solution.
|
||||
|
||||
**Warning signs:** A new "Phishing Report" ticket from a not-yet-listed KnowBe4/BSN domain gets
|
||||
classified as UNWANTED/THREAT instead of SPAM/UNWANTED-via-simulation — expected/acceptable
|
||||
degradation, not a bug, given D-07's explicit scope ("ship KnowBe4-only if BSN can't be confirmed
|
||||
... treat adding [more] as a fast follow-up").
|
||||
|
||||
## D-07 Findings: BSN Ticket Evidence (highest-priority research item)
|
||||
|
||||
Queried `tickets` table directly (`docker exec pulse-postgres psql ...`) for
|
||||
`title/description ILIKE '%breach secure now%' OR '%BSN%'` and cross-referenced against the
|
||||
existing `KNOWN_PHISHING_PATTERNS` matches. Findings:
|
||||
|
||||
1. **A real Breach Secure Now sender-domain pattern exists and is usable.** Ticket
|
||||
`#610787` / `T20260716.0291` — title `Phishing:cd85e335-e22d-4dc8-c4df-08ddc49a6050|no-reply@breachsecurenow.com|(Welcome to Your Security Training Program) 7/16/2025 11:43:53 PM`
|
||||
— is a genuine BSN training-program notification, reported via the same Microsoft "Report
|
||||
Message"-flavored flow that produces `userSubmissionsReportMessage`/`reported message
|
||||
destinations`/`Microsoft directly` matches in `KNOWN_PHISHING_PATTERNS`. Its raw headers show:
|
||||
- `smtp.mailfrom=em8721.breachsecurenow.com` (envelope/bounce subdomain — SendGrid-style VERP)
|
||||
- `header.d=breachsecurenow.com`, `header.from=breachsecurenow.com` (DKIM/DMARC domain)
|
||||
- A sibling ticket, `#610770` (same day), and `#650284` ("User inquiry about potential phishing
|
||||
email from breachsecurenow.com"), corroborate the same domain independently.
|
||||
**Recommendation:** add `breachsecurenow.com` to the sender-domain allowlist (match on exact
|
||||
domain OR any subdomain, since the envelope sender uses rotating `emNNNN.breachsecurenow.com`
|
||||
subdomains — confirm via suffix match, not exact-string match).
|
||||
|
||||
2. **A real, high-confidence KnowBe4 sender-domain pattern also exists** (not explicitly asked for
|
||||
in D-07, but directly relevant to D-06/CLASSIFY-04, and far more load-bearing than the BSN
|
||||
finding given `it-support.care` appears in **219 tickets** across **~37 distinct impersonated
|
||||
personas** — Apple, Amazon, Adobe, Discord, Instagram, iTunes, Microsoft, Netflix, ChatGPT,
|
||||
AT&T, Dropbox, "SSN Fraud Alert," "Google Security Alert" — all sharing the ONE sending domain
|
||||
`it-support.care`, with local-parts like `noreply@`, `security@`, `notification@`,
|
||||
`humanrec@`, `webinars@`). This one-domain/many-personas pattern is the hallmark of a
|
||||
security-awareness-training template library, not a real attacker's infrastructure (a real
|
||||
phishing campaign targeting one org rarely rotates through dozens of unrelated brand
|
||||
impersonations from the same domain over 3+ years). Ticket `#624439` /
|
||||
`Phishing:00000000-0000-0000-0000-000000000000|noreply@it-support.care|(Phishing Test Exempt)`
|
||||
— the literal subject "Phishing Test Exempt" plus the all-zero GUID — is strong direct
|
||||
confirmation this is KnowBe4's own "this was a known test, don't alert" marker baked into the
|
||||
simulated email itself.
|
||||
**Confidence:** MEDIUM-HIGH — grounded in real, voluminous ticket data and a highly distinctive
|
||||
pattern, but NOT independently confirmed against KnowBe4's own published domain list (no such
|
||||
public list was found; KnowBe4 support explicitly says the full ~53-domain CSV is
|
||||
account-specific and only available by contacting them directly). Tag `[ASSUMED]` per the
|
||||
package/claim-provenance rule — recommend the planner surface this to the user for a quick
|
||||
sanity check ("does `it-support.care` ring a bell as your KnowBe4 simulation domain?") before
|
||||
locking it in, even though the evidence is strong.
|
||||
**Recommendation:** add `it-support.care` to the allowlist as the KnowBe4 entry.
|
||||
|
||||
3. **Other candidate domains found in the same "Phishing Report -" ticket population were NOT
|
||||
independently confirmed** and should NOT be added without further evidence:
|
||||
`customer-portal.info` (118 tickets), `member-services.info` (106), `cloud-service-care.com`
|
||||
(60), `logineverification.com` (35), `secureaccess.biz` (17), `packagetrackingportal.com` (16).
|
||||
These share `it-support.care`'s generic-corporate-naming flavor and could plausibly be more
|
||||
KnowBe4 system domains — but some high-volume domains in the same list
|
||||
(`bankonlinesupport.com`, 50 tickets; `mlcrosoft.live`, 19, an obvious typosquat) are very
|
||||
likely GENUINE phishing/spam, not simulations, so volume alone doesn't distinguish
|
||||
vendor-owned from attacker-owned domains. Do not add these without a second, independent
|
||||
confirmation signal (e.g., asking the user, or finding a ticket with the same
|
||||
"Phishing Test Exempt"/all-zero-GUID marker seen for `it-support.care`).
|
||||
|
||||
4. **Report-pattern match does NOT imply simulation** — directly confirmed, not just theoretical.
|
||||
Ticket `#698321` ("Phishing Report - You have new held messages") matched
|
||||
`KNOWN_PHISHING_PATTERNS`'s "Source: KnowBe4 Phish Alert Button" and "Phishing Report" patterns
|
||||
(i.e., it WAS reported via the KnowBe4 PAB plugin) but its actual content is a suspicious
|
||||
Mimecast bulk-digest-styled email with obfuscated redirect URLs — almost certainly a genuine
|
||||
phishing/spam attempt that a user reported using the same button KnowBe4 simulations use to
|
||||
auto-report themselves. This is exactly D-06's stated rationale, independently reproduced with
|
||||
real data: the report-pattern match only proves how it was reported, never what it is.
|
||||
|
||||
## Code Examples
|
||||
|
||||
### Evidence assembly (report/message/indicator gathering, bounded)
|
||||
|
||||
```typescript
|
||||
// Source: pattern derived from app/api/phishing/campaigns/[id]/route.ts's existing
|
||||
// bulk-fetch-by-id-array shape (reportIds -> messagesRes -> messageIds -> indicatorsRes)
|
||||
async function gatherCampaignEvidence(campaignId: string) {
|
||||
const campaign = await postgresClient.query(/* SELECT * FROM campaigns WHERE id = $1 */);
|
||||
const reports = await postgresClient.query(
|
||||
`SELECT r.*, c.email_address AS requester_email
|
||||
FROM reports r LEFT JOIN contacts c ON c.id = r.requester_contact_id
|
||||
WHERE r.campaign_id = $1 ORDER BY r.created_at ASC`,
|
||||
[campaignId]
|
||||
);
|
||||
const reportIds = reports.rows.map((r) => r.id);
|
||||
const messages = reportIds.length
|
||||
? await postgresClient.query(
|
||||
`SELECT * FROM messages WHERE report_id = ANY($1::uuid[])`, [reportIds]
|
||||
)
|
||||
: { rows: [] };
|
||||
const messageIds = messages.rows.map((m) => m.id);
|
||||
const indicators = messageIds.length
|
||||
? await postgresClient.query(
|
||||
`SELECT * FROM indicators WHERE message_id = ANY($1::uuid[])`, [messageIds]
|
||||
)
|
||||
: { rows: [] };
|
||||
|
||||
// Blast radius: use the EARLIEST report as the canonical evidence source for
|
||||
// sender/subject/date-window (mirrors campaign-grouping-service.ts's own
|
||||
// `ORDER BY r.created_at ASC` convention for "the original" report).
|
||||
const primaryReport = reports.rows[0];
|
||||
const primaryMessage = messages.rows.find((m) => m.report_id === primaryReport?.id);
|
||||
const senderIndicator = indicators.rows.find(
|
||||
(i) => i.message_id === primaryMessage?.id && i.indicator_type === 'sender'
|
||||
);
|
||||
const blastRadius = primaryReport
|
||||
? await getBlastRadius({
|
||||
sender: senderIndicator?.value ?? primaryMessage?.headers?.from?.email ?? '',
|
||||
recipient: primaryReport.requester_email ?? '',
|
||||
subject: primaryMessage?.headers?.subject ?? primaryReport.title ?? '',
|
||||
dateWindow: {
|
||||
start: new Date(new Date(primaryReport.created_at).getTime() - 24 * 60 * 60 * 1000),
|
||||
end: new Date(new Date(primaryReport.created_at).getTime() + 24 * 60 * 60 * 1000),
|
||||
},
|
||||
})
|
||||
: { status: 'unavailable' as const, reason: 'not_configured' as const };
|
||||
|
||||
return { campaign: campaign.rows[0], reports: reports.rows, messages: messages.rows, indicators: indicators.rows, blastRadius };
|
||||
}
|
||||
```
|
||||
|
||||
### Auth-verdict precedence (Pitfall 1 fix)
|
||||
|
||||
```typescript
|
||||
// Prefer the pre-forwarding verdict when present (Pitfall 1) — do this per-message.
|
||||
function effectiveAuthResults(message: { headers: { authResults: AuthResults; authResultsOriginal: AuthResults | null } }): AuthResults {
|
||||
return message.headers.authResultsOriginal ?? message.headers.authResults;
|
||||
}
|
||||
|
||||
function hasHardAuthFail(effective: AuthResults): boolean {
|
||||
return effective.spf === 'fail' || effective.dkim === 'fail' || effective.dmarc === 'fail';
|
||||
}
|
||||
```
|
||||
|
||||
### KnowBe4/BSN sender-domain allowlist (D-06/D-07)
|
||||
|
||||
```typescript
|
||||
// lib/services/campaign-classifier.ts
|
||||
// NOT exhaustive — see 19-RESEARCH.md "Pitfall 4" and "D-07 Findings" for provenance and
|
||||
// known gaps. Refresh from new ticket evidence or a vendor domain export as needed.
|
||||
export const KNOWN_SIMULATION_SENDERS: readonly { vendor: string; domains: readonly string[] }[] = [
|
||||
{
|
||||
vendor: 'knowbe4',
|
||||
domains: ['it-support.care'], // 219 tickets, ~37 impersonated personas — see D-07 finding #2
|
||||
},
|
||||
{
|
||||
vendor: 'breach-secure-now',
|
||||
domains: ['breachsecurenow.com'], // confirmed via ticket #610787/#610770/#650284 — D-07 finding #1
|
||||
},
|
||||
];
|
||||
|
||||
function domainMatchesAllowlist(domain: string): boolean {
|
||||
const lower = domain.toLowerCase();
|
||||
return KNOWN_SIMULATION_SENDERS.some((entry) =>
|
||||
entry.domains.some((allowed) => lower === allowed || lower.endsWith(`.${allowed}`))
|
||||
);
|
||||
}
|
||||
|
||||
// Pitfall 3: check BOTH From domain and Return-Path domain — From may lack an address.
|
||||
function isKnownSimulationSender(message: NormalizedMessage): boolean {
|
||||
const fromDomain = message.from.domain;
|
||||
const returnPathDomain = message.returnPath?.split('@')[1] ?? null;
|
||||
return [fromDomain, returnPathDomain]
|
||||
.filter((d): d is string => d !== null)
|
||||
.some(domainMatchesAllowlist);
|
||||
}
|
||||
```
|
||||
|
||||
### Confidence scoring (D-05)
|
||||
|
||||
```typescript
|
||||
interface ConfidenceResult { confidence: number; reasons: string[] }
|
||||
|
||||
function computeConfidence(evidence: {
|
||||
hasAnyMessage: boolean;
|
||||
blastRadiusStatus: 'ok' | 'unavailable';
|
||||
hasAttachmentOrUrlIndicators: boolean;
|
||||
}): ConfidenceResult {
|
||||
let confidence = 1.0;
|
||||
const reasons: string[] = [];
|
||||
|
||||
if (!evidence.hasAnyMessage) {
|
||||
confidence -= 0.4;
|
||||
reasons.push('No .eml/message evidence parsed for any report in this campaign — sender-domain and auth-verdict signals unavailable');
|
||||
}
|
||||
if (evidence.blastRadiusStatus !== 'ok') {
|
||||
confidence -= 0.3;
|
||||
reasons.push('Mimecast blast-radius data unavailable — delivery/click evidence could not be confirmed');
|
||||
}
|
||||
if (!evidence.hasAttachmentOrUrlIndicators) {
|
||||
confidence -= 0.2;
|
||||
reasons.push('No attachment-hash or URL indicators found for this campaign');
|
||||
}
|
||||
|
||||
return { confidence: Math.round(confidence * 100) / 100, reasons };
|
||||
}
|
||||
```
|
||||
|
||||
Weights chosen so that: message-parse absence (0.4) is weighted heaviest since it starves every
|
||||
other evidence source (no sender domain, no auth verdicts, no indicators without a parsed
|
||||
message); Mimecast unavailability (0.3) is next since D-03's THREAT gate directly depends on it;
|
||||
missing indicators (0.2) is lightest since a genuinely clean message legitimately has none. The
|
||||
three sum to 0.9, leaving a natural floor of 0.1 confidence when all three evidence sources are
|
||||
missing — no extra clamping logic needed, and confidence is never presented as exactly 0.
|
||||
|
||||
### Recommended actions mapping (D-08)
|
||||
|
||||
```typescript
|
||||
const DESTRUCTIVE_ACTIONS = new Set(['block_sender', 'purge_message', 'reset_password', 'isolate_endpoint']);
|
||||
|
||||
function mapVerdictToActions(verdict: 'SPAM' | 'UNWANTED' | 'THREAT', evidence: { clicked: number }): string[] {
|
||||
switch (verdict) {
|
||||
case 'SPAM':
|
||||
return ['no_action'];
|
||||
case 'UNWANTED':
|
||||
return ['warn_user'];
|
||||
case 'THREAT': {
|
||||
const actions = ['block_sender', 'purge_message'];
|
||||
// Evidence of actual interaction (not just delivery) raises the bar to
|
||||
// credential/endpoint-compromise-level actions — reasoned interpretation,
|
||||
// not an explicit D-08 rule; flag for planner/user confirmation.
|
||||
if (evidence.clicked > 0) {
|
||||
actions.push('reset_password', 'isolate_endpoint', 'disable_forwarding_rule');
|
||||
}
|
||||
return actions;
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
function computeRequiresApproval(actions: string[]): boolean {
|
||||
return actions.some((a) => DESTRUCTIVE_ACTIONS.has(a));
|
||||
}
|
||||
```
|
||||
|
||||
### `classifications` INSERT (append-only, migration 097 exact columns)
|
||||
|
||||
```typescript
|
||||
// Source: migrations/097_phishing_triage_schema.sql lines 114-124
|
||||
await postgresClient.query(
|
||||
`INSERT INTO classifications (
|
||||
campaign_id, verdict, confidence, summary, reasons, recommended_actions, requires_approval
|
||||
) VALUES ($1, $2, $3, $4, $5::jsonb, $6::jsonb, $7)
|
||||
RETURNING id::text AS id, created_at::text AS created_at`,
|
||||
[campaignId, verdict, confidence, summary, JSON.stringify(reasons), JSON.stringify(recommendedActions), requiresApproval]
|
||||
);
|
||||
```
|
||||
|
||||
## State of the Art
|
||||
|
||||
Not directly applicable — this phase does not depend on any fast-moving external library or API
|
||||
version. The relevant "state" is entirely this codebase's own Phase 15-18 output, all read
|
||||
directly above.
|
||||
|
||||
| Old Approach | Current Approach | When Changed | Impact |
|
||||
|--------------|------------------|--------------|--------|
|
||||
| N/A | N/A | N/A | N/A |
|
||||
|
||||
**Deprecated/outdated:** None applicable this phase.
|
||||
|
||||
## Assumptions Log
|
||||
|
||||
| # | Claim | Section | Risk if Wrong |
|
||||
|---|-------|---------|---------------|
|
||||
| A1 | `it-support.care` is a KnowBe4-owned phishing-simulation sending domain | D-07 Findings #2, Code Examples (`KNOWN_SIMULATION_SENDERS`) | If wrong, genuine phishing sent from this domain (unlikely given the multi-persona pattern, but not impossible) would be forced to SPAM/UNWANTED and never reach THREAT — a false negative on a real attack. Mitigate: surface to user for a one-line confirmation before locking in; the "Phishing Test Exempt" subject-line ticket (#624439) is strong corroboration but not a KnowBe4-issued confirmation. |
|
||||
| A2 | `breachsecurenow.com` is Breach Secure Now's legitimate training-notification sending domain | D-07 Findings #1 | Same class of risk as A1, but lower — envelope/DKIM/DMARC domain all agree across 3 independent tickets, and BSN is a named, known-to-the-user vendor (unlike a cold ticket-volume inference). |
|
||||
| A3 | `customer-portal.info`, `member-services.info`, `cloud-service-care.com`, `logineverification.com`, `secureaccess.biz`, `packagetrackingportal.com` are additional KnowBe4 domains | D-07 Findings #3 | NOT recommended for inclusion in this research — listed only as candidates requiring further confirmation. Risk is null if the planner follows the recommendation to exclude them. |
|
||||
| A4 | "Known-bad indicator match" (D-03) means cross-report indicator correlation within a campaign, not external reputation lookup | Common Pitfalls #2, Code Examples | If the user actually intended an external reputation check, this interpretation under-detects THREAT (misses attacker infrastructure known-bad via a real feed) — but building an unrequested external integration is a larger, riskier assumption. Flagged explicitly for planner/user sign-off. |
|
||||
| A5 | TypeScript-constant allowlist (not DB table) is the right storage choice for D-06 | Standard Stack — Alternatives Considered | If wrong, ops will need a code deploy to add/remove a vendor domain rather than an admin UI edit — a process cost, not a correctness risk. Reversible later without a rework (per D-06's own multi-vendor-extensibility requirement). |
|
||||
| A6 | Primary blast-radius call should use the campaign's EARLIEST report as canonical sender/subject/date-window | Code Examples (`gatherCampaignEvidence`) | If wrong (e.g. a later report has better/more-complete data), blast-radius lookup could target a slightly wrong subject/date-window and return fewer/no matches — degrades to `status:'unavailable'`-like under-evidence, not a false verdict, since D-03's gate still requires a real signal to fire. |
|
||||
|
||||
## Open Questions
|
||||
|
||||
1. **Should `disable_forwarding_rule` ever be recommended by this phase's rules, given no
|
||||
evidence source in Phases 15-18 signals "a mailbox forwarding rule was created"?**
|
||||
- What we know: D-08 defines the action as valid vocabulary and requires the
|
||||
`requires_approval` invariant to hold even when it's the only recommended action — implying
|
||||
at least one code path must be able to produce it (to exercise the test).
|
||||
- What's unclear: no evidence source in this evidence model (blast radius, auth verdicts,
|
||||
indicators) actually detects a mailbox rule.
|
||||
- Recommendation: tie it to `clicked > 0` on a THREAT verdict (see Code Examples) as a
|
||||
reasoned proxy for "possible account compromise, worth checking for a malicious forwarding
|
||||
rule" — but confirm with the user/planner before locking in, since this is inference, not an
|
||||
explicit CONTEXT.md decision.
|
||||
|
||||
2. **Should the KnowBe4/BSN allowlist match check `Return-Path` via a parsed domain, or does
|
||||
`NormalizedMessage` need a new `returnPathDomain` field?**
|
||||
- What we know: `NormalizedMessage.returnPath` is currently a raw address STRING (e.g.
|
||||
`bounces...@em8721.breachsecurenow.com`), not pre-split into a domain.
|
||||
- What's unclear: whether the planner wants a small addition to `eml-parser.ts` (new
|
||||
`returnPathDomain` derived field) or wants `campaign-classifier.ts` to do the
|
||||
`.split('@')[1]` itself locally.
|
||||
- Recommendation: do it locally in `campaign-classifier.ts` (as shown in Code Examples) — Phase
|
||||
16 is marked "read, does not modify" in this phase's canonical_refs, and the split is a
|
||||
one-line, low-risk operation not worth reopening an already-shipped, tested module for.
|
||||
|
||||
3. **Is a campaign's blast-radius lookup re-run on every `/classify` call, or cached/reused across
|
||||
calls within the same campaign?**
|
||||
- What we know: `getBlastRadius()` already has its own 5-minute Redis cache keyed on
|
||||
messageId/composite sender+subject+date-window (Phase 17 D-?), and D-02 says every
|
||||
`/classify` call inserts a new row (no caching of the VERDICT).
|
||||
- What's unclear: nothing, actually — `getBlastRadius()`'s own cache already handles the
|
||||
"don't hammer Mimecast on repeated classify calls" concern transparently. No new caching
|
||||
logic needed in this phase.
|
||||
- Recommendation: no action needed; call `getBlastRadius()` on every `classifyCampaign()`
|
||||
invocation and let its existing cache absorb repeat calls.
|
||||
|
||||
## Environment Availability
|
||||
|
||||
| Dependency | Required By | Available | Version | Fallback |
|
||||
|------------|------------|-----------|---------|----------|
|
||||
| Postgres 16 (`pulse-postgres` container) | All evidence reads + `classifications` INSERT | ✓ | 16 (running, confirmed via live query) | — |
|
||||
| Mimecast (`MIMECAST_CLIENT_ID`/`SECRET`/`TOKEN_URL`) | `getBlastRadius()` D-03 delivery signal | ✓ (set in `.env`) | — | `getBlastRadius()` already degrades to `status:'unavailable'` if unset — no fallback needed in this phase's own code |
|
||||
| Autotask (`AUTOTASK_*`) | Upstream only (reports/tickets already synced) — not called directly by this phase | ✓ (set in `.env`) | — | — |
|
||||
| Backblaze B2 (`B2_*`) | Not used by this phase (raw `.eml` bytes are Phase 16's concern; classifier reads only parsed `messages`/`indicators` rows) | Not set in `.env` | — | N/A — this phase never touches B2 |
|
||||
|
||||
**Missing dependencies with no fallback:** None.
|
||||
|
||||
**Missing dependencies with fallback:** None relevant to this phase's own code — Mimecast
|
||||
unavailability is an explicit D-05 evidence-completeness signal, not a blocker.
|
||||
|
||||
## Validation Architecture
|
||||
|
||||
### Test Framework
|
||||
|
||||
| Property | Value |
|
||||
|----------|-------|
|
||||
| Framework | vitest 4.1.5 |
|
||||
| Config file | `vitest.config.ts` (`include: ['lib/**/*.test.ts']`, node environment) |
|
||||
| Quick run command | `npx vitest run lib/services/campaign-classifier.test.ts` |
|
||||
| Full suite command | `npm test` |
|
||||
|
||||
### Phase Requirements → Test Map
|
||||
|
||||
| Req ID | Behavior | Test Type | Automated Command | File Exists? |
|
||||
|--------|----------|-----------|-------------------|-------------|
|
||||
| CLASSIFY-01 | `classifyCampaign()` returns exactly one of SPAM/UNWANTED/THREAT + confidence + summary + reasons + recommendedActions + requiresApproval | unit | `npx vitest run lib/services/campaign-classifier.test.ts -t "returns exactly one verdict"` | ❌ Wave 0 |
|
||||
| CLASSIFY-02 | Destructive action always forces `requires_approval: true`; `disable_forwarding_rule` alone does NOT | unit | `npx vitest run lib/services/campaign-classifier.test.ts -t "requires_approval invariant"` | ❌ Wave 0 |
|
||||
| CLASSIFY-03 | Incomplete evidence lowers confidence and names the specific missing evidence | unit | `npx vitest run lib/services/campaign-classifier.test.ts -t "confidence deduction"` | ❌ Wave 0 |
|
||||
| CLASSIFY-04 | Synthetic KnowBe4 (`it-support.care`) and BSN (`breachsecurenow.com`) simulation fixtures are not classified THREAT absent contrary evidence | unit | `npx vitest run lib/services/campaign-classifier.test.ts -t "simulation allowlist"` | ❌ Wave 0 |
|
||||
| CLASSIFY-05 | `POST /classify` enforces `requirePermission('phishing','analyze')` and returns the classification | manual/smoke (matches existing convention — no route-level test files exist anywhere under `app/api/phishing/`) | `curl -X POST localhost:3100/api/phishing/campaigns/{id}/classify` (authenticated session) | N/A — route tests are not this codebase's convention |
|
||||
| CLASSIFY-06 | Evidence payload is bounded/structured — no raw unbounded body text reaches the classifier or persisted `reasons` | unit | `npx vitest run lib/services/campaign-classifier.test.ts -t "evidence bounding"` | ❌ Wave 0 |
|
||||
|
||||
### Sampling Rate
|
||||
|
||||
- **Per task commit:** `npx vitest run lib/services/campaign-classifier.test.ts`
|
||||
- **Per wave merge:** `npm test`
|
||||
- **Phase gate:** Full suite green before `/gsd:verify-work`
|
||||
|
||||
### Wave 0 Gaps
|
||||
|
||||
- [ ] `lib/services/campaign-classifier.test.ts` — new file, covers CLASSIFY-01/02/03/04/06;
|
||||
mock `postgresClient.query` (vi.mock, matching `campaign-grouping-service.test.ts`'s
|
||||
existing convention) and mock `getBlastRadius` (vi.mock('./mimecast-blast-radius'))
|
||||
- [ ] Fixture data for a synthetic KnowBe4 (`it-support.care`) message and a synthetic BSN
|
||||
(`breachsecurenow.com`) message, each WITH an `authResultsOriginal` block showing pass
|
||||
despite `authResults` showing fail (Pitfall 1 regression coverage) — no real customer data,
|
||||
synthetic only, matching Phase 16's own fixture convention (`eml-parser.fixtures.ts`)
|
||||
- [ ] Framework install: none — vitest already configured project-wide
|
||||
|
||||
## Security Domain
|
||||
|
||||
### Applicable ASVS Categories
|
||||
|
||||
| ASVS Category | Applies | Standard Control |
|
||||
|---------------|---------|-----------------|
|
||||
| V2 Authentication | No | Route relies on existing Better Auth session — no new auth surface this phase |
|
||||
| V3 Session Management | No | Unchanged from existing middleware/session handling |
|
||||
| V4 Access Control | Yes | `requirePermission('phishing', 'analyze')` — same action as the existing `/analyze` route, enforced server-side (not just middleware's cookie-presence check) |
|
||||
| V5 Input Validation | Yes | Campaign `id` path param must be validated as a UUID before querying (mirror `campaigns/[id]/route.ts`'s `UUID_RE` guard) — an unvalidated malformed UUID would otherwise surface as an unhandled Postgres error → uncaught 500 |
|
||||
| V6 Cryptography | No | No new crypto in this phase — auth-verdict parsing (SPF/DKIM/DMARC) is READ-ONLY interpretation of headers already verified by upstream mail infrastructure, never independent crypto verification (Phase 16 deliberately avoids `mailauth`'s live verification for exactly this reason) |
|
||||
|
||||
### Known Threat Patterns for this stack
|
||||
|
||||
| Pattern | STRIDE | Standard Mitigation |
|
||||
|---------|--------|---------------------|
|
||||
| Classifier evidence payload growing unbounded (many reports/messages/indicators in one large campaign) | Denial of Service | Cap sample arrays in the persisted `reasons`/evidence JSONB (e.g. summarize counts + up to N sample values) rather than embedding every report/message/indicator verbatim — CLASSIFY-06's "size-bounded evidence" requirement doubles as a DoS guard |
|
||||
| Simulation-allowlist bypass via spoofed sender domain lookalike (e.g. `it-support.care.evil.com`) | Spoofing | Match using exact-domain-or-proper-subdomain (`d === allowed \|\| d.endsWith('.' + allowed)`), never a bare `.includes(allowed)` substring check — a substring check would let `evil-it-support.care.attacker.net` or `it-support.care.attacker.net` falsely match |
|
||||
| Malformed/malicious campaign `id` path param | Tampering | UUID-shape validation before querying (see V5 above), matching the existing `campaigns/[id]/route.ts` precedent exactly |
|
||||
| Confidence-score/verdict logic drift silently weakening THREAT detection over time | Tampering (of business logic, not data) | Keep all D-03/D-04/D-06 rule functions pure, unit-tested, and reviewed via PR (TS constant, not a live-editable DB table) — per this research's Standard-Stack recommendation |
|
||||
|
||||
## Sources
|
||||
|
||||
### Primary (HIGH confidence — read directly from this session)
|
||||
|
||||
- `/opt/stacks/pulse/lib/services/robotic-classifier.ts` — full file read, D-01 architectural analog confirmed
|
||||
- `/opt/stacks/pulse/lib/services/mimecast-blast-radius.ts` — full file read, `BlastRadiusResult`/`BlastRadiusInput` exact shape
|
||||
- `/opt/stacks/pulse/lib/services/eml-parser.ts` — full file read, `NormalizedMessage`/`AuthResults` exact shape
|
||||
- `/opt/stacks/pulse/lib/services/phishing-detector.ts` — full file read, `KNOWN_PHISHING_PATTERNS`/`matchesPhishingPatterns` exact shape
|
||||
- `/opt/stacks/pulse/lib/services/phishing-eml-service.ts` — full file read, `messages`/`indicators` INSERT shape confirmed
|
||||
- `/opt/stacks/pulse/lib/services/campaign-grouping-service.ts` — full file read, campaign/report join conventions confirmed
|
||||
- `/opt/stacks/pulse/lib/services/mimecast-client.ts` (lines 1-120) — `MimecastThreatEvent` shape confirmed, `verdict`/`threatLevel` fields NOT surfaced by `getBlastRadius()`
|
||||
- `/opt/stacks/pulse/migrations/097_phishing_triage_schema.sql` — full file read, exact `classifications` columns/types
|
||||
- `/opt/stacks/pulse/migrations/099_indicators_metadata.sql` — `indicators.metadata` column confirmed
|
||||
- `/opt/stacks/pulse/lib/permissions.ts` — full file read, `phishing` resource/action vocabulary confirmed
|
||||
- `/opt/stacks/pulse/app/api/phishing/tickets/[ticket_id]/analyze/route.ts` — full file read, exact auth-early-return pattern
|
||||
- `/opt/stacks/pulse/app/api/phishing/campaigns/[id]/route.ts` and `.../campaigns/route.ts` — full files read, exact response-shape/UUID-validation/bulk-fetch conventions
|
||||
- Live query against `pulse-postgres` (docker exec psql) — `tickets` table, 219 `it-support.care` tickets, 11 `breachsecurenow.com` tickets, confirmed raw MIME headers for tickets #610787, #640079, #624439, #695851, #698321 (D-07's core evidence)
|
||||
- `.planning/phases/16-eml-mime-evidence-parser/16-RESEARCH.md` and `16-01-PLAN.md` — confirmed Phase 16's own anticipation of the auth-verdict-precedence concern for Phase 19
|
||||
- `.planning/phases/17-mimecast-blast-radius-lookup/17-VERIFICATION.md` — confirmed Phase 17's shipped behavior and D-05 multi-tenant gap
|
||||
- `.planning/ROADMAP.md` (Phase 19 section, lines 427-438) — exact success-criteria wording
|
||||
|
||||
### Secondary (MEDIUM confidence)
|
||||
|
||||
- WebSearch: KnowBe4 "Manage Phish Link Domains" knowledge-base article — confirms KnowBe4 issues
|
||||
a per-account CSV of ~53 system+custom phish-link domains, corroborating that `it-support.care`
|
||||
is plausibly one of many such domains, though the specific domain was not found in any public
|
||||
KnowBe4 document (searched, not found — see Assumptions Log A1)
|
||||
|
||||
### Tertiary (LOW confidence)
|
||||
|
||||
- The 6 additional candidate domains in D-07 Finding #3 (`customer-portal.info`,
|
||||
`member-services.info`, `cloud-service-care.com`, `logineverification.com`,
|
||||
`secureaccess.biz`, `packagetrackingportal.com`) — ticket-volume pattern only, explicitly NOT
|
||||
recommended for inclusion without further confirmation
|
||||
|
||||
## Metadata
|
||||
|
||||
**Confidence breakdown:**
|
||||
- Standard stack / architecture / schema: HIGH — all read directly from shipped, tested code in this repository
|
||||
- D-07 BSN/KnowBe4 domain findings: MEDIUM-HIGH for `it-support.care` and `breachsecurenow.com` (strong ticket-data pattern, not vendor-confirmed); LOW for the 6 additional candidate domains (excluded from recommendation)
|
||||
- Confidence-weight numbers (D-05) and action-recommendation mapping (D-08 triggers): MEDIUM — reasoned proposals grounded in the evidence model's actual shape, not locked user decisions; flagged in Open Questions/Assumptions Log for planner/user confirmation
|
||||
|
||||
**Research date:** 2026-07-16
|
||||
**Valid until:** 30 days (stable internal codebase; the KnowBe4/BSN domain findings should be
|
||||
re-validated against fresh ticket history if this phase's implementation slips more than ~60 days,
|
||||
since simulated-phishing template domains do rotate)
|
||||
Loading…
Add table
Add a link
Reference in a new issue