wulf-pulse/.planning/phases/19-classification-engine/19-VALIDATION.md

phase	slug	status	nyquist_compliant	wave_0_complete	created
19	classification-engine	draft	true	false	2026-07-16

Phase 19 — Validation Strategy

Per-phase validation contract for feedback sampling during execution.

---

Test Infrastructure

Property	Value
Framework	vitest 4.1.5
Config file	vitest.config.ts — include: ['lib/**/*.test.ts'] (does NOT include app/** — no route-handler test precedent exists in this repo)
Quick run command	npx vitest run lib/services/campaign-classifier.test.ts
Full suite command	npm test
Estimated runtime	~2 seconds (mocked postgresClient and getBlastRadius, no real DB/network)

---

Sampling Rate

• After every task commit: npx vitest run lib/services/campaign-classifier.test.ts
• After every plan wave: npm test (full suite)
• Before /gsd:verify-work: Full suite green + npx tsc --noEmit --pretty clean + manual curl verification of the /classify route's auth behavior
• Max feedback latency: ~2 seconds (automated) / manual curl pass for route auth (no automated route-handler test precedent in this repo)

---

Per-Task Verification Map

Task ID	Plan	Wave	Requirement	Threat Ref	Secure Behavior	Test Type	Automated Command	File Exists	Status
19-01-0x	01	1	CLASSIFY-01	—	classifyCampaign() returns exactly one of SPAM/UNWANTED/THREAT + confidence + summary + reasons + recommendedActions + requiresApproval	unit	npx vitest run lib/services/campaign-classifier.test.ts -t "returns exactly one verdict"	❌ W0	⬜ pending
19-01-0x	01	1	CLASSIFY-02	Tampering	Destructive recommended action always forces requires_approval: true; disable_forwarding_rule alone does NOT	unit	npx vitest run lib/services/campaign-classifier.test.ts -t "requires_approval invariant"	❌ W0	⬜ pending
19-01-0x	01	1	CLASSIFY-03	—	Incomplete evidence lowers confidence and names the specific missing evidence source in reasons	unit	npx vitest run lib/services/campaign-classifier.test.ts -t "confidence deduction"	❌ W0	⬜ pending
19-01-0x	01	1	CLASSIFY-04	Spoofing	Synthetic KnowBe4 (it-support.care) and BSN (breachsecurenow.com) simulation fixtures — each with authResultsOriginal pass despite authResults fail — are not classified THREAT absent contrary evidence	unit	npx vitest run lib/services/campaign-classifier.test.ts -t "simulation allowlist"	❌ W0	⬜ pending
19-0x-0x	0x	0x	CLASSIFY-05	V4/V5	POST /api/phishing/campaigns/{id}/classify enforces requirePermission('phishing','analyze'), validates campaign id as UUID, inserts a new classifications row, and rejects unauthorized/malformed requests	manual	curl against dev server (no route-handler test precedent)	n/a	⬜ pending
19-01-0x	01	1	CLASSIFY-06	Denial of Service	Evidence payload is bounded/structured — no raw unbounded body text reaches the classifier or persisted reasons; IT Glue-sourced evidence (if any) goes through itglue-search.ts	unit	npx vitest run lib/services/campaign-classifier.test.ts -t "evidence bounding"	❌ W0	⬜ pending

Status: ⬜ pending · ✅ green · ❌ red · ⚠️ flaky Exact task IDs finalized by the planner — this table's requirement/behavior mapping is fixed by research; row numbering will be updated once PLAN.md exists.

---

Wave 0 Requirements

• lib/services/campaign-classifier.test.ts — new file, covers CLASSIFY-01/02/03/04/06; mock postgresClient.query (vi.mock, matching campaign-grouping-service.test.ts's existing convention) and mock getBlastRadius (vi.mock('./mimecast-blast-radius'))
• Fixture data for a synthetic KnowBe4 (it-support.care) message and a synthetic BSN (breachsecurenow.com) message, each WITH an authResultsOriginal block showing pass despite authResults showing fail (auth-verdict-precedence pitfall regression coverage) — no real customer data, synthetic only, matching Phase 16's own fixture convention (eml-parser.fixtures.ts)
• Framework install: none — vitest already configured project-wide

---

Manual-Only Verifications

Behavior	Requirement	Why Manual	Test Instructions
POST /api/phishing/campaigns/{id}/classify orchestration + response shape + auth rejection	CLASSIFY-05	No route-handler test precedent in this repo; vitest.config.ts doesn't even discover app/** test files	curl -X POST http://localhost:3100/api/phishing/campaigns/{id}/classify with a valid session cookie against a dev server with a known campaign; confirm response includes verdict/confidence/summary/reasons/recommendedActions/requiresApproval. Repeat with no session cookie (expect 401) and a user-role session (expect 403, since analyze is only granted to admin/super-admin per Phase 18 D-05)

---

Validation Sign-Off

• All tasks have <automated> verify or Wave 0 dependencies (service-layer) or documented manual verification (route-layer)
• Sampling continuity: no 3 consecutive tasks without automated or manual verify
• Wave 0 covers all MISSING references
• No watch-mode flags
• Feedback latency < 5s (automated) — manual route check is a pre-merge/pre-verify gate, not per-commit
• nyquist_compliant: true set in frontmatter

Approval: pending