docs(phase-18): add validation strategy
This commit is contained in:
parent
e05d34640c
commit
1c6d2fdde5
1 changed files with 79 additions and 0 deletions
|
|
@ -0,0 +1,79 @@
|
|||
---
|
||||
phase: 18
|
||||
slug: campaign-grouping-phishing-analysis-api
|
||||
status: draft
|
||||
nyquist_compliant: false
|
||||
wave_0_complete: false
|
||||
created: 2026-07-15
|
||||
---
|
||||
|
||||
# Phase 18 — Validation Strategy
|
||||
|
||||
> Per-phase validation contract for feedback sampling during execution.
|
||||
|
||||
---
|
||||
|
||||
## Test Infrastructure
|
||||
|
||||
| Property | Value |
|
||||
|----------|-------|
|
||||
| **Framework** | vitest 4.1.5 |
|
||||
| **Config file** | `vitest.config.ts` — `include: ['lib/**/*.test.ts']` (does NOT include `app/**` — no route-handler test precedent exists in this repo) |
|
||||
| **Quick run command** | `npx vitest run lib/services/campaign-grouping-service.test.ts` |
|
||||
| **Full suite command** | `npm test` |
|
||||
| **Estimated runtime** | ~2 seconds (mocked postgresClient, no real DB/network) |
|
||||
|
||||
---
|
||||
|
||||
## Sampling Rate
|
||||
|
||||
- **After every task commit:** `npx vitest run lib/services/campaign-grouping-service.test.ts`
|
||||
- **After every plan wave:** `npm test` (full suite)
|
||||
- **Before `/gsd:verify-work`:** Full suite green (modulo the known pre-existing `itglue-search.test.ts` failure) + `npx tsc --noEmit --pretty` clean + manual curl verification of all 3 new routes' auth behavior
|
||||
- **Max feedback latency:** ~2 seconds (automated) / manual curl pass for route auth (no automated route-handler test precedent in this repo)
|
||||
|
||||
---
|
||||
|
||||
## Per-Task Verification Map
|
||||
|
||||
| Task ID | Plan | Wave | Requirement | Threat Ref | Secure Behavior | Test Type | Automated Command | File Exists | Status |
|
||||
|---------|------|------|-------------|------------|-----------------|-----------|-------------------|-------------|--------|
|
||||
| 18-01-0x | 01 | 1 | CAMP-01 | — | Tiered key computation (Message-ID > attach-hash/URL-domain+subject+sender+24h > sender+normalized-subject+client+24h); normalizeSubject() strips Re:/Fwd:/Fw: | unit | `npx vitest run lib/services/campaign-grouping-service.test.ts` | ❌ W0 | ⬜ pending |
|
||||
| 18-01-0x | 01 | 1 | CAMP-02 | — | Second report with matching key increments report_count, updates last_seen_at, links campaign_id, never creates a second campaign | unit (mocked postgresClient) | same file | ❌ W0 | ⬜ pending |
|
||||
| 18-0x-0x | 0x | 0x | DETECT-03 | V4/V5 | POST /analyze orchestrates detect→parse→group, returns campaign linkage, requirePermission('phishing','analyze') rejects unauthorized | manual | curl against dev server (no route-handler test precedent) | n/a | ⬜ pending |
|
||||
| 18-0x-0x | 0x | 0x | CAMP-03 | V4/V5 | GET /campaigns and /campaigns/{id} return expected nested shape, requirePermission('phishing','read') rejects unauthorized | manual | curl against dev server | n/a | ⬜ pending |
|
||||
| 18-0x-0x | 0x | 0x | ACCESS-01 | V4 | Every /api/phishing/* route rejects unauthenticated/unauthorized requests with 401/403 | manual | curl without/with session cookie | n/a | ⬜ pending |
|
||||
|
||||
*Status: ⬜ pending · ✅ green · ❌ red · ⚠️ flaky*
|
||||
*Exact task IDs finalized by the planner — this table's requirement/behavior mapping is fixed by research; row numbering will be updated once PLAN.md exists.*
|
||||
|
||||
---
|
||||
|
||||
## Wave 0 Requirements
|
||||
|
||||
- [ ] `lib/services/campaign-grouping-service.test.ts` — new file, covers CAMP-01/CAMP-02 pure-function and mocked-DB behavior (mirror `phishing-detector.test.ts`/`phishing-eml-service.test.ts`'s `vi.mock('./postgres-client')` approach)
|
||||
- [ ] No new fixtures needed — reuse `eml-parser.fixtures.ts` synthetic fixtures if Message-ID matching needs a fixture value (synthetic only, no real customer `.eml`)
|
||||
- [ ] Framework install: none — vitest already configured and used extensively
|
||||
|
||||
---
|
||||
|
||||
## Manual-Only Verifications
|
||||
|
||||
| Behavior | Requirement | Why Manual | Test Instructions |
|
||||
|----------|-------------|------------|--------------------|
|
||||
| `POST /api/phishing/tickets/{ticket_id}/analyze` orchestration + response shape | DETECT-03 | No route-handler test precedent in this repo; `vitest.config.ts` doesn't even discover `app/**` test files | `curl -X POST http://localhost:3100/api/phishing/tickets/{id}/analyze` with a valid session cookie against a dev server with a known phishing-pattern ticket; confirm response includes campaignId/groupMethod/created |
|
||||
| `GET /api/phishing/campaigns` and `/{id}` nested response shape | CAMP-03 | Same reasoning as above | `curl http://localhost:3100/api/phishing/campaigns` and `/campaigns/{id}` with a valid session cookie; confirm camelCase nested shape (linked reports/messages/indicators/classifications) |
|
||||
| Auth rejection (401/403) on all 3 new routes | ACCESS-01 | Same reasoning as above | `curl` each route with no session cookie (expect 401) and with a `user`-role session against `/analyze` (expect 403, since D-05 only grants `analyze` to admin/super-admin) |
|
||||
|
||||
---
|
||||
|
||||
## Validation Sign-Off
|
||||
|
||||
- [ ] All tasks have `<automated>` verify or Wave 0 dependencies (service-layer) or documented manual verification (route-layer)
|
||||
- [ ] Sampling continuity: no 3 consecutive tasks without automated or manual verify
|
||||
- [ ] Wave 0 covers all MISSING references
|
||||
- [ ] No watch-mode flags
|
||||
- [ ] Feedback latency < 5s (automated) — manual route checks are pre-merge/pre-verify gates, not per-commit
|
||||
- [ ] `nyquist_compliant: true` set in frontmatter
|
||||
|
||||
**Approval:** pending
|
||||
Loading…
Add table
Add a link
Reference in a new issue