chore: check in pending work — queue preferences, QBO AR diagnostics, mobile engagement fixes, ops scripts
Bundles several in-progress efforts that were sitting uncommitted: - User queue-preferences (migration 087, API route, popover component) - QBO invoice soft-delete (migration 088) and AR diagnostics route - Dashboard/mobile engagement route and page adjustments - Docker Compose log-rotation config - One-off ticket/RMM investigation scripts (scripts/) - Planning docs: phase verification/pattern notes, mobile shell design spec - .gitignore: exclude local scratch financial/inventory data and Claude Code worktree/local-settings runtime state (never meant for version control) Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01W6RuWdiUiXrPK6FLBHjtpY
This commit is contained in:
parent
b638189cb0
commit
672f17b7f9
35 changed files with 2801 additions and 92 deletions
146
.claude/skills/pulse-overshell-b2-evidence/SKILL.md
Normal file
146
.claude/skills/pulse-overshell-b2-evidence/SKILL.md
Normal file
|
|
@ -0,0 +1,146 @@
|
||||||
|
---
|
||||||
|
name: pulse-overshell-b2-evidence
|
||||||
|
description: Use when running PowerShell on a Pulse-managed Windows endpoint via Datto RMM Overshell, uploading large output to Backblaze B2, and retrieving the parsed evidence — covers both the stdout-only and B2-transport pipelines, the dispatch API, webhook receiver, presigned download, and where results land in rmm_executions.
|
||||||
|
---
|
||||||
|
|
||||||
|
# Pulse Overshell → B2 → Retrieve
|
||||||
|
|
||||||
|
Pulse has two PowerShell evidence pipelines, both terminating in the same
|
||||||
|
`rmm_executions` table. Pick the right one for the payload size, dispatch
|
||||||
|
via the API, then read the row back.
|
||||||
|
|
||||||
|
## Decision: which transport?
|
||||||
|
|
||||||
|
| Output size | Transport (`transport` column) | How results come back |
|
||||||
|
|---|---|---|
|
||||||
|
| ≤ ~50 KB stdout | `overshell_stdout` (default) | Worker polls Datto job → `raw_stdout` + `parsed_evidence` |
|
||||||
|
| Large (event logs, dumps) | `b2_upload` (LogLift) | Collector gzips → B2 PUT → webhook → Pulse downloads + slims |
|
||||||
|
|
||||||
|
**There is no ad-hoc PowerShell input.** Only registered `RmmScript`s in
|
||||||
|
`lib/services/rmm/scripts/index.ts` can run. To add one, create
|
||||||
|
`lib/services/rmm/scripts/<id>.ts` with a `parseOutput` that JSON-parses
|
||||||
|
a `ConvertTo-Json -Compress` tail, then add to `_all` in `index.ts` and
|
||||||
|
bump `version`. Registry test enforces uniqueness + credential-shape ban.
|
||||||
|
|
||||||
|
## Dispatch (both transports)
|
||||||
|
|
||||||
|
`POST /api/rmm/executions` (requires `rmm.execute` permission — admin/super-admin only):
|
||||||
|
|
||||||
|
```ts
|
||||||
|
// site-anchored (runs on the client's WNP endpoint)
|
||||||
|
{ scriptId: 'get-ad-health', target: { type: 'site_anchor', companyId: 29861375 } }
|
||||||
|
|
||||||
|
// asset-self (runs on a specific Datto device)
|
||||||
|
{ scriptId: 'loglift-eventlogs',
|
||||||
|
target: { type: 'asset_self', deviceUid: 'f7a8…', assetType: 'configuration', assetId: '12345' } }
|
||||||
|
```
|
||||||
|
|
||||||
|
Returns `{ executionId, status: 'queued' | 'running', ... }`. Rate limit:
|
||||||
|
**50 executions per user per 24h** — trips before `runQuickJob` and logs
|
||||||
|
to `analyzer_cost_audit` with `decision='blocked'`. Hard timeout: 5 min.
|
||||||
|
|
||||||
|
## How LogLift uses B2
|
||||||
|
|
||||||
|
For `loglift-eventlogs` (the only B2-transport script today), Pulse
|
||||||
|
passes the collector four `runQuickJob` variables — `RunId`, `ClientId`
|
||||||
|
(Datto site uid), `WebhookUrl` (`${BETTER_AUTH_URL}/api/rmm/loglift/upload`),
|
||||||
|
`WebhookSecret` (`OPENCLAW_API_KEY`). The collector script (registered
|
||||||
|
inside Datto, not stored in Pulse) gzips its JSON and PUTs to:
|
||||||
|
|
||||||
|
```
|
||||||
|
{datto_site_uid}/{computer_name}/eventlogs_{YYYYMMDD_HHMMSS}.json.gz
|
||||||
|
```
|
||||||
|
|
||||||
|
…in bucket `B2_BUCKET` (default `wulf-audits`, region `us-west-002`).
|
||||||
|
Object-key shape is enforced by `OBJECT_KEY_REGEX` in
|
||||||
|
`lib/services/b2/client.ts` — anything else is rejected at the webhook.
|
||||||
|
|
||||||
|
Then it POSTs to `/api/rmm/loglift/upload` with `x-openclaw-key: $WebhookSecret`
|
||||||
|
and the metadata body (`runId`, `clientId`, `computerName`, `objectKey`,
|
||||||
|
`summary`, etc. — see `docs/loglift-eventlog-pipeline-spec.md` for the
|
||||||
|
exact shape). Pulse calls `downloadToBuffer(objectKey)`, gunzips, runs
|
||||||
|
`redact()`, slims to `parsed_evidence`, and flips the row to `complete`.
|
||||||
|
|
||||||
|
## Retrieving results
|
||||||
|
|
||||||
|
### From an executionId you already have
|
||||||
|
|
||||||
|
```ts
|
||||||
|
const r = await fetch(`/api/rmm/executions/${id}`).then(r => r.json());
|
||||||
|
// r.execution.status, .raw_stdout (overshell_stdout), .parsed_evidence (both), .evidence_object_key (b2_upload)
|
||||||
|
```
|
||||||
|
|
||||||
|
### From SQL
|
||||||
|
|
||||||
|
```sql
|
||||||
|
-- Most-recent successful run per (company, script):
|
||||||
|
SELECT DISTINCT ON (target_company_id, script_id)
|
||||||
|
id, target_company_id, script_id, target_hostname, transport,
|
||||||
|
evidence_object_key, completed_at, jsonb_pretty(parsed_evidence) AS parsed
|
||||||
|
FROM rmm_executions
|
||||||
|
WHERE status = 'complete'
|
||||||
|
AND completed_at >= NOW() - INTERVAL '14 days'
|
||||||
|
ORDER BY target_company_id, script_id, completed_at DESC;
|
||||||
|
```
|
||||||
|
|
||||||
|
### Re-fetching the original gzip from B2
|
||||||
|
|
||||||
|
The slim payload is in Postgres; the **full gzip stays in B2 forever** for
|
||||||
|
forensic replay. From a Node script or API route:
|
||||||
|
|
||||||
|
```ts
|
||||||
|
import { downloadToBuffer, presignDownload } from '@/lib/services/b2/client';
|
||||||
|
|
||||||
|
// In-process (capped at 25 MB by MAX_DOWNLOAD_BYTES):
|
||||||
|
const buf = await downloadToBuffer(row.evidence_object_key);
|
||||||
|
|
||||||
|
// Or hand a short-lived URL to a human / external tool:
|
||||||
|
const url = presignDownload(row.evidence_object_key, 600); // 10-minute GET
|
||||||
|
```
|
||||||
|
|
||||||
|
`presignDownload` / `presignUpload` validate `OBJECT_KEY_REGEX` and sign
|
||||||
|
with SigV4 against `B2_KEY_ID` / `B2_APP_KEY` from env. They throw
|
||||||
|
`B2NotConfiguredError` if either is unset.
|
||||||
|
|
||||||
|
## Where things live
|
||||||
|
|
||||||
|
- Dispatch: `lib/services/rmm/executor.ts` (`queueExecution`)
|
||||||
|
- Worker (stdout polling + timeout sweep): `lib/services/rmm/worker.ts`
|
||||||
|
- LogLift receiver: `lib/services/rmm/loglift-receiver.ts` →
|
||||||
|
`app/api/rmm/loglift/upload/route.ts`
|
||||||
|
- B2 SigV4 client: `lib/services/b2/client.ts`
|
||||||
|
- Script registry: `lib/services/rmm/scripts/index.ts`
|
||||||
|
- Settings (component_uids, variable name): `lib/services/rmm/settings.ts` →
|
||||||
|
`/admin/rmm-overshell` for the UI + "Re-discover" buttons
|
||||||
|
- Schema: `migrations/077_rmm_overshell.sql` + `078_loglift_uploads.sql`
|
||||||
|
(adds `transport`, `evidence_object_key`, `run_id`)
|
||||||
|
- Specs: `docs/rmm-overshell-evidence-spec.md`, `docs/loglift-eventlog-pipeline-spec.md`
|
||||||
|
|
||||||
|
## Gotchas
|
||||||
|
|
||||||
|
- **The worker is a side-effect import.** Anything that needs it running
|
||||||
|
must `import '@/lib/services/rmm/worker'` somewhere on the server. The
|
||||||
|
POST `/api/rmm/executions` route already does this. Don't eager-import
|
||||||
|
from hot paths or shared utilities.
|
||||||
|
- **Worker ignores `transport='b2_upload'` rows for stdout polling.** The
|
||||||
|
webhook is the completion event. If the webhook never arrives, the
|
||||||
|
5-minute timeout sweep marks the row `timeout`.
|
||||||
|
- **Redaction runs twice.** `lib/services/analyzer/itglue-redact.ts:redact()`
|
||||||
|
strips anything keyed `/password|secret|key|token|credential|api[_-]?key/i`
|
||||||
|
before persistence and again before the LLM prompt sees it. Don't try
|
||||||
|
to surface those fields — they're gone by the time you read the row.
|
||||||
|
- **B2 download cap is 25 MB hard.** Refuses content-length over the cap
|
||||||
|
and aborts mid-stream if it exceeds. Decompress cap is 100 MB ISIZE
|
||||||
|
(zip-bomb defense).
|
||||||
|
- **Object-key shape is strict.** Custom prefixes won't work — must be
|
||||||
|
`{client_id_or_uuid}/{computer_name}/eventlogs_{timestamp}.json.gz`.
|
||||||
|
If you need a new payload type, add a new key regex + a new transport
|
||||||
|
rather than loosening the existing one.
|
||||||
|
- **Auto-audit fires only on single-match Configurations.** Multi-match
|
||||||
|
hostnames land the evidence but skip the audit (logged with
|
||||||
|
`rmm.loglift.matched`, no `audit_triggered`).
|
||||||
|
- **Webhook is not HMAC-signed** — `x-openclaw-key` is the auth boundary.
|
||||||
|
If you expose the endpoint to a new collector, rotate `OPENCLAW_API_KEY`.
|
||||||
|
- **`B2_*` env vars are required.** `isB2Configured()` checks only
|
||||||
|
`B2_KEY_ID + B2_APP_KEY`; bucket/region/endpoint fall back to
|
||||||
|
`wulf-audits` / `us-west-002` / `s3.us-west-002.backblazeb2.com`.
|
||||||
11
.gitignore
vendored
11
.gitignore
vendored
|
|
@ -44,3 +44,14 @@ routes.ts
|
||||||
|
|
||||||
# fonts
|
# fonts
|
||||||
*.woff2
|
*.woff2
|
||||||
|
|
||||||
|
# local scratch data (real customer/financial data — never commit)
|
||||||
|
dev/fin/
|
||||||
|
dev/seubert-laptops.csv
|
||||||
|
|
||||||
|
# Claude Code local runtime state — agent worktree checkouts, personal
|
||||||
|
# permission overrides, and the scheduler lock are per-machine, not project
|
||||||
|
# config. .claude/skills/ (project skills) is intentionally NOT ignored.
|
||||||
|
.claude/worktrees/
|
||||||
|
.claude/settings.local.json
|
||||||
|
.claude/scheduled_tasks.lock
|
||||||
|
|
|
||||||
121
.planning/phases/16-eml-mime-evidence-parser/16-VERIFICATION.md
Normal file
121
.planning/phases/16-eml-mime-evidence-parser/16-VERIFICATION.md
Normal file
|
|
@ -0,0 +1,121 @@
|
||||||
|
---
|
||||||
|
phase: 16-eml-mime-evidence-parser
|
||||||
|
verified: 2026-07-15T14:44:12Z
|
||||||
|
status: passed
|
||||||
|
score: 8/8 must-haves verified
|
||||||
|
overrides_applied: 0
|
||||||
|
---
|
||||||
|
|
||||||
|
# Phase 16: EML/MIME Evidence Parser Verification Report
|
||||||
|
|
||||||
|
**Phase Goal:** Given a ticket's attachments, Pulse selects the correct original reported message and parses its RFC822/MIME structure into normalized, actionable evidence — without ever executing or fetching anything from the message.
|
||||||
|
**Verified:** 2026-07-15T14:44:12Z
|
||||||
|
**Status:** passed
|
||||||
|
**Re-verification:** No — initial verification
|
||||||
|
|
||||||
|
## Goal Achievement
|
||||||
|
|
||||||
|
### Observable Truths (ROADMAP Success Criteria + PLAN must_haves, merged)
|
||||||
|
|
||||||
|
| # | Truth | Status | Evidence |
|
||||||
|
|---|-------|--------|----------|
|
||||||
|
| 1 | Given synthetic fixtures with both `rfc.eml` and `OriginatingEmail.eml` present, selection picks `rfc.eml` matching by `message/rfc822` content-type, not filename alone | ✓ VERIFIED | `lib/services/eml-parser.ts:50-65` `selectOriginalMessage` filters `isMessageRfc822` first, then matches exact `rfc.eml`; test `eml-parser.test.ts:28-37` asserts tier 1 + case-insensitive tier 1; also covers KnowBe4 tier 2 (`:39-43`) and OriginatingEmail-only tier 3 (`:45-49`) and ambiguous/null cases (`:51-69`). All pass live (`npx vitest run` — 49/49 across the 4 phase test files). |
|
||||||
|
| 2 | Parsing a synthetic `.eml` fixture produces normalized headers (From/displayName/senderEmail/senderDomain/Reply-To/Return-Path/To/Cc/Subject/Date/Message-ID/Received chain, SPF/DKIM/DMARC), URLs, and attachment metadata (name/content-type/size/hash) | ✓ VERIFIED | `parseEml` (`eml-parser.ts:227-282`) builds exactly this shape from `mailparser`'s `ParsedMail`; test `eml-parser.test.ts:137-164` asserts every field against `RICH_MULTIPART_EML`, including `authResults` structured verdicts (not raw text), sha256 checksum format, and `related` flag. Runs against real `simpleParser` (no mocking of the parser itself) — not a tautology. |
|
||||||
|
| 3 | The parser never executes or fetches any URL found in a message — verified by tests asserting no outbound network calls happen during parsing | ✓ VERIFIED | `eml-parser.test.ts:188-197` spies on `global.fetch` across 5 fixture parses and asserts zero calls; `extractUrls`/`buildBodyPreview` are pure string operations (`eml-parser.ts:140-184`) with no fetch/render path. Orchestration-level (service) invariant separately verified below (Truth 6). |
|
||||||
|
| 4 | Parsed output includes a sanitized/truncated body preview stored alongside raw evidence, distinct from the full raw body | ✓ VERIFIED | `buildBodyPreview` (`eml-parser.ts:176-184`) truncates to `MAX_BODY_PREVIEW_LENGTH=500` and strips HTML via `stripHtmlToText`; test `eml-parser.test.ts:128-133,178-181` confirms truncation and distinctness on a 2000-char fixture. Raw bytes are never persisted to Postgres — only to B2 (`raw_ref` is an object key, `phishing-eml-service.ts:97-106,142-157`), consistent with D-05. |
|
||||||
|
| 5 | `npx vitest run` for the new parser test file passes using synthetic fixtures only (no real customer email) | ✓ VERIFIED | Ran live: `eml-parser.test.ts` 26/26 pass. A dedicated test (`eml-parser.test.ts:205-218`) asserts no fixture contains `wulfconsulting.com` and all use RFC 2606 reserved `.test`/`.com`-fake (`evil-example.test`) domains. `deferred-items.md` documents the 2 pre-existing unrelated `itglue-search.test.ts` failures (confirmed via `git log` — last touched at commit `a0a6e7f`/`8f8b5ab`, predating all Phase 16 commits). |
|
||||||
|
| 6 | (16-02/16-03 supporting truth) AutotaskClient.getAttachmentContent reads `response.items?.[0]`, not `.item`; b2 EML_OBJECT_KEY_REGEX enforces path-traversal-safe `.eml` keys in parallel with the untouched LogLift regex; migration 099 adds `indicators.metadata` JSONB | ✓ VERIFIED | `autotask-client.ts:443-456` reads `response.items?.[0] ?? null`; test asserts an `{item:...}`-shaped response yields null (guards the exact regression risk called out in the plan). `b2/client.ts:41-42` adds `EML_OBJECT_KEY_REGEX` beside the unmodified `OBJECT_KEY_REGEX` (`:31-32`); `presignDownload`/`presignUpload`/`downloadToBuffer` (`:155-218`) take an optional `keyRegex` param defaulting to `OBJECT_KEY_REGEX`; `rmm/executor.ts:337`'s existing 2-arg call site still compiles (confirmed via `tsc --noEmit`, exit 0). Migration 099 applied live: `information_schema.columns` on the dev DB (`pulse-postgres`) reports `indicators.metadata` as `jsonb`. |
|
||||||
|
| 7 | Orchestration service (16-03) wires list→select→fetch→(B2 gated)→parse→persist end to end, writing one `messages` row + `indicators` rows with D-07 metadata, and no-op (no throw) when no `.eml` attachment or B2 unconfigured | ✓ VERIFIED | `phishing-eml-service.ts:49-224` imports and calls `getAutotaskClient().getAttachments`, `selectOriginalMessage`, `getAttachmentContent`, `isB2Configured`/`presignUpload`/`EML_OBJECT_KEY_REGEX`, `parseEml` — exact function signatures match Plan 01/02's exports (verified by reading both source files side by side). `messages` INSERT (`:142-157`) and 3 `indicators` INSERT loops (`:164-211`) match migration 097/099's actual column list (`report_id, message_id, headers, urls, attachments, body_preview, raw_ref` / `message_id, indicator_type, value, metadata`). Test suite (`phishing-eml-service.test.ts`, 6/6 passing) exercises the happy path (real `parseEml` invoked on `RICH_MULTIPART_EML`, only DB/Autotask/B2 boundaries mocked — not tautological), no-eml no-op, B2-configured/unconfigured branches, no-network invariant, and indicator metadata shape. |
|
||||||
|
| 8 | No outbound network call is made to any URL found in the message, end to end (service level) | ✓ VERIFIED | `phishing-eml-service.test.ts:155-170` asserts `global.fetch` is called exactly once (the B2 presigned PUT) and never with the fixture's embedded body URL (`http://evil-example.test/verify`). Source review confirms the only `fetch(` call in `phishing-eml-service.ts` targets `uploadUrl` (a B2-presigned URL Pulse itself constructed), never a value derived from `normalized.urls`. |
|
||||||
|
|
||||||
|
**Score:** 8/8 truths verified
|
||||||
|
|
||||||
|
### Required Artifacts
|
||||||
|
|
||||||
|
| Artifact | Expected | Status | Details |
|
||||||
|
|----------|----------|--------|---------|
|
||||||
|
| `lib/services/eml-parser.ts` | parseEml, selectOriginalMessage, parseAuthResults, extractUrls, buildBodyPreview, NormalizedMessage (min 120 lines) | ✓ VERIFIED | 282 lines; all 5 functions + types exported exactly per plan interface contract |
|
||||||
|
| `lib/services/eml-parser.fixtures.ts` | synthetic fixtures for all 3 selection tiers + parse fixtures | ✓ VERIFIED | Exists, imported and exercised by both `eml-parser.test.ts` and `phishing-eml-service.test.ts`; RFC 2606-safe synthetic content only |
|
||||||
|
| `lib/services/eml-parser.test.ts` | EVID-02/03/04 vitest coverage incl. no-network spy + 3-tier selection (min 100 lines) | ✓ VERIFIED | 219 lines, 26 tests, all passing live |
|
||||||
|
| `lib/services/autotask-client.ts` | getAttachmentContent(entityName, entityId, attachmentId) | ✓ VERIFIED | Present at line 443, reads `items?.[0] ?? null` |
|
||||||
|
| `lib/services/autotask-client.test.ts` | first AutotaskClient unit coverage — items[0] behavior (min 30 lines) | ✓ VERIFIED | 73 lines, 3 tests, all passing |
|
||||||
|
| `lib/services/b2/client.ts` | EML_OBJECT_KEY_REGEX + parameterized key validation | ✓ VERIFIED | Present, OBJECT_KEY_REGEX untouched (git-diff-style visual check against current content confirms LogLift regex line unchanged), all 3 functions parameterized |
|
||||||
|
| `migrations/099_indicators_metadata.sql` | ALTER TABLE indicators ADD COLUMN metadata JSONB | ✓ VERIFIED | Present, applied live to dev DB (confirmed via information_schema query) |
|
||||||
|
| `lib/services/phishing-eml-service.ts` | parseAndStoreMessage orchestration (min 90 lines) | ✓ VERIFIED | 224 lines, full list→select→fetch→size-guard→B2→parse→persist flow |
|
||||||
|
| `lib/services/phishing-eml-service.test.ts` | orchestration coverage with mocked autotask/b2/postgres incl. no-network + B2-gated + no-eml no-op (min 70 lines) | ✓ VERIFIED | 195 lines, 6 tests, all passing |
|
||||||
|
|
||||||
|
### Key Link Verification
|
||||||
|
|
||||||
|
| From | To | Via | Status | Details |
|
||||||
|
|------|-----|-----|--------|---------|
|
||||||
|
| `eml-parser.ts` | `mailparser` | `simpleParser` import w/ `checksumAlgo: 'sha256'` | ✓ WIRED | `eml-parser.ts:21,236` |
|
||||||
|
| `eml-parser.test.ts` | `global.fetch` | `vi.spyOn` zero-call assertion | ✓ WIRED | `eml-parser.test.ts:188-197` |
|
||||||
|
| `autotask-client.ts getAttachmentContent` | Autotask `Tickets/{id}/Attachments/{attachmentId}` | `makeApiCall` GET, `items?.[0]` | ✓ WIRED | `autotask-client.ts:443-456`; test confirms both shapes |
|
||||||
|
| `b2/client.ts presignUpload` | `EML_OBJECT_KEY_REGEX` | optional `keyRegex` param | ✓ WIRED | `b2/client.ts:165-173` |
|
||||||
|
| `phishing-eml-service.ts` | `eml-parser.ts` | `parseEml`+`selectOriginalMessage` import | ✓ WIRED | `phishing-eml-service.ts:27`; both invoked in the real flow (not stubbed in production code) |
|
||||||
|
| `phishing-eml-service.ts` | `AutotaskClient.getAttachmentContent` | `getAutotaskClient().getAttachmentContent(...)` | ✓ WIRED | `phishing-eml-service.ts:65-69` |
|
||||||
|
| `phishing-eml-service.ts` | `messages`/`indicators` tables | `postgresClient.query` INSERT ... RETURNING id::text | ✓ WIRED | `phishing-eml-service.ts:142-157,166-211`; column lists match migration 097+099 schema exactly |
|
||||||
|
| `phishing-eml-service.ts` | B2 (`presignUpload`+`EML_OBJECT_KEY_REGEX`) | `isB2Configured` gate then self-PUT | ✓ WIRED | `phishing-eml-service.ts:97-122`; gated, graceful-degrades on failure |
|
||||||
|
|
||||||
|
### Data-Flow Trace (Level 4)
|
||||||
|
|
||||||
|
Not applicable in the traditional UI-rendering sense — this phase is a pure backend service/library. Instead, traced data flow through the orchestration pipeline directly:
|
||||||
|
|
||||||
|
| Stage | Input | Output | Verified Real (not hardcoded) |
|
||||||
|
|-------|-------|--------|-------------------------------|
|
||||||
|
| `getAttachments` → `selectOriginalMessage` | live Autotask attachment list | selected `Attachment \| null` | ✓ Algorithm is a real filter/find chain over the input array, not a static return |
|
||||||
|
| `getAttachmentContent` → `Buffer.from(data,'base64')` | live Autotask base64 payload | decoded raw bytes | ✓ Real base64 decode, test proves items[0]-vs-item distinction |
|
||||||
|
| `parseEml` → `messages` INSERT | real `simpleParser` output | JSONB headers/urls/attachments payload | ✓ `headersPayload` built field-by-field from `normalized.*`, not a static object; test asserts `authResults` reaches the persisted param |
|
||||||
|
| `normalized.attachments/urls/from` → `indicators` INSERT loops | parsed message | per-indicator rows w/ metadata | ✓ Loops iterate real arrays from the parse result; test asserts attachment_hash metadata shape from an actual parsed fixture |
|
||||||
|
|
||||||
|
No hollow/static-return patterns found in the traced path.
|
||||||
|
|
||||||
|
### Behavioral Spot-Checks
|
||||||
|
|
||||||
|
| Behavior | Command | Result | Status |
|
||||||
|
|----------|---------|--------|--------|
|
||||||
|
| Full phase-16 vitest suite | `npx vitest run lib/services/eml-parser.test.ts lib/services/autotask-client.test.ts lib/services/b2/client.test.ts lib/services/phishing-eml-service.test.ts` | 4 files, 49/49 tests passed | ✓ PASS |
|
||||||
|
| Repo-wide type check | `npx tsc --noEmit --pretty` | exit 0, no output | ✓ PASS |
|
||||||
|
| Full test suite (regression check) | `npm test` | 25/26 files pass, 282/284 tests pass; only pre-existing unrelated `itglue-search.test.ts` failures (confirmed via `git log` predating Phase 16 commits) | ✓ PASS (no phase-16 regressions) |
|
||||||
|
| Migration 099 applied to dev DB | `docker exec pulse-postgres psql ... information_schema.columns` | returns `jsonb` | ✓ PASS |
|
||||||
|
| `rmm/executor.ts` presignUpload call site unaffected | `grep presignUpload lib/services/rmm/executor.ts` + tsc clean | 2-arg call, `presignUpload(objectKey, 1800)`, compiles | ✓ PASS |
|
||||||
|
|
||||||
|
### Probe Execution
|
||||||
|
|
||||||
|
No `scripts/*/tests/probe-*.sh` probes declared or discovered for this phase; not a migration/tooling phase in that sense. SKIPPED (no probe files apply — verification instead relied on the project's own vitest suite, run directly by the verifier, not narrated by SUMMARY.md).
|
||||||
|
|
||||||
|
### Requirements Coverage
|
||||||
|
|
||||||
|
| Requirement | Source Plan | Description | Status | Evidence |
|
||||||
|
|-------------|-------------|-------------|--------|----------|
|
||||||
|
| EVID-02 | 16-01 | Prefer `rfc.eml` over `OriginatingEmail.eml`, matching by content-type not filename alone | ✓ SATISFIED | `selectOriginalMessage` three-tier algorithm + full test coverage |
|
||||||
|
| EVID-03 | 16-01, 16-02, 16-03 | Parse RFC822/MIME into normalized headers/auth-verdicts/URLs/attachment metadata, persisted | ✓ SATISFIED | `parseEml` + `messages`/`indicators` persistence, both tested |
|
||||||
|
| EVID-04 | 16-01, 16-02, 16-03 | Sanitized/truncated body preview stored alongside raw evidence; never fetch/execute message content | ✓ SATISFIED | `buildBodyPreview` + B2 raw storage (D-05) + no-network spy at both parser and service level |
|
||||||
|
|
||||||
|
Note: `.planning/REQUIREMENTS.md` still shows EVID-02/03/04 as unchecked/"Pending" — this appears to be a tracking-doc staleness issue (the doc is not updated by the execute-phase workflow), not evidence of non-completion. All three requirements are satisfied by the code and tests as verified above.
|
||||||
|
|
||||||
|
### Anti-Patterns Found
|
||||||
|
|
||||||
|
None. Scanned all 10 phase-modified/created files for `TBD|FIXME|XXX|TODO|HACK|PLACEHOLDER`, "not yet implemented", "coming soon" — zero matches.
|
||||||
|
|
||||||
|
### Deviations Reviewed (from SUMMARY.md, checked against CLAUDE.md conventions)
|
||||||
|
|
||||||
|
1. **16-01: hand-rolled HTML stripper instead of `html-to-text`.** Acceptable — avoids depending on an unpinned transitive dependency; consistent with "don't introduce dependencies not required" project posture; fully test-covered.
|
||||||
|
2. **16-01: test title renamed for `-t` filter discoverability.** Cosmetic, no behavior change.
|
||||||
|
3. **16-02: migration 099 applied via direct `docker exec` instead of `scripts/apply-migrations.sh`.** Consistent with CLAUDE.md's own documented caveat ("check first; behavior varies") — the script's hardcoded `MIGRATIONS_DIR` didn't see the worktree's file; the same real credentials (`pulse_user`/`pulse_autotask`) were used, not fallback defaults. Verified live on the dev DB.
|
||||||
|
4. **16-03: self-PUT to B2 (first instance of Pulse's own server code PUTting to B2, vs. handing a presigned URL to an external collector).** Explicitly anticipated and researched in 16-RESEARCH.md as a deliberate new pattern, not an ad hoc deviation; gated behind `isB2Configured()` with graceful degrade on failure.
|
||||||
|
5. **16-03: Task 1/Task 2 executed as separate implementation-then-test-suite commits rather than interleaved RED/GREEN within one task.** Matches how the plan's own task structure was written (Task 1's verify was tsc-only; Task 2's verify included vitest). No coverage gap — the full suite passes and covers real behavior, not a rubber-stamp.
|
||||||
|
|
||||||
|
None of these deviations reduce scope or introduce risk beyond what's already accepted in the phase's own threat model.
|
||||||
|
|
||||||
|
### Human Verification Required
|
||||||
|
|
||||||
|
None. This phase is a pure backend library + orchestration service with no UI, no new API route, and no external-service live-credential dependency (Autotask/B2/Postgres are all mocked in tests; the live on-demand trigger route is explicitly deferred to Phase 18). All success criteria are mechanically verifiable via source review + live test execution, both performed above.
|
||||||
|
|
||||||
|
### Gaps Summary
|
||||||
|
|
||||||
|
No gaps found. All 3 requirements (EVID-02, EVID-03, EVID-04) are implemented, wired end-to-end from the pure parser (16-01) through supporting infrastructure (16-02) to the orchestration service (16-03), and covered by tests that exercise real parsing logic (not mocked-to-pass tautologies) with mocks confined to true I/O boundaries (Postgres, Autotask HTTP, B2 HTTP). Live verification (not just SUMMARY.md narrative) confirms: 49/49 phase-specific tests pass, `tsc --noEmit` is clean, the dev-DB migration is actually applied, and the pre-existing unrelated `itglue-search.test.ts` failures are correctly out of scope (confirmed via git history predating this phase).
|
||||||
|
|
||||||
|
---
|
||||||
|
_Verified: 2026-07-15T14:44:12Z_
|
||||||
|
_Verifier: Claude (gsd-verifier)_
|
||||||
|
|
@ -0,0 +1,94 @@
|
||||||
|
---
|
||||||
|
phase: 17-mimecast-blast-radius-lookup
|
||||||
|
verified: 2026-07-15T14:35:00Z
|
||||||
|
status: passed
|
||||||
|
score: 5/5 must-haves verified
|
||||||
|
overrides_applied: 0
|
||||||
|
---
|
||||||
|
|
||||||
|
# Phase 17: Mimecast Blast Radius Lookup Verification Report
|
||||||
|
|
||||||
|
**Phase Goal:** Pulse can ask "how far did this message spread" via a Mimecast blast-radius abstraction when Mimecast is configured, and gets a clean `unavailable` signal — never a crash or a block — when it isn't.
|
||||||
|
**Verified:** 2026-07-15T14:35:00Z
|
||||||
|
**Status:** passed
|
||||||
|
**Re-verification:** No — initial verification
|
||||||
|
|
||||||
|
## Goal Achievement
|
||||||
|
|
||||||
|
### Observable Truths
|
||||||
|
|
||||||
|
| # | Truth | Status | Evidence |
|
||||||
|
|---|-------|--------|----------|
|
||||||
|
| 1 | `isMimecastConfigured()` returns true only when both `MIMECAST_CLIENT_ID` and `MIMECAST_CLIENT_SECRET` are set, false otherwise | VERIFIED | `lib/services/mimecast-client.ts:647-649` — `!!(process.env.MIMECAST_CLIENT_ID && process.env.MIMECAST_CLIENT_SECRET)`. Tested in `mimecast-client.test.ts` (4 cases: neither/only-ID/only-secret/both), all pass. |
|
||||||
|
| 2 | `getBlastRadius()` returns `status:'unavailable' reason:'not_configured'` synchronously (no Mimecast call) when unconfigured | VERIFIED | `mimecast-blast-radius.ts:92-95` returns before any client construction. Test asserts `getMimecastClientMock`/all 3 fan-out mocks `.not.toHaveBeenCalled()`. |
|
||||||
|
| 3 | `getBlastRadius()` when configured returns normalized matched/delivered/held/rejected/clicked counts + per-recipient status array, built by fanning out to `searchDeliveredMessages` + `getHeldMessages` + `getThreatEvents` UNCONDITIONALLY (not gated behind a `getMessageInfo` miss — corrected D-01 / Pitfall 1) | VERIFIED | `mimecast-blast-radius.ts:104-128` — `getMessageInfo` (line 109-111) is called only for supplementary body/header evidence when `messageId` present, its result is discarded (not awaited into a variable used downstream), and is NOT inside any conditional that gates the `Promise.all([...])` fan-out at line 118, which always runs. Merge logic at 130-184 builds real counts, not fallback zeros. Test "merges delivered/held/threat-event fixtures..." confirms counts (`matched:2, delivered:1, held:1, rejected:1, clicked:1`) and `perRecipient` array. |
|
||||||
|
| 4 | An unexpected error thrown during the fan-out degrades to `status:'unavailable' reason:'lookup_failed'` rather than propagating | VERIFIED | `mimecast-blast-radius.ts:188-194` catch block. Test "degrades to unavailable/lookup_failed (never throws) when a fan-out call rejects" — `getHeldMessagesMock.mockRejectedValue(...)`, asserts resolved (not rejected) result equals `{status:'unavailable', reason:'lookup_failed', error:'Mimecast API timeout'}`. |
|
||||||
|
| 5 | A repeated lookup for the same message identity within the cache TTL returns the cached result without re-calling any MimecastClient method | VERIFIED | `mimecast-blast-radius.ts:101-102` — `getCachedData` checked and returned BEFORE `getMimecastClient()` is called at line 105. Test "returns the cached result on a cache hit..." asserts all 3 fan-out mocks and `setCachedDataMock` not called. |
|
||||||
|
|
||||||
|
**Score:** 5/5 truths verified
|
||||||
|
|
||||||
|
### Required Artifacts
|
||||||
|
|
||||||
|
| Artifact | Expected | Status | Details |
|
||||||
|
|----------|----------|--------|---------|
|
||||||
|
| `lib/services/mimecast-client.ts` | `isMimecastConfigured()` + `_resetMimecastClient()` config gate / test seam, existing exports untouched | VERIFIED | Lines 647-649 (`isMimecastConfigured`), 651-655 (`_resetMimecastClient`) added directly above unmodified `getMimecastClient()` (657-672) and `getMimecastClientForTenant()` (674+). |
|
||||||
|
| `lib/services/mimecast-blast-radius.ts` | `getBlastRadius()` orchestration + `BlastRadiusInput`/`BlastRadiusResult` types | VERIFIED | 195 lines; exports `getBlastRadius`, `BlastRadiusInput`, `BlastRadiusResult` (discriminated union) exactly as specced. |
|
||||||
|
| `lib/services/mimecast-client.test.ts` | Unit tests for `isMimecastConfigured()` + `getMimecastClient()` throw/cache behavior | VERIFIED | 63 lines, 7 tests, all pass (`npx vitest run` confirmed). |
|
||||||
|
| `lib/services/mimecast-blast-radius.test.ts` | Unit tests for config gate, fan-out merge, never-throw, cache-hit | VERIFIED | 227 lines, 6 tests, all pass. |
|
||||||
|
|
||||||
|
### Key Link Verification
|
||||||
|
|
||||||
|
| From | To | Via | Status | Details |
|
||||||
|
|------|----|----|--------|---------|
|
||||||
|
| `mimecast-blast-radius.ts` | `mimecast-client.ts` | `import { isMimecastConfigured, getMimecastClient, type MimecastDeliveredMessage, type MimecastHeldMessage } from './mimecast-client'` | WIRED | Import present (line 37-42); both interface types confirmed exported from `mimecast-client.ts` (lines 52, 76, 92 — `MimecastThreatEvent`, `MimecastHeldMessage`, `MimecastDeliveredMessage`). |
|
||||||
|
| `mimecast-blast-radius.ts` | `redis-client.ts` | `import { getCachedData, setCachedData } from './redis-client'` | WIRED | Import present (line 43); both functions called and awaited correctly (cache-check before client construction, cache-write after successful merge only). |
|
||||||
|
|
||||||
|
### D-05 Multi-Tenant Comment Verification
|
||||||
|
|
||||||
|
`grep -qi "D-05" lib/services/mimecast-blast-radius.ts` → present. The module doc-comment (lines 25-34) explicitly states: "KNOWN LIMITATION — MULTI-TENANT GAP (D-05): this module uses only the single global env-var-configured getMimecastClient(), NOT the per-company `mimecast_tenants` table / getMimecastClientForTenant(). Reports belonging to companies with their own registered Mimecast tenant ... will return `status: 'unavailable'`..." This is a genuine code comment in the shipped file, not just a plan/summary claim.
|
||||||
|
|
||||||
|
### Pitfall-Avoidance Verification (17-RESEARCH.md's 3 documented pitfalls)
|
||||||
|
|
||||||
|
| Pitfall | Research Concern | Shipped-Code Verification |
|
||||||
|
|---------|------------------|---------------------------|
|
||||||
|
| #1 — `getMessageInfo()` has no status/counts; fan-out must be unconditional, not gated on a miss | Implementer might skip fan-out when `getMessageInfo` hits | Confirmed avoided: `getMessageInfo` call (line 110) is a bare `await` whose return value is discarded; the `Promise.all` fan-out (lines 118-128) is unconditional — no `if` branch separates "exact match" from "fallback." Code comment at line 107-108 explicitly documents why. |
|
||||||
|
| #2 — `getThreatEvents()` click derivation is best-effort, not confirmed-zero | Risk of overstating confidence in `clicked: 0` | Confirmed avoided: doc-comment (b) at lines 17-23 states `clicked: 0` means "no click-type threat event found... NOT confirmed zero clicks." `isClickEvent()` helper (line 88-90) comment references D-02 explicitly. |
|
||||||
|
| #3 — Multi-tenant gap silently missed | Risk of a silent single-tenant assumption | Confirmed avoided: D-05 comment present and specific (see above); T-17-03 in the plan's threat model documents the accepted risk; not touched by this phase per explicit scope. |
|
||||||
|
|
||||||
|
### Requirements Coverage
|
||||||
|
|
||||||
|
| Requirement | Source Plan | Description | Status | Evidence |
|
||||||
|
|-------------|-------------|-------------|--------|----------|
|
||||||
|
| BLAST-01 | 17-01-PLAN.md | Query blast-radius abstraction for delivery data when configured, keyed on message ID/sender/recipient/subject/date-window | SATISFIED | `getBlastRadius()` implemented and tested per Truth #3 above; ROADMAP.md and REQUIREMENTS.md both mark BLAST-01 `[x]`/`Complete`. |
|
||||||
|
| BLAST-02 | 17-01-PLAN.md | Not configured → `status: unavailable`, never blocks; unexpected error also degrades | SATISFIED | Truths #2 and #4 above; both paths tested with explicit call-count/rejection assertions. |
|
||||||
|
|
||||||
|
No orphaned requirements found for Phase 17 in REQUIREMENTS.md (only BLAST-01/BLAST-02 map to this phase).
|
||||||
|
|
||||||
|
### Anti-Patterns Found
|
||||||
|
|
||||||
|
None. Scanned all 4 phase-modified/created files (`mimecast-client.ts` diff region, `mimecast-blast-radius.ts`, `mimecast-blast-radius.test.ts`, `mimecast-client.test.ts`) for `TBD|FIXME|XXX|TODO|HACK|PLACEHOLDER|placeholder|not yet implemented` — zero matches.
|
||||||
|
|
||||||
|
### Behavioral Spot-Checks
|
||||||
|
|
||||||
|
| Behavior | Command | Result | Status |
|
||||||
|
|----------|---------|--------|--------|
|
||||||
|
| New unit tests pass in isolation | `npx vitest run lib/services/mimecast-blast-radius.test.ts lib/services/mimecast-client.test.ts` | 2 files, 13 tests, all passed | PASS |
|
||||||
|
| Type-check clean | `npx tsc --noEmit --pretty` | No output / exit 0 | PASS |
|
||||||
|
| Full suite has no new regressions | `npm test` | 27 passed / 1 failed file (`itglue-search.test.ts`, 2 tests) — confirmed pre-existing via `git log` (`a0a6e7f`, predates this phase's commits `8b032c3`/`efbc437`) and unrelated to any file this phase touched | PASS (pre-existing failure correctly excluded per task instructions) |
|
||||||
|
|
||||||
|
### Test Isolation / Mocking Verification
|
||||||
|
|
||||||
|
Both test files declare `vi.mock('./mimecast-client', ...)` / `vi.mock('./redis-client', ...)` (blast-radius test) and import the real `mimecast-client.ts` module only for the config-gate/factory tests (which intentionally exercise the real singleton with env-var manipulation and `_resetMimecastClient()` — no network egress since `getMimecastClient()` only constructs the object, doesn't call out). No `MIMECAST_CLIENT_ID`/`MIMECAST_CLIENT_SECRET` real credentials are referenced; no live HTTP calls are made in either test file (confirmed by reading both files in full — no `fetch`/`request`/network imports present). Redis is fully mocked (`getCachedDataMock`/`setCachedDataMock`).
|
||||||
|
|
||||||
|
### Human Verification Required
|
||||||
|
|
||||||
|
None. This phase produces no UI, no HTTP route, and no externally-observable runtime behavior beyond the unit-testable function contract — all Success Criteria are objectively verifiable via code + tests.
|
||||||
|
|
||||||
|
### Gaps Summary
|
||||||
|
|
||||||
|
No gaps found. All 5 derived truths verified, both roadmap Success Criteria requirements (BLAST-01, BLAST-02) satisfied, all 3 documented research pitfalls confirmed avoided in the shipped code (not just claimed in SUMMARY.md), D-05 comment confirmed present in the actual file, cache-short-circuit-before-client-call confirmed via call-count assertions, and the full test suite has zero new regressions (the 2 failing `itglue-search.test.ts` tests are confirmed pre-existing and unrelated).
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
*Verified: 2026-07-15T14:35:00Z*
|
||||||
|
*Verifier: Claude (gsd-verifier)*
|
||||||
|
|
@ -0,0 +1,105 @@
|
||||||
|
---
|
||||||
|
phase: 20-remediation-approval-audit-safety
|
||||||
|
verified: 2026-07-16T11:00:00Z
|
||||||
|
status: passed
|
||||||
|
score: 6/6 must-haves verified
|
||||||
|
overrides_applied: 0
|
||||||
|
---
|
||||||
|
|
||||||
|
# Phase 20: Remediation Approval & Audit Safety Verification Report
|
||||||
|
|
||||||
|
**Phase Goal:** Deliver a remediation/approval/audit-safety layer where recommended
|
||||||
|
remediation actions are proposed-only until an operator explicitly approves them,
|
||||||
|
remediation execution is idempotent and gated, false-positive marking is
|
||||||
|
conflict-guarded, and every state-changing action (classify/approve/remediate/
|
||||||
|
mark-false-positive) is audit-logged and permission-gated.
|
||||||
|
|
||||||
|
**Verified:** 2026-07-16T11:00:00Z
|
||||||
|
**Status:** passed
|
||||||
|
**Re-verification:** No — initial verification
|
||||||
|
|
||||||
|
## Goal Achievement
|
||||||
|
|
||||||
|
### Observable Truths
|
||||||
|
|
||||||
|
| # | Truth | Status | Evidence |
|
||||||
|
|---|-------|--------|----------|
|
||||||
|
| 1 | REMED-01: Recommended actions are `proposed`, never auto-executed | VERIFIED | `lib/services/campaign-classifier.ts` has no INSERT into `remediation_actions`; `remediation_actions.status` defaults to `'proposed'` in migration 097; only `approveRemediationActions` creates `'approved'` rows, only on explicit operator call. |
|
||||||
|
| 2 | REMED-02: Operator can approve via `POST /approve`, recording approver, timestamp, exact params | VERIFIED | `app/api/phishing/campaigns/[id]/approve/route.ts` gates `requirePermission('phishing','approve')`, captures actor from `session.user.email` (never request body), delegates to `approveRemediationActions` which inserts `approved_by=actor, approved_at=NOW(), params=$3::jsonb` per action, validated against latest `classifications.recommended_actions`. Unit tests confirm reject-if-not-recommended and reject-if-no-classification. |
|
||||||
|
| 3 | REMED-03: `POST /remediate` proceeds only for approved actions, non-destructive-by-default (simulated), never silently succeeds | VERIFIED | `remediateApprovedActions` throws `RemediationValidationError` when zero `remediation_actions` rows exist (route maps to 400). Only `status='approved'` rows transition; effect is simulated (`UPDATE ... status='completed'`, no external provider call per D-01, confirmed via grep — 0 matches for `not_implemented` outside comments and no HTTP/client calls to Mimecast/other providers in this file). |
|
||||||
|
| 4 | REMED-04: Re-running remediation is idempotent | VERIFIED | `remediateApprovedActions` selects `FOR UPDATE` and only acts on `status='approved'` rows; already-`completed` rows are skipped (no UPDATE, no audit write). Test `is idempotent: a second call transitions nothing and writes no second audit row` asserts 0 additional UPDATE calls and audit-write count stays at 2 across two calls. |
|
||||||
|
| 5 | REMED-05: Operator can mark false positive via API, blocked when remediation approved/completed exists | VERIFIED | `mark-false-positive/route.ts` gated `requirePermission('phishing','approve')`, delegates to `markCampaignFalsePositive`, which runs a `SELECT ... WHERE status IN ('approved','completed') FOR UPDATE` guard before any write; a hit throws `RemediationConflictError` -> route returns 409. Test confirms guard trips and confirms happy-path sets `campaigns.status='false_positive'`. |
|
||||||
|
| 6 | REMED-06: Every state-changing action (classify/approve/remediate/mark-false-positive) writes an audit event | VERIFIED | `writeAuditEvent` is the single INSERT path into `audit_events` (grep confirms no other file inserts into this table). All three remediation-service functions call it inside the same `postgresClient.transaction` as their state write (atomic — rollback discards both). `classify/route.ts` was edited to call `writeAuditEvent({..., eventType:'campaign_classified'})` after `classifyCampaign()` succeeds, closing the fourth action. |
|
||||||
|
|
||||||
|
**Score:** 6/6 truths verified
|
||||||
|
|
||||||
|
### Required Artifacts
|
||||||
|
|
||||||
|
| Artifact | Expected | Status | Details |
|
||||||
|
|----------|----------|--------|---------|
|
||||||
|
| `lib/services/phishing-audit.ts` | `writeAuditEvent(input, client?)` single audit insert path | VERIFIED | 55 lines, parameterized INSERT (`$1`-`$4`), returns `RETURNING id::text`, accepts optional transaction client. |
|
||||||
|
| `lib/services/remediation-service.ts` | approve/remediate/mark-false-positive + typed errors | VERIFIED | 279 lines. Exports `approveRemediationActions`, `remediateApprovedActions`, `markCampaignFalsePositive`, `RemediationValidationError`, `RemediationConflictError` — all present and match plan signatures. |
|
||||||
|
| `lib/services/remediation-service.test.ts` | idempotency/audit/D-04/recommended-only proofs | VERIFIED | 259 lines, 8 real behavioral test cases (not tautological — fake transaction client records actual SQL calls and asserts on them). |
|
||||||
|
| `lib/services/phishing-audit.test.ts` | writer proof | VERIFIED | 3 tests: standalone insert, injected-client routing. |
|
||||||
|
| `lib/permissions.ts` | grants approve+remediate to admin/super-admin only | VERIFIED | `superAdminRole` and `adminRole` both `phishing: ["read","analyze","approve","remediate"]`; `userRole` remains `phishing: ["read"]`. `hasPermission()` enforces this at runtime (not just declared) — verified by reading `lib/auth-utils.ts` `requirePermission()` which calls `hasPermission(userRole, resource, action)` and returns 403 on failure. |
|
||||||
|
| `app/api/phishing/campaigns/[id]/approve/route.ts` | POST approve endpoint | VERIFIED | 77 lines. Permission gate, UUID guard, JSON body validation, actor from session, error mapping (400/409/500), delegates to service. |
|
||||||
|
| `app/api/phishing/campaigns/[id]/remediate/route.ts` | POST remediate endpoint | VERIFIED | 64 lines. Same shell, `phishing:remediate` gate, no body required. |
|
||||||
|
| `app/api/phishing/campaigns/[id]/mark-false-positive/route.ts` | POST mark-false-positive endpoint | VERIFIED | 76 lines. `phishing:approve` gate, optional-body tolerant JSON parse, 409 on conflict. |
|
||||||
|
| `app/api/phishing/campaigns/[id]/classify/route.ts` | audit event wiring | VERIFIED | Edited to destructure `session`, import `writeAuditEvent`, call it post-classify with `eventType:'campaign_classified'`. |
|
||||||
|
|
||||||
|
### Key Link Verification
|
||||||
|
|
||||||
|
| From | To | Via | Status | Details |
|
||||||
|
|------|-----|-----|--------|---------|
|
||||||
|
| `remediation-service.ts` | `audit_events` | `writeAuditEvent(client)` inside `postgresClient.transaction` | WIRED | All three functions call `writeAuditEvent(..., client)` using the same transaction client as their state write — atomic. |
|
||||||
|
| `remediation-service.ts` | `classifications.recommended_actions` | latest-classification read validates approvable types | WIRED | `approveRemediationActions` selects `ORDER BY created_at DESC LIMIT 1` and rejects non-recommended action types before any insert. |
|
||||||
|
| `remediation-service.ts` | `remediation_actions.status` | `FOR UPDATE` status filter drives idempotent completion | WIRED | Confirmed via grep + test (`FOR UPDATE` present in both `remediateApprovedActions` and the D-04 guard in `markCampaignFalsePositive`). |
|
||||||
|
| `approve/route.ts` | `approveRemediationActions` | `requirePermission('phishing','approve')` -> service delegation | WIRED | Route imports and calls the function directly with `(id, actions, actor)`; not a stub — errors from the service propagate to typed HTTP status mapping. |
|
||||||
|
| `remediate/route.ts` | `remediateApprovedActions` | `requirePermission('phishing','remediate')` -> service delegation | WIRED | Same pattern confirmed. |
|
||||||
|
| `mark-false-positive/route.ts` | `markCampaignFalsePositive` | `requirePermission('phishing','approve')` -> service delegation | WIRED | Same pattern confirmed. |
|
||||||
|
| `classify/route.ts` | `writeAuditEvent` | post-classify audit write | WIRED | Call is inside the existing try block after `classifyCampaign(id)` succeeds. |
|
||||||
|
| routes | `middleware.ts` | auth gate | WIRED | `/api/phishing/*` is NOT in `middleware.ts`'s public-route allowlist (confirmed via grep — zero matches), so unauthenticated requests are redirected/blocked before reaching route-level `requirePermission`. |
|
||||||
|
|
||||||
|
### Behavioral Spot-Checks
|
||||||
|
|
||||||
|
| Behavior | Command | Result | Status |
|
||||||
|
|----------|---------|--------|--------|
|
||||||
|
| Service unit tests (11 assertions across approve/remediate/mark-fp/audit-writer) | `npx vitest run lib/services/phishing-audit.test.ts lib/services/remediation-service.test.ts` | 2 files, 11 tests, all passed | PASS |
|
||||||
|
| Type check | `npx tsc --noEmit --pretty` | clean, no output | PASS |
|
||||||
|
| Full test suite regression | `npx vitest run` | 369/371 passing; 2 failures isolated to `lib/services/analyzer/itglue-search.test.ts` | PASS (pre-existing, out-of-scope failure confirmed — last touched by commit `8f8b5ab`, an unrelated earlier commit, not part of this phase's diff) |
|
||||||
|
| No auto-execution of remediation on classify | `grep -n "remediation_actions" lib/services/campaign-classifier.ts` | no matches | PASS |
|
||||||
|
| Single audit insert path | `grep -rn "INSERT INTO audit_events"` across `lib/` and `app/` | only in `phishing-audit.ts` | PASS |
|
||||||
|
|
||||||
|
### Requirements Coverage
|
||||||
|
|
||||||
|
| Requirement | Source Plan | Description | Status | Evidence |
|
||||||
|
|-------------|------------|-------------|--------|----------|
|
||||||
|
| REMED-01 | 20-01 | Proposed-only, never auto-executed | SATISFIED | See Truth 1 |
|
||||||
|
| REMED-02 | 20-01, 20-02 | Approve via API, records approver/timestamp/params | SATISFIED | See Truth 2 |
|
||||||
|
| REMED-03 | 20-01, 20-02 | Remediate only proceeds for approved, non-destructive, never silent success | SATISFIED | See Truth 3 |
|
||||||
|
| REMED-04 | 20-01 | Idempotent re-run | SATISFIED | See Truth 4 |
|
||||||
|
| REMED-05 | 20-01, 20-02 | Mark false positive via API | SATISFIED | See Truth 5 |
|
||||||
|
| REMED-06 | 20-01, 20-02 | Every state-changing action audited | SATISFIED | See Truth 6 |
|
||||||
|
|
||||||
|
Note: `.planning/REQUIREMENTS.md` still lists REMED-01..06 as "Pending" in its coverage table (lines 198-203) and unchecked (`[ ]`) in the requirement list — this is a tracking-document staleness issue, not a code gap. Recommend updating REQUIREMENTS.md status table as a follow-up, but it does not block phase goal achievement since the underlying code is verified.
|
||||||
|
|
||||||
|
### Anti-Patterns Found
|
||||||
|
|
||||||
|
None found in the phase's modified/created files. No `TODO`/`FIXME`/`HACK`/`PLACEHOLDER` markers, no empty handlers, no hardcoded empty returns feeding into rendering, no `not_implemented` code paths (only descriptive prose in comments, explicitly called out in 20-01-SUMMARY.md as a deliberate wording fix to avoid tripping the D-01 grep check).
|
||||||
|
|
||||||
|
### Human Verification Required
|
||||||
|
|
||||||
|
None. This phase is service + API layer only (no new UI in this phase — Phase 22 covers the approval UI). All behaviors are verifiable via code inspection, unit tests, and static grep checks.
|
||||||
|
|
||||||
|
### Gaps Summary
|
||||||
|
|
||||||
|
No gaps found. All six REMED requirements are backed by real, non-stub implementations:
|
||||||
|
- The service layer (`phishing-audit.ts`, `remediation-service.ts`) performs real parameterized SQL, real transactional atomicity between state writes and audit writes, and real validation logic (not pass-through stubs).
|
||||||
|
- The route layer enforces real permission checks (`requirePermission` calls `hasPermission` against actual role definitions, returning 403 on failure — not just declaring permissions without enforcing them).
|
||||||
|
- Idempotency and the D-04 conflict guard are proven by unit tests that assert on actual SQL call counts and audit-write counts, not just "does not throw."
|
||||||
|
- Actor identity is derived from the server session in all three new routes and in the classify route edit — never from request body, closing the spoofing/repudiation threat noted in the phase's own threat model.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
_Verified: 2026-07-16T11:00:00Z_
|
||||||
|
_Verifier: Claude (gsd-verifier)_
|
||||||
364
.planning/phases/21-autotask-triage-note/21-PATTERNS.md
Normal file
364
.planning/phases/21-autotask-triage-note/21-PATTERNS.md
Normal file
|
|
@ -0,0 +1,364 @@
|
||||||
|
# Phase 21: Autotask Triage Note - Pattern Map
|
||||||
|
|
||||||
|
**Mapped:** 2026-07-16
|
||||||
|
**Files analyzed:** 2 new (route + service), 1 optional (service test)
|
||||||
|
**Analogs found:** 2 / 2 (both exact/near-exact structural matches)
|
||||||
|
|
||||||
|
## File Classification
|
||||||
|
|
||||||
|
| New/Modified File | Role | Data Flow | Closest Analog | Match Quality |
|
||||||
|
|-------------------|------|-----------|----------------|---------------|
|
||||||
|
| `app/api/phishing/campaigns/[id]/triage-note/route.ts` (new) | controller (route) | request-response | `app/api/phishing/campaigns/[id]/classify/route.ts` | exact |
|
||||||
|
| `lib/services/triage-note-service.ts` (new — planner may name differently) | service | CRUD (read-many) + file/external-write (Autotask `TicketNotes` POST per linked ticket) | `lib/services/campaign-classifier.ts` (evidence gathering half) + `lib/services/workflow-engine.ts` `runAiTroubleshooting` (Autotask write half) | role-match (composite — no single existing file does both halves) |
|
||||||
|
| `lib/services/triage-note-service.test.ts` (optional, if planner follows sibling-test convention) | test | n/a | `lib/services/remediation-service.test.ts` / `lib/services/campaign-classifier.test.ts` | role-match |
|
||||||
|
|
||||||
|
## Pattern Assignments
|
||||||
|
|
||||||
|
### `app/api/phishing/campaigns/[id]/triage-note/route.ts` (controller, request-response)
|
||||||
|
|
||||||
|
**Analog:** `app/api/phishing/campaigns/[id]/classify/route.ts` (full file read — 67 lines)
|
||||||
|
|
||||||
|
This is a near-identical structural twin. Copy the whole shape: UUID guard,
|
||||||
|
`requirePermission`, campaign-exists pre-check, service delegation, try/catch
|
||||||
|
with typed error branches, `console.error` with a `[PHISHING-*]` tag.
|
||||||
|
|
||||||
|
**Imports pattern** (lines 12-16):
|
||||||
|
```typescript
|
||||||
|
import { NextRequest, NextResponse } from 'next/server';
|
||||||
|
import { requirePermission } from '@/lib/auth-utils';
|
||||||
|
import postgresClient from '@/lib/services/postgres-client';
|
||||||
|
import { classifyCampaign } from '@/lib/services/campaign-classifier';
|
||||||
|
import { writeAuditEvent } from '@/lib/services/phishing-audit';
|
||||||
|
```
|
||||||
|
For the new route, swap the service import for the new triage-note service
|
||||||
|
export (e.g. `generateAndPostTriageNote`) — `writeAuditEvent` is optional here
|
||||||
|
(no explicit audit event is required by CONTEXT.md D-01..D-06 for this phase;
|
||||||
|
if the planner wants one, `remediation-service.ts`'s in-transaction audit
|
||||||
|
pattern is the reference — see Shared Patterns below).
|
||||||
|
|
||||||
|
**UUID guard + permission gate** (lines 18, 20-32):
|
||||||
|
```typescript
|
||||||
|
const UUID_RE = /^[0-9a-f]{8}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{12}$/i;
|
||||||
|
|
||||||
|
export async function POST(
|
||||||
|
request: NextRequest,
|
||||||
|
{ params }: { params: Promise<{ id: string }> }
|
||||||
|
) {
|
||||||
|
const { session, error } = await requirePermission('phishing', 'analyze');
|
||||||
|
if (error) return error;
|
||||||
|
|
||||||
|
const { id } = await params;
|
||||||
|
// V5: validate UUID shape before querying — a malformed id would otherwise
|
||||||
|
// surface as an unhandled Postgres error -> uncaught 500.
|
||||||
|
if (!UUID_RE.test(id)) {
|
||||||
|
return NextResponse.json({ error: 'Invalid campaign id' }, { status: 400 });
|
||||||
|
}
|
||||||
|
```
|
||||||
|
Per CONTEXT.md's deferred discretion note, `'analyze'` (not `'approve'`) is
|
||||||
|
the recommended permission tier — matches `classify`'s tier since this is
|
||||||
|
informational, not a destructive state change.
|
||||||
|
|
||||||
|
**Campaign-exists pre-check + service delegation + response** (lines 34-43, 58):
|
||||||
|
```typescript
|
||||||
|
try {
|
||||||
|
const campaignRes = await postgresClient.query<{ id: string }>(
|
||||||
|
`SELECT id FROM campaigns WHERE id = $1`,
|
||||||
|
[id]
|
||||||
|
);
|
||||||
|
if (!campaignRes.rows[0]) {
|
||||||
|
return NextResponse.json({ error: 'Campaign not found' }, { status: 404 });
|
||||||
|
}
|
||||||
|
|
||||||
|
const result = await classifyCampaign(id); // -> generateAndPostTriageNote(id)
|
||||||
|
|
||||||
|
return NextResponse.json(result);
|
||||||
|
```
|
||||||
|
D-06's response shape (note text + per-ticket `posted`/error status list)
|
||||||
|
should be returned directly as the service's return value — no reshaping
|
||||||
|
needed in the route, matching how `classify`/`approve`/`remediate` all just
|
||||||
|
`NextResponse.json(result)` the service's return type verbatim.
|
||||||
|
|
||||||
|
**Error handling pattern** (lines 59-65 — the ONLY error branch this route
|
||||||
|
needs, since D-05 says individual write failures are captured *inside* the
|
||||||
|
service's return value, not thrown):
|
||||||
|
```typescript
|
||||||
|
} catch (err) {
|
||||||
|
console.error('[PHISHING-CLASSIFY] Failed to classify campaign', id, err);
|
||||||
|
return NextResponse.json(
|
||||||
|
{ error: 'Failed to classify campaign', message: err instanceof Error ? err.message : 'Unknown error' },
|
||||||
|
{ status: 500 }
|
||||||
|
);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
```
|
||||||
|
Rename the log tag (e.g. `[PHISHING-TRIAGE-NOTE]`) and message. This catch
|
||||||
|
block should only ever fire for a whole-request failure (e.g. campaign
|
||||||
|
evidence-gathering itself throws) — NOT for a single ticket's Autotask write
|
||||||
|
failing, which D-05/D-06 require to be caught per-ticket inside the service
|
||||||
|
and reported in the 200 response body instead.
|
||||||
|
|
||||||
|
**Optional: typed-error branches** if the service throws domain errors (see
|
||||||
|
`approve/route.ts` lines 63-69 for the pattern, not strictly needed here since
|
||||||
|
this phase has no validation-conflict states like approve/remediate do):
|
||||||
|
```typescript
|
||||||
|
if (err instanceof RemediationValidationError) {
|
||||||
|
return NextResponse.json({ error: err.message }, { status: 400 });
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
### `lib/services/triage-note-service.ts` (service, CRUD-read + external-write)
|
||||||
|
|
||||||
|
No single existing file does both halves this service needs, so it composes
|
||||||
|
two analogs: **evidence gathering** (read side, copy shape from
|
||||||
|
`campaign-classifier.ts`'s `gatherCampaignEvidence`) and **Autotask note
|
||||||
|
write** (write side, copy verbatim from `workflow-engine.ts`'s
|
||||||
|
`runAiTroubleshooting`).
|
||||||
|
|
||||||
|
**Imports pattern** — composite of `campaign-classifier.ts` (lines 17-19) and
|
||||||
|
the Autotask factory used across `workflow-engine.ts`:
|
||||||
|
```typescript
|
||||||
|
import { postgresClient } from './postgres-client';
|
||||||
|
import { getBlastRadius, type BlastRadiusResult } from './mimecast-blast-radius';
|
||||||
|
import { getAutotaskClient } from './autotask-factory';
|
||||||
|
import type { TicketNote } from '@/lib/types/autotask';
|
||||||
|
```
|
||||||
|
|
||||||
|
**Read-side pattern — bulk-fetch linked reports, then classification +
|
||||||
|
remediation state** (`campaign-classifier.ts` lines 270-296, adapted; also see
|
||||||
|
`app/api/phishing/campaigns/[id]/route.ts` lines 84-96 for the same
|
||||||
|
`reports WHERE campaign_id = $1 ORDER BY created_at ASC` bulk-fetch shape used
|
||||||
|
a third time in this codebase):
|
||||||
|
```typescript
|
||||||
|
const reportsRes = await postgresClient.query<ReportDbRow>(
|
||||||
|
`SELECT r.id::text AS id, r.ticket_id::text AS ticket_id, r.ticket_number,
|
||||||
|
r.title, r.created_at::text AS created_at,
|
||||||
|
c.email_address AS requester_email
|
||||||
|
FROM reports r
|
||||||
|
LEFT JOIN contacts c ON c.id = r.requester_contact_id
|
||||||
|
WHERE r.campaign_id = $1
|
||||||
|
ORDER BY r.created_at ASC`,
|
||||||
|
[campaignId]
|
||||||
|
);
|
||||||
|
```
|
||||||
|
|
||||||
|
**Most-recent classification** (`remediation-service.ts` lines 89-96 — same
|
||||||
|
`ORDER BY created_at DESC LIMIT 1` idiom used for "current" state per D-04):
|
||||||
|
```typescript
|
||||||
|
const classificationRes = await client.query<ClassificationRow>(
|
||||||
|
`SELECT recommended_actions
|
||||||
|
FROM classifications
|
||||||
|
WHERE campaign_id = $1
|
||||||
|
ORDER BY created_at DESC
|
||||||
|
LIMIT 1`,
|
||||||
|
[campaignId]
|
||||||
|
);
|
||||||
|
```
|
||||||
|
For the triage note, select the full row (`verdict, confidence, summary,
|
||||||
|
reasons, recommended_actions, requires_approval, created_at`), not just
|
||||||
|
`recommended_actions` — D-04 requires verdict/confidence/summary/reasons in
|
||||||
|
the note.
|
||||||
|
|
||||||
|
**Current remediation_actions state** (D-04 — "proposed only if no operator
|
||||||
|
has acted, else approved/completed rows") — same table/columns
|
||||||
|
`remediation-service.ts` already reads/writes (lines 133-148, 175-179):
|
||||||
|
```typescript
|
||||||
|
const remediationRes = await postgresClient.query<RemediationActionRow>(
|
||||||
|
`SELECT id::text, action_type, status, approved_by, approved_at::text
|
||||||
|
FROM remediation_actions
|
||||||
|
WHERE campaign_id = $1
|
||||||
|
ORDER BY created_at ASC`,
|
||||||
|
[campaignId]
|
||||||
|
);
|
||||||
|
```
|
||||||
|
|
||||||
|
**Blast radius — reuse the exact `getBlastRadius()` call shape** from
|
||||||
|
`campaign-classifier.ts` lines 332-351 (D-04/Claude's-Discretion: planner may
|
||||||
|
call fresh or reuse Phase 19's persisted `reasons` — either way this is the
|
||||||
|
call signature to copy if calling fresh):
|
||||||
|
```typescript
|
||||||
|
const blastRadius = await getBlastRadius({
|
||||||
|
sender: senderIndicator?.value ?? primaryMessage?.from.email ?? '',
|
||||||
|
recipient: primaryReport.requesterEmail ?? '',
|
||||||
|
subject: primaryMessage?.subject ?? primaryReport.title ?? '',
|
||||||
|
dateWindow: {
|
||||||
|
start: new Date(createdAt.getTime() - 24 * 60 * 60 * 1000),
|
||||||
|
end: new Date(createdAt.getTime() + 24 * 60 * 60 * 1000),
|
||||||
|
},
|
||||||
|
});
|
||||||
|
// BlastRadiusResult is a discriminated union — status: 'ok' | 'unavailable'.
|
||||||
|
// D-04 requires an explicit "unavailable" string in the note when this
|
||||||
|
// branch is hit, never a silent omission.
|
||||||
|
```
|
||||||
|
|
||||||
|
**Write-side pattern — one `createEntity('TicketNotes', ...)` call per linked
|
||||||
|
ticket, copied verbatim from `workflow-engine.ts` lines 581-589**:
|
||||||
|
```typescript
|
||||||
|
const client = getAutotaskClient();
|
||||||
|
await client.createEntity('TicketNotes', {
|
||||||
|
ticketID: ticket.id, // -> report.ticketId for each linked report (D-01)
|
||||||
|
title: 'Troubleshooting Steps (Auto-Generated)', // -> e.g. 'Phishing Triage Summary'
|
||||||
|
description: steps, // -> the generated sanitized note text
|
||||||
|
noteType: 1, // Internal
|
||||||
|
publish: 1,
|
||||||
|
});
|
||||||
|
```
|
||||||
|
`TicketNote` interface for reference (`lib/types/autotask.ts` lines 185-196):
|
||||||
|
```typescript
|
||||||
|
export interface TicketNote {
|
||||||
|
id: number;
|
||||||
|
ticketID: number;
|
||||||
|
title?: string;
|
||||||
|
description?: string;
|
||||||
|
noteType?: number;
|
||||||
|
publish?: number;
|
||||||
|
creatorResourceID?: number;
|
||||||
|
creatorType?: number;
|
||||||
|
lastActivityDate?: string;
|
||||||
|
createDateTime?: string;
|
||||||
|
}
|
||||||
|
```
|
||||||
|
`createEntity<T>` generic signature (`lib/services/autotask-client.ts` lines
|
||||||
|
175-189) — throws `Error('Failed to create entity')` if Autotask's response
|
||||||
|
has no `item`, and lets network/HTTP errors from `makeApiCall` propagate
|
||||||
|
uncaught. **This is exactly the failure mode D-05 requires the service to
|
||||||
|
catch per-ticket** — wrap each `createEntity` call in its own try/catch inside
|
||||||
|
a loop over linked tickets, not one try/catch around the whole loop:
|
||||||
|
```typescript
|
||||||
|
const ticketResults: Array<{ ticketId: string; posted: boolean; error?: string }> = [];
|
||||||
|
for (const report of reports) {
|
||||||
|
try {
|
||||||
|
await client.createEntity('TicketNotes', {
|
||||||
|
ticketID: Number(report.ticketId),
|
||||||
|
title: 'Phishing Triage Summary',
|
||||||
|
description: noteText,
|
||||||
|
noteType: 1,
|
||||||
|
publish: 1,
|
||||||
|
});
|
||||||
|
ticketResults.push({ ticketId: report.ticketId, posted: true });
|
||||||
|
} catch (err) {
|
||||||
|
console.error('[TRIAGE-NOTE] Failed to post note to ticket', report.ticketId, err);
|
||||||
|
ticketResults.push({
|
||||||
|
ticketId: report.ticketId,
|
||||||
|
posted: false,
|
||||||
|
error: err instanceof Error ? err.message : 'Unknown error',
|
||||||
|
});
|
||||||
|
}
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
**Sanitization precedent** — `lib/services/analyzer/itglue-redact.ts` (full
|
||||||
|
file, 65 lines) is the spirit-reference named in CONTEXT.md, though it
|
||||||
|
redacts by KEY NAME across an arbitrary object tree (IT Glue documents), which
|
||||||
|
doesn't map directly onto this phase's need (truncating URL query strings
|
||||||
|
inside plain prose text). Two concrete things to actually copy:
|
||||||
|
1. The **module-level "why" comment convention** — state plainly what must
|
||||||
|
never leak and why, mirroring lines 1-16 of `itglue-redact.ts`.
|
||||||
|
2. The **exported-for-tests + pure-function** shape — a small
|
||||||
|
`sanitizeIndicatorValue(value: string, type: string): string` (or similar)
|
||||||
|
function, unit-testable in isolation, same as `redact()`/`isSensitiveKey()`
|
||||||
|
are exported standalone in `itglue-redact.ts` lines 26-28 and 62-64. For
|
||||||
|
URL truncation specifically there is no existing analog in this codebase —
|
||||||
|
this is genuinely new logic (strip query string via `new URL(value).origin
|
||||||
|
+ new URL(value).pathname`, wrapped in try/catch for malformed URLs).
|
||||||
|
|
||||||
|
**No-op / synthesized-value pattern for missing evidence** — copy
|
||||||
|
`mimecast-blast-radius.ts`'s discriminated union (`status: 'ok' | 'unavailable'`,
|
||||||
|
never a thrown error for a missing/misconfigured integration) as the model for
|
||||||
|
how the note text should render "blast radius data unavailable" rather than
|
||||||
|
omitting the section — same spirit as `campaign-classifier.ts` line 350's
|
||||||
|
`{ status: 'unavailable', reason: 'not_configured' }` synthesis when there are
|
||||||
|
no linked reports at all.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
### `lib/services/triage-note-service.test.ts` (test, optional)
|
||||||
|
|
||||||
|
**Analog:** `lib/services/remediation-service.test.ts` and
|
||||||
|
`lib/services/campaign-classifier.test.ts` (not read in full — file names
|
||||||
|
only, per early-stopping guidance; both are existing Vitest suites under
|
||||||
|
`lib/services/` that test service functions directly against a real/fixture
|
||||||
|
Postgres, following the project's stated test coverage: `lib/services/**` is
|
||||||
|
covered). Structure to follow: mock or seed `campaigns`/`reports`/
|
||||||
|
`classifications`/`remediation_actions` rows, mock `getAutotaskClient()` (or
|
||||||
|
the whole `autotask-factory` module) to assert `createEntity` was called once
|
||||||
|
per linked ticket with the expected `ticketID`/`description`, and assert the
|
||||||
|
per-ticket failure path (D-05/D-06) when a mocked `createEntity` rejects for
|
||||||
|
one of several tickets.
|
||||||
|
|
||||||
|
## Shared Patterns
|
||||||
|
|
||||||
|
### Auth/Permission gate
|
||||||
|
**Source:** `lib/auth-utils.ts` lines 51-71 (`requirePermission`), used
|
||||||
|
identically by `classify/route.ts` line 24, `approve/route.ts` line 28,
|
||||||
|
`remediate/route.ts` line 27, `route.ts` (GET) line 62.
|
||||||
|
**Apply to:** the new triage-note route.
|
||||||
|
```typescript
|
||||||
|
const { session, error } = await requirePermission('phishing', 'analyze');
|
||||||
|
if (error) return error;
|
||||||
|
```
|
||||||
|
`lib/permissions.ts` line 33/50/64/78 confirms `'analyze'` is already granted
|
||||||
|
to admin/super-admin/user roles (only the read-only-ish role at line 78 lacks
|
||||||
|
it) — no new permission statement needed.
|
||||||
|
|
||||||
|
### UUID param validation
|
||||||
|
**Source:** identical `UUID_RE` regex + early-400 pattern in all four existing
|
||||||
|
`campaigns/[id]/*` routes (`classify`, `approve`, `remediate`, base `route.ts`).
|
||||||
|
**Apply to:** the new triage-note route — copy the exact regex, don't
|
||||||
|
re-derive it.
|
||||||
|
|
||||||
|
### Campaign-exists pre-check before service delegation
|
||||||
|
**Source:** `classify/route.ts` lines 34-41, `approve/route.ts` lines 52-59,
|
||||||
|
`remediate/route.ts` lines 40-46 — all three query `SELECT id FROM campaigns
|
||||||
|
WHERE id = $1` and return 404 before calling their service function.
|
||||||
|
**Apply to:** the new triage-note route, same shape.
|
||||||
|
|
||||||
|
### Error response shape
|
||||||
|
**Source:** every phishing route's catch block:
|
||||||
|
`NextResponse.json({ error: '...', message: err instanceof Error ? err.message : 'Unknown error' }, { status: 500 })`
|
||||||
|
with a `console.error('[PHISHING-<ACTION>] ...')` line immediately before.
|
||||||
|
**Apply to:** the new route's outer catch (whole-request failures only — see
|
||||||
|
D-05 note above about per-ticket failures NOT using this branch).
|
||||||
|
|
||||||
|
### Safe Autotask ticket-note write
|
||||||
|
**Source:** `lib/services/workflow-engine.ts` lines 581-589
|
||||||
|
(`runAiTroubleshooting`), backed by `lib/services/autotask-client.ts`
|
||||||
|
`createEntity<T>` (lines 175-189) and `lib/services/autotask-factory.ts`
|
||||||
|
`getAutotaskClient()`.
|
||||||
|
**Apply to:** the new service's write loop — `noteType: 1` (Internal),
|
||||||
|
`publish: 1` (All Autotask Users, still non-portal per
|
||||||
|
`AUTOTASK_API_GUIDE.md` line 365) are the existing codebase's only precedent
|
||||||
|
values; reuse them unless the planner has a specific reason to pick
|
||||||
|
`publish: 2` (Internal Users Only — even more restrictive, also non-portal).
|
||||||
|
|
||||||
|
### Audit trail (optional — not required by CONTEXT.md for this phase)
|
||||||
|
**Source:** `lib/services/phishing-audit.ts` (full file, 55 lines) —
|
||||||
|
`writeAuditEvent({ campaignId, actor, eventType, payload }, client?)`. Used by
|
||||||
|
every state-*changing* action (classify/approve/remediate/false-positive).
|
||||||
|
This phase is read+external-write, not a Postgres state change, so an audit
|
||||||
|
row is NOT strictly required by any D-0x decision — CONTEXT.md's Deferred
|
||||||
|
Ideas section explicitly puts "persisting sent-note history" out of scope.
|
||||||
|
If the planner still wants a lightweight audit trail of *when* a triage note
|
||||||
|
was requested (not full content), this is the write shape to reuse; `client`
|
||||||
|
param is optional so it can be called standalone (no transaction needed since
|
||||||
|
there's no corresponding state row to keep atomic with).
|
||||||
|
|
||||||
|
## No Analog Found
|
||||||
|
|
||||||
|
| File | Role | Data Flow | Reason |
|
||||||
|
|------|------|-----------|--------|
|
||||||
|
| URL/text sanitization helper (e.g. `lib/services/triage-note-sanitize.ts`, if split out) | utility | transform | No existing codebase function truncates URL query strings or formats human-readable prose from structured evidence — `itglue-redact.ts` redacts by object key name (a different technique for a different data shape); this is genuinely new logic per CONTEXT.md's "Claude's Discretion" section. |
|
||||||
|
| Note-text template/formatter | utility | transform | No existing "build human-readable prose from campaign+classification+remediation rows" function exists anywhere in the codebase — closest precedent is `campaign-classifier.ts`'s one-line `summary` string (line 487), which is far shorter than what D-04 requires here. |
|
||||||
|
|
||||||
|
## Metadata
|
||||||
|
|
||||||
|
**Analog search scope:** `app/api/phishing/**`, `lib/services/campaign-classifier.ts`,
|
||||||
|
`lib/services/remediation-service.ts`, `lib/services/campaign-grouping-service.ts`,
|
||||||
|
`lib/services/phishing-audit.ts`, `lib/services/workflow-engine.ts`,
|
||||||
|
`lib/services/autotask-client.ts`, `lib/services/autotask-factory.ts`,
|
||||||
|
`lib/services/mimecast-blast-radius.ts`, `lib/services/analyzer/itglue-search.ts`,
|
||||||
|
`lib/services/analyzer/itglue-redact.ts`, `lib/permissions.ts`, `lib/auth-utils.ts`,
|
||||||
|
`lib/types/autotask.ts`, `migrations/097_phishing_triage_schema.sql`.
|
||||||
|
**Files scanned:** 15
|
||||||
|
**Pattern extraction date:** 2026-07-16
|
||||||
|
|
@ -5,7 +5,8 @@
|
||||||
* volumeByDay — last 30 days, ticket creation count per day
|
* volumeByDay — last 30 days, ticket creation count per day
|
||||||
* resolutionByDay — last 30 days, mean resolution hours per day completed
|
* resolutionByDay — last 30 days, mean resolution hours per day completed
|
||||||
* queueHeatmap — open tickets grouped by (queue, priority)
|
* queueHeatmap — open tickets grouped by (queue, priority)
|
||||||
* activeEngineers — top engineers today by hours logged
|
* activeEngineers — working engineers today (top N by hours), each with their per-ticket time entries
|
||||||
|
* ptoEngineers — engineers whose only time today is on PTO/Vacation allocation codes
|
||||||
*
|
*
|
||||||
* All queries run in parallel. ~50 ms total against a warm DB.
|
* All queries run in parallel. ~50 ms total against a warm DB.
|
||||||
*/
|
*/
|
||||||
|
|
@ -18,6 +19,10 @@ import { getUserTimezone } from '@/lib/services/user-timezone';
|
||||||
const TREND_DAYS = 30;
|
const TREND_DAYS = 30;
|
||||||
const TOP_QUEUES = 10;
|
const TOP_QUEUES = 10;
|
||||||
const TOP_ENGINEERS = 8;
|
const TOP_ENGINEERS = 8;
|
||||||
|
// Autotask allocation codes treated as PTO/Vacation/time-off.
|
||||||
|
// These are the internal codes used at Wulf for non-billable time-off entries
|
||||||
|
// (vacation, personal day, etc.) — surfaced separately from working hours.
|
||||||
|
const PTO_ALLOCATION_CODE_IDS = [91206, 91207, 91209];
|
||||||
|
|
||||||
export async function GET() {
|
export async function GET() {
|
||||||
const { session, error } = await requireAuth();
|
const { session, error } = await requireAuth();
|
||||||
|
|
@ -63,6 +68,7 @@ export async function GET() {
|
||||||
[tz],
|
[tz],
|
||||||
),
|
),
|
||||||
// queueHeatmap: open-only counts (no day-boundary math) — tz does not apply.
|
// queueHeatmap: open-only counts (no day-boundary math) — tz does not apply.
|
||||||
|
// Filters out queues the calling user has hidden via /api/me/queue-preferences.
|
||||||
postgresClient.query<{
|
postgresClient.query<{
|
||||||
queue_id: number | null;
|
queue_id: number | null;
|
||||||
queue_label: string | null;
|
queue_label: string | null;
|
||||||
|
|
@ -77,29 +83,81 @@ export async function GET() {
|
||||||
LEFT JOIN queues q ON q.value = t.queue_id
|
LEFT JOIN queues q ON q.value = t.queue_id
|
||||||
WHERE t.completed_date IS NULL
|
WHERE t.completed_date IS NULL
|
||||||
AND (t.is_deleted = false OR t.is_deleted IS NULL)
|
AND (t.is_deleted = false OR t.is_deleted IS NULL)
|
||||||
|
AND (t.queue_id IS NULL OR t.queue_id NOT IN (
|
||||||
|
SELECT queue_id FROM user_queue_preferences WHERE user_id = $1
|
||||||
|
))
|
||||||
GROUP BY t.queue_id, q.label, t.priority
|
GROUP BY t.queue_id, q.label, t.priority
|
||||||
ORDER BY COUNT(*) DESC`,
|
ORDER BY COUNT(*) DESC`,
|
||||||
|
[session!.user.id],
|
||||||
),
|
),
|
||||||
postgresClient.query<{
|
postgresClient.query<{
|
||||||
resource_id: string;
|
resource_id: string;
|
||||||
resource_name: string;
|
resource_name: string;
|
||||||
hours: string;
|
hours: string;
|
||||||
tickets_touched: string;
|
tickets_touched: string;
|
||||||
|
ticket_id: string | null;
|
||||||
|
ticket_number: string | null;
|
||||||
|
ticket_title: string | null;
|
||||||
|
ticket_description: string | null;
|
||||||
|
ticket_status_label: string | null;
|
||||||
|
ticket_hours: string | null;
|
||||||
|
is_pto: boolean;
|
||||||
|
pto_note: string | null;
|
||||||
}>(
|
}>(
|
||||||
`SELECT te.resource_id::text,
|
`WITH today_entries AS (
|
||||||
|
SELECT te.resource_id,
|
||||||
|
te.ticket_id,
|
||||||
|
te.hours_worked,
|
||||||
|
te.allocation_code_id,
|
||||||
|
te.notes,
|
||||||
|
te.title
|
||||||
|
FROM time_entries te
|
||||||
|
WHERE te.entry_date::date = (NOW() AT TIME ZONE $1)::date
|
||||||
|
AND te.hours_worked > 0
|
||||||
|
),
|
||||||
|
engineer_totals AS (
|
||||||
|
SELECT resource_id,
|
||||||
|
SUM(hours_worked) AS hours,
|
||||||
|
COUNT(DISTINCT ticket_id) FILTER (WHERE ticket_id IS NOT NULL) AS tickets_touched,
|
||||||
|
BOOL_OR(allocation_code_id = ANY($2::int[])) AS has_pto,
|
||||||
|
BOOL_OR(allocation_code_id IS NULL OR NOT (allocation_code_id = ANY($2::int[]))) AS has_work,
|
||||||
|
MAX(CASE WHEN allocation_code_id = ANY($2::int[])
|
||||||
|
THEN NULLIF(COALESCE(notes, title), '') END) AS pto_note
|
||||||
|
FROM today_entries
|
||||||
|
GROUP BY resource_id
|
||||||
|
),
|
||||||
|
ticket_totals AS (
|
||||||
|
SELECT te.resource_id,
|
||||||
|
te.ticket_id,
|
||||||
|
SUM(te.hours_worked) AS ticket_hours
|
||||||
|
FROM today_entries te
|
||||||
|
WHERE te.ticket_id IS NOT NULL
|
||||||
|
GROUP BY te.resource_id, te.ticket_id
|
||||||
|
)
|
||||||
|
SELECT et.resource_id::text,
|
||||||
COALESCE(NULLIF(TRIM(r.first_name || ' ' || COALESCE(r.last_name, '')), ''),
|
COALESCE(NULLIF(TRIM(r.first_name || ' ' || COALESCE(r.last_name, '')), ''),
|
||||||
r.email,
|
r.email,
|
||||||
'Resource ' || te.resource_id) AS resource_name,
|
'Resource ' || et.resource_id) AS resource_name,
|
||||||
SUM(te.hours_worked)::text AS hours,
|
et.hours::text AS hours,
|
||||||
COUNT(DISTINCT te.ticket_id)::text AS tickets_touched
|
et.tickets_touched::text AS tickets_touched,
|
||||||
FROM time_entries te
|
tt.ticket_id::text AS ticket_id,
|
||||||
LEFT JOIN resources r ON r.id = te.resource_id
|
t.ticket_number,
|
||||||
WHERE ((te.entry_date AT TIME ZONE 'UTC') AT TIME ZONE $1)::date = (NOW() AT TIME ZONE $1)::date
|
t.title AS ticket_title,
|
||||||
AND te.hours_worked > 0
|
t.description AS ticket_description,
|
||||||
GROUP BY te.resource_id, r.first_name, r.last_name, r.email
|
s.label AS ticket_status_label,
|
||||||
ORDER BY SUM(te.hours_worked) DESC
|
tt.ticket_hours::text AS ticket_hours,
|
||||||
LIMIT ${TOP_ENGINEERS}`,
|
(et.has_pto AND NOT et.has_work) AS is_pto,
|
||||||
[tz],
|
et.pto_note
|
||||||
|
FROM engineer_totals et
|
||||||
|
LEFT JOIN resources r ON r.id = et.resource_id
|
||||||
|
LEFT JOIN ticket_totals tt ON tt.resource_id = et.resource_id
|
||||||
|
LEFT JOIN tickets t ON t.id = tt.ticket_id
|
||||||
|
LEFT JOIN statuses s ON s.value = t.status
|
||||||
|
ORDER BY (et.has_pto AND NOT et.has_work) ASC,
|
||||||
|
et.hours DESC,
|
||||||
|
et.resource_id,
|
||||||
|
tt.ticket_hours DESC NULLS LAST`,
|
||||||
|
[tz, PTO_ALLOCATION_CODE_IDS],
|
||||||
),
|
),
|
||||||
]);
|
]);
|
||||||
|
|
||||||
|
|
@ -129,6 +187,61 @@ export async function GET() {
|
||||||
return { queueId: q.id, queueLabel: q.label, total: q.total, byPriority: cells };
|
return { queueId: q.id, queueLabel: q.label, total: q.total, byPriority: cells };
|
||||||
});
|
});
|
||||||
|
|
||||||
|
// Fold the joined engineer/ticket rows into per-engineer records.
|
||||||
|
type Ticket = {
|
||||||
|
id: string;
|
||||||
|
ticketNumber: string | null;
|
||||||
|
title: string | null;
|
||||||
|
description: string | null;
|
||||||
|
statusLabel: string | null;
|
||||||
|
hours: number;
|
||||||
|
};
|
||||||
|
type Engineer = {
|
||||||
|
resourceId: string;
|
||||||
|
name: string;
|
||||||
|
hours: number;
|
||||||
|
ticketsTouched: number;
|
||||||
|
tickets: Ticket[];
|
||||||
|
isPto: boolean;
|
||||||
|
ptoNote: string | null;
|
||||||
|
};
|
||||||
|
const engineerById = new Map<string, Engineer>();
|
||||||
|
for (const row of engineersRes.rows) {
|
||||||
|
let eng = engineerById.get(row.resource_id);
|
||||||
|
if (!eng) {
|
||||||
|
eng = {
|
||||||
|
resourceId: row.resource_id,
|
||||||
|
name: row.resource_name,
|
||||||
|
hours: Math.round(parseFloat(row.hours) * 10) / 10,
|
||||||
|
ticketsTouched: parseInt(row.tickets_touched, 10),
|
||||||
|
tickets: [],
|
||||||
|
isPto: row.is_pto,
|
||||||
|
ptoNote: row.pto_note,
|
||||||
|
};
|
||||||
|
engineerById.set(row.resource_id, eng);
|
||||||
|
}
|
||||||
|
if (row.ticket_id) {
|
||||||
|
eng.tickets.push({
|
||||||
|
id: row.ticket_id,
|
||||||
|
ticketNumber: row.ticket_number,
|
||||||
|
title: row.ticket_title,
|
||||||
|
description: row.ticket_description,
|
||||||
|
statusLabel: row.ticket_status_label,
|
||||||
|
hours: row.ticket_hours == null
|
||||||
|
? 0
|
||||||
|
: Math.round(parseFloat(row.ticket_hours) * 10) / 10,
|
||||||
|
});
|
||||||
|
}
|
||||||
|
}
|
||||||
|
const allEngineers = [...engineerById.values()];
|
||||||
|
const activeEngineers = allEngineers
|
||||||
|
.filter((e) => !e.isPto)
|
||||||
|
.sort((a, b) => b.hours - a.hours)
|
||||||
|
.slice(0, TOP_ENGINEERS);
|
||||||
|
const ptoEngineers = allEngineers
|
||||||
|
.filter((e) => e.isPto)
|
||||||
|
.sort((a, b) => a.name.localeCompare(b.name));
|
||||||
|
|
||||||
return NextResponse.json({
|
return NextResponse.json({
|
||||||
volumeByDay: volumeRes.rows.map((r) => ({
|
volumeByDay: volumeRes.rows.map((r) => ({
|
||||||
date: r.d,
|
date: r.d,
|
||||||
|
|
@ -139,11 +252,7 @@ export async function GET() {
|
||||||
avgHours: r.avg_hours == null ? null : Math.round(parseFloat(r.avg_hours) * 10) / 10,
|
avgHours: r.avg_hours == null ? null : Math.round(parseFloat(r.avg_hours) * 10) / 10,
|
||||||
})),
|
})),
|
||||||
queueHeatmap: heatmap,
|
queueHeatmap: heatmap,
|
||||||
activeEngineers: engineersRes.rows.map((r) => ({
|
activeEngineers,
|
||||||
resourceId: r.resource_id,
|
ptoEngineers,
|
||||||
name: r.resource_name,
|
|
||||||
hours: Math.round(parseFloat(r.hours) * 10) / 10,
|
|
||||||
ticketsTouched: parseInt(r.tickets_touched, 10),
|
|
||||||
})),
|
|
||||||
});
|
});
|
||||||
}
|
}
|
||||||
|
|
|
||||||
101
app/api/me/queue-preferences/route.ts
Normal file
101
app/api/me/queue-preferences/route.ts
Normal file
|
|
@ -0,0 +1,101 @@
|
||||||
|
import { NextRequest, NextResponse } from 'next/server';
|
||||||
|
import { requireAuth } from '@/lib/auth-utils';
|
||||||
|
import { postgresClient } from '@/lib/services/postgres-client';
|
||||||
|
|
||||||
|
// GET /api/me/queue-preferences
|
||||||
|
// -> { queues: [{ id, label, hidden }], hiddenIds: number[] }
|
||||||
|
// Returns every active queue with a `hidden` flag for the calling user,
|
||||||
|
// plus the bare list of hidden queue IDs (for clients that only need that).
|
||||||
|
//
|
||||||
|
// PUT /api/me/queue-preferences
|
||||||
|
// body { hiddenIds: number[] } -> { hiddenIds: number[] }
|
||||||
|
// Replaces the user's hidden-queue set atomically (full set semantics).
|
||||||
|
//
|
||||||
|
// Per-user only — writes target session.user.id, never an id from the body.
|
||||||
|
|
||||||
|
export async function GET(): Promise<NextResponse> {
|
||||||
|
const { session, error } = await requireAuth();
|
||||||
|
if (error) return error;
|
||||||
|
|
||||||
|
try {
|
||||||
|
const result = await postgresClient.query<{
|
||||||
|
value: number;
|
||||||
|
label: string;
|
||||||
|
hidden: boolean;
|
||||||
|
}>(
|
||||||
|
`SELECT q.value, q.label,
|
||||||
|
(p.queue_id IS NOT NULL) AS hidden
|
||||||
|
FROM queues q
|
||||||
|
LEFT JOIN user_queue_preferences p
|
||||||
|
ON p.queue_id = q.value AND p.user_id = $1
|
||||||
|
WHERE q.is_active = true
|
||||||
|
AND (q.is_deleted = false OR q.is_deleted IS NULL)
|
||||||
|
ORDER BY q.label`,
|
||||||
|
[session!.user.id],
|
||||||
|
);
|
||||||
|
const queues = result.rows.map((r) => ({
|
||||||
|
id: r.value,
|
||||||
|
label: r.label,
|
||||||
|
hidden: r.hidden,
|
||||||
|
}));
|
||||||
|
const hiddenIds = queues.filter((q) => q.hidden).map((q) => q.id);
|
||||||
|
return NextResponse.json({ queues, hiddenIds });
|
||||||
|
} catch (e) {
|
||||||
|
console.error('GET /api/me/queue-preferences failed:', e);
|
||||||
|
return NextResponse.json(
|
||||||
|
{ error: 'Failed to read queue preferences', message: e instanceof Error ? e.message : 'unknown' },
|
||||||
|
{ status: 500 },
|
||||||
|
);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
export async function PUT(request: NextRequest): Promise<NextResponse> {
|
||||||
|
const { session, error } = await requireAuth();
|
||||||
|
if (error) return error;
|
||||||
|
|
||||||
|
let body: unknown;
|
||||||
|
try {
|
||||||
|
body = await request.json();
|
||||||
|
} catch {
|
||||||
|
return NextResponse.json(
|
||||||
|
{ error: 'Invalid JSON', message: 'Request body must be JSON' },
|
||||||
|
{ status: 400 },
|
||||||
|
);
|
||||||
|
}
|
||||||
|
|
||||||
|
const raw =
|
||||||
|
body && typeof body === 'object' && 'hiddenIds' in body
|
||||||
|
? (body as { hiddenIds: unknown }).hiddenIds
|
||||||
|
: undefined;
|
||||||
|
|
||||||
|
if (!Array.isArray(raw) || !raw.every((v) => Number.isInteger(v))) {
|
||||||
|
return NextResponse.json(
|
||||||
|
{ error: 'Invalid input', message: 'hiddenIds must be an array of integers' },
|
||||||
|
{ status: 400 },
|
||||||
|
);
|
||||||
|
}
|
||||||
|
const hiddenIds = [...new Set(raw as number[])];
|
||||||
|
|
||||||
|
const userId = session!.user.id;
|
||||||
|
|
||||||
|
try {
|
||||||
|
await postgresClient.transaction(async (client) => {
|
||||||
|
await client.query('DELETE FROM user_queue_preferences WHERE user_id = $1', [userId]);
|
||||||
|
if (hiddenIds.length > 0) {
|
||||||
|
const placeholders = hiddenIds.map((_, i) => `($1, $${i + 2})`).join(', ');
|
||||||
|
await client.query(
|
||||||
|
`INSERT INTO user_queue_preferences (user_id, queue_id) VALUES ${placeholders}
|
||||||
|
ON CONFLICT (user_id, queue_id) DO NOTHING`,
|
||||||
|
[userId, ...hiddenIds],
|
||||||
|
);
|
||||||
|
}
|
||||||
|
});
|
||||||
|
return NextResponse.json({ hiddenIds });
|
||||||
|
} catch (e) {
|
||||||
|
console.error('PUT /api/me/queue-preferences failed:', e);
|
||||||
|
return NextResponse.json(
|
||||||
|
{ error: 'Failed to update queue preferences', message: e instanceof Error ? e.message : 'unknown' },
|
||||||
|
{ status: 500 },
|
||||||
|
);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
@ -131,7 +131,7 @@ export async function GET(request: NextRequest): Promise<NextResponse> {
|
||||||
FROM time_entries te
|
FROM time_entries te
|
||||||
JOIN resources r ON r.id = te.resource_id AND (r.is_deleted = false OR r.is_deleted IS NULL)
|
JOIN resources r ON r.id = te.resource_id AND (r.is_deleted = false OR r.is_deleted IS NULL)
|
||||||
JOIN graph_users gu ON LOWER(gu.email) = LOWER(r.email)
|
JOIN graph_users gu ON LOWER(gu.email) = LOWER(r.email)
|
||||||
WHERE (te.entry_date AT TIME ZONE 'UTC' AT TIME ZONE $1) >= (NOW() AT TIME ZONE $1) - INTERVAL '${interval}'
|
WHERE te.entry_date >= (NOW() AT TIME ZONE $1)::date - INTERVAL '${interval}'
|
||||||
AND (te.is_deleted = false OR te.is_deleted IS NULL)
|
AND (te.is_deleted = false OR te.is_deleted IS NULL)
|
||||||
AND gu.account_enabled = true
|
AND gu.account_enabled = true
|
||||||
AND LOWER(gu.email) LIKE '%@wulfconsulting.%'
|
AND LOWER(gu.email) LIKE '%@wulfconsulting.%'
|
||||||
|
|
|
||||||
|
|
@ -56,18 +56,18 @@ export async function GET(request: NextRequest): Promise<NextResponse> {
|
||||||
)::date AS day
|
)::date AS day
|
||||||
),
|
),
|
||||||
daily_hours AS (
|
daily_hours AS (
|
||||||
SELECT (te.entry_date AT TIME ZONE 'UTC' AT TIME ZONE $1)::date AS day, COALESCE(SUM(te.hours_worked), 0) AS hours
|
SELECT te.entry_date::date AS day, COALESCE(SUM(te.hours_worked), 0) AS hours
|
||||||
FROM time_entries te
|
FROM time_entries te
|
||||||
JOIN resources r ON r.id = te.resource_id
|
JOIN resources r ON r.id = te.resource_id
|
||||||
AND (r.is_deleted = false OR r.is_deleted IS NULL)
|
AND (r.is_deleted = false OR r.is_deleted IS NULL)
|
||||||
JOIN graph_users gu ON LOWER(gu.email) = LOWER(r.email)
|
JOIN graph_users gu ON LOWER(gu.email) = LOWER(r.email)
|
||||||
WHERE (te.entry_date AT TIME ZONE 'UTC' AT TIME ZONE $1)::date >= ((NOW() AT TIME ZONE $1) - INTERVAL '${days - 1} days')::date
|
WHERE te.entry_date::date >= ((NOW() AT TIME ZONE $1) - INTERVAL '${days - 1} days')::date
|
||||||
AND (te.entry_date AT TIME ZONE 'UTC' AT TIME ZONE $1)::date <= (NOW() AT TIME ZONE $1)::date
|
AND te.entry_date::date <= (NOW() AT TIME ZONE $1)::date
|
||||||
AND (te.is_deleted = false OR te.is_deleted IS NULL)
|
AND (te.is_deleted = false OR te.is_deleted IS NULL)
|
||||||
AND gu.account_enabled = true
|
AND gu.account_enabled = true
|
||||||
AND LOWER(gu.email) LIKE '%@wulfconsulting.%'
|
AND LOWER(gu.email) LIKE '%@wulfconsulting.%'
|
||||||
AND LOWER(gu.email) NOT LIKE '%#ext#%'
|
AND LOWER(gu.email) NOT LIKE '%#ext#%'
|
||||||
GROUP BY (te.entry_date AT TIME ZONE 'UTC' AT TIME ZONE $1)::date
|
GROUP BY te.entry_date::date
|
||||||
)
|
)
|
||||||
SELECT to_char(ds.day, 'YYYY-MM-DD') AS date,
|
SELECT to_char(ds.day, 'YYYY-MM-DD') AS date,
|
||||||
COALESCE(dh.hours, 0)::numeric AS hours
|
COALESCE(dh.hours, 0)::numeric AS hours
|
||||||
|
|
|
||||||
|
|
@ -13,11 +13,11 @@ export async function GET() {
|
||||||
if (error) return error;
|
if (error) return error;
|
||||||
const tz = getUserTimezone(session);
|
const tz = getUserTimezone(session);
|
||||||
|
|
||||||
const [summary, aging, topCustomers, overdueInvoices, recentPayments, monthlyRevenue] = await Promise.all([
|
const [summary, aging, topCustomers, overdueInvoices, recentPayments, monthlyRevenue, credits] = await Promise.all([
|
||||||
postgresClient.query(
|
postgresClient.query(
|
||||||
`
|
`
|
||||||
SELECT
|
SELECT
|
||||||
SUM(balance) FILTER (WHERE status IN ('Open','Overdue')) as total_ar,
|
SUM(balance) FILTER (WHERE status IN ('Open','Overdue')) as total_ar_gross,
|
||||||
COUNT(*) FILTER (WHERE status IN ('Open','Overdue')) as total_ar_count,
|
COUNT(*) FILTER (WHERE status IN ('Open','Overdue')) as total_ar_count,
|
||||||
SUM(balance) FILTER (WHERE status = 'Open') as current_balance,
|
SUM(balance) FILTER (WHERE status = 'Open') as current_balance,
|
||||||
COUNT(*) FILTER (WHERE status = 'Open') as current_count,
|
COUNT(*) FILTER (WHERE status = 'Open') as current_count,
|
||||||
|
|
@ -26,6 +26,7 @@ export async function GET() {
|
||||||
SUM(total_amt) FILTER (WHERE status = 'Paid' AND (txn_date AT TIME ZONE 'UTC' AT TIME ZONE $1) >= DATE_TRUNC('month', NOW() AT TIME ZONE $1)) as paid_mtd,
|
SUM(total_amt) FILTER (WHERE status = 'Paid' AND (txn_date AT TIME ZONE 'UTC' AT TIME ZONE $1) >= DATE_TRUNC('month', NOW() AT TIME ZONE $1)) as paid_mtd,
|
||||||
SUM(total_amt) FILTER (WHERE status = 'Paid' AND (txn_date AT TIME ZONE 'UTC' AT TIME ZONE $1) >= DATE_TRUNC('year', NOW() AT TIME ZONE $1)) as paid_ytd
|
SUM(total_amt) FILTER (WHERE status = 'Paid' AND (txn_date AT TIME ZONE 'UTC' AT TIME ZONE $1) >= DATE_TRUNC('year', NOW() AT TIME ZONE $1)) as paid_ytd
|
||||||
FROM qbo_invoices
|
FROM qbo_invoices
|
||||||
|
WHERE is_deleted = false
|
||||||
`,
|
`,
|
||||||
[tz],
|
[tz],
|
||||||
),
|
),
|
||||||
|
|
@ -41,22 +42,43 @@ export async function GET() {
|
||||||
SUM(balance) FILTER (WHERE status = 'Overdue' AND due_date < (NOW() AT TIME ZONE $1)::date - 60) as days_60_plus,
|
SUM(balance) FILTER (WHERE status = 'Overdue' AND due_date < (NOW() AT TIME ZONE $1)::date - 60) as days_60_plus,
|
||||||
COUNT(*) FILTER (WHERE status = 'Overdue' AND due_date < (NOW() AT TIME ZONE $1)::date - 60) as cnt_60_plus
|
COUNT(*) FILTER (WHERE status = 'Overdue' AND due_date < (NOW() AT TIME ZONE $1)::date - 60) as cnt_60_plus
|
||||||
FROM qbo_invoices
|
FROM qbo_invoices
|
||||||
|
WHERE is_deleted = false
|
||||||
`,
|
`,
|
||||||
[tz],
|
[tz],
|
||||||
),
|
),
|
||||||
|
// Top customers — net out any unapplied payment credits the customer is
|
||||||
|
// sitting on, so the number matches QBO's customer A/R view.
|
||||||
postgresClient.query(`
|
postgresClient.query(`
|
||||||
SELECT customer_ref_name, SUM(balance) as balance, COUNT(*) as invoice_count
|
WITH inv AS (
|
||||||
|
SELECT customer_ref_id, customer_ref_name,
|
||||||
|
SUM(balance) AS gross_balance,
|
||||||
|
COUNT(*) AS invoice_count
|
||||||
FROM qbo_invoices
|
FROM qbo_invoices
|
||||||
WHERE status IN ('Open','Overdue')
|
WHERE is_deleted = false
|
||||||
GROUP BY customer_ref_name
|
AND status IN ('Open','Overdue')
|
||||||
ORDER BY balance DESC LIMIT 8
|
GROUP BY customer_ref_id, customer_ref_name
|
||||||
|
),
|
||||||
|
cred AS (
|
||||||
|
SELECT customer_ref_id, SUM(unapplied_amt) AS unapplied
|
||||||
|
FROM qbo_payments
|
||||||
|
WHERE unapplied_amt > 0
|
||||||
|
GROUP BY customer_ref_id
|
||||||
|
)
|
||||||
|
SELECT inv.customer_ref_name,
|
||||||
|
GREATEST(inv.gross_balance - COALESCE(cred.unapplied, 0), 0) AS balance,
|
||||||
|
inv.invoice_count
|
||||||
|
FROM inv
|
||||||
|
LEFT JOIN cred ON cred.customer_ref_id = inv.customer_ref_id
|
||||||
|
ORDER BY balance DESC
|
||||||
|
LIMIT 8
|
||||||
`),
|
`),
|
||||||
postgresClient.query(
|
postgresClient.query(
|
||||||
`
|
`
|
||||||
SELECT id, doc_number, txn_date, due_date, customer_ref_name, total_amt, balance, status,
|
SELECT id, doc_number, txn_date, due_date, customer_ref_name, total_amt, balance, status,
|
||||||
(NOW() AT TIME ZONE $1)::date - due_date::date as days_overdue
|
(NOW() AT TIME ZONE $1)::date - due_date::date as days_overdue
|
||||||
FROM qbo_invoices
|
FROM qbo_invoices
|
||||||
WHERE status IN ('Open','Overdue')
|
WHERE is_deleted = false
|
||||||
|
AND status IN ('Open','Overdue')
|
||||||
ORDER BY status DESC, balance DESC LIMIT 30
|
ORDER BY status DESC, balance DESC LIMIT 30
|
||||||
`,
|
`,
|
||||||
[tz],
|
[tz],
|
||||||
|
|
@ -72,16 +94,32 @@ export async function GET() {
|
||||||
SELECT DATE_TRUNC('month', txn_date) as month,
|
SELECT DATE_TRUNC('month', txn_date) as month,
|
||||||
SUM(total_amt) as revenue, COUNT(*) as invoice_count
|
SUM(total_amt) as revenue, COUNT(*) as invoice_count
|
||||||
FROM qbo_invoices
|
FROM qbo_invoices
|
||||||
WHERE status = 'Paid' AND txn_date >= NOW() - INTERVAL '12 months'
|
WHERE is_deleted = false
|
||||||
|
AND status = 'Paid'
|
||||||
|
AND txn_date >= NOW() - INTERVAL '12 months'
|
||||||
GROUP BY 1 ORDER BY 1 ASC
|
GROUP BY 1 ORDER BY 1 ASC
|
||||||
`),
|
`),
|
||||||
|
// Unapplied customer credits — payments not yet linked to an invoice.
|
||||||
|
// QBO nets these against A/R in its customer balance reports.
|
||||||
|
postgresClient.query(`
|
||||||
|
SELECT COALESCE(SUM(unapplied_amt), 0) AS total_unapplied
|
||||||
|
FROM qbo_payments
|
||||||
|
WHERE unapplied_amt > 0
|
||||||
|
`),
|
||||||
]);
|
]);
|
||||||
|
|
||||||
const s = summary.rows[0];
|
const s = summary.rows[0];
|
||||||
const a = aging.rows[0];
|
const a = aging.rows[0];
|
||||||
|
const unappliedCredits = parseFloat(credits.rows[0]?.total_unapplied ?? 0);
|
||||||
|
const grossAr = parseFloat(s.total_ar_gross ?? 0);
|
||||||
|
// QBO's A/R reports net unapplied customer credits against the gross
|
||||||
|
// invoice balance. Mirror that so the dashboard headline matches QBO.
|
||||||
|
const netAr = Math.max(grossAr - unappliedCredits, 0);
|
||||||
return NextResponse.json({
|
return NextResponse.json({
|
||||||
summary: {
|
summary: {
|
||||||
total_ar: parseFloat(s.total_ar ?? 0),
|
total_ar: netAr,
|
||||||
|
total_ar_gross: grossAr,
|
||||||
|
unapplied_credits: unappliedCredits,
|
||||||
total_ar_count: parseInt(s.total_ar_count ?? 0),
|
total_ar_count: parseInt(s.total_ar_count ?? 0),
|
||||||
current_balance: parseFloat(s.current_balance ?? 0),
|
current_balance: parseFloat(s.current_balance ?? 0),
|
||||||
current_count: parseInt(s.current_count ?? 0),
|
current_count: parseInt(s.current_count ?? 0),
|
||||||
|
|
|
||||||
182
app/api/qbo/diagnose-ar/route.ts
Normal file
182
app/api/qbo/diagnose-ar/route.ts
Normal file
|
|
@ -0,0 +1,182 @@
|
||||||
|
/**
|
||||||
|
* GET /api/qbo/diagnose-ar
|
||||||
|
* One-off reconciliation diagnostic — pulls QBO's Aged Receivable Detail
|
||||||
|
* report live and diffs it against Pulse's qbo_invoices to identify the
|
||||||
|
* gap between Pulse's headline A/R and what QBO reports.
|
||||||
|
*
|
||||||
|
* Returns per-invoice mismatches in three buckets:
|
||||||
|
* - in_qbo_not_in_pulse — QBO knows about it, Pulse doesn't (sync miss)
|
||||||
|
* - in_pulse_not_in_qbo — Pulse has live A/R for it, QBO doesn't
|
||||||
|
* (likely a tombstone candidate)
|
||||||
|
* - balance_diff — both have it but the balance differs
|
||||||
|
*/
|
||||||
|
|
||||||
|
import { NextResponse } from 'next/server';
|
||||||
|
import { requireAdmin } from '@/lib/auth-utils';
|
||||||
|
import postgresClient from '@/lib/services/postgres-client';
|
||||||
|
import { QboClient } from '@/lib/services/qbo-client';
|
||||||
|
|
||||||
|
// QBO report rows are deeply nested — walk them and yield every leaf "Data"
|
||||||
|
// row. Each leaf carries an array of column values matching the report's
|
||||||
|
// column header (Customer | Date | Transaction Type | Num | Due Date | Aging |
|
||||||
|
// Open Balance, etc.).
|
||||||
|
function* walkRows(node: unknown): Generator<{ values: string[]; group?: string }> {
|
||||||
|
if (!node || typeof node !== 'object') return;
|
||||||
|
const n = node as Record<string, unknown>;
|
||||||
|
if (Array.isArray(n.Row)) {
|
||||||
|
for (const child of n.Row as unknown[]) yield* walkRows(child);
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
if (n.Rows) {
|
||||||
|
yield* walkRows(n.Rows);
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
if (n.type === 'Data' && n.ColData && Array.isArray(n.ColData)) {
|
||||||
|
const values = (n.ColData as Array<{ value?: string }>).map((c) => c?.value ?? '');
|
||||||
|
yield { values, group: typeof n.group === 'string' ? n.group : undefined };
|
||||||
|
}
|
||||||
|
if (n.Header || n.Summary) {
|
||||||
|
// Section node — recurse into its rows
|
||||||
|
if (n.Rows) yield* walkRows(n.Rows);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
export async function GET() {
|
||||||
|
const { error } = await requireAdmin();
|
||||||
|
if (error) return error;
|
||||||
|
const client = new QboClient();
|
||||||
|
const report = await client.getAgedReceivableDetail();
|
||||||
|
|
||||||
|
// Map columns by ColTitle so we don't rely on positional order
|
||||||
|
const cols: Array<{ ColTitle?: string; ColType?: string }> =
|
||||||
|
(report.Columns?.Column ?? []) as Array<{ ColTitle?: string; ColType?: string }>;
|
||||||
|
const idx = (title: string) => cols.findIndex((c) => (c.ColTitle ?? '').toLowerCase() === title.toLowerCase());
|
||||||
|
|
||||||
|
const iNum = idx('Num');
|
||||||
|
const iCust = idx('Customer');
|
||||||
|
const iBalance = idx('Open Balance');
|
||||||
|
const iType = idx('Transaction Type');
|
||||||
|
const iAging = idx('Aging');
|
||||||
|
|
||||||
|
type QboLine = {
|
||||||
|
docNumber: string;
|
||||||
|
customer: string;
|
||||||
|
type: string;
|
||||||
|
aging: string;
|
||||||
|
balance: number;
|
||||||
|
};
|
||||||
|
const qboInvoices: QboLine[] = [];
|
||||||
|
let qboTotal = 0;
|
||||||
|
for (const row of walkRows(report.Rows)) {
|
||||||
|
const v = row.values;
|
||||||
|
if (!v.length) continue;
|
||||||
|
const balanceStr = iBalance >= 0 ? v[iBalance] : '';
|
||||||
|
const balance = balanceStr ? parseFloat(balanceStr) : 0;
|
||||||
|
const docNumber = iNum >= 0 ? v[iNum] : '';
|
||||||
|
const customer = iCust >= 0 ? v[iCust] : '';
|
||||||
|
const type = iType >= 0 ? v[iType] : '';
|
||||||
|
const aging = iAging >= 0 ? v[iAging] : '';
|
||||||
|
// Skip total/summary rows (no doc number, no customer)
|
||||||
|
if (!docNumber && !customer) continue;
|
||||||
|
qboInvoices.push({ docNumber, customer, type, aging, balance });
|
||||||
|
qboTotal += balance;
|
||||||
|
}
|
||||||
|
|
||||||
|
const pulseRes = await postgresClient.query<{
|
||||||
|
id: string;
|
||||||
|
doc_number: string | null;
|
||||||
|
customer_ref_name: string | null;
|
||||||
|
balance: string;
|
||||||
|
status: string;
|
||||||
|
}>(
|
||||||
|
`SELECT id, doc_number, customer_ref_name, balance::text, status
|
||||||
|
FROM qbo_invoices
|
||||||
|
WHERE is_deleted = false
|
||||||
|
AND status IN ('Open','Overdue')
|
||||||
|
AND balance <> 0`,
|
||||||
|
);
|
||||||
|
const pulseByDoc = new Map<string, { id: string; customer: string; balance: number; status: string }>();
|
||||||
|
let pulseTotal = 0;
|
||||||
|
for (const r of pulseRes.rows) {
|
||||||
|
const balance = parseFloat(r.balance);
|
||||||
|
pulseTotal += balance;
|
||||||
|
if (r.doc_number) {
|
||||||
|
pulseByDoc.set(r.doc_number, {
|
||||||
|
id: r.id,
|
||||||
|
customer: r.customer_ref_name ?? '',
|
||||||
|
balance,
|
||||||
|
status: r.status,
|
||||||
|
});
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// Diff
|
||||||
|
const inQboNotInPulse: QboLine[] = [];
|
||||||
|
const balanceDiff: Array<{
|
||||||
|
docNumber: string;
|
||||||
|
customer: string;
|
||||||
|
qboBalance: number;
|
||||||
|
pulseBalance: number;
|
||||||
|
delta: number;
|
||||||
|
}> = [];
|
||||||
|
const seenDocs = new Set<string>();
|
||||||
|
for (const inv of qboInvoices) {
|
||||||
|
if (!inv.docNumber) continue;
|
||||||
|
seenDocs.add(inv.docNumber);
|
||||||
|
const pulse = pulseByDoc.get(inv.docNumber);
|
||||||
|
if (!pulse) {
|
||||||
|
inQboNotInPulse.push(inv);
|
||||||
|
continue;
|
||||||
|
}
|
||||||
|
const delta = Math.round((inv.balance - pulse.balance) * 100) / 100;
|
||||||
|
if (Math.abs(delta) > 0.005) {
|
||||||
|
balanceDiff.push({
|
||||||
|
docNumber: inv.docNumber,
|
||||||
|
customer: inv.customer || pulse.customer,
|
||||||
|
qboBalance: inv.balance,
|
||||||
|
pulseBalance: pulse.balance,
|
||||||
|
delta,
|
||||||
|
});
|
||||||
|
}
|
||||||
|
}
|
||||||
|
const inPulseNotInQbo: Array<{
|
||||||
|
docNumber: string;
|
||||||
|
customer: string;
|
||||||
|
balance: number;
|
||||||
|
status: string;
|
||||||
|
}> = [];
|
||||||
|
for (const [docNumber, p] of pulseByDoc) {
|
||||||
|
if (!seenDocs.has(docNumber)) {
|
||||||
|
inPulseNotInQbo.push({
|
||||||
|
docNumber,
|
||||||
|
customer: p.customer,
|
||||||
|
balance: p.balance,
|
||||||
|
status: p.status,
|
||||||
|
});
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
const sumOf = <T extends { balance?: number; delta?: number }>(arr: T[], key: 'balance' | 'delta') =>
|
||||||
|
Math.round(arr.reduce((s, x) => s + (x[key] ?? 0), 0) * 100) / 100;
|
||||||
|
|
||||||
|
return NextResponse.json({
|
||||||
|
summary: {
|
||||||
|
qbo_aging_total: Math.round(qboTotal * 100) / 100,
|
||||||
|
pulse_gross_ar: Math.round(pulseTotal * 100) / 100,
|
||||||
|
qbo_invoice_count: qboInvoices.length,
|
||||||
|
pulse_invoice_count: pulseRes.rows.length,
|
||||||
|
gap: Math.round((pulseTotal - qboTotal) * 100) / 100,
|
||||||
|
},
|
||||||
|
differences: {
|
||||||
|
in_qbo_not_in_pulse: { count: inQboNotInPulse.length, sum: sumOf(inQboNotInPulse, 'balance'), rows: inQboNotInPulse },
|
||||||
|
in_pulse_not_in_qbo: { count: inPulseNotInQbo.length, sum: sumOf(inPulseNotInQbo, 'balance'), rows: inPulseNotInQbo },
|
||||||
|
balance_diff: { count: balanceDiff.length, sum: sumOf(balanceDiff, 'delta'), rows: balanceDiff },
|
||||||
|
},
|
||||||
|
report_meta: {
|
||||||
|
report_name: report.Header?.ReportName,
|
||||||
|
end_period: report.Header?.EndPeriod,
|
||||||
|
time: report.Header?.Time,
|
||||||
|
report_basis: report.Header?.ReportBasis,
|
||||||
|
},
|
||||||
|
});
|
||||||
|
}
|
||||||
|
|
@ -40,7 +40,7 @@ export async function POST(request: NextRequest) {
|
||||||
export async function GET() {
|
export async function GET() {
|
||||||
try {
|
try {
|
||||||
const [invoices, payments, deposits, transactions, reports] = await Promise.all([
|
const [invoices, payments, deposits, transactions, reports] = await Promise.all([
|
||||||
postgresClient.query(`SELECT COUNT(*) as count, MAX(synced_at) as last_sync FROM qbo_invoices`),
|
postgresClient.query(`SELECT COUNT(*) as count, MAX(synced_at) as last_sync FROM qbo_invoices WHERE is_deleted = false`),
|
||||||
postgresClient.query(`SELECT COUNT(*) as count, MAX(synced_at) as last_sync FROM qbo_payments`),
|
postgresClient.query(`SELECT COUNT(*) as count, MAX(synced_at) as last_sync FROM qbo_payments`),
|
||||||
postgresClient.query(`SELECT COUNT(*) as count, MAX(synced_at) as last_sync FROM qbo_deposits`),
|
postgresClient.query(`SELECT COUNT(*) as count, MAX(synced_at) as last_sync FROM qbo_deposits`),
|
||||||
postgresClient.query(`SELECT COUNT(*) as count, MAX(synced_at) as last_sync FROM qbo_transactions`),
|
postgresClient.query(`SELECT COUNT(*) as count, MAX(synced_at) as last_sync FROM qbo_transactions`),
|
||||||
|
|
|
||||||
|
|
@ -23,6 +23,7 @@ import { KpiCard } from '@/components/dashboard/kpi-card';
|
||||||
import { VolumeTrend } from '@/components/dashboard/volume-trend';
|
import { VolumeTrend } from '@/components/dashboard/volume-trend';
|
||||||
import { ResolutionTrend } from '@/components/dashboard/resolution-trend';
|
import { ResolutionTrend } from '@/components/dashboard/resolution-trend';
|
||||||
import { QueueHeatmap } from '@/components/dashboard/queue-heatmap';
|
import { QueueHeatmap } from '@/components/dashboard/queue-heatmap';
|
||||||
|
import { QueuePreferencesPopover } from '@/components/dashboard/queue-preferences-popover';
|
||||||
import { ActiveEngineers } from '@/components/dashboard/active-engineers';
|
import { ActiveEngineers } from '@/components/dashboard/active-engineers';
|
||||||
import {
|
import {
|
||||||
RefreshCw,
|
RefreshCw,
|
||||||
|
|
@ -89,6 +90,32 @@ interface Trends {
|
||||||
name: string;
|
name: string;
|
||||||
hours: number;
|
hours: number;
|
||||||
ticketsTouched: number;
|
ticketsTouched: number;
|
||||||
|
tickets: Array<{
|
||||||
|
id: string;
|
||||||
|
ticketNumber: string | null;
|
||||||
|
title: string | null;
|
||||||
|
description: string | null;
|
||||||
|
statusLabel: string | null;
|
||||||
|
hours: number;
|
||||||
|
}>;
|
||||||
|
isPto: boolean;
|
||||||
|
ptoNote: string | null;
|
||||||
|
}>;
|
||||||
|
ptoEngineers: Array<{
|
||||||
|
resourceId: string;
|
||||||
|
name: string;
|
||||||
|
hours: number;
|
||||||
|
ticketsTouched: number;
|
||||||
|
tickets: Array<{
|
||||||
|
id: string;
|
||||||
|
ticketNumber: string | null;
|
||||||
|
title: string | null;
|
||||||
|
description: string | null;
|
||||||
|
statusLabel: string | null;
|
||||||
|
hours: number;
|
||||||
|
}>;
|
||||||
|
isPto: boolean;
|
||||||
|
ptoNote: string | null;
|
||||||
}>;
|
}>;
|
||||||
}
|
}
|
||||||
|
|
||||||
|
|
@ -270,11 +297,12 @@ export default function DashboardPage() {
|
||||||
{/* QUEUE POSTURE ------------------------------------------------ */}
|
{/* QUEUE POSTURE ------------------------------------------------ */}
|
||||||
<div className="grid grid-cols-1 lg:grid-cols-12 gap-6">
|
<div className="grid grid-cols-1 lg:grid-cols-12 gap-6">
|
||||||
<Card className="lg:col-span-8">
|
<Card className="lg:col-span-8">
|
||||||
<CardHeader className="pb-3">
|
<CardHeader className="pb-3 flex flex-row items-center justify-between space-y-0">
|
||||||
<CardTitle className="text-base flex items-center gap-2">
|
<CardTitle className="text-base flex items-center gap-2">
|
||||||
<Layers className="h-4 w-4" />
|
<Layers className="h-4 w-4" />
|
||||||
Queue posture
|
Queue posture
|
||||||
</CardTitle>
|
</CardTitle>
|
||||||
|
<QueuePreferencesPopover onSaved={load} />
|
||||||
</CardHeader>
|
</CardHeader>
|
||||||
<CardContent>
|
<CardContent>
|
||||||
{!trends ? (
|
{!trends ? (
|
||||||
|
|
@ -295,7 +323,10 @@ export default function DashboardPage() {
|
||||||
{!trends ? (
|
{!trends ? (
|
||||||
<SkeletonChart height={196} />
|
<SkeletonChart height={196} />
|
||||||
) : (
|
) : (
|
||||||
<ActiveEngineers data={trends.activeEngineers} />
|
<ActiveEngineers
|
||||||
|
working={trends.activeEngineers}
|
||||||
|
pto={trends.ptoEngineers}
|
||||||
|
/>
|
||||||
)}
|
)}
|
||||||
</CardContent>
|
</CardContent>
|
||||||
</Card>
|
</Card>
|
||||||
|
|
|
||||||
|
|
@ -26,6 +26,7 @@ interface AgingBucket { balance: number; count: number; }
|
||||||
interface FinanceData {
|
interface FinanceData {
|
||||||
summary: {
|
summary: {
|
||||||
total_ar: number; total_ar_count: number;
|
total_ar: number; total_ar_count: number;
|
||||||
|
total_ar_gross?: number; unapplied_credits?: number;
|
||||||
current_balance: number; current_count: number;
|
current_balance: number; current_count: number;
|
||||||
overdue_balance: number; overdue_count: number;
|
overdue_balance: number; overdue_count: number;
|
||||||
paid_mtd: number; paid_ytd: number;
|
paid_mtd: number; paid_ytd: number;
|
||||||
|
|
@ -178,7 +179,11 @@ export default function MobileFinance() {
|
||||||
<KpiCardMobile
|
<KpiCardMobile
|
||||||
label="TOTAL AR"
|
label="TOTAL AR"
|
||||||
value={fmt$(summary.total_ar)}
|
value={fmt$(summary.total_ar)}
|
||||||
caption={`${summary.total_ar_count} open invoices`}
|
caption={
|
||||||
|
summary.unapplied_credits && summary.unapplied_credits > 0
|
||||||
|
? `${summary.total_ar_count} open · −${fmt$(summary.unapplied_credits)} credits`
|
||||||
|
: `${summary.total_ar_count} open invoices`
|
||||||
|
}
|
||||||
/>
|
/>
|
||||||
<KpiCardMobile
|
<KpiCardMobile
|
||||||
label="CURRENT"
|
label="CURRENT"
|
||||||
|
|
|
||||||
|
|
@ -1,26 +1,66 @@
|
||||||
/* ActiveEngineers — top engineers today by hours logged.
|
/* ActiveEngineers — today's engineers grouped by working vs PTO.
|
||||||
*
|
*
|
||||||
* Compact list: name + ticket count + hours bar. Sorted by hours
|
* Top section: engineers with billable/work time entries today. Each row's
|
||||||
* desc upstream. Empty when no time has been logged yet today. */
|
* ticket count is a button that opens a dialog listing the specific tickets
|
||||||
|
* they logged time against (and hours per ticket).
|
||||||
|
*
|
||||||
|
* Bottom section: collapsible list of engineers whose only time today is on a
|
||||||
|
* PTO/Vacation allocation code. Empty when nobody is out. */
|
||||||
|
|
||||||
'use client';
|
'use client';
|
||||||
|
|
||||||
import { Activity } from 'lucide-react';
|
import { useState } from 'react';
|
||||||
|
import { Activity, ChevronDown, Palmtree } from 'lucide-react';
|
||||||
import { EmptyState } from '@/components/ui/empty-state';
|
import { EmptyState } from '@/components/ui/empty-state';
|
||||||
|
import {
|
||||||
|
Dialog,
|
||||||
|
DialogContent,
|
||||||
|
DialogHeader,
|
||||||
|
DialogTitle,
|
||||||
|
DialogDescription,
|
||||||
|
} from '@/components/ui/dialog';
|
||||||
|
import {
|
||||||
|
Collapsible,
|
||||||
|
CollapsibleContent,
|
||||||
|
CollapsibleTrigger,
|
||||||
|
} from '@/components/ui/collapsible';
|
||||||
|
import {
|
||||||
|
Popover,
|
||||||
|
PopoverContent,
|
||||||
|
PopoverTrigger,
|
||||||
|
} from '@/components/ui/popover';
|
||||||
|
import { Badge } from '@/components/ui/badge';
|
||||||
|
import { cn } from '@/lib/utils';
|
||||||
|
|
||||||
|
interface Ticket {
|
||||||
|
id: string;
|
||||||
|
ticketNumber: string | null;
|
||||||
|
title: string | null;
|
||||||
|
description: string | null;
|
||||||
|
statusLabel: string | null;
|
||||||
|
hours: number;
|
||||||
|
}
|
||||||
|
|
||||||
interface Engineer {
|
interface Engineer {
|
||||||
resourceId: string;
|
resourceId: string;
|
||||||
name: string;
|
name: string;
|
||||||
hours: number;
|
hours: number;
|
||||||
ticketsTouched: number;
|
ticketsTouched: number;
|
||||||
|
tickets: Ticket[];
|
||||||
|
isPto: boolean;
|
||||||
|
ptoNote: string | null;
|
||||||
}
|
}
|
||||||
|
|
||||||
interface ActiveEngineersProps {
|
interface ActiveEngineersProps {
|
||||||
data: Engineer[];
|
working: Engineer[];
|
||||||
|
pto: Engineer[];
|
||||||
}
|
}
|
||||||
|
|
||||||
export function ActiveEngineers({ data }: ActiveEngineersProps) {
|
export function ActiveEngineers({ working, pto }: ActiveEngineersProps) {
|
||||||
if (data.length === 0) {
|
const [ticketsFor, setTicketsFor] = useState<Engineer | null>(null);
|
||||||
|
const [ptoOpen, setPtoOpen] = useState(false);
|
||||||
|
|
||||||
|
if (working.length === 0 && pto.length === 0) {
|
||||||
return (
|
return (
|
||||||
<EmptyState
|
<EmptyState
|
||||||
icon={Activity}
|
icon={Activity}
|
||||||
|
|
@ -31,16 +71,25 @@ export function ActiveEngineers({ data }: ActiveEngineersProps) {
|
||||||
);
|
);
|
||||||
}
|
}
|
||||||
|
|
||||||
const max = data.reduce((m, e) => Math.max(m, e.hours), 0) || 1;
|
const max = working.reduce((m, e) => Math.max(m, e.hours), 0) || 1;
|
||||||
const totalHours = data.reduce((s, e) => s + e.hours, 0);
|
const totalHours = working.reduce((s, e) => s + e.hours, 0);
|
||||||
const totalTickets = data.reduce((s, e) => s + e.ticketsTouched, 0);
|
const totalTickets = working.reduce((s, e) => s + e.ticketsTouched, 0);
|
||||||
|
|
||||||
return (
|
return (
|
||||||
|
<div className="space-y-3">
|
||||||
|
{working.length === 0 ? (
|
||||||
|
<div className="text-xs text-muted-foreground py-2">
|
||||||
|
No working time logged yet today.
|
||||||
|
</div>
|
||||||
|
) : (
|
||||||
<div className="space-y-1">
|
<div className="space-y-1">
|
||||||
{data.map((e) => {
|
{working.map((e) => {
|
||||||
const pct = (e.hours / max) * 100;
|
const pct = (e.hours / max) * 100;
|
||||||
return (
|
return (
|
||||||
<div key={e.resourceId} className="grid grid-cols-[1fr_auto] items-center gap-3 py-1">
|
<div
|
||||||
|
key={e.resourceId}
|
||||||
|
className="grid grid-cols-[1fr_auto] items-center gap-3 py-1"
|
||||||
|
>
|
||||||
<div className="min-w-0">
|
<div className="min-w-0">
|
||||||
<div className="text-sm font-medium truncate">{e.name}</div>
|
<div className="text-sm font-medium truncate">{e.name}</div>
|
||||||
<div className="relative h-1 w-full bg-muted rounded-sm mt-1 overflow-hidden">
|
<div className="relative h-1 w-full bg-muted rounded-sm mt-1 overflow-hidden">
|
||||||
|
|
@ -52,10 +101,20 @@ export function ActiveEngineers({ data }: ActiveEngineersProps) {
|
||||||
</div>
|
</div>
|
||||||
<div className="text-right shrink-0">
|
<div className="text-right shrink-0">
|
||||||
<div className="num text-sm">{e.hours.toFixed(1)}h</div>
|
<div className="num text-sm">{e.hours.toFixed(1)}h</div>
|
||||||
<div className="text-xs text-muted-foreground">
|
<button
|
||||||
|
type="button"
|
||||||
|
onClick={() => setTicketsFor(e)}
|
||||||
|
disabled={e.ticketsTouched === 0}
|
||||||
|
className={cn(
|
||||||
|
'text-xs text-muted-foreground',
|
||||||
|
e.ticketsTouched > 0
|
||||||
|
? 'hover:text-foreground hover:underline cursor-pointer'
|
||||||
|
: 'cursor-default',
|
||||||
|
)}
|
||||||
|
>
|
||||||
<span className="num">{e.ticketsTouched}</span>{' '}
|
<span className="num">{e.ticketsTouched}</span>{' '}
|
||||||
ticket{e.ticketsTouched === 1 ? '' : 's'}
|
ticket{e.ticketsTouched === 1 ? '' : 's'}
|
||||||
</div>
|
</button>
|
||||||
</div>
|
</div>
|
||||||
</div>
|
</div>
|
||||||
);
|
);
|
||||||
|
|
@ -69,5 +128,139 @@ export function ActiveEngineers({ data }: ActiveEngineersProps) {
|
||||||
</span>
|
</span>
|
||||||
</div>
|
</div>
|
||||||
</div>
|
</div>
|
||||||
|
)}
|
||||||
|
|
||||||
|
{pto.length > 0 && (
|
||||||
|
<Collapsible open={ptoOpen} onOpenChange={setPtoOpen}>
|
||||||
|
<CollapsibleTrigger
|
||||||
|
className={cn(
|
||||||
|
'flex w-full items-center justify-between gap-2 rounded-md border bg-muted/40',
|
||||||
|
'px-2 py-1.5 text-xs text-muted-foreground hover:bg-muted',
|
||||||
|
)}
|
||||||
|
>
|
||||||
|
<span className="flex items-center gap-1.5">
|
||||||
|
<Palmtree className="h-3.5 w-3.5" />
|
||||||
|
<span>
|
||||||
|
<span className="num">{pto.length}</span> on PTO / Vacation
|
||||||
|
</span>
|
||||||
|
</span>
|
||||||
|
<ChevronDown
|
||||||
|
className={cn(
|
||||||
|
'h-3.5 w-3.5 transition-transform',
|
||||||
|
ptoOpen && 'rotate-180',
|
||||||
|
)}
|
||||||
|
/>
|
||||||
|
</CollapsibleTrigger>
|
||||||
|
<CollapsibleContent className="pt-2">
|
||||||
|
<ul className="space-y-1">
|
||||||
|
{pto.map((e) => (
|
||||||
|
<li
|
||||||
|
key={e.resourceId}
|
||||||
|
className="flex items-center justify-between gap-3 text-sm py-0.5"
|
||||||
|
>
|
||||||
|
<span className="truncate">{e.name}</span>
|
||||||
|
{e.ptoNote ? (
|
||||||
|
<span
|
||||||
|
className="text-xs text-muted-foreground truncate max-w-[60%]"
|
||||||
|
title={e.ptoNote}
|
||||||
|
>
|
||||||
|
{e.ptoNote}
|
||||||
|
</span>
|
||||||
|
) : null}
|
||||||
|
</li>
|
||||||
|
))}
|
||||||
|
</ul>
|
||||||
|
</CollapsibleContent>
|
||||||
|
</Collapsible>
|
||||||
|
)}
|
||||||
|
|
||||||
|
<Dialog open={!!ticketsFor} onOpenChange={(open) => !open && setTicketsFor(null)}>
|
||||||
|
<DialogContent className="max-w-md">
|
||||||
|
<DialogHeader>
|
||||||
|
<DialogTitle>{ticketsFor?.name} · today</DialogTitle>
|
||||||
|
<DialogDescription>
|
||||||
|
{ticketsFor
|
||||||
|
? `${ticketsFor.hours.toFixed(1)}h across ${ticketsFor.ticketsTouched} ticket${
|
||||||
|
ticketsFor.ticketsTouched === 1 ? '' : 's'
|
||||||
|
}`
|
||||||
|
: ''}
|
||||||
|
</DialogDescription>
|
||||||
|
</DialogHeader>
|
||||||
|
<ul className="divide-y max-h-[60vh] overflow-y-auto">
|
||||||
|
{ticketsFor?.tickets.map((t) => (
|
||||||
|
<TicketRow key={t.id} ticket={t} />
|
||||||
|
))}
|
||||||
|
</ul>
|
||||||
|
</DialogContent>
|
||||||
|
</Dialog>
|
||||||
|
</div>
|
||||||
|
);
|
||||||
|
}
|
||||||
|
|
||||||
|
/* Single ticket row inside the per-engineer dialog. Hovering anywhere on the
|
||||||
|
* row reveals a popover with the ticket's current status and description. */
|
||||||
|
function TicketRow({ ticket: t }: { ticket: Ticket }) {
|
||||||
|
const [open, setOpen] = useState(false);
|
||||||
|
const hasHoverDetail = !!(t.description || t.statusLabel);
|
||||||
|
|
||||||
|
const row = (
|
||||||
|
<li
|
||||||
|
className="py-2 flex items-start justify-between gap-3"
|
||||||
|
onMouseEnter={hasHoverDetail ? () => setOpen(true) : undefined}
|
||||||
|
onMouseLeave={hasHoverDetail ? () => setOpen(false) : undefined}
|
||||||
|
>
|
||||||
|
<div className="min-w-0">
|
||||||
|
<a
|
||||||
|
href={`https://ww1.autotask.net/Mvc/ServiceDesk/TicketDetail.mvc?workspace=False&ids%5B0%5D=${t.id}&ticketId=${t.id}`}
|
||||||
|
target="_blank"
|
||||||
|
rel="noopener noreferrer"
|
||||||
|
className="text-sm font-medium hover:underline"
|
||||||
|
>
|
||||||
|
{t.ticketNumber ?? `#${t.id}`}
|
||||||
|
</a>
|
||||||
|
{t.title ? (
|
||||||
|
<div className="text-xs text-muted-foreground truncate">{t.title}</div>
|
||||||
|
) : null}
|
||||||
|
</div>
|
||||||
|
<span className="num text-sm shrink-0">{t.hours.toFixed(2)}h</span>
|
||||||
|
</li>
|
||||||
|
);
|
||||||
|
|
||||||
|
if (!hasHoverDetail) return row;
|
||||||
|
|
||||||
|
return (
|
||||||
|
<Popover open={open} onOpenChange={setOpen}>
|
||||||
|
<PopoverTrigger asChild>{row}</PopoverTrigger>
|
||||||
|
<PopoverContent
|
||||||
|
side="left"
|
||||||
|
align="start"
|
||||||
|
sideOffset={8}
|
||||||
|
className="w-80"
|
||||||
|
onOpenAutoFocus={(e) => e.preventDefault()}
|
||||||
|
>
|
||||||
|
<div className="space-y-2">
|
||||||
|
{t.statusLabel ? (
|
||||||
|
<div className="flex items-center gap-2">
|
||||||
|
<span className="text-xs text-muted-foreground">Status</span>
|
||||||
|
<Badge variant="secondary" className="text-xs">
|
||||||
|
{t.statusLabel}
|
||||||
|
</Badge>
|
||||||
|
</div>
|
||||||
|
) : null}
|
||||||
|
{t.description ? (
|
||||||
|
<div>
|
||||||
|
<div className="text-xs text-muted-foreground mb-1">Description</div>
|
||||||
|
<p className="text-sm whitespace-pre-wrap break-words max-h-48 overflow-y-auto">
|
||||||
|
{t.description}
|
||||||
|
</p>
|
||||||
|
</div>
|
||||||
|
) : (
|
||||||
|
<div className="text-xs text-muted-foreground italic">
|
||||||
|
No description on this ticket.
|
||||||
|
</div>
|
||||||
|
)}
|
||||||
|
</div>
|
||||||
|
</PopoverContent>
|
||||||
|
</Popover>
|
||||||
);
|
);
|
||||||
}
|
}
|
||||||
|
|
|
||||||
137
components/dashboard/queue-preferences-popover.tsx
Normal file
137
components/dashboard/queue-preferences-popover.tsx
Normal file
|
|
@ -0,0 +1,137 @@
|
||||||
|
'use client';
|
||||||
|
|
||||||
|
/* Queue preferences popover — gear button on the Queue posture card.
|
||||||
|
*
|
||||||
|
* Lists every active queue with a Switch per row. Toggling persists to
|
||||||
|
* /api/me/queue-preferences (full-set PUT) and calls onSaved() so the
|
||||||
|
* parent can refetch trends and the heatmap drops the hidden rows.
|
||||||
|
*
|
||||||
|
* State is owned by this component; it lazy-loads queue data on first
|
||||||
|
* open to keep the dashboard's initial paint cheap. */
|
||||||
|
|
||||||
|
import { useEffect, useState } from 'react';
|
||||||
|
import { Settings2, Loader2, Search } from 'lucide-react';
|
||||||
|
import { toast } from 'sonner';
|
||||||
|
import { Button } from '@/components/ui/button';
|
||||||
|
import { Popover, PopoverContent, PopoverTrigger } from '@/components/ui/popover';
|
||||||
|
import { Switch } from '@/components/ui/switch';
|
||||||
|
import { Input } from '@/components/ui/input';
|
||||||
|
|
||||||
|
interface QueueRow {
|
||||||
|
id: number;
|
||||||
|
label: string;
|
||||||
|
hidden: boolean;
|
||||||
|
}
|
||||||
|
|
||||||
|
interface Props {
|
||||||
|
onSaved: () => void;
|
||||||
|
}
|
||||||
|
|
||||||
|
export function QueuePreferencesPopover({ onSaved }: Props) {
|
||||||
|
const [open, setOpen] = useState(false);
|
||||||
|
const [queues, setQueues] = useState<QueueRow[] | null>(null);
|
||||||
|
const [loading, setLoading] = useState(false);
|
||||||
|
const [saving, setSaving] = useState<number | null>(null);
|
||||||
|
const [filter, setFilter] = useState('');
|
||||||
|
|
||||||
|
useEffect(() => {
|
||||||
|
if (!open || queues !== null) return;
|
||||||
|
setLoading(true);
|
||||||
|
fetch('/api/me/queue-preferences')
|
||||||
|
.then((r) => r.json())
|
||||||
|
.then((data: { queues: QueueRow[] }) => setQueues(data.queues))
|
||||||
|
.catch(() => toast.error('Failed to load queues'))
|
||||||
|
.finally(() => setLoading(false));
|
||||||
|
}, [open, queues]);
|
||||||
|
|
||||||
|
async function toggle(queueId: number, nextHidden: boolean) {
|
||||||
|
if (!queues) return;
|
||||||
|
const previous = queues;
|
||||||
|
const next = queues.map((q) => (q.id === queueId ? { ...q, hidden: nextHidden } : q));
|
||||||
|
setQueues(next);
|
||||||
|
setSaving(queueId);
|
||||||
|
try {
|
||||||
|
const hiddenIds = next.filter((q) => q.hidden).map((q) => q.id);
|
||||||
|
const res = await fetch('/api/me/queue-preferences', {
|
||||||
|
method: 'PUT',
|
||||||
|
headers: { 'Content-Type': 'application/json' },
|
||||||
|
body: JSON.stringify({ hiddenIds }),
|
||||||
|
});
|
||||||
|
if (!res.ok) throw new Error(`HTTP ${res.status}`);
|
||||||
|
onSaved();
|
||||||
|
} catch {
|
||||||
|
setQueues(previous);
|
||||||
|
toast.error('Failed to update queue preference');
|
||||||
|
} finally {
|
||||||
|
setSaving(null);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
const visible = queues?.filter((q) =>
|
||||||
|
filter ? q.label.toLowerCase().includes(filter.toLowerCase()) : true,
|
||||||
|
);
|
||||||
|
|
||||||
|
return (
|
||||||
|
<Popover open={open} onOpenChange={setOpen}>
|
||||||
|
<PopoverTrigger asChild>
|
||||||
|
<Button
|
||||||
|
variant="ghost"
|
||||||
|
size="icon"
|
||||||
|
className="h-7 w-7"
|
||||||
|
aria-label="Configure visible queues"
|
||||||
|
>
|
||||||
|
<Settings2 className="h-4 w-4" />
|
||||||
|
</Button>
|
||||||
|
</PopoverTrigger>
|
||||||
|
<PopoverContent align="end" className="w-80 p-0">
|
||||||
|
<div className="p-3 border-b">
|
||||||
|
<p className="text-sm font-medium">Visible queues</p>
|
||||||
|
<p className="text-xs text-muted-foreground mt-0.5">
|
||||||
|
Toggle off to hide a queue from your dashboard.
|
||||||
|
</p>
|
||||||
|
</div>
|
||||||
|
<div className="p-3 border-b">
|
||||||
|
<div className="relative">
|
||||||
|
<Search className="absolute left-2 top-1/2 -translate-y-1/2 h-3.5 w-3.5 text-muted-foreground" />
|
||||||
|
<Input
|
||||||
|
value={filter}
|
||||||
|
onChange={(e) => setFilter(e.target.value)}
|
||||||
|
placeholder="Filter queues…"
|
||||||
|
className="pl-7 h-8 text-sm"
|
||||||
|
/>
|
||||||
|
</div>
|
||||||
|
</div>
|
||||||
|
<div className="max-h-80 overflow-y-auto">
|
||||||
|
{loading || queues === null ? (
|
||||||
|
<div className="flex items-center justify-center py-8">
|
||||||
|
<Loader2 className="h-4 w-4 animate-spin text-muted-foreground" />
|
||||||
|
</div>
|
||||||
|
) : visible && visible.length === 0 ? (
|
||||||
|
<p className="text-sm text-muted-foreground text-center py-6">
|
||||||
|
No queues match.
|
||||||
|
</p>
|
||||||
|
) : (
|
||||||
|
<ul className="py-1">
|
||||||
|
{visible!.map((q) => (
|
||||||
|
<li
|
||||||
|
key={q.id}
|
||||||
|
className="flex items-center justify-between gap-3 px-3 py-2 hover:bg-accent/40"
|
||||||
|
>
|
||||||
|
<span className="text-sm truncate" title={q.label}>
|
||||||
|
{q.label}
|
||||||
|
</span>
|
||||||
|
<Switch
|
||||||
|
checked={!q.hidden}
|
||||||
|
onCheckedChange={(checked) => toggle(q.id, !checked)}
|
||||||
|
disabled={saving === q.id}
|
||||||
|
aria-label={`${q.hidden ? 'Show' : 'Hide'} ${q.label}`}
|
||||||
|
/>
|
||||||
|
</li>
|
||||||
|
))}
|
||||||
|
</ul>
|
||||||
|
)}
|
||||||
|
</div>
|
||||||
|
</PopoverContent>
|
||||||
|
</Popover>
|
||||||
|
);
|
||||||
|
}
|
||||||
|
|
@ -14,6 +14,11 @@ services:
|
||||||
interval: 5s
|
interval: 5s
|
||||||
timeout: 3s
|
timeout: 3s
|
||||||
retries: 5
|
retries: 5
|
||||||
|
logging:
|
||||||
|
driver: json-file
|
||||||
|
options:
|
||||||
|
max-size: "10m"
|
||||||
|
max-file: "5"
|
||||||
|
|
||||||
# PostgreSQL database for Autotask sync
|
# PostgreSQL database for Autotask sync
|
||||||
postgres:
|
postgres:
|
||||||
|
|
@ -36,6 +41,11 @@ services:
|
||||||
interval: 10s
|
interval: 10s
|
||||||
timeout: 5s
|
timeout: 5s
|
||||||
retries: 5
|
retries: 5
|
||||||
|
logging:
|
||||||
|
driver: json-file
|
||||||
|
options:
|
||||||
|
max-size: "10m"
|
||||||
|
max-file: "5"
|
||||||
|
|
||||||
# Next.js application on port 3100 (instead of 3000)
|
# Next.js application on port 3100 (instead of 3000)
|
||||||
app:
|
app:
|
||||||
|
|
@ -140,6 +150,11 @@ services:
|
||||||
volumes:
|
volumes:
|
||||||
# Mount .env.local for development (remove in production)
|
# Mount .env.local for development (remove in production)
|
||||||
- ./.env.local:/app/.env.local:ro
|
- ./.env.local:/app/.env.local:ro
|
||||||
|
logging:
|
||||||
|
driver: json-file
|
||||||
|
options:
|
||||||
|
max-size: "10m"
|
||||||
|
max-file: "5"
|
||||||
|
|
||||||
volumes:
|
volumes:
|
||||||
redis_data:
|
redis_data:
|
||||||
|
|
|
||||||
221
docs/superpowers/specs/2026-05-03-mobile-shell-design.md
Normal file
221
docs/superpowers/specs/2026-05-03-mobile-shell-design.md
Normal file
|
|
@ -0,0 +1,221 @@
|
||||||
|
# Mobile Shell Redesign — Design Spec
|
||||||
|
|
||||||
|
**Date:** 2026-05-03
|
||||||
|
**Scope:** `/mobile/*` shell, navigation, and per-page layouts on Pulse.
|
||||||
|
|
||||||
|
## 1. Goal & audience
|
||||||
|
|
||||||
|
Pulse's mobile shell exists for **managers on the go** — quick status checks,
|
||||||
|
triage decisions, and read-only awareness. It is not a full replacement for the
|
||||||
|
desktop app; pages where mobile editing isn't justified should link out to the
|
||||||
|
desktop equivalent.
|
||||||
|
|
||||||
|
Success means a manager can:
|
||||||
|
|
||||||
|
- See the state of the business at a glance (Dashboard).
|
||||||
|
- Triage and inspect tickets (Tickets).
|
||||||
|
- Read AR / invoice / payment status (Finance).
|
||||||
|
- Skim recent AI ticket analyses (Analyzer).
|
||||||
|
- Reach Engagement and sign-out without clutter on the primary nav.
|
||||||
|
|
||||||
|
## 2. Approach
|
||||||
|
|
||||||
|
**Rebuild `/mobile` in place.** Keep all existing `/mobile/*` route paths
|
||||||
|
(`/mobile/dashboard`, `/mobile/tickets`, `/mobile/tickets/[id]`,
|
||||||
|
`/mobile/finance`). Replace `/mobile/layout.tsx`, the four page files, and
|
||||||
|
delete `/mobile/nav` (its content moves into a Sheet drawer — see §3).
|
||||||
|
|
||||||
|
No new top-level routes. No parallel `/mobile-v2` directory. The existing pages
|
||||||
|
stay their canonical URLs throughout the redesign.
|
||||||
|
|
||||||
|
## 3. Navigation
|
||||||
|
|
||||||
|
### 3.1 Bottom tab bar (primary)
|
||||||
|
|
||||||
|
Four tabs, equal-width, fixed to the bottom of the viewport:
|
||||||
|
|
||||||
|
| Tab | Icon (lucide) | Route |
|
||||||
|
|------------|-------------------|---------------------|
|
||||||
|
| Dashboard | `LayoutDashboard` | `/mobile/dashboard` |
|
||||||
|
| Tickets | `Ticket` | `/mobile/tickets` |
|
||||||
|
| Finance | `DollarSign` | `/mobile/finance` |
|
||||||
|
| Analyzer | `Sparkles` | `/mobile/analyzer` |
|
||||||
|
|
||||||
|
Active state: `text-primary`; inactive: `text-muted-foreground`. Active
|
||||||
|
detection via `pathname.startsWith(href)`.
|
||||||
|
|
||||||
|
### 3.2 "More" Sheet drawer (secondary)
|
||||||
|
|
||||||
|
A fifth control on the bottom bar — `Menu` icon labelled **More** — opens a
|
||||||
|
shadcn `Sheet` (side="right", or "bottom" on phones; pick one and stay
|
||||||
|
consistent). The drawer **replaces the standalone `/mobile/nav` page entirely**.
|
||||||
|
|
||||||
|
Drawer contents, top to bottom:
|
||||||
|
|
||||||
|
1. **Mobile sections** — Engagement (`/mobile/engagement`, see §6.5).
|
||||||
|
2. **Full site** — link list to desktop pages that have no mobile view yet
|
||||||
|
(Quotes, Configuration Items, Backup Status, Ticket Digest, Admin / Sync).
|
||||||
|
Each link uses the `ExternalLink` icon hint to signal "leaves mobile shell".
|
||||||
|
3. **Account** — current user (avatar + email, read-only), then **Sign out**
|
||||||
|
(calls `signOut()` then `router.push('/auth/sign-in')`).
|
||||||
|
|
||||||
|
Delete `app/mobile/nav/page.tsx` after the drawer is in place.
|
||||||
|
|
||||||
|
## 4. PWA setup
|
||||||
|
|
||||||
|
- `public/manifest.json` — name "Pulse", short_name "Pulse", `display: "standalone"`,
|
||||||
|
`start_url: "/mobile"`, theme/background colors matching the dark and light
|
||||||
|
shells. Reference it from `app/layout.tsx` via `<link rel="manifest">`.
|
||||||
|
- `viewport` meta in `app/layout.tsx` (or root metadata): include
|
||||||
|
`viewport-fit=cover` so the shell can paint behind the home indicator.
|
||||||
|
- **Safe-area insets** — both the sticky header and the bottom tab bar add
|
||||||
|
`env(safe-area-inset-top)` / `env(safe-area-inset-bottom)` padding. Use
|
||||||
|
Tailwind arbitrary values (`pt-[env(safe-area-inset-top)]`,
|
||||||
|
`pb-[env(safe-area-inset-bottom)]`) or a small utility class.
|
||||||
|
- **No service worker, no offline mode** in this iteration. Don't ship
|
||||||
|
`next-pwa` or a custom SW — deferred until a clear offline use-case lands.
|
||||||
|
- **Tablet** — `md:max-w-2xl mx-auto` wrapper is noted as a follow-up but is
|
||||||
|
**not** part of this spec's deliverable. Keep the current `max-w-lg`.
|
||||||
|
|
||||||
|
## 5. Shell (`app/mobile/layout.tsx`)
|
||||||
|
|
||||||
|
### 5.1 Header
|
||||||
|
|
||||||
|
Sticky top, `bg-background/95 backdrop-blur`, bottom border. Three slots:
|
||||||
|
|
||||||
|
- **Left:** Wulf mark (existing logo asset) + "Pulse" wordmark, linked to
|
||||||
|
`/mobile/dashboard`. Use the actual brand mark, not the text-only fallback.
|
||||||
|
- **Right (in order):**
|
||||||
|
- `Bell` icon button — **placeholder only**, no menu, no badge logic. Wire
|
||||||
|
to `aria-label="Notifications"` and an empty `onClick` so it's
|
||||||
|
keyboard-accessible. A future phase will turn this into a real list.
|
||||||
|
- Compact user avatar (`Avatar` from shadcn, size `h-7 w-7`). On tap,
|
||||||
|
opens the More drawer (so the avatar is the entry point alongside the
|
||||||
|
bottom-bar More button).
|
||||||
|
|
||||||
|
No page title in the header — pages render their own H1 in the content area.
|
||||||
|
|
||||||
|
### 5.2 Content area
|
||||||
|
|
||||||
|
`<main>` between header and bottom nav, scrollable. Add bottom padding equal to
|
||||||
|
the bottom-nav height + safe-area inset so content doesn't hide under the bar.
|
||||||
|
|
||||||
|
### 5.3 Bottom nav
|
||||||
|
|
||||||
|
Fixed, full-width, `border-t bg-background`. Wraps in `max-w-lg mx-auto` so
|
||||||
|
content and nav share the same gutter. Five cells: 4 tabs (§3.1) + More (§3.2).
|
||||||
|
|
||||||
|
## 6. Pages
|
||||||
|
|
||||||
|
### 6.1 Dashboard (`/mobile/dashboard`)
|
||||||
|
|
||||||
|
Sections, top to bottom:
|
||||||
|
|
||||||
|
1. **2×2 KPI grid** — four primary metric cards. KPIs to choose from the
|
||||||
|
desktop dashboard's hero stats; final selection during build, but the layout
|
||||||
|
is fixed at 2×2 on phone widths.
|
||||||
|
2. **"Needs Attention" strip** — horizontal scroll of compact cards
|
||||||
|
surfacing items that require manager action (overdue tickets, failed
|
||||||
|
backups, stalled workflows). Each card opens its detail view directly.
|
||||||
|
3. **Backup / worker status** — compact status row showing the analyzer
|
||||||
|
worker, RMM worker, and backup-success-rate at a glance. Read-only; tap
|
||||||
|
opens the desktop admin page.
|
||||||
|
|
||||||
|
No charts on the mobile Dashboard in this iteration — recharts on small widths
|
||||||
|
isn't earning its weight.
|
||||||
|
|
||||||
|
### 6.2 Tickets (`/mobile/tickets`)
|
||||||
|
|
||||||
|
- **Collapsible filter strip** at top (`Collapsible` from shadcn, default
|
||||||
|
collapsed). When expanded: status, priority, queue, assigned-to-me toggle.
|
||||||
|
Filter state syncs to the URL query string so deep-linking works.
|
||||||
|
- **Priority-bar rows** — list rows have a left-edge color stripe by priority
|
||||||
|
(Critical/High/Medium/Low → red/orange/amber/slate). Body shows ticket #,
|
||||||
|
title, company, age, assignee. Single-tap opens detail.
|
||||||
|
- **Infinite scroll** — replace the current pagination with cursor-based
|
||||||
|
infinite scroll. Fetch in pages of ~25; trigger next page when the last row
|
||||||
|
enters the viewport (IntersectionObserver). Keep a "Load more" fallback button
|
||||||
|
for accessibility.
|
||||||
|
- **Detail page** — keep `/mobile/tickets/[id]/page.tsx` largely as-is; only
|
||||||
|
reskin the header to match the new shell (Wulf mark, breadcrumb back).
|
||||||
|
|
||||||
|
### 6.3 Finance (`/mobile/finance`)
|
||||||
|
|
||||||
|
Restyle of the existing page only. No new data, no new sections. Adopt the new
|
||||||
|
Card and typography scale, fix any spacing that breaks on small phones. If a
|
||||||
|
section currently relies on a wide table, swap it for a stacked list on mobile.
|
||||||
|
|
||||||
|
### 6.4 Analyzer (`/mobile/analyzer`) — NEW
|
||||||
|
|
||||||
|
**Read-only feed of recent AI ticket analyses.**
|
||||||
|
|
||||||
|
- List view: most-recent-first stream of analyses. Each row shows ticket #,
|
||||||
|
title, the analyzer's one-line summary, confidence badge, and a stage
|
||||||
|
indicator (Triage → Analyze → Deep Review). Tap opens a mobile summary view.
|
||||||
|
- Mobile summary view: renders **Summary**, **Next Step**, **Next Step
|
||||||
|
Rationale** (all already produced by the analyzer pipeline). Includes a "View
|
||||||
|
full analysis" link out to the desktop analyzer page.
|
||||||
|
- No editing, no re-run, no prompt tuning on mobile.
|
||||||
|
- Source data: existing `analyzer_analyses` rows via a new
|
||||||
|
`/api/mobile/analyzer/feed` endpoint (or reuse an existing list endpoint if
|
||||||
|
one already returns the right shape).
|
||||||
|
|
||||||
|
### 6.5 Engagement (`/mobile/engagement`) — NEW, accessed via More drawer
|
||||||
|
|
||||||
|
Engagement gets a **real mobile refactor**, not a thin adaptation. The desktop
|
||||||
|
page (`app/engagement/page.tsx`, ~1300 lines) and profile page
|
||||||
|
(`app/engagement/profile/page.tsx`, ~650 lines) both rely on wide tables, dense
|
||||||
|
charts, and modal patterns that don't translate to phone widths. Build the
|
||||||
|
mobile views from the same data sources but with phone-first layouts.
|
||||||
|
|
||||||
|
**Overview (`/mobile/engagement`):**
|
||||||
|
|
||||||
|
- Period selector chip row (today / 7d / 30d) — sticky just below the page H1.
|
||||||
|
- Summary cards stacked single-column (active users, total Graph hours, total
|
||||||
|
Autotask hours, hours-per-active-user). No 4-up grid; reading numbers across
|
||||||
|
a 4-up row on a phone is a non-starter.
|
||||||
|
- Per-employee list as stacked rows (avatar / initials, name, role, hours
|
||||||
|
bar). Sortable via a small control above the list (sort by hours, name,
|
||||||
|
utilization). Search input above the list.
|
||||||
|
- Charts: replace the desktop's wide bar/line charts with one compact "hours
|
||||||
|
trend" sparkline at the top of the list, period-scoped. No multi-series
|
||||||
|
chart on mobile in this iteration.
|
||||||
|
|
||||||
|
**User profile (`/mobile/engagement/profile?userId=...` or
|
||||||
|
`/mobile/engagement/[userId]`):**
|
||||||
|
|
||||||
|
- Reuses the existing profile data endpoints. Layout is single-column:
|
||||||
|
identity header → period selector → key metrics (compact) → activity
|
||||||
|
breakdown list → recent items.
|
||||||
|
- This **replaces** the desktop user-detail modal pattern on mobile —
|
||||||
|
tapping a row navigates to a real page, not a modal, so the shell's back
|
||||||
|
gesture works correctly.
|
||||||
|
- Pick exactly one of the two route shapes above during build (prefer the
|
||||||
|
`[userId]` segment form for shareable URLs); don't ship both.
|
||||||
|
|
||||||
|
Engagement is **not** on the bottom bar — it lives in the More drawer because
|
||||||
|
managers don't check it as often as the four primary surfaces.
|
||||||
|
|
||||||
|
## 7. Out of scope (explicit non-goals)
|
||||||
|
|
||||||
|
- Service worker, offline cache, push notifications.
|
||||||
|
- Tablet breakpoint (md:max-w-2xl) — noted, deferred.
|
||||||
|
- Real notification list behind the bell.
|
||||||
|
- Mobile editing on Engagement (user detail) or Analyzer (re-run, prompt edits).
|
||||||
|
- Charts / recharts on mobile Dashboard.
|
||||||
|
- Replacing or restyling the desktop pages reachable from the More drawer.
|
||||||
|
|
||||||
|
## 8. Build order
|
||||||
|
|
||||||
|
1. PWA scaffolding — `manifest.json`, viewport meta, safe-area utility.
|
||||||
|
2. New `app/mobile/layout.tsx` — header, More drawer (Sheet), 5-cell bottom nav.
|
||||||
|
Delete `app/mobile/nav/page.tsx` in the same change.
|
||||||
|
3. Dashboard restyle (2×2 grid, Needs Attention, status row).
|
||||||
|
4. Tickets restyle (filter Collapsible, priority-bar rows, infinite scroll).
|
||||||
|
Detail page header reskin only.
|
||||||
|
5. Finance restyle.
|
||||||
|
6. Analyzer page + `/api/mobile/analyzer/feed` endpoint.
|
||||||
|
7. Engagement Overview (`/mobile/engagement`) — phone-first layout from existing data sources.
|
||||||
|
8. Engagement user profile (`/mobile/engagement/[userId]`) — replaces the desktop modal pattern with a real mobile page.
|
||||||
|
|
||||||
|
Each step ships independently — no big-bang merge.
|
||||||
|
|
@ -62,8 +62,9 @@ export class QboSyncService {
|
||||||
updatedSince = await this.getLastSyncTime();
|
updatedSince = await this.getLastSyncTime();
|
||||||
}
|
}
|
||||||
|
|
||||||
// Sync invoices
|
// Sync invoices. Pass syncType so a full sync can tombstone any
|
||||||
entities.push(await this.syncInvoices(realmId, updatedSince));
|
// rows QBO no longer returns (voided / deleted there).
|
||||||
|
entities.push(await this.syncInvoices(realmId, updatedSince, syncType));
|
||||||
|
|
||||||
// Sync payments
|
// Sync payments
|
||||||
entities.push(await this.syncPayments(realmId, updatedSince));
|
entities.push(await this.syncPayments(realmId, updatedSince));
|
||||||
|
|
@ -111,14 +112,20 @@ export class QboSyncService {
|
||||||
|
|
||||||
// ─── Entity Sync Methods ───────────────────────────────────────────────────
|
// ─── Entity Sync Methods ───────────────────────────────────────────────────
|
||||||
|
|
||||||
private async syncInvoices(realmId: string, updatedSince?: Date): Promise<QboEntitySyncResult> {
|
private async syncInvoices(
|
||||||
|
realmId: string,
|
||||||
|
updatedSince?: Date,
|
||||||
|
syncType: 'full' | 'incremental' = 'incremental',
|
||||||
|
): Promise<QboEntitySyncResult> {
|
||||||
const start = Date.now();
|
const start = Date.now();
|
||||||
try {
|
try {
|
||||||
const invoices = await this.client.getInvoices(updatedSince);
|
const invoices = await this.client.getInvoices(updatedSince);
|
||||||
console.log(`[QboSync] Fetched ${invoices.length} invoices`);
|
console.log(`[QboSync] Fetched ${invoices.length} invoices`);
|
||||||
|
|
||||||
let upserted = 0;
|
let upserted = 0;
|
||||||
|
const seenIds: string[] = [];
|
||||||
for (const inv of invoices) {
|
for (const inv of invoices) {
|
||||||
|
if (inv.Id) seenIds.push(inv.Id);
|
||||||
const status = this.deriveInvoiceStatus(inv);
|
const status = this.deriveInvoiceStatus(inv);
|
||||||
await postgresClient.query(
|
await postgresClient.query(
|
||||||
`INSERT INTO qbo_invoices
|
`INSERT INTO qbo_invoices
|
||||||
|
|
@ -141,7 +148,9 @@ export class QboSyncService {
|
||||||
linked_txns = EXCLUDED.linked_txns,
|
linked_txns = EXCLUDED.linked_txns,
|
||||||
sync_token = EXCLUDED.sync_token,
|
sync_token = EXCLUDED.sync_token,
|
||||||
qbo_updated_at = EXCLUDED.qbo_updated_at,
|
qbo_updated_at = EXCLUDED.qbo_updated_at,
|
||||||
synced_at = NOW()`,
|
synced_at = NOW(),
|
||||||
|
is_deleted = false,
|
||||||
|
deleted_at = NULL`,
|
||||||
[
|
[
|
||||||
inv.Id, realmId, inv.DocNumber ?? null,
|
inv.Id, realmId, inv.DocNumber ?? null,
|
||||||
inv.TxnDate ?? null, inv.DueDate ?? null,
|
inv.TxnDate ?? null, inv.DueDate ?? null,
|
||||||
|
|
@ -159,7 +168,40 @@ export class QboSyncService {
|
||||||
upserted++;
|
upserted++;
|
||||||
}
|
}
|
||||||
|
|
||||||
return { entity: 'invoices', success: true, recordsUpserted: upserted, duration: Date.now() - start };
|
// Tombstone pass — full syncs only.
|
||||||
|
//
|
||||||
|
// QBO's Invoice query never reports deletions; voided/deleted invoices
|
||||||
|
// just stop appearing. A full sync just fetched every live invoice
|
||||||
|
// for this realm, so any Pulse row not in `seenIds` is an orphan and
|
||||||
|
// should be soft-deleted so it stops counting toward A/R.
|
||||||
|
//
|
||||||
|
// Incremental syncs are skipped here — they only see recently-changed
|
||||||
|
// rows, so applying the same logic would tombstone the entire ledger.
|
||||||
|
let tombstoned = 0;
|
||||||
|
if (syncType === 'full' && seenIds.length > 0) {
|
||||||
|
const tombstoneRes = await postgresClient.query<{ id: string }>(
|
||||||
|
`UPDATE qbo_invoices
|
||||||
|
SET is_deleted = true,
|
||||||
|
deleted_at = NOW()
|
||||||
|
WHERE realm_id = $1
|
||||||
|
AND is_deleted = false
|
||||||
|
AND id <> ALL($2::text[])
|
||||||
|
RETURNING id`,
|
||||||
|
[realmId, seenIds],
|
||||||
|
);
|
||||||
|
tombstoned = tombstoneRes.rowCount ?? tombstoneRes.rows.length;
|
||||||
|
if (tombstoned > 0) {
|
||||||
|
console.log(`[QboSync] Tombstoned ${tombstoned} invoice(s) absent from full-sync response`);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
return {
|
||||||
|
entity: 'invoices',
|
||||||
|
success: true,
|
||||||
|
recordsUpserted: upserted,
|
||||||
|
recordsDeleted: tombstoned,
|
||||||
|
duration: Date.now() - start,
|
||||||
|
};
|
||||||
} catch (err) {
|
} catch (err) {
|
||||||
const msg = err instanceof Error ? err.message : String(err);
|
const msg = err instanceof Error ? err.message : String(err);
|
||||||
console.error(`[QboSync] Invoice sync failed: ${msg}`);
|
console.error(`[QboSync] Invoice sync failed: ${msg}`);
|
||||||
|
|
|
||||||
23
migrations/087_user_queue_preferences.sql
Normal file
23
migrations/087_user_queue_preferences.sql
Normal file
|
|
@ -0,0 +1,23 @@
|
||||||
|
-- =============================================================================
|
||||||
|
-- Per-user queue preferences
|
||||||
|
-- =============================================================================
|
||||||
|
-- Lets each user hide specific queues from their own dashboard widgets
|
||||||
|
-- (notably the Queue posture heatmap). Presence of a row = that user has
|
||||||
|
-- hidden that queue. Absence = visible.
|
||||||
|
--
|
||||||
|
-- Independent of the global company_scope filter — that affects KPI counts
|
||||||
|
-- for everyone, this only affects what an individual user chooses to see.
|
||||||
|
-- =============================================================================
|
||||||
|
|
||||||
|
CREATE TABLE IF NOT EXISTS user_queue_preferences (
|
||||||
|
user_id TEXT NOT NULL REFERENCES "user"(id) ON DELETE CASCADE,
|
||||||
|
queue_id INTEGER NOT NULL REFERENCES queues(value) ON DELETE CASCADE,
|
||||||
|
hidden_at TIMESTAMPTZ NOT NULL DEFAULT NOW(),
|
||||||
|
PRIMARY KEY (user_id, queue_id)
|
||||||
|
);
|
||||||
|
|
||||||
|
CREATE INDEX IF NOT EXISTS idx_user_queue_preferences_user
|
||||||
|
ON user_queue_preferences(user_id);
|
||||||
|
|
||||||
|
COMMENT ON TABLE user_queue_preferences IS
|
||||||
|
'Per-user hidden-queue list. Row presence = queue is hidden for that user in dashboard widgets.';
|
||||||
17
migrations/088_qbo_invoices_soft_delete.sql
Normal file
17
migrations/088_qbo_invoices_soft_delete.sql
Normal file
|
|
@ -0,0 +1,17 @@
|
||||||
|
-- Soft-delete support for qbo_invoices.
|
||||||
|
--
|
||||||
|
-- QBO's Invoice query endpoint only returns rows that still exist in QBO.
|
||||||
|
-- When an invoice is voided or deleted in QBO, it simply stops appearing in
|
||||||
|
-- responses — there is no "deleted" flag we can subscribe to. Without a
|
||||||
|
-- tombstone pass these orphan rows linger in Pulse forever and keep
|
||||||
|
-- inflating Total A/R (see the TNT Pizza incident, 2026-05-15).
|
||||||
|
--
|
||||||
|
-- A full QBO sync now compares the set of returned invoice IDs against
|
||||||
|
-- what Pulse has on file and flips is_deleted = true for any row the
|
||||||
|
-- query no longer returns. Finance queries filter on is_deleted = false.
|
||||||
|
|
||||||
|
ALTER TABLE qbo_invoices
|
||||||
|
ADD COLUMN IF NOT EXISTS is_deleted BOOLEAN NOT NULL DEFAULT false,
|
||||||
|
ADD COLUMN IF NOT EXISTS deleted_at TIMESTAMPTZ;
|
||||||
|
|
||||||
|
CREATE INDEX IF NOT EXISTS idx_qbo_invoices_is_deleted ON qbo_invoices(is_deleted);
|
||||||
12
scripts/check-699456-parse.ts
Normal file
12
scripts/check-699456-parse.ts
Normal file
|
|
@ -0,0 +1,12 @@
|
||||||
|
import { config } from 'dotenv';
|
||||||
|
import { resolve } from 'path';
|
||||||
|
config({ path: resolve('/opt/stacks/pulse/.env.local') });
|
||||||
|
|
||||||
|
import { parseAndStoreMessage } from '../lib/services/phishing-eml-service';
|
||||||
|
|
||||||
|
async function main() {
|
||||||
|
const result = await parseAndStoreMessage({ reportId: '6bdd5c3f-0844-4673-941b-d5b438739383', ticketId: 699456 });
|
||||||
|
console.log(JSON.stringify(result, null, 2));
|
||||||
|
process.exit(0);
|
||||||
|
}
|
||||||
|
main().catch((err) => { console.error(err); process.exit(1); });
|
||||||
31
scripts/check-suspect-tickets.ts
Normal file
31
scripts/check-suspect-tickets.ts
Normal file
|
|
@ -0,0 +1,31 @@
|
||||||
|
import { config } from 'dotenv';
|
||||||
|
import { resolve } from 'path';
|
||||||
|
config({ path: resolve(__dirname, '../.env.local') });
|
||||||
|
|
||||||
|
const SUSPECTS = [573672, 568742, 541321, 532302];
|
||||||
|
|
||||||
|
const headers = {
|
||||||
|
Username: process.env.AUTOTASK_USERNAME!,
|
||||||
|
Secret: process.env.AUTOTASK_SECRET!,
|
||||||
|
APIIntegrationcode: process.env.AUTOTASK_API_INTEGRATION_CODE!,
|
||||||
|
Accept: 'application/json',
|
||||||
|
};
|
||||||
|
|
||||||
|
async function main() {
|
||||||
|
for (const id of SUSPECTS) {
|
||||||
|
const res = await fetch(`${process.env.AUTOTASK_API_URL}/Tickets/${id}`, { headers });
|
||||||
|
const data = await res.json();
|
||||||
|
const t = data.item;
|
||||||
|
if (!t) {
|
||||||
|
console.log(`Ticket ${id}: NOT FOUND IN AUTOTASK`);
|
||||||
|
continue;
|
||||||
|
}
|
||||||
|
console.log(`Ticket ${id} (${t.ticketNumber}):`);
|
||||||
|
for (const f of ['title','purchaseOrderNumber','ticketNumber','changeInfoField1','changeInfoField2','changeInfoField3','changeInfoField4','changeInfoField5']) {
|
||||||
|
const v = t[f];
|
||||||
|
const len = typeof v === 'string' ? v.length : 0;
|
||||||
|
if (len > 0) console.log(` ${f.padEnd(24)} len=${len}${len > (f === 'purchaseOrderNumber' || f === 'ticketNumber' ? 100 : 255) ? ' << OVERFLOW' : ''}`);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
main().catch(console.error);
|
||||||
192
scripts/diagnose-ticket-varchar-overflow.ts
Normal file
192
scripts/diagnose-ticket-varchar-overflow.ts
Normal file
|
|
@ -0,0 +1,192 @@
|
||||||
|
/**
|
||||||
|
* diagnose-ticket-varchar-overflow.ts
|
||||||
|
*
|
||||||
|
* One-shot: identify Autotask Tickets whose string fields exceed the
|
||||||
|
* varchar() limits declared on the local postgres `tickets` table.
|
||||||
|
*
|
||||||
|
* Why: every full tickets sync since 2026-05-21 16:14 fails with
|
||||||
|
* `value too long for type character varying(255)`
|
||||||
|
* during bulkUpsert. The pg error doesn't name the column, so we replicate
|
||||||
|
* the real sync's Autotask query EXACTLY (createDate filter, no IncludeFields)
|
||||||
|
* and walk the response checking every string field on every record.
|
||||||
|
*
|
||||||
|
* Stops after first 10 violations found.
|
||||||
|
*
|
||||||
|
* Usage:
|
||||||
|
* npx tsx scripts/diagnose-ticket-varchar-overflow.ts
|
||||||
|
*/
|
||||||
|
|
||||||
|
import { config } from 'dotenv';
|
||||||
|
import { resolve } from 'path';
|
||||||
|
|
||||||
|
config({ path: resolve(__dirname, '../.env.local') });
|
||||||
|
|
||||||
|
const API_BASE = process.env.AUTOTASK_API_URL!;
|
||||||
|
const USERNAME = process.env.AUTOTASK_USERNAME!;
|
||||||
|
const SECRET = process.env.AUTOTASK_SECRET!;
|
||||||
|
const INT_CODE = process.env.AUTOTASK_API_INTEGRATION_CODE!;
|
||||||
|
|
||||||
|
if (!API_BASE || !USERNAME || !SECRET || !INT_CODE) {
|
||||||
|
console.error('Missing Autotask credentials in env (.env.local)');
|
||||||
|
process.exit(1);
|
||||||
|
}
|
||||||
|
|
||||||
|
// Postgres tickets table varchar column limits (keep in sync with schema).
|
||||||
|
// Autotask camelCase field name -> postgres column limit.
|
||||||
|
const FIELD_LIMITS: Record<string, number> = {
|
||||||
|
title: 255,
|
||||||
|
purchaseOrderNumber: 100,
|
||||||
|
ticketNumber: 100,
|
||||||
|
changeInfoField1: 255,
|
||||||
|
changeInfoField2: 255,
|
||||||
|
changeInfoField3: 255,
|
||||||
|
changeInfoField4: 255,
|
||||||
|
changeInfoField5: 255,
|
||||||
|
};
|
||||||
|
|
||||||
|
function authHeaders(): Record<string, string> {
|
||||||
|
return {
|
||||||
|
Username: USERNAME,
|
||||||
|
Secret: SECRET,
|
||||||
|
APIIntegrationcode: INT_CODE,
|
||||||
|
'Content-Type': 'application/json',
|
||||||
|
Accept: 'application/json',
|
||||||
|
};
|
||||||
|
}
|
||||||
|
|
||||||
|
interface Violation {
|
||||||
|
ticketId: number;
|
||||||
|
ticketNumber?: string;
|
||||||
|
field: string;
|
||||||
|
length: number;
|
||||||
|
limit: number;
|
||||||
|
preview: string;
|
||||||
|
createDate?: string;
|
||||||
|
}
|
||||||
|
|
||||||
|
const MAX_VIOLATIONS = 10;
|
||||||
|
|
||||||
|
async function main() {
|
||||||
|
const since = new Date();
|
||||||
|
since.setUTCFullYear(since.getUTCFullYear() - 2);
|
||||||
|
const sinceIso = since.toISOString();
|
||||||
|
|
||||||
|
console.log(`Scanning Tickets with createDate >= ${sinceIso} (matches real sync filter exactly)`);
|
||||||
|
console.log(`Field limits: ${JSON.stringify(FIELD_LIMITS)}`);
|
||||||
|
console.log(`Will also report ANY string field >255 chars even if not in known limit list.`);
|
||||||
|
console.log('');
|
||||||
|
|
||||||
|
// Real sync sends ONLY {MaxRecords, filter} — no IncludeFields. Match exactly.
|
||||||
|
const baseBody = {
|
||||||
|
MaxRecords: 500,
|
||||||
|
filter: [{ field: 'createDate', op: 'gte', value: sinceIso }],
|
||||||
|
};
|
||||||
|
|
||||||
|
const violations: Violation[] = [];
|
||||||
|
const unknownFieldOverflows: Violation[] = [];
|
||||||
|
let totalScanned = 0;
|
||||||
|
let pageIdx = 0;
|
||||||
|
|
||||||
|
let url: string | null = `${API_BASE}/Tickets/query`;
|
||||||
|
while (url) {
|
||||||
|
pageIdx += 1;
|
||||||
|
const res: Response = await fetch(url, {
|
||||||
|
method: 'POST',
|
||||||
|
headers: authHeaders(),
|
||||||
|
body: JSON.stringify(baseBody),
|
||||||
|
});
|
||||||
|
if (!res.ok) {
|
||||||
|
const t = await res.text();
|
||||||
|
throw new Error(`Autotask query failed (page ${pageIdx}): ${res.status} ${t.slice(0, 500)}`);
|
||||||
|
}
|
||||||
|
const data = await res.json();
|
||||||
|
const items: any[] = data.items || [];
|
||||||
|
totalScanned += items.length;
|
||||||
|
|
||||||
|
for (const rec of items) {
|
||||||
|
for (const [field, v] of Object.entries(rec)) {
|
||||||
|
if (typeof v !== 'string') continue;
|
||||||
|
const limit = FIELD_LIMITS[field];
|
||||||
|
if (limit !== undefined && v.length > limit) {
|
||||||
|
violations.push({
|
||||||
|
ticketId: rec.id,
|
||||||
|
ticketNumber: rec.ticketNumber,
|
||||||
|
field,
|
||||||
|
length: v.length,
|
||||||
|
limit,
|
||||||
|
preview: v.slice(0, 100) + (v.length > 100 ? '…' : ''),
|
||||||
|
createDate: rec.createDate,
|
||||||
|
});
|
||||||
|
} else if (limit === undefined && v.length > 255) {
|
||||||
|
// Any other string field >255 (could explain why we missed it earlier).
|
||||||
|
unknownFieldOverflows.push({
|
||||||
|
ticketId: rec.id,
|
||||||
|
ticketNumber: rec.ticketNumber,
|
||||||
|
field,
|
||||||
|
length: v.length,
|
||||||
|
limit: -1,
|
||||||
|
preview: v.slice(0, 100) + '…',
|
||||||
|
createDate: rec.createDate,
|
||||||
|
});
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
process.stdout.write(
|
||||||
|
` page ${pageIdx}: scanned ${items.length} (total ${totalScanned}, hits known=${violations.length} other>255=${unknownFieldOverflows.length})\n`
|
||||||
|
);
|
||||||
|
|
||||||
|
if (violations.length >= MAX_VIOLATIONS) {
|
||||||
|
console.log(`\nReached ${MAX_VIOLATIONS} known-field violations, stopping early.`);
|
||||||
|
break;
|
||||||
|
}
|
||||||
|
|
||||||
|
url = data.pageDetails?.nextPageUrl || null;
|
||||||
|
}
|
||||||
|
|
||||||
|
console.log('');
|
||||||
|
console.log(`Scan ended. Pages: ${pageIdx}. Tickets scanned: ${totalScanned}.`);
|
||||||
|
console.log(`Violations on KNOWN columns: ${violations.length}`);
|
||||||
|
console.log(`Other string fields >255 chars: ${unknownFieldOverflows.length}`);
|
||||||
|
console.log('');
|
||||||
|
|
||||||
|
if (violations.length === 0 && unknownFieldOverflows.length === 0) {
|
||||||
|
console.log('NO violations found. The offending record may have been modified since the failed sync.');
|
||||||
|
process.exit(0);
|
||||||
|
}
|
||||||
|
|
||||||
|
if (violations.length > 0) {
|
||||||
|
const byField: Record<string, { count: number; maxLen: number }> = {};
|
||||||
|
for (const v of violations) {
|
||||||
|
const b = byField[v.field] || { count: 0, maxLen: 0 };
|
||||||
|
b.count += 1;
|
||||||
|
b.maxLen = Math.max(b.maxLen, v.length);
|
||||||
|
byField[v.field] = b;
|
||||||
|
}
|
||||||
|
console.log('Known-column violations by field:');
|
||||||
|
for (const [field, { count, maxLen }] of Object.entries(byField)) {
|
||||||
|
console.log(` ${field.padEnd(24)} count=${count} maxLen=${maxLen} limit=${FIELD_LIMITS[field]}`);
|
||||||
|
}
|
||||||
|
console.log('');
|
||||||
|
console.log('First 10 violations:');
|
||||||
|
for (const v of violations.slice(0, 10)) {
|
||||||
|
console.log(
|
||||||
|
` ticket ${v.ticketId} (${v.ticketNumber ?? '?'}) field=${v.field} length=${v.length} createDate=${v.createDate}`
|
||||||
|
);
|
||||||
|
console.log(` preview: ${v.preview.replace(/\s+/g, ' ')}`);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
if (unknownFieldOverflows.length > 0) {
|
||||||
|
console.log('');
|
||||||
|
console.log('Other (unmapped) string fields >255 chars — these would NOT cause the postgres error but worth noting:');
|
||||||
|
const byOther: Record<string, number> = {};
|
||||||
|
for (const v of unknownFieldOverflows) byOther[v.field] = (byOther[v.field] || 0) + 1;
|
||||||
|
for (const [f, c] of Object.entries(byOther)) console.log(` ${f.padEnd(28)} count=${c}`);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
main().catch((err) => {
|
||||||
|
console.error('FATAL:', err);
|
||||||
|
process.exit(1);
|
||||||
|
});
|
||||||
22
scripts/list-rmm-sites.ts
Normal file
22
scripts/list-rmm-sites.ts
Normal file
|
|
@ -0,0 +1,22 @@
|
||||||
|
import { config } from 'dotenv';
|
||||||
|
import { resolve } from 'path';
|
||||||
|
config({ path: resolve('/opt/stacks/pulse/.env.local') });
|
||||||
|
|
||||||
|
import { getDattoRMMClient } from '../lib/services/datto-rmm-factory';
|
||||||
|
|
||||||
|
async function main() {
|
||||||
|
const client = getDattoRMMClient();
|
||||||
|
const sites = await client.getSites();
|
||||||
|
|
||||||
|
console.log(`Found ${sites.length} sites\n`);
|
||||||
|
console.log('NAME\tUID\tDEVICES\tON-DEMAND');
|
||||||
|
for (const s of sites.sort((a, b) => a.name.localeCompare(b.name))) {
|
||||||
|
const devs = s.devicesStatus?.numberOfDevices ?? '';
|
||||||
|
console.log(`${s.name}\t${s.uid}\t${devs}\t${s.onDemand}`);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
main().catch((err) => {
|
||||||
|
console.error(err);
|
||||||
|
process.exit(1);
|
||||||
|
});
|
||||||
26
scripts/peek-ticket.ts
Normal file
26
scripts/peek-ticket.ts
Normal file
|
|
@ -0,0 +1,26 @@
|
||||||
|
import { config } from 'dotenv';
|
||||||
|
import { resolve } from 'path';
|
||||||
|
config({ path: resolve('/opt/stacks/pulse/.env.local') });
|
||||||
|
|
||||||
|
async function main() {
|
||||||
|
const res = await fetch(`${process.env.AUTOTASK_API_URL}/Tickets/query`, {
|
||||||
|
method: 'POST',
|
||||||
|
headers: {
|
||||||
|
Username: process.env.AUTOTASK_USERNAME!,
|
||||||
|
Secret: process.env.AUTOTASK_SECRET!,
|
||||||
|
APIIntegrationcode: process.env.AUTOTASK_API_INTEGRATION_CODE!,
|
||||||
|
'Content-Type': 'application/json',
|
||||||
|
Accept: 'application/json',
|
||||||
|
},
|
||||||
|
body: JSON.stringify({ MaxRecords: 1, filter: [{ field: 'id', op: 'gt', value: 0 }] }),
|
||||||
|
});
|
||||||
|
const data = await res.json();
|
||||||
|
const rec = data.items[0];
|
||||||
|
console.log('All string fields:');
|
||||||
|
for (const [k, v] of Object.entries(rec).sort()) {
|
||||||
|
if (typeof v === 'string') console.log(` ${k.padEnd(45)} len=${(v as string).length}`);
|
||||||
|
}
|
||||||
|
console.log('\nAll field names (for reference):');
|
||||||
|
console.log(Object.keys(rec).sort().join(', '));
|
||||||
|
}
|
||||||
|
main().catch(console.error);
|
||||||
105
scripts/reanalyze-seubert-burst.ts
Normal file
105
scripts/reanalyze-seubert-burst.ts
Normal file
|
|
@ -0,0 +1,105 @@
|
||||||
|
/**
|
||||||
|
* reanalyze-seubert-burst.ts
|
||||||
|
*
|
||||||
|
* One-shot: (1) reclassify ticket 699415's campaign now that the container
|
||||||
|
* has been rebuilt with the Phase 23 USER_AWARENESS classifier (the existing
|
||||||
|
* classification ran against the pre-Phase-23 code and is stale), and
|
||||||
|
* (2) run /analyze's parse+group steps for the 7 Seubert reports (699419,
|
||||||
|
* 699421, 699422, 699433, 699435, 699441, 699445) that landed before the
|
||||||
|
* company's automation gate was switched on and never got EML-parsed.
|
||||||
|
*
|
||||||
|
* Usage:
|
||||||
|
* POSTGRES_HOST=localhost npx tsx scripts/reanalyze-seubert-burst.ts
|
||||||
|
*/
|
||||||
|
|
||||||
|
import { config } from 'dotenv';
|
||||||
|
import { resolve } from 'path';
|
||||||
|
config({ path: resolve('/opt/stacks/pulse/.env.local') });
|
||||||
|
|
||||||
|
import postgresClient from '../lib/services/postgres-client';
|
||||||
|
import { detectPhishingTicket, type DetectableTicket } from '../lib/services/phishing-detector';
|
||||||
|
import { parseAndStoreMessage } from '../lib/services/phishing-eml-service';
|
||||||
|
import { groupReportIntoCampaign } from '../lib/services/campaign-grouping-service';
|
||||||
|
import { classifyCampaign } from '../lib/services/campaign-classifier';
|
||||||
|
|
||||||
|
const RECLASSIFY_CAMPAIGN_ID = '60a3613c-5b6c-45b1-8441-411a3fbf100c'; // ticket 699415
|
||||||
|
const UNPARSED_TICKET_IDS = [699419, 699421, 699422, 699433, 699435, 699441, 699445];
|
||||||
|
|
||||||
|
async function reanalyzeTicket(ticketId: number) {
|
||||||
|
const row = await postgresClient.query<{
|
||||||
|
id: string;
|
||||||
|
ticket_number: string | null;
|
||||||
|
title: string | null;
|
||||||
|
description: string | null;
|
||||||
|
company_id: number | null;
|
||||||
|
contact_id: number | null;
|
||||||
|
created_by_contact_id: number | null;
|
||||||
|
}>(
|
||||||
|
`SELECT id, ticket_number, title, description, company_id, contact_id, created_by_contact_id
|
||||||
|
FROM tickets WHERE id = $1`,
|
||||||
|
[ticketId]
|
||||||
|
);
|
||||||
|
const r = row.rows[0];
|
||||||
|
if (!r) {
|
||||||
|
console.log(` ticket ${ticketId}: NOT FOUND in Postgres`);
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
|
||||||
|
const ticket: DetectableTicket = {
|
||||||
|
id: Number(r.id),
|
||||||
|
ticket_number: r.ticket_number,
|
||||||
|
title: r.title,
|
||||||
|
description: r.description,
|
||||||
|
company_id: r.company_id,
|
||||||
|
contact_id: r.contact_id,
|
||||||
|
created_by_contact_id: r.created_by_contact_id,
|
||||||
|
};
|
||||||
|
|
||||||
|
const detection = await detectPhishingTicket(ticket);
|
||||||
|
if (!detection.flagged || !detection.reportId) {
|
||||||
|
console.log(` ticket ${ticketId}: detector did not flag it (unexpected — it already has a reports row)`);
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
|
||||||
|
const parseResult = await parseAndStoreMessage({ reportId: detection.reportId, ticketId });
|
||||||
|
const grouped = await groupReportIntoCampaign(detection.reportId);
|
||||||
|
|
||||||
|
console.log(
|
||||||
|
` ticket ${ticketId}: parse=${JSON.stringify(parseResult)} campaignId=${grouped?.campaignId ?? 'null'} groupMethod=${grouped?.groupMethod ?? 'null'}`
|
||||||
|
);
|
||||||
|
|
||||||
|
if (grouped?.campaignId) {
|
||||||
|
try {
|
||||||
|
const result = await classifyCampaign(grouped.campaignId);
|
||||||
|
console.log(` classified: verdict=${result.verdict}`);
|
||||||
|
} catch (err) {
|
||||||
|
console.log(` classify failed: ${err instanceof Error ? err.message : err}`);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
async function main() {
|
||||||
|
console.log('=== Step 1: Reclassify 699415 campaign (stale pre-Phase-23 verdict) ===');
|
||||||
|
const before = await postgresClient.query<{ verdict: string | null }>(
|
||||||
|
`SELECT verdict FROM classifications WHERE campaign_id = $1 ORDER BY created_at DESC LIMIT 1`,
|
||||||
|
[RECLASSIFY_CAMPAIGN_ID]
|
||||||
|
);
|
||||||
|
console.log(` verdict before: ${before.rows[0]?.verdict ?? 'none'}`);
|
||||||
|
|
||||||
|
const result = await classifyCampaign(RECLASSIFY_CAMPAIGN_ID);
|
||||||
|
console.log(` verdict after: ${result.verdict}`);
|
||||||
|
console.log(` reasons: ${JSON.stringify(result.reasons ?? [], null, 2)}`);
|
||||||
|
|
||||||
|
console.log('\n=== Step 2: Analyze the 7 unparsed Seubert reports ===');
|
||||||
|
for (const ticketId of UNPARSED_TICKET_IDS) {
|
||||||
|
await reanalyzeTicket(ticketId);
|
||||||
|
}
|
||||||
|
|
||||||
|
console.log('\nDone.');
|
||||||
|
process.exit(0);
|
||||||
|
}
|
||||||
|
|
||||||
|
main().catch((err) => {
|
||||||
|
console.error(err);
|
||||||
|
process.exit(1);
|
||||||
|
});
|
||||||
12
scripts/reclassify-699456.ts
Normal file
12
scripts/reclassify-699456.ts
Normal file
|
|
@ -0,0 +1,12 @@
|
||||||
|
import { config } from 'dotenv';
|
||||||
|
import { resolve } from 'path';
|
||||||
|
config({ path: resolve('/opt/stacks/pulse/.env.local') });
|
||||||
|
|
||||||
|
import { classifyCampaign } from '../lib/services/campaign-classifier';
|
||||||
|
|
||||||
|
async function main() {
|
||||||
|
const result = await classifyCampaign('5cdb31a3-64c9-4237-8edc-7e317f4f614c');
|
||||||
|
console.log(JSON.stringify(result, null, 2));
|
||||||
|
process.exit(0);
|
||||||
|
}
|
||||||
|
main().catch((err) => { console.error(err); process.exit(1); });
|
||||||
95
scripts/verify-blast-radius-held-bug.ts
Normal file
95
scripts/verify-blast-radius-held-bug.ts
Normal file
|
|
@ -0,0 +1,95 @@
|
||||||
|
/**
|
||||||
|
* verify-blast-radius-held-bug.ts
|
||||||
|
*
|
||||||
|
* Ticket 699687 (Tomlinson, Seubert) shows Blast Radius: Matched 16,
|
||||||
|
* Delivered 1, Held 15, with btomlinson@seubert.com's per-recipient status
|
||||||
|
* shown as "held" — despite this being the accidental Guardian Protection
|
||||||
|
* report we already confirmed was cleanly delivered (spamScore 0, no
|
||||||
|
* detection) via Mimecast's own trace. Hypothesis: getHeldMessages() in
|
||||||
|
* lib/services/mimecast-blast-radius.ts is scoped ONLY by recipient (Mimecast's
|
||||||
|
* get-hold-message-list API has no sender/subject/date filter), so it pulls
|
||||||
|
* in EVERY message currently sitting in btomlinson's hold queue — unrelated
|
||||||
|
* to Guardian Protection — and the per-recipient merge logic lets any held
|
||||||
|
* row silently overwrite a same-recipient delivered row. This script
|
||||||
|
* independently verifies by calling the same two Mimecast calls the blast
|
||||||
|
* radius module makes and inspecting what's actually in the held queue.
|
||||||
|
*
|
||||||
|
* Usage:
|
||||||
|
* POSTGRES_HOST=localhost npx tsx scripts/verify-blast-radius-held-bug.ts
|
||||||
|
*/
|
||||||
|
|
||||||
|
import { config } from 'dotenv';
|
||||||
|
import { resolve } from 'path';
|
||||||
|
config({ path: resolve('/opt/stacks/pulse/.env.local') });
|
||||||
|
|
||||||
|
import postgresClient from '../lib/services/postgres-client';
|
||||||
|
import { getMimecastClientForTenant } from '../lib/services/mimecast-client';
|
||||||
|
|
||||||
|
const SEUBERT_COMPANY_ID = 29683407;
|
||||||
|
const SENDER = 'guardian_protection_do_not_reply@guardianprotection.com';
|
||||||
|
const RECIPIENT = 'btomlinson@seubert.com';
|
||||||
|
const SUBJECT = "Unexpected Activity for Tomlinson’s Home: The Front Door was left unlocked at 1:39 pm";
|
||||||
|
|
||||||
|
async function main() {
|
||||||
|
const tenantRow = await postgresClient.query<{
|
||||||
|
client_id: string;
|
||||||
|
client_secret: string;
|
||||||
|
base_url: string | null;
|
||||||
|
account_name: string | null;
|
||||||
|
}>(
|
||||||
|
`SELECT client_id, client_secret, base_url, account_name FROM mimecast_tenants WHERE company_id = $1 AND enabled = true`,
|
||||||
|
[SEUBERT_COMPANY_ID]
|
||||||
|
);
|
||||||
|
const tenant = tenantRow.rows[0];
|
||||||
|
if (!tenant) {
|
||||||
|
console.log('No enabled Mimecast tenant for Seubert — cannot verify.');
|
||||||
|
process.exit(1);
|
||||||
|
}
|
||||||
|
console.log(`Using Mimecast tenant: ${tenant.account_name}`);
|
||||||
|
|
||||||
|
const client = getMimecastClientForTenant({
|
||||||
|
client_id: tenant.client_id,
|
||||||
|
client_secret: tenant.client_secret,
|
||||||
|
base_url: tenant.base_url ?? undefined,
|
||||||
|
});
|
||||||
|
|
||||||
|
const now = new Date();
|
||||||
|
const start = new Date(now.getTime() - 24 * 60 * 60 * 1000);
|
||||||
|
const startStr = start.toISOString().replace(/\.\d{3}Z$/, '+0000');
|
||||||
|
const endStr = now.toISOString().replace(/\.\d{3}Z$/, '+0000');
|
||||||
|
|
||||||
|
console.log(`\n=== 1. searchDeliveredMessages (scoped: sender+recipient+subject+24h window) ===`);
|
||||||
|
const delivered = await client.searchDeliveredMessages({
|
||||||
|
from: SENDER,
|
||||||
|
to: RECIPIENT,
|
||||||
|
subject: SUBJECT,
|
||||||
|
start: startStr,
|
||||||
|
end: endStr,
|
||||||
|
});
|
||||||
|
console.log(`Delivered rows found: ${delivered.messages?.length ?? 0}`);
|
||||||
|
console.log(JSON.stringify(delivered.messages, null, 2));
|
||||||
|
|
||||||
|
console.log(`\n=== 2. getHeldMessages (scoped ONLY by recipient — no sender/subject/date filter possible) ===`);
|
||||||
|
const held = await client.getHeldMessages({ recipient: RECIPIENT });
|
||||||
|
console.log(`Held rows found: ${held.messages.length} (totalCount reported: ${held.totalCount})`);
|
||||||
|
console.log(
|
||||||
|
JSON.stringify(
|
||||||
|
held.messages.map((m) => ({ from: m.from, subject: m.subject, dateReceived: m.dateReceived, reason: m.reason })),
|
||||||
|
null,
|
||||||
|
2
|
||||||
|
)
|
||||||
|
);
|
||||||
|
|
||||||
|
const heldFromGuardianProtection = held.messages.filter((m) =>
|
||||||
|
(m.from ?? '').toLowerCase().includes('guardianprotection.com')
|
||||||
|
);
|
||||||
|
console.log(`\n=== 3. Of those held messages, how many are actually from Guardian Protection? ===`);
|
||||||
|
console.log(`${heldFromGuardianProtection.length} of ${held.messages.length}`);
|
||||||
|
|
||||||
|
process.exit(0);
|
||||||
|
}
|
||||||
|
|
||||||
|
main().catch((err) => {
|
||||||
|
console.error(err);
|
||||||
|
process.exit(1);
|
||||||
|
});
|
||||||
94
scripts/verify-guardian-protection-mimecast.ts
Normal file
94
scripts/verify-guardian-protection-mimecast.ts
Normal file
|
|
@ -0,0 +1,94 @@
|
||||||
|
/**
|
||||||
|
* verify-guardian-protection-mimecast.ts
|
||||||
|
*
|
||||||
|
* Ticket 699687 (Tomlinson, Seubert) — an accidental phishing report of a
|
||||||
|
* genuine Guardian Protection (alarm.com-platform) home-security notification
|
||||||
|
* ("The Front Door was left unlocked"). The forwarded/reported copy's
|
||||||
|
* primary Authentication-Results shows spf=fail/dkim=fail/dmarc=fail (a
|
||||||
|
* known forwarding artifact — see authResultsOriginal), while the pre-
|
||||||
|
* forwarding authResultsOriginal shows all-pass. Before concluding Guardian
|
||||||
|
* Protection has bad email hygiene, independently verify against Mimecast's
|
||||||
|
* own real-time record of this exact message at Seubert's tenant.
|
||||||
|
*
|
||||||
|
* Usage:
|
||||||
|
* npx tsx scripts/verify-guardian-protection-mimecast.ts
|
||||||
|
*/
|
||||||
|
|
||||||
|
import { config } from 'dotenv';
|
||||||
|
import { resolve } from 'path';
|
||||||
|
config({ path: resolve('/opt/stacks/pulse/.env.local') });
|
||||||
|
|
||||||
|
import postgresClient from '../lib/services/postgres-client';
|
||||||
|
import { getMimecastClientForTenant } from '../lib/services/mimecast-client';
|
||||||
|
|
||||||
|
const SEUBERT_COMPANY_ID = 29683407;
|
||||||
|
const SENDER = 'guardian_protection_do_not_reply@guardianprotection.com';
|
||||||
|
const RECIPIENT = 'btomlinson@seubert.com';
|
||||||
|
const MESSAGE_ID = '<d12e6a25451a7f6dddac51480c302bef@guardianprotection.com>';
|
||||||
|
|
||||||
|
async function main() {
|
||||||
|
const tenantRow = await postgresClient.query<{
|
||||||
|
client_id: string;
|
||||||
|
client_secret: string;
|
||||||
|
base_url: string | null;
|
||||||
|
account_name: string | null;
|
||||||
|
}>(
|
||||||
|
`SELECT client_id, client_secret, base_url, account_name FROM mimecast_tenants WHERE company_id = $1 AND enabled = true`,
|
||||||
|
[SEUBERT_COMPANY_ID]
|
||||||
|
);
|
||||||
|
const tenant = tenantRow.rows[0];
|
||||||
|
if (!tenant) {
|
||||||
|
console.log('No enabled Mimecast tenant for Seubert — cannot verify.');
|
||||||
|
process.exit(1);
|
||||||
|
}
|
||||||
|
console.log(`Using Mimecast tenant: ${tenant.account_name}`);
|
||||||
|
|
||||||
|
const client = getMimecastClientForTenant({
|
||||||
|
client_id: tenant.client_id,
|
||||||
|
client_secret: tenant.client_secret,
|
||||||
|
base_url: tenant.base_url ?? undefined,
|
||||||
|
});
|
||||||
|
|
||||||
|
const now = new Date();
|
||||||
|
const start = new Date(now.getTime() - 3 * 24 * 60 * 60 * 1000);
|
||||||
|
const startStr = start.toISOString().replace(/\.\d{3}Z$/, '+0000');
|
||||||
|
const endStr = now.toISOString().replace(/\.\d{3}Z$/, '+0000');
|
||||||
|
|
||||||
|
console.log(`\n=== 1. Exact sender+recipient match, Mimecast's own trace ===`);
|
||||||
|
const exact = await client.searchDeliveredMessages({
|
||||||
|
from: SENDER,
|
||||||
|
to: RECIPIENT,
|
||||||
|
start: startStr,
|
||||||
|
end: endStr,
|
||||||
|
});
|
||||||
|
console.log(JSON.stringify(exact, null, 2));
|
||||||
|
|
||||||
|
console.log(`\n=== 2. getMessageInfo by Message-ID ===`);
|
||||||
|
try {
|
||||||
|
const info = await client.getMessageInfo(MESSAGE_ID);
|
||||||
|
console.log(JSON.stringify(info, null, 2));
|
||||||
|
} catch (err) {
|
||||||
|
console.log('getMessageInfo failed:', err instanceof Error ? err.message : err);
|
||||||
|
}
|
||||||
|
|
||||||
|
console.log(`\n=== 3. Held messages for Tomlinson ===`);
|
||||||
|
const held = await client.getHeldMessages({ recipient: RECIPIENT });
|
||||||
|
console.log(JSON.stringify(held, null, 2));
|
||||||
|
|
||||||
|
console.log(`\n=== 4. Recent threat events mentioning guardianprotection.com or alarm.com ===`);
|
||||||
|
const threats = await client.getThreatEvents({ pageSize: 500 });
|
||||||
|
const relevant = threats.items.filter((t) =>
|
||||||
|
(t.actorEmail ?? '').toLowerCase().includes('guardianprotection.com') ||
|
||||||
|
(t.actorEmail ?? '').toLowerCase().includes('alarm.com')
|
||||||
|
);
|
||||||
|
console.log(`Total threat events fetched: ${threats.items.length}`);
|
||||||
|
console.log(`Events involving guardianprotection.com/alarm.com: ${relevant.length}`);
|
||||||
|
console.log(JSON.stringify(relevant, null, 2));
|
||||||
|
|
||||||
|
process.exit(0);
|
||||||
|
}
|
||||||
|
|
||||||
|
main().catch((err) => {
|
||||||
|
console.error(err);
|
||||||
|
process.exit(1);
|
||||||
|
});
|
||||||
56
scripts/verify-held-date-scope.ts
Normal file
56
scripts/verify-held-date-scope.ts
Normal file
|
|
@ -0,0 +1,56 @@
|
||||||
|
import { config } from 'dotenv';
|
||||||
|
import { resolve } from 'path';
|
||||||
|
config({ path: resolve('/opt/stacks/pulse/.env.local') });
|
||||||
|
|
||||||
|
import postgresClient from '../lib/services/postgres-client';
|
||||||
|
import { getMimecastClientForTenant } from '../lib/services/mimecast-client';
|
||||||
|
|
||||||
|
const SEUBERT_COMPANY_ID = 29683407;
|
||||||
|
const RECIPIENT = 'btomlinson@seubert.com';
|
||||||
|
|
||||||
|
async function main() {
|
||||||
|
const tenantRow = await postgresClient.query<any>(
|
||||||
|
`SELECT client_id, client_secret, base_url, account_name FROM mimecast_tenants WHERE company_id = $1 AND enabled = true`,
|
||||||
|
[SEUBERT_COMPANY_ID]
|
||||||
|
);
|
||||||
|
const tenant = tenantRow.rows[0];
|
||||||
|
const client: any = getMimecastClientForTenant({
|
||||||
|
client_id: tenant.client_id,
|
||||||
|
client_secret: tenant.client_secret,
|
||||||
|
base_url: tenant.base_url ?? undefined,
|
||||||
|
});
|
||||||
|
|
||||||
|
// Exact production window: createdAt (report creation, 2026-07-17T18:10:55Z) +/- 24h, end clamped to now.
|
||||||
|
const createdAt = new Date('2026-07-17T18:10:55.844Z');
|
||||||
|
const start = new Date(createdAt.getTime() - 24 * 60 * 60 * 1000);
|
||||||
|
const end = new Date(Math.min(createdAt.getTime() + 24 * 60 * 60 * 1000, Date.now()));
|
||||||
|
const startStr = start.toISOString().replace(/\.\d{3}Z$/, '+0000');
|
||||||
|
const endStr = end.toISOString().replace(/\.\d{3}Z$/, '+0000');
|
||||||
|
|
||||||
|
console.log(`Querying held messages for ${RECIPIENT} scoped to the EXACT production window: ${startStr}..${endStr}`);
|
||||||
|
|
||||||
|
const body = {
|
||||||
|
data: [
|
||||||
|
{
|
||||||
|
admin: true,
|
||||||
|
start: startStr,
|
||||||
|
end: endStr,
|
||||||
|
searchBy: { fieldName: 'recipient', value: RECIPIENT },
|
||||||
|
},
|
||||||
|
],
|
||||||
|
};
|
||||||
|
|
||||||
|
const result = await client.request('POST', '/api/gateway/get-hold-message-list', body);
|
||||||
|
const msgs = result?.data ?? [];
|
||||||
|
console.log(`Rows returned: ${msgs.length}`);
|
||||||
|
console.log(JSON.stringify(msgs.map((m: any) => ({
|
||||||
|
from: m.fromHeader?.emailAddress ?? m.from?.emailAddress,
|
||||||
|
subject: m.subject,
|
||||||
|
dateReceived: m.dateReceived,
|
||||||
|
reason: m.reason,
|
||||||
|
})), null, 2));
|
||||||
|
|
||||||
|
process.exit(0);
|
||||||
|
}
|
||||||
|
|
||||||
|
main().catch((err) => { console.error(err); process.exit(1); });
|
||||||
109
scripts/verify-mansfield-mimecast.ts
Normal file
109
scripts/verify-mansfield-mimecast.ts
Normal file
|
|
@ -0,0 +1,109 @@
|
||||||
|
/**
|
||||||
|
* verify-mansfield-mimecast.ts
|
||||||
|
*
|
||||||
|
* Ticket 699451 (Richard Mansfield, Seubert) was manually forwarded by a
|
||||||
|
* technician, not reported via KnowBe4's Phish Alert button or Microsoft's
|
||||||
|
* "Report Message" add-in — so the detector never created a `reports` row,
|
||||||
|
* and everything we know about the sender ("eserver@it-support.care") comes
|
||||||
|
* from a human's free-text ticket description, not a parsed EML/header.
|
||||||
|
*
|
||||||
|
* it-support.care IS on the KNOWN_SIMULATION_SENDERS allowlist
|
||||||
|
* (lib/services/campaign-classifier.ts) for KnowBe4. Before trusting the
|
||||||
|
* ticket text at face value, independently verify against Mimecast's actual
|
||||||
|
* message-trace data for Seubert's tenant: does a message from that sender
|
||||||
|
* to Rich, with that subject, in that time window, actually exist and what
|
||||||
|
* does Mimecast say about it (delivered/held/rejected, real headers)?
|
||||||
|
*
|
||||||
|
* Usage:
|
||||||
|
* POSTGRES_HOST=localhost npx tsx scripts/verify-mansfield-mimecast.ts
|
||||||
|
*/
|
||||||
|
|
||||||
|
import { config } from 'dotenv';
|
||||||
|
import { resolve } from 'path';
|
||||||
|
config({ path: resolve('/opt/stacks/pulse/.env.local') });
|
||||||
|
|
||||||
|
import postgresClient from '../lib/services/postgres-client';
|
||||||
|
import { getMimecastClientForTenant } from '../lib/services/mimecast-client';
|
||||||
|
|
||||||
|
const SEUBERT_COMPANY_ID = 29683407;
|
||||||
|
const REPORTED_SENDER = 'eserver@it-support.care';
|
||||||
|
const RECIPIENT = 'rmansfield@seubert.com';
|
||||||
|
const SUBJECT = 'Internal Email Problems';
|
||||||
|
|
||||||
|
async function main() {
|
||||||
|
const tenantRow = await postgresClient.query<{
|
||||||
|
client_id: string;
|
||||||
|
client_secret: string;
|
||||||
|
base_url: string | null;
|
||||||
|
account_name: string | null;
|
||||||
|
}>(
|
||||||
|
`SELECT client_id, client_secret, base_url, account_name FROM mimecast_tenants WHERE company_id = $1 AND enabled = true`,
|
||||||
|
[SEUBERT_COMPANY_ID]
|
||||||
|
);
|
||||||
|
const tenant = tenantRow.rows[0];
|
||||||
|
if (!tenant) {
|
||||||
|
console.log('No enabled Mimecast tenant for Seubert — cannot verify.');
|
||||||
|
process.exit(1);
|
||||||
|
}
|
||||||
|
console.log(`Using Mimecast tenant: ${tenant.account_name}`);
|
||||||
|
|
||||||
|
const client = getMimecastClientForTenant({
|
||||||
|
client_id: tenant.client_id,
|
||||||
|
client_secret: tenant.client_secret,
|
||||||
|
base_url: tenant.base_url ?? undefined,
|
||||||
|
});
|
||||||
|
|
||||||
|
const now = new Date();
|
||||||
|
const start = new Date(now.getTime() - 5 * 24 * 60 * 60 * 1000); // 5 days back
|
||||||
|
const startStr = start.toISOString().replace(/\.\d{3}Z$/, '+0000');
|
||||||
|
const endStr = now.toISOString().replace(/\.\d{3}Z$/, '+0000');
|
||||||
|
|
||||||
|
console.log(`\n=== 1. Exact sender+recipient+subject match ===`);
|
||||||
|
console.log(`from=${REPORTED_SENDER} to=${RECIPIENT} subject="${SUBJECT}" window=${startStr}..${endStr}`);
|
||||||
|
const exact = await client.searchDeliveredMessages({
|
||||||
|
from: REPORTED_SENDER,
|
||||||
|
to: RECIPIENT,
|
||||||
|
subject: SUBJECT,
|
||||||
|
start: startStr,
|
||||||
|
end: endStr,
|
||||||
|
});
|
||||||
|
console.log(JSON.stringify(exact, null, 2));
|
||||||
|
|
||||||
|
console.log(`\n=== 2. Sender domain only (it-support.care), any recipient at seubert.com ===`);
|
||||||
|
const domainOnly = await client.searchDeliveredMessages({
|
||||||
|
from: 'it-support.care',
|
||||||
|
to: 'seubert.com',
|
||||||
|
start: startStr,
|
||||||
|
end: endStr,
|
||||||
|
});
|
||||||
|
console.log(JSON.stringify(domainOnly, null, 2));
|
||||||
|
|
||||||
|
console.log(`\n=== 3. All mail TO Rich in the window (no sender filter) ===`);
|
||||||
|
const toRich = await client.searchDeliveredMessages({
|
||||||
|
to: RECIPIENT,
|
||||||
|
subject: SUBJECT,
|
||||||
|
start: startStr,
|
||||||
|
end: endStr,
|
||||||
|
});
|
||||||
|
console.log(JSON.stringify(toRich, null, 2));
|
||||||
|
|
||||||
|
console.log(`\n=== 4. Held messages for Rich ===`);
|
||||||
|
const held = await client.getHeldMessages({ recipient: RECIPIENT });
|
||||||
|
console.log(JSON.stringify(held, null, 2));
|
||||||
|
|
||||||
|
console.log(`\n=== 5. Recent threat events on this tenant (last 500) ===`);
|
||||||
|
const threats = await client.getThreatEvents({ pageSize: 500 });
|
||||||
|
const relevant = threats.items.filter(
|
||||||
|
(t) => (t.actorEmail ?? '').toLowerCase().includes('it-support.care')
|
||||||
|
);
|
||||||
|
console.log(`Total threat events fetched: ${threats.items.length}`);
|
||||||
|
console.log(`Events involving it-support.care: ${relevant.length}`);
|
||||||
|
console.log(JSON.stringify(relevant, null, 2));
|
||||||
|
|
||||||
|
process.exit(0);
|
||||||
|
}
|
||||||
|
|
||||||
|
main().catch((err) => {
|
||||||
|
console.error(err);
|
||||||
|
process.exit(1);
|
||||||
|
});
|
||||||
Loading…
Add table
Add a link
Reference in a new issue