# AI Ticket Analyzer — Build Notes A running journal of the multi-phase build for the AI Ticket Analyzer feature. Captures what was delivered each phase, design decisions worth flagging, and what was deliberately left out. Spec lives in `wulf-pulse-ticket-analyzer-prompt.md`. This file is updated after each phase ships. --- ## Phase 1 — Migration + Zod schemas **Delivered** - `migrations/069_create_analyzer_tables.sql` — three tables (`analyzer_analyses`, `analyzer_shares`, `analyzer_jobs`) with `pgcrypto` extension guard, indexes per spec, status `CHECK` constraints, and FK types corrected to `TEXT` (not `UUID`) to match Better Auth's `user.id`. - `lib/types/analyzer.ts` — Zod schemas for every LLM-stage parsed JSON (`TaggedEvent`, `TriageResponse`, `DeepAnalysisResponse`, `OpusResponse`), persisted row shapes, job status, API request bodies, and the internal `PreprocessedTicket` payload that flows through the pipeline. **Decisions worth flagging** - `triggered_by_user_id` is **nullable + `ON DELETE SET NULL`** (not `NOT NULL`). An analysis should still be readable in history if the triggering user is later deleted. `analyzer_shares.shared_by_user_id` is `NOT NULL + ON DELETE CASCADE` (audit-log style — share rows go with the user). - `evidence_timestamps` typed as `string().datetime({offset: true})` (ISO timestamps), not numeric indices. More robust to model hallucination and reads better in the UI. - `TaggedEvent` does NOT support a "two separate events" form for time entries with both Summary + Internal Notes — single event with both fields, per the spec's preference for a cleaner timeline. **Deliberately left out** - Migration is committed but **not applied** to any running DB. Postgres only re-applies migrations on first init of a fresh volume; the existing DB needs a manual run of this migration when phase 5 is exercised. --- ## Phase 2 — Stage 0 pre-processor + IT Glue redaction **Delivered** - `lib/services/analyzer/itglue-redact.ts` — recursive walk; matches keys against `/password|secret|key|token|credential|api[_-]?key/i`; replaces values with `[REDACTED]`. Subtree redaction (key matching `auth` → no leaves leak), defensive copy, cycle guard (`[CIRCULAR]` marker). - `lib/services/analyzer/preprocessor.ts` — filters workflow noise + Service Desk Notification rows, tags ticket_create/notes/time entries with `actor / actor_type / source / visibility / summary_notes / internal_notes / hours`, sorts chronologically, computes `sha256` content hash over canonical JSON of `(events, status, priority, queue)`. - `vitest@^4.1.5` added as devDep with `vitest.config.ts` setting up the `@/` alias. Two test files at this phase: 32 redaction tests + 36 preprocessor tests including the regression run against the `T20260424.0045` fixture. - Type-aliasing fix in `lib/types/analyzer.ts`: added `export type X = z.infer` for `ActorType`, `EventSource`, `Visibility`, `Severity`, `ComplexityTier`, `TicketType` — the Zod-enum const exports alone don't produce a usable TypeScript type. **Decisions worth flagging** - `actor_type` is classified by **email domain, not author text**, per the spec. `lorentz@wulfconsulting.com` is `wulf_tech` regardless of how the message reads. Sonnet handles the role nuance at Stage 3. - Time entries with no narrative content (no summary, no internal notes) are dropped — a purely numeric entry adds nothing. - Vendor-domain allowlist is intentionally short and conservative (Vertafore, Datto, Microsoft, etc.). Misclassifying a customer domain as "vendor" is worse than the default `client_contact`. **Deliberately left out** - No real protection for secrets embedded in **free text** (e.g. a notes field containing the literal string "the password is hunter2"). The redaction guarantee is on field **keys**, not values. The IT Glue search test documents this contract explicitly so it isn't "fixed" without thought. --- ## Phase 3 — Anthropic SDK setup + Stage 1 (Haiku triage) **Delivered** - `lib/services/llm/{models,pricing,client,call}.ts` — model ID constants, per-model rate table (Haiku $1/$5, Sonnet $3/$15, Opus $5/$25 per 1M tokens + cache read/write tiers), lazy SDK singleton, generic `callLLMStage({model, system, user, schema, maxTokens, client?})` helper that: - Marks the system prompt with `cache_control: {type: "ephemeral"}` - Sends NO `temperature`/`top_p`/`top_k` (Opus 4.7 would 400) - Strips a single ` ```json ` fence before parsing - On parse failure, retries **once** with prior-attempt + error in a follow-up user turn - Returns `{data, usage, estimated_cost_usd, attempts, raw_response}` - `lib/services/analyzer/stages/stage1-triage.ts` — Haiku caller with the spec's verbatim system prompt and a 50KB user-payload cap that drops oldest internal-only events first when oversized. - `lib/services/analyzer/itglue-search.ts` — search facade that runs every result through `redact()` before returning. Snippets capped at 2000 chars, doc count capped at 10. `itglue-aliases.json` skeleton for known fuzzy org-name mappings. - 32 new tests across pricing/call/Stage 1/IT Glue search. **Decisions worth flagging** - **Manual JSON.parse + Zod validate, not `output_config.format` / `client.messages.parse()`.** The spec said retry-once on Zod parse failure, and Zod 4 ↔ JSON Schema conversion has edge cases I didn't want to depend on (e.g. `.datetime({offset: true})` → JSON Schema `format`). Manual parse is what the spec asks for and is more transparent. - **Prompt caching probably won't fire on these stages.** Our system prompts are ~1-2 KB (~250-500 tokens); minimum cacheable prefix is 2048 (Sonnet) or 4096 (Haiku/Opus) tokens. The `cache_control` marker is a no-op below threshold and incurs no cost — left in defensively, but caching is not a meaningful lever for this workload. - **Redaction is on KEY names, not free text** — re-stated for emphasis. The IT Glue search test asserts this contract. - **`itglue-aliases.json` is a skeleton** with `_comment` / `_example` keys documenting the format. Real org-id entries get added when phase 5 wires this into the pipeline. **Deliberately left out** - No live API integration test. All Stage 1 tests use a mocked `Anthropic` client. A real-API smoke test belongs in phase 5+ when we run end-to-end. - Did **not** wrap `lib/services/itglue-client.ts` in a redacting search facade for non-LLM callers. The redaction primitive is ready; non-LLM callers (sync service, data browsers) intentionally have full data — they are not the path that needs protection. --- ## Phase 4 — Pipeline + Job worker (Stages 3, 4, 5) **Delivered** - `lib/services/analyzer/stages/stage3-deep-analysis.ts` — Sonnet caller, spec's verbatim prompt, 80KB payload cap. - `lib/services/analyzer/stages/stage4-deep-reasoning.ts` — Opus caller plus pure helpers `shouldRunDeepReasoning()` and `applyOpusUpdates()`. - `lib/services/analyzer/data-access.ts` — `loadTicketBundle(ticketNumber)` joins `tickets` / `statuses` / `priorities` / `queues` / `companies` / `contacts` / `resources` / `ticket_notes` / `time_entries` and returns the exact `RawTicketBundle` shape the preprocessor expects. Throws typed `TicketNotFoundError`. - `lib/services/analyzer/persistence.ts` — `getNextAnalysisVersion`, `findExistingAnalysisByContentHash`, `insertAnalysis`, plus job `claimQueuedJob` / `updateJobStatus` / `completeJob` / `failJob` / `queueJob` / `getJob`. Job claim uses `FOR UPDATE SKIP LOCKED` so multiple Next.js workers can poll safely. - `lib/services/analyzer/pipeline.ts` — composes Stage 0 → idempotency check → Stage 1 → (Stage 2 if `itglue_lookup_needed`) → Stage 3 → (Stage 4 if trigger fires AND cost ceiling not reached) → result. Cost circuit breaker trips at $2.00 before Opus, sets `needs_human_review=true` with a reason. Returns full `model_traces` for debugging. - `lib/services/analyzer/worker.ts` — singleton with 2-second poll loop; auto-starts in production; opt-in in dev via `ANALYZER_WORKER_AUTOSTART=1`; **skipped** under vitest. `runJob()` exposed for tests + manual triggers. `TicketNotFoundError` produces a user-facing job error message that fingers the sync as the culprit. - 28 new tests bringing the total to 128. **Decisions worth flagging** - **Worker auto-start is more conservative than `sync-scheduler.ts`.** That one auto-starts on any non-browser import (including tests). I gated this one because the worker hits the database AND runs LLM calls — much higher blast radius. To run locally, set `ANALYZER_WORKER_AUTOSTART=1`. - **Cost circuit breaker only fires before Opus.** Sonnet runs unconditionally even if it would push past $2. Spec wording matches; if the team wants stricter control, the natural place is next to `COST_CEILING_USD` in `pipeline.ts`. - **Idempotency check matches `status='complete'` only.** Failed runs don't poison the cache. - **IT Glue failures are tolerated.** Search throws → analysis continues without context, doesn't fail the run. - `worker.test.ts` casts `mockImplementationOnce` to `as never` because vitest's overload resolution fights us when the mocked function has multiple call signatures. Functional, just ugly. **Deliberately left out** - No live API integration test (still). Phase 5+ is the natural place for an end-to-end smoke test. - Stage 3 doesn't have a fixture-driven happy-path test like Stage 1; it's exercised at the orchestration layer via `pipeline.test.ts` only. Worth adding direct Stage 3 tests later. - Migration `069` still not applied — same as phase 1. --- ## Phase 5 — API routes **Delivered** - Persistence read paths added to `lib/services/analyzer/persistence.ts`: `getAnalysisById`, `listAnalysesByTicketNumber`, `listNeedsReview`, `createShare`, plus a shared `rowToPersistedAnalysis` row mapper. - 6 routes under `app/api/analyzer/`: | Route | Method | Returns | |---|---|---| | `tickets/[ticketNumber]/analyze` | POST | `{status, jobId?, existingAnalysisId?}` | | `tickets/[ticketNumber]/analyses` | GET | `{analyses: PersistedAnalysis[]}` | | `jobs/[jobId]` | GET | `{job}` | | `analyses/[id]` | GET | `{analysis}` | | `analyses/[id]/share` | POST | `{share}` | | `needs-review?limit=&offset=` | GET | `{analyses}` | **Decisions worth flagging** - **Analyze runs preprocess inline.** The route does `loadTicketBundle → preprocessTicket → findExistingAnalysisByContentHash` synchronously to support the spec's `existingAnalysisId?` immediate response. The worker also runs this — duplicated work, but preprocess is fast (deterministic, one DB load) and the alternative (always queue, frontend polls to discover the short-circuit) is worse UX. - **Email send deferred to phase 8** per the spec's delivery order. The share route persists the audit row and validates the recipient domain against `ALLOWED_SHARE_DOMAINS`. Until phase 8, share rows have `viewed_at = null` indefinitely. - **`requireAuth()` everywhere — not `requireAdmin()`.** Any authenticated user can analyze a ticket they have access to. If `/needs-review` should be admin-only later, swap that one to `requirePermission('analyzer', 'review')` once the permission map is decided. - **No new permission entries added to `lib/permissions.ts`.** Adding scoped permissions for a feature still under build risks getting them wrong. - Routes are NOT in `middleware.ts` `publicRoutes` — they require a session. **Deliberately left out** - No API route tests. The repo has zero `app/api/**/*.test.ts` files; the routes are thin orchestration on top of already-tested persistence. Adding integration tests means setting up a test harness for the auth helpers + Postgres, which is a separate effort. --- ## Phase 6 — Frontend pages **Delivered** - `components/analyzer/analyze-button.tsx` — `` with the full state machine: POSTs to the analyze endpoint, navigates straight to an existing analysis if `existingAnalysisId` came back, otherwise polls `/api/analyzer/jobs/:jobId` every 2s and renders stage labels (Queued → Fetching → Triaging → Searching IT Glue → Analyzing → Deep review → Done). 5-minute hard timeout. Failures surface as toast errors. Supports `force` for explicit re-run. - `components/analyzer/share-modal.tsx` — `` with a shadcn Dialog + email field + optional note (max 2000 chars). POSTs to the share endpoint and surfaces server-side validation errors (domain not allowed → toast). - `components/analyzer/analysis-view.tsx` — the full 10-section analysis layout per spec. Header with model-tier badges (Haiku/Sonnet/Opus pills), confidence score, total cost, Share + Re-analyze buttons. Summary, Next Step (with rationale collapsed), Timeline (vertical list with 🟢 / 🔒 / 🔄 markers, click to expand), What Was Done / Should Have Been Done side-by-side on wide screens, Gaps colored by severity with "Evidence:" links that scroll-and-expand the matching timeline event, Post-Resolution (only if present), Human Review Flags (only if needed), IT Glue References. - `app/analyzer/ticket/[ticketNumber]/page.tsx` — ticket detail with the Analyze button and a list of historical versions; latest is badged. - `app/analyzer/analysis/[id]/page.tsx` — fetches one analysis and renders it via ``. - `app/analyzer/queue/page.tsx` — needs-review queue, shows ticket number, version, summary preview, top reasons, confidence badge, and a `high gap` badge if any gap is high severity. **Decisions worth flagging** - **Imperative `useState` + `useEffect` + `fetch`**, no SWR / react-query — matches CLAUDE.md and the rest of the repo. Don't refactor to a global cache layer for these three pages; if it becomes a real pain, that's a whole-app concern. - **`'use client'` everywhere.** The pages use `use(params)` (the React hook) to unwrap Next.js 16's `params: Promise<...>` shape on the client. Server components weren't appropriate here — every page does interactive state (analyze flow, expand events, share modal). - **Stage labels render directly from `JobStatus` enum values**, not a separate label list, so any new statuses added to the enum auto-render with their default name. - **Re-analyze banner not implemented yet.** The spec calls for "New activity since last analysis · Re-analyze" when the live content_hash drifts from the persisted one. That requires a live-preprocess endpoint (or running preprocess on the page render). Skipped for now — the user can always click Re-analyze. Worth adding once we have real-world signal on whether activity drift is common. - **No autocomplete on the share-recipient field.** Spec says "autocomplete from existing wulf-pulse user list if available". Skipped — the existing user list is in Better Auth's `user` table; exposing it requires a small API endpoint. Easy follow-up. - **No navigation entry yet.** `components/navigation/app-navigation.tsx` doesn't yet have an "Analyzer" link. Adding that is a one-line edit; I left it for the operator to opt in once the feature is staged. - **Printable analysis view** — the spec mentions "printable" for the analysis page. The current layout is print-friendly by accident (no fixed sidebars, sectioned cards), but no explicit `@media print` styles yet. Add when someone asks. **Deliberately left out** - No frontend tests. vitest is configured for `lib/**/*.test.ts` only; the pages and components are visually verified. Component tests with testing-library would be a separate setup decision. - Analysis view doesn't auto-refresh while a job is running on a *different* version. If a user navigates to an old version while a new one is in progress, they don't see the in-progress state. Acceptable — the queue view + ticket history give that signal. --- ## Phase 7 — Share-via-email integration **Delivered** - `sendAnalysisShareEmail()` added to `lib/services/email.ts` — re-uses the existing nodemailer SMTP transport that already serves magic-link and invitation mail. Subject `Pulse analysis · v — `, gradient-header HTML body matching the other Pulse emails, plain text fallback, `replyTo` set to the sharer so a recipient reply lands with the right person. - Share route `app/api/analyzer/analyses/[id]/share/route.ts` now resolves the sharer's session, builds the analysis URL from `BETTER_AUTH_URL` (falling back to `NEXT_PUBLIC_BETTER_AUTH_URL`, then `localhost:3100`), persists the audit row, then attempts the email send. - Share-modal frontend handles the new `emailSent`/`emailError` fields: success → green toast; row-saved-but-send-failed → orange `toast.warning` carrying the SMTP error. **Decisions worth flagging** - **Audit row persists even if email send fails.** The `analyzer_shares` row is the audit log, not just a delivery receipt. SMTP outages should not erase the record that the user attempted a share. - **Response is HTTP 200 on email failure.** The route returns `{share, emailSent: false, emailError}`. A non-2xx would imply the share itself failed; surfacing `emailSent: false` is the more honest signal. - **No `email_sent_at` column added.** Adding it requires a migration and a way to retry — neither is asked for by the spec. Send state lives only in the response and the server log line `[ANALYZER-SHARE] email send failed for share `. If retries become a real need, a dedicated `analyzer_share_email_attempts` table is the natural shape. - **Used existing nodemailer SMTP, not Graph sendMail.** Spec said "via M365 Graph using existing wulf-pulse mail integration if one exists; otherwise use a new module" — `email.ts` *is* the existing module. The Graph integration in this codebase is read-only (mailbox search, user reports), not send-capable. - **HTML escaping is hand-rolled.** Five-replace function for amp/lt/gt/quote/apos. The user-controlled fields (sender name, note, analysis summary, next step) all flow through it. The repo has no HTML escaper utility and the magic-link/invitation emails don't need one because their inputs are URLs and admin-set names. **Deliberately left out** - **No retries.** A failed send is logged once and surfaced to the user. They can re-share if they want — that creates a fresh audit row, which is correct behavior. - **No tests.** `lib/services/email.ts` has no existing tests, mocking nodemailer fully would add a non-trivial test scaffold for one function, and the share route is route-handler thin. Consistent with phases 5/6. - **No `viewed_at` tracking yet.** The migration has the column but nothing writes to it. A `/share/:id/viewed` endpoint with an unguessable token would be the smallest addition; spec didn't ask, so skipped. --- ## Phase 8 — Operator runbook **Delivered** - `docs/wulf-pulse-ticket-analyzer-runbook.md` — cost monitoring queries (daily spend, top expensive analyses, multi-run tickets), the cost ceiling explanation (`COST_CEILING_USD = 2.00` in `pipeline.ts`), the IT Glue alias workflow (where to find org IDs, how to verify via `model_traces.itglue.alias_used`), failure triage tables (failed jobs vs. `needs_human_review` flagged completes, with the actual error strings the worker emits), and a manual-ops section for queueing jobs from psql. - README.md "Documentation" section links the runbook so operators finding the project for the first time discover it. **Decisions worth flagging** - **Runbook lives in `docs/`, not the README.** Per CLAUDE.md, long-form per-feature docs go in `docs/` and the README points to them. Matches the existing pattern (workflow-editor-guide, sync guides, etc.). - **Manual migration step is documented.** Postgres only re-applies migrations on first init — operators applying analyzer to an existing DB need to run `psql -f migrations/069_*.sql`. The runbook leads with this. - **Listed what's *not* implemented.** Retries, `viewed_at`, share `email_sent_at`, pre-Sonnet cost gate, share-recipient autocomplete. Better to advertise the gaps than have an operator trip over them. **Deliberately left out** - No production-monitoring dashboard (Grafana etc.) — Pulse doesn't have one for any other feature, and the SQL queries cover the same ground. - No README expansion for the analyzer feature itself. Pulse's README is intentionally short; the runbook + spec + build notes are the deep docs. --- ## Phase 9 — Ticket browser + analysis-view formatting Reactive to first-use feedback: the analysis page Summary / Next Step text was bare and dense, and there was no way to discover tickets to analyze without typing the URL. **Delivered** - `` helper inside `analysis-view.tsx` — splits text on blank lines, renders each chunk as a separate `

` with `leading-7` and `whitespace-pre-line`. Applied to Summary, Next Step, Next Step rationale, and Post-Resolution Analysis. Single-paragraph text still renders cleanly. - Summary + Post-Resolution headers got an uppercase tracking-wide treatment to act as section dividers, and body text bumped to `text-base text-foreground` so it reads as a finding rather than a caption. - Next Step card now has a subtle `bg-primary/5` tint, an `ArrowRight` icon next to "Recommended Next Step", a stronger separator before the rationale collapsible, and the rationale itself renders in a bordered indented block. - New `/analyzer/tickets` browse page — pill-style period chips (Today, Yesterday, This week, Last week, Last 30/60 days, All time), a client (company) Select, an issue-type Select, and a debounced free-text search across ticket_number/title. Compact table with per-row Analyze/Re-analyze button (reusing ``) and a "View" button shortcut to the existing analysis when one is recorded. Active-filter count + clear-all in the filter card header. - New API `GET /api/analyzer/tickets/list` — filters by `period` (computed in Postgres against `last_activity_date`), `companyId`, `issueType`, `search`. Returns 50 rows + total via `COUNT(*) OVER ()`, plus `latestAnalysisId` from a LATERAL join into `analyzer_analyses`. - New API `GET /api/analyzer/tickets/filter-options` — companies that have at least one non-deleted ticket (drops dormant accounts) + active issue types ordered by `sort_order, label`. - Top-level "Analyzer" nav menu added to `app-navigation.tsx`, with "Browse Tickets" + "Needs Review". Earlier phases left this off intentionally; this phase opts in. **Decisions worth flagging** - **Period filters on `last_activity_date`, not `create_date`.** "Today" surfaces tickets that had activity today (new tickets, re-opened, status churn) — much more useful for an analyzer-driven triage flow than tickets created today. A new ticket created today also has activity today, so we don't lose those. - **Period math runs in Postgres via `date_trunc('day', NOW())` etc.** Database server-clock = app-process clock for an internal Docker stack, so naive timestamps and naive `NOW()` agree. If users complain about edge-of-day drift, swap to `NOW() AT TIME ZONE 'America/New_York'` — Pulse's primary user base. - **Default period is `last_30d`.** "All time" pulls many thousands of rows; defaulting wide-open hurts first-page latency. 30 days hits ~7K rows in our DB, paginates cleanly. - **Per-row analyze button reuses `` directly.** Each row gets its own component instance — no shared state, the running state lives per-button. The button navigates on completion, which feels right: click Analyze, watch the stages, land on the analysis page. - **`force=true` is set automatically on tickets that already have an analysis.** Re-analyze should re-run, not short-circuit to the cached row. The "View" button covers the cached path. - **No Linear/JIRA-style multi-select filters.** Single-value Selects are simpler and match the rest of Pulse. **Deliberately left out** - **No saved views.** A filter URL is shareable, but there's no bookmark / saved-view UX. Add when someone asks. - **No `latest_analysis_status` exposure.** A failed analysis doesn't show up — the LATERAL join filters by `status='complete'`. So a ticket whose only analysis failed looks like an un-analyzed ticket. Acceptable: re-running is the intended action there anyway. - **Search is `ILIKE '%...%'`.** No tsvector / trigram index. 7K-row scans are sub-100ms in this DB; if the corpus grows past low six digits, swap in `pg_trgm`. --- ## Status after each phase | Phase | Tests | tsc | Notes | |---|---|---|---| | 1 | 0 | clean | migration + types only | | 2 | 68 | clean | redaction + preprocessor regression | | 3 | 100 | clean | + LLM scaffolding + Stage 1 | | 4 | 128 | clean | + pipeline + worker | | 5 | 128 | clean | API routes (no route tests) | | 6 | 128 | clean | frontend (no FE tests) | | 7 | 128 | clean | share email via existing SMTP transport | | 8 | 128 | clean | operator runbook + README link | | 9 | 128 | clean | browse page + analysis-view formatting + nav entry | --- # Phase 2 (cross-ticket analysis) Spec: `docs/ticket-analyzer-phase2-spec.md`. Eight sub-phases delivered as one Phase 2 push. ## 2.1 — Schema additions **Delivered** - Migration 070: `analyzer_stage_executions` table (per-stage I/O for every analyzer run, including failed attempts) + three columns on `analyzer_analyses`: `source_snapshot`, `aggregate_fingerprint`, `fingerprint_generated_at`. - `model_traces` column annotated with a `LEGACY` `COMMENT ON COLUMN` for the SQL side and a `// LEGACY` doc comment in TS — kept for back-compat until aggregate reports have soaked. - Zod schemas: `AggregateFingerprint`, `StageName`, `StageExecution` (read-back), `StageExecutionRecord` (write-time interface). **Decisions worth flagging** - `analyzer_stage_executions.analysis_id` starts NOT NULL in 070. Migration 071 (Phase 2.6) relaxes it and adds the mutually-exclusive CHECK with `aggregate_report_id`. - 070 is idempotent (`IF NOT EXISTS` on every object) so re-running against an already-applied DB is safe. ## 2.2 — Pipeline writes to stage executions **Delivered** - `recordedStage(meta, fn, callbacks, outputSelector)` helper in `pipeline.ts` wraps each stage call, emits a `StageExecutionRecord` on success or failure (re-throws after recording). Pipeline wires it into Stage 1 (triage), Stage 3 (analyze), Stage 4 (deep_review). Stage 0 (preprocess) and Stage 2 (itglue) are recorded inline since they're not LLM calls. - Pipeline result now includes `triage_response`, `sonnet_response`, `opus_response` for downstream stages (Stage 6 fingerprint). - Worker's `runJob` collects records via `onStageRecord` callback, bulk-inserts them after `insertAnalysis` succeeds. On pipeline throw, worker captures the preprocessed bundle via `onPreprocessed` callback, persists a `status='failed'` analyzer_analyses row with source_snapshot intact, and bulk-inserts the partial stage records linked to it. - New persistence functions: `bulkInsertStageExecutions`, `insertFailedAnalysis`, `updateAnalysisFingerprint`. **Decisions worth flagging** - **Failure-tolerant audit**: spec says "Every stage that runs MUST insert a row, including stages that fail." We persist a failed analyzer_analyses row even on pipeline crash so the partial stage records have a parent. Without this the FK would be orphaned. - **Single bulk insert**: ~5–6 stage rows per pipeline run. One multi-VALUES INSERT is fast enough; no need for COPY. - **`model_traces` double-write retained**: the legacy column still receives the old payload. Drop it in a future migration once aggregate reports have soaked through prod. ## 2.3 — Prose formatting **Delivered** - Stage 3 system prompt updated with the markdown formatting rules from the spec verbatim (banned filler phrases, action-verb emphasis, no headers). - `react-markdown@10`, `remark-gfm@4`, `@tailwindcss/typography@0.5` added. Tailwind 4 plugin registered via `@plugin "@tailwindcss/typography"` in `app/globals.css`. - `` component at `components/analyzer/analysis-markdown.tsx` renders prose with `prose prose-sm dark:prose-invert max-w-none prose-p:leading-7`. Coerces stray model headers into bold paragraphs (the prompt forbids them but defense-in-depth). - `` removed. Summary, Recommended Next Step, next_step_rationale, and post_resolution_analysis all use ``. **Decisions worth flagging** - Tailwind 4 syntax: `@plugin "@tailwindcss/typography"` in CSS, no JS config needed. - Stage 3 prompt change is back-compatible — old analyses with plain-text summaries still render fine through ReactMarkdown. ## 2.4 — Stage 6 fingerprint + backfill CLI **Delivered** - `lib/services/analyzer/stages/stage6-fingerprint.ts` — Haiku call with the spec's verbatim system prompt. Server-overrides `generated_by_model` and `generated_at` after parse so the model's guess for those fields can't drift. - Worker integration: after `insertAnalysis` succeeds, run fingerprint with try/catch. Failure logs a warn, fingerprint stays NULL on the row, but the analysis is still complete and usable. The fingerprint stage record is added to the bulk insert whether it succeeded or failed. - `scripts/backfill-fingerprints.ts` — idempotent CLI. Reads `analyzer_analyses.model_traces.{triage_response, sonnet_response, opus_response}` (which Phase 1 was already storing), runs Stage 6, writes `aggregate_fingerprint`. Supports `--dry-run` and `--limit=N`. Skips analyses where model_traces is incomplete. **Decisions worth flagging** - Stage 6 input is just the analysis content (triage + sonnet + optional opus). No `pre` payload needed — fingerprinting is about the produced *analysis*, not the source ticket. - Backfill processes oldest-first (triggered_at ASC). Lets us observe a few rounds before chewing through hundreds. - Stage 6 failure is non-fatal. Spec: "If fingerprinting fails, do NOT fail the overall analysis." ## 2.5 — Browse / filter UI rebuild **Delivered** - New endpoint: `GET /api/analyzer/tickets` (replaces the simpler `GET /api/analyzer/tickets/list` from Phase 1.9). Multi-select CSV-style query params (clientId, issueType, queue, status, priority, assignedTo); analyzed segmented filter (any/yes/no/stale); needsReview toggle; search; sort. - `/api/analyzer/tickets/filter-options` extended with queues, statuses, priorities, resources (joined to "has at least one ticket" so the dropdowns aren't padded). - New `` component at `components/ui/multi-select.tsx` — Popover + checkbox list with optional search box (auto-shown above 8 options). One trigger + one popover, no shadcn Command dependency. - `/analyzer/tickets` page rebuilt: - Sticky filter bar with period pills, multi-selects, search, analyzed segmented, needs-review checkbox, sort - Active-filter chips (click to clear individual filter) - Bulk selection persisted via localStorage (key `analyzer:ticket-selection:v1`) — survives pagination - "Analyze N selected" — sequential job queue, forces re-analyze on `stale` rows - "Generate aggregate report" — routes to `/analyzer/reports/new`; only enabled when all selected are `current` - Top nav reorganized: Browse Tickets / Aggregate Reports / Needs Review under "Analyzer". **Decisions worth flagging** - **Staleness via `last_activity_date > completed_at`**, not content-hash compare. The spec lets either; the date heuristic is good enough and avoids per-row preprocessing on 50-row paginated responses. - **`MultiSelect` is a one-popover-per-instance design** — multiple popovers can be open across the bar. Acceptable; matches how Linear / Vercel's table filters behave. - **No "Select all matching filters" semantic**. Selection is an explicit per-row action stored as ticket numbers in localStorage. Filter-level selection adds significant complexity (server has to resolve filter→IDs, two-modes everywhere). Skipped for V1; the spec's intent (don't lose selection on pagination) is met. - **Bulk Analyze is sequential, not parallel.** N concurrent calls would all hit `claimQueuedJob` and the worker would process them one at a time anyway (single in-process worker). Sequential POSTs are more honest about that. ## 2.6 — Aggregate reports **Delivered** - Migration 071: `analyzer_aggregate_reports` table + ALTER on `analyzer_stage_executions` to drop NOT NULL on `analysis_id`, add `aggregate_report_id` FK, add the `analyzer_stage_executions_parent_check` CHECK constraint (`(analysis_id IS NOT NULL) <> (aggregate_report_id IS NOT NULL)`). - `lib/services/analyzer/stages/aggregate-reduce.ts` — Sonnet (Opus opt-in) reduce stage with the spec's verbatim system prompt and `AggregateReduceResponse` Zod schema. - `lib/services/analyzer/aggregate-persistence.ts` — `createAggregateReport`, `getAggregateReport`, `listAggregateReports`, `runAggregateReport`, `bulkInsertReportStageExecutions`. The runner is fire-and-forget (called via `void runAggregateReport(id)` from the POST endpoint); it persists distributions immediately so the UI can show partial results during the LLM call. - IT Glue context fetcher: per-client `findOrganizationByName` + `getFlexibleAssets`, capped at 200 doc titles total per spec. Failure tolerant — per-client errors don't fail the report. - API endpoints: - `POST /api/analyzer/aggregate-reports` — validates (≤100, fingerprint exists, not stale), creates pending row, fires runner - `GET /api/analyzer/aggregate-reports/:id` — full report row (UI polls this every 3s while pending/running) - `GET /api/analyzer/aggregate-reports` — paginated list - Pages: - `/analyzer/reports/new?ids=T...,T...` — pre-flight: shows selected tickets, options (title, IT Glue context toggle), Generate button - `/analyzer/reports/[id]` — pending → distributions → completed. Sections: header, executive summary, four distribution mini-bar cards, documentation gaps (with `itglue_check` color tone), process gaps (severity tone), recurrence clusters, recommended actions (sorted by priority), narrative summary. - `/analyzer/reports` — table list of past reports **Decisions worth flagging** - **Fire-and-forget runner**, no separate worker module. The POST endpoint kicks `void runAggregateReport(id)`; updates land in the row when the LLM call completes. Frontend polls. Avoids adding a second polling worker alongside `analyzerWorker`. - **Stage names reused for aggregate sub-stages.** The CHECK constraint on `analyzer_stage_executions.stage` enumerates the per-analysis stage names. Aggregate sub-stages (SQL aggregation, IT Glue context, reduce LLM) are recorded with `stage='analyze'` / `'itglue'` plus `aggregate_report_id` set. A future migration could add `aggregate_sql` / `aggregate_reduce` to the enum and re-emit those rows; for now the existing names are good enough for forensics. - **`generated_by_user_id` is TEXT nullable**, not `uuid NOT NULL` per spec. Better Auth's `user.id` is text, and we want the report to remain readable if the generating user is later deleted — matches the pattern from `analyzer_analyses.triggered_by_user_id`. - **Distributions persist before LLM call** so partial-state UI doesn't have to wait the full 30–90s for anything to render. ## 2.7 — Cost guards **Delivered** - Migration 072: `analyzer_cost_audit` table. - `lib/services/analyzer/cost-guard.ts`: `estimateAggregateReportCost`, `getUserDailySpend`, `evaluateCost`, `recordCostAuditDecision`. Thresholds: `REQUIRES_CONFIRMATION_USD = 5`, `SOFT_WARN_DAILY_USD = 20`, `HARD_BLOCK_DAILY_USD = 50`. - `evaluateCost` produces a four-state decision (`approved` / `requires_confirmation` / `blocked` / `overridden`) plus boolean `softWarn`/`hardBlocked`/`requiresConfirmation`/`isOverride` fields the API can return for UX. - POST `/api/analyzer/aggregate-reports` enforces: - `requires_confirmation` → 400 with `requiresConfirmation:true, estimatedCost, dailySpendBefore` so the frontend can show `confirm()` and re-POST with `confirmedCost:true`. - `blocked` → 403 with the daily spend in the body. - Every decision (including `approved`) writes a row to `analyzer_cost_audit`. - Override env var `ANALYZER_DAILY_COST_OVERRIDE_USERS` (comma-separated user ids). - Frontend new-report page: catches `requiresConfirmation`, shows `window.confirm()` with the dollar figure, retries with `confirmedCost: true`. **Decisions worth flagging** - **Cost estimate is char/4 → tokens × Sonnet pricing.** Crude but pessimistic in the right direction. At 100 tickets the estimate comes in under $0.50 — far below the $5 threshold — so the confirmation modal almost never fires in practice. Ceiling exists to catch payload bloat / Opus-opt-in scenarios. - **Daily window is trailing 24h, not "today UTC".** Avoids midnight-edge-of-day reset gaming; rolling window is what the spec calls "$X/day" naturally. - **Soft warn at $20/day is informational only.** Fields exposed in the cost evaluation; UI can choose to surface, but the API doesn't refuse to proceed. Hard block at $50/day is the only enforcement. ## 2.8 — Documentation **Delivered** - `docs/wulf-pulse-ticket-analyzer-runbook.md` — added Phase 2 sections covering stage execution forensics, fingerprint backfill, aggregate report flow + SQL queries, cost guard configuration + override env var, and the rebuilt browse UI behavior. - This file — the per-sub-phase notes above. --- ## Status after Phase 2 | Phase | Tests | tsc | Notes | |---|---|---|---| | 2.1 | 128 | clean | schema (070) | | 2.2 | 128 | clean | stage_executions writes + failure-tolerant persistence | | 2.3 | 128 | clean | markdown rendering + Stage 3 prompt | | 2.4 | 128 | clean | Stage 6 fingerprint + backfill CLI | | 2.5 | 128 | clean | browse UI rebuild | | 2.6 | 128 | clean | aggregate reports (071, runner, 3 endpoints, 3 pages) | | 2.7 | 128 | clean | cost guards (072, audit log, threshold gating) | | 2.8 | 128 | clean | runbook + build notes |