# AI Ticket Analyzer — Build Notes A running journal of the multi-phase build for the AI Ticket Analyzer feature. Captures what was delivered each phase, design decisions worth flagging, and what was deliberately left out. Spec lives in `wulf-pulse-ticket-analyzer-prompt.md`. This file is updated after each phase ships. --- ## Phase 1 — Migration + Zod schemas **Delivered** - `migrations/069_create_analyzer_tables.sql` — three tables (`analyzer_analyses`, `analyzer_shares`, `analyzer_jobs`) with `pgcrypto` extension guard, indexes per spec, status `CHECK` constraints, and FK types corrected to `TEXT` (not `UUID`) to match Better Auth's `user.id`. - `lib/types/analyzer.ts` — Zod schemas for every LLM-stage parsed JSON (`TaggedEvent`, `TriageResponse`, `DeepAnalysisResponse`, `OpusResponse`), persisted row shapes, job status, API request bodies, and the internal `PreprocessedTicket` payload that flows through the pipeline. **Decisions worth flagging** - `triggered_by_user_id` is **nullable + `ON DELETE SET NULL`** (not `NOT NULL`). An analysis should still be readable in history if the triggering user is later deleted. `analyzer_shares.shared_by_user_id` is `NOT NULL + ON DELETE CASCADE` (audit-log style — share rows go with the user). - `evidence_timestamps` typed as `string().datetime({offset: true})` (ISO timestamps), not numeric indices. More robust to model hallucination and reads better in the UI. - `TaggedEvent` does NOT support a "two separate events" form for time entries with both Summary + Internal Notes — single event with both fields, per the spec's preference for a cleaner timeline. **Deliberately left out** - Migration is committed but **not applied** to any running DB. Postgres only re-applies migrations on first init of a fresh volume; the existing DB needs a manual run of this migration when phase 5 is exercised. --- ## Phase 2 — Stage 0 pre-processor + IT Glue redaction **Delivered** - `lib/services/analyzer/itglue-redact.ts` — recursive walk; matches keys against `/password|secret|key|token|credential|api[_-]?key/i`; replaces values with `[REDACTED]`. Subtree redaction (key matching `auth` → no leaves leak), defensive copy, cycle guard (`[CIRCULAR]` marker). - `lib/services/analyzer/preprocessor.ts` — filters workflow noise + Service Desk Notification rows, tags ticket_create/notes/time entries with `actor / actor_type / source / visibility / summary_notes / internal_notes / hours`, sorts chronologically, computes `sha256` content hash over canonical JSON of `(events, status, priority, queue)`. - `vitest@^4.1.5` added as devDep with `vitest.config.ts` setting up the `@/` alias. Two test files at this phase: 32 redaction tests + 36 preprocessor tests including the regression run against the `T20260424.0045` fixture. - Type-aliasing fix in `lib/types/analyzer.ts`: added `export type X = z.infer` for `ActorType`, `EventSource`, `Visibility`, `Severity`, `ComplexityTier`, `TicketType` — the Zod-enum const exports alone don't produce a usable TypeScript type. **Decisions worth flagging** - `actor_type` is classified by **email domain, not author text**, per the spec. `lorentz@wulfconsulting.com` is `wulf_tech` regardless of how the message reads. Sonnet handles the role nuance at Stage 3. - Time entries with no narrative content (no summary, no internal notes) are dropped — a purely numeric entry adds nothing. - Vendor-domain allowlist is intentionally short and conservative (Vertafore, Datto, Microsoft, etc.). Misclassifying a customer domain as "vendor" is worse than the default `client_contact`. **Deliberately left out** - No real protection for secrets embedded in **free text** (e.g. a notes field containing the literal string "the password is hunter2"). The redaction guarantee is on field **keys**, not values. The IT Glue search test documents this contract explicitly so it isn't "fixed" without thought. --- ## Phase 3 — Anthropic SDK setup + Stage 1 (Haiku triage) **Delivered** - `lib/services/llm/{models,pricing,client,call}.ts` — model ID constants, per-model rate table (Haiku $1/$5, Sonnet $3/$15, Opus $5/$25 per 1M tokens + cache read/write tiers), lazy SDK singleton, generic `callLLMStage({model, system, user, schema, maxTokens, client?})` helper that: - Marks the system prompt with `cache_control: {type: "ephemeral"}` - Sends NO `temperature`/`top_p`/`top_k` (Opus 4.7 would 400) - Strips a single ` ```json ` fence before parsing - On parse failure, retries **once** with prior-attempt + error in a follow-up user turn - Returns `{data, usage, estimated_cost_usd, attempts, raw_response}` - `lib/services/analyzer/stages/stage1-triage.ts` — Haiku caller with the spec's verbatim system prompt and a 50KB user-payload cap that drops oldest internal-only events first when oversized. - `lib/services/analyzer/itglue-search.ts` — search facade that runs every result through `redact()` before returning. Snippets capped at 2000 chars, doc count capped at 10. `itglue-aliases.json` skeleton for known fuzzy org-name mappings. - 32 new tests across pricing/call/Stage 1/IT Glue search. **Decisions worth flagging** - **Manual JSON.parse + Zod validate, not `output_config.format` / `client.messages.parse()`.** The spec said retry-once on Zod parse failure, and Zod 4 ↔ JSON Schema conversion has edge cases I didn't want to depend on (e.g. `.datetime({offset: true})` → JSON Schema `format`). Manual parse is what the spec asks for and is more transparent. - **Prompt caching probably won't fire on these stages.** Our system prompts are ~1-2 KB (~250-500 tokens); minimum cacheable prefix is 2048 (Sonnet) or 4096 (Haiku/Opus) tokens. The `cache_control` marker is a no-op below threshold and incurs no cost — left in defensively, but caching is not a meaningful lever for this workload. - **Redaction is on KEY names, not free text** — re-stated for emphasis. The IT Glue search test asserts this contract. - **`itglue-aliases.json` is a skeleton** with `_comment` / `_example` keys documenting the format. Real org-id entries get added when phase 5 wires this into the pipeline. **Deliberately left out** - No live API integration test. All Stage 1 tests use a mocked `Anthropic` client. A real-API smoke test belongs in phase 5+ when we run end-to-end. - Did **not** wrap `lib/services/itglue-client.ts` in a redacting search facade for non-LLM callers. The redaction primitive is ready; non-LLM callers (sync service, data browsers) intentionally have full data — they are not the path that needs protection. --- ## Phase 4 — Pipeline + Job worker (Stages 3, 4, 5) **Delivered** - `lib/services/analyzer/stages/stage3-deep-analysis.ts` — Sonnet caller, spec's verbatim prompt, 80KB payload cap. - `lib/services/analyzer/stages/stage4-deep-reasoning.ts` — Opus caller plus pure helpers `shouldRunDeepReasoning()` and `applyOpusUpdates()`. - `lib/services/analyzer/data-access.ts` — `loadTicketBundle(ticketNumber)` joins `tickets` / `statuses` / `priorities` / `queues` / `companies` / `contacts` / `resources` / `ticket_notes` / `time_entries` and returns the exact `RawTicketBundle` shape the preprocessor expects. Throws typed `TicketNotFoundError`. - `lib/services/analyzer/persistence.ts` — `getNextAnalysisVersion`, `findExistingAnalysisByContentHash`, `insertAnalysis`, plus job `claimQueuedJob` / `updateJobStatus` / `completeJob` / `failJob` / `queueJob` / `getJob`. Job claim uses `FOR UPDATE SKIP LOCKED` so multiple Next.js workers can poll safely. - `lib/services/analyzer/pipeline.ts` — composes Stage 0 → idempotency check → Stage 1 → (Stage 2 if `itglue_lookup_needed`) → Stage 3 → (Stage 4 if trigger fires AND cost ceiling not reached) → result. Cost circuit breaker trips at $2.00 before Opus, sets `needs_human_review=true` with a reason. Returns full `model_traces` for debugging. - `lib/services/analyzer/worker.ts` — singleton with 2-second poll loop; auto-starts in production; opt-in in dev via `ANALYZER_WORKER_AUTOSTART=1`; **skipped** under vitest. `runJob()` exposed for tests + manual triggers. `TicketNotFoundError` produces a user-facing job error message that fingers the sync as the culprit. - 28 new tests bringing the total to 128. **Decisions worth flagging** - **Worker auto-start is more conservative than `sync-scheduler.ts`.** That one auto-starts on any non-browser import (including tests). I gated this one because the worker hits the database AND runs LLM calls — much higher blast radius. To run locally, set `ANALYZER_WORKER_AUTOSTART=1`. - **Cost circuit breaker only fires before Opus.** Sonnet runs unconditionally even if it would push past $2. Spec wording matches; if the team wants stricter control, the natural place is next to `COST_CEILING_USD` in `pipeline.ts`. - **Idempotency check matches `status='complete'` only.** Failed runs don't poison the cache. - **IT Glue failures are tolerated.** Search throws → analysis continues without context, doesn't fail the run. - `worker.test.ts` casts `mockImplementationOnce` to `as never` because vitest's overload resolution fights us when the mocked function has multiple call signatures. Functional, just ugly. **Deliberately left out** - No live API integration test (still). Phase 5+ is the natural place for an end-to-end smoke test. - Stage 3 doesn't have a fixture-driven happy-path test like Stage 1; it's exercised at the orchestration layer via `pipeline.test.ts` only. Worth adding direct Stage 3 tests later. - Migration `069` still not applied — same as phase 1. --- ## Phase 5 — API routes **Delivered** - Persistence read paths added to `lib/services/analyzer/persistence.ts`: `getAnalysisById`, `listAnalysesByTicketNumber`, `listNeedsReview`, `createShare`, plus a shared `rowToPersistedAnalysis` row mapper. - 6 routes under `app/api/analyzer/`: | Route | Method | Returns | |---|---|---| | `tickets/[ticketNumber]/analyze` | POST | `{status, jobId?, existingAnalysisId?}` | | `tickets/[ticketNumber]/analyses` | GET | `{analyses: PersistedAnalysis[]}` | | `jobs/[jobId]` | GET | `{job}` | | `analyses/[id]` | GET | `{analysis}` | | `analyses/[id]/share` | POST | `{share}` | | `needs-review?limit=&offset=` | GET | `{analyses}` | **Decisions worth flagging** - **Analyze runs preprocess inline.** The route does `loadTicketBundle → preprocessTicket → findExistingAnalysisByContentHash` synchronously to support the spec's `existingAnalysisId?` immediate response. The worker also runs this — duplicated work, but preprocess is fast (deterministic, one DB load) and the alternative (always queue, frontend polls to discover the short-circuit) is worse UX. - **Email send deferred to phase 8** per the spec's delivery order. The share route persists the audit row and validates the recipient domain against `ALLOWED_SHARE_DOMAINS`. Until phase 8, share rows have `viewed_at = null` indefinitely. - **`requireAuth()` everywhere — not `requireAdmin()`.** Any authenticated user can analyze a ticket they have access to. If `/needs-review` should be admin-only later, swap that one to `requirePermission('analyzer', 'review')` once the permission map is decided. - **No new permission entries added to `lib/permissions.ts`.** Adding scoped permissions for a feature still under build risks getting them wrong. - Routes are NOT in `middleware.ts` `publicRoutes` — they require a session. **Deliberately left out** - No API route tests. The repo has zero `app/api/**/*.test.ts` files; the routes are thin orchestration on top of already-tested persistence. Adding integration tests means setting up a test harness for the auth helpers + Postgres, which is a separate effort. --- ## Phase 6 — Frontend pages **Delivered** - `components/analyzer/analyze-button.tsx` — `` with the full state machine: POSTs to the analyze endpoint, navigates straight to an existing analysis if `existingAnalysisId` came back, otherwise polls `/api/analyzer/jobs/:jobId` every 2s and renders stage labels (Queued → Fetching → Triaging → Searching IT Glue → Analyzing → Deep review → Done). 5-minute hard timeout. Failures surface as toast errors. Supports `force` for explicit re-run. - `components/analyzer/share-modal.tsx` — `` with a shadcn Dialog + email field + optional note (max 2000 chars). POSTs to the share endpoint and surfaces server-side validation errors (domain not allowed → toast). - `components/analyzer/analysis-view.tsx` — the full 10-section analysis layout per spec. Header with model-tier badges (Haiku/Sonnet/Opus pills), confidence score, total cost, Share + Re-analyze buttons. Summary, Next Step (with rationale collapsed), Timeline (vertical list with 🟢 / 🔒 / 🔄 markers, click to expand), What Was Done / Should Have Been Done side-by-side on wide screens, Gaps colored by severity with "Evidence:" links that scroll-and-expand the matching timeline event, Post-Resolution (only if present), Human Review Flags (only if needed), IT Glue References. - `app/analyzer/ticket/[ticketNumber]/page.tsx` — ticket detail with the Analyze button and a list of historical versions; latest is badged. - `app/analyzer/analysis/[id]/page.tsx` — fetches one analysis and renders it via ``. - `app/analyzer/queue/page.tsx` — needs-review queue, shows ticket number, version, summary preview, top reasons, confidence badge, and a `high gap` badge if any gap is high severity. **Decisions worth flagging** - **Imperative `useState` + `useEffect` + `fetch`**, no SWR / react-query — matches CLAUDE.md and the rest of the repo. Don't refactor to a global cache layer for these three pages; if it becomes a real pain, that's a whole-app concern. - **`'use client'` everywhere.** The pages use `use(params)` (the React hook) to unwrap Next.js 16's `params: Promise<...>` shape on the client. Server components weren't appropriate here — every page does interactive state (analyze flow, expand events, share modal). - **Stage labels render directly from `JobStatus` enum values**, not a separate label list, so any new statuses added to the enum auto-render with their default name. - **Re-analyze banner not implemented yet.** The spec calls for "New activity since last analysis · Re-analyze" when the live content_hash drifts from the persisted one. That requires a live-preprocess endpoint (or running preprocess on the page render). Skipped for now — the user can always click Re-analyze. Worth adding once we have real-world signal on whether activity drift is common. - **No autocomplete on the share-recipient field.** Spec says "autocomplete from existing wulf-pulse user list if available". Skipped — the existing user list is in Better Auth's `user` table; exposing it requires a small API endpoint. Easy follow-up. - **No navigation entry yet.** `components/navigation/app-navigation.tsx` doesn't yet have an "Analyzer" link. Adding that is a one-line edit; I left it for the operator to opt in once the feature is staged. - **Printable analysis view** — the spec mentions "printable" for the analysis page. The current layout is print-friendly by accident (no fixed sidebars, sectioned cards), but no explicit `@media print` styles yet. Add when someone asks. **Deliberately left out** - No frontend tests. vitest is configured for `lib/**/*.test.ts` only; the pages and components are visually verified. Component tests with testing-library would be a separate setup decision. - Analysis view doesn't auto-refresh while a job is running on a *different* version. If a user navigates to an old version while a new one is in progress, they don't see the in-progress state. Acceptable — the queue view + ticket history give that signal. --- ## Status after each phase | Phase | Tests | tsc | Notes | |---|---|---|---| | 1 | 0 | clean | migration + types only | | 2 | 68 | clean | redaction + preprocessor regression | | 3 | 100 | clean | + LLM scaffolding + Stage 1 | | 4 | 128 | clean | + pipeline + worker | | 5 | 128 | clean | API routes (no route tests) | | 6 | 128 | clean | frontend (no FE tests) |