wulf-pulse/docs/wulf-pulse-ticket-analyzer-build-notes.md

# AI Ticket Analyzer — Build Notes

A running journal of the multi-phase build for the AI Ticket Analyzer feature.
Captures what was delivered each phase, design decisions worth flagging, and
what was deliberately left out. Spec lives in
`wulf-pulse-ticket-analyzer-prompt.md`.

This file is updated after each phase ships.

---

## Phase 1 — Migration + Zod schemas

**Delivered**

- `migrations/069_create_analyzer_tables.sql` — three tables
  (`analyzer_analyses`, `analyzer_shares`, `analyzer_jobs`) with `pgcrypto`
  extension guard, indexes per spec, status `CHECK` constraints, and FK types
  corrected to `TEXT` (not `UUID`) to match Better Auth's `user.id`.
- `lib/types/analyzer.ts` — Zod schemas for every LLM-stage parsed JSON
  (`TaggedEvent`, `TriageResponse`, `DeepAnalysisResponse`, `OpusResponse`),
  persisted row shapes, job status, API request bodies, and the internal
  `PreprocessedTicket` payload that flows through the pipeline.

**Decisions worth flagging**

- `triggered_by_user_id` is **nullable + `ON DELETE SET NULL`** (not `NOT NULL`).
  An analysis should still be readable in history if the triggering user is
  later deleted. `analyzer_shares.shared_by_user_id` is `NOT NULL +
  ON DELETE CASCADE` (audit-log style — share rows go with the user).
- `evidence_timestamps` typed as `string().datetime({offset: true})` (ISO
  timestamps), not numeric indices. More robust to model hallucination and
  reads better in the UI.
- `TaggedEvent` does NOT support a "two separate events" form for time entries
  with both Summary + Internal Notes — single event with both fields, per the
  spec's preference for a cleaner timeline.

**Deliberately left out**

- Migration is committed but **not applied** to any running DB. Postgres only
  re-applies migrations on first init of a fresh volume; the existing DB needs
  a manual run of this migration when phase 5 is exercised.

---

## Phase 2 — Stage 0 pre-processor + IT Glue redaction

**Delivered**

- `lib/services/analyzer/itglue-redact.ts` — recursive walk; matches keys
  against `/password|secret|key|token|credential|api[_-]?key/i`; replaces
  values with `[REDACTED]`. Subtree redaction (key matching `auth` → no leaves
  leak), defensive copy, cycle guard (`[CIRCULAR]` marker).
- `lib/services/analyzer/preprocessor.ts` — filters workflow noise + Service
  Desk Notification rows, tags ticket_create/notes/time entries with
  `actor / actor_type / source / visibility / summary_notes / internal_notes /
  hours`, sorts chronologically, computes `sha256` content hash over canonical
  JSON of `(events, status, priority, queue)`.
- `vitest@^4.1.5` added as devDep with `vitest.config.ts` setting up the
  `@/` alias. Two test files at this phase: 32 redaction tests + 36
  preprocessor tests including the regression run against the
  `T20260424.0045` fixture.
- Type-aliasing fix in `lib/types/analyzer.ts`: added `export type X =
  z.infer<typeof X>` for `ActorType`, `EventSource`, `Visibility`, `Severity`,
  `ComplexityTier`, `TicketType` — the Zod-enum const exports alone don't
  produce a usable TypeScript type.

**Decisions worth flagging**

- `actor_type` is classified by **email domain, not author text**, per the
  spec. `lorentz@wulfconsulting.com` is `wulf_tech` regardless of how the
  message reads. Sonnet handles the role nuance at Stage 3.
- Time entries with no narrative content (no summary, no internal notes) are
  dropped — a purely numeric entry adds nothing.
- Vendor-domain allowlist is intentionally short and conservative (Vertafore,
  Datto, Microsoft, etc.). Misclassifying a customer domain as "vendor" is
  worse than the default `client_contact`.

**Deliberately left out**

- No real protection for secrets embedded in **free text** (e.g. a notes field
  containing the literal string "the password is hunter2"). The redaction
  guarantee is on field **keys**, not values. The IT Glue search test
  documents this contract explicitly so it isn't "fixed" without thought.

---

## Phase 3 — Anthropic SDK setup + Stage 1 (Haiku triage)

**Delivered**

- `lib/services/llm/{models,pricing,client,call}.ts` — model ID constants,
  per-model rate table (Haiku $1/$5, Sonnet $3/$15, Opus $5/$25 per 1M tokens
  + cache read/write tiers), lazy SDK singleton, generic
  `callLLMStage<T>({model, system, user, schema, maxTokens, client?})`
  helper that:
  - Marks the system prompt with `cache_control: {type: "ephemeral"}`
  - Sends NO `temperature`/`top_p`/`top_k` (Opus 4.7 would 400)
  - Strips a single ` ```json ` fence before parsing
  - On parse failure, retries **once** with prior-attempt + error in a
    follow-up user turn
  - Returns `{data, usage, estimated_cost_usd, attempts, raw_response}`
- `lib/services/analyzer/stages/stage1-triage.ts` — Haiku caller with the
  spec's verbatim system prompt and a 50KB user-payload cap that drops oldest
  internal-only events first when oversized.
- `lib/services/analyzer/itglue-search.ts` — search facade that runs every
  result through `redact()` before returning. Snippets capped at 2000 chars,
  doc count capped at 10. `itglue-aliases.json` skeleton for known fuzzy
  org-name mappings.
- 32 new tests across pricing/call/Stage 1/IT Glue search.

**Decisions worth flagging**

- **Manual JSON.parse + Zod validate, not `output_config.format` /
  `client.messages.parse()`.** The spec said retry-once on Zod parse failure,
  and Zod 4 ↔ JSON Schema conversion has edge cases I didn't want to depend
  on (e.g. `.datetime({offset: true})` → JSON Schema `format`). Manual parse
  is what the spec asks for and is more transparent.
- **Prompt caching probably won't fire on these stages.** Our system prompts
  are ~1-2 KB (~250-500 tokens); minimum cacheable prefix is 2048 (Sonnet) or
  4096 (Haiku/Opus) tokens. The `cache_control` marker is a no-op below
  threshold and incurs no cost — left in defensively, but caching is not a
  meaningful lever for this workload.
- **Redaction is on KEY names, not free text** — re-stated for emphasis. The
  IT Glue search test asserts this contract.
- **`itglue-aliases.json` is a skeleton** with `_comment` / `_example` keys
  documenting the format. Real org-id entries get added when phase 5 wires
  this into the pipeline.

**Deliberately left out**

- No live API integration test. All Stage 1 tests use a mocked `Anthropic`
  client. A real-API smoke test belongs in phase 5+ when we run end-to-end.
- Did **not** wrap `lib/services/itglue-client.ts` in a redacting search
  facade for non-LLM callers. The redaction primitive is ready; non-LLM
  callers (sync service, data browsers) intentionally have full data — they
  are not the path that needs protection.

---

## Phase 4 — Pipeline + Job worker (Stages 3, 4, 5)

**Delivered**

- `lib/services/analyzer/stages/stage3-deep-analysis.ts` — Sonnet caller,
  spec's verbatim prompt, 80KB payload cap.
- `lib/services/analyzer/stages/stage4-deep-reasoning.ts` — Opus caller plus
  pure helpers `shouldRunDeepReasoning()` and `applyOpusUpdates()`.
- `lib/services/analyzer/data-access.ts` — `loadTicketBundle(ticketNumber)`
  joins `tickets` / `statuses` / `priorities` / `queues` / `companies` /
  `contacts` / `resources` / `ticket_notes` / `time_entries` and returns the
  exact `RawTicketBundle` shape the preprocessor expects. Throws typed
  `TicketNotFoundError`.
- `lib/services/analyzer/persistence.ts` — `getNextAnalysisVersion`,
  `findExistingAnalysisByContentHash`, `insertAnalysis`, plus job
  `claimQueuedJob` / `updateJobStatus` / `completeJob` / `failJob` /
  `queueJob` / `getJob`. Job claim uses `FOR UPDATE SKIP LOCKED` so multiple
  Next.js workers can poll safely.
- `lib/services/analyzer/pipeline.ts` — composes Stage 0 → idempotency check
  → Stage 1 → (Stage 2 if `itglue_lookup_needed`) → Stage 3 → (Stage 4 if
  trigger fires AND cost ceiling not reached) → result. Cost circuit breaker
  trips at $2.00 before Opus, sets `needs_human_review=true` with a
  reason. Returns full `model_traces` for debugging.
- `lib/services/analyzer/worker.ts` — singleton with 2-second poll loop;
  auto-starts in production; opt-in in dev via
  `ANALYZER_WORKER_AUTOSTART=1`; **skipped** under vitest. `runJob()`
  exposed for tests + manual triggers. `TicketNotFoundError` produces a
  user-facing job error message that fingers the sync as the culprit.
- 28 new tests bringing the total to 128.

**Decisions worth flagging**

- **Worker auto-start is more conservative than `sync-scheduler.ts`.** That
  one auto-starts on any non-browser import (including tests). I gated this
  one because the worker hits the database AND runs LLM calls — much higher
  blast radius. To run locally, set `ANALYZER_WORKER_AUTOSTART=1`.
- **Cost circuit breaker only fires before Opus.** Sonnet runs unconditionally
  even if it would push past $2. Spec wording matches; if the team wants
  stricter control, the natural place is next to `COST_CEILING_USD` in
  `pipeline.ts`.
- **Idempotency check matches `status='complete'` only.** Failed runs don't
  poison the cache.
- **IT Glue failures are tolerated.** Search throws → analysis continues
  without context, doesn't fail the run.
- `worker.test.ts` casts `mockImplementationOnce` to `as never` because
  vitest's overload resolution fights us when the mocked function has multiple
  call signatures. Functional, just ugly.

**Deliberately left out**

- No live API integration test (still). Phase 5+ is the natural place for an
  end-to-end smoke test.
- Stage 3 doesn't have a fixture-driven happy-path test like Stage 1; it's
  exercised at the orchestration layer via `pipeline.test.ts` only. Worth
  adding direct Stage 3 tests later.
- Migration `069` still not applied — same as phase 1.

---

## Phase 5 — API routes

**Delivered**

- Persistence read paths added to `lib/services/analyzer/persistence.ts`:
  `getAnalysisById`, `listAnalysesByTicketNumber`, `listNeedsReview`,
  `createShare`, plus a shared `rowToPersistedAnalysis` row mapper.
- 6 routes under `app/api/analyzer/`:
  | Route | Method | Returns |
  |---|---|---|
  | `tickets/[ticketNumber]/analyze` | POST | `{status, jobId?, existingAnalysisId?}` |
  | `tickets/[ticketNumber]/analyses` | GET | `{analyses: PersistedAnalysis[]}` |
  | `jobs/[jobId]` | GET | `{job}` |
  | `analyses/[id]` | GET | `{analysis}` |
  | `analyses/[id]/share` | POST | `{share}` |
  | `needs-review?limit=&offset=` | GET | `{analyses}` |

**Decisions worth flagging**

- **Analyze runs preprocess inline.** The route does
  `loadTicketBundle → preprocessTicket → findExistingAnalysisByContentHash`
  synchronously to support the spec's `existingAnalysisId?` immediate
  response. The worker also runs this — duplicated work, but preprocess is
  fast (deterministic, one DB load) and the alternative (always queue,
  frontend polls to discover the short-circuit) is worse UX.
- **Email send deferred to phase 8** per the spec's delivery order. The share
  route persists the audit row and validates the recipient domain against
  `ALLOWED_SHARE_DOMAINS`. Until phase 8, share rows have `viewed_at = null`
  indefinitely.
- **`requireAuth()` everywhere — not `requireAdmin()`.** Any authenticated
  user can analyze a ticket they have access to. If `/needs-review` should
  be admin-only later, swap that one to `requirePermission('analyzer',
  'review')` once the permission map is decided.
- **No new permission entries added to `lib/permissions.ts`.** Adding scoped
  permissions for a feature still under build risks getting them wrong.
- Routes are NOT in `middleware.ts` `publicRoutes` — they require a session.

**Deliberately left out**

- No API route tests. The repo has zero `app/api/**/*.test.ts` files; the
  routes are thin orchestration on top of already-tested persistence. Adding
  integration tests means setting up a test harness for the auth helpers +
  Postgres, which is a separate effort.

---

## Phase 6 — Frontend pages

**Delivered**

- `components/analyzer/analyze-button.tsx` — `<AnalyzeButton>` with the full
  state machine: POSTs to the analyze endpoint, navigates straight to an
  existing analysis if `existingAnalysisId` came back, otherwise polls
  `/api/analyzer/jobs/:jobId` every 2s and renders stage labels (Queued →
  Fetching → Triaging → Searching IT Glue → Analyzing → Deep review → Done).
  5-minute hard timeout. Failures surface as toast errors. Supports `force`
  for explicit re-run.
- `components/analyzer/share-modal.tsx` — `<ShareModal>` with a shadcn Dialog
  + email field + optional note (max 2000 chars). POSTs to the share
  endpoint and surfaces server-side validation errors (domain not allowed →
  toast).
- `components/analyzer/analysis-view.tsx` — the full 10-section analysis
  layout per spec. Header with model-tier badges (Haiku/Sonnet/Opus pills),
  confidence score, total cost, Share + Re-analyze buttons. Summary, Next
  Step (with rationale collapsed), Timeline (vertical list with 🟢 / 🔒 /
  🔄 markers, click to expand), What Was Done / Should Have Been Done
  side-by-side on wide screens, Gaps colored by severity with "Evidence:"
  links that scroll-and-expand the matching timeline event,
  Post-Resolution (only if present), Human Review Flags (only if needed),
  IT Glue References.
- `app/analyzer/ticket/[ticketNumber]/page.tsx` — ticket detail with the
  Analyze button and a list of historical versions; latest is badged.
- `app/analyzer/analysis/[id]/page.tsx` — fetches one analysis and renders
  it via `<AnalysisView>`.
- `app/analyzer/queue/page.tsx` — needs-review queue, shows ticket number,
  version, summary preview, top reasons, confidence badge, and a `high gap`
  badge if any gap is high severity.

**Decisions worth flagging**

- **Imperative `useState` + `useEffect` + `fetch`**, no SWR / react-query —
  matches CLAUDE.md and the rest of the repo. Don't refactor to a global
  cache layer for these three pages; if it becomes a real pain, that's a
  whole-app concern.
- **`'use client'` everywhere.** The pages use `use(params)` (the React
  hook) to unwrap Next.js 16's `params: Promise<...>` shape on the client.
  Server components weren't appropriate here — every page does interactive
  state (analyze flow, expand events, share modal).
- **Stage labels render directly from `JobStatus` enum values**, not a
  separate label list, so any new statuses added to the enum auto-render
  with their default name.
- **Re-analyze banner not implemented yet.** The spec calls for "New
  activity since last analysis · Re-analyze" when the live content_hash
  drifts from the persisted one. That requires a live-preprocess endpoint
  (or running preprocess on the page render). Skipped for now — the user
  can always click Re-analyze. Worth adding once we have real-world signal
  on whether activity drift is common.
- **No autocomplete on the share-recipient field.** Spec says "autocomplete
  from existing wulf-pulse user list if available". Skipped — the existing
  user list is in Better Auth's `user` table; exposing it requires a
  small API endpoint. Easy follow-up.
- **No navigation entry yet.** `components/navigation/app-navigation.tsx`
  doesn't yet have an "Analyzer" link. Adding that is a one-line edit; I
  left it for the operator to opt in once the feature is staged.
- **Printable analysis view** — the spec mentions "printable" for the
  analysis page. The current layout is print-friendly by accident (no
  fixed sidebars, sectioned cards), but no explicit `@media print` styles
  yet. Add when someone asks.

**Deliberately left out**

- No frontend tests. vitest is configured for `lib/**/*.test.ts` only; the
  pages and components are visually verified. Component tests with
  testing-library would be a separate setup decision.
- Analysis view doesn't auto-refresh while a job is running on a
  *different* version. If a user navigates to an old version while a new
  one is in progress, they don't see the in-progress state. Acceptable —
  the queue view + ticket history give that signal.

---

## Phase 7 — Share-via-email integration

**Delivered**

- `sendAnalysisShareEmail()` added to `lib/services/email.ts` — re-uses the
  existing nodemailer SMTP transport that already serves magic-link and
  invitation mail. Subject `Pulse analysis · <ticket> v<n> — <summary
  excerpt>`, gradient-header HTML body matching the other Pulse emails, plain
  text fallback, `replyTo` set to the sharer so a recipient reply lands with
  the right person.
- Share route `app/api/analyzer/analyses/[id]/share/route.ts` now resolves
  the sharer's session, builds the analysis URL from `BETTER_AUTH_URL`
  (falling back to `NEXT_PUBLIC_BETTER_AUTH_URL`, then `localhost:3100`),
  persists the audit row, then attempts the email send.
- Share-modal frontend handles the new `emailSent`/`emailError` fields:
  success → green toast; row-saved-but-send-failed → orange `toast.warning`
  carrying the SMTP error.

**Decisions worth flagging**

- **Audit row persists even if email send fails.** The `analyzer_shares`
  row is the audit log, not just a delivery receipt. SMTP outages should
  not erase the record that the user attempted a share.
- **Response is HTTP 200 on email failure.** The route returns
  `{share, emailSent: false, emailError}`. A non-2xx would imply the share
  itself failed; surfacing `emailSent: false` is the more honest signal.
- **No `email_sent_at` column added.** Adding it requires a migration and
  a way to retry — neither is asked for by the spec. Send state lives only
  in the response and the server log line `[ANALYZER-SHARE] email send
  failed for share <id>`. If retries become a real need, a dedicated
  `analyzer_share_email_attempts` table is the natural shape.
- **Used existing nodemailer SMTP, not Graph sendMail.** Spec said "via
  M365 Graph using existing wulf-pulse mail integration if one exists;
  otherwise use a new module" — `email.ts` *is* the existing module. The
  Graph integration in this codebase is read-only (mailbox search, user
  reports), not send-capable.
- **HTML escaping is hand-rolled.** Five-replace function for
  amp/lt/gt/quote/apos. The user-controlled fields (sender name, note,
  analysis summary, next step) all flow through it. The repo has no HTML
  escaper utility and the magic-link/invitation emails don't need one
  because their inputs are URLs and admin-set names.

**Deliberately left out**

- **No retries.** A failed send is logged once and surfaced to the user.
  They can re-share if they want — that creates a fresh audit row, which
  is correct behavior.
- **No tests.** `lib/services/email.ts` has no existing tests, mocking
  nodemailer fully would add a non-trivial test scaffold for one
  function, and the share route is route-handler thin. Consistent with
  phases 5/6.
- **No `viewed_at` tracking yet.** The migration has the column but
  nothing writes to it. A `/share/:id/viewed` endpoint with an
  unguessable token would be the smallest addition; spec didn't ask, so
  skipped.

---

## Phase 8 — Operator runbook

**Delivered**

- `docs/wulf-pulse-ticket-analyzer-runbook.md` — cost monitoring queries
  (daily spend, top expensive analyses, multi-run tickets), the cost
  ceiling explanation (`COST_CEILING_USD = 2.00` in `pipeline.ts`), the
  IT Glue alias workflow (where to find org IDs, how to verify via
  `model_traces.itglue.alias_used`), failure triage tables (failed jobs
  vs. `needs_human_review` flagged completes, with the actual error
  strings the worker emits), and a manual-ops section for queueing jobs
  from psql.
- README.md "Documentation" section links the runbook so operators
  finding the project for the first time discover it.

**Decisions worth flagging**

- **Runbook lives in `docs/`, not the README.** Per CLAUDE.md, long-form
  per-feature docs go in `docs/` and the README points to them. Matches
  the existing pattern (workflow-editor-guide, sync guides, etc.).
- **Manual migration step is documented.** Postgres only re-applies
  migrations on first init — operators applying analyzer to an existing
  DB need to run `psql -f migrations/069_*.sql`. The runbook leads with
  this.
- **Listed what's *not* implemented.** Retries, `viewed_at`, share
  `email_sent_at`, pre-Sonnet cost gate, share-recipient autocomplete.
  Better to advertise the gaps than have an operator trip over them.

**Deliberately left out**

- No production-monitoring dashboard (Grafana etc.) — Pulse doesn't have
  one for any other feature, and the SQL queries cover the same ground.
- No README expansion for the analyzer feature itself. Pulse's README is
  intentionally short; the runbook + spec + build notes are the deep
  docs.

---

## Status after each phase

| Phase | Tests | tsc | Notes |
|---|---|---|---|
| 1 | 0 | clean | migration + types only |
| 2 | 68 | clean | redaction + preprocessor regression |
| 3 | 100 | clean | + LLM scaffolding + Stage 1 |
| 4 | 128 | clean | + pipeline + worker |
| 5 | 128 | clean | API routes (no route tests) |
| 6 | 128 | clean | frontend (no FE tests) |
| 7 | 128 | clean | share email via existing SMTP transport |
| 8 | 128 | clean | operator runbook + README link |