docs(analyzer): phase 8 — operator runbook

docs/wulf-pulse-ticket-analyzer-runbook.md covers cost monitoring
queries, the $2 cost ceiling, IT Glue alias workflow, failure triage
(failed jobs vs needs_human_review), and manual ops (queue from psql,
force re-run, inspect model_traces). Calls out the manual migration
step for existing DBs and lists the unimplemented surfaces (no auto
retries, no viewed_at, no email_sent_at) so operators don't trip on
them. Linked from README.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

README.md

@@ -74,6 +74,7 @@ See `AUTOTASK_API_GUIDE.md` and `ADDIGY_API_GUIDE.md` for credential setup.
 - **`POSTGRES_SYNC_SETUP.md`** — database initialization
 - **`DOCKER_README.md`** — Docker workflow
 - **`PULSE_DATABASE_SKILL.md`** — diagnostic SQL queries
+- **`docs/wulf-pulse-ticket-analyzer-runbook.md`** — AI ticket analyzer operator runbook (cost monitoring, IT Glue alias map, failure triage)
 - **`docs/`** — sync behavior, webhook setup, workflow editor, per-integration guides
 
 ## Deployment

docs/wulf-pulse-ticket-analyzer-build-notes.md

@@ -376,6 +376,44 @@ This file is updated after each phase ships.
 
 ---
 
+## Phase 8 — Operator runbook
+
+**Delivered**
+
+- `docs/wulf-pulse-ticket-analyzer-runbook.md` — cost monitoring queries
+  (daily spend, top expensive analyses, multi-run tickets), the cost
+  ceiling explanation (`COST_CEILING_USD = 2.00` in `pipeline.ts`), the
+  IT Glue alias workflow (where to find org IDs, how to verify via
+  `model_traces.itglue.alias_used`), failure triage tables (failed jobs
+  vs. `needs_human_review` flagged completes, with the actual error
+  strings the worker emits), and a manual-ops section for queueing jobs
+  from psql.
+- README.md "Documentation" section links the runbook so operators
+  finding the project for the first time discover it.
+
+**Decisions worth flagging**
+
+- **Runbook lives in `docs/`, not the README.** Per CLAUDE.md, long-form
+  per-feature docs go in `docs/` and the README points to them. Matches
+  the existing pattern (workflow-editor-guide, sync guides, etc.).
+- **Manual migration step is documented.** Postgres only re-applies
+  migrations on first init — operators applying analyzer to an existing
+  DB need to run `psql -f migrations/069_*.sql`. The runbook leads with
+  this.
+- **Listed what's *not* implemented.** Retries, `viewed_at`, share
+  `email_sent_at`, pre-Sonnet cost gate, share-recipient autocomplete.
+  Better to advertise the gaps than have an operator trip over them.
+
+**Deliberately left out**
+
+- No production-monitoring dashboard (Grafana etc.) — Pulse doesn't have
+  one for any other feature, and the SQL queries cover the same ground.
+- No README expansion for the analyzer feature itself. Pulse's README is
+  intentionally short; the runbook + spec + build notes are the deep
+  docs.
+
+---
+
 ## Status after each phase
 
 | Phase | Tests | tsc | Notes |
@@ -387,3 +425,4 @@ This file is updated after each phase ships.
 | 5 | 128 | clean | API routes (no route tests) |
 | 6 | 128 | clean | frontend (no FE tests) |
 | 7 | 128 | clean | share email via existing SMTP transport |
+| 8 | 128 | clean | operator runbook + README link |

docs/wulf-pulse-ticket-analyzer-runbook.md

@@ -0,0 +1,318 @@
+# AI Ticket Analyzer — Operator Runbook
+
+How to keep the analyzer healthy in production. Audience is whoever is on
+call for Pulse. The feature spec is in
+`wulf-pulse-ticket-analyzer-prompt.md`; the implementation log is in
+`wulf-pulse-ticket-analyzer-build-notes.md`.
+
+---
+
+## At a glance
+
+- **Pipeline**: Haiku triage → (optional) IT Glue retrieval → Sonnet deep
+  analysis → (optional) Opus deep reasoning → persistence.
+- **Trigger**: user clicks "Analyze" on a ticket page; that POSTs to
+  `/api/analyzer/tickets/:ticketNumber/analyze`. The route preprocesses
+  inline, short-circuits to the cached row when content hasn't changed,
+  otherwise queues a job for the worker.
+- **Worker**: `analyzerWorker` polls `analyzer_jobs` every 2 s in-process.
+  Auto-starts in production (`NODE_ENV=production`). Opt in for dev with
+  `ANALYZER_WORKER_AUTOSTART=1`.
+- **Cost ceiling**: hard-coded `COST_CEILING_USD = 2.00` in
+  `lib/services/analyzer/pipeline.ts`. Trips before Opus only — Sonnet runs
+  unconditionally. When tripped, the analysis is flagged
+  `needs_human_review = true` with reason "cost ceiling reached".
+
+---
+
+## Required environment
+
+Add to `~/projects_env/wulf-pulse.env` (do not commit values):
+
+```
+ANTHROPIC_API_KEY=
+ITGLUE_API_KEY=
+ITGLUE_API_BASE=https://api.itglue.com
+ALLOWED_SHARE_DOMAINS=wulfconsulting.com
+# Optional in dev:
+ANALYZER_WORKER_AUTOSTART=1
+```
+
+`SMTP_*` and `BETTER_AUTH_URL` are already set elsewhere in Pulse — share
+emails reuse the existing magic-link transport.
+
+---
+
+## First-time setup on an existing database
+
+Migration `069_create_analyzer_tables.sql` creates `analyzer_analyses`,
+`analyzer_shares`, and `analyzer_jobs`. Postgres only re-applies migrations
+on a fresh data volume — for an existing DB, run the migration directly:
+
+```bash
+psql "$DATABASE_URL" -f migrations/069_create_analyzer_tables.sql
+```
+
+The script is idempotent (`IF NOT EXISTS` on every object).
+
+---
+
+## Monitoring cost
+
+Every completed analysis stores `estimated_cost_usd`, `total_input_tokens`,
+`total_output_tokens`, and per-stage tier flags
+(`haiku_used` / `sonnet_used` / `opus_used`). The number is a forward
+estimate from `lib/services/llm/pricing.ts` and will drift if Anthropic
+changes published rates — re-check that file before believing big spend
+numbers.
+
+### Daily spend (last 14 days)
+
+```sql
+SELECT date_trunc('day', triggered_at) AS day,
+       count(*) FILTER (WHERE status = 'complete')        AS analyses,
+       count(*) FILTER (WHERE opus_used)                  AS opus_runs,
+       count(*) FILTER (WHERE needs_human_review)         AS needs_review,
+       round(sum(estimated_cost_usd)::numeric, 2)         AS spend_usd
+  FROM analyzer_analyses
+ WHERE triggered_at > now() - interval '14 days'
+ GROUP BY 1
+ ORDER BY 1 DESC;
+```
+
+### Top 20 most expensive analyses (last 30 days)
+
+```sql
+SELECT id, ticket_number, analysis_version, estimated_cost_usd,
+       opus_used, needs_human_review, triggered_at
+  FROM analyzer_analyses
+ WHERE triggered_at > now() - interval '30 days'
+ ORDER BY estimated_cost_usd DESC
+ LIMIT 20;
+```
+
+### Heaviest tickets (multiple re-runs against the same ticket)
+
+```sql
+SELECT ticket_number,
+       count(*)                              AS analyses,
+       round(sum(estimated_cost_usd)::numeric, 2) AS spend_usd,
+       max(triggered_at)                     AS last_run
+  FROM analyzer_analyses
+ GROUP BY 1
+HAVING count(*) > 1
+ ORDER BY spend_usd DESC
+ LIMIT 20;
+```
+
+### Cost ceiling — what to do when it trips
+
+`needs_human_review = true` with `human_review_reasons` containing "cost
+ceiling reached" means Sonnet ran but Opus was skipped. The Sonnet output
+is still persisted and is usually fine. To re-run with Opus forced on,
+trigger a new analysis (the route accepts `force=true`); the cost ceiling
+will reapply, so check the prior `estimated_cost_usd` before committing.
+To raise the ceiling, edit `COST_CEILING_USD` in
+`lib/services/analyzer/pipeline.ts` and redeploy.
+
+To reduce spend without code changes: discourage operators from clicking
+Re-analyze unless content has changed (the idempotency short-circuit
+already prevents charged re-runs against the same `content_hash`).
+
+---
+
+## Adding IT Glue org aliases
+
+Stage 2 resolves an Autotask company name to an IT Glue organization in
+two steps:
+
+1. **Alias map** — `lib/services/analyzer/itglue-aliases.json`. Keys are
+   normalized company names (lowercase, single-spaced); values are IT Glue
+   organization IDs as **strings**. Exact-match wins immediately.
+2. **Live IT Glue search** — fallback when the alias map misses.
+
+When the live search misses or returns the wrong org, add an alias.
+
+### Finding the IT Glue organization ID
+
+In the IT Glue UI, navigate to the organization. The numeric segment of
+the URL (`/organizations/12345`) is the ID. Or via API:
+
+```bash
+curl -s -H "x-api-key: $ITGLUE_API_KEY" \
+  "$ITGLUE_API_BASE/organizations?filter[name]=Seubert%20and%20Associates" \
+  | jq '.data[] | {id, name: .attributes.name}'
+```
+
+### Updating the alias file
+
+Edit `lib/services/analyzer/itglue-aliases.json`. Keys must be **already
+normalized** — lowercase, single-spaced — because the matcher applies the
+same normalization to the Autotask company name and looks up an exact key.
+
+```json
+{
+  "seubert": "1234567",
+  "seubert and associates": "1234567",
+  "acme insurance group": "7654321"
+}
+```
+
+Strip the `_comment` / `_example` keys when adding the first real entry,
+or leave them — they're ignored at match time (no real org name normalizes
+to a leading underscore).
+
+Changes require a redeploy. The file is bundled into the build by Next's
+`import` — there is no runtime reload.
+
+### Verifying an alias works
+
+After deploy, run an analysis on a ticket from that company. Check the
+`model_traces.itglue` field on the resulting row:
+
+```sql
+SELECT model_traces->'itglue'
+  FROM analyzer_analyses
+ WHERE id = '<analysis-id>';
+```
+
+`alias_used: true` confirms the alias hit. `org_id` should be the value
+you put in the JSON.
+
+---
+
+## Triaging failed analyses
+
+Two failure surfaces to know about:
+
+### 1. Failed jobs (`analyzer_jobs.status = 'failed'`)
+
+The job hit an exception before the analysis row was written. The error
+message is in `analyzer_jobs.error_message`. Common causes:
+
+| Error message | Cause | Fix |
+|---|---|---|
+| `Ticket T... not found in local mirror — confirm sync is current.` | Autotask sync hasn't pulled the ticket yet. | Wait for the next sync, or trigger a manual sync from `/admin`. The webhook usually fills the gap; if not, see `scripts/backfill-ticket-notes-gap.ts` for the reconciliation pattern. |
+| `LLM stage on claude-... failed twice` | Two Anthropic call attempts both failed (network / parse / 5xx). | Check the API key is valid (`curl` Anthropic), and check Anthropic status. Re-queue by clicking Re-analyze. |
+| `Anthropic API error: 429 ...` | Rate limit. | Wait and re-queue. |
+| `IT Glue search failed: ...` should NOT appear here — the pipeline tolerates IT Glue failures and continues without context. | If you see one, the failure was outside the catch block and is a bug. |
+
+```sql
+-- Recent failures
+SELECT id, ticket_number, error_message, queued_at, finished_at
+  FROM analyzer_jobs
+ WHERE status = 'failed'
+   AND queued_at > now() - interval '7 days'
+ ORDER BY queued_at DESC;
+```
+
+To retry a failed job, queue a fresh one — there is no in-place retry.
+Either click Re-analyze in the UI or:
+
+```sql
+INSERT INTO analyzer_jobs (ticket_number, queued_by_user_id)
+SELECT ticket_number, queued_by_user_id
+  FROM analyzer_jobs
+ WHERE id = '<failed-job-id>';
+```
+
+### 2. Completed but flagged for human review (`needs_human_review = true`)
+
+The pipeline finished and produced an analysis, but the model itself
+reported low confidence or the cost circuit breaker tripped before Opus.
+`human_review_reasons` is a JSONB array of short strings — read it.
+
+The UI exposes this queue at `/analyzer/queue`. SQL equivalent:
+
+```sql
+SELECT id, ticket_number, analysis_version,
+       confidence_score, estimated_cost_usd,
+       human_review_reasons
+  FROM analyzer_analyses
+ WHERE needs_human_review = true
+   AND status = 'complete'
+ ORDER BY triggered_at DESC
+ LIMIT 50;
+```
+
+Reasons you'll see:
+
+- `"cost ceiling reached"` — Opus skipped to save money. Sonnet output is
+  what you have. Re-run with `force=true` to override (and accept the
+  spend).
+- `"low confidence: <score>"` — the model itself reports uncertainty. The
+  Sonnet output may still be useful as a starting point; treat as
+  guidance, not verdict.
+- A model-emitted reason from Stage 3 or Stage 4. The full string is in
+  the JSONB — don't truncate before reading.
+
+### 3. Worker not picking up jobs
+
+If `analyzer_jobs` has rows in `status='queued'` for more than ~30s, the
+worker is wedged or not running.
+
+- Check the application log for `[ANALYZER-WORKER] starting` at boot. If
+  missing in production, `NODE_ENV` may not be set to `production` —
+  bring up `ANALYZER_WORKER_AUTOSTART=1` as a workaround.
+- The worker is in-process; restarting the Pulse app restarts it.
+- Concurrency: multiple replicas all run a worker, but `claimQueuedJob`
+  uses `FOR UPDATE SKIP LOCKED` so each job is processed exactly once.
+
+---
+
+## Manual ops
+
+### Trigger an analysis from psql
+
+```sql
+INSERT INTO analyzer_jobs (ticket_number, queued_by_user_id)
+VALUES ('T20260424.0045', NULL);
+```
+
+The worker will pick it up within 2 seconds. `queued_by_user_id` may be
+null (`ON DELETE SET NULL`).
+
+### Force a re-run despite a cached analysis
+
+The `force=true` flag on `POST /api/analyzer/tickets/:ticketNumber/analyze`
+bypasses the idempotency short-circuit. The Re-analyze button in the UI
+sends this flag.
+
+### Inspect raw stage outputs for a debugging session
+
+```sql
+SELECT model_traces
+  FROM analyzer_analyses
+ WHERE id = '<analysis-id>';
+```
+
+Per-stage payload includes input/output tokens, attempt count, full parsed
+model responses, and IT Glue resolution metadata. Don't paste this into
+external systems — it can include redacted-but-still-sensitive snippets
+of customer data.
+
+### Ad hoc: how much have we spent this month?
+
+```sql
+SELECT round(sum(estimated_cost_usd)::numeric, 2) AS spend_usd,
+       count(*)                                   AS analyses
+  FROM analyzer_analyses
+ WHERE triggered_at >= date_trunc('month', now());
+```
+
+---
+
+## Things that are not implemented yet
+
+- No automated retries on failed jobs.
+- No `viewed_at` tracking on share rows (column exists, nothing writes
+  to it).
+- No `email_sent_at` column — share email send state lives only in the
+  HTTP response and the server log.
+- No pre-Sonnet cost gate. The circuit breaker only protects against
+  Opus.
+- No autocomplete on the share-recipient field.
+
+If any of these become a real operational problem, file the work — the
+stubs are intentional and called out in
+`wulf-pulse-ticket-analyzer-build-notes.md`.