feat(analyzer): Phase 2 — full stage persistence, fingerprints, aggregate reports, cost guards

Eight sub-phases per docs/ticket-analyzer-phase2-spec.md:

2.1 Schema (migration 070): analyzer_stage_executions table; source_snapshot,
    aggregate_fingerprint, fingerprint_generated_at columns on analyzer_analyses.
    model_traces marked LEGACY (kept for back-compat).
2.2 Every pipeline stage records a row to analyzer_stage_executions, success
    or failure. Worker persists a status='failed' analyzer_analyses row when
    the pipeline throws so partial stage records have a parent. Pipeline
    exposes raw triage/sonnet/opus responses for downstream stages.
2.3 Stage 3 prompt updated with markdown formatting rules + banned filler
    phrases. Added react-markdown + remark-gfm + @tailwindcss/typography.
    New <AnalysisMarkdown> component replaces <ProseText>; coerces stray
    headers to bold paragraphs.
2.4 Stage 6 fingerprint (Haiku) runs after persistence, failure-tolerant.
    scripts/backfill-fingerprints.ts reconstructs Stage 6 input from the
    legacy model_traces blob.
2.5 Browse UI rebuild at /analyzer/tickets: multi-select for client/issue/
    queue/status/priority/assignee, sticky filter bar, active-filter chips,
    bulk selection persisted via localStorage, "Analyze N selected" +
    "Generate aggregate report" actions. New <MultiSelect> primitive.
    Staleness uses last_activity_date > completed_at heuristic per spec C.1.
2.6 Aggregate reports (migration 071): runner is fire-and-forget, persists
    SQL distributions immediately so UI shows partial state during the
    Sonnet reduce call. Three endpoints, three pages (/analyzer/reports[/new
    /:id]). IT Glue context fetcher capped at 200 doc titles.
2.7 Cost guards (migration 072): per-request $5 confirmation, soft-warn at
    $20/day, hard-block at $50/day with ANALYZER_DAILY_COST_OVERRIDE_USERS
    override. Every gating decision audited.
2.8 Runbook + build notes updated.

128 vitest tests passing, tsc clean. Migrations 070/071/072 idempotent
(IF NOT EXISTS). model_traces double-write retained — drop in a future
migration once aggregate reports have soaked.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
lorentz 2026-04-29 14:00:22 -04:00
parent b20c94ea1a
commit bd3401df1c
33 changed files with 7132 additions and 554 deletions

View file

@ -316,3 +316,172 @@ SELECT round(sum(estimated_cost_usd)::numeric, 2) AS spend_usd,
If any of these become a real operational problem, file the work — the
stubs are intentional and called out in
`wulf-pulse-ticket-analyzer-build-notes.md`.
---
## Phase 2 additions
### Stage execution history (per-analysis forensics)
Migration 070 introduced `analyzer_stage_executions`. Every stage of the
pipeline (preprocess, triage, itglue, analyze, deep_review, fingerprint)
now writes a row including the input it saw and the output it produced.
Failed stages get a row with `error_message` populated.
Read it like this:
```sql
-- Full per-stage trace for one analysis
SELECT stage_order, stage, model_id,
input_tokens, output_tokens, latency_ms,
error_message
FROM analyzer_stage_executions
WHERE analysis_id = '<analysis-id>'
ORDER BY stage_order;
-- Inspect a single stage's full input/output
SELECT input_payload, output_payload
FROM analyzer_stage_executions
WHERE analysis_id = '<analysis-id>' AND stage = 'analyze';
```
The legacy `analyzer_analyses.model_traces` JSONB column is preserved for
back-compat. New analyses double-write to both. A future migration will
drop `model_traces` once aggregate reports have soaked.
### Fingerprint stage (Stage 6)
After persistence, the worker runs Haiku-tier fingerprint extraction and
writes the result to `analyzer_analyses.aggregate_fingerprint`. Failure
is non-fatal — the analysis row stays usable, fingerprint stays NULL.
Re-fingerprint on demand:
```sql
SELECT id, ticket_number, analysis_version
FROM analyzer_analyses
WHERE aggregate_fingerprint IS NULL
AND status = 'complete'
ORDER BY triggered_at ASC;
```
Or run the backfill script (idempotent — the SQL filter skips already-fingerprinted rows):
```bash
npx tsx scripts/backfill-fingerprints.ts # process all missing
npx tsx scripts/backfill-fingerprints.ts --limit=50 # cap work
npx tsx scripts/backfill-fingerprints.ts --dry-run # show what would run
```
The script reconstructs the Stage 6 input from
`analyzer_analyses.model_traces.{triage_response, sonnet_response, opus_response}`.
If a row's model_traces is missing those fields (very old format) the script
logs `[skip]` for it and moves on.
Cost: Haiku at ~$0.001 per fingerprint. 1000 analyses ≈ $1.
### Aggregate reports
Migration 071 added `analyzer_aggregate_reports` and relaxed
`analyzer_stage_executions.analysis_id` to be nullable; rows now carry
either an `analysis_id` or an `aggregate_report_id` (CHECK enforces
exactly one).
Workflow:
1. User selects tickets on `/analyzer/tickets` and clicks
**Generate aggregate report**.
2. POST `/api/analyzer/aggregate-reports` validates: ≤100 tickets, all
have a fingerprint, none are stale. Inserts a 'pending' row, fires
the runner via `void runAggregateReport(id)`, returns immediately.
3. Runner does:
- Loads fingerprints
- Computes SQL distributions (categories, clients, root cause,
resolution path) and writes them to the row immediately
- Fetches IT Glue doc titles for affected clients (if requested)
- Calls Sonnet (Opus opt-in) with a structured payload
- Writes the LLM-derived fields, marks `status='complete'`
4. UI polls `GET /api/analyzer/aggregate-reports/:id` every 3s while
pending/running.
Inspect a report's sub-stage trace (linked via `aggregate_report_id`):
```sql
SELECT stage_order, stage, model_id, latency_ms, error_message
FROM analyzer_stage_executions
WHERE aggregate_report_id = '<report-id>'
ORDER BY stage_order;
```
Daily spend on aggregate reports:
```sql
SELECT date_trunc('day', generated_at) AS day,
count(*) AS reports,
round(sum(estimated_cost_usd)::numeric, 2) AS spend_usd
FROM analyzer_aggregate_reports
WHERE generated_at > now() - interval '14 days'
GROUP BY 1
ORDER BY 1 DESC;
```
### Cost guards (Phase 2.7)
Three thresholds enforce per-user daily spend:
- **$5 per request** — a single aggregate report estimated above $5
triggers a confirmation modal. Frontend re-POSTs with
`confirmedCost: true` if the user clicks through.
- **$20 per user/day** — soft warn. Surfaced via `softWarn:true` in the
cost evaluation; no enforcement.
- **$50 per user/day** — hard block. POST returns 403 unless the user
is in `ANALYZER_DAILY_COST_OVERRIDE_USERS` (comma-separated list of
Better Auth user ids).
Daily spend is computed as the trailing 24h sum of
`analyzer_analyses.estimated_cost_usd` + `analyzer_aggregate_reports.estimated_cost_usd`
for the user.
Every gating decision writes a row to `analyzer_cost_audit`:
```sql
SELECT created_at, user_id, action, estimated_cost,
daily_spend_before, decision, decision_reason
FROM analyzer_cost_audit
ORDER BY created_at DESC
LIMIT 50;
```
To grant a user override capability:
```bash
# in ~/projects_env/wulf-pulse.env
ANALYZER_DAILY_COST_OVERRIDE_USERS=user_id_1,user_id_2
```
Restart `pulse-app` to pick up the change.
### Browse / filter ticket list
`/analyzer/tickets` rebuilt as the primary entry point in Phase 2.5:
- Multi-select for client, issue type, queue, status, priority, assignee
- Sticky filter bar with period chips (today/yesterday/this+last week,
30d/60d, custom range, all time)
- Active-filter chips below the bar; click to remove
- Bulk row selection persisted in localStorage
(`analyzer:ticket-selection:v1`) — survives pagination but is
session-scoped (no user id baked in)
- Bulk "Analyze N selected" sequentially queues jobs for each ticket
(forces re-analyze on stale; analyzes from scratch on un-analyzed)
- Bulk "Generate aggregate report" routes selected tickets to
`/analyzer/reports/new`; only enabled when all selected are
`analyzedState === 'current'`
**Staleness heuristic**: a ticket's `analyzedState` is computed as
`stale` when `tickets.last_activity_date > latest_analysis.completed_at`.
The Phase 2 spec calls for content-hash-based comparison; that requires
either caching the current hash on the tickets row (sync change) or
computing it on read for the visible page (slow). The date heuristic
gets ~95% of the value at zero compute cost; revisit when there's real
load signal.