# Feature: IT Glue Asset Audit & Documentation Write-back **Status:** shipped (Phases 4 + 4.1 of the AI Ticket Analyzer) **Migrations:** `075_itglue_audit.sql`, `076_itglue_ticket_xrefs.sql` **Build notes:** `docs/wulf-pulse-ticket-analyzer-build-notes.md` → Phase 4 + 4.1 **Operator runbook:** `docs/wulf-pulse-ticket-analyzer-runbook.md` → "IT Glue asset audits" --- ## What it is, in one paragraph For each IT Glue **flexible-asset Application record** *and* **Configuration record** (server, workstation, network device), Pulse runs an LLM audit that compares the record's current contents against (a) the field schema with hints, (b) other well-filled records at the same client, (c) recent ticket history that mentions the asset. It surfaces **field gaps** ("Wulf Application Champion is empty — Jake Hammel is the de-facto SME per T20260502.0033"), **note promotions** (free text in Notes that belongs in a structured field), and **contradictions** (Notes say "2-3 VMs" but Application-on-Device tags only 1). Admins can apply a suggestion with one click — Pulse PATCHes IT Glue and records every change with full before/after diff and revert capability. Two complementary entry points: - **Asset-first** (Phase 4) — admin browses lowest-scoring Application or Configuration records and runs an audit against all-time history. Useful for periodic backlog sweeps. - **Ticket-first** (Phase 4.1) — every analyzed ticket gets a "Check IT Glue documentation" button on its analysis page. Click → matched IT Glue records appear → audit per record uses *just this ticket's evidence*. Findings frame as "what did this ticket teach us that the documentation doesn't say?" Drives a forward-only **cross-reference index** (`itglue_ticket_xrefs`) of which tickets referenced or updated which documentation, which is also the lookup index for a future RAG automation. --- ## Why we built it The analyzer already extracts `documentation_gaps_observed` per ticket (Phase 2.4 fingerprint). Until Phase 4 nothing acted on the signal. The catalyzing example was T20260502.0033 (Hynes — tags not printing from Simple Shop Floor), resolved by Collin + Jake identifying a stopped Windows service on MISYS-SQL processing BarTender scan-folder text files. The IT Glue Application record `MISYS 6.3` had **5/17 fields filled**, missing: - Wulf Application Champion (Jake is the SME, but only by tribal knowledge) - Vendor Maintenance/Support (escalation contact "Steve Cianflone" buried in free-text Notes) - The integration architecture (Simple Shop Floor → MISYS-SQL → BarTender scan folder) - The named services on each VM The next tech who hits this issue would re-discover everything. Phase 4 turns the analyzer's passive "documentation_gaps_observed" output into an actionable backlog with a write-back path. --- ## User-facing flows ### List view — `/analyzer/itglue/applications` Shows every Application record across all clients, sorted by **lowest audit score first** (so unaudited and worst-scored assets bubble up). Each row: asset name, client, populated-field count, last-audit timestamp + provider, score badge. ``` ┌────────────────────────────────────────────────────────────────────┐ │ MISYS 6.3 ▌ 55% │ │ Hynes Industries · 5 fields populated · last audited today (Claude)│ └────────────────────────────────────────────────────────────────────┘ ``` ### Detail view — `/analyzer/itglue/applications/[id]` Three regions: 1. **Header** — asset name, client, score badge, "Open in IT Glue" link, the `` (Claude vs DeepSeek), and a primary **Run audit** button. 2. **Audit findings** (visible after a run) — - **Field gaps** — severity-toned cards (red/amber/blue by confidence), each with: field name, why-it-matters, suggested value (when the LLM has evidence), evidence ticket links, and an **Apply** button. - **Promote from Notes** — quoted substring → target field → suggested structured value, with **Apply**. - **Contradictions** — pure observations (no Apply, by design). 3. **Current fields** — every field rendered in IT Glue's order; populated shown normally, empty shown muted with the field's hint inline as guidance. 4. **Write history** — every Apply/Revert for this asset, with inline before/after diffs and a **Revert** button on committed writes. 5. **Audit history** — score over time across runs. ### Admin view — `/admin/itglue-writes` Cross-asset write log, status filters (pending / committed / failed / reverted), full diff per row, ordered most-recent-first. --- ## Permissions | Action | Permission | |---|---| | Read audit / View asset detail | `requireAuth()` (any signed-in user) | | Run a fresh audit | `requireAuth()` | | Apply a suggestion (PATCH IT Glue) | `requirePermission('itglue', 'write')` | | Revert a previous write | `requirePermission('itglue', 'write')` | | `/admin/itglue-writes` cross-asset feed | `requirePermission('admin', 'access')` | `itglue.write` is granted to `admin` and `super-admin` roles only (see `lib/permissions.ts`). The Apply / Revert buttons render on the page for non-admins but are **disabled** with a tooltip explaining why. --- ## Pipeline Single LLM call, provider-aware via `stageModelsFor(provider).deep_analysis` (Claude Sonnet for Anthropic; DeepSeek V4 Pro for OpenRouter). ### Inputs (six context arms) 1. **Asset snapshot** — the asset's current `traits` JSONB, redacted via `lib/services/analyzer/itglue-redact.ts` to strip any password/secret/ key/token/credential-keyed values. 2. **Field schema with hints** — every field on the asset type, in IT Glue's display order, with the `hint` text IT Glue surfaces in its own editor (e.g. *"Tag any application servers or Devices it's installed on"*). 3. **Peer exemplars from same client** — top 5 most-filled flexible assets of the same type at the same client (ranked by populated trait keys), redacted. 4. **Best-in-class peers across all clients** — top 3 most-filled instances of the same asset type globally, redacted. 5. **Per-field fill-rate stats** — the % of (a) this client's assets and (b) all assets of this type that have field X populated. Pure SQL aggregate; no token cost. Lets the LLM rank gaps by "unusual to be missing" vs "usually missing anyway." 6. **Ticket evidence** — up to 20 most-recent complete analyses for the client whose summary or fingerprint mentions the asset name, with their `aggregate_fingerprint` payload included in full. ### Output schema (`AssetAuditResponse` in `lib/types/analyzer.ts`) ```ts { field_gaps: Array<{ field_name: string, // must exist on the asset type why_missing_matters: string, suggested_value: string | null, // null if no concrete evidence evidence_ticket_numbers: string[], confidence: 'high' | 'medium' | 'low', }>, notes_promotions: Array<{ quoted_note_text: string, // exact substring of Notes target_field: string, suggested_value: string, confidence: 'high' | 'medium' | 'low', }>, contradictions: Array<{ description: string, evidence: string, }>, overall_score: number, // 0..1 self-rated completeness } ``` ### System-prompt rules (highlights) - Never invent a field name not present in the schema. - Never suggest a value you cannot point to evidence for — use `null` instead. - Don't suggest password/secret/key/token/credential-shaped fields. (The Apply endpoint also blocks these defensively.) - Use fill-rate stats: a gap that's empty here but populated >80% elsewhere is a stronger gap than one that's empty 80% of the time globally. ### Pipeline files - `lib/services/analyzer/asset-audit/data-builder.ts` — collects all six inputs. - `lib/services/analyzer/asset-audit/prompt.ts` — system prompt + payload builder (with 80KB cap; trims peer_global first, oldest tickets next). - `lib/services/analyzer/asset-audit/runner.ts` — single `callLLMStage` call, persistence on success or failure. - `lib/services/analyzer/asset-audit/persistence.ts` — typed read/write of both DB tables. --- ## Data model ### `itglue_asset_audits` (one row per audit run) | Column | Notes | |---|---| | `id` UUID | PK | | `asset_type` TEXT | Currently `'flexible_asset'` only | | `asset_id` BIGINT | IT Glue resource id | | `asset_type_id`, `organization_id` BIGINT | Denormalized for fast filters | | `generated_by_user_id` TEXT | FK `user(id)`, nullable on user delete | | `generated_at` TIMESTAMPTZ | | | `provider`, `model_used` | Which LLM produced the analysis | | `asset_snapshot` JSONB | Redacted traits at audit time | | `ticket_count` INT | How many fingerprints fed in | | `field_gaps`, `notes_promotions`, `contradictions` JSONB | LLM output | | `overall_score` NUMERIC(3,2) | | | `estimated_cost_usd`, `total_input_tokens`, `total_output_tokens` | Cost telemetry | | `status` | `pending` / `running` / `complete` / `failed` | | `error_message` TEXT | | ### `itglue_writes` (one row per write attempt) | Column | Notes | |---|---| | `id` UUID | PK | | `audit_id` UUID | FK to the audit that prompted the change (nullable) | | `asset_type`, `asset_id` | What was written | | `field_name` TEXT | Human field name (mapped to trait key on apply) | | `before_value`, `after_value` JSONB | Pre/post diff | | `performed_by_user_id` TEXT | FK `user(id)` | | `performed_at` TIMESTAMPTZ | | | `status` | `pending` → `committed` / `failed` / `reverted` | | `itglue_response` JSONB | Raw API response for forensics | | `error_message` TEXT | | | `source_evidence` JSONB | `{ ticket_numbers, gap_description }` or `{ reverts_write_id }` | ### Generic `audit_log` (existing — also written for every change) `audit.log()` is called on every successful Apply/Revert with action `itglue.write` or `itglue.revert`, resource `flexible_asset`, resourceId = asset_id, and details = `{ field_name, before, after, audit_id }`. Surfaces in `/admin/audit-log` next to every other admin action. --- ## API contract All routes under `/api/analyzer/itglue/`. ### `GET /applications` List all Application records joined to their latest audit. Returns: `{ applications: [{ id, name, organizationId, organizationName, traitCount, latestAudit }] }`. ### `GET /applications/[id]` Asset detail + field schema for rendering. Returns: `{ asset, fields }`. ### `GET /applications/[id]/audit?history=1` Latest audit (and optionally history of last 20). Returns: `{ audit: AssetAuditRow | null, history?: AssetAuditRow[] }`. ### `POST /applications/[id]/audit` Body: `{ provider?: 'anthropic' | 'openrouter' }`. Runs a fresh audit. Cost-guard records `action='itglue_audit'` in `analyzer_cost_audit`. Returns: `{ audit: AssetAuditRow }`. ### `POST /applications/[id]/apply` *(admin)* Body: `{ auditId, fieldName, suggestedValue, sourceEvidence? }`. Inserts pending `itglue_writes` row → calls `updateFlexibleAsset` on the IT Glue client → marks committed/failed → refreshes the local mirror → writes generic `audit_log`. Returns: `{ writeId, status: 'committed', asset }` or 502 on IT Glue error. ### `POST /applications/[id]/revert/[writeId]` *(admin)* Re-applies the original `before_value`. Inserts a new write row with swapped before/after; original row → `status='reverted'`. Returns: `{ writeId, revertedWriteId, status: 'committed', asset }`. ### `GET /applications/[id]/writes` Per-asset write history. Returns: `{ writes: AssetWriteRow[] }`. ### `GET /writes` *(admin)* Cross-asset write log with status filter. Query: `?status=committed&limit=100&offset=0`. Returns: `{ writes: AssetWriteRow[] }`. --- ## Safeguards ### Three-layer credential refusal 1. **Prompt** — system prompt instructs the LLM not to suggest password, secret, key, token, or credential fields. 2. **Redaction** — `redact()` from `lib/services/analyzer/itglue-redact.ts` strips matching keys from the asset snapshot, peer traits, and all payloads before they reach the LLM. 3. **Endpoint** — `POST /apply` regex-blocks any `field_name` matching `/(password|secret|key|token|credential)/i` and returns 400 even if a compromised payload made it through the first two layers. ### Three-layer audit trail | Layer | Captures | Retention | |---|---|---| | `itglue_asset_audits` | Every audit run with full LLM context (asset snapshot, model used, cost, ticket count) | Forever | | `itglue_writes` | Every PATCH attempt with before/after diff, status (pending → committed \| failed \| reverted), audit provenance, raw IT Glue API response, source_evidence | Forever | | `audit_log` (generic) | One row per Apply/Revert in the format admins are used to | Per existing policy | ### Revert chain semantics A revert produces a **new** `itglue_writes` row whose `before_value` and `after_value` are swapped from the original. The new row carries `source_evidence = { reverts_write_id }`. The original row's status flips to `'reverted'`. To trace any state, walk the `audit_id` and `reverts_write_id` graph — it's always complete. ### Per-record sync after every write `refreshFlexibleAssetById()` runs after a successful PATCH (or a successful revert) so `itg_flexible_assets.traits` reflects the change immediately. The route also returns the freshly-PATCH'd asset in the response so the UI can update without waiting for the mirror. --- ## Cost & latency | Provider | Per-audit cost (typical) | Per-audit latency | |---|---|---| | Anthropic (Sonnet) | ~$0.10 | 30–60s | | OpenRouter (DeepSeek V4 Pro) | ~$0.01 | 60–180s | Apply is one IT Glue PATCH + one local upsert + one audit_log insert — typically < 1s. A full audit + 5 applies on a typical Application record: ~$0.10 (Claude) or ~$0.01 (DeepSeek), 3–5 minutes including click-through. --- ## Limitations & non-goals - **Applications only in v1.** The schema generalizes (`asset_type` is a column), but Configurations / Procedures / Domains / Passwords each have different prompt nuances. Adding more asset types is straightforward — duplicate the data-builder and prompt files, add a new asset detail page. - **No bulk-apply.** Admin clicks each suggestion individually. If a single audit produces 10 gaps that's 10 clicks. Bulk-apply is an easy fast-follow once we trust quality. - **No two-step approval workflow.** Admin-direct write per the design decision; the audit log is the safety net. The existing `approval_requests` table from the pipeline engine is available if we later want to require tech-proposes / admin-approves. - **No auto-create of new asset records.** Apply only updates existing assets. If an audit reveals a totally missing record, the suggestion is shown but Apply is disabled with an explainer; admins create the stub manually in IT Glue. - **No inline editor for suggested values.** Admin sees the LLM's suggestion verbatim; if they want to tweak, they edit the value in IT Glue afterward. v2 could add an inline editor. - **No password / secret / key / token / credential writes through this surface — ever.** Refused at three layers (above). --- ## Inspecting state (operational SQL) ### Audits, lowest score first ```sql SELECT a.asset_id, fa.name AS application_name, fa.organization_name, a.overall_score, jsonb_array_length(a.field_gaps) AS gap_count, jsonb_array_length(a.notes_promotions) AS promo_count, a.provider, a.estimated_cost_usd, a.generated_at FROM itglue_asset_audits a JOIN itg_flexible_assets fa ON fa.id = a.asset_id::bigint WHERE a.status = 'complete' ORDER BY a.generated_at DESC, a.overall_score ASC; ``` ### Writes in the last 7 days ```sql SELECT performed_at, field_name, status, before_value, after_value, performed_by_user_id, audit_id FROM itglue_writes WHERE performed_at >= NOW() - INTERVAL '7 days' ORDER BY performed_at DESC; ``` ### Find what's still waiting on which write ```sql SELECT id, field_name, before_value, after_value, source_evidence ->> 'reverts_write_id' AS reverts_id, status, performed_at FROM itglue_writes WHERE asset_id = '17096940' ORDER BY performed_at; ``` ### Audit cost-guard decisions ```sql SELECT created_at, user_id, action, estimated_cost, decision, decision_reason FROM analyzer_cost_audit WHERE action = 'itglue_audit' ORDER BY created_at DESC LIMIT 50; ``` --- ## How a typical session looks 1. Tech opens `/analyzer/itglue/applications`. MISYS 6.3 is at the top with no audit yet. 2. Tech clicks the row → lands on the detail page. They see 12/17 fields are empty. 3. Tech clicks **Run audit** (DeepSeek selected for cost). 60–180s later, the audit panel populates: 4 high-confidence field gaps, 1 notes promotion, 1 contradiction, score 0.55. 4. Tech (admin) clicks **Apply** on `Wulf Application Champion` → suggested value "Jake Hammel". A PATCH lands on IT Glue, the local mirror refreshes, the field shows the new value, a write row appears in the History section. 5. Tech clicks **Apply** on the Vendor Maintenance/Support promotion ("Steve Cianflone" extracted from Notes). 6. The next time someone opens this asset in IT Glue, the documentation reflects what tickets have been telling us all along. --- ## Files (quick reference) **Schema**: `migrations/075_itglue_audit.sql` **Pipeline**: `lib/services/analyzer/asset-audit/` - `data-builder.ts`, `prompt.ts`, `runner.ts`, `persistence.ts`, `runner.test.ts` **API**: `app/api/analyzer/itglue/applications/` - `route.ts` (list) - `[id]/route.ts` (detail) - `[id]/audit/route.ts` (GET/POST audit) - `[id]/apply/route.ts` (admin write) - `[id]/revert/[writeId]/route.ts` (admin revert) - `[id]/writes/route.ts` (per-asset history) Plus `app/api/analyzer/itglue/writes/route.ts` (admin cross-asset feed). **UI**: - `app/analyzer/itglue/applications/page.tsx` (list) - `app/analyzer/itglue/applications/[id]/page.tsx` (detail) - `app/admin/itglue-writes/page.tsx` (admin cross-asset) **Modified**: - `lib/services/itglue-client.ts` — `updateFlexibleAsset`, `refreshFlexibleAsset`, `getRawSingle`, `isITGlueConfigured`, internal `patch` - `lib/services/itglue-sync-service.ts` — `refreshFlexibleAssetById` - `lib/permissions.ts` — `itglue: ['read', 'write']` - `lib/types/analyzer.ts` — `AssetAuditResponse` schema + request types - `components/navigation/app-navigation.tsx` — nav entry --- ## Phase 4.1 additions ### Ticket-first capture flow Trigger: opt-in. The user clicks **"Check IT Glue documentation"** on an analysis detail page (`/analyzer/analysis/[id]`). Pipeline: 1. `GET /api/analyzer/analyses/[id]/itglue-suggestions` — runs `matchAssetsForAnalysis(id)` and returns matched flexible_assets + configurations (top 5 each by score) along with any existing ticket-scoped audits. 2. User clicks **"Audit for this ticket"** on a matched asset → `POST` to the same endpoint with `{ assetType, assetId, provider }`. 3. The runner builds context with `ticketScopeAnalysisId` set, so ticket evidence is exactly the one analysis the user clicked from. The system prompt picks up a ticket-scoped suffix instructing the LLM to frame findings as "what *this ticket* taught us." 4. The audit row is persisted with `triggered_by_ticket_number` + `triggered_by_analysis_id` populated. 5. Apply works from the same per-asset routes as the asset-first flow. When the audit is ticket-scoped, the resulting `itglue_writes` row also carries `triggered_by_ticket_number`, and the apply path inserts an xref row with `relationship='updated'`. Asset matching is loose: substring + word-boundary match between the ticket's `aggregate_fingerprint.{applications_involved, device_classes, vendors_involved}` and `itg_flexible_assets.name` (Application type only) + `itg_configurations.{name, hostname}` for the same client. Score 3 (exact) > 2 (word-boundary) > 1 (substring). Top 5 per kind. ### Configuration support Same audit/apply/revert pattern as Applications, with these differences: - **Flat schema** — Configurations have ~17 editable top-level columns rather than a `traits` JSONB blob. The data-builder synthesizes a trait-style map for prompt consistency, then the apply route maps field name → IT Glue dash-case attribute (`primary_ip` → `'primary-ip'`). - **Hand-curated field hints** — IT Glue Configurations don't expose a `_fields` table; the 16 hand-written hints live in `lib/services/analyzer/asset-audit/data-builder.ts` (`CONFIGURATION_FIELDS` constant). Each hint is the per-field documentation the LLM sees. - **Configuration-flavored prompt** — focuses on hostname/FQDN consistency, OS version currency, named services capture (in `operating_system_notes`), IP/MAC hygiene, contact ownership. - **Column allowlist on apply** — even though the audit's output schema doesn't restrict `field_name`, the configuration apply route refuses anything outside `name | hostname | primary_ip | mac_address | serial_number | asset_tag | position | notes | operating_system_notes`. FK-shaped fields (manufacturer_id, model_id, operating_system_id, contact_id, location_id) are read-only via this surface for v1. - **Per-record sync helper** — `refreshConfigurationById(id)` mirrors `refreshFlexibleAssetById(id)`; called after every successful Apply. ### Cross-reference index — `itglue_ticket_xrefs` ``` ticket_number | analysis_id | asset_type | asset_id | relationship | source | confidence | details | created_at ``` Three relationship types: | Relationship | Meaning | Source | |---|---|---| | `referenced` | The analyzer cited this asset/doc when analyzing the ticket. Comes from `analyzer_analyses.itglue_docs_referenced`. | `analyzer_referenced` | | `updated` | A ticket-driven audit produced a write on this asset. | `audit_write` | | `should_have_referenced` | Reserved — gap text suggests we should have found this asset/doc but didn't. Not auto-populated yet. | `manual` (future) | Populated by: - **Post-analysis hook** in `lib/services/analyzer/worker.ts` — `insertReferencedXrefsFromAnalysis` runs after every successful analysis insertion. Maps each `ITGlueDocReference.doc_type` → asset_type. Best-effort. - **Post-apply hook** in the apply routes — `insertUpdatedXref` runs after a successful PATCH if the audit was ticket-scoped. Best-effort. - **No backfill** — table fills forward. The unique index `(ticket_number, analysis_id, asset_type, asset_id, relationship, source)` makes ingestion idempotent. Two views consume it: 1. **Asset detail page** — "Tickets that touched this asset" section under the audit panel, with sub-sections for `Referenced by` (with the LLM's relevance reason) and `Updated by` (with the field name and write history link). 2. **`GET /api/analyzer/tickets/[ticketNumber]/itglue-xrefs`** — ticket-side view; available for future ticket-page surfacing. This table is also the lookup index a future RAG automation will use: given a new ticket's entities, fetch historically-referenced docs as warm candidates for the analyzer's IT Glue retrieval stage. ### Ticket-linkage columns Both audit and write tables carry denormalized ticket linkage: - `itglue_asset_audits.triggered_by_ticket_number` (and `triggered_by_analysis_id`) — set on ticket-scoped audits. Null on asset-first audits. - `itglue_writes.triggered_by_ticket_number` — copied forward from the audit on Apply, so "every write a given ticket drove" is a one-query lookup. Null when the audit was asset-first. ### Files added in 4.1 - `migrations/076_itglue_ticket_xrefs.sql` - `lib/services/analyzer/asset-audit/xrefs.ts` - `lib/services/analyzer/asset-audit/asset-matcher.ts` - `app/api/analyzer/analyses/[id]/itglue-suggestions/route.ts` - Configuration parallel route tree under `app/api/analyzer/itglue/configurations/` - xref endpoints: `app/api/analyzer/itglue/applications/[id]/xrefs/route.ts`, `app/api/analyzer/itglue/configurations/[id]/xrefs/route.ts`, `app/api/analyzer/tickets/[ticketNumber]/itglue-xrefs/route.ts` - `app/analyzer/itglue/configurations/page.tsx` (list) - `app/analyzer/itglue/configurations/[id]/page.tsx` (detail) - `components/analyzer/itglue-suggestions-panel.tsx` ### Files modified in 4.1 - `lib/services/itglue-client.ts` — `updateConfiguration`, `refreshConfiguration` - `lib/services/itglue-sync-service.ts` — `refreshConfigurationById` - `lib/services/analyzer/asset-audit/data-builder.ts` — assetType dispatch + ticket-scope mode + Configuration field schema - `lib/services/analyzer/asset-audit/prompt.ts` — Configuration prompt + ticket-scoped suffix + assetType-aware payload - `lib/services/analyzer/asset-audit/runner.ts` — accepts `assetType` and `ticketScopeAnalysisId`; persists triggered_by_* - `lib/services/analyzer/asset-audit/persistence.ts` — asset_type union extended; `getLatestTicketScopedAudit`; `createPendingWrite` accepts asset_type + ticket linkage - `lib/services/analyzer/worker.ts` — post-analysis xref ingestion hook - `app/api/analyzer/itglue/applications/[id]/apply/route.ts` — ticket linkage + xref insert on Apply - `app/api/analyzer/itglue/applications/[id]/revert/[writeId]/route.ts` — asset_type now passed to createPendingWrite - `app/analyzer/analysis/[id]/page.tsx` — `` rendered - `app/analyzer/itglue/applications/[id]/page.tsx` — "Tickets that touched this asset" section - `components/navigation/app-navigation.tsx` — split entries