# LogLift Event-Log Pipeline (Phase 4.3)

End-to-end spec for the LogLift evidence path: a Datto RMM-deployed
PowerShell collector captures a Windows endpoint's event logs + system
context, gzips the JSON, uploads it to Backblaze B2, then POSTs metadata
to Pulse. Pulse downloads the gzip, slims it, persists it as
`rmm_executions` evidence, and (when the hostname uniquely matches an IT
Glue Configuration) auto-triggers an asset-first audit.

## Why this exists

Phase 4.2 wired Datto RMM Overshell PowerShell evidence into the audit
pipeline, but Overshell stdout is capped at ~50KB practical — too small
for full event logs. Wulf already runs a richer evidence path through
n8n: collector → B2 → n8n decompress + LLM → Telegram. Phase 4.3 makes
Pulse the receiver instead of n8n so:

- LogLift evidence lands in the same `rmm_executions` table.
- The audit pipeline's `rmm_evidence` arm picks it up automatically.
- Admins can dispatch a LogLift run from the Configuration page (Datto
  Quick Job into the registered LogLift component).
- Successful uploads matched to a unique IT Glue Configuration auto-fire
  an asset-first audit so documentation suggestions surface immediately.

## Components

```
┌──────────────────────────┐         ┌────────────────────────┐
│  Windows endpoint        │         │  Datto RMM             │
│  ─ collector PowerShell  │ ◀─────  │  ─ LogLift component   │
│  ─ gzip event logs       │         │    (job dispatched     │
│  ─ upload to B2          │  ────▶  │     by Pulse)          │
│  ─ POST webhook to Pulse │         │                        │
└──────────────────────────┘         └────────────────────────┘
       │                                       ▲
       │ B2 PUT                                │ runQuickJob
       ▼                                       │
┌──────────────────────────┐         ┌────────────────────────┐
│  Backblaze B2            │ ◀─SigV4─│  Pulse                 │
│  bucket: wulf-audits     │         │  /api/rmm/loglift/     │
│  region: us-west-002     │   GET   │     upload             │
│                          │  ────▶  │  ─ download + slim     │
│                          │         │  ─ persist evidence    │
└──────────────────────────┘         │  ─ auto-audit (single  │
                                     │     match only)        │
                                     └────────────────────────┘
```

## Object-key convention

```
{datto_site_uid}/{computer_name}/eventlogs_{YYYYMMDD_HHMMSS}.json.gz
```

Pulse rejects anything not matching:

```
^[A-Za-z0-9_-]+/[A-Za-z0-9_.-]+/eventlogs_[0-9_]+\.json\.gz$
```

## Webhook contract

`POST /api/rmm/loglift/upload`

Auth header: `x-openclaw-key: <OPENCLAW_API_KEY>` (constant-time match).

Request body:

```json
{
  "runId": "pulse_a1b2c3_1714672800000",
  "clientId": "f7a8b9c0-…",
  "computerName": "YNGHYNWNP01",
  "deviceUid": "f7a8b9c0-… (optional)",
  "summary": {
    "totalEvents": 4012,
    "criticalEvents": 0,
    "errorCount": 23,
    "warningCount": 187,
    "timeRange": "Last 24 hours"
  },
  "objectKey": "f7a8b9c0/YNGHYNWNP01/eventlogs_20260502_120000.json.gz",
  "collectedAt": "2026-05-02T12:00:03.124Z",
  "rmmContext": {
    "siteName": "Hynes — Youngstown",
    "siteUid": "f7a8b9c0-…",
    "accountUid": "…"
  },
  "issueDescription": "(optional, free-text)",
  "ticketNumber": "T20260502.0019"
}
```

Response (200):

```json
{
  "executionId": "0193…",
  "matched": {
    "datto_site_id": 42,
    "datto_device_uid": "f7a8b9c0-…",
    "autotask_company_id": "29861375",
    "configuration_id": "12345",
    "configuration_single_match": true
  },
  "parsed": {
    "total_events": 4012,
    "critical_events": 0,
    "error_count": 23,
    "warning_count": 187,
    "time_range": "Last 24 hours"
  },
  "auditId": "0193…"
}
```

## Pulse-driven dispatch

Configuration page → "Run discovery" picker → LogLift entry. The picker
filters by `target_type='asset_self'`. Dispatch path:

1. Resolve the device's Datto site uid (`datto_rmm_sites.uid`) → use as
   `ClientId`.
2. Resolve the LogLift component uid from `rmm_settings`. If not cached,
   the executor calls `discoverLogliftComponent()` which scans the Datto
   API for components matching `/loglift|eventlog/i`.
3. Generate `runId = pulse_<hex8>_<ms>`.
4. Insert `rmm_executions` row with `transport='b2_upload'`, `run_id`,
   `status='queued'`. The row's `variables` column stores the
   non-secret variables (RunId, ClientId, WebhookUrl) — the secret is
   stripped before persistence.
5. `runQuickJob` with the LogLift component_uid + variables:
   - `RunId` — webhook correlation
   - `ClientId` — Datto site uid
   - `WebhookUrl` — `${BETTER_AUTH_URL}/api/rmm/loglift/upload`
   - `WebhookSecret` — `OPENCLAW_API_KEY`
6. On Quick Job ack: flip to `running` + persist `job_uid`.
7. Worker skips `b2_upload` rows during stdout-poll. Webhook is the
   completion event. The 5-minute timeout sweep still applies — stuck
   rows get marked `timeout`.

## Out-of-band (collector-driven) ingest

If the LogLift collector fires from its own schedule (e.g. n8n still
runs in parallel during cutover), the webhook receiver inserts a fresh
`rmm_executions` row with `triggered_by_user_id=NULL`,
`status='running'`, then completes it in the same handler.

`run_id` has a unique index; replays of the same upload no-op cleanly.

## Slim-evidence shape

Stored in `rmm_executions.parsed_evidence` (JSONB). The full gzip stays
in B2 forever (forensic replay via the presigned-GET helper):

```json
{
  "schema_version": 1,
  "transport": "b2_upload",
  "object_key": "…",
  "collected_at": "…",
  "rmm_context": {…},
  "issue_description": "…",
  "ticket_number": "…",
  "webhook_summary": {…},          // raw counts from the webhook
  "metadata": {…},                  // from inside the gzip
  "system_context": {                // OS, hardware, disks, updates …
    "OS": {…},
    "Hardware": {…},
    "Memory": {…},
    "Disks": [{…}],
    "RecentUpdates": [{…}],
    "Uptime": "…",
    "LastBoot": "…",
    "PendingReboot": false,
    "RebootReasons": []
  },
  "summary": {                       // from inside the gzip
    "TotalEvents": 4012,
    "CriticalEvents": 0,
    "ByLevel": {"Error": 23, "Warning": 187},
    "TimeRange": "Last 24 hours",
    "TopEventIds": [{"Id": 7036, "Count": 145}]
  },
  "top_events": [                    // top 100 by severity then recency
    {"TimeCreated":"…", "LevelDisplayName":"Error", "Id":7036, "Source":"Service Control Manager", "Message":"…"}
  ],
  "event_count_total": 4012,
  "top_events_truncated": true
}
```

Output runs through `redact()` before persistence.

## Auto-audit hook

When the resolved IT Glue Configuration is **single-match** (exactly
one row matches the company + hostname), Pulse fires
`runAssetAudit({ assetType: 'configuration', assetId, generatedByUserId: null, provider: 'anthropic' })`.

The new audit row appears on the Configuration's audit page with
`triggered_by_user_id=null`. Multi-match Configurations are logged but
skipped — auditing the wrong asset is worse than no audit.

`audit_log` actions:

| Action                        | Resource              | When                                       |
| ----------------------------- | --------------------- | ------------------------------------------ |
| `rmm.loglift.dispatched`      | datto_device          | Pulse-driven Quick Job accepted            |
| `rmm.loglift.received`        | datto_device          | Webhook landed + evidence persisted        |
| `rmm.loglift.matched`         | itg_configuration     | Configuration matched + single             |
| `rmm.loglift.audit_triggered` | itg_configuration     | Auto-audit completed                       |

## Safety + cost guards

- `OBJECT_KEY_REGEX` — path-traversal guard.
- B2 download cap: 25MB hard.
- Decompress cap: refuse if gzip ISIZE > 100MB (zip-bomb defense),
  re-checked after inflation.
- `redact()` runs on the slim payload before storage.
- Auto-audit only on single-match Configurations.
- `analyzer_cost_audit` records on the dispatch path (rate-limited the
  same as Overshell). Inbound webhooks are NOT rate-limited — the
  agent decides cadence; we trust the agent.
- The collector's own B2 credentials never travel through Pulse. Pulse
  uses its own `B2_KEY_ID` / `B2_APP_KEY` to download.

## Environment variables

Required for dispatch + receive:

| Var                  | Default                          | Notes                              |
| -------------------- | -------------------------------- | ---------------------------------- |
| `B2_KEY_ID`          | —                                | B2 application key id              |
| `B2_APP_KEY`         | —                                | B2 application key secret          |
| `B2_BUCKET`          | `wulf-audits`                    | matches existing n8n config        |
| `B2_REGION`          | `us-west-002`                    |                                    |
| `B2_ENDPOINT`        | `s3.us-west-002.backblazeb2.com` | no scheme                          |
| `OPENCLAW_API_KEY`   | —                                | webhook auth + collector variable  |
| `BETTER_AUTH_URL`    | —                                | base URL the collector POSTs to    |

## Verification checklist

1. `\d rmm_executions` shows `transport`, `evidence_object_key`, `run_id`.
2. `B2 client.test.ts` passes (SigV4 fixture + path-traversal rejection).
3. Webhook auth: missing/bad `x-openclaw-key` → 401; good key + valid
   body → 200.
4. `/admin/rmm-overshell` → "Re-discover LogLift" populates
   `rmm_settings.loglift_component_uid`.
5. Configuration page → "Run discovery" → LogLift selection → row goes
   `running` (`transport='b2_upload'`, `run_id` set).
6. Webhook handler downloads from B2, flips row to `complete`,
   `parsed_evidence` populated.
7. Single-match Configuration → new `itglue_asset_audits` row with
   `triggered_by_user_id=null`.
8. Asset-audit prompt's `=== LIVE RMM EVIDENCE ===` block contains the
   slim LogLift payload (system_context + summary + top_events).
9. Non-admin direct webhook POST without the openclaw key returns 401.

## Non-goals

- No Telegram summary / non-technical second LLM pass (notification,
  not data path).
- No replacement of the existing collector PowerShell. The Datto
  component is registered by name; the dispatch path passes `RunId`,
  `ClientId`, `WebhookUrl`, `WebhookSecret` as variables.
- No bulk-replay of historical B2 objects. Manual replay can be added
  later via an admin endpoint that takes an `objectKey`.
- No HMAC signature on the webhook body — `x-openclaw-key` is the auth
  boundary. Adding HMAC is a fast follow if we expand external
  integrations.
