Bundles several in-progress efforts that were sitting uncommitted: - User queue-preferences (migration 087, API route, popover component) - QBO invoice soft-delete (migration 088) and AR diagnostics route - Dashboard/mobile engagement route and page adjustments - Docker Compose log-rotation config - One-off ticket/RMM investigation scripts (scripts/) - Planning docs: phase verification/pattern notes, mobile shell design spec - .gitignore: exclude local scratch financial/inventory data and Claude Code worktree/local-settings runtime state (never meant for version control) Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01W6RuWdiUiXrPK6FLBHjtpY
6.7 KiB
| name | description |
|---|---|
| pulse-overshell-b2-evidence | Use when running PowerShell on a Pulse-managed Windows endpoint via Datto RMM Overshell, uploading large output to Backblaze B2, and retrieving the parsed evidence — covers both the stdout-only and B2-transport pipelines, the dispatch API, webhook receiver, presigned download, and where results land in rmm_executions. |
Pulse Overshell → B2 → Retrieve
Pulse has two PowerShell evidence pipelines, both terminating in the same
rmm_executions table. Pick the right one for the payload size, dispatch
via the API, then read the row back.
Decision: which transport?
| Output size | Transport (transport column) |
How results come back |
|---|---|---|
| ≤ ~50 KB stdout | overshell_stdout (default) |
Worker polls Datto job → raw_stdout + parsed_evidence |
| Large (event logs, dumps) | b2_upload (LogLift) |
Collector gzips → B2 PUT → webhook → Pulse downloads + slims |
There is no ad-hoc PowerShell input. Only registered RmmScripts in
lib/services/rmm/scripts/index.ts can run. To add one, create
lib/services/rmm/scripts/<id>.ts with a parseOutput that JSON-parses
a ConvertTo-Json -Compress tail, then add to _all in index.ts and
bump version. Registry test enforces uniqueness + credential-shape ban.
Dispatch (both transports)
POST /api/rmm/executions (requires rmm.execute permission — admin/super-admin only):
// site-anchored (runs on the client's WNP endpoint)
{ scriptId: 'get-ad-health', target: { type: 'site_anchor', companyId: 29861375 } }
// asset-self (runs on a specific Datto device)
{ scriptId: 'loglift-eventlogs',
target: { type: 'asset_self', deviceUid: 'f7a8…', assetType: 'configuration', assetId: '12345' } }
Returns { executionId, status: 'queued' | 'running', ... }. Rate limit:
50 executions per user per 24h — trips before runQuickJob and logs
to analyzer_cost_audit with decision='blocked'. Hard timeout: 5 min.
How LogLift uses B2
For loglift-eventlogs (the only B2-transport script today), Pulse
passes the collector four runQuickJob variables — RunId, ClientId
(Datto site uid), WebhookUrl (${BETTER_AUTH_URL}/api/rmm/loglift/upload),
WebhookSecret (OPENCLAW_API_KEY). The collector script (registered
inside Datto, not stored in Pulse) gzips its JSON and PUTs to:
{datto_site_uid}/{computer_name}/eventlogs_{YYYYMMDD_HHMMSS}.json.gz
…in bucket B2_BUCKET (default wulf-audits, region us-west-002).
Object-key shape is enforced by OBJECT_KEY_REGEX in
lib/services/b2/client.ts — anything else is rejected at the webhook.
Then it POSTs to /api/rmm/loglift/upload with x-openclaw-key: $WebhookSecret
and the metadata body (runId, clientId, computerName, objectKey,
summary, etc. — see docs/loglift-eventlog-pipeline-spec.md for the
exact shape). Pulse calls downloadToBuffer(objectKey), gunzips, runs
redact(), slims to parsed_evidence, and flips the row to complete.
Retrieving results
From an executionId you already have
const r = await fetch(`/api/rmm/executions/${id}`).then(r => r.json());
// r.execution.status, .raw_stdout (overshell_stdout), .parsed_evidence (both), .evidence_object_key (b2_upload)
From SQL
-- Most-recent successful run per (company, script):
SELECT DISTINCT ON (target_company_id, script_id)
id, target_company_id, script_id, target_hostname, transport,
evidence_object_key, completed_at, jsonb_pretty(parsed_evidence) AS parsed
FROM rmm_executions
WHERE status = 'complete'
AND completed_at >= NOW() - INTERVAL '14 days'
ORDER BY target_company_id, script_id, completed_at DESC;
Re-fetching the original gzip from B2
The slim payload is in Postgres; the full gzip stays in B2 forever for forensic replay. From a Node script or API route:
import { downloadToBuffer, presignDownload } from '@/lib/services/b2/client';
// In-process (capped at 25 MB by MAX_DOWNLOAD_BYTES):
const buf = await downloadToBuffer(row.evidence_object_key);
// Or hand a short-lived URL to a human / external tool:
const url = presignDownload(row.evidence_object_key, 600); // 10-minute GET
presignDownload / presignUpload validate OBJECT_KEY_REGEX and sign
with SigV4 against B2_KEY_ID / B2_APP_KEY from env. They throw
B2NotConfiguredError if either is unset.
Where things live
- Dispatch:
lib/services/rmm/executor.ts(queueExecution) - Worker (stdout polling + timeout sweep):
lib/services/rmm/worker.ts - LogLift receiver:
lib/services/rmm/loglift-receiver.ts→app/api/rmm/loglift/upload/route.ts - B2 SigV4 client:
lib/services/b2/client.ts - Script registry:
lib/services/rmm/scripts/index.ts - Settings (component_uids, variable name):
lib/services/rmm/settings.ts→/admin/rmm-overshellfor the UI + "Re-discover" buttons - Schema:
migrations/077_rmm_overshell.sql+078_loglift_uploads.sql(addstransport,evidence_object_key,run_id) - Specs:
docs/rmm-overshell-evidence-spec.md,docs/loglift-eventlog-pipeline-spec.md
Gotchas
- The worker is a side-effect import. Anything that needs it running
must
import '@/lib/services/rmm/worker'somewhere on the server. The POST/api/rmm/executionsroute already does this. Don't eager-import from hot paths or shared utilities. - Worker ignores
transport='b2_upload'rows for stdout polling. The webhook is the completion event. If the webhook never arrives, the 5-minute timeout sweep marks the rowtimeout. - Redaction runs twice.
lib/services/analyzer/itglue-redact.ts:redact()strips anything keyed/password|secret|key|token|credential|api[_-]?key/ibefore persistence and again before the LLM prompt sees it. Don't try to surface those fields — they're gone by the time you read the row. - B2 download cap is 25 MB hard. Refuses content-length over the cap and aborts mid-stream if it exceeds. Decompress cap is 100 MB ISIZE (zip-bomb defense).
- Object-key shape is strict. Custom prefixes won't work — must be
{client_id_or_uuid}/{computer_name}/eventlogs_{timestamp}.json.gz. If you need a new payload type, add a new key regex + a new transport rather than loosening the existing one. - Auto-audit fires only on single-match Configurations. Multi-match
hostnames land the evidence but skip the audit (logged with
rmm.loglift.matched, noaudit_triggered). - Webhook is not HMAC-signed —
x-openclaw-keyis the auth boundary. If you expose the endpoint to a new collector, rotateOPENCLAW_API_KEY. B2_*env vars are required.isB2Configured()checks onlyB2_KEY_ID + B2_APP_KEY; bucket/region/endpoint fall back towulf-audits/us-west-002/s3.us-west-002.backblazeb2.com.