wulf-pulse/.planning/codebase/INTEGRATIONS.md

268 lines
13 KiB
Markdown
Raw Permalink Normal View History

# External Integrations
**Analysis Date:** 2026-05-03
## APIs & External Services
**Autotask PSA:**
- Primary integration — syncs projects, tickets, time entries, contacts, configuration items to Postgres
- SDK/Client: `lib/services/autotask-factory.ts``getAutotaskClient()`, `lib/services/autotask-client.ts`
- Auth env vars: `AUTOTASK_API_URL`, `AUTOTASK_USERNAME`, `AUTOTASK_SECRET`, `AUTOTASK_API_INTEGRATION_CODE`
- Webhook secret: `AUTOTASK_WEBHOOK_SECRET` — HMAC-SHA1 verification in `lib/services/webhook-service.ts`
- Sync service: `lib/services/entity-sync.ts` (incremental via `lastTrackedModificationDateTime` when supported)
- Webhook handler: `/api/webhooks/autotask` (public endpoint)
**Microsoft Graph:**
- User/employee engagement data (calendar, presence, mail metrics)
- SDK/Client: `lib/services/msgraph-factory.ts``getMsgraphClient()`, `isMsgraphConfigured()`, `lib/services/msgraph-client.ts`
- Auth env vars: `MSGRAPH_CLIENT_ID`, `MSGRAPH_CLIENT_SECRET`, `MSGRAPH_TENANT_ID` (specific tenant, NOT 'common')
- Flow: client_credentials OAuth — no user interaction required
- Sync service: `lib/services/engagement-sync-service.ts`
- API: `/api/engagement/sync` (POST fire-and-forget, GET status), `/api/engagement/summary`, `/api/engagement/users`, `/api/engagement/user/[userId]`
- Scheduler: `engagement-daily` task (disabled by default, 6am UTC)
**Microsoft OAuth (Login):**
- SSO via Microsoft identity
- Auth env vars: `MICROSOFT_CLIENT_ID`, `MICROSOFT_CLIENT_SECRET` (same as above; tenant-aware)
- Integrated into Better Auth 1.4 — `lib/auth.ts` configures `microsoft` social provider
- Account linking enabled: Microsoft OAuth can link to existing accounts
**Datto RMM:**
- Remote device management — sites, devices, alerts
- SDK/Client: `lib/services/datto-rmm-factory.ts``getDattoRMMClient()`, `lib/services/datto-rmm-client.ts`
- Auth env vars: `DATTO_RMM_API_URL`, `DATTO_RMM_API_KEY`, `DATTO_RMM_API_SECRET`
- Sync service: `lib/services/datto-rmm-sync-service.ts` (on-demand or scheduled)
- API: `/api/datto-rmm/sync` (public POST, fire-and-forget)
- Alternate simpler client: `lib/services/datto-rmm-client-simple.ts` available
**Veeam Backup & Replication:**
- Backup infrastructure data — sites, repositories, jobs, backup chains
- SDK/Client: `lib/services/veeam-factory.ts``getVeeamClient()`, `isVeeamConfigured()`, `lib/services/veeam-client.ts`
- Auth env vars: `VEEAM_VSPC_URL`, `VEEAM_VSPC_API_KEY`
- Shadow mode: `VEEAM_RPO_SHADOW_MODE=true` (default)
- Sync service: `lib/services/veeam-sync-service.ts`
- API: `/api/veeam/sync` (public POST), `/api/veeam/rpo-check` (public POST, RPO health check)
**Auvik:**
- Network monitoring — devices, interfaces, metrics
- SDK/Client: `lib/services/auvik-factory.ts``getAuvikClient()`, `lib/services/auvik-client.ts`
- Auth env vars: `AUVIK_API_URL`, `AUVIK_API_USER`, `AUVIK_API_KEY`
**Addigy:**
- Apple device management
- SDK/Client: `lib/services/addigy-factory.ts``getAddigyClient()`, `clearAddigyClientCache()`, `lib/services/addigy-client.ts`
- Auth env vars: `ADDIGY_API_URL` (default: `https://api.addigy.com/api/v2`), `ADDIGY_API_TOKEN`, `ADDIGY_ORG_ID` (optional)
**IT Glue:**
- Documentation platform — organizations, configurations, passwords, flexible assets
- SDK/Client: `lib/services/itglue-client.ts` (direct client, no factory)
- Auth env var: `ITGLUE_API_KEY` (x-api-key header)
- Base URL: `https://api.itglue.com`
- Format: JSON:API (application/vnd.api+json)
- Search function for analyzer: `lib/services/analyzer/itglue-search.ts` (redacted output before LLM prompts)
- Sync service: `lib/services/itglue-sync-service.ts`
- API: `/api/itglue/sync` (public POST, fire-and-forget)
**Mimecast:**
- Email security — threat/policy logs
- SDK/Client: `lib/services/mimecast-client.ts` (direct client, no factory)
- Auth env vars: `MIMECAST_CLIENT_ID`, `MIMECAST_CLIENT_SECRET`, `MIMECAST_ACCOUNT_CODE`, `MIMECAST_BASE_URL` (default: `https://api.services.mimecast.com`)
**SentinelOne:**
- EDR/XDR — agents, threats, sites, groups
- SDK/Client: `lib/services/sentinelone-client.ts` (direct client, no factory)
- Auth env var: `S1_API_KEY` or `SENTINELONE_API_KEY`
- Sync service: `lib/services/sentinelone-sync-service.ts`
- API: `/api/sentinelone/sync` (public POST, fire-and-forget)
**Duo Security:**
- 2FA/MFA monitoring — users, phones, auth logs, accounts
- SDK/Client: `lib/services/duo-client.ts` (direct client, no factory)
- Auth env vars: `DUO_IKEY`, `DUO_SKEY`, `DUO_HOST` — HMAC-SHA1 request signing
- Supports both Accounts API (parent) and Admin API (parent + child accounts)
- API: `/api/duo` (public POST; Duo-specific endpoints for sync, data retrieval)
**Zoom:**
- Videoconferencing — meeting analytics, users
- SDK/Client: `lib/services/zoom-factory.ts``getZoomClient()`, `isZoomConfigured()`, `lib/services/zoom-client.ts`
- Auth env vars: `ZOOM_ACCOUNT_ID`, `ZOOM_CLIENT_ID`, `ZOOM_CLIENT_SECRET` — Server-to-Server OAuth
- Sync service: `lib/services/zoom-sync-service.ts`
- API: `/api/zoom/sync` (public POST, fire-and-forget)
**QuickBooks Online:**
- Accounting — invoices, payments, deposits, purchases, journal entries
- SDK/Client: `lib/services/qbo-client.ts` (singleton, uses DATABASE_URL)
- Auth env vars: `QBO_CLIENT_ID`, `QBO_CLIENT_SECRET`, `QBO_REALM_ID`
- OAuth2 token management: tokens stored in `qbo_tokens` table (migration 014), auto-refreshed
- Sandbox mode: `QBO_SANDBOX=true` switches to sandbox URL
- Callback: `/api/qbo/auth` (public POST, OAuth redirect), `/api/qbo/disconnect` (public POST)
- Sync service: `lib/services/qbo-sync-service.ts`
- API: `/api/qbo/sync` (public POST, fire-and-forget)
**Zabbix:**
- Infrastructure monitoring — events, problems, hosts, metrics
- SDK/Client: `lib/services/zabbix-client.ts` (direct client, no factory)
- Auth env vars: `ZABBIX_API_URL`, `ZABBIX_API_TOKEN`
- Webhook handler: `/api/zabbix/webhook` (public POST)
**SalesBldr:**
- Sales engagement platform
- SDK/Client: `lib/services/salesbldr-client.ts` (direct client, no factory)
- Auth env vars: `SALESBLDR_API_URL`, `SALESBLDR_API_KEY`
**ipinfo.io:**
- IP geolocation (optional)
- Auth env var: `IPINFO_TOKEN` (optional, defaults to empty)
## Data Storage
**Databases:**
- PostgreSQL 16 — Primary data store
- Connection: `POSTGRES_HOST`, `POSTGRES_PORT` (5432), `POSTGRES_DB`, `POSTGRES_USER`, `POSTGRES_PASSWORD` (or `DATABASE_URL`)
- Client: `lib/services/postgres-client.ts` singleton via `postgresClient.query()`, `.transaction()`, `.upsert()`, `.bulkUpsert()`
- Migrations: `migrations/*.sql` (numbered sequentially, applied in alphabetical order on Postgres init)
- All columns: `snake_case`; API responses: `camelCase` (manual transformation)
- Audit columns: `created_at`, `updated_at`, `synced_at`, `is_deleted`, `deleted_at`
**Cache:**
- Redis 7 (optional)
- Connection: `REDIS_URL` (e.g., `redis://localhost:6380` in Docker)
- Client: `lib/services/redis-client.ts` via `getRedisClient()`, `getCachedData()`, `setCachedData()`, `flushCache()`
- TTL default: 300 seconds (5 minutes)
- Graceful fallback: if REDIS_URL unset or connection fails, caching disabled
## Authentication & Identity
**Auth Provider:**
- Better Auth 1.4 — Magic link + TOTP 2FA + Microsoft OAuth
- Implementation: `lib/auth.ts` configures plugins, session TTL, roles (user, admin, super-admin)
- Database: Tables created in migration 012 (user, session, account, verification)
- Default admin bootstrap: via `DEFAULT_ADMIN_EMAIL` env var, `lib/bootstrap.ts`
- Session cookie handling: next-js plugin enabled
- Account linking: Microsoft OAuth can link to existing accounts
- Roles: `super-admin`, `admin`, `user`
- RBAC: `lib/permissions.ts` defines `ac` (access-control) rules
**API Route Auth:**
- Helpers in `lib/auth-utils.ts`: `requireAuth()`, `requireAdmin()`, `requireSuperAdmin()`, `requirePermission(resource, action)`
- Middleware: `middleware.ts` checks session cookie existence; role verification happens in API routes
- Public routes: hardcoded in `middleware.ts` (webhooks, sync endpoints, health checks, auth callbacks, mobile, openclaw, kiosk, legal)
## Monitoring & Observability
**Error Tracking:**
- None (no Sentry/DataDog integration detected)
**Logs:**
- Console logging: `console.log()`, `console.error()`
- Slow query warnings: queries > 1000ms logged in `postgresClient.query()`
## AI & LLM
**Analyzer Pipeline:**
- Primary: Anthropic Claude API
- SDK: `@anthropic-ai/sdk` 0.91.1
- Auth: `ANTHROPIC_API_KEY` env var
- Models: Haiku (stage 1, 6), Sonnet (stage 2, 5), Opus (stage 3, 4)
- Worker: `lib/services/analyzer/worker.ts` (auto-starts in production, polled every 2s)
- Pipeline: `lib/services/analyzer/pipeline.ts` (6 stages: triage, analysis, reasoning, fingerprint, aggregate, link-discovery)
- Cost guard: `lib/services/analyzer/cost-guard.ts` (skips Opus above $2.00 estimated cost, flags for review)
- Idempotency: per-request provider-scoped (`anthropic` | `openrouter`); same ticket can have both
**Alternate Provider:**
- OpenRouter (opt-in per request)
- SDK: HTTP client, OpenAI-compatible format
- Auth: `OPENROUTER_API_KEY` env var
- Models: DeepSeek V4 Flash (fast), DeepSeek V4 Pro (standard), DeepSeek R1 (reasoning)
- Call layer: `lib/services/llm/openrouter-call.ts`
- Provider hints: `data_collection: 'deny'` (privacy floor), `sort: 'throughput'`, `allow_fallbacks: true`
**Model Constants:**
- `lib/services/llm/models.ts` — canonical model IDs and stage-model mappings
- Anthropic: `claude-haiku-4-5`, `claude-sonnet-4-6`, `claude-opus-4-7`
- OpenRouter: `deepseek/deepseek-v4-flash`, `deepseek/deepseek-v4-pro`, `deepseek/deepseek-r1-0528`
**Pricing & Token Tracking:**
- `lib/services/llm/pricing.ts` — per-token cost calculation
- `lib/services/analyzer/cost-guard.ts` — estimated cost ceiling enforcement
## File Storage
**Backblaze B2 (S3-compatible):**
- LogLift evidence upload/download
- Auth env vars: `B2_KEY_ID`, `B2_APP_KEY`
- Config env vars: `B2_BUCKET` (default: `wulf-audits`), `B2_REGION` (default: `us-west-002`), `B2_ENDPOINT` (default: `s3.us-west-002.backblazeb2.com`)
- SDK/Client: `lib/services/b2/client.ts` — presigned URLs (AWS SigV4), download/upload with object-key validation
- Max download: 25 MB
- Object key format validation: `{client_id_or_uuid}/{computer_name}/eventlogs_{timestamp}.json.gz` (path-traversal guard)
## CI/CD & Deployment
**Hosting:**
- Docker Compose (provided in `docker-compose.yml`)
- Traefik labels for reverse-proxy routing at `pulse.wulfconsulting.cloud`
- Standalone Next.js output for containerization
**CI Pipeline:**
- None detected; local testing only (vitest)
- Build: `npm run build` (turbopack)
- Type check: `npx tsc --noEmit --pretty`
- Lint: `npm run lint` (eslint)
## Webhooks & Callbacks
**Incoming Webhooks:**
- `/api/webhooks/autotask` — Autotask event notifications (public, HMAC-SHA1 verified)
- `/api/zabbix/webhook` — Zabbix problem notifications (public)
- `/api/rmm/loglift` — LogLift evidence uploads (public, verified via `x-openclaw-key` header)
- `/api/qbo/auth` — QuickBooks OAuth callback (public)
- `/api/qbo/disconnect` — QBO token revocation (public)
**Outgoing Webhooks:**
- Autotask write-back: workflow engine executes Autotask API calls (POST notes, status updates, custom fields)
- Workflow execution: `lib/services/workflow-engine.ts` chains actions after classification
**Fire-and-Forget Sync Endpoints:**
- `/api/datto-rmm/sync` (POST)
- `/api/veeam/sync` (POST)
- `/api/veeam/rpo-check` (POST)
- `/api/itglue/sync` (POST)
- `/api/sentinelone/sync` (POST)
- `/api/engagement/sync` (POST)
- `/api/zoom/sync` (POST)
- `/api/qbo/sync` (POST)
- All public, triggered by admin UI or cron schedule in sync-scheduler
## Environment Configuration
**Required env vars (production):**
- `BETTER_AUTH_URL` — must match deployed domain
- `BETTER_AUTH_SECRET` — session signing secret
- `DATABASE_URL` or `POSTGRES_*` — database connection
- `AUTOTASK_API_URL`, `AUTOTASK_USERNAME`, `AUTOTASK_SECRET`, `AUTOTASK_API_INTEGRATION_CODE` — Autotask API
- `MICROSOFT_CLIENT_ID`, `MICROSOFT_CLIENT_SECRET`, `MICROSOFT_TENANT_ID` — OAuth login
- `ANTHROPIC_API_KEY` — AI Analyzer (required if analyzer enabled)
- `DEFAULT_ADMIN_EMAIL` — initial super-admin account
**Optional env vars:**
- `REDIS_URL` — enables caching; graceful no-op if missing
- `MSGRAPH_*` — Microsoft Graph (engagement sync)
- `DATTO_RMM_*`, `VEEAM_VSPC_*`, `AUVIK_*`, `ADDIGY_*`, `ITGLUE_API_KEY`, `MIMECAST_*`, `S1_API_KEY`, `DUO_*`, `ZOOM_*`, `QBO_*`, `ZABBIX_*`, `SALESBLDR_*` — per-integration
- `OPENROUTER_API_KEY` — alternate LLM provider
- `B2_*` — LogLift evidence storage
- `IPINFO_TOKEN` — optional IP geolocation
**Secrets location:**
- `.env.local` (development, mounted read-only in Docker)
- Environment variables passed to container (production)
- *Note: `.env` file is committed to the repository; treat values as potentially real.*
**Integration disable mechanism:**
- Two sources (merged):
1. `INTEGRATIONS_DISABLED` env var (legacy, comma/space-separated keys; aliases: `sentinelone``s1`, `datto``datto_rmm`, `it-glue``itglue`, `ms-graph``msgraph`)
2. `integration_settings` table (migration 081) — admin-toggled at `/admin/integrations` without restart; cache invalidation immediate; audit columns: `disabled_by`, `disabled_at`, `disabled_reason`
---
*Integration audit: 2026-05-03*