diff --git a/docs/DUO_INTEGRATION.md b/docs/DUO_INTEGRATION.md new file mode 100644 index 0000000..ac7443d --- /dev/null +++ b/docs/DUO_INTEGRATION.md @@ -0,0 +1,283 @@ +# Duo Security Integration + +## Overview + +Pulse syncs data from **Duo Security** across all managed child accounts using two Duo APIs: + +- **Accounts API** — Lists child accounts under the parent MSP account +- **Admin API** — Pulls users, phones, auth logs, groups, and integrations from each child (and the parent) + +The parent Accounts API credentials can sign requests against any child account's API hostname, so no per-child API keys are needed. + +--- + +## Architecture + +``` +┌──────────────────┐ POST /accounts/v1/account/list ┌──────────────┐ +│ Duo Accounts │ ◄──────────────────────────────────────│ │ +│ API │ (returns 32 child accounts) │ │ +└──────────────────┘ │ DuoClient │ + │ (HMAC-SHA1 │ +┌──────────────────┐ GET /admin/v1/users, phones, ... │ signing) │ +│ Duo Admin API │ ◄──────────────────────────────────────│ │ +│ (per-child host │ (with account_id param) │ │ +│ + parent) │ └──────┬───────┘ +└──────────────────┘ │ + │ + ┌──────▼───────┐ + │ DuoSyncService│ + │ (orchestrate │ + │ full sync) │ + └──────┬───────┘ + │ upsert + ┌──────▼───────┐ + │ PostgreSQL │ + │ (6 tables) │ + └──────────────┘ +``` + +--- + +## Data Synced + +### 1. Accounts (`duo_accounts`) + +| Field | Description | +|-------|-------------| +| `account_id` | Duo's unique account identifier | +| `name` | Account display name | +| `api_hostname` | API hostname for Admin API calls | +| `autotask_company_id` | FK to matched Autotask company | +| `user_count` | Number of users in this account | +| `integration_count` | Number of integrations | +| `edition` | Duo edition (if available) | +| `is_parent` | `true` for the parent Wulf Consulting account | + +**Source:** Accounts API `POST /accounts/v1/account/list` + parent account row added manually. + +### 2. Users (`duo_users`) + +| Field | Description | +|-------|-------------| +| `user_id` | Duo user ID | +| `username` | Login username | +| `email` | Email address | +| `realname` | Display name | +| `status` | `active`, `bypass`, `disabled`, `locked out` | +| `is_enrolled` | Whether the user has completed enrollment | +| `last_login` | Last authentication timestamp | +| `phones_count` | Number of registered devices | +| `groups` | JSONB array of group memberships | +| `aliases` | JSONB array of username aliases | + +**Source:** Admin API `GET /admin/v1/users` per account. + +### 3. Phones (`duo_phones`) + +| Field | Description | +|-------|-------------| +| `phone_id` | Duo phone ID | +| `name` | Device name | +| `number` | Phone number | +| `type` | `mobile`, `landline`, etc. | +| `platform` | `Apple iOS`, `Google Android`, etc. | +| `model` | Device model | +| `os_version` | Operating system version | +| `activated` | Whether Duo Mobile is activated | +| `last_seen` | Last device activity | +| `capabilities` | JSONB array of supported auth methods | +| `users` | JSONB array of associated users | + +**Source:** Admin API `GET /admin/v1/phones` per account. + +### 4. Auth Logs (`duo_auth_logs`) + +| Field | Description | +|-------|-------------| +| `txid` | Unique transaction ID | +| `timestamp` | When the authentication occurred | +| `user_name` | Username that authenticated | +| `factor` | Auth method: `duo_push`, `phone`, `sms`, `passcode`, etc. | +| `result` | `success`, `denied`, `fraud` | +| `reason` | Detailed reason for result | +| `application_name` | Which integration triggered the auth | +| `access_device_ip` | IP address of the access device (INET type) | +| `access_device_location` | JSONB with city/state/country of access device | +| `auth_device_ip` | IP of the authenticating device | +| `event_type` | `authentication`, `enrollment` | + +**Source:** Admin API `GET /admin/v2/logs/authentication` per account. Uses incremental sync — only pulls logs newer than the last synced timestamp (default: 30-day lookback on first sync). + +### 5. Groups (`duo_groups`) + +| Field | Description | +|-------|-------------| +| `group_id` | Duo group ID | +| `name` | Group name | +| `description` | Group description | +| `member_count` | Number of members | +| `status` | Group status | + +**Source:** Admin API `GET /admin/v1/groups` per account. + +### 6. Integrations (`duo_integrations`) + +| Field | Description | +|-------|-------------| +| `integration_key` | Duo integration key | +| `name` | Integration name | +| `type` | Integration type (e.g., `websdk`, `rdgateway`) | +| `enabled` | Whether the integration is active | +| `notes` | Admin notes | + +**Source:** Admin API `GET /admin/v1/integrations` per account. + +--- + +## Company Matching + +After syncing accounts, the service automatically matches Duo account names to Autotask companies: + +1. **Exact match** — `duo_accounts.name` = `companies.company_name` +2. **Case-insensitive containment** — company name contains account name or vice versa +3. **No match** — left unlinked (can be manually linked later) + +Current match rate: **30 of 32 child accounts (94%)**. + +--- + +## Sync Frequency + +The sync is currently **on-demand only** — triggered via API call or the UI "Sync Now" button. + +To schedule automatic syncing, add an entry to `sync_schedules`: + +```sql +INSERT INTO sync_schedules (id, sync_type, cron_expression, enabled, description) +VALUES ('duo-sync', 'duo', '0 3 * * *', true, 'Duo Security full sync daily at 3am'); +``` + +A full sync takes approximately **75 seconds** (sequential processing across 33 accounts to respect Duo rate limits). + +--- + +## Sync Process + +1. **List child accounts** via Accounts API +2. **For each child account** (sequentially, to respect rate limits): + - Sync users → upsert into `duo_users` + - Sync phones → upsert into `duo_phones` + - Sync groups → upsert into `duo_groups` + - Sync integrations → upsert into `duo_integrations` + - Sync auth logs (incremental) → insert new logs into `duo_auth_logs` +3. **Sync parent account** (same as child but uses parent host directly) +4. **Run company matching** — link Duo accounts to Autotask companies +5. **Update account stats** — refresh `user_count` and `integration_count` + +### Rate Limiting + +- The client handles HTTP 429 responses automatically +- Reads the `Retry-After` header and waits before retrying +- Child accounts are processed **sequentially** (not in parallel) to avoid hitting rate limits + +### Incremental Auth Logs + +- On first sync: pulls last 30 days of auth logs +- On subsequent syncs: uses `MAX(timestamp)` from existing logs as `mintime` +- Auth logs v2 uses a different pagination model (`next_offset` array) which is handled separately from standard v1 pagination + +--- + +## API Endpoints + +### Internal (session-authenticated via middleware bypass) + +| Method | Endpoint | Description | +|--------|----------|-------------| +| `POST` | `/api/duo/sync` | Trigger a full sync (non-blocking) | +| `GET` | `/api/duo/sync` | Check sync status (`inProgress`, `currentSyncId`) | +| `GET` | `/api/duo/status` | Connection status + record counts (accounts, users, phones, auth logs, groups, integrations, bypass, disabled) | +| `GET` | `/api/duo/accounts` | List all accounts with stats and matched Autotask company | +| `GET` | `/api/duo/accounts/[id]/users` | List users for a specific account | +| `GET` | `/api/duo/users/flagged` | List bypass and disabled users separately with account names | + +### OpenClaw (external, API key auth via `x-openclaw-key` header) + +| Method | Endpoint | Description | +|--------|----------|-------------| +| `POST` | `/api/openclaw/sync/duo` | Trigger full sync from external agent | + +--- + +## UI Elements + +### Sync Overview Card (`/admin/sync`) + +The Duo card appears on the integrations overview page with: + +- **Category:** 2FA / MFA +- **Color:** Green theme +- **Stats shown:** Accounts / Users, Phones, Auth logs +- **Status icon:** + - Green checkmark — all clear + - Red alert — bypass users detected (MFA not enforced) + - Gray clock — not yet synced + +### Duo Detail Page (`/admin/sync/duo`) + +Full management page with: + +- **Header** — Duo logo, title, "Sync Now" button with progress polling +- **Stat cards** (6) — Accounts, Users, Phones, Auth Logs, Groups, Integrations +- **Last sync timestamp** +- **Bypass Users panel** (expandable, red) — Clickable warning banner shows users with `status = 'bypass'`. These users can authenticate **without MFA** — this is a security risk. Table shows: user, email, account, enrolled status, last login, notes. +- **Disabled Users panel** (expandable, muted gray) — Clickable info banner shows users with `status = 'disabled'`. These users are locked out and cannot authenticate — not a security concern. Same table columns. +- **Parent Account section** — Name, user count, integrations, last sync +- **Child Accounts table** — Sortable list with name, user count, integrations, matched Autotask company (with checkmark), last sync time + +--- + +## Environment Variables + +| Variable | Description | +|----------|-------------| +| `DUOACCOUNTS_INTEGRATION_KEY` | Parent Accounts API integration key | +| `DUOACCOUNTS_SECRET_KEY` | Parent Accounts API secret key | +| `DUOACCOUNTS_API_HOSTNAME` | Parent Accounts API hostname | +| `DUOADMIN_INTEGRATION_KEY` | Parent Admin API integration key (used for parent account only) | +| `DUOADMIN_SECRET_KEY` | Parent Admin API secret key | +| `DUOADMIN_API_HOSTNAME` | Parent Admin API hostname | + +> **Important:** The Accounts API credentials (not Admin API) are used for all child account data access. The client signs requests against each child's `api_hostname` and passes `account_id` as a parameter. + +--- + +## Files + +| File | Purpose | +|------|---------| +| `lib/services/duo-client.ts` | API client — HMAC-SHA1 signing, GET/POST, pagination, rate-limit handling | +| `lib/services/duo-sync-service.ts` | Sync orchestration — full sync, per-account data sync, company matching | +| `migrations/058_create_duo_tables.sql` | Database schema — 6 tables with indexes and foreign keys | +| `app/api/duo/sync/route.ts` | Sync trigger + status endpoint | +| `app/api/duo/status/route.ts` | Connection test + record counts | +| `app/api/duo/accounts/route.ts` | Account listing endpoint | +| `app/api/duo/accounts/[id]/users/route.ts` | Per-account user listing | +| `app/api/duo/users/flagged/route.ts` | Bypass + disabled user listing | +| `app/api/openclaw/sync/duo/route.ts` | External sync trigger (OpenClaw) | +| `app/admin/sync/duo/page.tsx` | Admin detail page | +| `app/admin/sync/page.tsx` | Overview card (Duo entry) | + +--- + +## Current Data Volume + +| Table | Records | +|-------|---------| +| `duo_accounts` | 33 (32 children + 1 parent) | +| `duo_users` | 832 | +| `duo_phones` | 925 | +| `duo_auth_logs` | ~5,900+ (growing with each sync) | +| `duo_groups` | 46 | +| `duo_integrations` | 78 |