lorentz
e3aba93857
Duo API Client (lib/services/duo-client.ts): - HMAC-SHA1 request signing, GET/POST, automatic pagination - Rate-limit handling (429 + Retry-After), configurable timeout - Accounts API: listAccounts() via POST /accounts/v1/account/list - Admin API: getUsers, getPhones, getGroups, getIntegrations, getAuthLogs - Child account access: parent creds signed against child api_hostname + account_id - Factory helpers: getDuoAccountsClient(), getDuoAdminClient() Database (migration 058): - 6 tables: duo_accounts, duo_users, duo_phones, duo_auth_logs, duo_groups, duo_integrations - All with proper FKs, indexes, JSONB fields for capabilities/location/groups Sync Service (lib/services/duo-sync-service.ts): - syncAll() orchestration, per-child sequential sync, incremental auth logs - Company matching: exact then case-insensitive containment (30/32 = 94% matched) - Non-blocking with sync ID tracking API Routes: - POST/GET /api/duo/sync — trigger sync / check status - GET /api/duo/accounts — list all accounts with stats + matched company - GET /api/duo/accounts/[id]/users — users for a specific account - POST /api/openclaw/sync/duo — OpenClaw trigger with API key auth Verified data: 33 accounts, 832 users, 925 phones, 5927 auth logs, 46 groups, 78 integrations Also: entity-mapper company fields update, task list marked complete
65 lines
5.5 KiB
Markdown
65 lines
5.5 KiB
Markdown
# Tasks: Duo Security Integration — Data Sync & Storage
|
|
|
|
> Generated from [prd-duo-integration.md](./prd-duo-integration.md)
|
|
|
|
## Relevant Files
|
|
|
|
- `lib/services/duo-client.ts` - Duo API client with HMAC-SHA1 signing, pagination, rate-limit handling
|
|
- `lib/services/duo-sync-service.ts` - Sync service orchestrating data pull from all Duo accounts into PostgreSQL
|
|
- `migrations/058_create_duo_tables.sql` - Database migration creating 6 Duo tables with indexes and FKs
|
|
- `app/api/duo/sync/route.ts` - Internal API route to trigger/check Duo sync status (POST/GET)
|
|
- `app/api/duo/accounts/route.ts` - API route to list all Duo child accounts with stats (GET)
|
|
- `app/api/duo/accounts/[id]/users/route.ts` - API route to list users for a specific Duo account (GET)
|
|
- `app/api/openclaw/sync/duo/route.ts` - OpenClaw trigger route for Duo sync (POST, API key auth)
|
|
- `middleware.ts` - Add `/api/duo` to public routes
|
|
- `scripts/test-duo.mjs` - Existing test script (already created)
|
|
|
|
### Notes
|
|
|
|
- No new npm packages required — uses Node.js built-in `crypto` and `https`.
|
|
- Duo Accounts API uses POST for all endpoints (including list). Admin API uses GET.
|
|
- Parent Accounts API creds can call Admin API on any child by signing against child's `api_hostname` + passing `account_id`.
|
|
- Migration number 058 is next available.
|
|
|
|
## Tasks
|
|
|
|
- [x] 1.0 Create the Duo API Client (`lib/services/duo-client.ts`)
|
|
- [x] 1.1 Implement `DuoClient` class with constructor accepting `ikey`, `skey`, `host`
|
|
- [x] 1.2 Implement HMAC-SHA1 request signing method (canon string → signature → Basic auth header)
|
|
- [x] 1.3 Implement `get(path, params)` and `post(path, params)` methods with signed HTTPS requests
|
|
- [x] 1.4 Implement automatic pagination — follow `metadata.next_offset` until all pages retrieved
|
|
- [x] 1.5 Implement rate-limit handling — detect HTTP 429, read `Retry-After` header, wait and retry
|
|
- [x] 1.6 Add configurable timeout (default 30s) on all requests
|
|
- [x] 1.7 Implement Accounts API method: `listAccounts()` → `POST /accounts/v1/account/list`
|
|
- [x] 1.8 Implement Admin API methods: `getAccountSummary()`, `getUsers()`, `getPhones()`, `getGroups()`, `getIntegrations()`, `getAuthLogs()`
|
|
- [x] 1.9 Support child account access pattern — accept override `host` + `account_id` param for Admin API calls
|
|
- [x] 2.0 Create Database Migration (`migrations/058_create_duo_tables.sql`)
|
|
- [x] 2.1 Create `duo_accounts` table with all columns from PRD (account_id, name, api_hostname, autotask_company_id FK, user_count, integration_count, edition, is_parent, synced_at, created_at)
|
|
- [x] 2.2 Create `duo_users` table (user_id, duo_account_id FK, username, email, realname, status, is_enrolled, last_login, groups JSONB, aliases JSONB, etc.)
|
|
- [x] 2.3 Create `duo_phones` table (phone_id, duo_account_id FK, name, number, type, platform, model, os_version, activated, last_seen, capabilities JSONB, users JSONB)
|
|
- [x] 2.4 Create `duo_auth_logs` table (txid, duo_account_id FK, timestamp, user_name, factor, result, reason, access_device_ip INET, access_device_location JSONB, etc.)
|
|
- [x] 2.5 Create `duo_groups` table (group_id, duo_account_id FK, name, description, member_count, status)
|
|
- [x] 2.6 Create `duo_integrations` table (integration_key, duo_account_id FK, name, type, enabled, notes)
|
|
- [x] 2.7 Add indexes: duo_account_id on all child tables, timestamp on auth_logs, status on users, is_parent on accounts
|
|
- [x] 3.0 Create the Duo Sync Service (`lib/services/duo-sync-service.ts`)
|
|
- [x] 3.1 Implement `syncAccounts()` — list child accounts via Accounts API, upsert into `duo_accounts`, add parent account row
|
|
- [x] 3.2 Implement `syncAccountData(account)` — for a single account, sync users, phones, groups, integrations via Admin API upserts
|
|
- [x] 3.3 Implement `syncAuthLogs(account, since?)` — incremental auth log sync using `mintime` from last synced timestamp
|
|
- [x] 3.4 Implement `syncAll()` — orchestrate full sync: syncAccounts → loop each child sequentially → syncAccountData + syncAuthLogs → sync parent account
|
|
- [x] 3.5 Implement company matching — after syncing accounts, match `duo_accounts.name` to `companies.company_name` (exact → case-insensitive containment → skip)
|
|
- [x] 3.6 Add sync ID tracking, progress logging, and per-account stats (records added/updated)
|
|
- [x] 4.0 Create Internal API Routes
|
|
- [x] 4.1 Create `app/api/duo/sync/route.ts` — POST to trigger full sync (non-blocking), GET to return sync status
|
|
- [x] 4.2 Create `app/api/duo/accounts/route.ts` — GET to list all Duo accounts with user_count, integration_count, matched company
|
|
- [x] 4.3 Create `app/api/duo/accounts/[id]/users/route.ts` — GET to list users for a specific Duo account
|
|
- [x] 5.0 Create OpenClaw Route and Register in Middleware
|
|
- [x] 5.1 Create `app/api/openclaw/sync/duo/route.ts` — POST with API key auth, triggers `syncAll()` non-blocking
|
|
- [x] 5.2 Add `/api/duo` to publicRoutes in `middleware.ts`
|
|
- [x] 6.0 End-to-End Testing — Run Migration, Build, Sync, Verify
|
|
- [x] 6.1 Run migration 058 against pulse-postgres
|
|
- [x] 6.2 Rebuild and restart the app container
|
|
- [x] 6.3 Trigger full sync via `/api/duo/sync` POST and verify it completes
|
|
- [x] 6.4 Verify all 6 tables populated: duo_accounts (33 rows), duo_users, duo_phones, duo_auth_logs, duo_groups, duo_integrations
|
|
- [x] 6.5 Verify company matching — check duo_accounts.autotask_company_id is populated for matched accounts
|
|
- [x] 6.6 Verify OpenClaw trigger works via `/api/openclaw/sync/duo`
|
|
- [x] 6.7 Git commit all changes
|