Duo API Client (lib/services/duo-client.ts): - HMAC-SHA1 request signing, GET/POST, automatic pagination - Rate-limit handling (429 + Retry-After), configurable timeout - Accounts API: listAccounts() via POST /accounts/v1/account/list - Admin API: getUsers, getPhones, getGroups, getIntegrations, getAuthLogs - Child account access: parent creds signed against child api_hostname + account_id - Factory helpers: getDuoAccountsClient(), getDuoAdminClient() Database (migration 058): - 6 tables: duo_accounts, duo_users, duo_phones, duo_auth_logs, duo_groups, duo_integrations - All with proper FKs, indexes, JSONB fields for capabilities/location/groups Sync Service (lib/services/duo-sync-service.ts): - syncAll() orchestration, per-child sequential sync, incremental auth logs - Company matching: exact then case-insensitive containment (30/32 = 94% matched) - Non-blocking with sync ID tracking API Routes: - POST/GET /api/duo/sync — trigger sync / check status - GET /api/duo/accounts — list all accounts with stats + matched company - GET /api/duo/accounts/[id]/users — users for a specific account - POST /api/openclaw/sync/duo — OpenClaw trigger with API key auth Verified data: 33 accounts, 832 users, 925 phones, 5927 auth logs, 46 groups, 78 integrations Also: entity-mapper company fields update, task list marked complete
5.5 KiB
5.5 KiB
Tasks: Duo Security Integration — Data Sync & Storage
Generated from prd-duo-integration.md
Relevant Files
lib/services/duo-client.ts- Duo API client with HMAC-SHA1 signing, pagination, rate-limit handlinglib/services/duo-sync-service.ts- Sync service orchestrating data pull from all Duo accounts into PostgreSQLmigrations/058_create_duo_tables.sql- Database migration creating 6 Duo tables with indexes and FKsapp/api/duo/sync/route.ts- Internal API route to trigger/check Duo sync status (POST/GET)app/api/duo/accounts/route.ts- API route to list all Duo child accounts with stats (GET)app/api/duo/accounts/[id]/users/route.ts- API route to list users for a specific Duo account (GET)app/api/openclaw/sync/duo/route.ts- OpenClaw trigger route for Duo sync (POST, API key auth)middleware.ts- Add/api/duoto public routesscripts/test-duo.mjs- Existing test script (already created)
Notes
- No new npm packages required — uses Node.js built-in
cryptoandhttps. - Duo Accounts API uses POST for all endpoints (including list). Admin API uses GET.
- Parent Accounts API creds can call Admin API on any child by signing against child's
api_hostname+ passingaccount_id. - Migration number 058 is next available.
Tasks
- 1.0 Create the Duo API Client (
lib/services/duo-client.ts)- 1.1 Implement
DuoClientclass with constructor acceptingikey,skey,host - 1.2 Implement HMAC-SHA1 request signing method (canon string → signature → Basic auth header)
- 1.3 Implement
get(path, params)andpost(path, params)methods with signed HTTPS requests - 1.4 Implement automatic pagination — follow
metadata.next_offsetuntil all pages retrieved - 1.5 Implement rate-limit handling — detect HTTP 429, read
Retry-Afterheader, wait and retry - 1.6 Add configurable timeout (default 30s) on all requests
- 1.7 Implement Accounts API method:
listAccounts()→POST /accounts/v1/account/list - 1.8 Implement Admin API methods:
getAccountSummary(),getUsers(),getPhones(),getGroups(),getIntegrations(),getAuthLogs() - 1.9 Support child account access pattern — accept override
host+account_idparam for Admin API calls
- 1.1 Implement
- 2.0 Create Database Migration (
migrations/058_create_duo_tables.sql)- 2.1 Create
duo_accountstable with all columns from PRD (account_id, name, api_hostname, autotask_company_id FK, user_count, integration_count, edition, is_parent, synced_at, created_at) - 2.2 Create
duo_userstable (user_id, duo_account_id FK, username, email, realname, status, is_enrolled, last_login, groups JSONB, aliases JSONB, etc.) - 2.3 Create
duo_phonestable (phone_id, duo_account_id FK, name, number, type, platform, model, os_version, activated, last_seen, capabilities JSONB, users JSONB) - 2.4 Create
duo_auth_logstable (txid, duo_account_id FK, timestamp, user_name, factor, result, reason, access_device_ip INET, access_device_location JSONB, etc.) - 2.5 Create
duo_groupstable (group_id, duo_account_id FK, name, description, member_count, status) - 2.6 Create
duo_integrationstable (integration_key, duo_account_id FK, name, type, enabled, notes) - 2.7 Add indexes: duo_account_id on all child tables, timestamp on auth_logs, status on users, is_parent on accounts
- 2.1 Create
- 3.0 Create the Duo Sync Service (
lib/services/duo-sync-service.ts)- 3.1 Implement
syncAccounts()— list child accounts via Accounts API, upsert intoduo_accounts, add parent account row - 3.2 Implement
syncAccountData(account)— for a single account, sync users, phones, groups, integrations via Admin API upserts - 3.3 Implement
syncAuthLogs(account, since?)— incremental auth log sync usingmintimefrom last synced timestamp - 3.4 Implement
syncAll()— orchestrate full sync: syncAccounts → loop each child sequentially → syncAccountData + syncAuthLogs → sync parent account - 3.5 Implement company matching — after syncing accounts, match
duo_accounts.nametocompanies.company_name(exact → case-insensitive containment → skip) - 3.6 Add sync ID tracking, progress logging, and per-account stats (records added/updated)
- 3.1 Implement
- 4.0 Create Internal API Routes
- 4.1 Create
app/api/duo/sync/route.ts— POST to trigger full sync (non-blocking), GET to return sync status - 4.2 Create
app/api/duo/accounts/route.ts— GET to list all Duo accounts with user_count, integration_count, matched company - 4.3 Create
app/api/duo/accounts/[id]/users/route.ts— GET to list users for a specific Duo account
- 4.1 Create
- 5.0 Create OpenClaw Route and Register in Middleware
- 5.1 Create
app/api/openclaw/sync/duo/route.ts— POST with API key auth, triggerssyncAll()non-blocking - 5.2 Add
/api/duoto publicRoutes inmiddleware.ts
- 5.1 Create
- 6.0 End-to-End Testing — Run Migration, Build, Sync, Verify
- 6.1 Run migration 058 against pulse-postgres
- 6.2 Rebuild and restart the app container
- 6.3 Trigger full sync via
/api/duo/syncPOST and verify it completes - 6.4 Verify all 6 tables populated: duo_accounts (33 rows), duo_users, duo_phones, duo_auth_logs, duo_groups, duo_integrations
- 6.5 Verify company matching — check duo_accounts.autotask_company_id is populated for matched accounts
- 6.6 Verify OpenClaw trigger works via
/api/openclaw/sync/duo - 6.7 Git commit all changes