Commit graph

14 commits

Author SHA1 Message Date
e3aba93857 feat: Duo Security integration — full data sync from Accounts + Admin API
Duo API Client (lib/services/duo-client.ts):
- HMAC-SHA1 request signing, GET/POST, automatic pagination
- Rate-limit handling (429 + Retry-After), configurable timeout
- Accounts API: listAccounts() via POST /accounts/v1/account/list
- Admin API: getUsers, getPhones, getGroups, getIntegrations, getAuthLogs
- Child account access: parent creds signed against child api_hostname + account_id
- Factory helpers: getDuoAccountsClient(), getDuoAdminClient()

Database (migration 058):
- 6 tables: duo_accounts, duo_users, duo_phones, duo_auth_logs, duo_groups, duo_integrations
- All with proper FKs, indexes, JSONB fields for capabilities/location/groups

Sync Service (lib/services/duo-sync-service.ts):
- syncAll() orchestration, per-child sequential sync, incremental auth logs
- Company matching: exact then case-insensitive containment (30/32 = 94% matched)
- Non-blocking with sync ID tracking

API Routes:
- POST/GET /api/duo/sync — trigger sync / check status
- GET /api/duo/accounts — list all accounts with stats + matched company
- GET /api/duo/accounts/[id]/users — users for a specific account
- POST /api/openclaw/sync/duo — OpenClaw trigger with API key auth

Verified data: 33 accounts, 832 users, 925 phones, 5927 auth logs, 46 groups, 78 integrations

Also: entity-mapper company fields update, task list marked complete
2026-03-27 11:10:30 -04:00
9a448d111c feat: contacts sync — add all missing fields + UDFs
Migration 057:
- ADD user_defined_fields JSONB (GIN indexed)
- ADD middle_initial, note, external_id, country_id, company_location_id
- ADD create_date, impersonator_creator_resource_id
- ADD is_opted_out_from_bulk_email, bulk_email_opt_out_time
- ADD solicitation_opt_out_time, survey_opt_out
- ADD receives_email_notifications, billing_contact

entity-mapper.ts mapContact():
- Fix broken snake_case field refs → correct camelCase API names:
  alternate_phone → alternatePhone, mobile_phone → mobilePhone
  name_prefix/suffix → namePrefix/nameSuffix
  facebook/twitter/linkedin_url → facebookUrl/twitterUrl/linkedInUrl
  primary_contact → primaryContact, solicitation_opt_out → solicitationOptOut
  last_activity/modified_date → lastActivityDate/lastModifiedDate
  api_vendor_id → apiVendorID, is_active → isActive, is_deleted → isDeleted
- Add UDF conversion: userDefinedFields[] → JSONB {name: value}
- 17 UDFs stored: UserID, Birthday, O365License, VIP User, Department,
  Password, Email Password, Archive Email, User System Profile, etc.

Results: 4234 contacts synced, 3318 with UDFs, 4140 with create_date
2026-03-26 13:16:13 -04:00
dd4cf68def feat: add project_phases entity sync with task project_id backfill
The Autotask Tasks bulk API does not return projectID in its response,
causing all tasks.project_id to be NULL. This fixes it by:

- Adding project_phases as a synced entity (Autotask endpoint: /Phases)
- Migration 059: project_phases table with project_id, phase_number,
  estimated_hours, start/due dates, parent_phase_id, is_scheduled
- EntityType.PROJECT_PHASES added to all sync maps and dependency graph
  (depends on PROJECTS, runs before TASKS in sync order)
- buildProjectPhasesFilter: Phases endpoint requires a filter (id > 0)
- mapProjectPhase: maps Autotask field names to DB columns
- Post-sync backfill in syncEntity: after each project_phases sync,
  UPDATE tasks SET project_id = pp.project_id FROM project_phases pp
  JOIN projects p WHERE tasks.phase_id = pp.id
  Only backfills where the project exists in our DB (FK constraint on
  tasks.project_id; archived projects are skipped gracefully)

Result: 2,455 of 4,966 tasks now have project_id populated. Tasks
belonging to archived/completed projects have phase_id resolvable via
project_phases even when project_id remains NULL.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-24 13:57:33 -04:00
ff9e34cafe feat: add Autotask tags sync (tag groups, tags, ticket tag associations)
- Migration 057: autotask_tag_groups, autotask_tags, and junction tables
  (ticket_tags, company_tags, configuration_item_tags, contact_tags)
- Add TAG_GROUPS and TAGS to EntityType enum and dependency map
- Add mapTagGroup() and mapTag() entity mapper functions
- Add syncTagGroups(), syncTags(), syncTicketTagAssociations() methods
- Wire TicketTagAssociations bulk sync into full/incremental sync flow
- Add 'exist' operator to QueryFilter type
- No FK on ticket_id (tagged tickets may be outside 2yr sync window)

Synced: 26 tag groups, 7299 tags, 10141 ticket-tag associations
2026-03-20 09:22:40 -04:00
c8be128c5c feat: companies sync — add UDFs and fix all field mappings
Migration 056:
- ADD user_defined_fields JSONB (GIN indexed)
- ADD parent_company_id, web_address, create_date, company_category_id
- ADD is_client_portal_active, is_task_fire_active, is_comanaged
- ADD country_id, bill_to_country_id, bill_to_address_to_use
- ADD invoice_email_message_id, purchase_order_template_id
- ADD additional_address_information, sic_code, stock_market, etc.

entity-mapper.ts mapCompany():
- Fix broken snake_case field refs (was data.invoice_method, data.last_activity_date etc.)
- All corrected to actual Autotask camelCase API names (invoiceMethod, lastActivityDate, etc.)
- was data.webSiteURL → data.webAddress
- was data.taxExempt → data.isTaxExempt
- was data.lastTrackedModifiedDateTime (various) → data.lastTrackedModifiedDateTime
- Add UDF conversion: userDefinedFields[] array → JSONB {name: value} object
- 38 UDFs now stored: PassportalID, MimecastID, MSP Service Model, Short Name,
  Workstation Lifecycle Plan, Seats (End Users), CSProfileUID, UniFiID, etc.
2026-03-17 23:56:40 -04:00
cb293ea843 fix: configuration_items sync — correct camelCase field mapping, store all UDFs in udfs JSONB
- Fix mapConfigurationItem() to use Autotask camelCase field names (was using
  snake_case keys that never matched, leaving costs/Datto/RMM fields empty)
- Map all 60+ fields explicitly: costs, Datto device fields, RMM audit fields,
  location, vendor, SNMP, backup dates, source cost, etc.
- Store entire userDefinedFields array as udfs JSONB (keyed by UDF name)
- Retain backup_type_udf VARCHAR column for backward compat
- Migration 052: add udfs JSONB column + GIN index to configuration_items
2026-03-17 15:39:42 -04:00
c518eefdb2 feat: Morning NOC Summary adaptive card for Teams
- Add MorningSummaryService with Zabbix aggregation and adaptive card builder
- Add webhook delivery system with Teams incoming webhooks
- Add admin UI at /admin/morning-summary for webhook/config management
- Add API routes: /send, /test, /webhooks, /webhooks/[id], /config, /history
- Register morning-summary cron job in SyncScheduler (Mon-Fri 6:30 AM)
- Add outages_only filter (Unavailable triggers only)
- Fix host resolution: use getTriggerEnabledHosts to exclude disabled hosts
- Fix resolved events: event.get value:1 scoped to window with r_eventid filter
- Remove emojis from fact rows and section headers in card
- Remove Open Zabbix button (duplicate of View Problems)
- Add migrations: morning_summary_config + morning_summaries tables
- Add outages_only column to morning_summary_config
2026-03-11 09:34:51 -04:00
d7c3dc7168 feat: Autotask webhook integration, TicketNotes, Datto RMM, workflow engine, Veeam agents/alarms, AI triage, misc improvements 2026-02-20 10:28:15 -05:00
5dc7a7e66b feat: Veeam VSPC backup integration - sync, compliance, UI
- Database: 7 Veeam tables + backup_type_udf column on configuration_items
- API Client: VSPC REST API v3 client with pagination, rate limiting, Bearer auth
- Sync Service: full/incremental sync for orgs, servers, repos, jobs, agent jobs, workloads
- Scheduler: veeam-incremental (30min) and veeam-full (daily 2AM) schedules
- Compliance Engine: cross-references Autotask config items vs Veeam workloads
- API Endpoints: backup-status, companies, workloads, jobs, repos, compliance, sync
- UI: Backup Status page with Overview + Contract Compliance tabs
- Navigation: added Backup Status link with HardDrive icon
- Docker: added VEEAM_VSPC_URL and VEEAM_VSPC_API_KEY env vars to compose
2026-02-11 21:04:28 -05:00
root
8405c813a3 feat: add missing Tickets entity fields from Autotask API
Added 10 critical missing fields to Tickets entity:

Database Migration (016):
- billing_code_id: Billing code assignment
- configuration_item_id: Primary asset/CI
- creator_resource_id: Ticket creator
- creator_type: Creator type (resource/contact)
- problem_ticket_id: Link to problem ticket
- rma_status: RMA status tracking
- rma_type: RMA type classification
- service_level_agreement_paused_next_event_hours: SLA pause tracking
- is_assigned_to_comanaged: Co-managed assignment flag
- is_visible_to_comanaged: Co-managed visibility flag

Entity Mapper Updates:
- Added all new fields with correct camelCase mapping
- Ensures all Autotask Tickets API fields are captured

This completes the Tickets entity to match the full Autotask API
specification and should resolve issues with missing ticket data.
2026-01-26 15:26:34 -05:00
root
cc16d9b32d fix: correct Tickets entity field name mappings from Autotask API
Fixed field names to use proper camelCase from Autotask API instead of
snake_case. This should resolve the issue where assigned_resource_id
was NULL because the mapper was looking for incorrect field names.

Changed mappings:
- firstResponseDateTime (was first_response_date_time)
- resolutionPlanDateTime (was resolution_plan_date_time)
- resolvedDateTime (was resolved_date_time)
- firstResponseAssignedResourceID (was first_response_assigned_resource_id)
- firstResponseInitiatingResourceID (was first_response_initiating_resource_id)
- projectID, opportunityID, contractID, monitorID (were snake_case)
- All change approval and ticket fields converted to camelCase
- All service thermometer fields converted to camelCase

This fix should populate assigned_resource_id and other fields correctly
on the next ticket sync.
2026-01-26 15:25:03 -05:00
root
bc01a6aafb fix: handle non-numeric values in Resources integer fields
Autotask API returns string values like 'PRIMARY' for some integer
fields (e.g., emailTypeCode). Added safeInt() helper function to:
- Parse numeric values correctly
- Return null for non-numeric strings
- Handle null/undefined/empty values

This prevents database errors when syncing Resources with non-standard
field values from Autotask API.
2026-01-26 14:48:37 -05:00
root
8e0ca0a63b feat: expand Resources entity to include all Autotask API fields
Add comprehensive field support for Resources entity including:
- Name fields: middleInitial, namePrefix, nameSuffix
- Contact: emailAddress2, emailAddress3, homePhone
- Employment: accountingReferenceID, payrollType, internalCost
- System: emailTypeCode, numberFormat, timeFormat, dateFormat
- Demographics: gender
- Security: licenseType, securityLevel
- Location: defaultServiceDeskRoleID

Changes:
- Migration 015: Add new columns to resources table
- Updated entity mapper to map all Autotask Resource fields
- Expanded TypeScript Resource interface with all fields
- Maintains backward compatibility with existing field names

This ensures all data exposed by the Autotask Resources API is
now captured and stored in the database.
2026-01-26 14:28:51 -05:00
root
6eee14f8af Add comprehensive admin features and multi-system integration
- Add admin dashboard with sync controls and data browser
- Implement RMM, Auvik, and Addigy organization mappings
- Add chunked ticket sync with progress tracking
- Implement entity sync service with rate limiting
- Add analytics engine and performance optimizer
- Create data browser for all PSA entities
- Add navigation components and UI improvements
- Implement background processing and sync services
- Add comprehensive documentation and migration scripts
- Update configuration items with multi-system support
- Enhance contact management and purchase history
- Add issue type assignment and LLM analyzer
- Improve error handling and logging utilities
2025-11-19 14:18:16 -05:00