feat: Display Settings UI + Company Category/Type sync
- Add /admin/display-settings page with Kiosk and Mobile sections - Company category checkbox filter + excluded companies searchable multi-select - New DB tables: company_categories, company_types (migration 064) - Sync COMPANY_CATEGORIES via CompanyCategories entity (id/name/isActive) - Sync COMPANY_TYPES via Companies.companyType picklist - Add to EntityType, ENTITY_DEPENDENCIES, sync-helpers, entity-mapper, entity-sync - New API routes: /api/admin/display-settings (GET/POST), /api/data/company-categories, /api/data/companies-list - Update all 4 routes (kiosk/stats, kiosk/activity, mobile/tickets, mobile/dashboard) to filter by kiosk_settings company_category_ids + excluded_company_ids - Add Display Settings nav link (SlidersHorizontal icon) to Admin menu - Seed kiosk_settings: kiosk_company_category_ids=1, mobile_company_category_ids=1
This commit is contained in:
parent
89dbe6155b
commit
07067bef19
16 changed files with 847 additions and 85 deletions
156
docs/AUTOTASK_SYNC.md
Normal file
156
docs/AUTOTASK_SYNC.md
Normal file
|
|
@ -0,0 +1,156 @@
|
|||
# Autotask Sync — Reference Guide
|
||||
|
||||
## Overview
|
||||
|
||||
Pulse maintains a local PostgreSQL mirror of Autotask data in the `pulse_autotask` database. Three mechanisms keep it current: **scheduled syncs**, **real-time webhooks**, and **manual/API-triggered syncs**.
|
||||
|
||||
---
|
||||
|
||||
## Automatic Syncs (Scheduled)
|
||||
|
||||
All schedules are managed via the **Admin → Sync Scheduler** UI or directly in the `sync_schedules` table. They can be enabled/disabled individually.
|
||||
|
||||
| Schedule ID | Name | Cron (UTC) | Local (EST) | Type | Enabled |
|
||||
|---|---|---|---|---|---|
|
||||
| `sync-6am` | Morning Sync | `0 11 * * *` | 6:00 AM daily | incremental | ✅ |
|
||||
| `sync-830am` | Morning Sync | `30 13 * * *` | 8:30 AM daily | incremental | ✅ |
|
||||
| `sync-11am` | Midday Sync | `0 16 * * *` | 11:00 AM daily | incremental | ✅ |
|
||||
| `sync-230pm` | Afternoon Sync | `30 19 * * *` | 2:30 PM daily | incremental | ✅ |
|
||||
| `weekly-full` | Weekly Full Sync | `0 3 * * 0` | 3:00 AM Sunday | full | ✅ |
|
||||
| `contract-services` | Contract Services | `0 4 * * *` | 4:00 AM daily | contract-services | ✅ |
|
||||
|
||||
### Incremental vs Full
|
||||
|
||||
**Incremental sync** — pulls only records modified since the last successful sync timestamp (stored in `sync_history`). Much faster. Runs 4× per day.
|
||||
|
||||
**Full sync** — pulls everything regardless of modification date, going back `yearsBack` years (default: 2). Runs every Sunday at 3 AM. Also used for initial setup or repair.
|
||||
|
||||
### What incremental syncs cover
|
||||
|
||||
Every incremental run syncs all entities in dependency order:
|
||||
|
||||
1. Companies, Resources, Statuses, Issue Types, Sub-Issue Types, Work Types, Queues, Priorities, Ticket Categories *(no dependencies)*
|
||||
2. Contacts *(requires Companies)*
|
||||
3. Projects *(requires Companies, Resources)*
|
||||
4. Tickets *(requires Companies, Resources, Contacts)*
|
||||
5. Tasks *(requires Resources, Projects, Tickets)*
|
||||
6. Configuration Items *(requires Companies, Contacts)*
|
||||
7. Contracts *(requires Companies, Contacts)*
|
||||
8. Autotask Services *(standalone)*
|
||||
9. Billing Items *(requires Companies, Tasks, Tickets, Projects)*
|
||||
10. Time Entries *(requires Companies, Resources, Contacts, Projects, Tasks, Tickets)*
|
||||
11. Ticket Notes *(requires Tickets)*
|
||||
12. Tag Groups, Tags, Ticket Tag Associations
|
||||
|
||||
> **Note:** Companies and Resources do not support date-based filtering in the Autotask API, so they always do a full pull on every run (fast — ~57 resources, ~236 companies).
|
||||
|
||||
### Time-windowed entities
|
||||
|
||||
Some entities are too large to sync in full and are filtered by date:
|
||||
|
||||
| Entity | Filter field | Default window |
|
||||
|---|---|---|
|
||||
| Tickets | `createDate` | Last 2 years |
|
||||
| Tasks | `createDate` | Last 2 years |
|
||||
| Time Entries | `dateWorked` | Last 2 years |
|
||||
| Billing Items | `itemDate` | Last 2 years |
|
||||
|
||||
The `yearsBack` parameter (default `2`) controls how far back these go. The full sync and chunked sync accept a custom `yearsBack` value.
|
||||
|
||||
---
|
||||
|
||||
## Real-Time Webhooks
|
||||
|
||||
Autotask pushes change events to `/api/webhooks/autotask` immediately when a ticket (or other entity) is created or updated. The webhook handler fetches the full record from the Autotask API and upserts it directly — no waiting for the next scheduled sync.
|
||||
|
||||
This means tickets assigned, updated, or closed in Autotask appear in Pulse within seconds. The webhook path **bypasses** the resource ID validation step, so it always writes exactly what Autotask sends.
|
||||
|
||||
---
|
||||
|
||||
## Manual / API-Triggered Syncs
|
||||
|
||||
### Via Admin UI
|
||||
|
||||
**Admin → Sync** — the sync overview page has buttons to trigger syncs per integration.
|
||||
|
||||
### Via API endpoints
|
||||
|
||||
All endpoints are POST and non-blocking (fire-and-forget) — they return immediately and run in the background.
|
||||
|
||||
| Endpoint | Payload | Description |
|
||||
|---|---|---|
|
||||
| `POST /api/sync/full` | `{ yearsBack?, triggeredBy? }` | Full sync of all entities |
|
||||
| `POST /api/sync/entity` | `{ entities: [...], syncType?, yearsBack? }` | Sync specific entity types only |
|
||||
| `POST /api/sync/tickets-chunked` | `{ yearsBack? }` | Chunked ticket sync with progress logging (good for large date ranges) |
|
||||
|
||||
**Example — sync only tickets and resources:**
|
||||
```bash
|
||||
curl -X POST http://localhost:3100/api/sync/entity \
|
||||
-H "Content-Type: application/json" \
|
||||
-d '{"entities": ["tickets", "resources"], "triggeredBy": "manual"}'
|
||||
```
|
||||
|
||||
**Example — full sync going back 3 years:**
|
||||
```bash
|
||||
curl -X POST http://localhost:3100/api/sync/full \
|
||||
-H "Content-Type: application/json" \
|
||||
-d '{"yearsBack": 3, "triggeredBy": "manual"}'
|
||||
```
|
||||
|
||||
### Via OpenClaw (external agent API)
|
||||
|
||||
```bash
|
||||
# Incremental sync
|
||||
POST /api/openclaw/sync/autotask/incremental
|
||||
|
||||
# Full sync
|
||||
POST /api/openclaw/sync/autotask/full { "yearsBack": 2 }
|
||||
|
||||
# Specific entities
|
||||
POST /api/openclaw/sync/autotask/entity { "entities": ["tickets"] }
|
||||
```
|
||||
Auth: `x-openclaw-key: <OPENCLAW_API_KEY>`
|
||||
|
||||
---
|
||||
|
||||
## Valid Entity Names
|
||||
|
||||
Use these string values in the `entities` array:
|
||||
|
||||
```
|
||||
companies, resources, statuses, issue_types, sub_issue_types, work_types,
|
||||
queues, priorities, ticket_categories, contacts, projects, tickets, tasks,
|
||||
configuration_items, contracts, contract_services, autotask_services,
|
||||
billing_items, time_entries, ticket_notes, tag_groups, tags
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Sync History & Status
|
||||
|
||||
Every sync run writes to `sync_history`. You can query it directly:
|
||||
|
||||
```sql
|
||||
-- Last sync per entity type
|
||||
SELECT entity_type, sync_type, status, records_added, records_updated, completed_at
|
||||
FROM sync_history
|
||||
WHERE status = 'completed'
|
||||
ORDER BY completed_at DESC;
|
||||
|
||||
-- Check when tickets last synced successfully
|
||||
SELECT completed_at FROM sync_history
|
||||
WHERE entity_type = 'tickets' AND status = 'completed'
|
||||
ORDER BY completed_at DESC LIMIT 1;
|
||||
```
|
||||
|
||||
The incremental sync uses `completed_at` from `sync_history` to determine how far back to pull. **If a sync fails, the timestamp does not advance** — the next run re-fetches from the last successful point.
|
||||
|
||||
---
|
||||
|
||||
## Known Constraints & Gotchas
|
||||
|
||||
- **One sync at a time.** If a sync is already running, new requests return `409 Conflict`.
|
||||
- **`assigned_resource_id` is the only resource assignment on a ticket.** There is no separate "primary resource" — `assigned_resource_id` is it. Other resource fields (`first_response_assigned_resource_id`, `last_activity_resource_id`, `creator_resource_id`) are audit/SLA tracking fields.
|
||||
- **BIGINT IDs come back as strings from pg.** The sync code uses `Number()` coercion when building validation sets to avoid false-negative ID comparisons.
|
||||
- **FK constraints on tickets are deferrable.** `tickets_company_id_fkey` and all resource FKs are `DEFERRABLE INITIALLY DEFERRED ON DELETE SET NULL`. This means a ticket can be inserted even if its referenced company or resource isn't in the local DB yet — the field is set to NULL rather than rejecting the row.
|
||||
- **Webhook and scheduled sync can race.** If a webhook fires during a scheduled sync batch, it may be overwritten by the batch. This is harmless — both write the same Autotask data.
|
||||
Loading…
Add table
Add a link
Reference in a new issue