wulf-pulse/docs/AUTOTASK_SYNC.md

# Autotask Sync — Reference Guide

## Overview

Pulse maintains a local PostgreSQL mirror of Autotask data in the `pulse_autotask` database. Three mechanisms keep it current: **scheduled syncs**, **real-time webhooks**, and **manual/API-triggered syncs**.

---

## Automatic Syncs (Scheduled)

All schedules are managed via the **Admin → Sync Scheduler** UI or directly in the `sync_schedules` table. They can be enabled/disabled individually.

| Schedule ID | Name | Cron (UTC) | Local (EST) | Type | Enabled |
|---|---|---|---|---|---|
| `sync-6am` | Morning Sync | `0 11 * * *` | 6:00 AM daily | incremental | ✅ |
| `sync-830am` | Morning Sync | `30 13 * * *` | 8:30 AM daily | incremental | ✅ |
| `sync-11am` | Midday Sync | `0 16 * * *` | 11:00 AM daily | incremental | ✅ |
| `sync-230pm` | Afternoon Sync | `30 19 * * *` | 2:30 PM daily | incremental | ✅ |
| `weekly-full` | Weekly Full Sync | `0 3 * * 0` | 3:00 AM Sunday | full | ✅ |
| `contract-services` | Contract Services | `0 4 * * *` | 4:00 AM daily | contract-services | ✅ |

### Incremental vs Full

**Incremental sync** — pulls only records modified since the last successful sync timestamp (stored in `sync_history`). Much faster. Runs 4× per day.

**Full sync** — pulls everything regardless of modification date, going back `yearsBack` years (default: 2). Runs every Sunday at 3 AM. Also used for initial setup or repair.

### What incremental syncs cover

Every incremental run syncs all entities in dependency order:

1. Companies, Resources, Statuses, Issue Types, Sub-Issue Types, Work Types, Queues, Priorities, Ticket Categories *(no dependencies)*
2. Contacts *(requires Companies)*
3. Projects *(requires Companies, Resources)*
4. Tickets *(requires Companies, Resources, Contacts)*
5. Tasks *(requires Resources, Projects, Tickets)*
6. Configuration Items *(requires Companies, Contacts)*
7. Contracts *(requires Companies, Contacts)*
8. Autotask Services *(standalone)*
9. Billing Items *(requires Companies, Tasks, Tickets, Projects)*
10. Time Entries *(requires Companies, Resources, Contacts, Projects, Tasks, Tickets)*
11. Ticket Notes *(requires Tickets)*
12. Tag Groups, Tags, Ticket Tag Associations

> **Note:** Companies and Resources do not support date-based filtering in the Autotask API, so they always do a full pull on every run (fast — ~57 resources, ~236 companies).

### Time-windowed entities

Some entities are too large to sync in full and are filtered by date:

| Entity | Filter field | Default window |
|---|---|---|
| Tickets | `createDate` | Last 2 years |
| Tasks | `createDate` | Last 2 years |
| Time Entries | `dateWorked` | Last 2 years |
| Billing Items | `itemDate` | Last 2 years |

The `yearsBack` parameter (default `2`) controls how far back these go. The full sync and chunked sync accept a custom `yearsBack` value.

---

## Real-Time Webhooks

Autotask pushes change events to `/api/webhooks/autotask` immediately when a ticket (or other entity) is created or updated. The webhook handler fetches the full record from the Autotask API and upserts it directly — no waiting for the next scheduled sync.

This means tickets assigned, updated, or closed in Autotask appear in Pulse within seconds. The webhook path **bypasses** the resource ID validation step, so it always writes exactly what Autotask sends.

---

## Manual / API-Triggered Syncs

### Via Admin UI

**Admin → Sync** — the sync overview page has buttons to trigger syncs per integration.

### Via API endpoints

All endpoints are POST and non-blocking (fire-and-forget) — they return immediately and run in the background.

| Endpoint | Payload | Description |
|---|---|---|
| `POST /api/sync/full` | `{ yearsBack?, triggeredBy? }` | Full sync of all entities |
| `POST /api/sync/entity` | `{ entities: [...], syncType?, yearsBack? }` | Sync specific entity types only |
| `POST /api/sync/tickets-chunked` | `{ yearsBack? }` | Chunked ticket sync with progress logging (good for large date ranges) |

**Example — sync only tickets and resources:**
```bash
curl -X POST http://localhost:3100/api/sync/entity \
  -H "Content-Type: application/json" \
  -d '{"entities": ["tickets", "resources"], "triggeredBy": "manual"}'
```

**Example — full sync going back 3 years:**
```bash
curl -X POST http://localhost:3100/api/sync/full \
  -H "Content-Type: application/json" \
  -d '{"yearsBack": 3, "triggeredBy": "manual"}'
```

### Via OpenClaw (external agent API)

```bash
# Incremental sync
POST /api/openclaw/sync/autotask/incremental

# Full sync
POST /api/openclaw/sync/autotask/full          { "yearsBack": 2 }

# Specific entities
POST /api/openclaw/sync/autotask/entity        { "entities": ["tickets"] }
```
Auth: `x-openclaw-key: <OPENCLAW_API_KEY>`

---

## Valid Entity Names

Use these string values in the `entities` array:

```
companies, resources, statuses, issue_types, sub_issue_types, work_types,
queues, priorities, ticket_categories, contacts, projects, tickets, tasks,
configuration_items, contracts, contract_services, autotask_services,
billing_items, time_entries, ticket_notes, tag_groups, tags
```

---

## Sync History & Status

Every sync run writes to `sync_history`. You can query it directly:

```sql
-- Last sync per entity type
SELECT entity_type, sync_type, status, records_added, records_updated, completed_at
FROM sync_history
WHERE status = 'completed'
ORDER BY completed_at DESC;

-- Check when tickets last synced successfully
SELECT completed_at FROM sync_history
WHERE entity_type = 'tickets' AND status = 'completed'
ORDER BY completed_at DESC LIMIT 1;
```

The incremental sync uses `completed_at` from `sync_history` to determine how far back to pull. **If a sync fails, the timestamp does not advance** — the next run re-fetches from the last successful point.

---

## Known Constraints & Gotchas

- **One sync at a time.** If a sync is already running, new requests return `409 Conflict`.
- **`assigned_resource_id` is the only resource assignment on a ticket.** There is no separate "primary resource" — `assigned_resource_id` is it. Other resource fields (`first_response_assigned_resource_id`, `last_activity_resource_id`, `creator_resource_id`) are audit/SLA tracking fields.
- **BIGINT IDs come back as strings from pg.** The sync code uses `Number()` coercion when building validation sets to avoid false-negative ID comparisons.
- **FK constraints on tickets are deferrable.** `tickets_company_id_fkey` and all resource FKs are `DEFERRABLE INITIALLY DEFERRED ON DELETE SET NULL`. This means a ticket can be inserted even if its referenced company or resource isn't in the local DB yet — the field is set to NULL rather than rejecting the row.
- **Webhook and scheduled sync can race.** If a webhook fires during a scheduled sync batch, it may be overwritten by the batch. This is harmless — both write the same Autotask data.