wulf-pulse/docs/WEBHOOK_SETUP.md
root 1f83456199 feat: add webhook support for real-time Autotask updates
Implements comprehensive webhook infrastructure to receive and process
real-time entity updates from Autotask, reducing API calls and improving
data freshness.

Features:
- Webhook receiver endpoint: POST /api/webhooks/autotask
- Automatic entity mapping and upsert to PostgreSQL
- Event logging and tracking in webhook_logs table
- Duplicate event prevention via unique event_id
- Failed event tracking with error messages
- Statistics and monitoring APIs
- Support for 8 entity types: Companies, Tickets, Tasks, Projects,
  Time Entries, Contacts, Contracts, Configuration Items

Architecture:
- WebhookService: Core processing logic
- Database tables: webhook_logs, webhook_configs
- API endpoints: /autotask (receiver), /logs, /stats
- Automatic data mapping using existing entity-mapper

Benefits:
- Near real-time updates (<1 minute vs 24 hours)
- Reduced API usage (webhooks vs polling)
- Complements daily incremental sync for redundancy
- Automatic recovery from webhook failures

Files Added:
- lib/types/webhook.ts - TypeScript types and interfaces
- lib/services/webhook-service.ts - Webhook processing service
- app/api/webhooks/autotask/route.ts - Webhook receiver
- app/api/webhooks/logs/route.ts - Logs API
- app/api/webhooks/stats/route.ts - Statistics API
- migrations/004_webhook_support.sql - Database schema
- docs/WEBHOOK_SETUP.md - Complete setup guide (47 sections)
- docs/WEBHOOKS_README.md - Quick start guide

Next Steps:
1. Run database migration
2. Configure webhooks in Autotask
3. Test endpoint and monitor logs

See docs/WEBHOOK_SETUP.md for detailed setup instructions.
2026-01-24 10:07:20 -05:00

12 KiB

Autotask Webhook Setup Guide

Overview

This guide explains how to configure Autotask webhooks for real-time data synchronization with Pulse. Webhooks provide near-instant updates when entities change in Autotask, reducing API calls and keeping your data fresh.


Table of Contents


Benefits of Webhooks

Real-Time Updates

  • Instant notification when entities are created or updated in Autotask
  • No polling delay - changes appear within seconds

Reduced API Usage

  • Fewer API calls compared to frequent polling
  • Lower risk of hitting API rate limits
  • More efficient use of resources

Better Data Freshness

  • Combined with daily incremental syncs for redundancy
  • Ensures no events are missed
  • Automatic recovery from webhook failures

Supported Entities

The following entities support webhook notifications:

Entity Events Notes
Companies Create, Update Real-time company changes
Tickets Create, Update New tickets and status updates
Tasks Create, Update Task creation and modifications
Projects Create, Update Project changes
Time Entries Create, Update New time entries
Contacts Create, Update Contact information changes
Contracts Create, Update Contract updates
Configuration Items Create, Update Asset changes

Note: Delete events are supported but less common in Autotask workflows.


Prerequisites

Before setting up webhooks, ensure you have:

  1. Autotask API Access

    • API user with appropriate permissions
    • API integration enabled in Autotask
  2. Public Webhook Endpoint

    • Your Pulse instance must be accessible from the internet
    • HTTPS endpoint (required by Autotask)
    • Example: https://your-pulse-instance.com/api/webhooks/autotask
  3. Database Migration

    • Run migration 004_webhook_support.sql to create webhook tables
    docker exec pulse-postgres psql -U pulse_user -d pulse_autotask -f /migrations/004_webhook_support.sql
    

Setup Steps

Step 1: Configure Webhook Endpoint in Autotask

  1. Log in to Autotask as an administrator
  2. Navigate to AdminFeatures & SettingsAPI & Integrations
  3. Click on Webhooks
  4. Click New Webhook

Step 2: Create Webhook for Each Entity

For each entity you want to track (e.g., Tickets):

  1. Webhook Name: Pulse - Tickets
  2. Endpoint URL: https://your-pulse-instance.com/api/webhooks/autotask
  3. Entity Type: Select the entity (e.g., Tickets)
  4. Events: Select events to track:
    • Create
    • Update
    • Delete (optional)
  5. Include Entity Data: Enabled (recommended)
    • This includes full entity data in the webhook payload
    • Reduces need for additional API calls
  6. Active: Enabled
  7. Click Save

Step 3: Record Webhook IDs

After creating each webhook, Autotask will provide a Webhook ID. Record these for reference:

Companies: webhook_12345
Tickets: webhook_12346
Tasks: webhook_12347
Projects: webhook_12348
TimeEntries: webhook_12349
Contacts: webhook_12350

Step 4: Verify Webhook Configuration

Check that webhooks are configured in the database:

SELECT entity_type, is_active, autotask_webhook_id
FROM webhook_configs
ORDER BY entity_type;

Update webhook IDs if needed:

UPDATE webhook_configs
SET autotask_webhook_id = 'webhook_12345'
WHERE entity_type = 'Companies';

Webhook Endpoint

Endpoint Details

  • URL: https://your-domain.com/api/webhooks/autotask
  • Method: POST
  • Content-Type: application/json
  • Authentication: None (Autotask doesn't support webhook authentication)
    • Secure via IP whitelisting or network-level security

Payload Structure

Autotask sends webhooks with this structure:

{
  "eventId": "evt_abc123",
  "eventType": "create",
  "entityType": "Tickets",
  "entityId": 12345,
  "eventTimestamp": "2026-01-24T10:00:00Z",
  "entity": {
    "id": 12345,
    "title": "New Ticket",
    "status": 1,
    "companyID": 67890,
    ...
  }
}

Response

The endpoint returns:

{
  "success": true,
  "eventId": "evt_abc123",
  "action": "created",
  "processingTime": 45
}

Testing Webhooks

1. Health Check

Verify the webhook endpoint is accessible:

curl https://your-pulse-instance.com/api/webhooks/autotask

Expected response:

{
  "status": "active",
  "endpoint": "/api/webhooks/autotask",
  "message": "Autotask webhook receiver is ready"
}

2. Test Webhook from Autotask

  1. In Autotask, go to the webhook configuration
  2. Click Test Webhook
  3. Autotask will send a test event
  4. Check webhook logs in Pulse

3. Create Test Entity

Create a test ticket or company in Autotask and verify:

  1. Webhook is received (check logs)
  2. Entity appears in database
  3. Processing time is reasonable (<1 second)

4. View Webhook Logs

# Via API
curl https://your-pulse-instance.com/api/webhooks/logs?limit=10

# Via Database
docker exec pulse-postgres psql -U pulse_user -d pulse_autotask -c \
  "SELECT event_id, entity_type, entity_id, status, processing_time_ms 
   FROM webhook_logs 
   ORDER BY received_at DESC 
   LIMIT 10;"

Monitoring

Webhook Statistics

View webhook statistics via API:

curl https://your-pulse-instance.com/api/webhooks/stats?hours=24

Response:

{
  "success": true,
  "stats": {
    "total": 150,
    "processed": 148,
    "failed": 2,
    "pending": 0,
    "byEntityType": {
      "Tickets": 75,
      "Companies": 25,
      "Tasks": 30,
      "TimeEntries": 20
    }
  },
  "period": "24 hours"
}

Failed Webhooks

Check for failed webhook processing:

SELECT event_id, entity_type, entity_id, error_message, received_at
FROM webhook_logs
WHERE status = 'failed'
ORDER BY received_at DESC
LIMIT 20;

Processing Performance

Monitor webhook processing times:

SELECT 
  entity_type,
  COUNT(*) as total,
  AVG(processing_time_ms) as avg_ms,
  MAX(processing_time_ms) as max_ms
FROM webhook_logs
WHERE status = 'processed'
  AND received_at >= NOW() - INTERVAL '24 hours'
GROUP BY entity_type
ORDER BY avg_ms DESC;

Troubleshooting

Webhooks Not Received

Check 1: Endpoint Accessibility

# From external network
curl https://your-pulse-instance.com/api/webhooks/autotask

Check 2: Firewall/Network

  • Ensure port 443 (HTTPS) is open
  • Check firewall rules allow Autotask IPs
  • Verify SSL certificate is valid

Check 3: Autotask Configuration

  • Verify webhook is Active in Autotask
  • Check endpoint URL is correct
  • Ensure entity type matches

Webhooks Failing to Process

Check Logs:

SELECT event_id, entity_type, error_message, payload
FROM webhook_logs
WHERE status = 'failed'
ORDER BY received_at DESC
LIMIT 5;

Common Issues:

  1. Missing Entity Data

    • Enable "Include Entity Data" in Autotask webhook config
    • Without this, webhook only includes entity ID
  2. Foreign Key Violations

    • Run full sync for dependent entities first
    • Example: Sync Companies before Tickets
  3. Invalid Data

    • Check entity mapper for missing field mappings
    • Review error_message in webhook_logs

High Failure Rate

If >5% of webhooks fail:

  1. Run Full Sync for affected entities
  2. Check Dependencies - ensure parent entities are synced
  3. Review Error Patterns - look for common error messages
  4. Contact Support - if issues persist

Optimal Configuration

Combine webhooks with scheduled syncs for best results:

1. Webhooks (Real-Time)

  • Enabled for: Companies, Tickets, Tasks, Projects, Time Entries, Contacts
  • Events: Create, Update
  • Purpose: Instant updates for active entities

2. Daily Incremental Sync (Scheduled)

  • Time: 2:00 AM daily
  • Entities: All entities
  • Purpose:
    • Catch any missed webhooks
    • Sync entities without webhooks (Contracts, Config Items, Picklists)
    • Ensure data consistency

3. Weekly Full Sync (Scheduled)

  • Time: Sunday 3:00 AM
  • Entities: All entities
  • Date Range: Last 2 years
  • Purpose:
    • Complete data refresh
    • Verify data integrity
    • Sync historical changes

Cron Schedule Example

# Daily incremental sync at 2 AM
0 2 * * * curl -X POST https://your-pulse-instance.com/api/sync/incremental

# Weekly full sync on Sunday at 3 AM
0 3 * * 0 curl -X POST https://your-pulse-instance.com/api/sync/full

Expected Data Freshness

Entity Freshness Method
Tickets <1 minute Webhook
Tasks <1 minute Webhook
Companies <1 minute Webhook
Projects <1 minute Webhook
Time Entries <1 minute Webhook
Contacts <1 minute Webhook
Contracts 24 hours Daily sync
Config Items 24 hours Daily sync
Picklists 7 days Weekly sync

Security Considerations

1. Network Security

Option A: IP Whitelisting

  • Restrict webhook endpoint to Autotask IP ranges
  • Configure in firewall or reverse proxy

Option B: VPN/Private Network

  • Use VPN tunnel for webhook traffic
  • More secure but complex setup

2. Payload Validation

The webhook service validates:

  • Required fields (eventId, eventType, entityType, entityId)
  • Event ID uniqueness (prevents duplicate processing)
  • Entity type is supported

3. Rate Limiting

Consider adding rate limiting to prevent abuse:

# Nginx example
limit_req_zone $binary_remote_addr zone=webhook:10m rate=100r/m;

location /api/webhooks/autotask {
    limit_req zone=webhook burst=20;
    proxy_pass http://pulse-app:3100;
}

Database Schema

webhook_logs Table

Tracks all incoming webhook events:

CREATE TABLE webhook_logs (
  id SERIAL PRIMARY KEY,
  event_id VARCHAR(255) NOT NULL UNIQUE,
  entity_type VARCHAR(100) NOT NULL,
  entity_id INTEGER NOT NULL,
  event_type VARCHAR(50) NOT NULL,
  status VARCHAR(50) NOT NULL DEFAULT 'pending',
  error_message TEXT,
  received_at TIMESTAMP NOT NULL DEFAULT NOW(),
  processed_at TIMESTAMP,
  processing_time_ms INTEGER,
  payload JSONB NOT NULL
);

webhook_configs Table

Manages webhook configurations:

CREATE TABLE webhook_configs (
  id SERIAL PRIMARY KEY,
  entity_type VARCHAR(100) NOT NULL UNIQUE,
  event_types JSONB NOT NULL,
  is_active BOOLEAN NOT NULL DEFAULT true,
  autotask_webhook_id VARCHAR(255)
);

API Reference

Receive Webhook

POST /api/webhooks/autotask
Content-Type: application/json

{
  "eventId": "evt_abc123",
  "eventType": "create",
  "entityType": "Tickets",
  "entityId": 12345,
  "entity": { ... }
}

Get Webhook Logs

GET /api/webhooks/logs?limit=100&entityType=Tickets

Get Webhook Statistics

GET /api/webhooks/stats?hours=24

Support

For issues or questions:

  1. Check webhook logs for error messages
  2. Verify Autotask webhook configuration
  3. Review this documentation
  4. Contact system administrator