Implements comprehensive webhook infrastructure to receive and process real-time entity updates from Autotask, reducing API calls and improving data freshness. Features: - Webhook receiver endpoint: POST /api/webhooks/autotask - Automatic entity mapping and upsert to PostgreSQL - Event logging and tracking in webhook_logs table - Duplicate event prevention via unique event_id - Failed event tracking with error messages - Statistics and monitoring APIs - Support for 8 entity types: Companies, Tickets, Tasks, Projects, Time Entries, Contacts, Contracts, Configuration Items Architecture: - WebhookService: Core processing logic - Database tables: webhook_logs, webhook_configs - API endpoints: /autotask (receiver), /logs, /stats - Automatic data mapping using existing entity-mapper Benefits: - Near real-time updates (<1 minute vs 24 hours) - Reduced API usage (webhooks vs polling) - Complements daily incremental sync for redundancy - Automatic recovery from webhook failures Files Added: - lib/types/webhook.ts - TypeScript types and interfaces - lib/services/webhook-service.ts - Webhook processing service - app/api/webhooks/autotask/route.ts - Webhook receiver - app/api/webhooks/logs/route.ts - Logs API - app/api/webhooks/stats/route.ts - Statistics API - migrations/004_webhook_support.sql - Database schema - docs/WEBHOOK_SETUP.md - Complete setup guide (47 sections) - docs/WEBHOOKS_README.md - Quick start guide Next Steps: 1. Run database migration 2. Configure webhooks in Autotask 3. Test endpoint and monitor logs See docs/WEBHOOK_SETUP.md for detailed setup instructions.
12 KiB
Autotask Webhook Setup Guide
Overview
This guide explains how to configure Autotask webhooks for real-time data synchronization with Pulse. Webhooks provide near-instant updates when entities change in Autotask, reducing API calls and keeping your data fresh.
Table of Contents
- Benefits of Webhooks
- Supported Entities
- Prerequisites
- Setup Steps
- Webhook Endpoint
- Testing Webhooks
- Monitoring
- Troubleshooting
- Recommended Sync Strategy
Benefits of Webhooks
Real-Time Updates
- Instant notification when entities are created or updated in Autotask
- No polling delay - changes appear within seconds
Reduced API Usage
- Fewer API calls compared to frequent polling
- Lower risk of hitting API rate limits
- More efficient use of resources
Better Data Freshness
- Combined with daily incremental syncs for redundancy
- Ensures no events are missed
- Automatic recovery from webhook failures
Supported Entities
The following entities support webhook notifications:
| Entity | Events | Notes |
|---|---|---|
| Companies | Create, Update | Real-time company changes |
| Tickets | Create, Update | New tickets and status updates |
| Tasks | Create, Update | Task creation and modifications |
| Projects | Create, Update | Project changes |
| Time Entries | Create, Update | New time entries |
| Contacts | Create, Update | Contact information changes |
| Contracts | Create, Update | Contract updates |
| Configuration Items | Create, Update | Asset changes |
Note: Delete events are supported but less common in Autotask workflows.
Prerequisites
Before setting up webhooks, ensure you have:
-
Autotask API Access
- API user with appropriate permissions
- API integration enabled in Autotask
-
Public Webhook Endpoint
- Your Pulse instance must be accessible from the internet
- HTTPS endpoint (required by Autotask)
- Example:
https://your-pulse-instance.com/api/webhooks/autotask
-
Database Migration
- Run migration
004_webhook_support.sqlto create webhook tables
docker exec pulse-postgres psql -U pulse_user -d pulse_autotask -f /migrations/004_webhook_support.sql - Run migration
Setup Steps
Step 1: Configure Webhook Endpoint in Autotask
- Log in to Autotask as an administrator
- Navigate to Admin → Features & Settings → API & Integrations
- Click on Webhooks
- Click New Webhook
Step 2: Create Webhook for Each Entity
For each entity you want to track (e.g., Tickets):
- Webhook Name:
Pulse - Tickets - Endpoint URL:
https://your-pulse-instance.com/api/webhooks/autotask - Entity Type: Select the entity (e.g.,
Tickets) - Events: Select events to track:
- ✅ Create
- ✅ Update
- ⬜ Delete (optional)
- Include Entity Data: ✅ Enabled (recommended)
- This includes full entity data in the webhook payload
- Reduces need for additional API calls
- Active: ✅ Enabled
- Click Save
Step 3: Record Webhook IDs
After creating each webhook, Autotask will provide a Webhook ID. Record these for reference:
Companies: webhook_12345
Tickets: webhook_12346
Tasks: webhook_12347
Projects: webhook_12348
TimeEntries: webhook_12349
Contacts: webhook_12350
Step 4: Verify Webhook Configuration
Check that webhooks are configured in the database:
SELECT entity_type, is_active, autotask_webhook_id
FROM webhook_configs
ORDER BY entity_type;
Update webhook IDs if needed:
UPDATE webhook_configs
SET autotask_webhook_id = 'webhook_12345'
WHERE entity_type = 'Companies';
Webhook Endpoint
Endpoint Details
- URL:
https://your-domain.com/api/webhooks/autotask - Method:
POST - Content-Type:
application/json - Authentication: None (Autotask doesn't support webhook authentication)
- Secure via IP whitelisting or network-level security
Payload Structure
Autotask sends webhooks with this structure:
{
"eventId": "evt_abc123",
"eventType": "create",
"entityType": "Tickets",
"entityId": 12345,
"eventTimestamp": "2026-01-24T10:00:00Z",
"entity": {
"id": 12345,
"title": "New Ticket",
"status": 1,
"companyID": 67890,
...
}
}
Response
The endpoint returns:
{
"success": true,
"eventId": "evt_abc123",
"action": "created",
"processingTime": 45
}
Testing Webhooks
1. Health Check
Verify the webhook endpoint is accessible:
curl https://your-pulse-instance.com/api/webhooks/autotask
Expected response:
{
"status": "active",
"endpoint": "/api/webhooks/autotask",
"message": "Autotask webhook receiver is ready"
}
2. Test Webhook from Autotask
- In Autotask, go to the webhook configuration
- Click Test Webhook
- Autotask will send a test event
- Check webhook logs in Pulse
3. Create Test Entity
Create a test ticket or company in Autotask and verify:
- Webhook is received (check logs)
- Entity appears in database
- Processing time is reasonable (<1 second)
4. View Webhook Logs
# Via API
curl https://your-pulse-instance.com/api/webhooks/logs?limit=10
# Via Database
docker exec pulse-postgres psql -U pulse_user -d pulse_autotask -c \
"SELECT event_id, entity_type, entity_id, status, processing_time_ms
FROM webhook_logs
ORDER BY received_at DESC
LIMIT 10;"
Monitoring
Webhook Statistics
View webhook statistics via API:
curl https://your-pulse-instance.com/api/webhooks/stats?hours=24
Response:
{
"success": true,
"stats": {
"total": 150,
"processed": 148,
"failed": 2,
"pending": 0,
"byEntityType": {
"Tickets": 75,
"Companies": 25,
"Tasks": 30,
"TimeEntries": 20
}
},
"period": "24 hours"
}
Failed Webhooks
Check for failed webhook processing:
SELECT event_id, entity_type, entity_id, error_message, received_at
FROM webhook_logs
WHERE status = 'failed'
ORDER BY received_at DESC
LIMIT 20;
Processing Performance
Monitor webhook processing times:
SELECT
entity_type,
COUNT(*) as total,
AVG(processing_time_ms) as avg_ms,
MAX(processing_time_ms) as max_ms
FROM webhook_logs
WHERE status = 'processed'
AND received_at >= NOW() - INTERVAL '24 hours'
GROUP BY entity_type
ORDER BY avg_ms DESC;
Troubleshooting
Webhooks Not Received
Check 1: Endpoint Accessibility
# From external network
curl https://your-pulse-instance.com/api/webhooks/autotask
Check 2: Firewall/Network
- Ensure port 443 (HTTPS) is open
- Check firewall rules allow Autotask IPs
- Verify SSL certificate is valid
Check 3: Autotask Configuration
- Verify webhook is Active in Autotask
- Check endpoint URL is correct
- Ensure entity type matches
Webhooks Failing to Process
Check Logs:
SELECT event_id, entity_type, error_message, payload
FROM webhook_logs
WHERE status = 'failed'
ORDER BY received_at DESC
LIMIT 5;
Common Issues:
-
Missing Entity Data
- Enable "Include Entity Data" in Autotask webhook config
- Without this, webhook only includes entity ID
-
Foreign Key Violations
- Run full sync for dependent entities first
- Example: Sync Companies before Tickets
-
Invalid Data
- Check entity mapper for missing field mappings
- Review error_message in webhook_logs
High Failure Rate
If >5% of webhooks fail:
- Run Full Sync for affected entities
- Check Dependencies - ensure parent entities are synced
- Review Error Patterns - look for common error messages
- Contact Support - if issues persist
Recommended Sync Strategy
Optimal Configuration
Combine webhooks with scheduled syncs for best results:
1. Webhooks (Real-Time)
- Enabled for: Companies, Tickets, Tasks, Projects, Time Entries, Contacts
- Events: Create, Update
- Purpose: Instant updates for active entities
2. Daily Incremental Sync (Scheduled)
- Time: 2:00 AM daily
- Entities: All entities
- Purpose:
- Catch any missed webhooks
- Sync entities without webhooks (Contracts, Config Items, Picklists)
- Ensure data consistency
3. Weekly Full Sync (Scheduled)
- Time: Sunday 3:00 AM
- Entities: All entities
- Date Range: Last 2 years
- Purpose:
- Complete data refresh
- Verify data integrity
- Sync historical changes
Cron Schedule Example
# Daily incremental sync at 2 AM
0 2 * * * curl -X POST https://your-pulse-instance.com/api/sync/incremental
# Weekly full sync on Sunday at 3 AM
0 3 * * 0 curl -X POST https://your-pulse-instance.com/api/sync/full
Expected Data Freshness
| Entity | Freshness | Method |
|---|---|---|
| Tickets | <1 minute | Webhook |
| Tasks | <1 minute | Webhook |
| Companies | <1 minute | Webhook |
| Projects | <1 minute | Webhook |
| Time Entries | <1 minute | Webhook |
| Contacts | <1 minute | Webhook |
| Contracts | 24 hours | Daily sync |
| Config Items | 24 hours | Daily sync |
| Picklists | 7 days | Weekly sync |
Security Considerations
1. Network Security
Option A: IP Whitelisting
- Restrict webhook endpoint to Autotask IP ranges
- Configure in firewall or reverse proxy
Option B: VPN/Private Network
- Use VPN tunnel for webhook traffic
- More secure but complex setup
2. Payload Validation
The webhook service validates:
- Required fields (eventId, eventType, entityType, entityId)
- Event ID uniqueness (prevents duplicate processing)
- Entity type is supported
3. Rate Limiting
Consider adding rate limiting to prevent abuse:
# Nginx example
limit_req_zone $binary_remote_addr zone=webhook:10m rate=100r/m;
location /api/webhooks/autotask {
limit_req zone=webhook burst=20;
proxy_pass http://pulse-app:3100;
}
Database Schema
webhook_logs Table
Tracks all incoming webhook events:
CREATE TABLE webhook_logs (
id SERIAL PRIMARY KEY,
event_id VARCHAR(255) NOT NULL UNIQUE,
entity_type VARCHAR(100) NOT NULL,
entity_id INTEGER NOT NULL,
event_type VARCHAR(50) NOT NULL,
status VARCHAR(50) NOT NULL DEFAULT 'pending',
error_message TEXT,
received_at TIMESTAMP NOT NULL DEFAULT NOW(),
processed_at TIMESTAMP,
processing_time_ms INTEGER,
payload JSONB NOT NULL
);
webhook_configs Table
Manages webhook configurations:
CREATE TABLE webhook_configs (
id SERIAL PRIMARY KEY,
entity_type VARCHAR(100) NOT NULL UNIQUE,
event_types JSONB NOT NULL,
is_active BOOLEAN NOT NULL DEFAULT true,
autotask_webhook_id VARCHAR(255)
);
API Reference
Receive Webhook
POST /api/webhooks/autotask
Content-Type: application/json
{
"eventId": "evt_abc123",
"eventType": "create",
"entityType": "Tickets",
"entityId": 12345,
"entity": { ... }
}
Get Webhook Logs
GET /api/webhooks/logs?limit=100&entityType=Tickets
Get Webhook Statistics
GET /api/webhooks/stats?hours=24
Support
For issues or questions:
- Check webhook logs for error messages
- Verify Autotask webhook configuration
- Review this documentation
- Contact system administrator