feat: add webhook support for real-time Autotask updates
Implements comprehensive webhook infrastructure to receive and process
real-time entity updates from Autotask, reducing API calls and improving
data freshness.
Features:
- Webhook receiver endpoint: POST /api/webhooks/autotask
- Automatic entity mapping and upsert to PostgreSQL
- Event logging and tracking in webhook_logs table
- Duplicate event prevention via unique event_id
- Failed event tracking with error messages
- Statistics and monitoring APIs
- Support for 8 entity types: Companies, Tickets, Tasks, Projects,
Time Entries, Contacts, Contracts, Configuration Items
Architecture:
- WebhookService: Core processing logic
- Database tables: webhook_logs, webhook_configs
- API endpoints: /autotask (receiver), /logs, /stats
- Automatic data mapping using existing entity-mapper
Benefits:
- Near real-time updates (<1 minute vs 24 hours)
- Reduced API usage (webhooks vs polling)
- Complements daily incremental sync for redundancy
- Automatic recovery from webhook failures
Files Added:
- lib/types/webhook.ts - TypeScript types and interfaces
- lib/services/webhook-service.ts - Webhook processing service
- app/api/webhooks/autotask/route.ts - Webhook receiver
- app/api/webhooks/logs/route.ts - Logs API
- app/api/webhooks/stats/route.ts - Statistics API
- migrations/004_webhook_support.sql - Database schema
- docs/WEBHOOK_SETUP.md - Complete setup guide (47 sections)
- docs/WEBHOOKS_README.md - Quick start guide
Next Steps:
1. Run database migration
2. Configure webhooks in Autotask
3. Test endpoint and monitor logs
See docs/WEBHOOK_SETUP.md for detailed setup instructions.
2026-01-24 10:07:20 -05:00
|
|
|
-- Migration: Add webhook support tables
|
|
|
|
|
-- Description: Tables for tracking webhook events and configurations
|
|
|
|
|
|
|
|
|
|
-- Webhook logs table - tracks all incoming webhook events
|
|
|
|
|
CREATE TABLE IF NOT EXISTS webhook_logs (
|
|
|
|
|
id SERIAL PRIMARY KEY,
|
|
|
|
|
event_id VARCHAR(255) NOT NULL UNIQUE,
|
|
|
|
|
entity_type VARCHAR(100) NOT NULL,
|
|
|
|
|
entity_id INTEGER NOT NULL,
|
|
|
|
|
event_type VARCHAR(50) NOT NULL, -- create, update, delete
|
|
|
|
|
status VARCHAR(50) NOT NULL DEFAULT 'pending', -- pending, processed, failed
|
|
|
|
|
error_message TEXT,
|
2026-01-24 17:28:56 -05:00
|
|
|
source_ip VARCHAR(45), -- IPv4 or IPv6 address
|
|
|
|
|
user_agent TEXT, -- User agent string from request
|
feat: add webhook support for real-time Autotask updates
Implements comprehensive webhook infrastructure to receive and process
real-time entity updates from Autotask, reducing API calls and improving
data freshness.
Features:
- Webhook receiver endpoint: POST /api/webhooks/autotask
- Automatic entity mapping and upsert to PostgreSQL
- Event logging and tracking in webhook_logs table
- Duplicate event prevention via unique event_id
- Failed event tracking with error messages
- Statistics and monitoring APIs
- Support for 8 entity types: Companies, Tickets, Tasks, Projects,
Time Entries, Contacts, Contracts, Configuration Items
Architecture:
- WebhookService: Core processing logic
- Database tables: webhook_logs, webhook_configs
- API endpoints: /autotask (receiver), /logs, /stats
- Automatic data mapping using existing entity-mapper
Benefits:
- Near real-time updates (<1 minute vs 24 hours)
- Reduced API usage (webhooks vs polling)
- Complements daily incremental sync for redundancy
- Automatic recovery from webhook failures
Files Added:
- lib/types/webhook.ts - TypeScript types and interfaces
- lib/services/webhook-service.ts - Webhook processing service
- app/api/webhooks/autotask/route.ts - Webhook receiver
- app/api/webhooks/logs/route.ts - Logs API
- app/api/webhooks/stats/route.ts - Statistics API
- migrations/004_webhook_support.sql - Database schema
- docs/WEBHOOK_SETUP.md - Complete setup guide (47 sections)
- docs/WEBHOOKS_README.md - Quick start guide
Next Steps:
1. Run database migration
2. Configure webhooks in Autotask
3. Test endpoint and monitor logs
See docs/WEBHOOK_SETUP.md for detailed setup instructions.
2026-01-24 10:07:20 -05:00
|
|
|
received_at TIMESTAMP NOT NULL DEFAULT NOW(),
|
|
|
|
|
processed_at TIMESTAMP,
|
|
|
|
|
processing_time_ms INTEGER,
|
|
|
|
|
payload JSONB NOT NULL,
|
|
|
|
|
created_at TIMESTAMP NOT NULL DEFAULT NOW(),
|
|
|
|
|
updated_at TIMESTAMP NOT NULL DEFAULT NOW()
|
|
|
|
|
);
|
|
|
|
|
|
|
|
|
|
-- Webhook configurations table - tracks which webhooks are configured
|
|
|
|
|
CREATE TABLE IF NOT EXISTS webhook_configs (
|
|
|
|
|
id SERIAL PRIMARY KEY,
|
|
|
|
|
entity_type VARCHAR(100) NOT NULL,
|
|
|
|
|
event_types JSONB NOT NULL, -- array of event types: ["create", "update", "delete"]
|
|
|
|
|
is_active BOOLEAN NOT NULL DEFAULT true,
|
|
|
|
|
autotask_webhook_id VARCHAR(255), -- ID from Autotask webhook registration
|
|
|
|
|
created_at TIMESTAMP NOT NULL DEFAULT NOW(),
|
|
|
|
|
updated_at TIMESTAMP NOT NULL DEFAULT NOW(),
|
|
|
|
|
UNIQUE(entity_type)
|
|
|
|
|
);
|
|
|
|
|
|
|
|
|
|
-- Indexes for webhook_logs
|
|
|
|
|
CREATE INDEX IF NOT EXISTS idx_webhook_logs_event_id ON webhook_logs(event_id);
|
|
|
|
|
CREATE INDEX IF NOT EXISTS idx_webhook_logs_entity ON webhook_logs(entity_type, entity_id);
|
|
|
|
|
CREATE INDEX IF NOT EXISTS idx_webhook_logs_status ON webhook_logs(status);
|
|
|
|
|
CREATE INDEX IF NOT EXISTS idx_webhook_logs_received_at ON webhook_logs(received_at DESC);
|
|
|
|
|
CREATE INDEX IF NOT EXISTS idx_webhook_logs_entity_type ON webhook_logs(entity_type);
|
2026-01-24 17:28:56 -05:00
|
|
|
CREATE INDEX IF NOT EXISTS idx_webhook_logs_source_ip ON webhook_logs(source_ip);
|
feat: add webhook support for real-time Autotask updates
Implements comprehensive webhook infrastructure to receive and process
real-time entity updates from Autotask, reducing API calls and improving
data freshness.
Features:
- Webhook receiver endpoint: POST /api/webhooks/autotask
- Automatic entity mapping and upsert to PostgreSQL
- Event logging and tracking in webhook_logs table
- Duplicate event prevention via unique event_id
- Failed event tracking with error messages
- Statistics and monitoring APIs
- Support for 8 entity types: Companies, Tickets, Tasks, Projects,
Time Entries, Contacts, Contracts, Configuration Items
Architecture:
- WebhookService: Core processing logic
- Database tables: webhook_logs, webhook_configs
- API endpoints: /autotask (receiver), /logs, /stats
- Automatic data mapping using existing entity-mapper
Benefits:
- Near real-time updates (<1 minute vs 24 hours)
- Reduced API usage (webhooks vs polling)
- Complements daily incremental sync for redundancy
- Automatic recovery from webhook failures
Files Added:
- lib/types/webhook.ts - TypeScript types and interfaces
- lib/services/webhook-service.ts - Webhook processing service
- app/api/webhooks/autotask/route.ts - Webhook receiver
- app/api/webhooks/logs/route.ts - Logs API
- app/api/webhooks/stats/route.ts - Statistics API
- migrations/004_webhook_support.sql - Database schema
- docs/WEBHOOK_SETUP.md - Complete setup guide (47 sections)
- docs/WEBHOOKS_README.md - Quick start guide
Next Steps:
1. Run database migration
2. Configure webhooks in Autotask
3. Test endpoint and monitor logs
See docs/WEBHOOK_SETUP.md for detailed setup instructions.
2026-01-24 10:07:20 -05:00
|
|
|
|
|
|
|
|
-- Indexes for webhook_configs
|
|
|
|
|
CREATE INDEX IF NOT EXISTS idx_webhook_configs_entity_type ON webhook_configs(entity_type);
|
|
|
|
|
CREATE INDEX IF NOT EXISTS idx_webhook_configs_is_active ON webhook_configs(is_active);
|
|
|
|
|
|
|
|
|
|
-- Function to update updated_at timestamp
|
|
|
|
|
CREATE OR REPLACE FUNCTION update_webhook_updated_at()
|
|
|
|
|
RETURNS TRIGGER AS $$
|
|
|
|
|
BEGIN
|
|
|
|
|
NEW.updated_at = NOW();
|
|
|
|
|
RETURN NEW;
|
|
|
|
|
END;
|
|
|
|
|
$$ LANGUAGE plpgsql;
|
|
|
|
|
|
|
|
|
|
-- Triggers for updated_at
|
|
|
|
|
CREATE TRIGGER webhook_logs_updated_at
|
|
|
|
|
BEFORE UPDATE ON webhook_logs
|
|
|
|
|
FOR EACH ROW
|
|
|
|
|
EXECUTE FUNCTION update_webhook_updated_at();
|
|
|
|
|
|
|
|
|
|
CREATE TRIGGER webhook_configs_updated_at
|
|
|
|
|
BEFORE UPDATE ON webhook_configs
|
|
|
|
|
FOR EACH ROW
|
|
|
|
|
EXECUTE FUNCTION update_webhook_updated_at();
|
|
|
|
|
|
|
|
|
|
-- Insert default webhook configurations for key entities
|
|
|
|
|
INSERT INTO webhook_configs (entity_type, event_types, is_active) VALUES
|
|
|
|
|
('Companies', '["create", "update"]', true),
|
|
|
|
|
('Tickets', '["create", "update"]', true),
|
|
|
|
|
('Tasks', '["create", "update"]', true),
|
|
|
|
|
('Projects', '["create", "update"]', true),
|
|
|
|
|
('TimeEntries', '["create", "update"]', true),
|
|
|
|
|
('Contacts', '["create", "update"]', true)
|
|
|
|
|
ON CONFLICT (entity_type) DO NOTHING;
|
|
|
|
|
|
|
|
|
|
-- Comments
|
|
|
|
|
COMMENT ON TABLE webhook_logs IS 'Tracks all incoming webhook events from Autotask';
|
|
|
|
|
COMMENT ON TABLE webhook_configs IS 'Configuration for which webhooks are enabled';
|
|
|
|
|
COMMENT ON COLUMN webhook_logs.event_id IS 'Unique identifier from Autotask webhook event';
|
|
|
|
|
COMMENT ON COLUMN webhook_logs.payload IS 'Full webhook payload as JSON';
|
|
|
|
|
COMMENT ON COLUMN webhook_logs.processing_time_ms IS 'Time taken to process the webhook in milliseconds';
|
|
|
|
|
COMMENT ON COLUMN webhook_configs.autotask_webhook_id IS 'Webhook ID returned by Autotask API when webhook was registered';
|