wulf-pulse/docs/WEBHOOK_IP_WHITELISTING.md

# Webhook IP Whitelisting Guide

## Overview

This guide explains how to identify Autotask's webhook source IPs and configure IP whitelisting to secure your webhook endpoint.

---

## Why IP Whitelisting?

**Benefits:**
- ✅ Only allow webhooks from Autotask's servers
- ✅ Block unauthorized webhook attempts
- ✅ Additional security layer beyond HTTPS
- ✅ Prevent abuse and DDoS attacks

**When to Use:**
- Production environments
- High-security requirements
- After identifying Autotask's IP ranges

---

## Step 1: Identify Autotask IPs

### **Method 1: View Webhook Logs (Recommended)**

After receiving a few webhooks from Autotask, check the logged IPs:

**Via API:**
```bash
curl https://your-domain.com/api/webhooks/ips?hours=168
```

Response:
```json
{
  "success": true,
  "ips": [
    {
      "ip": "52.1.2.3",
      "userAgent": "Autotask-Webhook/1.0",
      "requestCount": 150,
      "successfulCount": 148,
      "failedCount": 2,
      "firstSeen": "2026-01-20T10:00:00Z",
      "lastSeen": "2026-01-24T15:00:00Z",
      "entityTypes": ["Tickets", "Companies", "Tasks"]
    },
    {
      "ip": "52.5.6.7",
      "userAgent": "Autotask-Webhook/1.0",
      "requestCount": 75,
      "successfulCount": 75,
      "failedCount": 0,
      "firstSeen": "2026-01-22T08:00:00Z",
      "lastSeen": "2026-01-24T14:30:00Z",
      "entityTypes": ["TimeEntries", "Projects"]
    }
  ],
  "count": 2,
  "period": "168 hours"
}
```

**Via Database:**
```sql
-- Get unique IPs with request counts
SELECT 
  source_ip,
  user_agent,
  COUNT(*) as request_count,
  MIN(received_at) as first_seen,
  MAX(received_at) as last_seen,
  COUNT(*) FILTER (WHERE status = 'processed') as successful,
  COUNT(*) FILTER (WHERE status = 'failed') as failed
FROM webhook_logs
WHERE received_at >= NOW() - INTERVAL '7 days'
  AND source_ip IS NOT NULL
GROUP BY source_ip, user_agent
ORDER BY request_count DESC;
```

**Identify Autotask IPs:**
- Look for IPs with `Autotask-Webhook` user agent
- High request counts (if you have active webhooks)
- Multiple entity types
- Consistent activity pattern

### **Method 2: Contact Autotask Support**

Request official IP ranges from Autotask:
- Open support ticket
- Ask for "Webhook source IP ranges"
- They may provide CIDR blocks (e.g., `52.1.0.0/16`)

### **Method 3: Check Autotask Documentation**

Some Autotask zones publish their IP ranges:
- Check Autotask developer documentation
- Look for "Webhook IP ranges" or "API IP addresses"
- May vary by zone (ww1, ww5, ww15, etc.)

---

## Step 2: Configure IP Whitelist

### **Option A: Nginx**

Edit your nginx configuration:

```nginx
server {
    listen 443 ssl http2;
    server_name webhooks.yourdomain.com;
    
    # ... SSL configuration ...
    
    location = /api/webhooks/autotask {
        # IP Whitelist - Autotask webhook IPs
        allow 52.1.2.3;      # Autotask IP 1
        allow 52.5.6.7;      # Autotask IP 2
        allow 52.10.0.0/16;  # Autotask IP range (CIDR)
        deny all;            # Block everything else
        
        # Rate limiting
        limit_req zone=webhook_limit burst=20 nodelay;
        
        # Proxy to Pulse app
        proxy_pass http://pulse_app;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
    }
}
```

**Test and reload:**
```bash
sudo nginx -t
sudo systemctl reload nginx
```

### **Option B: Pangolin**

Update your Pangolin configuration:

```yaml
# /etc/pangolin/config.yml

name: pulse-webhooks

ingress:
  - hostname: webhooks.yourdomain.com
    path: /api/webhooks/autotask
    service: http://localhost:3100

security:
  # IP Whitelist
  ipWhitelist:
    - "52.1.2.3"        # Autotask IP 1
    - "52.5.6.7"        # Autotask IP 2
    - "52.10.0.0/16"    # Autotask IP range
  
  # Rate limiting
  rateLimit:
    enabled: true
    requestsPerMinute: 100
    burstSize: 20
```

**Restart tunnel:**
```bash
sudo systemctl restart pangolin-tunnel
```

### **Option C: Cloudflare Tunnel**

Use Cloudflare Access rules:

1. **Go to Cloudflare Dashboard** → Zero Trust → Access → Applications
2. **Create Application:**
   - Name: `Pulse Webhooks`
   - Domain: `webhooks.yourdomain.com`
   - Path: `/api/webhooks/autotask`
3. **Add Policy:**
   - Name: `Autotask IPs Only`
   - Action: `Allow`
   - Rule: `IP ranges`
   - Values: `52.1.2.3`, `52.5.6.7`, `52.10.0.0/16`
4. **Save**

### **Option D: Firewall (UFW)**

If using UFW on the server:

```bash
# Allow specific IPs to port 443
sudo ufw allow from 52.1.2.3 to any port 443
sudo ufw allow from 52.5.6.7 to any port 443
sudo ufw allow from 52.10.0.0/16 to any port 443

# Deny all other traffic to port 443 (if not already blocked)
# Be careful with this - ensure you have other access methods!
# sudo ufw deny 443

# Check rules
sudo ufw status numbered
```

**Warning:** Be careful with firewall rules. Ensure you don't lock yourself out!

---

## Step 3: Test IP Whitelist

### **1. Test from Allowed IP (Autotask)**

Trigger a webhook from Autotask:
- Create a test ticket or company
- Check webhook logs to verify it was received

```bash
# Check recent webhooks
curl https://your-domain.com/api/webhooks/logs?limit=5
```

### **2. Test from Blocked IP**

From a different IP (your computer):

```bash
curl -X POST https://webhooks.yourdomain.com/api/webhooks/autotask \
  -H "Content-Type: application/json" \
  -d '{"eventId":"test","eventType":"create","entityType":"Tickets","entityId":1}'
```

**Expected result:** 
- **Nginx:** `403 Forbidden`
- **Pangolin/Cloudflare:** Connection refused or 403
- **Firewall:** Connection timeout

### **3. Verify Logs**

Check that only Autotask IPs are getting through:

```sql
-- Recent webhook IPs
SELECT source_ip, COUNT(*) as count
FROM webhook_logs
WHERE received_at >= NOW() - INTERVAL '1 hour'
GROUP BY source_ip;
```

Should only show Autotask IPs.

---

## Monitoring IP Changes

### **Set Up Alerts for New IPs**

Create a monitoring query to detect new source IPs:

```sql
-- Find IPs seen in last 24 hours that weren't seen before
SELECT DISTINCT wl.source_ip, wl.user_agent, MIN(wl.received_at) as first_seen
FROM webhook_logs wl
WHERE wl.received_at >= NOW() - INTERVAL '24 hours'
  AND wl.source_ip NOT IN (
    SELECT DISTINCT source_ip 
    FROM webhook_logs 
    WHERE received_at < NOW() - INTERVAL '24 hours'
      AND received_at >= NOW() - INTERVAL '30 days'
  )
GROUP BY wl.source_ip, wl.user_agent;
```

### **Regular IP Audit**

Run weekly to check for IP changes:

```bash
# Get IPs from last 7 days
curl https://your-domain.com/api/webhooks/ips?hours=168 > webhook-ips-$(date +%Y%m%d).json

# Compare with previous week
diff webhook-ips-20260117.json webhook-ips-20260124.json
```

If new IPs appear:
1. Verify they're legitimate Autotask IPs
2. Check user agent matches `Autotask-Webhook`
3. Update whitelist configuration
4. Reload/restart proxy or tunnel

---

## Troubleshooting

### **Webhooks Stopped Working After Whitelisting**

**Check 1: Verify Autotask IP**
```sql
-- Check recent failed webhooks
SELECT source_ip, user_agent, error_message, received_at
FROM webhook_logs
WHERE status = 'failed'
  AND received_at >= NOW() - INTERVAL '1 hour'
ORDER BY received_at DESC;
```

If you see legitimate Autotask IPs being blocked, add them to whitelist.

**Check 2: Proxy Logs**

**Nginx:**
```bash
tail -f /var/log/nginx/pulse-webhooks-error.log | grep 403
```

**Pangolin:**
```bash
sudo journalctl -u pangolin-tunnel -f | grep denied
```

**Check 3: Test Without Whitelist**

Temporarily disable IP whitelist to confirm that's the issue:

**Nginx:** Comment out `allow`/`deny` lines and reload
**Pangolin:** Remove `ipWhitelist` section and restart

If webhooks work without whitelist, the issue is IP configuration.

### **Autotask Changed IPs**

Autotask may change their webhook source IPs:

1. **Check webhook logs** for new IPs
2. **Verify** they're legitimate (user agent, request pattern)
3. **Update whitelist** with new IPs
4. **Keep old IPs** for a few days (in case of rollback)
5. **Remove old IPs** after confirming they're no longer used

### **Getting Blocked Yourself**

If you accidentally block legitimate traffic:

**Quick Fix (Nginx):**
```bash
# Edit config to remove IP whitelist
sudo nano /etc/nginx/sites-available/pulse-webhooks

# Comment out the deny rules
# deny all;

# Reload
sudo systemctl reload nginx
```

**Quick Fix (Pangolin):**
```bash
# Edit config
sudo nano /etc/pangolin/config.yml

# Remove or comment ipWhitelist section

# Restart
sudo systemctl restart pangolin-tunnel
```

---

## Best Practices

### **1. Use CIDR Blocks When Possible**

Instead of individual IPs:
```nginx
# Better
allow 52.10.0.0/16;

# Less flexible
allow 52.10.1.1;
allow 52.10.1.2;
allow 52.10.1.3;
```

### **2. Document Your Whitelist**

Keep a record of:
- Which IPs are whitelisted
- When they were added
- Source of information (Autotask support, logs, etc.)
- Last verified date

Example:
```yaml
# IP Whitelist Documentation
# Last updated: 2026-01-24

ipWhitelist:
  - "52.1.2.3"        # Added: 2026-01-20, Source: Webhook logs
  - "52.5.6.7"        # Added: 2026-01-22, Source: Webhook logs  
  - "52.10.0.0/16"    # Added: 2026-01-24, Source: Autotask support ticket #12345
```

### **3. Monitor Regularly**

- Weekly: Check for new IPs in logs
- Monthly: Audit whitelist against active IPs
- After Autotask updates: Verify webhooks still work

### **4. Combine with Other Security**

IP whitelisting is one layer. Also use:
- ✅ HTTPS/TLS encryption
- ✅ Rate limiting
- ✅ Path-based access control
- ✅ Request validation
- ✅ Monitoring and alerting

### **5. Have a Rollback Plan**

Keep configuration backups:
```bash
# Backup before changes
sudo cp /etc/nginx/sites-available/pulse-webhooks \
       /etc/nginx/sites-available/pulse-webhooks.backup-$(date +%Y%m%d)

# Or for Pangolin
sudo cp /etc/pangolin/config.yml \
       /etc/pangolin/config.yml.backup-$(date +%Y%m%d)
```

---

## API Reference

### **GET /api/webhooks/ips**

Get unique source IPs from webhook logs.

**Parameters:**
- `hours` (optional): Number of hours to look back (default: 168 = 7 days)
- `entityType` (optional): Filter by entity type

**Example:**
```bash
# Last 7 days
curl https://your-domain.com/api/webhooks/ips?hours=168

# Last 24 hours, tickets only
curl https://your-domain.com/api/webhooks/ips?hours=24&entityType=Tickets
```

**Response:**
```json
{
  "success": true,
  "ips": [
    {
      "ip": "52.1.2.3",
      "userAgent": "Autotask-Webhook/1.0",
      "requestCount": 150,
      "successfulCount": 148,
      "failedCount": 2,
      "firstSeen": "2026-01-20T10:00:00Z",
      "lastSeen": "2026-01-24T15:00:00Z",
      "entityTypes": ["Tickets", "Companies"]
    }
  ],
  "count": 1,
  "period": "168 hours"
}
```

---

## Database Queries

### **Get All Unique IPs**
```sql
SELECT DISTINCT source_ip, user_agent
FROM webhook_logs
WHERE source_ip IS NOT NULL
ORDER BY source_ip;
```

### **Get IP Activity Summary**
```sql
SELECT 
  source_ip,
  COUNT(*) as total_requests,
  COUNT(DISTINCT entity_type) as entity_types_count,
  MIN(received_at) as first_seen,
  MAX(received_at) as last_seen,
  ROUND(AVG(processing_time_ms), 2) as avg_processing_ms
FROM webhook_logs
WHERE source_ip IS NOT NULL
GROUP BY source_ip
ORDER BY total_requests DESC;
```

### **Detect Suspicious IPs**
```sql
-- IPs with high failure rates
SELECT 
  source_ip,
  user_agent,
  COUNT(*) as total,
  COUNT(*) FILTER (WHERE status = 'failed') as failed,
  ROUND(100.0 * COUNT(*) FILTER (WHERE status = 'failed') / COUNT(*), 2) as failure_rate
FROM webhook_logs
WHERE source_ip IS NOT NULL
  AND received_at >= NOW() - INTERVAL '24 hours'
GROUP BY source_ip, user_agent
HAVING COUNT(*) FILTER (WHERE status = 'failed') > 5
ORDER BY failure_rate DESC;
```

---

## Summary

**Steps to Enable IP Whitelisting:**

1. ✅ **Run migration** to add IP logging columns
2. ✅ **Deploy updated code** with IP capture
3. ✅ **Receive webhooks** from Autotask (let it run for a day)
4. ✅ **Identify Autotask IPs** via `/api/webhooks/ips` endpoint
5. ✅ **Configure whitelist** in your proxy/tunnel
6. ✅ **Test** that webhooks still work
7. ✅ **Monitor** for new IPs regularly

**Result:**
- Only Autotask's IPs can send webhooks
- Unauthorized requests are blocked
- Additional security layer for your webhook endpoint
- Logged IP data for audit and troubleshooting