A container restart leaves analyzer_jobs rows stuck in
fetching/triaging/itglue/analyzing/deep_review forever — the worker's
claimQueuedJob only picks up status='queued', so a job mid-pipeline
when the process died gets orphaned.
resetStaleJobsToQueued() reverts any active-state row whose started_at
is older than 10 min back to 'queued' with started_at=NULL. The worker
calls it once on start() before scheduling the first poll. 10 min is
3x the realistic pipeline ceiling — well past Sonnet+Opus combined.
Logs the count when nonzero so restarts that recover work are visible.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
IT Glue's /flexible_assets endpoint refuses requests without a
filter[flexible-asset-type-id] (returns 422 "Cannot index flexible
assets without providing a flexible asset type ID filter"). The
analyzer's Stage 2 search was caught and tolerated, but never returned
docs.
Added getFlexibleAssetsForOrganization(orgId) on ITGlueClient. It
fetches the type list once per process (memoized), then fans out
per-type fetches with Promise.allSettled so a permission-restricted
type doesn't poison the whole org. Wired into itglue-search and
aggregate-persistence.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Eight sub-phases per docs/ticket-analyzer-phase2-spec.md:
2.1 Schema (migration 070): analyzer_stage_executions table; source_snapshot,
aggregate_fingerprint, fingerprint_generated_at columns on analyzer_analyses.
model_traces marked LEGACY (kept for back-compat).
2.2 Every pipeline stage records a row to analyzer_stage_executions, success
or failure. Worker persists a status='failed' analyzer_analyses row when
the pipeline throws so partial stage records have a parent. Pipeline
exposes raw triage/sonnet/opus responses for downstream stages.
2.3 Stage 3 prompt updated with markdown formatting rules + banned filler
phrases. Added react-markdown + remark-gfm + @tailwindcss/typography.
New <AnalysisMarkdown> component replaces <ProseText>; coerces stray
headers to bold paragraphs.
2.4 Stage 6 fingerprint (Haiku) runs after persistence, failure-tolerant.
scripts/backfill-fingerprints.ts reconstructs Stage 6 input from the
legacy model_traces blob.
2.5 Browse UI rebuild at /analyzer/tickets: multi-select for client/issue/
queue/status/priority/assignee, sticky filter bar, active-filter chips,
bulk selection persisted via localStorage, "Analyze N selected" +
"Generate aggregate report" actions. New <MultiSelect> primitive.
Staleness uses last_activity_date > completed_at heuristic per spec C.1.
2.6 Aggregate reports (migration 071): runner is fire-and-forget, persists
SQL distributions immediately so UI shows partial state during the
Sonnet reduce call. Three endpoints, three pages (/analyzer/reports[/new
/:id]). IT Glue context fetcher capped at 200 doc titles.
2.7 Cost guards (migration 072): per-request $5 confirmation, soft-warn at
$20/day, hard-block at $50/day with ANALYZER_DAILY_COST_OVERRIDE_USERS
override. Every gating decision audited.
2.8 Runbook + build notes updated.
128 vitest tests passing, tsc clean. Migrations 070/071/072 idempotent
(IF NOT EXISTS). model_traces double-write retained — drop in a future
migration once aggregate reports have soaked.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
sendAnalysisShareEmail() reuses the existing nodemailer SMTP transport
(same path as magic-link/invitation mail). Share route persists the
audit row first, then attempts send; on failure returns
{share, emailSent:false, emailError} at HTTP 200 so the audit log
stays intact. Modal surfaces send failures as a warning toast.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
- Add /admin/display-settings page with Kiosk and Mobile sections
- Company category checkbox filter + excluded companies searchable multi-select
- New DB tables: company_categories, company_types (migration 064)
- Sync COMPANY_CATEGORIES via CompanyCategories entity (id/name/isActive)
- Sync COMPANY_TYPES via Companies.companyType picklist
- Add to EntityType, ENTITY_DEPENDENCIES, sync-helpers, entity-mapper, entity-sync
- New API routes: /api/admin/display-settings (GET/POST), /api/data/company-categories, /api/data/companies-list
- Update all 4 routes (kiosk/stats, kiosk/activity, mobile/tickets, mobile/dashboard)
to filter by kiosk_settings company_category_ids + excluded_company_ids
- Add Display Settings nav link (SlidersHorizontal icon) to Admin menu
- Seed kiosk_settings: kiosk_company_category_ids=1, mobile_company_category_ids=1
- bulkUpsert now accepts preserveExistingOnNull column list, using
COALESCE(EXCLUDED.col, table.col) so null incoming values never
overwrite existing non-null DB values
- bulkUpsertRecords passes resource ID columns as preserve-on-null
for TICKETS and TASKS entities
- getValidResourceIds now throws on DB error instead of returning
empty set (which would nullify every resource reference)
- Fix mimecast mailbox-remediate fetch handlers to check res.ok and
content-type before calling res.json(), preventing JSON parse crash
on 502 Bad Gateway responses
- Add searchMailboxMessages, deleteMailboxMessage, moveToDeletedItems to MsGraphClient
- POST /api/mimecast/mailbox-remediate: search, move, delete actions with permission error handling
- DeliveredAnalysisDialog: Remove from mailbox panel with search → confirm → delete flow
- Shows matching messages in mailbox with checkboxes, received time, read/unread status
- Moves selected to Deleted Items (recoverable) via Graph API
- Surfaces clear permission guidance if Mail.ReadWrite not yet granted
- New POST /api/mimecast/delivered route using message-finder/search API
- DeliveredMailTab: search by recipient, sender, subject, time range (6h–7d)
- Results table with status badge, spam score, row tinting for high/moderate risk
- Summary stats bar: total / high spam (≥10) / moderate (5-9) / clean counts
- Sort by date or spam score; filter by status (accepted/held/rejected/bounced)
- DeliveredAnalysisDialog: explains why high-score mail got through, envelope mismatch detection, actionable remediation steps (block domain, adjust policy threshold, report)
- MimecastDeliveredMessage interface + searchDeliveredMessages() method in client
- Add releaseHeldMessage() to MimecastClient (POST /api/gateway/hold-release)
- Add POST /api/mimecast/held/release route
- HeldMailTab: tenant selector before load (defaults to Wulf), only fetches selected tenant
- Release button per row with spinner + optimistic removal on success
- Error shown inline under Release button if release fails
getMimecastClientForTenant now always uses accountCode='' so the
x-mc-account header is never sent. Sending your own account code
with tenant-specific OAuth credentials causes Mimecast to 403
(it interprets it as an invalid impersonation attempt).
- export const dynamic = 'force-dynamic' on /api/mimecast/held to prevent Next.js caching
- Add AbortController timeout (20s) per request in MimecastClient.request()
- getHeldMessages: 403 fallback without admin:true flag for tenants lacking permission
- Reduce maxMessages default to 100 (10 pages) to stay within route timeout
- Show 'permission denied' tooltip in tenant badge for 403 errors
- Surface HTTP errors in HeldMailTab instead of silently failing
- Add missing favicons: sentinelone.ico, itglue.ico, mimecast.ico
The Autotask Tasks bulk API does not return projectID in its response,
causing all tasks.project_id to be NULL. This fixes it by:
- Adding project_phases as a synced entity (Autotask endpoint: /Phases)
- Migration 059: project_phases table with project_id, phase_number,
estimated_hours, start/due dates, parent_phase_id, is_scheduled
- EntityType.PROJECT_PHASES added to all sync maps and dependency graph
(depends on PROJECTS, runs before TASKS in sync order)
- buildProjectPhasesFilter: Phases endpoint requires a filter (id > 0)
- mapProjectPhase: maps Autotask field names to DB columns
- Post-sync backfill in syncEntity: after each project_phases sync,
UPDATE tasks SET project_id = pp.project_id FROM project_phases pp
JOIN projects p WHERE tasks.phase_id = pp.id
Only backfills where the project exists in our DB (FK constraint on
tasks.project_id; archived projects are skipped gracefully)
Result: 2,455 of 4,966 tasks now have project_id populated. Tasks
belonging to archived/completed projects have phase_id resolvable via
project_phases even when project_id remains NULL.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- API key auth via x-openclaw-key header (OPENCLAW_API_KEY env var)
- GET /api/openclaw/sync/status
- POST /api/openclaw/sync/autotask/incremental
- POST /api/openclaw/sync/autotask/full
- POST /api/openclaw/sync/autotask/entity { entities: [...] }
- POST /api/openclaw/sync/datto-rmm { syncType: full|incremental }
- POST /api/openclaw/sync/sentinelone
- POST /api/openclaw/sync/veeam { syncType: full|incremental }
- POST /api/openclaw/sync/zoom
- POST /api/openclaw/sync/engagement
- POST /api/openclaw/sync/qbo { syncType: full|incremental }
- POST /api/openclaw/sync/zabbix
- POST /api/openclaw/sync/itglue
All routes bypass Better Auth middleware, delegate to existing sync services
Previously the incremental filter compared dateWorked >= lastSyncTime,
meaning edits to existing entries (hours adjusted, notes changed, PTO
blocks entered in advance) were never picked up after initial ingestion.
Switch both the lastModifiedField mapping and buildTimeEntriesFilter to
use lastModifiedDateTime so any created or modified entry is captured
on the next scheduled sync.
- Autotask REST API doesn't expose ClassificationIcons endpoint
- Created migration to populate 17 standard classification icons
- Removed sync button from UI (classifications are pre-loaded)
- Includes: Tools Only, Co-Managed, Wulf 365 Essentials, Partner, etc.
- Users can now select any classification to exclude from kiosk
- Created company_classifications table to store Autotask classification icons
- Added getClassificationIcons() method to AutotaskClient
- Created /api/sync/classifications endpoint (GET/POST)
- Updated kiosk settings UI to dynamically load classifications
- Added 'Sync from Autotask' button to pull latest classifications
- Removed hardcoded classification list
- Display classification name and description in checkboxes
- Allow excluding any classification synced from Autotask
The scheduled sync was using AUTOTASK_INTEGRATION_CODE but the actual
environment variable is AUTOTASK_API_INTEGRATION_CODE. This caused
scheduled syncs to fail with 'IntegrationCode is invalid' while
manual syncs worked fine.
Changed:
- AUTOTASK_INTEGRATION_CODE -> AUTOTASK_API_INTEGRATION_CODE
This aligns the scheduler with autotask-factory.ts which was already
using the correct variable name.
Fixed critical logic bug where Companies and Resources were not getting
proper filters applied when falling back from incremental to full sync.
Root Cause:
- During scheduled incremental sync, Companies and Resources correctly
identify they don't support incremental sync
- However, the filter-building logic was in an else block that only
executed for non-incremental syncs
- This caused Companies and Resources to sync with NO filters at all
- Autotask API now rejects queries without filters, returning:
'Value cannot be null. Parameter name: filters'
Fix:
- Restructured logic so filter building happens for BOTH:
1. Non-incremental (full) syncs
2. Incremental syncs that fall back to full sync
- Companies and Resources now get active filters applied even during
scheduled incremental syncs
This resolves the scheduled sync failures for Companies, Resources,
and all other entities that were failing due to the cascading effect
of early failures.
Added 10 critical missing fields to Tickets entity:
Database Migration (016):
- billing_code_id: Billing code assignment
- configuration_item_id: Primary asset/CI
- creator_resource_id: Ticket creator
- creator_type: Creator type (resource/contact)
- problem_ticket_id: Link to problem ticket
- rma_status: RMA status tracking
- rma_type: RMA type classification
- service_level_agreement_paused_next_event_hours: SLA pause tracking
- is_assigned_to_comanaged: Co-managed assignment flag
- is_visible_to_comanaged: Co-managed visibility flag
Entity Mapper Updates:
- Added all new fields with correct camelCase mapping
- Ensures all Autotask Tickets API fields are captured
This completes the Tickets entity to match the full Autotask API
specification and should resolve issues with missing ticket data.
Fixed field names to use proper camelCase from Autotask API instead of
snake_case. This should resolve the issue where assigned_resource_id
was NULL because the mapper was looking for incorrect field names.
Changed mappings:
- firstResponseDateTime (was first_response_date_time)
- resolutionPlanDateTime (was resolution_plan_date_time)
- resolvedDateTime (was resolved_date_time)
- firstResponseAssignedResourceID (was first_response_assigned_resource_id)
- firstResponseInitiatingResourceID (was first_response_initiating_resource_id)
- projectID, opportunityID, contractID, monitorID (were snake_case)
- All change approval and ticket fields converted to camelCase
- All service thermometer fields converted to camelCase
This fix should populate assigned_resource_id and other fields correctly
on the next ticket sync.
Autotask API returns string values like 'PRIMARY' for some integer
fields (e.g., emailTypeCode). Added safeInt() helper function to:
- Parse numeric values correctly
- Return null for non-numeric strings
- Handle null/undefined/empty values
This prevents database errors when syncing Resources with non-standard
field values from Autotask API.
Add comprehensive field support for Resources entity including:
- Name fields: middleInitial, namePrefix, nameSuffix
- Contact: emailAddress2, emailAddress3, homePhone
- Employment: accountingReferenceID, payrollType, internalCost
- System: emailTypeCode, numberFormat, timeFormat, dateFormat
- Demographics: gender
- Security: licenseType, securityLevel
- Location: defaultServiceDeskRoleID
Changes:
- Migration 015: Add new columns to resources table
- Updated entity mapper to map all Autotask Resource fields
- Expanded TypeScript Resource interface with all fields
- Maintains backward compatibility with existing field names
This ensures all data exposed by the Autotask Resources API is
now captured and stored in the database.
Companies and Resources entities in Autotask API do not support
date-based filtering for incremental syncs. When attempting incremental
sync, Autotask returns errors:
- Companies: 'Unable to find lastTrackedModificationDateTime'
- Resources: 'Unable to find lastModifiedDate'
Solution:
- Skip incremental sync for these entities
- Always perform full sync for Companies and Resources
- Log informational message when falling back to full sync
- Other entities continue to support incremental sync normally
This prevents sync failures while maintaining data freshness for
Companies and Resources through full syncs.
Fixed Autotask API errors for Billing Items and Time Entries during
incremental syncs by using the correct field names:
- Billing Items: Changed from 'createDate' to 'itemDate'
- Time Entries: Changed from 'lastModifiedDate' to 'dateWorked'
These field names match what's used in full sync filters and are
accepted by the Autotask API.
Errors fixed:
- 'Unable to find createDate in the BillingItem Entity'
- 'Unable to find lastModifiedDate in the TimeEntry Entity'
This allows incremental syncs to work properly for these entities.
Fixed TypeScript errors in sync-scheduler:
- Added constructor to initialize SyncService with AutotaskClient
- Fixed AutotaskConfig property names (password, apiIntegrationCode)
- Changed method calls to match SyncService API (incrementalSync, fullSync)
- Fixed syncService references to use this.syncService
This resolves build errors preventing Docker image creation.
Implements comprehensive IP logging for webhook requests to enable
IP whitelisting and security monitoring.
Features:
- Capture source IP from webhook requests (x-forwarded-for, x-real-ip)
- Capture user agent for identification
- Store in webhook_logs table
- New API endpoint: GET /api/webhooks/ips
- View unique IPs with request counts and statistics
- Identify Autotask IPs for whitelisting
Database Changes:
- Added source_ip column (VARCHAR 45) to webhook_logs
- Added user_agent column (TEXT) to webhook_logs
- Added index on source_ip for efficient queries
- Migration 005 for existing installations
API Endpoints:
- GET /api/webhooks/ips?hours=168&entityType=Tickets
Returns unique IPs with:
* Request counts (total, successful, failed)
* First/last seen timestamps
* Entity types accessed
* User agent strings
Use Cases:
1. Identify Autotask webhook IPs
2. Configure IP whitelist in nginx/Pangolin/Cloudflare
3. Monitor for unauthorized webhook attempts
4. Audit webhook sources
5. Detect IP changes from Autotask
Security Benefits:
- Enable IP whitelisting for webhook endpoint
- Block unauthorized webhook attempts
- Monitor for suspicious activity
- Audit trail of webhook sources
Documentation:
- Complete IP whitelisting guide (WEBHOOK_IP_WHITELISTING.md)
- Configuration examples for nginx, Pangolin, Cloudflare
- Monitoring queries and best practices
- Troubleshooting guide
Files Modified:
- migrations/004_webhook_support.sql - Added IP columns
- migrations/005_add_webhook_ip_logging.sql - Migration for existing installs
- lib/types/webhook.ts - Added IP fields to WebhookLog
- lib/services/webhook-service.ts - Capture and log IPs
- app/api/webhooks/autotask/route.ts - Extract IP from headers
- app/api/webhooks/ips/route.ts - New IP viewing endpoint
- docs/WEBHOOK_IP_WHITELISTING.md - Complete guide
Next Steps:
1. Run migration (004 for new, 005 for existing)
2. Deploy updated code
3. Receive webhooks from Autotask
4. View IPs via /api/webhooks/ips
5. Configure IP whitelist in proxy/tunnel
