Commit graph

23 commits

Author SHA1 Message Date
ff9e34cafe feat: add Autotask tags sync (tag groups, tags, ticket tag associations)
- Migration 057: autotask_tag_groups, autotask_tags, and junction tables
  (ticket_tags, company_tags, configuration_item_tags, contact_tags)
- Add TAG_GROUPS and TAGS to EntityType enum and dependency map
- Add mapTagGroup() and mapTag() entity mapper functions
- Add syncTagGroups(), syncTags(), syncTicketTagAssociations() methods
- Wire TicketTagAssociations bulk sync into full/incremental sync flow
- Add 'exist' operator to QueryFilter type
- No FK on ticket_id (tagged tickets may be outside 2yr sync window)

Synced: 26 tag groups, 7299 tags, 10141 ticket-tag associations
2026-03-20 09:22:40 -04:00
c8be128c5c feat: companies sync — add UDFs and fix all field mappings
Migration 056:
- ADD user_defined_fields JSONB (GIN indexed)
- ADD parent_company_id, web_address, create_date, company_category_id
- ADD is_client_portal_active, is_task_fire_active, is_comanaged
- ADD country_id, bill_to_country_id, bill_to_address_to_use
- ADD invoice_email_message_id, purchase_order_template_id
- ADD additional_address_information, sic_code, stock_market, etc.

entity-mapper.ts mapCompany():
- Fix broken snake_case field refs (was data.invoice_method, data.last_activity_date etc.)
- All corrected to actual Autotask camelCase API names (invoiceMethod, lastActivityDate, etc.)
- was data.webSiteURL → data.webAddress
- was data.taxExempt → data.isTaxExempt
- was data.lastTrackedModifiedDateTime (various) → data.lastTrackedModifiedDateTime
- Add UDF conversion: userDefinedFields[] array → JSONB {name: value} object
- 38 UDFs now stored: PassportalID, MimecastID, MSP Service Model, Short Name,
  Workstation Lifecycle Plan, Seats (End Users), CSProfileUID, UniFiID, etc.
2026-03-17 23:56:40 -04:00
25bb70cfa6 feat: Mimecast email integration — message logs, threat events, 120d retention, admin UI 2026-03-17 16:23:47 -04:00
597ffbee9f fix: widen all VARCHAR columns in configuration_items to TEXT
Serial numbers, hostnames, SNMP strings, RMM identifiers, backup type UDF
all exceed VARCHAR limits in real data. Convert all to TEXT to prevent
future truncation errors during sync.
2026-03-17 15:52:57 -04:00
7762cf284e fix: widen VARCHAR(50) columns in configuration_items that exceed limit
IP addresses, MACs, and mobile numbers can exceed 50 chars (IPv6, UUIDs).
Widen datto_internal_ip, datto_remote_ip, rmm_device_audit_external_ip_address,
rmm_device_audit_ip_address to VARCHAR(255); mac_address and mobile_number to VARCHAR(100).
2026-03-17 15:48:35 -04:00
cb293ea843 fix: configuration_items sync — correct camelCase field mapping, store all UDFs in udfs JSONB
- Fix mapConfigurationItem() to use Autotask camelCase field names (was using
  snake_case keys that never matched, leaving costs/Datto/RMM fields empty)
- Map all 60+ fields explicitly: costs, Datto device fields, RMM audit fields,
  location, vendor, SNMP, backup dates, source cost, etc.
- Store entire userDefinedFields array as udfs JSONB (keyed by UDF name)
- Retain backup_type_udf VARCHAR column for backward compat
- Migration 052: add udfs JSONB column + GIN index to configuration_items
2026-03-17 15:39:42 -04:00
b98c67482a feat: QuickBooks Online integration
- Add QBO OAuth2 client with token refresh (lib/services/qbo-client.ts)
- Add QBO sync service for invoices, payments, deposits, purchases, journal entries, reports (lib/services/qbo-sync-service.ts)
- Add QBO types (lib/types/qbo.ts)
- Add API routes: /api/qbo/auth, /api/qbo/sync, /api/qbo/disconnect
- Add /admin/qbo status and sync management page
- Add legal pages: /legal/eula, /legal/privacy (Intuit app assessment)
- Add QBO nav link under Admin
- Fix reports: remove invalid summarize_column_by, add accounting_method from Preferences API, add showrows=all&showcols=all
- Add CashFlow report type alongside P&L and BalanceSheet
- Add NoReportData check to skip empty report months
- Add intuit_tid capture in error messages
- Add redirect: follow for cluster routing
- Migration 051: qbo_tokens, qbo_invoices, qbo_payments, qbo_deposits, qbo_transactions, qbo_reports tables

Also includes earlier work:
- Ping flap suppression pipeline step
- Ticket digest reports with LLM analysis
- Zabbix WAN monitor and gap analysis
- Kiosk is_deleted filter fixes
- Datto RMM ping target enrichment
- Entity sync soft-delete detection
2026-03-17 07:39:55 -04:00
c518eefdb2 feat: Morning NOC Summary adaptive card for Teams
- Add MorningSummaryService with Zabbix aggregation and adaptive card builder
- Add webhook delivery system with Teams incoming webhooks
- Add admin UI at /admin/morning-summary for webhook/config management
- Add API routes: /send, /test, /webhooks, /webhooks/[id], /config, /history
- Register morning-summary cron job in SyncScheduler (Mon-Fri 6:30 AM)
- Add outages_only filter (Unavailable triggers only)
- Fix host resolution: use getTriggerEnabledHosts to exclude disabled hosts
- Fix resolved events: event.get value:1 scoped to window with r_eventid filter
- Remove emojis from fact rows and section headers in card
- Remove Open Zabbix button (duplicate of View Problems)
- Add migrations: morning_summary_config + morning_summaries tables
- Add outages_only column to morning_summary_config
2026-03-11 09:34:51 -04:00
19605f82aa feat: IT Glue integration, workflow engine, pipelines, Zabbix WAN, notification channels, backup status UI improvements, nav alignment fixes 2026-02-27 14:52:14 -05:00
ed6c4a8b65 feat: Add SentinelOne integration
- Add SentinelOne API client (lib/services/sentinelone-client.ts)
  - Paginated fetching for sites, agents, threats
  - JWT token auth via S1_API_URL / S1_API_TOKEN env vars

- Add SentinelOne sync service (lib/services/sentinelone-sync-service.ts)
  - Full sync: sites, agents, threats into s1_* tables
  - Sync history tracking with per-entity results

- Add DB migration 038: s1_sites, s1_agents, s1_threats,
  s1_company_mappings, s1_sync_history tables

- Add API routes:
  - POST/GET /api/sentinelone/sync
  - GET/POST/DELETE /api/sentinelone/company-mappings
  - GET /api/sentinelone/coverage (fixed Cartesian product bug)

- Add UI pages:
  - /admin/sync/sentinelone — sync admin with history + stats
  - /sentinelone/coverage — AV coverage report per site
  - /sentinelone/mappings — map S1 sites to Autotask companies

- Wire SentinelOne into admin sync overview card grid
- Add SentinelOne Sync to app navigation
- Fix docker-compose: remove explicit S1 env var entries that
  were overwriting env_file values with empty strings
2026-02-27 05:31:31 -05:00
d7c3dc7168 feat: Autotask webhook integration, TicketNotes, Datto RMM, workflow engine, Veeam agents/alarms, AI triage, misc improvements 2026-02-20 10:28:15 -05:00
5dc7a7e66b feat: Veeam VSPC backup integration - sync, compliance, UI
- Database: 7 Veeam tables + backup_type_udf column on configuration_items
- API Client: VSPC REST API v3 client with pagination, rate limiting, Bearer auth
- Sync Service: full/incremental sync for orgs, servers, repos, jobs, agent jobs, workloads
- Scheduler: veeam-incremental (30min) and veeam-full (daily 2AM) schedules
- Compliance Engine: cross-references Autotask config items vs Veeam workloads
- API Endpoints: backup-status, companies, workloads, jobs, repos, compliance, sync
- UI: Backup Status page with Overview + Contract Compliance tabs
- Navigation: added Backup Status link with HardDrive icon
- Docker: added VEEAM_VSPC_URL and VEEAM_VSPC_API_KEY env vars to compose
2026-02-11 21:04:28 -05:00
f27b23bec8 fix: pre-populate classifications instead of syncing from Autotask API
- Autotask REST API doesn't expose ClassificationIcons endpoint
- Created migration to populate 17 standard classification icons
- Removed sync button from UI (classifications are pre-loaded)
- Includes: Tools Only, Co-Managed, Wulf 365 Essentials, Partner, etc.
- Users can now select any classification to exclude from kiosk
2026-02-03 20:54:35 -05:00
b222d97225 feat: add classification icons sync from Autotask
- Created company_classifications table to store Autotask classification icons
- Added getClassificationIcons() method to AutotaskClient
- Created /api/sync/classifications endpoint (GET/POST)
- Updated kiosk settings UI to dynamically load classifications
- Added 'Sync from Autotask' button to pull latest classifications
- Removed hardcoded classification list
- Display classification name and description in checkboxes
- Allow excluding any classification synced from Autotask
2026-02-03 20:48:08 -05:00
root
1d99a8f659 feat: add classification-based filtering and fix company name display
- Added classification column to companies table
- Created excluded_classifications setting (default: Tools Only)
- Updated kiosk stats and activity APIs to filter by classification
- Added comprehensive classification filtering UI with checkboxes
- Support for common classifications (Tools Only, Wulff Consulting Client, Customer)
- Allow custom classification entry
- Fixed company name display in settings (convert string IDs properly)
- Classification filtering works alongside company ID exclusions
2026-02-03 09:17:07 -05:00
root
26bfd503f1 feat: add kiosk settings UI and co-managed client filtering
- Created kiosk_settings table for configuration storage
- Added API endpoints for kiosk settings (GET/POST)
- Filter out co-managed clients (configurable company exclusions)
- Built comprehensive settings UI at /kiosk/settings
- Allow excluding specific companies from kiosk display
- Configurable cycle interval, refresh interval, and RMM alert toggle
- Updated dashboard link to point to settings page
- Applied company exclusion filter to all ticket queries and activity feed
- Default excludes Thrasher Group (ID: 29861361)
2026-02-03 08:32:42 -05:00
root
a050157836 feat: create auvik_tenants and rmm_sites tables for accurate dashboard stats
Added tables to store all tenants/sites from external APIs.
Populated with existing mapped data. Full sync of all tenants/sites
needed for accurate unmapped counts.
2026-02-02 23:01:54 -05:00
root
9f912aed24 feat: add authentication, user management, and admin features
Added comprehensive authentication and authorization system:

Authentication System:
- Better Auth integration with session management
- Login/logout pages and API routes
- Middleware for route protection
- Auth utilities and client libraries

User Management:
- User list, detail, and invite pages
- User API endpoints (CRUD operations)
- Session management for users
- Profile settings page

Role-Based Access Control:
- Role management pages (list, create, edit)
- Permission system with granular controls
- Role assignment to users
- Role API endpoints

Admin Features:
- Audit log page for tracking system events
- Admin settings page
- Audit service for logging user actions

Additional Features:
- Quotes management pages and components
- SalesBldr API integration
- Email service for notifications

Configuration & Documentation:
- Updated docker-compose.yml
- MCP server configuration (mcp.json)
- CVE-2025-55182 security review documentation
- Standards guide and PRD documents
- Re-enabling authentication documentation

Database Migrations:
- 012: Auth tables (users, sessions, accounts, verifications)
- 013: Role tables (roles, permissions, role_permissions, user_roles)
- 014: Admin settings table

UI Updates:
- Updated dashboard layout
- Enhanced app layout with auth integration
2026-01-31 12:43:14 -05:00
root
8405c813a3 feat: add missing Tickets entity fields from Autotask API
Added 10 critical missing fields to Tickets entity:

Database Migration (016):
- billing_code_id: Billing code assignment
- configuration_item_id: Primary asset/CI
- creator_resource_id: Ticket creator
- creator_type: Creator type (resource/contact)
- problem_ticket_id: Link to problem ticket
- rma_status: RMA status tracking
- rma_type: RMA type classification
- service_level_agreement_paused_next_event_hours: SLA pause tracking
- is_assigned_to_comanaged: Co-managed assignment flag
- is_visible_to_comanaged: Co-managed visibility flag

Entity Mapper Updates:
- Added all new fields with correct camelCase mapping
- Ensures all Autotask Tickets API fields are captured

This completes the Tickets entity to match the full Autotask API
specification and should resolve issues with missing ticket data.
2026-01-26 15:26:34 -05:00
root
8e0ca0a63b feat: expand Resources entity to include all Autotask API fields
Add comprehensive field support for Resources entity including:
- Name fields: middleInitial, namePrefix, nameSuffix
- Contact: emailAddress2, emailAddress3, homePhone
- Employment: accountingReferenceID, payrollType, internalCost
- System: emailTypeCode, numberFormat, timeFormat, dateFormat
- Demographics: gender
- Security: licenseType, securityLevel
- Location: defaultServiceDeskRoleID

Changes:
- Migration 015: Add new columns to resources table
- Updated entity mapper to map all Autotask Resource fields
- Expanded TypeScript Resource interface with all fields
- Maintains backward compatibility with existing field names

This ensures all data exposed by the Autotask Resources API is
now captured and stored in the database.
2026-01-26 14:28:51 -05:00
root
a093f8787c feat: add IP address logging to webhooks for whitelisting
Implements comprehensive IP logging for webhook requests to enable
IP whitelisting and security monitoring.

Features:
- Capture source IP from webhook requests (x-forwarded-for, x-real-ip)
- Capture user agent for identification
- Store in webhook_logs table
- New API endpoint: GET /api/webhooks/ips
- View unique IPs with request counts and statistics
- Identify Autotask IPs for whitelisting

Database Changes:
- Added source_ip column (VARCHAR 45) to webhook_logs
- Added user_agent column (TEXT) to webhook_logs
- Added index on source_ip for efficient queries
- Migration 005 for existing installations

API Endpoints:
- GET /api/webhooks/ips?hours=168&entityType=Tickets
  Returns unique IPs with:
  * Request counts (total, successful, failed)
  * First/last seen timestamps
  * Entity types accessed
  * User agent strings

Use Cases:
1. Identify Autotask webhook IPs
2. Configure IP whitelist in nginx/Pangolin/Cloudflare
3. Monitor for unauthorized webhook attempts
4. Audit webhook sources
5. Detect IP changes from Autotask

Security Benefits:
- Enable IP whitelisting for webhook endpoint
- Block unauthorized webhook attempts
- Monitor for suspicious activity
- Audit trail of webhook sources

Documentation:
- Complete IP whitelisting guide (WEBHOOK_IP_WHITELISTING.md)
- Configuration examples for nginx, Pangolin, Cloudflare
- Monitoring queries and best practices
- Troubleshooting guide

Files Modified:
- migrations/004_webhook_support.sql - Added IP columns
- migrations/005_add_webhook_ip_logging.sql - Migration for existing installs
- lib/types/webhook.ts - Added IP fields to WebhookLog
- lib/services/webhook-service.ts - Capture and log IPs
- app/api/webhooks/autotask/route.ts - Extract IP from headers
- app/api/webhooks/ips/route.ts - New IP viewing endpoint
- docs/WEBHOOK_IP_WHITELISTING.md - Complete guide

Next Steps:
1. Run migration (004 for new, 005 for existing)
2. Deploy updated code
3. Receive webhooks from Autotask
4. View IPs via /api/webhooks/ips
5. Configure IP whitelist in proxy/tunnel
2026-01-24 17:28:56 -05:00
root
1f83456199 feat: add webhook support for real-time Autotask updates
Implements comprehensive webhook infrastructure to receive and process
real-time entity updates from Autotask, reducing API calls and improving
data freshness.

Features:
- Webhook receiver endpoint: POST /api/webhooks/autotask
- Automatic entity mapping and upsert to PostgreSQL
- Event logging and tracking in webhook_logs table
- Duplicate event prevention via unique event_id
- Failed event tracking with error messages
- Statistics and monitoring APIs
- Support for 8 entity types: Companies, Tickets, Tasks, Projects,
  Time Entries, Contacts, Contracts, Configuration Items

Architecture:
- WebhookService: Core processing logic
- Database tables: webhook_logs, webhook_configs
- API endpoints: /autotask (receiver), /logs, /stats
- Automatic data mapping using existing entity-mapper

Benefits:
- Near real-time updates (<1 minute vs 24 hours)
- Reduced API usage (webhooks vs polling)
- Complements daily incremental sync for redundancy
- Automatic recovery from webhook failures

Files Added:
- lib/types/webhook.ts - TypeScript types and interfaces
- lib/services/webhook-service.ts - Webhook processing service
- app/api/webhooks/autotask/route.ts - Webhook receiver
- app/api/webhooks/logs/route.ts - Logs API
- app/api/webhooks/stats/route.ts - Statistics API
- migrations/004_webhook_support.sql - Database schema
- docs/WEBHOOK_SETUP.md - Complete setup guide (47 sections)
- docs/WEBHOOKS_README.md - Quick start guide

Next Steps:
1. Run database migration
2. Configure webhooks in Autotask
3. Test endpoint and monitor logs

See docs/WEBHOOK_SETUP.md for detailed setup instructions.
2026-01-24 10:07:20 -05:00
root
6eee14f8af Add comprehensive admin features and multi-system integration
- Add admin dashboard with sync controls and data browser
- Implement RMM, Auvik, and Addigy organization mappings
- Add chunked ticket sync with progress tracking
- Implement entity sync service with rate limiting
- Add analytics engine and performance optimizer
- Create data browser for all PSA entities
- Add navigation components and UI improvements
- Implement background processing and sync services
- Add comprehensive documentation and migration scripts
- Update configuration items with multi-system support
- Enhance contact management and purchase history
- Add issue type assignment and LLM analyzer
- Improve error handling and logging utilities
2025-11-19 14:18:16 -05:00