docs(260721-fy8): pre-dispatch plan for fix mimecast and qbo sync scheduler dispatch and reschedule mimecast cron
This commit is contained in:
parent
a0d01cfe8b
commit
d41d6e7c0b
1 changed files with 239 additions and 0 deletions
|
|
@ -0,0 +1,239 @@
|
||||||
|
---
|
||||||
|
phase: quick-260721-fy8
|
||||||
|
plan: 01
|
||||||
|
type: execute
|
||||||
|
wave: 1
|
||||||
|
depends_on: []
|
||||||
|
files_modified:
|
||||||
|
- lib/services/sync-scheduler.ts
|
||||||
|
- migrations/101_reschedule_mimecast_sync.sql
|
||||||
|
autonomous: true
|
||||||
|
requirements: [FIX-SCHED-DISPATCH, FIX-CRON-COLLISION]
|
||||||
|
|
||||||
|
must_haves:
|
||||||
|
truths:
|
||||||
|
- "The mimecast-sync schedule invokes real Mimecast sync logic, not a generic Autotask full sync"
|
||||||
|
- "The qbo schedules invoke the QBO sync service, not a generic Autotask full sync"
|
||||||
|
- "mimecast-sync no longer shares the 0 2 * * * cron slot with qbo-sync-2am and veeam-full"
|
||||||
|
- "npx tsc --noEmit --pretty passes clean"
|
||||||
|
artifacts:
|
||||||
|
- path: "lib/services/sync-scheduler.ts"
|
||||||
|
provides: "mimecast-sync and qbo dispatch branches in executeScheduledSync()"
|
||||||
|
contains: "config.sync_type === 'mimecast-sync'"
|
||||||
|
- path: "migrations/101_reschedule_mimecast_sync.sql"
|
||||||
|
provides: "Guarded UPDATE moving mimecast-sync off 0 2 * * *"
|
||||||
|
contains: "UPDATE sync_schedules"
|
||||||
|
key_links:
|
||||||
|
- from: "lib/services/sync-scheduler.ts"
|
||||||
|
to: "lib/services/mimecast-sync-service.ts"
|
||||||
|
via: "runMimecastIncrementalSync"
|
||||||
|
pattern: "runMimecastIncrementalSync"
|
||||||
|
- from: "lib/services/sync-scheduler.ts"
|
||||||
|
to: "lib/services/qbo-sync-service.ts"
|
||||||
|
via: "getQboSyncService().incrementalSync"
|
||||||
|
pattern: "getQboSyncService"
|
||||||
|
---
|
||||||
|
|
||||||
|
<objective>
|
||||||
|
Fix the sync scheduler so the `mimecast-sync` and `qbo` scheduled jobs call their real
|
||||||
|
sync logic instead of silently falling through to the generic Autotask `fullSync()`
|
||||||
|
catch-all, and reschedule `mimecast-sync`'s cron off the `0 2 * * *` collision it shares
|
||||||
|
with `qbo-sync-2am` and `veeam-full`.
|
||||||
|
|
||||||
|
Purpose: Two nightly integrations (Mimecast, QBO) never actually run on schedule — they
|
||||||
|
trigger a full Autotask entity sync instead — and their misfire contends for the
|
||||||
|
SyncService singleton mutex, causing the "A sync operation is already in progress" lock
|
||||||
|
errors seen at 2 AM.
|
||||||
|
Output: Two new dispatch branches in `executeScheduledSync()`, `'mimecast-sync'` added to
|
||||||
|
the `sync_type` union, and a new guarded migration (plus live DB apply) moving
|
||||||
|
`mimecast-sync` to a collision-free time.
|
||||||
|
</objective>
|
||||||
|
|
||||||
|
<execution_context>
|
||||||
|
@$HOME/.claude/get-shit-done/workflows/execute-plan.md
|
||||||
|
@$HOME/.claude/get-shit-done/templates/summary.md
|
||||||
|
</execution_context>
|
||||||
|
|
||||||
|
<context>
|
||||||
|
@CLAUDE.md
|
||||||
|
@lib/services/sync-scheduler.ts
|
||||||
|
|
||||||
|
<interfaces>
|
||||||
|
<!-- Verified from the codebase — use these directly, no exploration needed. -->
|
||||||
|
|
||||||
|
From lib/services/sync-scheduler.ts (line 25) — the sync_type union currently
|
||||||
|
INCLUDES 'qbo' but NOT 'mimecast-sync'. Comparing config.sync_type === 'mimecast-sync'
|
||||||
|
without adding it to the union is a TS "no overlap" error, so the union MUST be
|
||||||
|
extended.
|
||||||
|
|
||||||
|
From lib/services/mimecast-sync-service.ts:
|
||||||
|
export async function runMimecastIncrementalSync(): Promise<MimecastSyncResult>
|
||||||
|
export interface MimecastSyncResult {
|
||||||
|
messagesUpserted: number;
|
||||||
|
threatsUpserted: number;
|
||||||
|
bodiesFetched: number;
|
||||||
|
purgedMessages: number;
|
||||||
|
errors: string[];
|
||||||
|
durationMs: number;
|
||||||
|
}
|
||||||
|
|
||||||
|
From lib/services/mimecast-client.ts:
|
||||||
|
export function isMimecastConfigured(): boolean
|
||||||
|
|
||||||
|
From lib/services/qbo-sync-service.ts:
|
||||||
|
async incrementalSync(triggeredBy = 'system'): Promise<QboSyncResult> // class method
|
||||||
|
async fullSync(triggeredBy = 'system'): Promise<QboSyncResult>
|
||||||
|
export function getQboSyncService(): QboSyncService
|
||||||
|
// Manual route app/api/qbo/sync/route.ts uses incrementalSync for the non-full path.
|
||||||
|
|
||||||
|
Existing pax8-daily branch (lines 478-493) — the disable-check pattern to mirror for QBO:
|
||||||
|
const disabledRes = await postgresClient.query...(
|
||||||
|
"SELECT disabled FROM integration_settings WHERE key = 'pax8'"
|
||||||
|
);
|
||||||
|
const isDisabled = disabledRes.rows[0]?.disabled === true;
|
||||||
|
|
||||||
|
Confirmed live: integration_settings has both a 'qbo' row and a 'mimecast' row
|
||||||
|
(disabled = false for both).
|
||||||
|
|
||||||
|
Recent branches (device-link-reconcile, integration-health, appgate, tickets-reconcile,
|
||||||
|
phishing-sweep, pax8-daily) use dynamic `await import('@/lib/services/...')` inside the
|
||||||
|
branch. Match that style for the two new branches — it keeps the change localized and
|
||||||
|
avoids adding top-of-file imports.
|
||||||
|
</interfaces>
|
||||||
|
|
||||||
|
Live sync_schedules cron map (queried 2026-07-21) — used to pick a collision-free slot:
|
||||||
|
0 2 * * * -> mimecast-sync (STALE, moving), qbo-sync-2am, veeam-full
|
||||||
|
0 3 * * 0 -> weekly-full
|
||||||
|
0 4 * * * -> contract-services, pax8-daily
|
||||||
|
30 4 * * * -> tickets-reconcile
|
||||||
|
0 5 * * * -> phishing-sweep
|
||||||
|
15 * * * * -> device-link-reconcile (fires :15 EVERY hour — avoid minute 15)
|
||||||
|
slash-30 -> veeam-* (fires :00 and :30 — avoid minutes 0 and 30)
|
||||||
|
New target: 45 4 * * * (4:45 AM — minute 45 is unused anywhere; no collision)
|
||||||
|
</context>
|
||||||
|
|
||||||
|
<tasks>
|
||||||
|
|
||||||
|
<task type="auto">
|
||||||
|
<name>Task 1: Add mimecast-sync and qbo dispatch branches to executeScheduledSync()</name>
|
||||||
|
<files>lib/services/sync-scheduler.ts</files>
|
||||||
|
<action>
|
||||||
|
Two edits, both minimal and matching the existing if/else-if style. Do NOT refactor
|
||||||
|
the chain into a lookup table and do NOT touch any other branch.
|
||||||
|
|
||||||
|
(a) Extend the sync_type union on line 25 (the ScheduleConfig.sync_type type) by
|
||||||
|
adding 'mimecast-sync'. 'qbo' is already present — leave it. Without this, the string
|
||||||
|
comparison in the new mimecast branch is a TS "no overlap" error.
|
||||||
|
|
||||||
|
(b) Insert two new else-if branches into the chain in executeScheduledSync(). Place
|
||||||
|
them BEFORE the final `else if (config.sync_type === 'incremental')` and the final
|
||||||
|
catch-all `else` block, so the catch-all remains reachable only for 'full'/legacy
|
||||||
|
full-sync types. Do not alter the 'incremental' branch or the final else (generic
|
||||||
|
fullSync) behavior.
|
||||||
|
|
||||||
|
Mimecast branch (mirror the engagement-daily/zoom-daily configured-gate pattern, but
|
||||||
|
use dynamic import to match the recent branch style):
|
||||||
|
- else if (config.sync_type === 'mimecast-sync')
|
||||||
|
- Dynamically import isMimecastConfigured from @/lib/services/mimecast-client and
|
||||||
|
runMimecastIncrementalSync from @/lib/services/mimecast-sync-service.
|
||||||
|
- If not configured, console.log a skip line matching the engagement/zoom wording,
|
||||||
|
e.g. "[SCHEDULER] Skipping mimecast-sync — Mimecast not configured".
|
||||||
|
- Otherwise call runMimecastIncrementalSync(), capture the result, and log a one-line
|
||||||
|
summary using the ACTUAL MimecastSyncResult fields (do not invent fields):
|
||||||
|
messagesUpserted, threatsUpserted, bodiesFetched, purgedMessages,
|
||||||
|
errors.length, durationMs — following the one-line summary format used by the
|
||||||
|
phishing-sweep and device-link-reconcile branches.
|
||||||
|
|
||||||
|
QBO branch (mirror the pax8-daily integration_settings disable-check pattern):
|
||||||
|
- else if (config.sync_type === 'qbo')
|
||||||
|
- Query "SELECT disabled FROM integration_settings WHERE key = 'qbo'" exactly like the
|
||||||
|
pax8 branch, compute isDisabled = rows[0]?.disabled === true.
|
||||||
|
- If disabled, console.log "[SCHEDULER] Skipping qbo sync — QBO disabled via /admin/integrations"
|
||||||
|
and do nothing else.
|
||||||
|
- Otherwise dynamically import getQboSyncService from @/lib/services/qbo-sync-service
|
||||||
|
and call getQboSyncService().incrementalSync('scheduled'). Use incrementalSync, not
|
||||||
|
fullSync — that matches the manual /api/qbo/sync non-full path and is correct for a
|
||||||
|
twice-daily recurring job. QBO has no env-var configured check like pax8 (it uses
|
||||||
|
stored OAuth tokens), so only the disabled check is needed.
|
||||||
|
|
||||||
|
Because both branches replace a fall-through to the SyncService singleton, this also
|
||||||
|
removes mimecast-sync and qbo from contending for that mutex.
|
||||||
|
</action>
|
||||||
|
<verify>
|
||||||
|
<automated>cd /opt/stacks/pulse && npx tsc --noEmit --pretty 2>&1 | tail -5 && grep -c "config.sync_type === 'mimecast-sync'" lib/services/sync-scheduler.ts && grep -c "getQboSyncService" lib/services/sync-scheduler.ts && grep -c "runMimecastIncrementalSync" lib/services/sync-scheduler.ts</automated>
|
||||||
|
</verify>
|
||||||
|
<done>
|
||||||
|
tsc passes clean; both new branches present; catch-all fullSync and incremental
|
||||||
|
branches unchanged; 'mimecast-sync' added to the sync_type union. No test file exists
|
||||||
|
for sync-scheduler (confirmed — only mimecast-client.test.ts), so no test changes are
|
||||||
|
made, per the constraint against adding a new scheduler test file.
|
||||||
|
</done>
|
||||||
|
</task>
|
||||||
|
|
||||||
|
<task type="auto">
|
||||||
|
<name>Task 2: Create migration 101 to reschedule mimecast-sync + apply to live DB</name>
|
||||||
|
<files>migrations/101_reschedule_mimecast_sync.sql</files>
|
||||||
|
<action>
|
||||||
|
Create migrations/101_reschedule_mimecast_sync.sql following the repo's schedule-table
|
||||||
|
migration precedent (migrations/098_phishing_sweep_schedule.sql,
|
||||||
|
migrations/090_ticket_reconcile_schedule.sql) — but an UPDATE, not an INSERT, since the
|
||||||
|
mimecast-sync row already exists.
|
||||||
|
|
||||||
|
The UPDATE must be safe to re-run and must NOT clobber an admin's manual change. Guard
|
||||||
|
it on the stale value so it is a no-op if already moved:
|
||||||
|
UPDATE sync_schedules
|
||||||
|
SET cron_expression = '45 4 * * *', updated_at = NOW()
|
||||||
|
WHERE id = 'mimecast-sync' AND cron_expression = '0 2 * * *';
|
||||||
|
|
||||||
|
Add a leading SQL comment block explaining WHY (2 AM 3-way collision with qbo-sync-2am
|
||||||
|
+ veeam-full, plus SyncService mutex contention) — matching the commenting style of
|
||||||
|
migration 098.
|
||||||
|
|
||||||
|
Chosen new time: 45 4 * * * (4:45 AM). Verified collision-free against the live table:
|
||||||
|
minute 45 is used by nothing; it avoids 0 4 (contract-services, pax8-daily), 30 4
|
||||||
|
(tickets-reconcile), the 15 * * * * hourly device-link job, and the */30 :00/:30 veeam
|
||||||
|
jobs.
|
||||||
|
|
||||||
|
Then apply the same UPDATE directly to the live container so it takes effect without
|
||||||
|
waiting for a fresh-volume Postgres init (per CLAUDE.md, migrations only auto-apply on
|
||||||
|
first volume boot). Run:
|
||||||
|
docker exec pulse-postgres psql -U pulse_user -d pulse_autotask -c "UPDATE sync_schedules SET cron_expression = '45 4 * * *', updated_at = NOW() WHERE id = 'mimecast-sync' AND cron_expression = '0 2 * * *';"
|
||||||
|
|
||||||
|
Note in the SUMMARY: the running in-memory cron task keeps the old time until the app
|
||||||
|
container restarts (the scheduler re-loads schedules from the DB on init) or
|
||||||
|
reloadAllSchedules() is invoked. Deploying the Task 1 code change restarts the
|
||||||
|
container, which reloads the new cron from the DB — so no separate restart step is
|
||||||
|
needed as long as the code fix is deployed.
|
||||||
|
</action>
|
||||||
|
<verify>
|
||||||
|
<automated>cd /opt/stacks/pulse && test -f migrations/101_reschedule_mimecast_sync.sql && grep -q "UPDATE sync_schedules" migrations/101_reschedule_mimecast_sync.sql && echo MIGRATION_FILE_OK && docker exec pulse-postgres psql -U pulse_user -d pulse_autotask -t -c "SELECT cron_expression FROM sync_schedules WHERE id = 'mimecast-sync';"</automated>
|
||||||
|
</verify>
|
||||||
|
<done>
|
||||||
|
migrations/101_reschedule_mimecast_sync.sql exists with a guarded UPDATE keyed on the
|
||||||
|
stale '0 2 * * *' value; live DB query returns '45 4 * * *' for mimecast-sync; no other
|
||||||
|
schedule row modified.
|
||||||
|
</done>
|
||||||
|
</task>
|
||||||
|
|
||||||
|
</tasks>
|
||||||
|
|
||||||
|
<verification>
|
||||||
|
- npx tsc --noEmit --pretty passes clean.
|
||||||
|
- executeScheduledSync() contains a 'mimecast-sync' branch calling runMimecastIncrementalSync()
|
||||||
|
behind isMimecastConfigured(), and a 'qbo' branch calling getQboSyncService().incrementalSync('scheduled')
|
||||||
|
behind an integration_settings disabled check.
|
||||||
|
- The generic catch-all and the 'incremental' branch behave exactly as before.
|
||||||
|
- No existing branch (phishing-sweep, pax8-daily, veeam-*, etc.) is modified.
|
||||||
|
- Live sync_schedules shows mimecast-sync at '45 4 * * *' with no cron collision.
|
||||||
|
</verification>
|
||||||
|
|
||||||
|
<success_criteria>
|
||||||
|
- Mimecast and QBO scheduled jobs run their real sync logic on schedule.
|
||||||
|
- The 0 2 * * * three-way collision is eliminated (mimecast-sync moved to 4:45 AM).
|
||||||
|
- Migration 101 is safe to re-run and safe against an admin's manual cron change.
|
||||||
|
- Type check passes; change is minimal and matches existing dispatch style.
|
||||||
|
</success_criteria>
|
||||||
|
|
||||||
|
<output>
|
||||||
|
Create `.planning/quick/260721-fy8-fix-mimecast-and-qbo-sync-scheduler-disp/260721-fy8-SUMMARY.md` when done.
|
||||||
|
</output>
|
||||||
Loading…
Add table
Add a link
Reference in a new issue