docs(260721-fy8): pre-dispatch plan for fix mimecast and qbo sync scheduler dispatch and reschedule mimecast cron
This commit is contained in:
parent
a0d01cfe8b
commit
d41d6e7c0b
1 changed files with 239 additions and 0 deletions
|
|
@ -0,0 +1,239 @@
|
|||
---
|
||||
phase: quick-260721-fy8
|
||||
plan: 01
|
||||
type: execute
|
||||
wave: 1
|
||||
depends_on: []
|
||||
files_modified:
|
||||
- lib/services/sync-scheduler.ts
|
||||
- migrations/101_reschedule_mimecast_sync.sql
|
||||
autonomous: true
|
||||
requirements: [FIX-SCHED-DISPATCH, FIX-CRON-COLLISION]
|
||||
|
||||
must_haves:
|
||||
truths:
|
||||
- "The mimecast-sync schedule invokes real Mimecast sync logic, not a generic Autotask full sync"
|
||||
- "The qbo schedules invoke the QBO sync service, not a generic Autotask full sync"
|
||||
- "mimecast-sync no longer shares the 0 2 * * * cron slot with qbo-sync-2am and veeam-full"
|
||||
- "npx tsc --noEmit --pretty passes clean"
|
||||
artifacts:
|
||||
- path: "lib/services/sync-scheduler.ts"
|
||||
provides: "mimecast-sync and qbo dispatch branches in executeScheduledSync()"
|
||||
contains: "config.sync_type === 'mimecast-sync'"
|
||||
- path: "migrations/101_reschedule_mimecast_sync.sql"
|
||||
provides: "Guarded UPDATE moving mimecast-sync off 0 2 * * *"
|
||||
contains: "UPDATE sync_schedules"
|
||||
key_links:
|
||||
- from: "lib/services/sync-scheduler.ts"
|
||||
to: "lib/services/mimecast-sync-service.ts"
|
||||
via: "runMimecastIncrementalSync"
|
||||
pattern: "runMimecastIncrementalSync"
|
||||
- from: "lib/services/sync-scheduler.ts"
|
||||
to: "lib/services/qbo-sync-service.ts"
|
||||
via: "getQboSyncService().incrementalSync"
|
||||
pattern: "getQboSyncService"
|
||||
---
|
||||
|
||||
<objective>
|
||||
Fix the sync scheduler so the `mimecast-sync` and `qbo` scheduled jobs call their real
|
||||
sync logic instead of silently falling through to the generic Autotask `fullSync()`
|
||||
catch-all, and reschedule `mimecast-sync`'s cron off the `0 2 * * *` collision it shares
|
||||
with `qbo-sync-2am` and `veeam-full`.
|
||||
|
||||
Purpose: Two nightly integrations (Mimecast, QBO) never actually run on schedule — they
|
||||
trigger a full Autotask entity sync instead — and their misfire contends for the
|
||||
SyncService singleton mutex, causing the "A sync operation is already in progress" lock
|
||||
errors seen at 2 AM.
|
||||
Output: Two new dispatch branches in `executeScheduledSync()`, `'mimecast-sync'` added to
|
||||
the `sync_type` union, and a new guarded migration (plus live DB apply) moving
|
||||
`mimecast-sync` to a collision-free time.
|
||||
</objective>
|
||||
|
||||
<execution_context>
|
||||
@$HOME/.claude/get-shit-done/workflows/execute-plan.md
|
||||
@$HOME/.claude/get-shit-done/templates/summary.md
|
||||
</execution_context>
|
||||
|
||||
<context>
|
||||
@CLAUDE.md
|
||||
@lib/services/sync-scheduler.ts
|
||||
|
||||
<interfaces>
|
||||
<!-- Verified from the codebase — use these directly, no exploration needed. -->
|
||||
|
||||
From lib/services/sync-scheduler.ts (line 25) — the sync_type union currently
|
||||
INCLUDES 'qbo' but NOT 'mimecast-sync'. Comparing config.sync_type === 'mimecast-sync'
|
||||
without adding it to the union is a TS "no overlap" error, so the union MUST be
|
||||
extended.
|
||||
|
||||
From lib/services/mimecast-sync-service.ts:
|
||||
export async function runMimecastIncrementalSync(): Promise<MimecastSyncResult>
|
||||
export interface MimecastSyncResult {
|
||||
messagesUpserted: number;
|
||||
threatsUpserted: number;
|
||||
bodiesFetched: number;
|
||||
purgedMessages: number;
|
||||
errors: string[];
|
||||
durationMs: number;
|
||||
}
|
||||
|
||||
From lib/services/mimecast-client.ts:
|
||||
export function isMimecastConfigured(): boolean
|
||||
|
||||
From lib/services/qbo-sync-service.ts:
|
||||
async incrementalSync(triggeredBy = 'system'): Promise<QboSyncResult> // class method
|
||||
async fullSync(triggeredBy = 'system'): Promise<QboSyncResult>
|
||||
export function getQboSyncService(): QboSyncService
|
||||
// Manual route app/api/qbo/sync/route.ts uses incrementalSync for the non-full path.
|
||||
|
||||
Existing pax8-daily branch (lines 478-493) — the disable-check pattern to mirror for QBO:
|
||||
const disabledRes = await postgresClient.query...(
|
||||
"SELECT disabled FROM integration_settings WHERE key = 'pax8'"
|
||||
);
|
||||
const isDisabled = disabledRes.rows[0]?.disabled === true;
|
||||
|
||||
Confirmed live: integration_settings has both a 'qbo' row and a 'mimecast' row
|
||||
(disabled = false for both).
|
||||
|
||||
Recent branches (device-link-reconcile, integration-health, appgate, tickets-reconcile,
|
||||
phishing-sweep, pax8-daily) use dynamic `await import('@/lib/services/...')` inside the
|
||||
branch. Match that style for the two new branches — it keeps the change localized and
|
||||
avoids adding top-of-file imports.
|
||||
</interfaces>
|
||||
|
||||
Live sync_schedules cron map (queried 2026-07-21) — used to pick a collision-free slot:
|
||||
0 2 * * * -> mimecast-sync (STALE, moving), qbo-sync-2am, veeam-full
|
||||
0 3 * * 0 -> weekly-full
|
||||
0 4 * * * -> contract-services, pax8-daily
|
||||
30 4 * * * -> tickets-reconcile
|
||||
0 5 * * * -> phishing-sweep
|
||||
15 * * * * -> device-link-reconcile (fires :15 EVERY hour — avoid minute 15)
|
||||
slash-30 -> veeam-* (fires :00 and :30 — avoid minutes 0 and 30)
|
||||
New target: 45 4 * * * (4:45 AM — minute 45 is unused anywhere; no collision)
|
||||
</context>
|
||||
|
||||
<tasks>
|
||||
|
||||
<task type="auto">
|
||||
<name>Task 1: Add mimecast-sync and qbo dispatch branches to executeScheduledSync()</name>
|
||||
<files>lib/services/sync-scheduler.ts</files>
|
||||
<action>
|
||||
Two edits, both minimal and matching the existing if/else-if style. Do NOT refactor
|
||||
the chain into a lookup table and do NOT touch any other branch.
|
||||
|
||||
(a) Extend the sync_type union on line 25 (the ScheduleConfig.sync_type type) by
|
||||
adding 'mimecast-sync'. 'qbo' is already present — leave it. Without this, the string
|
||||
comparison in the new mimecast branch is a TS "no overlap" error.
|
||||
|
||||
(b) Insert two new else-if branches into the chain in executeScheduledSync(). Place
|
||||
them BEFORE the final `else if (config.sync_type === 'incremental')` and the final
|
||||
catch-all `else` block, so the catch-all remains reachable only for 'full'/legacy
|
||||
full-sync types. Do not alter the 'incremental' branch or the final else (generic
|
||||
fullSync) behavior.
|
||||
|
||||
Mimecast branch (mirror the engagement-daily/zoom-daily configured-gate pattern, but
|
||||
use dynamic import to match the recent branch style):
|
||||
- else if (config.sync_type === 'mimecast-sync')
|
||||
- Dynamically import isMimecastConfigured from @/lib/services/mimecast-client and
|
||||
runMimecastIncrementalSync from @/lib/services/mimecast-sync-service.
|
||||
- If not configured, console.log a skip line matching the engagement/zoom wording,
|
||||
e.g. "[SCHEDULER] Skipping mimecast-sync — Mimecast not configured".
|
||||
- Otherwise call runMimecastIncrementalSync(), capture the result, and log a one-line
|
||||
summary using the ACTUAL MimecastSyncResult fields (do not invent fields):
|
||||
messagesUpserted, threatsUpserted, bodiesFetched, purgedMessages,
|
||||
errors.length, durationMs — following the one-line summary format used by the
|
||||
phishing-sweep and device-link-reconcile branches.
|
||||
|
||||
QBO branch (mirror the pax8-daily integration_settings disable-check pattern):
|
||||
- else if (config.sync_type === 'qbo')
|
||||
- Query "SELECT disabled FROM integration_settings WHERE key = 'qbo'" exactly like the
|
||||
pax8 branch, compute isDisabled = rows[0]?.disabled === true.
|
||||
- If disabled, console.log "[SCHEDULER] Skipping qbo sync — QBO disabled via /admin/integrations"
|
||||
and do nothing else.
|
||||
- Otherwise dynamically import getQboSyncService from @/lib/services/qbo-sync-service
|
||||
and call getQboSyncService().incrementalSync('scheduled'). Use incrementalSync, not
|
||||
fullSync — that matches the manual /api/qbo/sync non-full path and is correct for a
|
||||
twice-daily recurring job. QBO has no env-var configured check like pax8 (it uses
|
||||
stored OAuth tokens), so only the disabled check is needed.
|
||||
|
||||
Because both branches replace a fall-through to the SyncService singleton, this also
|
||||
removes mimecast-sync and qbo from contending for that mutex.
|
||||
</action>
|
||||
<verify>
|
||||
<automated>cd /opt/stacks/pulse && npx tsc --noEmit --pretty 2>&1 | tail -5 && grep -c "config.sync_type === 'mimecast-sync'" lib/services/sync-scheduler.ts && grep -c "getQboSyncService" lib/services/sync-scheduler.ts && grep -c "runMimecastIncrementalSync" lib/services/sync-scheduler.ts</automated>
|
||||
</verify>
|
||||
<done>
|
||||
tsc passes clean; both new branches present; catch-all fullSync and incremental
|
||||
branches unchanged; 'mimecast-sync' added to the sync_type union. No test file exists
|
||||
for sync-scheduler (confirmed — only mimecast-client.test.ts), so no test changes are
|
||||
made, per the constraint against adding a new scheduler test file.
|
||||
</done>
|
||||
</task>
|
||||
|
||||
<task type="auto">
|
||||
<name>Task 2: Create migration 101 to reschedule mimecast-sync + apply to live DB</name>
|
||||
<files>migrations/101_reschedule_mimecast_sync.sql</files>
|
||||
<action>
|
||||
Create migrations/101_reschedule_mimecast_sync.sql following the repo's schedule-table
|
||||
migration precedent (migrations/098_phishing_sweep_schedule.sql,
|
||||
migrations/090_ticket_reconcile_schedule.sql) — but an UPDATE, not an INSERT, since the
|
||||
mimecast-sync row already exists.
|
||||
|
||||
The UPDATE must be safe to re-run and must NOT clobber an admin's manual change. Guard
|
||||
it on the stale value so it is a no-op if already moved:
|
||||
UPDATE sync_schedules
|
||||
SET cron_expression = '45 4 * * *', updated_at = NOW()
|
||||
WHERE id = 'mimecast-sync' AND cron_expression = '0 2 * * *';
|
||||
|
||||
Add a leading SQL comment block explaining WHY (2 AM 3-way collision with qbo-sync-2am
|
||||
+ veeam-full, plus SyncService mutex contention) — matching the commenting style of
|
||||
migration 098.
|
||||
|
||||
Chosen new time: 45 4 * * * (4:45 AM). Verified collision-free against the live table:
|
||||
minute 45 is used by nothing; it avoids 0 4 (contract-services, pax8-daily), 30 4
|
||||
(tickets-reconcile), the 15 * * * * hourly device-link job, and the */30 :00/:30 veeam
|
||||
jobs.
|
||||
|
||||
Then apply the same UPDATE directly to the live container so it takes effect without
|
||||
waiting for a fresh-volume Postgres init (per CLAUDE.md, migrations only auto-apply on
|
||||
first volume boot). Run:
|
||||
docker exec pulse-postgres psql -U pulse_user -d pulse_autotask -c "UPDATE sync_schedules SET cron_expression = '45 4 * * *', updated_at = NOW() WHERE id = 'mimecast-sync' AND cron_expression = '0 2 * * *';"
|
||||
|
||||
Note in the SUMMARY: the running in-memory cron task keeps the old time until the app
|
||||
container restarts (the scheduler re-loads schedules from the DB on init) or
|
||||
reloadAllSchedules() is invoked. Deploying the Task 1 code change restarts the
|
||||
container, which reloads the new cron from the DB — so no separate restart step is
|
||||
needed as long as the code fix is deployed.
|
||||
</action>
|
||||
<verify>
|
||||
<automated>cd /opt/stacks/pulse && test -f migrations/101_reschedule_mimecast_sync.sql && grep -q "UPDATE sync_schedules" migrations/101_reschedule_mimecast_sync.sql && echo MIGRATION_FILE_OK && docker exec pulse-postgres psql -U pulse_user -d pulse_autotask -t -c "SELECT cron_expression FROM sync_schedules WHERE id = 'mimecast-sync';"</automated>
|
||||
</verify>
|
||||
<done>
|
||||
migrations/101_reschedule_mimecast_sync.sql exists with a guarded UPDATE keyed on the
|
||||
stale '0 2 * * *' value; live DB query returns '45 4 * * *' for mimecast-sync; no other
|
||||
schedule row modified.
|
||||
</done>
|
||||
</task>
|
||||
|
||||
</tasks>
|
||||
|
||||
<verification>
|
||||
- npx tsc --noEmit --pretty passes clean.
|
||||
- executeScheduledSync() contains a 'mimecast-sync' branch calling runMimecastIncrementalSync()
|
||||
behind isMimecastConfigured(), and a 'qbo' branch calling getQboSyncService().incrementalSync('scheduled')
|
||||
behind an integration_settings disabled check.
|
||||
- The generic catch-all and the 'incremental' branch behave exactly as before.
|
||||
- No existing branch (phishing-sweep, pax8-daily, veeam-*, etc.) is modified.
|
||||
- Live sync_schedules shows mimecast-sync at '45 4 * * *' with no cron collision.
|
||||
</verification>
|
||||
|
||||
<success_criteria>
|
||||
- Mimecast and QBO scheduled jobs run their real sync logic on schedule.
|
||||
- The 0 2 * * * three-way collision is eliminated (mimecast-sync moved to 4:45 AM).
|
||||
- Migration 101 is safe to re-run and safe against an admin's manual cron change.
|
||||
- Type check passes; change is minimal and matches existing dispatch style.
|
||||
</success_criteria>
|
||||
|
||||
<output>
|
||||
Create `.planning/quick/260721-fy8-fix-mimecast-and-qbo-sync-scheduler-disp/260721-fy8-SUMMARY.md` when done.
|
||||
</output>
|
||||
Loading…
Add table
Add a link
Reference in a new issue