- Add MorningSummaryService with Zabbix aggregation and adaptive card builder - Add webhook delivery system with Teams incoming webhooks - Add admin UI at /admin/morning-summary for webhook/config management - Add API routes: /send, /test, /webhooks, /webhooks/[id], /config, /history - Register morning-summary cron job in SyncScheduler (Mon-Fri 6:30 AM) - Add outages_only filter (Unavailable triggers only) - Fix host resolution: use getTriggerEnabledHosts to exclude disabled hosts - Fix resolved events: event.get value:1 scoped to window with r_eventid filter - Remove emojis from fact rows and section headers in card - Remove Open Zabbix button (duplicate of View Problems) - Add migrations: morning_summary_config + morning_summaries tables - Add outages_only column to morning_summary_config
18 KiB
PRD: Morning NOC Summary — Teams Adaptive Card
Version: 1.1 Last updated: 2026-03-10
1. Introduction / Overview
Wulf Consulting management needs a daily morning briefing on overnight infrastructure activity across all ~46 client sites monitored by Zabbix. This feature automates that into a rich Teams direct message delivered before the workday starts, with configurable recipients and optional channel escalation actions directly from the card.
Goal: Post a Zabbix-sourced morning summary as a Teams Adaptive Card to one or more configured Teams channel webhooks. The primary targets are the On-Call and/or Outages-Issues channels in the Technical team. The card runs at 6:30 AM ET on weekdays (with a Monday extended window option) and can be triggered on-demand from the Pulse admin UI. No bot registration required — delivery is via incoming webhook URLs stored in Pulse config.
2. Goals
- Post a formatted Teams Adaptive Card to configured channel webhook(s) at 6:30 AM Mon–Fri.
- Surface currently-open Zabbix problems (client name, trigger name, duration, severity).
- Surface problems that resolved overnight (during the report window).
- Show headline stats: open count, resolved count, avg MTTR, clients affected.
- Highlight open problems active > 4 hours without acknowledgement ("Needs Attention" flag).
- Support posting to multiple webhooks (e.g. both On-Call and Outages-Issues simultaneously).
- Provide a manual trigger in the Pulse admin UI with per-webhook test send capability.
- Support configurable weekend suppression; when Monday is the run day, extend the report window to cover the full weekend (Friday 6 PM → Monday 6:30 AM).
- Persist each generated summary to the Pulse database and show it as a dashboard widget.
3. User Stories
- As Tom Carlin / Lorentz Hinrichsen, I want to see the morning NOC summary posted to the Teams channels I already monitor so I get the briefing without checking a separate tool.
- As a Pulse admin, I want to configure which channel webhooks receive the summary without touching code or env files.
- As a Pulse admin, I want to send a test post to a specific webhook to verify formatting before the scheduled run.
- As a Pulse admin, I want to toggle weekend suppression and Monday extended window.
- As any Pulse user, I want to see the most recent summary on the Pulse dashboard.
4. Functional Requirements
4.1 Data Collection (Zabbix only — v1)
- The system must query
problem.get(withrecent: true,selectHosts: extend,selectAcknowledges: extend,severities: [2,3,4,5]) to retrieve all currently open problems at run time. - The system must query
event.getwithvalue: 0(recovery events) betweentime_from= window start andtime_to= run time to retrieve resolved problems. - The system must query
host.getwithselectHostGroups: ['name']once per run to build ahostid → { clientName, hostName }map using groups prefixedClients/. - The report window must be determined as follows:
- Tuesday–Friday runs: window start = previous calendar day at 18:00 ET.
- Monday run (weekend mode on): window start = the preceding Friday at 18:00 ET, covering the full Sat + Sun + Mon pre-6:30 AM period.
- Monday run (weekend mode off): same as Tue–Fri (Sunday 18:00 ET → Monday 6:30 AM).
- The system must calculate:
- Open count — number of active problems meeting severity filter.
- Resolved count — recovery events in the window.
- Avg MTTR (minutes) — mean of
(r_clock - clock)for resolved events;nullif none. - Clients affected — deduplicated client names from open problems.
- Any open problem where
clockis > 4 hours before run time andacknowledgedarray is empty must be flaggedneedsAttention: true.
4.2 Webhook Configuration — Pulse UI
- The admin UI must provide a Webhooks section on
/admin/morning-summaryto manage the list of Teams incoming webhook URLs that receive the summary. - The admin must be able to add a webhook by providing:
- A display label (e.g.
Technical / On-Call) - The incoming webhook URL
- An enabled toggle
- A display label (e.g.
- The admin must be able to remove or disable any webhook.
- Webhooks must be stored in Postgres (new table
morning_summary_webhooks) with:id(serial PK)label(text) — human-friendly namewebhook_url(text)enabled(boolean, default true)last_delivered_at(timestamptz, nullable)last_status(text, nullable) —successorfailedcreated_at(timestamptz)
- The admin UI must provide a "Test" button next to each webhook that posts the current summary card to that URL only, independent of the scheduled run.
- The test endpoint must be
POST /api/notifications/morning-summary/testwith body{ "webhookId": <id> }. - The UI must display per-webhook last delivery status inline.
4.3 Teams Adaptive Card — Structure
- The card must include a header
TextBlock:☀️ Morning NOC Summary — {date}.- On Monday with weekend mode: append
(Weekend Coverage)to the header.
- On Monday with weekend mode: append
- The card must include a stat
ColumnSet: Open (red), Resolved (green), Avg MTTR. - If there are open problems, the card must include a red-styled
Containerlabelled🔴 OPEN ISSUESwith aFactSet:- Title:
{ClientName} / {HostName} - Value:
{TriggerName} — {duration}(+⚠️ Needs Attentionif flagged)
- Title:
- If there are no open problems, the card must show a green "✅ All Clear" container.
- If there are resolved overnight events, the card must include a green-styled
Containerlabelled🟢 RESOLVED OVERNIGHT. If more than 5, show 5 and append+{N} more — all resolved. - The card must include three action buttons:
Open Zabbix→https://zabbix.wulfconsulting.cloudOpen Pulse→https://pulse.wulfconsulting.cloudView Problems→https://zabbix.wulfconsulting.cloud/zabbix.php?action=problem.view
- The card must use Adaptive Card schema version
1.4. - The card payload must be wrapped in the Teams incoming webhook envelope:
{ "type": "message", "attachments": [{ "contentType": "application/vnd.microsoft.card.adaptive", "content": <card> }] }
4.4 Delivery via Incoming Webhooks
- The system must POST the Adaptive Card envelope to each enabled webhook URL.
- Delivery must be attempted for all enabled webhooks regardless of individual failures.
- Each delivery result (HTTP status, error message) must be recorded in
morning_summaries.delivery_statuskeyed by webhook ID. - The send endpoint
POST /api/notifications/morning-summary/sendmust accept an optional{ "webhookIds": [1, 2] }body to target specific webhooks; if omitted, all enabled webhooks are used. - Incoming webhook URLs are obtained from Teams: channel → connectors → "Incoming Webhook". The admin pastes the URL into the Pulse webhook config UI.
4.5 MS Graph Usage (read-only)
- MS Graph is not used for delivery in v1 (webhooks handle that).
- MS Graph
getUsers()may still be used in the admin UI to resolve display names when configuring webhook labels, but is not required for the core send flow. - No
ChatMessage.SendorChannelMessage.SendGraph permissions are needed for v1.
4.6 Scheduling & Weekend Mode
- The job must be registered in
SyncSchedulerwith cron30 6 * * 1-5. - A
morning_summary_configtable (single-row settings) must store:weekend_suppression(boolean) — if true, skip Sat/Sun runs.monday_extended_window(boolean) — if true and today is Monday, set window start to preceding Friday 18:00 ET.severity_filter(int, default 2) — minimum Zabbix severity to include.updated_at(timestamptz)
- The admin UI must expose toggles for
weekend_suppressionandmonday_extended_windowwith a clear label: "On Mondays, extend window to cover the full weekend (Fri 6 PM → Mon 6:30 AM)". - The admin UI must display the next scheduled run time calculated from the cron expression and current settings.
4.7 Manual Trigger — Pulse UI
- The admin UI at
/admin/morning-summarymust provide:- Schedule status: next run time, last run time, last run result.
- "Send Now" button — fires
POST /api/notifications/morning-summary/sendto all enabled webhooks immediately. - Per-webhook "Test" buttons (see §4.2 req 11).
- Weekend mode toggles (see §4.6 req 35).
- Webhooks section — add/remove/enable-disable webhook entries (see §4.2).
- Last card preview — most recent
card_payloadas formatted JSON (collapsed by default).
- The UI must show a toast on send and display per-webhook delivery results inline.
4.8 Dashboard Widget & Persistence
- Each run (scheduled or manual/test) must persist to
morning_summaries:id(serial PK)generated_at(timestamptz)window_from(timestamptz)window_to(timestamptz)open_count(int)resolved_count(int)mttr_minutes(int, nullable)clients_affected(text[])is_weekend_window(boolean)card_payload(jsonb)delivery_status(jsonb) —{ [webhookId]: { success: bool, httpStatus?: number, error?: string } }
- The Pulse home/dashboard must display a "Morning Summary" widget with: open count, resolved count, MTTR, window label, last generated timestamp, and a link to the admin page.
5. Non-Goals (Out of Scope for v1)
- Veeam backup failure data (deferred to v2).
- PSA/Autotask ticket correlation and "unmatched alerts" (deferred to v2).
- Direct Teams DMs to individual users (requires bot registration — out of scope).
- Email or ntfy delivery channels (webhooks only for v1).
- ISP outage detection / grouping by ISP.
- Per-channel content customization (all webhooks receive the same card).
6. Design Considerations
Adaptive Card Layout
┌──────────────────────────────────────────────────┐
│ ☀️ Morning NOC Summary — Mon Mar 16 (Weekend) │
├──────────┬────────────┬────────────────────────── │
│ 3 Open │ 12 Resolved│ 22m Avg MTTR │
├──────────────────────────────────────────────────┤
│ 🔴 OPEN ISSUES │
│ Kuhn's / FW-01 Host Unreachable — 6h 12m ⚠️ │
│ ADM / SW-Core High Packet Loss — 2h 5m │
│ Seubert / DC-01 Host Unreachable — 1h 20m │
├──────────────────────────────────────────────────┤
│ 🟢 RESOLVED OVERNIGHT │
│ Brodaks / RTR-01 Slow Response — 22m │
│ +11 more — all resolved │
├──────────────────────────────────────────────────┤
│[Open Zabbix][Open Pulse][View Problems] │
│[📣 Notify On Call] [🚨 Post to Outages] │
└──────────────────────────────────────────────────┘
"style": "attention"container → red in Teams (open issues)."style": "good"container → green in Teams (resolved).- Duration:
Xh Ymfor ≥ 1 hour,Xmfor < 1 hour. - Escalation buttons open a Pulse URL in browser; Pulse posts to channel server-side and returns a simple confirmation page.
Recipient Management UI (on /admin/morning-summary)
Recipients
┌────────────────────────────┬──────────┬────────────────────┐
│ Name / Email │ Status │ Actions │
├────────────────────────────┼──────────┼────────────────────┤
│ Tom Carlin │ ✅ Last: │ [Test] [Remove] │
│ tom@wulfconsulting.com │ today │ │
├────────────────────────────┼──────────┼────────────────────┤
│ Lorentz Hinrichsen │ ✅ Last: │ [Test] [Remove] │
│ lorentz@wulfconsulting.com │ today │ │
└────────────────────────────┴──────────┴────────────────────┘
[+ Add Recipient ▾] ← searchable dropdown of tenant users
Webhook Delivery Flow
// For each enabled webhook in morning_summary_webhooks:
POST {webhook_url}
Content-Type: application/json
{
"type": "message",
"attachments": [{
"contentType": "application/vnd.microsoft.card.adaptive",
"contentUrl": null,
"content": { ...adaptive card JSON... }
}]
}
To get a webhook URL in Teams: go to the target channel → ... → Connectors → Incoming Webhook → Configure → copy URL → paste into Pulse admin.
7. Technical Considerations
Existing Infrastructure to Reuse
ZabbixClient(lib/services/zabbix-client.ts) — addgetOpenProblems()andgetResolvedEvents(from: Date, to: Date)methods.MsGraphClient— no changes needed for v1.SyncScheduler— add'morning-summary'sync type; register ininitialize().- Postgres — three new tables:
morning_summaries,morning_summary_webhooks,morning_summary_config.
New Files to Create
| File | Purpose |
|---|---|
lib/services/morning-summary-service.ts |
Aggregation, card building, webhook delivery |
app/api/notifications/morning-summary/send/route.ts |
POST — send to all/selected webhooks |
app/api/notifications/morning-summary/test/route.ts |
POST — test send to single webhook |
app/api/notifications/morning-summary/webhooks/route.ts |
GET/POST — manage webhook list |
app/api/notifications/morning-summary/webhooks/[id]/route.ts |
PUT/DELETE — update/remove webhook |
app/api/notifications/morning-summary/config/route.ts |
GET/PUT — read/update settings |
app/admin/morning-summary/page.tsx |
Admin UI: webhooks, schedule, config, preview |
db/migrations/XXXX_morning_summary.sql |
All three new tables |
Zabbix API Notes
problem.getreturnsacknowledgedas"0"/"1"strings andclockas Unix timestamp string.- Use
selectAcknowledges: 'extend'— problem is unacknowledged if the array is empty. event.getwithvalue: 0returns recoveries; duration =r_clock - clock(both seconds).- Enrich problems with host/client by cross-referencing
objectid(triggerid) against a trigger→host map built fromtrigger.getwithselectHosts: ['hostid', 'name'], or directly viaproblem.getwithselectSuppressionData+ a pre-built hostid→client map.
MS Graph Permissions
No Graph permissions are required for v1 delivery. Delivery uses Teams incoming webhook URLs (plain HTTPS POST — no auth needed beyond the secret embedded in the URL).
Existing granted permissions (Chat.Create, Team.ReadBasic.All, Channel.ReadBasic.All,
ChatMessage.Send, ChannelMessage.Send) are unused by this feature but can remain for
future use.
Timezone
- Cron
30 6 * * 1-5fires at 6:30 AM server time — ensure server is set to ET or adjust cron. - Report window computed in ET using
Intl.DateTimeFormatordate-fns-tz. - Monday extended window:
startOfDay(subDays(monday, 3)) + 18h= Friday 18:00 ET.
8. Success Metrics
- All configured recipients receive the DM at 6:30 AM on weekdays without manual action.
- Test send from the UI delivers to the selected individual within 30 seconds.
- Escalation button (On Call or Outages) posts to the correct Teams channel within 10 seconds and the user sees a confirmation page.
- Monday card correctly shows the extended weekend window label and data.
- Dashboard widget reflects the most recent summary within 1 minute of generation.
- Zero unhandled errors surfaced to end users; all failures logged and shown in the admin UI.
9. Open Questions
MS Graph permissions— Not needed for v1 webhook delivery.Team/channel names— Confirmed: team =Technical, channels =On-CallandOutages-Issues.App membership in Technical team— Not needed; webhook URLs bypass Graph entirely.- Error alerting — If the 6:30 AM scheduled job fails entirely, how should that be surfaced? (ntfy push, Pulse admin badge, or both?)
- Severity filter default — Spec defaults to Warning (2) and above. Should Information (1) problems be included? Confirm with Tom/Lorentz.
- Webhook URLs — Lorentz needs to create incoming webhooks in the
On-CallandOutages-Issueschannels (Teams channel → ... → Connectors → Incoming Webhook → Configure) and paste the URLs into the Pulse admin UI once the feature is deployed.