wulf-pulse/tasks/prd-morning-summary-teams.md
lorentz c518eefdb2 feat: Morning NOC Summary adaptive card for Teams
- Add MorningSummaryService with Zabbix aggregation and adaptive card builder
- Add webhook delivery system with Teams incoming webhooks
- Add admin UI at /admin/morning-summary for webhook/config management
- Add API routes: /send, /test, /webhooks, /webhooks/[id], /config, /history
- Register morning-summary cron job in SyncScheduler (Mon-Fri 6:30 AM)
- Add outages_only filter (Unavailable triggers only)
- Fix host resolution: use getTriggerEnabledHosts to exclude disabled hosts
- Fix resolved events: event.get value:1 scoped to window with r_eventid filter
- Remove emojis from fact rows and section headers in card
- Remove Open Zabbix button (duplicate of View Problems)
- Add migrations: morning_summary_config + morning_summaries tables
- Add outages_only column to morning_summary_config
2026-03-11 09:34:51 -04:00

18 KiB
Raw Permalink Blame History

PRD: Morning NOC Summary — Teams Adaptive Card

Version: 1.1 Last updated: 2026-03-10


1. Introduction / Overview

Wulf Consulting management needs a daily morning briefing on overnight infrastructure activity across all ~46 client sites monitored by Zabbix. This feature automates that into a rich Teams direct message delivered before the workday starts, with configurable recipients and optional channel escalation actions directly from the card.

Goal: Post a Zabbix-sourced morning summary as a Teams Adaptive Card to one or more configured Teams channel webhooks. The primary targets are the On-Call and/or Outages-Issues channels in the Technical team. The card runs at 6:30 AM ET on weekdays (with a Monday extended window option) and can be triggered on-demand from the Pulse admin UI. No bot registration required — delivery is via incoming webhook URLs stored in Pulse config.


2. Goals

  1. Post a formatted Teams Adaptive Card to configured channel webhook(s) at 6:30 AM MonFri.
  2. Surface currently-open Zabbix problems (client name, trigger name, duration, severity).
  3. Surface problems that resolved overnight (during the report window).
  4. Show headline stats: open count, resolved count, avg MTTR, clients affected.
  5. Highlight open problems active > 4 hours without acknowledgement ("Needs Attention" flag).
  6. Support posting to multiple webhooks (e.g. both On-Call and Outages-Issues simultaneously).
  7. Provide a manual trigger in the Pulse admin UI with per-webhook test send capability.
  8. Support configurable weekend suppression; when Monday is the run day, extend the report window to cover the full weekend (Friday 6 PM → Monday 6:30 AM).
  9. Persist each generated summary to the Pulse database and show it as a dashboard widget.

3. User Stories

  • As Tom Carlin / Lorentz Hinrichsen, I want to see the morning NOC summary posted to the Teams channels I already monitor so I get the briefing without checking a separate tool.
  • As a Pulse admin, I want to configure which channel webhooks receive the summary without touching code or env files.
  • As a Pulse admin, I want to send a test post to a specific webhook to verify formatting before the scheduled run.
  • As a Pulse admin, I want to toggle weekend suppression and Monday extended window.
  • As any Pulse user, I want to see the most recent summary on the Pulse dashboard.

4. Functional Requirements

4.1 Data Collection (Zabbix only — v1)

  1. The system must query problem.get (with recent: true, selectHosts: extend, selectAcknowledges: extend, severities: [2,3,4,5]) to retrieve all currently open problems at run time.
  2. The system must query event.get with value: 0 (recovery events) between time_from = window start and time_to = run time to retrieve resolved problems.
  3. The system must query host.get with selectHostGroups: ['name'] once per run to build a hostid → { clientName, hostName } map using groups prefixed Clients/.
  4. The report window must be determined as follows:
    • TuesdayFriday runs: window start = previous calendar day at 18:00 ET.
    • Monday run (weekend mode on): window start = the preceding Friday at 18:00 ET, covering the full Sat + Sun + Mon pre-6:30 AM period.
    • Monday run (weekend mode off): same as TueFri (Sunday 18:00 ET → Monday 6:30 AM).
  5. The system must calculate:
    • Open count — number of active problems meeting severity filter.
    • Resolved count — recovery events in the window.
    • Avg MTTR (minutes) — mean of (r_clock - clock) for resolved events; null if none.
    • Clients affected — deduplicated client names from open problems.
  6. Any open problem where clock is > 4 hours before run time and acknowledged array is empty must be flagged needsAttention: true.

4.2 Webhook Configuration — Pulse UI

  1. The admin UI must provide a Webhooks section on /admin/morning-summary to manage the list of Teams incoming webhook URLs that receive the summary.
  2. The admin must be able to add a webhook by providing:
    • A display label (e.g. Technical / On-Call)
    • The incoming webhook URL
    • An enabled toggle
  3. The admin must be able to remove or disable any webhook.
  4. Webhooks must be stored in Postgres (new table morning_summary_webhooks) with:
    • id (serial PK)
    • label (text) — human-friendly name
    • webhook_url (text)
    • enabled (boolean, default true)
    • last_delivered_at (timestamptz, nullable)
    • last_status (text, nullable) — success or failed
    • created_at (timestamptz)
  5. The admin UI must provide a "Test" button next to each webhook that posts the current summary card to that URL only, independent of the scheduled run.
  6. The test endpoint must be POST /api/notifications/morning-summary/test with body { "webhookId": <id> }.
  7. The UI must display per-webhook last delivery status inline.

4.3 Teams Adaptive Card — Structure

  1. The card must include a header TextBlock: ☀️ Morning NOC Summary — {date}.
    • On Monday with weekend mode: append (Weekend Coverage) to the header.
  2. The card must include a stat ColumnSet: Open (red), Resolved (green), Avg MTTR.
  3. If there are open problems, the card must include a red-styled Container labelled 🔴 OPEN ISSUES with a FactSet:
    • Title: {ClientName} / {HostName}
    • Value: {TriggerName} — {duration} (+ ⚠️ Needs Attention if flagged)
  4. If there are no open problems, the card must show a green " All Clear" container.
  5. If there are resolved overnight events, the card must include a green-styled Container labelled 🟢 RESOLVED OVERNIGHT. If more than 5, show 5 and append +{N} more — all resolved.
  6. The card must include three action buttons:
    • Open Zabbixhttps://zabbix.wulfconsulting.cloud
    • Open Pulsehttps://pulse.wulfconsulting.cloud
    • View Problemshttps://zabbix.wulfconsulting.cloud/zabbix.php?action=problem.view
  7. The card must use Adaptive Card schema version 1.4.
  8. The card payload must be wrapped in the Teams incoming webhook envelope: { "type": "message", "attachments": [{ "contentType": "application/vnd.microsoft.card.adaptive", "content": <card> }] }

4.4 Delivery via Incoming Webhooks

  1. The system must POST the Adaptive Card envelope to each enabled webhook URL.
  2. Delivery must be attempted for all enabled webhooks regardless of individual failures.
  3. Each delivery result (HTTP status, error message) must be recorded in morning_summaries.delivery_status keyed by webhook ID.
  4. The send endpoint POST /api/notifications/morning-summary/send must accept an optional { "webhookIds": [1, 2] } body to target specific webhooks; if omitted, all enabled webhooks are used.
  5. Incoming webhook URLs are obtained from Teams: channel → connectors → "Incoming Webhook". The admin pastes the URL into the Pulse webhook config UI.

4.5 MS Graph Usage (read-only)

  1. MS Graph is not used for delivery in v1 (webhooks handle that).
  2. MS Graph getUsers() may still be used in the admin UI to resolve display names when configuring webhook labels, but is not required for the core send flow.
  3. No ChatMessage.Send or ChannelMessage.Send Graph permissions are needed for v1.

4.6 Scheduling & Weekend Mode

  1. The job must be registered in SyncScheduler with cron 30 6 * * 1-5.
  2. A morning_summary_config table (single-row settings) must store:
    • weekend_suppression (boolean) — if true, skip Sat/Sun runs.
    • monday_extended_window (boolean) — if true and today is Monday, set window start to preceding Friday 18:00 ET.
    • severity_filter (int, default 2) — minimum Zabbix severity to include.
    • updated_at (timestamptz)
  3. The admin UI must expose toggles for weekend_suppression and monday_extended_window with a clear label: "On Mondays, extend window to cover the full weekend (Fri 6 PM → Mon 6:30 AM)".
  4. The admin UI must display the next scheduled run time calculated from the cron expression and current settings.

4.7 Manual Trigger — Pulse UI

  1. The admin UI at /admin/morning-summary must provide:
    • Schedule status: next run time, last run time, last run result.
    • "Send Now" button — fires POST /api/notifications/morning-summary/send to all enabled webhooks immediately.
    • Per-webhook "Test" buttons (see §4.2 req 11).
    • Weekend mode toggles (see §4.6 req 35).
    • Webhooks section — add/remove/enable-disable webhook entries (see §4.2).
    • Last card preview — most recent card_payload as formatted JSON (collapsed by default).
  2. The UI must show a toast on send and display per-webhook delivery results inline.

4.8 Dashboard Widget & Persistence

  1. Each run (scheduled or manual/test) must persist to morning_summaries:
    • id (serial PK)
    • generated_at (timestamptz)
    • window_from (timestamptz)
    • window_to (timestamptz)
    • open_count (int)
    • resolved_count (int)
    • mttr_minutes (int, nullable)
    • clients_affected (text[])
    • is_weekend_window (boolean)
    • card_payload (jsonb)
    • delivery_status (jsonb) — { [webhookId]: { success: bool, httpStatus?: number, error?: string } }
  2. The Pulse home/dashboard must display a "Morning Summary" widget with: open count, resolved count, MTTR, window label, last generated timestamp, and a link to the admin page.

5. Non-Goals (Out of Scope for v1)

  • Veeam backup failure data (deferred to v2).
  • PSA/Autotask ticket correlation and "unmatched alerts" (deferred to v2).
  • Direct Teams DMs to individual users (requires bot registration — out of scope).
  • Email or ntfy delivery channels (webhooks only for v1).
  • ISP outage detection / grouping by ISP.
  • Per-channel content customization (all webhooks receive the same card).

6. Design Considerations

Adaptive Card Layout

┌──────────────────────────────────────────────────┐
│ ☀️ Morning NOC Summary — Mon Mar 16 (Weekend)     │
├──────────┬────────────┬────────────────────────── │
│  3 Open  │ 12 Resolved│  22m Avg MTTR             │
├──────────────────────────────────────────────────┤
│ 🔴 OPEN ISSUES                                   │
│  Kuhn's / FW-01    Host Unreachable — 6h 12m ⚠️  │
│  ADM / SW-Core     High Packet Loss — 2h 5m      │
│  Seubert / DC-01   Host Unreachable — 1h 20m     │
├──────────────────────────────────────────────────┤
│ 🟢 RESOLVED OVERNIGHT                            │
│  Brodaks / RTR-01  Slow Response — 22m           │
│  +11 more — all resolved                         │
├──────────────────────────────────────────────────┤
│[Open Zabbix][Open Pulse][View Problems]          │
│[📣 Notify On Call]  [🚨 Post to Outages]         │
└──────────────────────────────────────────────────┘
  • "style": "attention" container → red in Teams (open issues).
  • "style": "good" container → green in Teams (resolved).
  • Duration: Xh Ym for ≥ 1 hour, Xm for < 1 hour.
  • Escalation buttons open a Pulse URL in browser; Pulse posts to channel server-side and returns a simple confirmation page.

Recipient Management UI (on /admin/morning-summary)

Recipients
┌────────────────────────────┬──────────┬────────────────────┐
│ Name / Email               │ Status   │ Actions            │
├────────────────────────────┼──────────┼────────────────────┤
│ Tom Carlin                 │ ✅ Last: │ [Test] [Remove]    │
│ tom@wulfconsulting.com     │ today    │                    │
├────────────────────────────┼──────────┼────────────────────┤
│ Lorentz Hinrichsen         │ ✅ Last: │ [Test] [Remove]    │
│ lorentz@wulfconsulting.com │ today    │                    │
└────────────────────────────┴──────────┴────────────────────┘
[+ Add Recipient ▾]  ← searchable dropdown of tenant users

Webhook Delivery Flow

// For each enabled webhook in morning_summary_webhooks:
POST {webhook_url}
Content-Type: application/json

{
  "type": "message",
  "attachments": [{
    "contentType": "application/vnd.microsoft.card.adaptive",
    "contentUrl": null,
    "content": { ...adaptive card JSON... }
  }]
}

To get a webhook URL in Teams: go to the target channel → ... → Connectors → Incoming Webhook → Configure → copy URL → paste into Pulse admin.


7. Technical Considerations

Existing Infrastructure to Reuse

  • ZabbixClient (lib/services/zabbix-client.ts) — add getOpenProblems() and getResolvedEvents(from: Date, to: Date) methods.
  • MsGraphClient — no changes needed for v1.
  • SyncScheduler — add 'morning-summary' sync type; register in initialize().
  • Postgres — three new tables: morning_summaries, morning_summary_webhooks, morning_summary_config.

New Files to Create

File Purpose
lib/services/morning-summary-service.ts Aggregation, card building, webhook delivery
app/api/notifications/morning-summary/send/route.ts POST — send to all/selected webhooks
app/api/notifications/morning-summary/test/route.ts POST — test send to single webhook
app/api/notifications/morning-summary/webhooks/route.ts GET/POST — manage webhook list
app/api/notifications/morning-summary/webhooks/[id]/route.ts PUT/DELETE — update/remove webhook
app/api/notifications/morning-summary/config/route.ts GET/PUT — read/update settings
app/admin/morning-summary/page.tsx Admin UI: webhooks, schedule, config, preview
db/migrations/XXXX_morning_summary.sql All three new tables

Zabbix API Notes

  • problem.get returns acknowledged as "0"/"1" strings and clock as Unix timestamp string.
  • Use selectAcknowledges: 'extend' — problem is unacknowledged if the array is empty.
  • event.get with value: 0 returns recoveries; duration = r_clock - clock (both seconds).
  • Enrich problems with host/client by cross-referencing objectid (triggerid) against a trigger→host map built from trigger.get with selectHosts: ['hostid', 'name'], or directly via problem.get with selectSuppressionData + a pre-built hostid→client map.

MS Graph Permissions

No Graph permissions are required for v1 delivery. Delivery uses Teams incoming webhook URLs (plain HTTPS POST — no auth needed beyond the secret embedded in the URL).

Existing granted permissions (Chat.Create, Team.ReadBasic.All, Channel.ReadBasic.All, ChatMessage.Send, ChannelMessage.Send) are unused by this feature but can remain for future use.

Timezone

  • Cron 30 6 * * 1-5 fires at 6:30 AM server time — ensure server is set to ET or adjust cron.
  • Report window computed in ET using Intl.DateTimeFormat or date-fns-tz.
  • Monday extended window: startOfDay(subDays(monday, 3)) + 18h = Friday 18:00 ET.

8. Success Metrics

  • All configured recipients receive the DM at 6:30 AM on weekdays without manual action.
  • Test send from the UI delivers to the selected individual within 30 seconds.
  • Escalation button (On Call or Outages) posts to the correct Teams channel within 10 seconds and the user sees a confirmation page.
  • Monday card correctly shows the extended weekend window label and data.
  • Dashboard widget reflects the most recent summary within 1 minute of generation.
  • Zero unhandled errors surfaced to end users; all failures logged and shown in the admin UI.

9. Open Questions

  1. MS Graph permissions — Not needed for v1 webhook delivery.
  2. Team/channel names — Confirmed: team = Technical, channels = On-Call and Outages-Issues.
  3. App membership in Technical team — Not needed; webhook URLs bypass Graph entirely.
  4. Error alerting — If the 6:30 AM scheduled job fails entirely, how should that be surfaced? (ntfy push, Pulse admin badge, or both?)
  5. Severity filter default — Spec defaults to Warning (2) and above. Should Information (1) problems be included? Confirm with Tom/Lorentz.
  6. Webhook URLs — Lorentz needs to create incoming webhooks in the On-Call and Outages-Issues channels (Teams channel → ... → Connectors → Incoming Webhook → Configure) and paste the URLs into the Pulse admin UI once the feature is deployed.