wulf-pulse/tasks/prd-morning-summary-teams.md
lorentz c518eefdb2 feat: Morning NOC Summary adaptive card for Teams
- Add MorningSummaryService with Zabbix aggregation and adaptive card builder
- Add webhook delivery system with Teams incoming webhooks
- Add admin UI at /admin/morning-summary for webhook/config management
- Add API routes: /send, /test, /webhooks, /webhooks/[id], /config, /history
- Register morning-summary cron job in SyncScheduler (Mon-Fri 6:30 AM)
- Add outages_only filter (Unavailable triggers only)
- Fix host resolution: use getTriggerEnabledHosts to exclude disabled hosts
- Fix resolved events: event.get value:1 scoped to window with r_eventid filter
- Remove emojis from fact rows and section headers in card
- Remove Open Zabbix button (duplicate of View Problems)
- Add migrations: morning_summary_config + morning_summaries tables
- Add outages_only column to morning_summary_config
2026-03-11 09:34:51 -04:00

339 lines
18 KiB
Markdown
Raw Permalink Blame History

This file contains ambiguous Unicode characters

This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.

# PRD: Morning NOC Summary — Teams Adaptive Card
**Version:** 1.1
**Last updated:** 2026-03-10
---
## 1. Introduction / Overview
Wulf Consulting management needs a daily morning briefing on overnight infrastructure activity
across all ~46 client sites monitored by Zabbix. This feature automates that into a rich Teams
direct message delivered before the workday starts, with configurable recipients and optional
channel escalation actions directly from the card.
**Goal:** Post a Zabbix-sourced morning summary as a Teams Adaptive Card to one or more
configured Teams channel webhooks. The primary targets are the **On-Call** and/or
**Outages-Issues** channels in the **Technical** team. The card runs at 6:30 AM ET on
weekdays (with a Monday extended window option) and can be triggered on-demand from the
Pulse admin UI. No bot registration required — delivery is via incoming webhook URLs stored
in Pulse config.
---
## 2. Goals
1. Post a formatted Teams Adaptive Card to configured channel webhook(s) at 6:30 AM MonFri.
2. Surface currently-open Zabbix problems (client name, trigger name, duration, severity).
3. Surface problems that resolved overnight (during the report window).
4. Show headline stats: open count, resolved count, avg MTTR, clients affected.
5. Highlight open problems active > 4 hours without acknowledgement ("Needs Attention" flag).
6. Support posting to multiple webhooks (e.g. both On-Call and Outages-Issues simultaneously).
7. Provide a **manual trigger** in the Pulse admin UI with per-webhook test send capability.
8. Support configurable weekend suppression; when Monday is the run day, extend the report
window to cover the full weekend (Friday 6 PM → Monday 6:30 AM).
9. Persist each generated summary to the Pulse database and show it as a dashboard widget.
---
## 3. User Stories
- **As Tom Carlin / Lorentz Hinrichsen**, I want to see the morning NOC summary posted to
the Teams channels I already monitor so I get the briefing without checking a separate tool.
- **As a Pulse admin**, I want to configure which channel webhooks receive the summary without
touching code or env files.
- **As a Pulse admin**, I want to send a test post to a specific webhook to verify formatting
before the scheduled run.
- **As a Pulse admin**, I want to toggle weekend suppression and Monday extended window.
- **As any Pulse user**, I want to see the most recent summary on the Pulse dashboard.
---
## 4. Functional Requirements
### 4.1 Data Collection (Zabbix only — v1)
1. The system **must** query `problem.get` (with `recent: true`, `selectHosts: extend`,
`selectAcknowledges: extend`, `severities: [2,3,4,5]`) to retrieve all currently open
problems at run time.
2. The system **must** query `event.get` with `value: 0` (recovery events) between
`time_from` = window start and `time_to` = run time to retrieve resolved problems.
3. The system **must** query `host.get` with `selectHostGroups: ['name']` once per run to
build a `hostid → { clientName, hostName }` map using groups prefixed `Clients/`.
4. The report window **must** be determined as follows:
- **TuesdayFriday runs:** window start = previous calendar day at 18:00 ET.
- **Monday run (weekend mode on):** window start = the preceding Friday at 18:00 ET,
covering the full Sat + Sun + Mon pre-6:30 AM period.
- **Monday run (weekend mode off):** same as TueFri (Sunday 18:00 ET → Monday 6:30 AM).
5. The system **must** calculate:
- **Open count** — number of active problems meeting severity filter.
- **Resolved count** — recovery events in the window.
- **Avg MTTR (minutes)** — mean of `(r_clock - clock)` for resolved events; `null` if none.
- **Clients affected** — deduplicated client names from open problems.
6. Any open problem where `clock` is > 4 hours before run time **and** `acknowledged` array
is empty **must** be flagged `needsAttention: true`.
### 4.2 Webhook Configuration — Pulse UI
7. The admin UI **must** provide a **Webhooks** section on `/admin/morning-summary` to manage
the list of Teams incoming webhook URLs that receive the summary.
8. The admin **must** be able to **add** a webhook by providing:
- A display label (e.g. `Technical / On-Call`)
- The incoming webhook URL
- An enabled toggle
9. The admin **must** be able to **remove** or **disable** any webhook.
10. Webhooks **must** be stored in Postgres (new table `morning_summary_webhooks`) with:
- `id` (serial PK)
- `label` (text) — human-friendly name
- `webhook_url` (text)
- `enabled` (boolean, default true)
- `last_delivered_at` (timestamptz, nullable)
- `last_status` (text, nullable) — `success` or `failed`
- `created_at` (timestamptz)
11. The admin UI **must** provide a **"Test"** button next to each webhook that posts the
current summary card to that URL only, independent of the scheduled run.
12. The test endpoint **must** be `POST /api/notifications/morning-summary/test` with
body `{ "webhookId": <id> }`.
13. The UI **must** display per-webhook last delivery status inline.
### 4.3 Teams Adaptive Card — Structure
14. The card **must** include a header `TextBlock`: `☀️ Morning NOC Summary — {date}`.
- On Monday with weekend mode: append ` (Weekend Coverage)` to the header.
15. The card **must** include a stat `ColumnSet`: Open (red), Resolved (green), Avg MTTR.
16. If there are **open problems**, the card **must** include a red-styled `Container`
labelled `🔴 OPEN ISSUES` with a `FactSet`:
- **Title:** `{ClientName} / {HostName}`
- **Value:** `{TriggerName} — {duration}` (+ ` ⚠️ Needs Attention` if flagged)
17. If there are **no open problems**, the card **must** show a green "✅ All Clear" container.
18. If there are **resolved overnight** events, the card **must** include a green-styled
`Container` labelled `🟢 RESOLVED OVERNIGHT`. If more than 5, show 5 and append
`+{N} more — all resolved`.
19. The card **must** include three action buttons:
- `Open Zabbix``https://zabbix.wulfconsulting.cloud`
- `Open Pulse``https://pulse.wulfconsulting.cloud`
- `View Problems``https://zabbix.wulfconsulting.cloud/zabbix.php?action=problem.view`
20. The card **must** use Adaptive Card schema version `1.4`.
21. The card payload **must** be wrapped in the Teams incoming webhook envelope:
`{ "type": "message", "attachments": [{ "contentType": "application/vnd.microsoft.card.adaptive", "content": <card> }] }`
### 4.4 Delivery via Incoming Webhooks
22. The system **must** POST the Adaptive Card envelope to each enabled webhook URL.
23. Delivery **must** be attempted for all enabled webhooks regardless of individual failures.
24. Each delivery result (HTTP status, error message) **must** be recorded in
`morning_summaries.delivery_status` keyed by webhook ID.
25. The send endpoint `POST /api/notifications/morning-summary/send` **must** accept an
optional `{ "webhookIds": [1, 2] }` body to target specific webhooks; if omitted, all
enabled webhooks are used.
26. Incoming webhook URLs are obtained from Teams: channel → connectors → "Incoming Webhook".
The admin pastes the URL into the Pulse webhook config UI.
### 4.5 MS Graph Usage (read-only)
27. MS Graph is **not** used for delivery in v1 (webhooks handle that).
28. MS Graph `getUsers()` **may** still be used in the admin UI to resolve display names when
configuring webhook labels, but is not required for the core send flow.
29. No `ChatMessage.Send` or `ChannelMessage.Send` Graph permissions are needed for v1.
### 4.6 Scheduling & Weekend Mode
33. The job **must** be registered in `SyncScheduler` with cron `30 6 * * 1-5`.
34. A `morning_summary_config` table (single-row settings) **must** store:
- `weekend_suppression` (boolean) — if true, skip Sat/Sun runs.
- `monday_extended_window` (boolean) — if true and today is Monday, set window start
to preceding Friday 18:00 ET.
- `severity_filter` (int, default 2) — minimum Zabbix severity to include.
- `updated_at` (timestamptz)
35. The admin UI **must** expose toggles for `weekend_suppression` and `monday_extended_window`
with a clear label: _"On Mondays, extend window to cover the full weekend (Fri 6 PM → Mon
6:30 AM)"_.
36. The admin UI **must** display the next scheduled run time calculated from the cron
expression and current settings.
### 4.7 Manual Trigger — Pulse UI
37. The admin UI at `/admin/morning-summary` **must** provide:
- **Schedule status:** next run time, last run time, last run result.
- **"Send Now"** button — fires `POST /api/notifications/morning-summary/send` to all
enabled webhooks immediately.
- **Per-webhook "Test"** buttons (see §4.2 req 11).
- **Weekend mode toggles** (see §4.6 req 35).
- **Webhooks section** — add/remove/enable-disable webhook entries (see §4.2).
- **Last card preview** — most recent `card_payload` as formatted JSON (collapsed by default).
38. The UI **must** show a toast on send and display per-webhook delivery results inline.
### 4.8 Dashboard Widget & Persistence
39. Each run (scheduled or manual/test) **must** persist to `morning_summaries`:
- `id` (serial PK)
- `generated_at` (timestamptz)
- `window_from` (timestamptz)
- `window_to` (timestamptz)
- `open_count` (int)
- `resolved_count` (int)
- `mttr_minutes` (int, nullable)
- `clients_affected` (text[])
- `is_weekend_window` (boolean)
- `card_payload` (jsonb)
- `delivery_status` (jsonb) — `{ [webhookId]: { success: bool, httpStatus?: number, error?: string } }`
40. The Pulse home/dashboard **must** display a "Morning Summary" widget with: open count,
resolved count, MTTR, window label, last generated timestamp, and a link to the admin
page.
---
## 5. Non-Goals (Out of Scope for v1)
- Veeam backup failure data (deferred to v2).
- PSA/Autotask ticket correlation and "unmatched alerts" (deferred to v2).
- Direct Teams DMs to individual users (requires bot registration — out of scope).
- Email or ntfy delivery channels (webhooks only for v1).
- ISP outage detection / grouping by ISP.
- Per-channel content customization (all webhooks receive the same card).
---
## 6. Design Considerations
### Adaptive Card Layout
```
┌──────────────────────────────────────────────────┐
│ ☀️ Morning NOC Summary — Mon Mar 16 (Weekend) │
├──────────┬────────────┬────────────────────────── │
│ 3 Open │ 12 Resolved│ 22m Avg MTTR │
├──────────────────────────────────────────────────┤
│ 🔴 OPEN ISSUES │
│ Kuhn's / FW-01 Host Unreachable — 6h 12m ⚠️ │
│ ADM / SW-Core High Packet Loss — 2h 5m │
│ Seubert / DC-01 Host Unreachable — 1h 20m │
├──────────────────────────────────────────────────┤
│ 🟢 RESOLVED OVERNIGHT │
│ Brodaks / RTR-01 Slow Response — 22m │
│ +11 more — all resolved │
├──────────────────────────────────────────────────┤
│[Open Zabbix][Open Pulse][View Problems] │
│[📣 Notify On Call] [🚨 Post to Outages] │
└──────────────────────────────────────────────────┘
```
- `"style": "attention"` container → red in Teams (open issues).
- `"style": "good"` container → green in Teams (resolved).
- Duration: `Xh Ym` for ≥ 1 hour, `Xm` for < 1 hour.
- Escalation buttons open a Pulse URL in browser; Pulse posts to channel server-side
and returns a simple confirmation page.
### Recipient Management UI (on `/admin/morning-summary`)
```
Recipients
┌────────────────────────────┬──────────┬────────────────────┐
│ Name / Email │ Status │ Actions │
├────────────────────────────┼──────────┼────────────────────┤
│ Tom Carlin │ ✅ Last: │ [Test] [Remove] │
│ tom@wulfconsulting.com │ today │ │
├────────────────────────────┼──────────┼────────────────────┤
│ Lorentz Hinrichsen │ ✅ Last: │ [Test] [Remove] │
│ lorentz@wulfconsulting.com │ today │ │
└────────────────────────────┴──────────┴────────────────────┘
[+ Add Recipient ▾] ← searchable dropdown of tenant users
```
### Webhook Delivery Flow
```
// For each enabled webhook in morning_summary_webhooks:
POST {webhook_url}
Content-Type: application/json
{
"type": "message",
"attachments": [{
"contentType": "application/vnd.microsoft.card.adaptive",
"contentUrl": null,
"content": { ...adaptive card JSON... }
}]
}
```
To get a webhook URL in Teams: go to the target channel **...** Connectors
**Incoming Webhook** Configure copy URL paste into Pulse admin.
---
## 7. Technical Considerations
### Existing Infrastructure to Reuse
- **`ZabbixClient`** (`lib/services/zabbix-client.ts`) add `getOpenProblems()` and
`getResolvedEvents(from: Date, to: Date)` methods.
- **`MsGraphClient`** no changes needed for v1.
- **`SyncScheduler`** add `'morning-summary'` sync type; register in `initialize()`.
- **Postgres** three new tables: `morning_summaries`, `morning_summary_webhooks`,
`morning_summary_config`.
### New Files to Create
| File | Purpose |
|------|---------|
| `lib/services/morning-summary-service.ts` | Aggregation, card building, webhook delivery |
| `app/api/notifications/morning-summary/send/route.ts` | `POST` send to all/selected webhooks |
| `app/api/notifications/morning-summary/test/route.ts` | `POST` test send to single webhook |
| `app/api/notifications/morning-summary/webhooks/route.ts` | `GET`/`POST` manage webhook list |
| `app/api/notifications/morning-summary/webhooks/[id]/route.ts` | `PUT`/`DELETE` update/remove webhook |
| `app/api/notifications/morning-summary/config/route.ts` | `GET`/`PUT` read/update settings |
| `app/admin/morning-summary/page.tsx` | Admin UI: webhooks, schedule, config, preview |
| `db/migrations/XXXX_morning_summary.sql` | All three new tables |
### Zabbix API Notes
- `problem.get` returns `acknowledged` as `"0"/"1"` strings and `clock` as Unix timestamp string.
- Use `selectAcknowledges: 'extend'` problem is unacknowledged if the array is empty.
- `event.get` with `value: 0` returns recoveries; duration = `r_clock - clock` (both seconds).
- Enrich problems with host/client by cross-referencing `objectid` (triggerid) against a
triggerhost map built from `trigger.get` with `selectHosts: ['hostid', 'name']`, or
directly via `problem.get` with `selectSuppressionData` + a pre-built hostidclient map.
### MS Graph Permissions
No Graph permissions are required for v1 delivery. Delivery uses Teams incoming webhook URLs
(plain HTTPS POST no auth needed beyond the secret embedded in the URL).
Existing granted permissions (`Chat.Create`, `Team.ReadBasic.All`, `Channel.ReadBasic.All`,
`ChatMessage.Send`, `ChannelMessage.Send`) are unused by this feature but can remain for
future use.
### Timezone
- Cron `30 6 * * 1-5` fires at 6:30 AM server time ensure server is set to ET or adjust cron.
- Report window computed in ET using `Intl.DateTimeFormat` or `date-fns-tz`.
- Monday extended window: `startOfDay(subDays(monday, 3)) + 18h` = Friday 18:00 ET.
---
## 8. Success Metrics
- All configured recipients receive the DM at 6:30 AM on weekdays without manual action.
- Test send from the UI delivers to the selected individual within 30 seconds.
- Escalation button (On Call or Outages) posts to the correct Teams channel within 10 seconds
and the user sees a confirmation page.
- Monday card correctly shows the extended weekend window label and data.
- Dashboard widget reflects the most recent summary within 1 minute of generation.
- Zero unhandled errors surfaced to end users; all failures logged and shown in the admin UI.
---
## 9. Open Questions
1. ~~**MS Graph permissions**~~ Not needed for v1 webhook delivery.
2. ~~**Team/channel names**~~ Confirmed: team = `Technical`, channels = `On-Call` and `Outages-Issues`.
3. ~~**App membership in Technical team**~~ Not needed; webhook URLs bypass Graph entirely.
4. **Error alerting** If the 6:30 AM scheduled job fails entirely, how should that be
surfaced? (ntfy push, Pulse admin badge, or both?)
5. **Severity filter default** Spec defaults to Warning (2) and above. Should
Information (1) problems be included? Confirm with Tom/Lorentz.
6. **Webhook URLs** Lorentz needs to create incoming webhooks in the `On-Call` and
`Outages-Issues` channels (Teams channel ... Connectors Incoming Webhook Configure)
and paste the URLs into the Pulse admin UI once the feature is deployed.