# Architecture: Pulse Morning NOC Summary
## Context
Wulf Consulting (MSP) needs a daily morning summary sent to management showing overnight activity
and open issues across client infrastructure. Pulse (Node.js/TypeScript) is the internal web app
with existing API integrations to Zabbix, PSA, RMM, Veeam, Zoom, and MS Graph. It already has a
job scheduler (bull/agenda/node-cron). Apprise is running on the monitoring server for notification routing.
## Architecture Overview
```
┌──────────────┐ schedule ┌──────────────────────┐
│ Job Scheduler├─────────────►│ Summary Aggregator │
│ (existing) │ 6:30 AM │ Service │
└──────────────┘ └──────┬───────────────┘
│ parallel queries
┌─────────────────┼─────────────────┐
▼ ▼ ▼
┌──────────┐ ┌──────────┐ ┌──────────┐
│ Zabbix │ │ PSA │ │ Veeam │
│ API │ │ API │ │ API │
└────┬─────┘ └────┬─────┘ └────┬─────┘
│ │ │
└───────┬───────┘ │
▼ │
┌─────────────────┐ │
│ Data Correlator │◄──────────────┘
│ & Formatter │
└────────┬────────┘
│
┌───────────┼───────────┐
▼ ▼ ▼
┌──────────┐ ┌────────┐ ┌─────────┐
│ Teams │ │ Apprise│ │ Pulse │
│ MS Graph│ │ (ntfy, │ │ DB + │
│ Adaptive│ │ email)│ │ Widget │
│ Card │ │ │ │ │
└──────────┘ └────────┘ └─────────┘
```
## Hybrid Notification Strategy
| Channel | Method | Why |
|---------|--------|-----|
| **Teams Adaptive Card** | Direct via MS Graph | Rich formatting, action buttons, inline rendering — can't get this through Apprise |
| **Email (HTML)** | Via Apprise | Apprise handles SMTP config, templating is simpler for email |
| **ntfy** | Via Apprise | Already configured, Apprise knows the topic/auth |
| **Pulse Dashboard** | Direct DB write | Store the summary as a record, render as a widget |
---
## Data Model
### 1. Summary Aggregator Service
Single service class/module: `MorningSummaryService`
```typescript
interface MorningSummary {
generatedAt: Date;
reportWindow: { from: Date; to: Date }; // e.g. 6pm → 6:30am
openProblems: Problem[]; // currently active — most important section
resolvedOvernight: Problem[]; // resolved during the window
backupFailures: BackupJob[]; // from Veeam API
unmatchedAlerts: Problem[]; // Zabbix problems with no PSA ticket (action needed!)
stats: {
totalIncidents: number;
resolved: number;
stillOpen: number;
mttrMinutes: number; // mean time to resolve (overnight only)
clientsAffected: string[];
};
}
interface Problem {
host: string;
client: string; // from Zabbix host group "Clients/..."
triggerName: string; // "Host Unreachable", "High Packet Loss", etc.
severity: string;
startedAt: Date;
resolvedAt?: Date;
duration: string;
psaTicketId?: string; // correlated from PSA
psaTicketUrl?: string;
acknowledged: boolean;
}
interface BackupJob {
client: string;
server: string;
jobName: string;
status: string;
lastRun: Date;
message?: string;
}
```
### 2. Zabbix API Queries Needed
**Open problems:**
```json
{
"method": "problem.get",
"params": {
"recent": true,
"sortfield": ["eventid"],
"sortorder": "DESC"
}
}
```
**Resolved overnight:**
```json
{
"method": "event.get",
"params": {
"source": 0,
"object": 0,
"value": 0,
"time_from": "<6pm_yesterday_unix>",
"time_to": "<6:30am_today_unix>",
"selectHosts": ["name"],
"selectRelatedObject": ["description"]
}
}
```
**Host → Client mapping:**
```json
{
"method": "host.get",
"params": {
"selectHostGroups": ["name"]
}
}
```
Filter groups starting with `Clients/` to determine the client name per host.
**Host groups available for reference:**
- `Clients/Kuhn's Quality Foods`, `Clients/ADM Signs`, `Clients/Seubert and Associates`, etc. (50+ clients)
- `ISP/Comcast Cable Communications, LLC`, `ISP/AT&T Enterprises, LLC`, etc. (25+ ISPs)
### 3. PSA Correlation
For each open Zabbix problem, query the PSA for matching tickets:
- Match on hostname or client name + date range
- Flag any Zabbix problem that has NO corresponding PSA ticket — these are "unmatched"
and should be highlighted as needing attention
### 4. Veeam Backup Failures
Query Veeam API for jobs that ran overnight with status != Success.
Include in a separate section of the summary.
---
## Teams Adaptive Card Design
```json
{
"type": "AdaptiveCard",
"$schema": "http://adaptivecards.io/schemas/adaptive-card.json",
"version": "1.4",
"body": [
{
"type": "TextBlock",
"text": "☀️ Morning NOC Summary — Mar 12, 2026",
"weight": "bolder",
"size": "large"
},
{
"type": "ColumnSet",
"columns": [
{
"type": "Column",
"width": "auto",
"items": [
{ "type": "TextBlock", "text": "3", "size": "extraLarge", "color": "attention", "weight": "bolder" },
{ "type": "TextBlock", "text": "Open", "spacing": "none" }
]
},
{
"type": "Column",
"width": "auto",
"items": [
{ "type": "TextBlock", "text": "5", "size": "extraLarge", "color": "good", "weight": "bolder" },
{ "type": "TextBlock", "text": "Resolved", "spacing": "none" }
]
},
{
"type": "Column",
"width": "auto",
"items": [
{ "type": "TextBlock", "text": "18m", "size": "extraLarge", "weight": "bolder" },
{ "type": "TextBlock", "text": "Avg MTTR", "spacing": "none" }
]
}
]
},
{
"type": "Container",
"style": "attention",
"bleed": true,
"items": [
{ "type": "TextBlock", "text": "🔴 OPEN ISSUES", "weight": "bolder", "spacing": "small" },
{
"type": "FactSet",
"facts": [
{ "title": "Kuhn's / FW-01", "value": "Host Unreachable — 4h 12m" },
{ "title": "ADM / SW-Core", "value": "High Packet Loss — 2h 5m" },
{ "title": "Seubert / DC-01", "value": "⚠️ Backup Failed — 6h (no ticket!)" }
]
}
]
},
{
"type": "Container",
"style": "good",
"bleed": true,
"items": [
{ "type": "TextBlock", "text": "🟢 RESOLVED OVERNIGHT", "weight": "bolder", "spacing": "small" },
{
"type": "FactSet",
"facts": [
{ "title": "LWHResTest", "value": "Host Unreachable — resolved in 15m" },
{ "title": "Brodaks / RTR-01", "value": "Slow Response — resolved in 22m" },
{ "title": "+3 more", "value": "All auto-resolved" }
]
}
]
}
],
"actions": [
{ "type": "Action.OpenUrl", "title": "Open Zabbix", "url": "https://zabbix.wulfconsulting.cloud" },
{ "type": "Action.OpenUrl", "title": "Open Pulse", "url": "https://pulse.wulfconsulting.cloud" },
{ "type": "Action.OpenUrl", "title": "Ack All Open", "url": "https://pulse.wulfconsulting.cloud/ack-all" }
]
}
```
Post via MS Graph:
```
POST https://graph.microsoft.com/v1.0/teams/{teamId}/channels/{channelId}/messages
Content-Type: application/json
{
"body": {
"contentType": "html",
"content": ""
},
"attachments": [{
"id": "card",
"contentType": "application/vnd.microsoft.card.adaptive",
"content": ""
}]
}
```
---
## User Preference System
```typescript
interface NotificationPreferences {
userId: string;
morningSummary: {
enabled: boolean;
channels: ('teams' | 'email' | 'ntfy' | 'pulse')[];
schedule: string; // cron expression, default "30 6 * * 1-5"
timezone: string; // "America/New_York"
includeBackups: boolean;
includeResolvedDetail: boolean; // some execs just want open issues
severityFilter: number; // minimum severity to include (default: 2/Warning)
};
}
```
Store in Pulse's existing user/settings table. Expose in Pulse UI as a settings page.
---
## Job Scheduler Integration
Use the existing scheduler to run the aggregation:
```typescript
// Register the job
scheduler.register('morning-summary', '30 6 * * 1-5', async () => {
const users = await getUsersWithMorningSummaryEnabled();
// Aggregate once (shared data)
const summary = await morningSummaryService.aggregate();
// Store for Pulse dashboard widget
await morningSummaryService.persist(summary);
// Deliver per user preferences
for (const user of users) {
const prefs = user.notificationPreferences.morningSummary;
if (prefs.channels.includes('teams'))
await teamsService.postAdaptiveCard(user, summary);
if (prefs.channels.includes('email'))
await appriseService.sendEmail(user, formatEmailHtml(summary));
if (prefs.channels.includes('ntfy'))
await appriseService.sendNtfy(formatNtfySummary(summary));
}
});
```
### Aggregator Pseudocode
```typescript
class MorningSummaryService {
async aggregate(): Promise {
const now = new Date();
const windowStart = yesterday6pm(now);
const windowEnd = now;
// Run all API calls in parallel
const [zabbixOpen, zabbixResolved, backups, hostGroups] = await Promise.all([
this.zabbixApi.getOpenProblems(),
this.zabbixApi.getResolvedEvents(windowStart, windowEnd),
this.veeamApi.getOvernightJobs(windowStart, windowEnd),
this.zabbixApi.getHostGroupMapping() // cache this, changes rarely
]);
// Map hosts → client names via host groups
const clientMap = buildClientMap(hostGroups);
// Correlate with PSA tickets
const openWithTickets = await this.psaApi.correlateProblems(zabbixOpen, clientMap);
// Find unmatched (no PSA ticket)
const unmatched = openWithTickets.filter(p => !p.psaTicketId);
// Calculate stats
const mttr = calculateMTTR(zabbixResolved);
return {
generatedAt: now,
reportWindow: { from: windowStart, to: windowEnd },
openProblems: openWithTickets,
resolvedOvernight: zabbixResolved.map(e => enrichWithClient(e, clientMap)),
backupFailures: backups.filter(b => b.status !== 'Success'),
unmatchedAlerts: unmatched,
stats: {
totalIncidents: zabbixOpen.length + zabbixResolved.length,
resolved: zabbixResolved.length,
stillOpen: zabbixOpen.length,
mttrMinutes: mttr,
clientsAffected: [...new Set(openWithTickets.map(p => p.client))]
}
};
}
}
```
---
## Key Recommendations
1. **Aggregate once, deliver many** — Don't re-query Zabbix/PSA/Veeam per user. Run the
aggregation once, then fan out to each user's preferred channels.
2. **"Unmatched alerts" section is the killer feature** — Highlighting Zabbix problems that
have no PSA ticket is what will make management love this. It shows gaps in the process.
3. **Teams adaptive card direct, everything else through Apprise** — Adaptive cards need
the MS Graph payload format which Apprise can't produce. For simpler formats (email body,
ntfy text), Apprise handles routing without Pulse needing SMTP config.
4. **Pulse dashboard widget** — Persist each summary to the DB. Show the latest on the
Pulse home screen so anyone can check it anytime, not just at 6:30 AM.
5. **Weekend mode** — Consider a different schedule or suppression for weekends. The cron
`30 6 * * 1-5` only fires Mon-Fri. But you may want a Monday morning summary that covers
the full weekend window (Friday 6pm → Monday 6:30am).
6. **Escalation hint** — If any problem has been open > 4 hours with no PSA ticket and no
acknowledgement, flag it red in the card with "Needs Attention" — gives management
actionable signal, not just data.
7. **Cache the host → client mapping** — The `Clients/` host group mapping rarely changes.
Cache it in Pulse (refresh every few hours) to avoid an API call on every summary run.
---
## Zabbix Host Group Reference
Your host groups are well-organized for this feature:
**Client groups (50+):** `Clients/Kuhn's Quality Foods`, `Clients/ADM Signs`, `Clients/Seubert and Associates`, `Clients/Brodaks`, etc.
**ISP groups (25+):** `ISP/Comcast`, `ISP/AT&T`, `ISP/Armstrong`, `ISP/Bigleaf`, etc.
The ISP grouping can be used for a future enhancement: "ISP Outage Detection" — if 3+ hosts
on the same ISP go down simultaneously, flag it as a likely ISP outage rather than individual
site problems.
---
## ntfy Summary Format (via Apprise)
For the condensed ntfy version:
```
☀️ Morning Summary — Mar 12
🔴 3 Open
· Host Unreachable — Kuhn's / FW-01 (4h)
· High Packet Loss — ADM / SW-Core (2h)
· Backup Failed — Seubert / DC-01 (6h)
🟢 5 Resolved overnight (avg 18m)
⚠️ 1 issue with no ticket
```
---
## Email HTML Format (via Apprise)
Use a clean responsive HTML template with:
- Header with date and stats (open/resolved/MTTR)
- Red-bordered table for open issues
- Green-bordered table for resolved
- Yellow callout box for unmatched alerts
- Footer with links to Zabbix and Pulse
Keep it mobile-friendly — management reads email on phones.