feat: Morning NOC Summary adaptive card for Teams
- Add MorningSummaryService with Zabbix aggregation and adaptive card builder - Add webhook delivery system with Teams incoming webhooks - Add admin UI at /admin/morning-summary for webhook/config management - Add API routes: /send, /test, /webhooks, /webhooks/[id], /config, /history - Register morning-summary cron job in SyncScheduler (Mon-Fri 6:30 AM) - Add outages_only filter (Unavailable triggers only) - Fix host resolution: use getTriggerEnabledHosts to exclude disabled hosts - Fix resolved events: event.get value:1 scoped to window with r_eventid filter - Remove emojis from fact rows and section headers in card - Remove Open Zabbix button (duplicate of View Problems) - Add migrations: morning_summary_config + morning_summaries tables - Add outages_only column to morning_summary_config
This commit is contained in:
parent
19605f82aa
commit
c518eefdb2
61 changed files with 11236 additions and 237 deletions
435
dev/pulse-morning-summary-architecture.md
Normal file
435
dev/pulse-morning-summary-architecture.md
Normal file
|
|
@ -0,0 +1,435 @@
|
|||
# Architecture: Pulse Morning NOC Summary
|
||||
|
||||
## Context
|
||||
Wulf Consulting (MSP) needs a daily morning summary sent to management showing overnight activity
|
||||
and open issues across client infrastructure. Pulse (Node.js/TypeScript) is the internal web app
|
||||
with existing API integrations to Zabbix, PSA, RMM, Veeam, Zoom, and MS Graph. It already has a
|
||||
job scheduler (bull/agenda/node-cron). Apprise is running on the monitoring server for notification routing.
|
||||
|
||||
## Architecture Overview
|
||||
|
||||
```
|
||||
┌──────────────┐ schedule ┌──────────────────────┐
|
||||
│ Job Scheduler├─────────────►│ Summary Aggregator │
|
||||
│ (existing) │ 6:30 AM │ Service │
|
||||
└──────────────┘ └──────┬───────────────┘
|
||||
│ parallel queries
|
||||
┌─────────────────┼─────────────────┐
|
||||
▼ ▼ ▼
|
||||
┌──────────┐ ┌──────────┐ ┌──────────┐
|
||||
│ Zabbix │ │ PSA │ │ Veeam │
|
||||
│ API │ │ API │ │ API │
|
||||
└────┬─────┘ └────┬─────┘ └────┬─────┘
|
||||
│ │ │
|
||||
└───────┬───────┘ │
|
||||
▼ │
|
||||
┌─────────────────┐ │
|
||||
│ Data Correlator │◄──────────────┘
|
||||
│ & Formatter │
|
||||
└────────┬────────┘
|
||||
│
|
||||
┌───────────┼───────────┐
|
||||
▼ ▼ ▼
|
||||
┌──────────┐ ┌────────┐ ┌─────────┐
|
||||
│ Teams │ │ Apprise│ │ Pulse │
|
||||
│ MS Graph│ │ (ntfy, │ │ DB + │
|
||||
│ Adaptive│ │ email)│ │ Widget │
|
||||
│ Card │ │ │ │ │
|
||||
└──────────┘ └────────┘ └─────────┘
|
||||
```
|
||||
|
||||
## Hybrid Notification Strategy
|
||||
|
||||
| Channel | Method | Why |
|
||||
|---------|--------|-----|
|
||||
| **Teams Adaptive Card** | Direct via MS Graph | Rich formatting, action buttons, inline rendering — can't get this through Apprise |
|
||||
| **Email (HTML)** | Via Apprise | Apprise handles SMTP config, templating is simpler for email |
|
||||
| **ntfy** | Via Apprise | Already configured, Apprise knows the topic/auth |
|
||||
| **Pulse Dashboard** | Direct DB write | Store the summary as a record, render as a widget |
|
||||
|
||||
---
|
||||
|
||||
## Data Model
|
||||
|
||||
### 1. Summary Aggregator Service
|
||||
|
||||
Single service class/module: `MorningSummaryService`
|
||||
|
||||
```typescript
|
||||
interface MorningSummary {
|
||||
generatedAt: Date;
|
||||
reportWindow: { from: Date; to: Date }; // e.g. 6pm → 6:30am
|
||||
|
||||
openProblems: Problem[]; // currently active — most important section
|
||||
resolvedOvernight: Problem[]; // resolved during the window
|
||||
backupFailures: BackupJob[]; // from Veeam API
|
||||
unmatchedAlerts: Problem[]; // Zabbix problems with no PSA ticket (action needed!)
|
||||
|
||||
stats: {
|
||||
totalIncidents: number;
|
||||
resolved: number;
|
||||
stillOpen: number;
|
||||
mttrMinutes: number; // mean time to resolve (overnight only)
|
||||
clientsAffected: string[];
|
||||
};
|
||||
}
|
||||
|
||||
interface Problem {
|
||||
host: string;
|
||||
client: string; // from Zabbix host group "Clients/..."
|
||||
triggerName: string; // "Host Unreachable", "High Packet Loss", etc.
|
||||
severity: string;
|
||||
startedAt: Date;
|
||||
resolvedAt?: Date;
|
||||
duration: string;
|
||||
psaTicketId?: string; // correlated from PSA
|
||||
psaTicketUrl?: string;
|
||||
acknowledged: boolean;
|
||||
}
|
||||
|
||||
interface BackupJob {
|
||||
client: string;
|
||||
server: string;
|
||||
jobName: string;
|
||||
status: string;
|
||||
lastRun: Date;
|
||||
message?: string;
|
||||
}
|
||||
```
|
||||
|
||||
### 2. Zabbix API Queries Needed
|
||||
|
||||
**Open problems:**
|
||||
```json
|
||||
{
|
||||
"method": "problem.get",
|
||||
"params": {
|
||||
"recent": true,
|
||||
"sortfield": ["eventid"],
|
||||
"sortorder": "DESC"
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
**Resolved overnight:**
|
||||
```json
|
||||
{
|
||||
"method": "event.get",
|
||||
"params": {
|
||||
"source": 0,
|
||||
"object": 0,
|
||||
"value": 0,
|
||||
"time_from": "<6pm_yesterday_unix>",
|
||||
"time_to": "<6:30am_today_unix>",
|
||||
"selectHosts": ["name"],
|
||||
"selectRelatedObject": ["description"]
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
**Host → Client mapping:**
|
||||
```json
|
||||
{
|
||||
"method": "host.get",
|
||||
"params": {
|
||||
"selectHostGroups": ["name"]
|
||||
}
|
||||
}
|
||||
```
|
||||
Filter groups starting with `Clients/` to determine the client name per host.
|
||||
|
||||
**Host groups available for reference:**
|
||||
- `Clients/Kuhn's Quality Foods`, `Clients/ADM Signs`, `Clients/Seubert and Associates`, etc. (50+ clients)
|
||||
- `ISP/Comcast Cable Communications, LLC`, `ISP/AT&T Enterprises, LLC`, etc. (25+ ISPs)
|
||||
|
||||
### 3. PSA Correlation
|
||||
|
||||
For each open Zabbix problem, query the PSA for matching tickets:
|
||||
- Match on hostname or client name + date range
|
||||
- Flag any Zabbix problem that has NO corresponding PSA ticket — these are "unmatched"
|
||||
and should be highlighted as needing attention
|
||||
|
||||
### 4. Veeam Backup Failures
|
||||
|
||||
Query Veeam API for jobs that ran overnight with status != Success.
|
||||
Include in a separate section of the summary.
|
||||
|
||||
---
|
||||
|
||||
## Teams Adaptive Card Design
|
||||
|
||||
```json
|
||||
{
|
||||
"type": "AdaptiveCard",
|
||||
"$schema": "http://adaptivecards.io/schemas/adaptive-card.json",
|
||||
"version": "1.4",
|
||||
"body": [
|
||||
{
|
||||
"type": "TextBlock",
|
||||
"text": "☀️ Morning NOC Summary — Mar 12, 2026",
|
||||
"weight": "bolder",
|
||||
"size": "large"
|
||||
},
|
||||
{
|
||||
"type": "ColumnSet",
|
||||
"columns": [
|
||||
{
|
||||
"type": "Column",
|
||||
"width": "auto",
|
||||
"items": [
|
||||
{ "type": "TextBlock", "text": "3", "size": "extraLarge", "color": "attention", "weight": "bolder" },
|
||||
{ "type": "TextBlock", "text": "Open", "spacing": "none" }
|
||||
]
|
||||
},
|
||||
{
|
||||
"type": "Column",
|
||||
"width": "auto",
|
||||
"items": [
|
||||
{ "type": "TextBlock", "text": "5", "size": "extraLarge", "color": "good", "weight": "bolder" },
|
||||
{ "type": "TextBlock", "text": "Resolved", "spacing": "none" }
|
||||
]
|
||||
},
|
||||
{
|
||||
"type": "Column",
|
||||
"width": "auto",
|
||||
"items": [
|
||||
{ "type": "TextBlock", "text": "18m", "size": "extraLarge", "weight": "bolder" },
|
||||
{ "type": "TextBlock", "text": "Avg MTTR", "spacing": "none" }
|
||||
]
|
||||
}
|
||||
]
|
||||
},
|
||||
{
|
||||
"type": "Container",
|
||||
"style": "attention",
|
||||
"bleed": true,
|
||||
"items": [
|
||||
{ "type": "TextBlock", "text": "🔴 OPEN ISSUES", "weight": "bolder", "spacing": "small" },
|
||||
{
|
||||
"type": "FactSet",
|
||||
"facts": [
|
||||
{ "title": "Kuhn's / FW-01", "value": "Host Unreachable — 4h 12m" },
|
||||
{ "title": "ADM / SW-Core", "value": "High Packet Loss — 2h 5m" },
|
||||
{ "title": "Seubert / DC-01", "value": "⚠️ Backup Failed — 6h (no ticket!)" }
|
||||
]
|
||||
}
|
||||
]
|
||||
},
|
||||
{
|
||||
"type": "Container",
|
||||
"style": "good",
|
||||
"bleed": true,
|
||||
"items": [
|
||||
{ "type": "TextBlock", "text": "🟢 RESOLVED OVERNIGHT", "weight": "bolder", "spacing": "small" },
|
||||
{
|
||||
"type": "FactSet",
|
||||
"facts": [
|
||||
{ "title": "LWHResTest", "value": "Host Unreachable — resolved in 15m" },
|
||||
{ "title": "Brodaks / RTR-01", "value": "Slow Response — resolved in 22m" },
|
||||
{ "title": "+3 more", "value": "All auto-resolved" }
|
||||
]
|
||||
}
|
||||
]
|
||||
}
|
||||
],
|
||||
"actions": [
|
||||
{ "type": "Action.OpenUrl", "title": "Open Zabbix", "url": "https://zabbix.wulfconsulting.cloud" },
|
||||
{ "type": "Action.OpenUrl", "title": "Open Pulse", "url": "https://pulse.wulfconsulting.cloud" },
|
||||
{ "type": "Action.OpenUrl", "title": "Ack All Open", "url": "https://pulse.wulfconsulting.cloud/ack-all" }
|
||||
]
|
||||
}
|
||||
```
|
||||
|
||||
Post via MS Graph:
|
||||
```
|
||||
POST https://graph.microsoft.com/v1.0/teams/{teamId}/channels/{channelId}/messages
|
||||
Content-Type: application/json
|
||||
|
||||
{
|
||||
"body": {
|
||||
"contentType": "html",
|
||||
"content": "<attachment id=\"card\"></attachment>"
|
||||
},
|
||||
"attachments": [{
|
||||
"id": "card",
|
||||
"contentType": "application/vnd.microsoft.card.adaptive",
|
||||
"content": "<adaptive card JSON string>"
|
||||
}]
|
||||
}
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## User Preference System
|
||||
|
||||
```typescript
|
||||
interface NotificationPreferences {
|
||||
userId: string;
|
||||
morningSummary: {
|
||||
enabled: boolean;
|
||||
channels: ('teams' | 'email' | 'ntfy' | 'pulse')[];
|
||||
schedule: string; // cron expression, default "30 6 * * 1-5"
|
||||
timezone: string; // "America/New_York"
|
||||
includeBackups: boolean;
|
||||
includeResolvedDetail: boolean; // some execs just want open issues
|
||||
severityFilter: number; // minimum severity to include (default: 2/Warning)
|
||||
};
|
||||
}
|
||||
```
|
||||
|
||||
Store in Pulse's existing user/settings table. Expose in Pulse UI as a settings page.
|
||||
|
||||
---
|
||||
|
||||
## Job Scheduler Integration
|
||||
|
||||
Use the existing scheduler to run the aggregation:
|
||||
|
||||
```typescript
|
||||
// Register the job
|
||||
scheduler.register('morning-summary', '30 6 * * 1-5', async () => {
|
||||
const users = await getUsersWithMorningSummaryEnabled();
|
||||
|
||||
// Aggregate once (shared data)
|
||||
const summary = await morningSummaryService.aggregate();
|
||||
|
||||
// Store for Pulse dashboard widget
|
||||
await morningSummaryService.persist(summary);
|
||||
|
||||
// Deliver per user preferences
|
||||
for (const user of users) {
|
||||
const prefs = user.notificationPreferences.morningSummary;
|
||||
|
||||
if (prefs.channels.includes('teams'))
|
||||
await teamsService.postAdaptiveCard(user, summary);
|
||||
|
||||
if (prefs.channels.includes('email'))
|
||||
await appriseService.sendEmail(user, formatEmailHtml(summary));
|
||||
|
||||
if (prefs.channels.includes('ntfy'))
|
||||
await appriseService.sendNtfy(formatNtfySummary(summary));
|
||||
}
|
||||
});
|
||||
```
|
||||
|
||||
### Aggregator Pseudocode
|
||||
|
||||
```typescript
|
||||
class MorningSummaryService {
|
||||
async aggregate(): Promise<MorningSummary> {
|
||||
const now = new Date();
|
||||
const windowStart = yesterday6pm(now);
|
||||
const windowEnd = now;
|
||||
|
||||
// Run all API calls in parallel
|
||||
const [zabbixOpen, zabbixResolved, backups, hostGroups] = await Promise.all([
|
||||
this.zabbixApi.getOpenProblems(),
|
||||
this.zabbixApi.getResolvedEvents(windowStart, windowEnd),
|
||||
this.veeamApi.getOvernightJobs(windowStart, windowEnd),
|
||||
this.zabbixApi.getHostGroupMapping() // cache this, changes rarely
|
||||
]);
|
||||
|
||||
// Map hosts → client names via host groups
|
||||
const clientMap = buildClientMap(hostGroups);
|
||||
|
||||
// Correlate with PSA tickets
|
||||
const openWithTickets = await this.psaApi.correlateProblems(zabbixOpen, clientMap);
|
||||
|
||||
// Find unmatched (no PSA ticket)
|
||||
const unmatched = openWithTickets.filter(p => !p.psaTicketId);
|
||||
|
||||
// Calculate stats
|
||||
const mttr = calculateMTTR(zabbixResolved);
|
||||
|
||||
return {
|
||||
generatedAt: now,
|
||||
reportWindow: { from: windowStart, to: windowEnd },
|
||||
openProblems: openWithTickets,
|
||||
resolvedOvernight: zabbixResolved.map(e => enrichWithClient(e, clientMap)),
|
||||
backupFailures: backups.filter(b => b.status !== 'Success'),
|
||||
unmatchedAlerts: unmatched,
|
||||
stats: {
|
||||
totalIncidents: zabbixOpen.length + zabbixResolved.length,
|
||||
resolved: zabbixResolved.length,
|
||||
stillOpen: zabbixOpen.length,
|
||||
mttrMinutes: mttr,
|
||||
clientsAffected: [...new Set(openWithTickets.map(p => p.client))]
|
||||
}
|
||||
};
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Key Recommendations
|
||||
|
||||
1. **Aggregate once, deliver many** — Don't re-query Zabbix/PSA/Veeam per user. Run the
|
||||
aggregation once, then fan out to each user's preferred channels.
|
||||
|
||||
2. **"Unmatched alerts" section is the killer feature** — Highlighting Zabbix problems that
|
||||
have no PSA ticket is what will make management love this. It shows gaps in the process.
|
||||
|
||||
3. **Teams adaptive card direct, everything else through Apprise** — Adaptive cards need
|
||||
the MS Graph payload format which Apprise can't produce. For simpler formats (email body,
|
||||
ntfy text), Apprise handles routing without Pulse needing SMTP config.
|
||||
|
||||
4. **Pulse dashboard widget** — Persist each summary to the DB. Show the latest on the
|
||||
Pulse home screen so anyone can check it anytime, not just at 6:30 AM.
|
||||
|
||||
5. **Weekend mode** — Consider a different schedule or suppression for weekends. The cron
|
||||
`30 6 * * 1-5` only fires Mon-Fri. But you may want a Monday morning summary that covers
|
||||
the full weekend window (Friday 6pm → Monday 6:30am).
|
||||
|
||||
6. **Escalation hint** — If any problem has been open > 4 hours with no PSA ticket and no
|
||||
acknowledgement, flag it red in the card with "Needs Attention" — gives management
|
||||
actionable signal, not just data.
|
||||
|
||||
7. **Cache the host → client mapping** — The `Clients/` host group mapping rarely changes.
|
||||
Cache it in Pulse (refresh every few hours) to avoid an API call on every summary run.
|
||||
|
||||
---
|
||||
|
||||
## Zabbix Host Group Reference
|
||||
|
||||
Your host groups are well-organized for this feature:
|
||||
|
||||
**Client groups (50+):** `Clients/Kuhn's Quality Foods`, `Clients/ADM Signs`, `Clients/Seubert and Associates`, `Clients/Brodaks`, etc.
|
||||
|
||||
**ISP groups (25+):** `ISP/Comcast`, `ISP/AT&T`, `ISP/Armstrong`, `ISP/Bigleaf`, etc.
|
||||
|
||||
The ISP grouping can be used for a future enhancement: "ISP Outage Detection" — if 3+ hosts
|
||||
on the same ISP go down simultaneously, flag it as a likely ISP outage rather than individual
|
||||
site problems.
|
||||
|
||||
---
|
||||
|
||||
## ntfy Summary Format (via Apprise)
|
||||
|
||||
For the condensed ntfy version:
|
||||
|
||||
```
|
||||
☀️ Morning Summary — Mar 12
|
||||
|
||||
🔴 3 Open
|
||||
· Host Unreachable — Kuhn's / FW-01 (4h)
|
||||
· High Packet Loss — ADM / SW-Core (2h)
|
||||
· Backup Failed — Seubert / DC-01 (6h)
|
||||
|
||||
🟢 5 Resolved overnight (avg 18m)
|
||||
|
||||
⚠️ 1 issue with no ticket
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Email HTML Format (via Apprise)
|
||||
|
||||
Use a clean responsive HTML template with:
|
||||
- Header with date and stats (open/resolved/MTTR)
|
||||
- Red-bordered table for open issues
|
||||
- Green-bordered table for resolved
|
||||
- Yellow callout box for unmatched alerts
|
||||
- Footer with links to Zabbix and Pulse
|
||||
|
||||
Keep it mobile-friendly — management reads email on phones.
|
||||
774
dev/windsurf-zabbix-development-guide.md
Normal file
774
dev/windsurf-zabbix-development-guide.md
Normal file
|
|
@ -0,0 +1,774 @@
|
|||
# Windsurf + Sonnet Development Guide — Zabbix Monitoring System
|
||||
|
||||
> **For:** AI-assisted development in Windsurf using Claude Sonnet 4.6
|
||||
> **Organization:** Wulf Consulting (MSP)
|
||||
> **Last updated:** 2026-03-11
|
||||
|
||||
---
|
||||
|
||||
## Table of Contents
|
||||
|
||||
1. [System Overview](#1-system-overview)
|
||||
2. [Infrastructure & Network Topology](#2-infrastructure--network-topology)
|
||||
3. [Zabbix API Reference](#3-zabbix-api-reference)
|
||||
4. [Host Organization & Data Model](#4-host-organization--data-model)
|
||||
5. [Notification Pipeline](#5-notification-pipeline)
|
||||
6. [Trigger Naming Conventions](#6-trigger-naming-conventions)
|
||||
7. [Grafana Integration](#7-grafana-integration)
|
||||
8. [Pulse Integration Architecture](#8-pulse-integration-architecture)
|
||||
9. [Development Patterns & Gotchas](#9-development-patterns--gotchas)
|
||||
10. [API Cookbook](#10-api-cookbook)
|
||||
|
||||
---
|
||||
|
||||
## 1. System Overview
|
||||
|
||||
Wulf Consulting is an MSP managing ~46 client sites. The monitoring stack runs on a single
|
||||
Ubuntu 24.04 server (public IP: 209.166.162.245) with all services containerized in Docker.
|
||||
|
||||
### Stack Components
|
||||
|
||||
| Service | Container | Image | Purpose |
|
||||
|---------|-----------|-------|---------|
|
||||
| **Zabbix Server** | `zabbix-server` | `zabbix/zabbix-server-pgsql:alpine-7.4-latest` | Core monitoring engine |
|
||||
| **Zabbix Frontend** | `zabbix-frontend` | `zabbix/zabbix-web-nginx-pgsql:alpine-7.4-latest` | Web UI + API endpoint |
|
||||
| **PostgreSQL** | `zabbix-postgres` | `postgres:17-alpine` | Zabbix database |
|
||||
| **Grafana** | `grafana` | `grafana/grafana:latest` | Dashboards + Zabbix plugin |
|
||||
| **ntfy** | `ntfy` | `binwiederhier/ntfy:latest` | Push notification server |
|
||||
| **Apprise** | `apprise-api` | `caronc/apprise:latest` | Multi-channel notification router |
|
||||
| **Authentik** | `authentik` | `ghcr.io/goauthentik/server:2025.8.1` | SSO/identity provider |
|
||||
| **Newt** | `newt` | `fosrl/newt` | Pangolin tunnel agent |
|
||||
|
||||
### External Access
|
||||
|
||||
All services are exposed through **Pangolin** (reverse proxy/tunnel), not direct port mappings.
|
||||
|
||||
| Service | External URL |
|
||||
|---------|-------------|
|
||||
| Zabbix | `https://zabbix.wulfconsulting.cloud` |
|
||||
| Grafana | *(via Pangolin — check Pangolin config for exact URL)* |
|
||||
|
||||
SSO is handled by Authentik with SAML integration to Zabbix.
|
||||
|
||||
### Software Versions
|
||||
|
||||
- **Zabbix:** 7.4.7
|
||||
- **PostgreSQL:** 17 (Alpine)
|
||||
- **Grafana:** Latest (with `alexanderzobnin-zabbix-app` 6.2.1)
|
||||
- **Host OS:** Ubuntu 24.04.4 LTS
|
||||
|
||||
---
|
||||
|
||||
## 2. Infrastructure & Network Topology
|
||||
|
||||
### Docker Networks
|
||||
|
||||
```
|
||||
┌─────────────────────────────────────────────────────────────┐
|
||||
│ pangolin (172.18.0.0/16) │
|
||||
│ ┌─────────────┐ ┌──────────┐ ┌──────┐ ┌───────────────┐ │
|
||||
│ │zabbix-front │ │ grafana │ │ ntfy │ │ apprise-api │ │
|
||||
│ │ 172.18.0.4 │ │172.18.0.3│ │.0.7 │ │ 172.18.0.5 │ │
|
||||
│ └──────┬──────┘ └────┬─────┘ └──────┘ └───────────────┘ │
|
||||
│ │ │ │
|
||||
│ ┌──────┴──────┐ │ ┌──────────┐ ┌───────────┐ │
|
||||
│ │zabbix-server│ │ │authentik │ │ newt │ │
|
||||
│ │ 172.18.0.8 │ │ │172.18.0.6│ │172.18.0.2 │ │
|
||||
│ └──────┬──────┘ │ └──────────┘ └───────────┘ │
|
||||
│ │ │ │
|
||||
└─────────┼──────────────┼─────────────────────────────────────┘
|
||||
│ │
|
||||
┌─────────┼──────────────┼──────────────────────┐
|
||||
│ │ zabbix_zabbix_internal (172.19.0.0/16) │
|
||||
│ ┌──────┴──────┐ ┌────┴─────┐ ┌──────────────┐│
|
||||
│ │zabbix-server│ │ grafana │ │zabbix-frontend││
|
||||
│ │ 172.19.0.4 │ │172.19.0.3│ │ 172.19.0.5 ││
|
||||
│ └─────────────┘ └──────────┘ └──────────────┘│
|
||||
│ ┌──────────────┐│
|
||||
│ │zabbix-postgres││
|
||||
│ │ 172.19.0.2 ││
|
||||
│ └──────────────┘│
|
||||
└─────────────────────────────────────────────────┘
|
||||
```
|
||||
|
||||
### Key Connectivity Facts
|
||||
|
||||
- **Zabbix API (internal):** `http://zabbix-frontend:8080/api_jsonrpc.php` — accessible from `zabbix-server`, `grafana`, and anything on `zabbix_zabbix_internal`
|
||||
- **Zabbix API (external):** `https://zabbix.wulfconsulting.cloud/api_jsonrpc.php` — via Pangolin
|
||||
- **ntfy (internal):** `http://ntfy:80` — accessible from `zabbix-server` and `zabbix-frontend` via `pangolin` network
|
||||
- **Apprise API:** `http://apprise-api:8000` (also `0.0.0.0:8000` on host)
|
||||
- **Zabbix agent port:** `10051` (mapped to host `0.0.0.0:10051`)
|
||||
- **Grafana** has NO external port mapping — accessed only through Pangolin
|
||||
|
||||
### Docker Compose Location
|
||||
|
||||
All stack definitions: `/opt/stacks/zabbix/compose.yml`
|
||||
|
||||
Environment variables: `/opt/stacks/zabbix/.env`
|
||||
|
||||
---
|
||||
|
||||
## 3. Zabbix API Reference
|
||||
|
||||
### Authentication
|
||||
|
||||
Zabbix 7.4 uses Bearer token authentication:
|
||||
|
||||
```
|
||||
Authorization: Bearer <api_token>
|
||||
```
|
||||
|
||||
API tokens are generated in Zabbix UI: **User Settings → API tokens**
|
||||
|
||||
> **Important:** The `apiinfo.version` method MUST be called WITHOUT the Authorization header.
|
||||
> All other methods require it.
|
||||
|
||||
### Base Request Format
|
||||
|
||||
```json
|
||||
{
|
||||
"jsonrpc": "2.0",
|
||||
"method": "<method_name>",
|
||||
"params": { ... },
|
||||
"id": 1
|
||||
}
|
||||
```
|
||||
|
||||
### API Endpoint
|
||||
|
||||
| Context | URL |
|
||||
|---------|-----|
|
||||
| From `zabbix-server` or `grafana` container | `http://zabbix-frontend:8080/api_jsonrpc.php` |
|
||||
| From the Docker host | `docker exec zabbix-frontend curl -s -X POST http://localhost:8080/api_jsonrpc.php ...` |
|
||||
| From external / Pulse | `https://zabbix.wulfconsulting.cloud/api_jsonrpc.php` |
|
||||
|
||||
### Key API Methods Used
|
||||
|
||||
| Method | Purpose | Notes |
|
||||
|--------|---------|-------|
|
||||
| `host.get` | List hosts, get groups/templates | Use `selectHostGroups`, `selectParentTemplates` |
|
||||
| `hostgroup.get` | List host groups | Filter by `Clients/` or `ISP/` prefix |
|
||||
| `trigger.get` | Get triggers, active problems | `only_true: true` for currently-firing |
|
||||
| `problem.get` | Get current problems | `recent: true` for unresolved |
|
||||
| `event.get` | Get events (problems + recoveries) | Use `time_from`/`time_to`, `value: 0` for recovery |
|
||||
| `mediatype.get` | Get notification media types | `selectMessageTemplates` for templates |
|
||||
| `mediatype.update` | Update webhook scripts/templates | Include full `script`, `parameters`, `message_templates` |
|
||||
| `action.get` | Get trigger actions | `selectOperations`, `selectRecoveryOperations` |
|
||||
| `action.update` | Update actions | Add `recovery_operations` |
|
||||
| `trigger.update` | Rename triggers, update descriptions | Use `description` (name) and `comments` (description text) |
|
||||
| `user.get` | Get users and their media | `selectMedias` for notification channels |
|
||||
| `template.get` | Get templates and their triggers | `selectTriggers` |
|
||||
|
||||
### Zabbix API Quirks (Zabbix 7.4)
|
||||
|
||||
- **Trigger `description` = trigger name** (not the description text). The description text is in `comments`.
|
||||
- **Status codes:** `0` = enabled, `1` = disabled for both media types and actions.
|
||||
- **`templateid` on host triggers:** If `0`, the trigger was created directly on the host (not inherited from a template). Non-zero = the parent trigger ID on the template.
|
||||
- **Severity levels:** `0`=Not classified, `1`=Information, `2`=Warning, `3`=Average, `4`=High, `5`=Disaster
|
||||
- **`{EVENT.VALUE}`:** `1` = problem, `0` = OK/recovery
|
||||
- **`{EVENT.NSEVERITY}`:** Numeric severity (0-5)
|
||||
- **Webhook scripts** run in Zabbix's built-in Duktape JavaScript engine (ES5 only — no `let`, `const`, arrow functions, template literals, `Array.find`, etc.)
|
||||
|
||||
---
|
||||
|
||||
## 4. Host Organization & Data Model
|
||||
|
||||
### Host Group Hierarchy
|
||||
|
||||
Every monitored host is assigned to multiple groups:
|
||||
|
||||
```
|
||||
Host: "Kuhn's Quality Foods"
|
||||
├── Clients/Kuhn's Quality Foods ← client identity
|
||||
├── ISP/Zito Media, L.P. ← internet provider
|
||||
└── Datto RMM Sites ← RMM platform category
|
||||
```
|
||||
|
||||
This triple-grouping enables:
|
||||
- **Per-client dashboards** — filter by `Clients/` group
|
||||
- **ISP outage detection** — if 3+ hosts on same ISP go down = likely ISP issue
|
||||
- **RMM correlation** — cross-reference Zabbix with Datto RMM agent status
|
||||
|
||||
### Client Groups (46)
|
||||
|
||||
| ID | Name |
|
||||
|----|------|
|
||||
| 84 | Clients/3 Rivers Express |
|
||||
| 90 | Clients/ABC Fire Extinguisher Inc |
|
||||
| 49 | Clients/ADM Signs |
|
||||
| 40 | Clients/Advanced Masonry |
|
||||
| 50 | Clients/Alabek Commercial Roofing Corp. |
|
||||
| 33 | Clients/All Saints Catholic Church |
|
||||
| 51 | Clients/Attica Hub/Seneca Publishing |
|
||||
| 42 | Clients/Bella Diamond LLC |
|
||||
| 28 | Clients/Blake Dentistry |
|
||||
| 85 | Clients/Bosak Eyecare & Optical |
|
||||
| 55 | Clients/Bridges Health Partners Services, LLC |
|
||||
| 37 | Clients/Brodaks |
|
||||
| 45 | Clients/Broker's Settlement Services, Inc. |
|
||||
| 43 | Clients/Brooks Diamonds |
|
||||
| 57 | Clients/Buffalo Glass Block |
|
||||
| 60 | Clients/CUMI America |
|
||||
| 61 | Clients/Chartiers Animal Hospital Ltd |
|
||||
| 58 | Clients/Cincinnati Glass Block |
|
||||
| 56 | Clients/Clista Electric Inc. |
|
||||
| 35 | Clients/ConnecTel, Inc. |
|
||||
| 26 | Clients/Finn Chiropractic Group |
|
||||
| 59 | Clients/Frew Plumbing, Heating, & Air |
|
||||
| 53 | Clients/Greco Gas |
|
||||
| 65 | Clients/Heart Prints Center for Early Education |
|
||||
| 47 | Clients/Hergenroeder, Rega, Ewing, & Kennedy, LLC |
|
||||
| 69 | Clients/Hynes Industries |
|
||||
| 48 | Clients/Insurance Restoration Consultants, Inc. |
|
||||
| 32 | Clients/Kuhn's Quality Foods |
|
||||
| 44 | Clients/Loss Prevention Services |
|
||||
| 80 | Clients/MDS Energy Development, LLC |
|
||||
| 88 | Clients/Marsico Financial Group, LLC |
|
||||
| 62 | Clients/Nordmann Roofing |
|
||||
| 68 | Clients/North Eastern Uniforms & Equipment Inc |
|
||||
| 38 | Clients/POH+W Architects |
|
||||
| 75 | Clients/Penn Energy Resources |
|
||||
| 86 | Clients/Pittsburgh Financial Consultants |
|
||||
| 87 | Clients/Premier Automation Holdings, Inc. |
|
||||
| 23 | Clients/Seubert and Associates |
|
||||
| 91 | Clients/Superior Distributing Co |
|
||||
| 30 | Clients/TK Plastics Company, Inc. |
|
||||
| 70 | Clients/Thoroughbred Construction Group |
|
||||
| 66 | Clients/Thrasher Group, Inc. |
|
||||
| 71 | Clients/Universal Plastics |
|
||||
| 73 | Clients/Universal Plastics Latrobe |
|
||||
| 54 | Clients/V-Systems |
|
||||
| 76 | Clients/Vorteq Coil Finishers |
|
||||
|
||||
### ISP Groups (24)
|
||||
|
||||
| ID | Name |
|
||||
|----|------|
|
||||
| 25 | ISP/AT&T Enterprises, LLC |
|
||||
| 27 | ISP/Armstrong |
|
||||
| 24 | ISP/Bigleaf Networks, Inc. |
|
||||
| 34 | ISP/Buckeye Cablevision, Inc. |
|
||||
| 52 | ISP/Charter Communications Inc |
|
||||
| 74 | ISP/Citizens Telecommunication Technologies, Inc |
|
||||
| 67 | ISP/CityNet |
|
||||
| 31 | ISP/Comcast Cable Communications, LLC |
|
||||
| 46 | ISP/DQE Communications LLC |
|
||||
| 63 | ISP/Expedient |
|
||||
| 77 | ISP/Fidium |
|
||||
| 89 | ISP/Frontier Communications of America, Inc. |
|
||||
| 82 | ISP/GeoLinks |
|
||||
| 78 | ISP/JACKSON ENERGY AUTHORITY |
|
||||
| 39 | ISP/Level 3 Parent, LLC |
|
||||
| 92 | ISP/Metalink Technologies, Inc. |
|
||||
| 72 | ISP/OneCleveland |
|
||||
| 64 | ISP/Space Exploration Technologies Corporation |
|
||||
| 83 | ISP/UPMC |
|
||||
| 79 | ISP/Ultimate Internet Access, Inc |
|
||||
| 36 | ISP/Verizon Business |
|
||||
| 81 | ISP/Wave Broadband |
|
||||
| 41 | ISP/Windstream Communications LLC |
|
||||
| 29 | ISP/Zito Media, L.P. |
|
||||
|
||||
### Other Groups
|
||||
|
||||
| ID | Name | Purpose |
|
||||
|----|------|---------|
|
||||
| 19 | Applications | Application-level monitoring |
|
||||
| 20 | Databases | Database servers |
|
||||
| 22 | Datto RMM Sites | Hosts also managed by Datto RMM |
|
||||
| 5 | Discovered hosts | Auto-discovered hosts |
|
||||
| 7 | Hypervisors | Virtualization hosts |
|
||||
| 2 | Linux servers | Linux-based systems |
|
||||
| 93 | Studio Imagine | Internal project |
|
||||
| 6 | Virtual machines | VMs |
|
||||
| 4 | Zabbix servers | Zabbix infrastructure |
|
||||
|
||||
### Host Pattern
|
||||
|
||||
All client-site hosts currently use the **ICMP Ping** template (templateid: 10564) for
|
||||
basic up/down monitoring. A typical host:
|
||||
|
||||
```
|
||||
Name: "Kuhn's Quality Foods"
|
||||
Status: enabled
|
||||
Groups: [Clients/Kuhn's Quality Foods, ISP/Zito Media L.P., Datto RMM Sites]
|
||||
Templates: [ICMP Ping]
|
||||
```
|
||||
|
||||
**77 total hosts** across the system (including Zabbix infrastructure hosts).
|
||||
|
||||
### Global Macros
|
||||
|
||||
| Macro | Value |
|
||||
|-------|-------|
|
||||
| `{$SNMP_COMMUNITY}` | `public` |
|
||||
|
||||
---
|
||||
|
||||
## 5. Notification Pipeline
|
||||
|
||||
### Current Architecture
|
||||
|
||||
```
|
||||
Zabbix Trigger Fires
|
||||
│
|
||||
▼
|
||||
Action: "Send ntfy to NOC" (actionid: 7)
|
||||
├── Problem → ntfy webhook (mediatypeid: 102)
|
||||
└── Recovery → ntfy webhook (same media type, different template)
|
||||
│
|
||||
▼
|
||||
ntfy Webhook Script (Duktape JS)
|
||||
├── Determines problem vs recovery (EVENT.VALUE)
|
||||
├── Sets 🔴 red_circle (problem) or 🟢 green_circle (recovery)
|
||||
├── Sets ntfy priority (high for problems, low for recovery)
|
||||
├── Adds Click URL → Zabbix event page
|
||||
├── For problems: queries Zabbix API for other active issues on same client
|
||||
└── POSTs to ntfy → http://ntfy/noc-alerts
|
||||
│
|
||||
▼
|
||||
ntfy Push Notification → user's phone
|
||||
```
|
||||
|
||||
### Action Configuration
|
||||
|
||||
**Action:** "Send ntfy to NOC" (actionid: 7)
|
||||
- **Trigger condition:** Severity >= High (conditiontype 4, operator 5, value 4)
|
||||
- **Operations:** Send message to user group "Executive" (usrgrpid: 14)
|
||||
- **Recovery operations:** Notify all involved (operationtype: 11)
|
||||
- **Uses default messages** (default_msg: 1) — pulls from media type templates
|
||||
|
||||
### Users & Media
|
||||
|
||||
| User | Role | ntfy Media |
|
||||
|------|------|------------|
|
||||
| Lorentz Hinrichsen (`lorentz@wulfconsulting.com`) | Super admin (3) | `noc-alerts` topic (enabled) |
|
||||
| Tom Carlin (`tom@wulfconsulting.com`) | Super admin (3) | *(removed — was duplicate to same topic)* |
|
||||
| Admin | Super admin (3) | *(none)* |
|
||||
| guest | Guest (4) | *(none)* |
|
||||
|
||||
### ntfy Webhook Script (Current)
|
||||
|
||||
The webhook script runs inside Zabbix Server's Duktape JS engine. It:
|
||||
|
||||
1. Parses parameters from Zabbix macros
|
||||
2. Determines if this is a problem or recovery event
|
||||
3. For problems: makes 2 internal Zabbix API calls to find related active issues for the same client
|
||||
4. Formats and sends the ntfy notification with appropriate tags, priority, and click URL
|
||||
|
||||
**Full script and parameters** are documented in `/opt/stacks/zabbix/notification-changes.md`
|
||||
|
||||
### Message Templates (Current)
|
||||
|
||||
| Event | ntfy Tag | Subject | Body |
|
||||
|-------|----------|---------|------|
|
||||
| Problem | 🔴 `red_circle` | `{HOST.NAME} — {EVENT.NAME}` | Trigger description + started time + duration + related issues |
|
||||
| Recovery | 🟢 `green_circle` | `Resolved: {HOST.NAME} — {EVENT.NAME}` | "Host is back online." + downtime + restored time |
|
||||
| Update | *(inherits)* | `Updated: {HOST.NAME} — {EVENT.NAME}` | Who updated + action + status |
|
||||
|
||||
### Notification Example
|
||||
|
||||
**Problem:**
|
||||
```
|
||||
🔴 Kuhn's Quality Foods — Host Unreachable
|
||||
|
||||
Host failed to respond to 3 consecutive ICMP ping
|
||||
requests. The site may be offline, the network path
|
||||
disrupted, or the device powered off.
|
||||
|
||||
Started: 2026.03.11 at 14:22:10
|
||||
Duration: 5m 30s
|
||||
|
||||
── Other active issues (Kuhn's Quality Foods) ──
|
||||
· High Packet Loss — Kuhn's SW-Core
|
||||
```
|
||||
|
||||
**Recovery:**
|
||||
```
|
||||
🟢 Resolved: Kuhn's Quality Foods — Host Unreachable
|
||||
|
||||
Host is back online.
|
||||
|
||||
Downtime: 22m 15s
|
||||
Restored: 2026.03.11 at 14:44:25
|
||||
```
|
||||
|
||||
### Available but Disabled Media Types
|
||||
|
||||
These exist in Zabbix but are disabled. Can be enabled if needed:
|
||||
- Email, Email (HTML), Gmail, Office365, SMS
|
||||
- Discord, Slack, MS Teams, MS Teams Workflow, Telegram
|
||||
- Jira, Jira Service Management, ServiceNow, Zendesk, PagerDuty, Opsgenie
|
||||
- Many others (40+ webhook integrations available)
|
||||
|
||||
---
|
||||
|
||||
## 6. Trigger Naming Conventions
|
||||
|
||||
### Design Principles (Established 2026-03-11)
|
||||
|
||||
Triggers were renamed from Zabbix defaults to be **executive-friendly**:
|
||||
|
||||
| Default Zabbix Name | Current Name | Severity |
|
||||
|---------------------|-------------|----------|
|
||||
| `ICMP Ping: Unavailable by ICMP ping` | **Host Unreachable** | High (4) |
|
||||
| `ICMP Ping: High ICMP ping loss` | **High Packet Loss** | Warning (2) |
|
||||
| `ICMP Ping: High ICMP ping response time` | **Slow Response Time** | Warning (2) |
|
||||
|
||||
### Naming Rules
|
||||
|
||||
1. **No technical jargon in trigger names** — use impact-based language
|
||||
2. **Trigger descriptions (comments field)** contain the technical detail: what the check does, thresholds, possible causes
|
||||
3. **Keep names short** — they appear in ntfy subjects, Teams cards, dashboards
|
||||
4. **No host name in trigger name** — `{HOST.NAME}` is added by the notification template
|
||||
|
||||
### Cisco Triggers (Not Yet Renamed)
|
||||
|
||||
The Cisco Catalyst SNMP templates have their own ICMP triggers that still use the old naming:
|
||||
- `Cisco Catalyst 3750V2-24FS: Unavailable by ICMP ping`
|
||||
- `Cisco Catalyst 3750V2-24FS: High ICMP ping loss`
|
||||
- etc.
|
||||
|
||||
These are on separate templates (not the "ICMP Ping" template) and have `templateid: 0` on host triggers. Renaming these requires updating each Cisco template individually.
|
||||
|
||||
---
|
||||
|
||||
## 7. Grafana Integration
|
||||
|
||||
### Plugin
|
||||
|
||||
- **alexanderzobnin-zabbix-app** v6.2.1 — connects directly to Zabbix API
|
||||
- Datasource URL (internal): `http://zabbix-frontend:8080/api_jsonrpc.php`
|
||||
|
||||
### Planned: MSP Executive Status Board
|
||||
|
||||
Design goal: at-a-glance dashboard for management showing:
|
||||
|
||||
| Panel | Type | Data Source |
|
||||
|-------|------|-------------|
|
||||
| Current problems by severity | Stat panels | Zabbix problems |
|
||||
| Problems per client | Bar gauge | Zabbix trigger groups |
|
||||
| Active problem list | Table | Zabbix problems |
|
||||
| SLA/uptime per client | Zabbix SLA panel | Zabbix SLA |
|
||||
| Host status grid | Status map plugin | Zabbix hosts |
|
||||
|
||||
Filter by `Clients/` host groups. Use Grafana variables for client selection dropdown.
|
||||
|
||||
---
|
||||
|
||||
## 8. Pulse Integration Architecture
|
||||
|
||||
### Overview
|
||||
|
||||
**Pulse** is the internal Node.js/TypeScript web app with API access to:
|
||||
- Zabbix (this system)
|
||||
- PSA (ticketing)
|
||||
- Datto RMM
|
||||
- Veeam (backups)
|
||||
- Microsoft Graph (Teams, email)
|
||||
- Zoom
|
||||
- Apprise (notification routing)
|
||||
|
||||
### Planned: Morning NOC Summary
|
||||
|
||||
Full architecture document: `/opt/stacks/zabbix/pulse-morning-summary-architecture.md`
|
||||
|
||||
**Key points:**
|
||||
- Scheduled job at 6:30 AM weekdays
|
||||
- Aggregates data from Zabbix + PSA + Veeam in parallel
|
||||
- Sends Teams adaptive card (direct via MS Graph), email/ntfy (via Apprise)
|
||||
- User-configurable delivery preferences
|
||||
- Killer feature: **unmatched alerts** — Zabbix problems with no PSA ticket
|
||||
|
||||
### Zabbix API Queries for Pulse
|
||||
|
||||
**Get all open problems with host/client context:**
|
||||
```typescript
|
||||
// 1. Get open problems
|
||||
const problems = await zabbixApi('problem.get', {
|
||||
recent: true,
|
||||
sortfield: ['eventid'],
|
||||
sortorder: 'DESC'
|
||||
});
|
||||
|
||||
// 2. Get host → client mapping (cache this)
|
||||
const hosts = await zabbixApi('host.get', {
|
||||
output: ['hostid', 'host', 'name'],
|
||||
selectHostGroups: ['groupid', 'name']
|
||||
});
|
||||
|
||||
// 3. Build client map
|
||||
const clientMap = {};
|
||||
for (const host of hosts) {
|
||||
const clientGroup = host.hostgroups.find(g => g.name.startsWith('Clients/'));
|
||||
if (clientGroup) {
|
||||
clientMap[host.hostid] = clientGroup.name.replace('Clients/', '');
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
**Get overnight resolved events:**
|
||||
```typescript
|
||||
const resolved = await zabbixApi('event.get', {
|
||||
source: 0,
|
||||
object: 0,
|
||||
value: 0, // recovery events only
|
||||
time_from: Math.floor(yesterday6pm.getTime() / 1000),
|
||||
time_to: Math.floor(today630am.getTime() / 1000),
|
||||
selectHosts: ['name'],
|
||||
selectRelatedObject: ['description'],
|
||||
sortfield: ['clock'],
|
||||
sortorder: 'DESC'
|
||||
});
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 9. Development Patterns & Gotchas
|
||||
|
||||
### Zabbix Webhook Script Constraints
|
||||
|
||||
The webhook script runs in **Duktape** (ES5 JavaScript engine):
|
||||
|
||||
**DO:**
|
||||
```javascript
|
||||
var x = 'hello'; // var only
|
||||
for (var i = 0; i < arr.length; i++) { } // classic for loops
|
||||
JSON.parse(), JSON.stringify() // available
|
||||
new HttpRequest() // Zabbix's HTTP client
|
||||
btoa() // Base64 encoding available
|
||||
```
|
||||
|
||||
**DON'T:**
|
||||
```javascript
|
||||
let x = 'hello'; // NO let/const
|
||||
const y = () => {}; // NO arrow functions
|
||||
`template ${literal}`; // NO template literals
|
||||
arr.find(x => x.id === 1); // NO Array.find/includes/map
|
||||
for (const x of arr) {} // NO for...of
|
||||
```
|
||||
|
||||
### HttpRequest in Webhooks
|
||||
|
||||
```javascript
|
||||
var req = new HttpRequest();
|
||||
req.addHeader('Content-Type: application/json');
|
||||
req.addHeader('Authorization: Bearer TOKEN');
|
||||
|
||||
var resp = req.post(url, body); // POST
|
||||
var resp = req.put(url, body); // PUT
|
||||
var resp = req.get(url); // GET
|
||||
|
||||
var status = req.getStatus(); // HTTP status code
|
||||
```
|
||||
|
||||
Multiple HttpRequest instances CAN be created in the same script (used for the related-problems enrichment that calls Zabbix API before calling ntfy).
|
||||
|
||||
### API Update Patterns
|
||||
|
||||
When updating a media type, you must include the FULL array for `parameters` and `message_templates` — they're **replaced entirely**, not merged.
|
||||
|
||||
```python
|
||||
# WRONG — this deletes all other parameters
|
||||
mediatype.update({ parameters: [{"name": "new_param", "value": "x"}] })
|
||||
|
||||
# RIGHT — include ALL parameters
|
||||
mediatype.update({ parameters: [
|
||||
{"name": "endpoint", "value": "http://ntfy/noc-alerts"},
|
||||
{"name": "username", "value": "monitoring"},
|
||||
# ... all existing params ...
|
||||
{"name": "new_param", "value": "x"}
|
||||
] })
|
||||
```
|
||||
|
||||
### Trigger Description vs Comments
|
||||
|
||||
This is confusing in the Zabbix API:
|
||||
- `description` field = **the trigger name** (what you see in the UI)
|
||||
- `comments` field = **the description text** (the explanation paragraph)
|
||||
|
||||
```json
|
||||
{
|
||||
"triggerid": "23176",
|
||||
"description": "Host Unreachable", // ← this is the NAME
|
||||
"comments": "Host failed to respond to..." // ← this is the DESCRIPTION
|
||||
}
|
||||
```
|
||||
|
||||
### Host Group Filtering Pattern
|
||||
|
||||
To get the client name for a host:
|
||||
|
||||
```javascript
|
||||
// Get host groups
|
||||
var groups = hostData.hostgroups;
|
||||
var clientName = null;
|
||||
for (var i = 0; i < groups.length; i++) {
|
||||
if (groups[i].name.indexOf("Clients/") === 0) {
|
||||
clientName = groups[i].name.replace("Clients/", "");
|
||||
break;
|
||||
}
|
||||
}
|
||||
// clientName = "Kuhn's Quality Foods"
|
||||
```
|
||||
|
||||
### Docker Exec for API Calls
|
||||
|
||||
Since the Zabbix frontend doesn't have ports mapped to the host, API calls from the host must go through `docker exec`:
|
||||
|
||||
```bash
|
||||
docker exec zabbix-frontend curl -s -X POST \
|
||||
"http://localhost:8080/api_jsonrpc.php" \
|
||||
-H "Content-Type: application/json" \
|
||||
-H "Authorization: Bearer <token>" \
|
||||
-d '{"jsonrpc":"2.0","method":"...","params":{...},"id":1}'
|
||||
```
|
||||
|
||||
Or use Python for complex payloads (escaping JSON in bash is fragile):
|
||||
|
||||
```python
|
||||
import json, subprocess
|
||||
|
||||
payload = json.dumps({...})
|
||||
result = subprocess.run(
|
||||
["docker", "exec", "zabbix-frontend", "curl", "-s", "-X", "POST",
|
||||
"http://localhost:8080/api_jsonrpc.php",
|
||||
"-H", "Content-Type: application/json",
|
||||
"-H", f"Authorization: Bearer {TOKEN}",
|
||||
"-d", payload],
|
||||
capture_output=True, text=True
|
||||
)
|
||||
data = json.loads(result.stdout)
|
||||
```
|
||||
|
||||
### External API Access (from Pulse or other servers)
|
||||
|
||||
```typescript
|
||||
const response = await fetch('https://zabbix.wulfconsulting.cloud/api_jsonrpc.php', {
|
||||
method: 'POST',
|
||||
headers: {
|
||||
'Content-Type': 'application/json',
|
||||
'Authorization': `Bearer ${ZABBIX_API_TOKEN}`
|
||||
},
|
||||
body: JSON.stringify({
|
||||
jsonrpc: '2.0',
|
||||
method: 'problem.get',
|
||||
params: { recent: true },
|
||||
id: 1
|
||||
})
|
||||
});
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 10. API Cookbook
|
||||
|
||||
### Get All Active Problems
|
||||
|
||||
```json
|
||||
{
|
||||
"method": "problem.get",
|
||||
"params": {
|
||||
"output": "extend",
|
||||
"recent": true,
|
||||
"sortfield": ["eventid"],
|
||||
"sortorder": "DESC"
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
### Get Active Problems for a Specific Client
|
||||
|
||||
```json
|
||||
{
|
||||
"method": "trigger.get",
|
||||
"params": {
|
||||
"output": ["triggerid", "description", "lastchange", "priority"],
|
||||
"groupids": ["32"],
|
||||
"only_true": true,
|
||||
"selectHosts": ["name"],
|
||||
"skipDependent": true
|
||||
}
|
||||
}
|
||||
```
|
||||
*(groupid 32 = Clients/Kuhn's Quality Foods)*
|
||||
|
||||
### Get All Hosts with Client + ISP Mapping
|
||||
|
||||
```json
|
||||
{
|
||||
"method": "host.get",
|
||||
"params": {
|
||||
"output": ["hostid", "host", "name", "status"],
|
||||
"selectHostGroups": ["groupid", "name"],
|
||||
"selectParentTemplates": ["templateid", "name"]
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
### Get Recovery Events in a Time Window
|
||||
|
||||
```json
|
||||
{
|
||||
"method": "event.get",
|
||||
"params": {
|
||||
"source": 0,
|
||||
"object": 0,
|
||||
"value": 0,
|
||||
"time_from": 1741647600,
|
||||
"time_to": 1741692600,
|
||||
"selectHosts": ["name"],
|
||||
"selectRelatedObject": ["description"],
|
||||
"sortfield": ["clock"],
|
||||
"sortorder": "DESC"
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
### Update a Trigger Name + Description
|
||||
|
||||
```json
|
||||
{
|
||||
"method": "trigger.update",
|
||||
"params": {
|
||||
"triggerid": "23176",
|
||||
"description": "Host Unreachable",
|
||||
"comments": "Host failed to respond to 3 consecutive ICMP ping requests."
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
### Get Media Type with Full Details
|
||||
|
||||
```json
|
||||
{
|
||||
"method": "mediatype.get",
|
||||
"params": {
|
||||
"output": "extend",
|
||||
"selectMessageTemplates": "extend",
|
||||
"mediatypeids": ["102"]
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
### Test ntfy Notification Manually
|
||||
|
||||
```bash
|
||||
docker exec zabbix-frontend curl -s -X PUT \
|
||||
-H "Authorization: Basic $(echo -n 'monitoring:GBigt1231#' | base64)" \
|
||||
-H "Title: Test Notification" \
|
||||
-H "Priority: 3" \
|
||||
-H "Tags: red_circle" \
|
||||
"http://ntfy/noc-alerts" \
|
||||
-d "This is a test notification from Zabbix"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Related Documentation
|
||||
|
||||
- **Notification changes (before/after + restoration):** `/opt/stacks/zabbix/notification-changes.md`
|
||||
- **Pulse morning summary architecture:** `/opt/stacks/zabbix/pulse-morning-summary-architecture.md`
|
||||
- **Docker Compose:** `/opt/stacks/zabbix/compose.yml`
|
||||
- **Zabbix SSO setup:** `/opt/stacks/zabbix/ENTRA-AUTHENTIK-ZABBIX-SSO.md`
|
||||
- **Zabbix 7.4 API docs:** https://www.zabbix.com/documentation/7.4/en/manual/api
|
||||
Loading…
Add table
Add a link
Reference in a new issue