feat: Veeam RPO analysis, comparison, ticket analysis + company teams table
- Add Veeam RPO analysis page (/veeam-analysis) and comparison page (/veeam-comparison) - Add API routes: /api/veeam/rpo-analyze, rpo-comparison, rpo-offline-log, ticket-analysis - Add veeam-rpo-service.ts enhancements (RPO logic, offline detection, comparison) - Add veeam-analysis-state.ts and rmm-device-resolver.ts services - Add migrations 065-068: company_teams, veeam_rpo_offline_log, rpo_comparison_tables, veeam_ticket_analysis - Add backup-status page updates and nav links for new Veeam pages - Add scripts: deactivate-cis-for-inactive-companies, workstation category updates - Add docs: mimecast-api-guide, veeam-backup-alerting-recommendation, workstation-backup-overview, ticket-analyzer-prompt - Minor: webhook-service, entity-sync, entity-mapper, sync-helpers, sync.ts, middleware.ts updates
This commit is contained in:
parent
07067bef19
commit
ea3471d38d
36 changed files with 5604 additions and 217 deletions
341
docs/mimecast-api-guide.md
Normal file
341
docs/mimecast-api-guide.md
Normal file
|
|
@ -0,0 +1,341 @@
|
|||
# Mimecast API Guide — Threat Dashboard Integration
|
||||
|
||||
## Overview
|
||||
|
||||
This guide covers authentication and data access for the Mimecast API v2, oriented toward building a threat dashboard in a client portal. All examples use OAuth 2.0 (the current recommended approach).
|
||||
|
||||
---
|
||||
|
||||
## 1. Authentication (OAuth 2.0 — Client Credentials)
|
||||
|
||||
### Setup in Mimecast Console
|
||||
|
||||
1. Navigate to **Administration > Services > API and Platform Integrations** (or Integrations Hub in newer tenants)
|
||||
2. Create a new API 2.0 application
|
||||
3. Assign only the permission scopes you need (see §3 below)
|
||||
4. Record the `client_id` and `client_secret`
|
||||
|
||||
### Getting an Access Token
|
||||
|
||||
```http
|
||||
POST https://api.services.mimecast.com/oauth/token
|
||||
Content-Type: application/x-www-form-urlencoded
|
||||
|
||||
grant_type=client_credentials&client_id=YOUR_CLIENT_ID&client_secret=YOUR_CLIENT_SECRET
|
||||
```
|
||||
|
||||
```json
|
||||
{
|
||||
"access_token": "eyJ...",
|
||||
"token_type": "Bearer",
|
||||
"expires_in": 3600
|
||||
}
|
||||
```
|
||||
|
||||
Cache the token and refresh before expiry. Plan for a 401 → refresh → retry flow.
|
||||
|
||||
### Using the Token
|
||||
|
||||
All API requests:
|
||||
|
||||
```http
|
||||
Authorization: Bearer {access_token}
|
||||
Content-Type: application/json
|
||||
x-mc-req-id: {uuid-per-request}
|
||||
```
|
||||
|
||||
> `x-mc-req-id` should be a fresh UUID per call — used for idempotency and support tracing.
|
||||
|
||||
### Region Discovery
|
||||
|
||||
Mimecast routes requests by region. If you get unexpected 401s or 404s, call the discovery endpoint first to resolve the correct base URL for a given account:
|
||||
|
||||
```http
|
||||
GET https://api.services.mimecast.com/oauth/discover
|
||||
Authorization: Bearer {access_token}
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 2. Request / Response Structure
|
||||
|
||||
Nearly all Mimecast endpoints use `POST`, even for reads. The body follows a consistent envelope:
|
||||
|
||||
```json
|
||||
{
|
||||
"data": [
|
||||
{ /* endpoint-specific params */ }
|
||||
],
|
||||
"meta": {
|
||||
"pagination": {
|
||||
"pageToken": "next-page-token-from-previous-response"
|
||||
}
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
Response:
|
||||
|
||||
```json
|
||||
{
|
||||
"data": [ /* results */ ],
|
||||
"meta": { /* status info */ },
|
||||
"fail": [ /* per-item errors */ ],
|
||||
"pagination": {
|
||||
"next": "token_for_next_page",
|
||||
"previous": "token_for_prev_page"
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
**Important**: HTTP 200 means the request was received. Always check `fail` array and `meta.status` for functional errors — a 200 can still contain failures.
|
||||
|
||||
---
|
||||
|
||||
## 3. Threat Dashboard Endpoints
|
||||
|
||||
### A. SIEM Logs (Core Email Threat Stream)
|
||||
|
||||
The broadest threat feed — covers all MTA events including blocked senders, malicious URLs, attachment verdicts, spoofing attempts.
|
||||
|
||||
```http
|
||||
POST /api/audit/get-siem-logs
|
||||
|
||||
{
|
||||
"data": [{
|
||||
"type": "MTA",
|
||||
"fileFormat": "json",
|
||||
"compress": false,
|
||||
"token": ""
|
||||
}]
|
||||
}
|
||||
```
|
||||
|
||||
- **Returns**: `application/octet-stream` — newline-delimited JSON events
|
||||
- **Token-based cursor**: response includes `mc-siem-token` header — pass this as `token` on the next call to get the next batch
|
||||
- **Retention**: 7 days only — pull at least every 6 hours, ideally hourly
|
||||
- **Update frequency**: new batches available every ~30 minutes
|
||||
- **Rate limit**: 300 calls/hour for this endpoint
|
||||
- **Required scope**: `Gateway | Tracking | Read`
|
||||
- **Prerequisite**: Enable **Enhanced Logging** in Administration Console > Account Settings
|
||||
|
||||
Key fields in each event:
|
||||
| Field | Description |
|
||||
|-------|-------------|
|
||||
| `Act` | Action taken (Acc=accepted, Rej=rejected, Hld=held) |
|
||||
| `ThreatDictionary` | Threat categories triggered |
|
||||
| `SpamInfo` | Spam score and verdict |
|
||||
| `URL` | Rewritten URL if TTP URL protection triggered |
|
||||
| `FileName` | Attachment name if scanned |
|
||||
| `SHA256` | Hash of attachment |
|
||||
| `SenderIP` | Originating IP |
|
||||
|
||||
---
|
||||
|
||||
### B. TTP Attachment Protection Logs
|
||||
|
||||
Sandboxing verdicts for email attachments — malware, ransomware, etc.
|
||||
|
||||
```http
|
||||
POST /api/ttp/attachment/get-logs
|
||||
|
||||
{
|
||||
"data": [{
|
||||
"from": "2026-01-01T00:00:00+0000",
|
||||
"to": "2026-01-02T00:00:00+0000",
|
||||
"route": "inbound",
|
||||
"result": "malicious",
|
||||
"pageSize": 100
|
||||
}]
|
||||
}
|
||||
```
|
||||
|
||||
- `route`: `inbound` | `outbound` | `internal` | `all`
|
||||
- `result`: `safe` | `malicious` | `timeout` | `error` | `unsafe` | `all`
|
||||
- **Required scope**: `Monitoring | Attachment Protection | Read`
|
||||
|
||||
Key response fields:
|
||||
| Field | Description |
|
||||
|-------|-------------|
|
||||
| `result` | Sandbox verdict |
|
||||
| `fileName` | Original attachment filename |
|
||||
| `sha256` | File hash (pivot to threat intel) |
|
||||
| `senderAddress` | From address |
|
||||
| `recipientAddress` | Target mailbox |
|
||||
| `actionTriggered` | What Mimecast did (block, sandbox, etc.) |
|
||||
| `date` | ISO 8601 timestamp |
|
||||
|
||||
---
|
||||
|
||||
### C. TTP URL Protection — Managed URL List
|
||||
|
||||
Retrieve blocked/tracked URLs from the managed threat list:
|
||||
|
||||
```http
|
||||
GET /api/ttp/url/get-managed-url
|
||||
```
|
||||
|
||||
Or decode a rewritten Mimecast URL back to the original for IOC extraction:
|
||||
|
||||
```http
|
||||
POST /api/ttp/url/decode-url
|
||||
|
||||
{
|
||||
"data": [{
|
||||
"url": "https://protect-eu.mimecast.com/s/ABC..."
|
||||
}]
|
||||
}
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### D. Audit Events (Security Configuration Changes)
|
||||
|
||||
Policy changes, admin actions — useful for detecting misconfigurations or insider threats:
|
||||
|
||||
```http
|
||||
POST /api/audit/get-audit-events
|
||||
|
||||
{
|
||||
"data": [{
|
||||
"startDateTime": "2026-01-01T00:00:00+0000",
|
||||
"endDateTime": "2026-01-02T00:00:00+0000",
|
||||
"categories": ["policy", "account"]
|
||||
}]
|
||||
}
|
||||
```
|
||||
|
||||
- **Required scope**: `Account | Logs | Read`
|
||||
|
||||
---
|
||||
|
||||
## 4. Pagination
|
||||
|
||||
All paginated endpoints use cursor tokens:
|
||||
|
||||
```typescript
|
||||
async function fetchAllPages(endpoint: string, basePayload: object): Promise<any[]> {
|
||||
const results: any[] = [];
|
||||
let pageToken: string | undefined;
|
||||
|
||||
do {
|
||||
const body = {
|
||||
...basePayload,
|
||||
meta: pageToken ? { pagination: { pageToken } } : undefined
|
||||
};
|
||||
|
||||
const res = await fetch(`https://api.services.mimecast.com${endpoint}`, {
|
||||
method: 'POST',
|
||||
headers: {
|
||||
Authorization: `Bearer ${token}`,
|
||||
'Content-Type': 'application/json',
|
||||
'x-mc-req-id': crypto.randomUUID()
|
||||
},
|
||||
body: JSON.stringify(body)
|
||||
});
|
||||
|
||||
const json = await res.json();
|
||||
results.push(...(json.data ?? []));
|
||||
pageToken = json.pagination?.next;
|
||||
} while (pageToken);
|
||||
|
||||
return results;
|
||||
}
|
||||
```
|
||||
|
||||
Default page size: 100. Maximum: 500. Set `pageSize` in the `data` array params.
|
||||
|
||||
---
|
||||
|
||||
## 5. Rate Limiting
|
||||
|
||||
| Header | Meaning |
|
||||
|--------|---------|
|
||||
| `X-RateLimit-Limit` | Total requests allowed in window |
|
||||
| `X-RateLimit-Remaining` | Calls left |
|
||||
| `X-RateLimit-Reset` | Milliseconds until reset |
|
||||
|
||||
On HTTP 429, back off exponentially. SIEM logs endpoint has a hard limit of **300 calls/hour**.
|
||||
|
||||
```typescript
|
||||
async function fetchWithRetry(url: string, options: RequestInit, retries = 3): Promise<Response> {
|
||||
const res = await fetch(url, options);
|
||||
if (res.status === 429 && retries > 0) {
|
||||
const resetMs = parseInt(res.headers.get('X-RateLimit-Reset') ?? '5000');
|
||||
await new Promise(r => setTimeout(r, resetMs));
|
||||
return fetchWithRetry(url, options, retries - 1);
|
||||
}
|
||||
return res;
|
||||
}
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 6. Multi-Tenant (MSP) Access
|
||||
|
||||
When accessing multiple client accounts with a single credential set, pass `accountCode` in the meta:
|
||||
|
||||
```json
|
||||
{
|
||||
"data": [{ /* params */ }],
|
||||
"meta": {
|
||||
"accountCode": "CLIENT_ACCOUNT_CODE"
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
Each Mimecast tenant has a unique account code visible in the administration console.
|
||||
|
||||
---
|
||||
|
||||
## 7. Required Permission Scopes by Endpoint
|
||||
|
||||
| Endpoint | Required Scope |
|
||||
|----------|---------------|
|
||||
| SIEM Logs | `Gateway > Tracking > Read` |
|
||||
| Audit Events | `Account > Logs > Read` |
|
||||
| TTP Attachment Logs | `Monitoring > Attachment Protection > Read` |
|
||||
| TTP URL Management | `Gateway > Policies > Read` |
|
||||
|
||||
Request only the scopes needed. Grant least privilege per API application.
|
||||
|
||||
---
|
||||
|
||||
## 8. Dashboard Data Model Suggestion
|
||||
|
||||
For a threat dashboard showing a summary per client:
|
||||
|
||||
```sql
|
||||
-- Suggested local cache table
|
||||
CREATE TABLE mimecast_threat_events (
|
||||
id TEXT PRIMARY KEY,
|
||||
account_code TEXT NOT NULL,
|
||||
event_type TEXT NOT NULL, -- 'siem_mta' | 'ttp_attachment' | 'ttp_url'
|
||||
event_ts TIMESTAMPTZ NOT NULL,
|
||||
sender TEXT,
|
||||
recipient TEXT,
|
||||
sender_ip INET,
|
||||
threat_type TEXT, -- malicious | spam | spoofing | etc.
|
||||
action TEXT, -- blocked | sandboxed | delivered
|
||||
sha256 TEXT,
|
||||
url TEXT,
|
||||
raw JSONB,
|
||||
synced_at TIMESTAMPTZ DEFAULT now()
|
||||
);
|
||||
|
||||
CREATE INDEX ON mimecast_threat_events (account_code, event_ts DESC);
|
||||
CREATE INDEX ON mimecast_threat_events (event_type, threat_type);
|
||||
```
|
||||
|
||||
Sync strategy: poll SIEM logs hourly using stored `mc-siem-token` per account. Store the cursor token per `account_code` in a config table.
|
||||
|
||||
---
|
||||
|
||||
## 9. Gotchas
|
||||
|
||||
- **Enhanced Logging must be enabled** in the Mimecast console before SIEM data appears — without it the endpoint returns empty.
|
||||
- **SIEM uses a streaming cursor**, not date ranges — don't skip token management or you'll re-read old events.
|
||||
- **HTTP 200 ≠ success** — always check `fail[]` in the response body.
|
||||
- **POST for reads** — don't expect REST conventions; almost everything is POST.
|
||||
- **Token expiry** — implement proactive refresh (check `expires_in`, refresh at 80% of TTL).
|
||||
- **Region routing** — if a client's tenant is in EU/AU, the base URL differs; use the discovery endpoint per account.
|
||||
Loading…
Add table
Add a link
Reference in a new issue