wulf-pulse/docs/mimecast-api-guide.md

# Mimecast API Guide — Threat Dashboard Integration

## Overview

This guide covers authentication and data access for the Mimecast API v2, oriented toward building a threat dashboard in a client portal. All examples use OAuth 2.0 (the current recommended approach).

---

## 1. Authentication (OAuth 2.0 — Client Credentials)

### Setup in Mimecast Console

1. Navigate to **Administration > Services > API and Platform Integrations** (or Integrations Hub in newer tenants)
2. Create a new API 2.0 application
3. Assign only the permission scopes you need (see §3 below)
4. Record the `client_id` and `client_secret`

### Getting an Access Token

```http
POST https://api.services.mimecast.com/oauth/token
Content-Type: application/x-www-form-urlencoded

grant_type=client_credentials&client_id=YOUR_CLIENT_ID&client_secret=YOUR_CLIENT_SECRET
```

```json
{
  "access_token": "eyJ...",
  "token_type": "Bearer",
  "expires_in": 3600
}
```

Cache the token and refresh before expiry. Plan for a 401 → refresh → retry flow.

### Using the Token

All API requests:

```http
Authorization: Bearer {access_token}
Content-Type: application/json
x-mc-req-id: {uuid-per-request}
```

> `x-mc-req-id` should be a fresh UUID per call — used for idempotency and support tracing.

### Region Discovery

Mimecast routes requests by region. If you get unexpected 401s or 404s, call the discovery endpoint first to resolve the correct base URL for a given account:

```http
GET https://api.services.mimecast.com/oauth/discover
Authorization: Bearer {access_token}
```

---

## 2. Request / Response Structure

Nearly all Mimecast endpoints use `POST`, even for reads. The body follows a consistent envelope:

```json
{
  "data": [
    { /* endpoint-specific params */ }
  ],
  "meta": {
    "pagination": {
      "pageToken": "next-page-token-from-previous-response"
    }
  }
}
```

Response:

```json
{
  "data": [ /* results */ ],
  "meta": { /* status info */ },
  "fail": [ /* per-item errors */ ],
  "pagination": {
    "next": "token_for_next_page",
    "previous": "token_for_prev_page"
  }
}
```

**Important**: HTTP 200 means the request was received. Always check `fail` array and `meta.status` for functional errors — a 200 can still contain failures.

---

## 3. Threat Dashboard Endpoints

### A. SIEM Logs (Core Email Threat Stream)

The broadest threat feed — covers all MTA events including blocked senders, malicious URLs, attachment verdicts, spoofing attempts.

```http
POST /api/audit/get-siem-logs

{
  "data": [{
    "type": "MTA",
    "fileFormat": "json",
    "compress": false,
    "token": ""
  }]
}
```

- **Returns**: `application/octet-stream` — newline-delimited JSON events
- **Token-based cursor**: response includes `mc-siem-token` header — pass this as `token` on the next call to get the next batch
- **Retention**: 7 days only — pull at least every 6 hours, ideally hourly
- **Update frequency**: new batches available every ~30 minutes
- **Rate limit**: 300 calls/hour for this endpoint
- **Required scope**: `Gateway | Tracking | Read`
- **Prerequisite**: Enable **Enhanced Logging** in Administration Console > Account Settings

Key fields in each event:
| Field | Description |
|-------|-------------|
| `Act` | Action taken (Acc=accepted, Rej=rejected, Hld=held) |
| `ThreatDictionary` | Threat categories triggered |
| `SpamInfo` | Spam score and verdict |
| `URL` | Rewritten URL if TTP URL protection triggered |
| `FileName` | Attachment name if scanned |
| `SHA256` | Hash of attachment |
| `SenderIP` | Originating IP |

---

### B. TTP Attachment Protection Logs

Sandboxing verdicts for email attachments — malware, ransomware, etc.

```http
POST /api/ttp/attachment/get-logs

{
  "data": [{
    "from": "2026-01-01T00:00:00+0000",
    "to": "2026-01-02T00:00:00+0000",
    "route": "inbound",
    "result": "malicious",
    "pageSize": 100
  }]
}
```

- `route`: `inbound` | `outbound` | `internal` | `all`
- `result`: `safe` | `malicious` | `timeout` | `error` | `unsafe` | `all`
- **Required scope**: `Monitoring | Attachment Protection | Read`

Key response fields:
| Field | Description |
|-------|-------------|
| `result` | Sandbox verdict |
| `fileName` | Original attachment filename |
| `sha256` | File hash (pivot to threat intel) |
| `senderAddress` | From address |
| `recipientAddress` | Target mailbox |
| `actionTriggered` | What Mimecast did (block, sandbox, etc.) |
| `date` | ISO 8601 timestamp |

---

### C. TTP URL Protection — Managed URL List

Retrieve blocked/tracked URLs from the managed threat list:

```http
GET /api/ttp/url/get-managed-url
```

Or decode a rewritten Mimecast URL back to the original for IOC extraction:

```http
POST /api/ttp/url/decode-url

{
  "data": [{
    "url": "https://protect-eu.mimecast.com/s/ABC..."
  }]
}
```

---

### D. Audit Events (Security Configuration Changes)

Policy changes, admin actions — useful for detecting misconfigurations or insider threats:

```http
POST /api/audit/get-audit-events

{
  "data": [{
    "startDateTime": "2026-01-01T00:00:00+0000",
    "endDateTime": "2026-01-02T00:00:00+0000",
    "categories": ["policy", "account"]
  }]
}
```

- **Required scope**: `Account | Logs | Read`

---

### E. Held Message Queue (Blast Radius / Phishing Triage)

Used by Pulse's phishing-triage Blast Radius panel (`lib/services/mimecast-blast-radius.ts`) to check what's sitting in a recipient's spam/policy hold queue.

```http
POST /api/gateway/get-hold-message-list

{
  "data": [{
    "admin": true,
    "start": "2026-07-16T18:10:55+0000",
    "end": "2026-07-18T09:38:00+0000",
    "searchBy": {
      "fieldName": "recipient",
      "value": "user@example.com"
    }
  }]
}
```

- `admin`: `true` returns held messages for all recipients (the client falls back to omitting it on a 403 for tenants lacking that permission); `false`/omitted returns only the authenticated user's own held mail.
- `start`/`end`: date range, same `yyyy-MM-dd'T'HH:mm:ssZ` format as every other endpoint in this guide — confirmed live in 2026-07 against [Mimecast's official docs](https://developer.services.mimecast.com/docs/cloudgateway/1/routes/api/gateway/get-hold-message-list/post) and a real tenant; easy to miss since without it you query a recipient's ENTIRE hold queue with no date bound.
- `searchBy.fieldName`: exactly one of `all`, `subject`, `sender`, `recipient`, `reasonCode`, `senderIP` per call — `searchBy` is a single object, not an array, so `sender` + `recipient` cannot be combined in one request.
- **Required permission**: `Account | Dashboard | Read`
- **Pagination**: `pageSize` in the request is ignored — always returns 10 rows; paginate via `meta.pagination.next`.

Key response fields:
| Field | Description |
|-------|-------------|
| `from.emailAddress` / `fromHeader.emailAddress` | Envelope vs. header sender — use `fromHeader` when present |
| `to.emailAddress` | Recipient |
| `subject` | Message subject |
| `reason` / `reasonCode` | Why it's held (e.g. "Message Hold Applied - Spam Signature policy", "... - DMARC Quarantine") |
| `dateReceived` | ISO 8601 timestamp |
| `route` | `INBOUND` \| `OUTBOUND` \| `INTERNAL` \| `EXTERNAL` |

Official docs: https://developer.services.mimecast.com/docs/cloudgateway/1/routes/api/gateway/get-hold-message-list/post

---

## 4. Pagination

All paginated endpoints use cursor tokens:

```typescript
async function fetchAllPages(endpoint: string, basePayload: object): Promise<any[]> {
  const results: any[] = [];
  let pageToken: string | undefined;

  do {
    const body = {
      ...basePayload,
      meta: pageToken ? { pagination: { pageToken } } : undefined
    };

    const res = await fetch(`https://api.services.mimecast.com${endpoint}`, {
      method: 'POST',
      headers: {
        Authorization: `Bearer ${token}`,
        'Content-Type': 'application/json',
        'x-mc-req-id': crypto.randomUUID()
      },
      body: JSON.stringify(body)
    });

    const json = await res.json();
    results.push(...(json.data ?? []));
    pageToken = json.pagination?.next;
  } while (pageToken);

  return results;
}
```

Default page size: 100. Maximum: 500. Set `pageSize` in the `data` array params.

---

## 5. Rate Limiting

| Header | Meaning |
|--------|---------|
| `X-RateLimit-Limit` | Total requests allowed in window |
| `X-RateLimit-Remaining` | Calls left |
| `X-RateLimit-Reset` | Milliseconds until reset |

On HTTP 429, back off exponentially. SIEM logs endpoint has a hard limit of **300 calls/hour**.

```typescript
async function fetchWithRetry(url: string, options: RequestInit, retries = 3): Promise<Response> {
  const res = await fetch(url, options);
  if (res.status === 429 && retries > 0) {
    const resetMs = parseInt(res.headers.get('X-RateLimit-Reset') ?? '5000');
    await new Promise(r => setTimeout(r, resetMs));
    return fetchWithRetry(url, options, retries - 1);
  }
  return res;
}
```

---

## 6. Multi-Tenant (MSP) Access

When accessing multiple client accounts with a single credential set, pass `accountCode` in the meta:

```json
{
  "data": [{ /* params */ }],
  "meta": {
    "accountCode": "CLIENT_ACCOUNT_CODE"
  }
}
```

Each Mimecast tenant has a unique account code visible in the administration console.

---

## 7. Required Permission Scopes by Endpoint

| Endpoint | Required Scope |
|----------|---------------|
| SIEM Logs | `Gateway > Tracking > Read` |
| Audit Events | `Account > Logs > Read` |
| TTP Attachment Logs | `Monitoring > Attachment Protection > Read` |
| TTP URL Management | `Gateway > Policies > Read` |

Request only the scopes needed. Grant least privilege per API application.

---

## 8. Dashboard Data Model Suggestion

For a threat dashboard showing a summary per client:

```sql
-- Suggested local cache table
CREATE TABLE mimecast_threat_events (
  id            TEXT PRIMARY KEY,
  account_code  TEXT NOT NULL,
  event_type    TEXT NOT NULL,  -- 'siem_mta' | 'ttp_attachment' | 'ttp_url'
  event_ts      TIMESTAMPTZ NOT NULL,
  sender        TEXT,
  recipient     TEXT,
  sender_ip     INET,
  threat_type   TEXT,           -- malicious | spam | spoofing | etc.
  action        TEXT,           -- blocked | sandboxed | delivered
  sha256        TEXT,
  url           TEXT,
  raw           JSONB,
  synced_at     TIMESTAMPTZ DEFAULT now()
);

CREATE INDEX ON mimecast_threat_events (account_code, event_ts DESC);
CREATE INDEX ON mimecast_threat_events (event_type, threat_type);
```

Sync strategy: poll SIEM logs hourly using stored `mc-siem-token` per account. Store the cursor token per `account_code` in a config table.

---

## 9. Gotchas

- **Enhanced Logging must be enabled** in the Mimecast console before SIEM data appears — without it the endpoint returns empty.
- **SIEM uses a streaming cursor**, not date ranges — don't skip token management or you'll re-read old events.
- **HTTP 200 ≠ success** — always check `fail[]` in the response body.
- **POST for reads** — don't expect REST conventions; almost everything is POST.
- **Token expiry** — implement proactive refresh (check `expires_in`, refresh at 80% of TTL).
- **Region routing** — if a client's tenant is in EU/AU, the base URL differs; use the discovery endpoint per account.
- **Held-message queue has no combined filter** — `get-hold-message-list`'s `searchBy` only accepts one field at a time (`recipient`, `sender`, `subject`, `reasonCode`, `senderIP`, or `all`). Querying by recipient alone returns their entire hold queue for any sender/reason unless you also pass `start`/`end` — and even then, an unrelated held message can coincidentally fall in the same window. Always cross-check the held row's sender against the message you actually care about before treating it as related (this exact false positive surfaced on a real ticket and is now guarded in `lib/services/mimecast-blast-radius.ts`'s `domainsMatch()` check).