Initial commit: Howl M365 email management daemon
Full implementation of the Howl email triage system: - Microsoft Graph API integration with MSAL auth (client-credentials and delegated modes) - Claude LLM classification via tool use for structured output - PostgreSQL database with customers, vendors, whitelist, and email_log tables - Alembic migration for full schema - APScheduler daemon with graceful shutdown - Typer CLI (run, dry-run, status commands) - Business rule classifier with overrides (whitelist protection, low-confidence fallback) - Action executor (move to folders, flag, escalate with webhook) - 35 passing unit tests - README, SETUP, and NEXT_STEPS documentation Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
This commit is contained in:
commit
3bfda9e585
39 changed files with 3656 additions and 0 deletions
90
NEXT_STEPS.md
Normal file
90
NEXT_STEPS.md
Normal file
|
|
@ -0,0 +1,90 @@
|
|||
# Next Steps & Roadmap
|
||||
|
||||
## Immediate (before going live)
|
||||
|
||||
### 1. Azure app registration
|
||||
Follow `SETUP.md` to register an app in Azure AD and grant the required Graph API permissions. This is a prerequisite for any real email processing.
|
||||
|
||||
### 2. Seed your database
|
||||
Populate `customers`, `customer_emails`, `vendors`, `vendor_emails`, and `whitelist` with your real contact data before running the daemon. The more complete this data is, the better the LLM's classifications will be.
|
||||
|
||||
### 3. Run a dry-run validation
|
||||
```bash
|
||||
DRY_RUN=true howl dry-run --limit 20
|
||||
```
|
||||
Review the `email_log` table. Check that:
|
||||
- Known customers/vendors are being matched correctly
|
||||
- LLM classifications feel accurate for your email types
|
||||
- Confidence scores make sense (low scores = good candidates for whitelist/notes improvements)
|
||||
|
||||
### 4. Tune confidence threshold
|
||||
The default `LLM_CONFIDENCE_THRESHOLD=0.60` sends uncertain emails to `Needs Review`. Monitor the `Needs Review` folder for the first week and adjust:
|
||||
- Too many emails going there → lower the threshold or add more DB records
|
||||
- Wrong actions being taken → raise the threshold
|
||||
|
||||
---
|
||||
|
||||
## Short-term improvements
|
||||
|
||||
### Admin UI for the email log
|
||||
A simple read-only web view (Flask or FastAPI) to browse `email_log` entries, see LLM reasoning, and mark records as reviewed. Currently you need `psql` or `howl status` to inspect results.
|
||||
|
||||
### CSV / bulk import for contacts
|
||||
A `howl import-customers customers.csv` and `howl import-vendors vendors.csv` CLI command to populate the database from spreadsheet exports. Would accept columns like `name, company, email, notes`.
|
||||
|
||||
### Human review workflow
|
||||
A way to mark emails in the `Needs Review` folder as approved/rejected after human inspection, writing back to `email_log.action_overridden`. This data could seed a feedback loop for improving prompts.
|
||||
|
||||
### Multiple mailbox support
|
||||
Currently `GRAPH_MAILBOX` is a single address. Supporting a list of mailboxes (e.g., `info@`, `support@`, `billing@`) would let one daemon instance manage several inboxes, each potentially with different folder structures.
|
||||
|
||||
### Auto-reply drafting
|
||||
For emails classified as `customer_inquiry`, have Claude draft a suggested reply and save it as a draft in M365 (Graph supports creating draft messages). A human reviews and sends — no risk of auto-sending.
|
||||
|
||||
---
|
||||
|
||||
## Medium-term improvements
|
||||
|
||||
### Domain-based routing rules
|
||||
Beyond exact email address matching, support routing rules like:
|
||||
- All `@acme.com` email → Customers folder regardless of LLM classification
|
||||
- All `@noreply.*` addresses → skip LLM, move directly to Vendors
|
||||
|
||||
### Feedback loop / fine-tuning data collection
|
||||
Export `email_log` rows where `action_overridden=true` as training/evaluation data. Use these to improve prompts or evaluate whether a switch in model or threshold would reduce overrides.
|
||||
|
||||
### Attachment handling
|
||||
Currently only the email body is analysed. Adding attachment type awareness (e.g., flagging `.pdf` attachments from vendors as likely invoices) would improve invoice detection accuracy.
|
||||
|
||||
### Metrics and alerting
|
||||
Export processing counts, failure rates, and LLM token usage to a metrics system (Prometheus, Datadog, etc.). Alert on elevated failure rates or processing backlogs.
|
||||
|
||||
### Scheduled summaries
|
||||
A daily digest email or Teams/Slack message summarising: emails processed, actions taken, items needing review. Useful for staying aware of daemon activity without querying the database.
|
||||
|
||||
---
|
||||
|
||||
## Database considerations
|
||||
|
||||
### Adding indexes for reporting queries
|
||||
As `email_log` grows, queries by date range or `sender_type` will benefit from additional indexes:
|
||||
```sql
|
||||
CREATE INDEX idx_email_log_type_date ON email_log (sender_type, received_at DESC);
|
||||
CREATE INDEX idx_email_log_classification ON email_log (llm_classification);
|
||||
```
|
||||
|
||||
### Archiving old records
|
||||
After 90 days, `email_log` rows for completed messages can be moved to a cold archive table or exported to object storage to keep the active table fast.
|
||||
|
||||
### Contact data enrichment
|
||||
Add fields to `customers` and `vendors` for priority tier, account manager, and SLA level. Pass these to the LLM prompt so high-priority customers get escalated rather than just moved to the folder.
|
||||
|
||||
---
|
||||
|
||||
## Security considerations
|
||||
|
||||
- **Rotate the Azure client secret** every 6–12 months and update `.env`
|
||||
- **Restrict `.env` and `.msal_cache.bin` permissions**: `chmod 600 .env .msal_cache.bin`
|
||||
- **Do not log email bodies** — the current implementation only logs metadata; ensure `LOG_LEVEL=DEBUG` is not used in production as it may expose content
|
||||
- **Review whitelist entries regularly** — expired entries are automatically ignored, but stale permanent entries should be audited
|
||||
- **Monitor for phishing mis-classifications** — review the `Junk Email` folder periodically to check that legitimate email isn't being over-aggressively filtered
|
||||
Loading…
Add table
Add a link
Reference in a new issue