howl/NEXT_STEPS.md

Next Steps & Roadmap

Immediate (before going live)

1. Azure app registration

Follow SETUP.md to register an app in Azure AD and grant the required Graph API permissions. This is a prerequisite for any real email processing.

2. Seed your database

Populate customers, customer_emails, vendors, vendor_emails, and whitelist with your real contact data before running the daemon. The more complete this data is, the better the LLM's classifications will be.

3. Run a dry-run validation

DRY_RUN=true howl dry-run --limit 20

Review the email_log table. Check that:

• Known customers/vendors are being matched correctly
• LLM classifications feel accurate for your email types
• Confidence scores make sense (low scores = good candidates for whitelist/notes improvements)

4. Tune confidence threshold

The default LLM_CONFIDENCE_THRESHOLD=0.60 sends uncertain emails to Needs Review. Monitor the Needs Review folder for the first week and adjust:

• Too many emails going there → lower the threshold or add more DB records
• Wrong actions being taken → raise the threshold

---

Short-term improvements

Admin UI for the email log

A simple read-only web view (Flask or FastAPI) to browse email_log entries, see LLM reasoning, and mark records as reviewed. Currently you need psql or howl status to inspect results.

CSV / bulk import for contacts

A howl import-customers customers.csv and howl import-vendors vendors.csv CLI command to populate the database from spreadsheet exports. Would accept columns like name, company, email, notes.

Human review workflow

A way to mark emails in the Needs Review folder as approved/rejected after human inspection, writing back to email_log.action_overridden. This data could seed a feedback loop for improving prompts.

Multiple mailbox support

Currently GRAPH_MAILBOX is a single address. Supporting a list of mailboxes (e.g., info@, support@, billing@) would let one daemon instance manage several inboxes, each potentially with different folder structures.

Auto-reply drafting

For emails classified as customer_inquiry, have Claude draft a suggested reply and save it as a draft in M365 (Graph supports creating draft messages). A human reviews and sends — no risk of auto-sending.

---

Medium-term improvements

Domain-based routing rules

Beyond exact email address matching, support routing rules like:

• All @acme.com email → Customers folder regardless of LLM classification
• All @noreply.* addresses → skip LLM, move directly to Vendors

Feedback loop / fine-tuning data collection

Export email_log rows where action_overridden=true as training/evaluation data. Use these to improve prompts or evaluate whether a switch in model or threshold would reduce overrides.

Attachment handling

Currently only the email body is analysed. Adding attachment type awareness (e.g., flagging .pdf attachments from vendors as likely invoices) would improve invoice detection accuracy.

Metrics and alerting

Export processing counts, failure rates, and LLM token usage to a metrics system (Prometheus, Datadog, etc.). Alert on elevated failure rates or processing backlogs.

Scheduled summaries

A daily digest email or Teams/Slack message summarising: emails processed, actions taken, items needing review. Useful for staying aware of daemon activity without querying the database.

---

Database considerations

Adding indexes for reporting queries

As email_log grows, queries by date range or sender_type will benefit from additional indexes:

CREATE INDEX idx_email_log_type_date ON email_log (sender_type, received_at DESC);
CREATE INDEX idx_email_log_classification ON email_log (llm_classification);

Archiving old records

After 90 days, email_log rows for completed messages can be moved to a cold archive table or exported to object storage to keep the active table fast.

Contact data enrichment

Add fields to customers and vendors for priority tier, account manager, and SLA level. Pass these to the LLM prompt so high-priority customers get escalated rather than just moved to the folder.

---

Security considerations

• Rotate the Azure client secret every 6–12 months and update .env
• Restrict .env and .msal_cache.bin permissions: chmod 600 .env .msal_cache.bin
• Do not log email bodies — the current implementation only logs metadata; ensure LOG_LEVEL=DEBUG is not used in production as it may expose content
• Review whitelist entries regularly — expired entries are automatically ignored, but stale permanent entries should be audited
• Monitor for phishing mis-classifications — review the Junk Email folder periodically to check that legitimate email isn't being over-aggressively filtered