wulf-pulse/docs/TIME_ENTRIES_ENRICHMENT.md
root 6eee14f8af Add comprehensive admin features and multi-system integration
- Add admin dashboard with sync controls and data browser
- Implement RMM, Auvik, and Addigy organization mappings
- Add chunked ticket sync with progress tracking
- Implement entity sync service with rate limiting
- Add analytics engine and performance optimizer
- Create data browser for all PSA entities
- Add navigation components and UI improvements
- Implement background processing and sync services
- Add comprehensive documentation and migration scripts
- Update configuration items with multi-system support
- Enhance contact management and purchase history
- Add issue type assignment and LLM analyzer
- Improve error handling and logging utilities
2025-11-19 14:18:16 -05:00

9.8 KiB

Time Entries Enrichment & Filtering

Overview

Added two powerful features to the Time Entries data browser:

  1. Data Enrichment - Replace IDs with human-readable names
  2. Ticket Filtering - Filter to show only entries with tickets (default ON)

Features

1. Tickets Only Filter 🎫

Default State: ON (enabled by default)

Purpose: Hide time entries without associated tickets to focus on billable work.

Button States:

  • Active (Blue): "Tickets Only" - Showing only entries with tickets
  • Inactive (Outline): "Show All" - Showing all entries

Implementation:

  • Adds has_ticket=true query parameter to API
  • Filters at database level: WHERE ticket_id IS NOT NULL
  • Persists across pagination and sorting
  • Resets to page 1 when toggled

Usage:

// Click button to toggle
<Button onClick={toggleHideNonTicket}>
  {hideNonTicket ? 'Tickets Only' : 'Show All'}
</Button>

2. Data Enrichment

Default State: OFF (click to enable)

Purpose: Replace numeric IDs with human-readable names for better readability.

Enriches:

  • Resource IDFull Name (e.g., 30861536John Smith)
  • Ticket IDTicket Number (e.g., 638476T20241104.0001)

Button States:

  • Active (Purple): "Enriched" - Data is enriched
  • Inactive (Outline): "Enrich" - Click to enrich
  • Disabled: No data to enrich

How It Works:

  1. Extracts unique resource and ticket IDs from current page
  2. Fetches resource and ticket details in parallel
  3. Builds lookup map for fast rendering
  4. Updates display without refetching time entries
  5. Toggle off to return to ID view

Performance:

  • Only enriches visible page (not all data)
  • Parallel API calls for resources and tickets
  • Cached in component state
  • No database overhead

Implementation:

const handleEnrichData = async () => {
  // Extract IDs
  const resourceIds = [...new Set(timeEntries.map(e => e.resource_id))];
  const ticketIds = [...new Set(timeEntries.map(e => e.ticket_id))];
  
  // Fetch in parallel
  const [resources, tickets] = await Promise.all([
    fetch(`/api/data/resources?ids=${resourceIds.join(',')}`),
    fetch(`/api/data/tickets?ids=${ticketIds.join(',')}`)
  ]);
  
  // Build lookup map
  const enrichmentMap = {};
  resources.forEach(r => {
    enrichmentMap[`resource_${r.id}`] = `${r.first_name} ${r.last_name}`;
  });
  tickets.forEach(t => {
    enrichmentMap[`ticket_${t.id}`] = t.ticket_number;
  });
  
  setEnrichedData(enrichmentMap);
  setEnriched(true);
};

UI/UX Design

Button Placement

Located in the header row, before Analytics/Export/Refresh buttons:

[Tickets Only] [Enrich] | [Analytics] [Export] [Refresh]

Visual States

Tickets Only Button

  • ON: Blue background (bg-blue-600), white text
  • OFF: Outline style, default text color
  • Icon: TicketX from Lucide

Enrich Button

  • ON: Purple background (bg-purple-600), white text
  • OFF: Outline style, default text color
  • Disabled: Grayed out when no data
  • Icon: Sparkles from Lucide

Dark Mode Support

Both buttons fully support dark mode with appropriate color variants.

API Changes

1. Time Entries API (/api/data/time-entries)

New Parameter:

  • has_ticket (string): Filter for entries with tickets
    • "true" - Only entries with ticket_id IS NOT NULL
    • Omit or any other value - Show all entries

Example:

GET /api/data/time-entries?has_ticket=true&limit=100&offset=0

Implementation:

if (hasTicket === 'true') {
  conditions.push(`te.ticket_id IS NOT NULL`);
}

2. Resources API (/api/data/resources)

New Parameter:

  • ids (string): Comma-separated resource IDs

Example:

GET /api/data/resources?ids=30861536,30861443,29823085

Response:

{
  "resources": [
    {
      "id": 30861536,
      "first_name": "John",
      "last_name": "Smith",
      "email": "john.smith@example.com"
    }
  ]
}

Implementation:

if (ids) {
  const idArray = ids.split(',').map(id => parseInt(id.trim()));
  conditions.push(`id = ANY($${params.length + 1})`);
  params.push(idArray);
}

3. Tickets API (/api/data/tickets)

New Parameter:

  • ids (string): Comma-separated ticket IDs

Example:

GET /api/data/tickets?ids=638476,638477,638478

Response:

{
  "tickets": [
    {
      "id": 638476,
      "ticket_number": "T20241104.0001",
      "title": "Server Issue",
      "status": 1,
      "priority": 2,
      "company_id": 12345
    }
  ]
}

Implementation:

if (ids) {
  const idArray = ids.split(',').map(id => parseInt(id.trim()));
  const query = `
    SELECT id, ticket_number, title, status, priority, company_id
    FROM tickets
    WHERE id = ANY($1) AND is_deleted = false
  `;
  const result = await postgresClient.query(query, [idArray]);
  return NextResponse.json({ tickets: result.rows });
}

User Workflow

Typical Usage

  1. Page Load

    • Tickets Only filter is ON by default
    • Shows only time entries with tickets
    • IDs displayed (not enriched)
  2. Enrich Data

    • Click "Enrich" button
    • Wait ~1-2 seconds for data fetch
    • Resource names and ticket numbers appear
    • Button turns purple showing active state
  3. Toggle Enrichment

    • Click "Enriched" button to turn off
    • Returns to ID display
    • No API call needed
  4. Show All Entries

    • Click "Tickets Only" to toggle off
    • Button changes to "Show All"
    • Page refetches with all entries
    • Enrichment state preserved
  5. Navigate Pages

    • Enrichment clears on page change
    • Tickets Only filter persists
    • Click "Enrich" again for new page

Technical Details

State Management

// Filter state
const [hideNonTicket, setHideNonTicket] = useState(true); // Default ON

// Enrichment state
const [enriched, setEnriched] = useState(false);
const [enrichedData, setEnrichedData] = useState<Record<string, any>>({});

Column Rendering

Before Enrichment:

<Badge variant="outline">
  <Users className="w-3 h-3 mr-1" />
  30861536
</Badge>

After Enrichment:

<Badge variant="outline">
  <Users className="w-3 h-3 mr-1" />
  John Smith
</Badge>

Performance Considerations

Enrichment:

  • Only fetches data for current page
  • Parallel API calls (resources + tickets)
  • Typical load time: 1-2 seconds
  • No impact on pagination/sorting

Filtering:

  • Database-level filtering (efficient)
  • No client-side processing
  • Indexed columns for fast queries

Benefits

1. Improved Readability

  • Human names instead of IDs
  • Ticket numbers instead of internal IDs
  • Easier to scan and understand data

2. Focus on Billable Work

  • Default filter shows only ticket-related entries
  • Reduces noise from non-billable time
  • Better for invoicing and reporting

3. Flexible Workflow

  • Toggle enrichment on/off as needed
  • Show all entries when needed
  • No permanent changes to data

4. Performance

  • Enrichment only when requested
  • Page-level enrichment (not all data)
  • Fast toggle off (no API call)

Future Enhancements

Potential Improvements

  1. Persistent Enrichment - Remember enrichment preference
  2. Auto-Enrich - Option to always enrich on page load
  3. More Fields - Enrich company names, project names
  4. Caching - Cache enrichment data across pages
  5. Batch Enrichment - Enrich all pages at once
  6. Export Enriched - Export with names instead of IDs

Additional Filters

  1. Has Task - Filter for entries with tasks
  2. Has Project - Filter for entries with projects
  3. Billable Only - Quick filter for billable entries
  4. Approved Only - Quick filter for approved entries

Files Modified

Frontend

  • /app/admin/data-browser/time-entries/page.tsx
    • Added enrichment state and logic
    • Added ticket filter state (default ON)
    • Updated column rendering
    • Added header buttons

Backend

  • /app/api/data/time-entries/route.ts

    • Added has_ticket parameter
    • Added filter condition
  • /app/api/data/resources/route.ts

    • Added ids parameter
    • Added bulk fetch by IDs
  • /app/api/data/tickets/route.ts

    • Added ids parameter
    • Added bulk fetch by IDs

Documentation

  • /docs/TIME_ENTRIES_ENRICHMENT.md - This file

Testing

Manual Test Steps

  1. Test Tickets Only Filter (Default ON)

    1. Navigate to /admin/data-browser/time-entries
    2. Verify "Tickets Only" button is blue (active)
    3. Verify all entries have ticket IDs
    4. Click button to toggle off
    5. Verify "Show All" appears
    6. Verify entries without tickets appear
    7. Toggle back on
    
  2. Test Enrichment

    1. Navigate to time entries page
    2. Verify "Enrich" button is outline style
    3. Click "Enrich" button
    4. Wait for loading
    5. Verify resource IDs become names
    6. Verify ticket IDs become ticket numbers
    7. Verify button turns purple
    8. Click "Enriched" to toggle off
    9. Verify IDs return
    
  3. Test Persistence

    1. Enable Tickets Only filter
    2. Sort by different column
    3. Verify filter persists
    4. Change page
    5. Verify filter persists
    6. Enrich data
    7. Change page
    8. Verify enrichment clears
    
  4. Test Performance

    1. Load page with 100 entries
    2. Click Enrich
    3. Measure load time (should be < 3s)
    4. Toggle off (should be instant)
    5. Toggle on (should be instant, no refetch)
    

Conclusion

The enrichment and filtering features significantly improve the usability of the Time Entries data browser by:

  • Making data more readable with human names
  • Focusing on relevant ticket-related entries by default
  • Providing flexible, performant data views
  • Following shadcn/ui design patterns

These features enhance the user experience without compromising performance or adding complexity to the data model.