Ticket Triage + Action Plan Agent

What it does

• Takes support or dev tickets as input and produces triage results
• Outputs severity, category, suggested assignee, and an actionable plan
• Repair pass fixes common LLM mistakes (invalid enums, missing fields) before committing outputs

Architecture

┌──────────────────┐     ┌───────────────────┐      ┌──────────────────┐
│  Ticket Input    │────►│  Triage LLM       │────► │  Schema          │
│  (JSON/text)     │     │  Extraction       │      │  Validation      │
└──────────────────┘     └───────────────────┘      │  (Zod)           │
         │                         │                └────────┬─────────┘
         │                         │                         │
         ▼                         ▼                         ▼
┌──────────────────┐     ┌───────────────────┐     ┌──────────────────┐
│  Trace /         │     │  Repair Pass      │     │  Triage          │
│  Audit Log       │     │  (if invalid)     │     │  Output          │
└──────────────────┘     └─────────┬─────────┘     └──────────────────┘
                                   │
                                   ▼

Reliability & Guardrails

• Zod validates all triage fields (severity, category, etc.)
• Repair pass re-invokes the LLM with validation errors when the initial output fails, reducing hallucination
• Eval harness enforces invariants (e.g., severity in allowed set, action plan non-empty) on golden cases

Key Engineering Decisions

• Repair pass instead of fail-fast on validation errors — Many LLM mistakes (wrong enum, missing field) are fixable with a second pass; avoids discarding otherwise-good output.
• Strict enum validation for severity/category — Invalid values break downstream routing; repair pass can correct, but schema enforces allowed set.
• Eval harness with both golden cases and invariants — Golden cases catch regression on known tickets; invariants catch new failure modes (e.g., empty action plan).

Failure Modes Handled

• LLM returns severity not in allowed enum — repair pass re-invokes with error context
• Action plan is empty or generic — invariant fails in eval
• Category drift over time — golden cases flag; periodic review of golden set
• Multi-label tickets (e.g., bug + feature) — single-category output; documented limitation
• Repair pass itself fails — fallback to manual triage; trace logged for debugging

How to demo in 2 minutes

1. Clone the repo and install deps: npm install
2. Set OPENAI_API_KEY in .env
3. Run the dev server: npm run dev
4. POST a sample ticket to /api/triage or use the web UI
5. Inspect triage output; trigger repair pass by sending an edge-case ticket

Links