Back to Portfolio

Support Ticket Router · AI Recommendation with Human Gate

SupportFlow Mini
Production System

A customer support ticket router with AI recommendation, human review gate, and webhook delivery. Built end-to-end with FastAPI, Pydantic, SQLite, OpenAI, and deterministic evaluation. One problem, one model call, one human decision, one safe integration.

Customer IssueAI RecommendationHuman ReviewSafe DeliveryMeasure Quality
View Code

Status: In Progress

A production-grade support ticket routing system. Handles ticket ingestion, AI-powered categorization and priority assignment, human review and approval workflow, webhook delivery to external systems, and quality evaluation across a held-out test set.

The problem

Manual support ticket triage is slow and inconsistent. Tickets arrive with a subject and description. A human reviewer reads each one and routes it to the right team, but:

How it works

SupportFlow Mini routes each ticket through a simple, auditable workflow: submit the ticket, get an AI recommendation, let a human decide, deliver the routing to the team, measure accuracy.

Step 1

Submit

Customer or support agent submits a ticket: customer name, tier, subject, and description.

Step 2

Recommend

AI model recommends a category (billing, technical, account, feature request, other) and priority (low, medium, high, urgent).

Step 3

Review

Human reviewer sees the recommendation with confidence score and can approve, correct, or reassign the ticket.

Step 4

Deliver

Approved routing is sent via webhook to the assigned team. Webhook includes the full ticket and final routing.

Step 5

Measure

Evaluate recommendation accuracy (agreement with human decision) and confidence calibration across the evaluation set.

Architecture

FastAPI backend with Pydantic validation, SQLite for ticket and event storage, mock-first workflow logic, browser-based review interface, OpenAI for AI recommendation, and webhook integration for delivery. Deterministic evaluation harness measures model accuracy.

API & Database

FastAPI service with Pydantic schemas, SQLite for tickets and events, dependency injection for clean routes.

POST /api/tickets creates and stores tickets with full audit trail. GET endpoints surface data for review UI.

Typed Data Contract

Pydantic models enforce ticket shape, AI output schema, and human decision structure.

TicketCreate validates input. TriageResult defines category, priority, team, summary, suggested response, confidence. ReviewResult captures final decision and notes.

Mock-First Workflow

Deterministic triage function runs the workflow without AI, proving logic before adding the model.

Uses keyword-matching rules to categorize tickets, letting you test form submission, storage, and review UI without LLM calls.

AI Recommendation Engine

Calls OpenAI API with ticket content, returns structured category, priority, team, and confidence.

Model sees only the ticket; never sees the expected answer or reviewer decision. Calls happen in production, not local testing.

Browser Review Interface

HTML form displays ticket, AI recommendation, and lets reviewer approve, correct, or reassign.

POST to /api/tickets/(id)/review sends final routing decision and optional notes. Stores reviewer name and timestamp.

Webhook Integration

Sends approved routing to external system via HTTP POST with full ticket and final categorization.

Retry logic, idempotency keys, and integration testing happen in later chapters. Early chapters prove the happy path.

Evaluation Harness

Scores recommendation accuracy against 5 holdout test cases, computes agreement and confidence metrics.

Reports show how often the model agrees with humans and whether high-confidence predictions are actually correct.

Why mock first

Building with mock data first isolates the workflow, routing logic, database behavior, review state and UI integration from model uncertainty. You prove the core works before adding AI.

Chapter 4 walks through deterministic triage with keyword-matching rules. Once forms, storage, and review all work correctly, Chapter 6 swaps in the real model call. Same data schema, same API contract, same webhook flow.

Implementation approach

Deterministic Testing

Mock the model to prove workflow logic. Same seed produces same recommendations, same review UI renders correctly, same webhook fires.

Typed Contracts

Pydantic enforces field types and presence. Invalid requests are rejected before business logic runs. Model outputs are always schema-valid.

One Problem, One Model Call

Ticket routing has exactly one decision: where should this go and how urgent is it? One model call, one confidence score, one human gate.

Human Gate Separates Review from Decision

Reviewer decision is stored separately from AI recommendation. You can compute agreement, measure confidence calibration, and maintain an audit trail.

Workflow First, Model Second

Build the full end-to-end flow with mock data. Prove forms, storage, review, and webhooks work. Then add the model call — same integration points, same data schema.

Production Practices from Day One

Dependency injection, environment config, secret boundaries, error handling, audit logs. No refactoring required when you ship.

Stack

PythonFastAPIUvicornPydanticSQLiteOpenAI APIStructured OutputsHTMLJavaScriptCSSWebhooksREST APIDependency Injection

System capabilities

✓Ticket ingestion: Accept customer support tickets via REST API with full validation. Store in SQLite with complete audit trail.

✓AI categorization: Recommend ticket category (billing, technical, account, feature request, other) and priority (low, medium, high, urgent) with confidence score.

✓Human review workflow: Web interface for reviewers to see recommendation, approve, correct, or reassign. Separate human decision from AI recommendation for audit trail.

✓Webhook delivery: Send approved routing to external systems via HTTP with idempotency, retry logic, and error handling.

✓Quality evaluation: Measure model accuracy against held-out test set. Track agreement with human reviewers and confidence calibration.

✓Deterministic testing: Mock workflow without AI to prove forms, storage, review, and delivery work before adding the model.

✓Production foundation: Pydantic validation, dependency injection, environment config, secret boundaries, and comprehensive error handling.

View the implementation

Complete, production-ready system from API through evaluation. Covers ticket ingestion, AI recommendation, human review workflow, webhook delivery, and quality measurement.

Code