Agents Lookup

Recipes / Customer support

Support Ticket and Review Triage with Policy-Cited Draft Replies

An agent tags every helpdesk ticket and public review by intent, urgency, and sentiment, routes it, and drafts a reply that cites your help center. People approve refunds and all public replies.

CategoryCustomer support
Steps10
Human sign-offs5

The problem

Small support teams lose hours every week just reading and sorting. Tickets arrive by email, chat, and contact forms while public reviews pile up on marketplaces and review sites. Someone has to decide what each message is about, how urgent it is, who should own it, and what the correct policy answer is, and that work happens before anyone writes a reply. Urgent items such as a safety complaint, a legal threat, or a chargeback notice can sit unnoticed behind a dozen where-is-my-order questions. Replies drift from the actual return or warranty policy because each person paraphrases from memory. Public review responses get written in a hurry or not at all. This recipe puts an agent in front of the queue to classify, route, and draft, while a person stays in charge of every refund, every escalation, and every public-facing word.

Who it's for

Not a fit when

Tools

Helpdesk (Zendesk, Gorgias, Help Scout, or shared Gmail)Source of tickets; receives tags, assignments, and internal-note drafts
Review platform exports or notificationsSource of public reviews; many platforms offer email alerts or CSV export
Claude or another LLMClassifies intent, urgency, and sentiment and drafts replies grounded in policy text
Help center or policy documents (Google Drive or helpdesk knowledge base)The only approved source of facts the agent may cite
Zapier or MakeMoves new tickets and reviews to the agent and writes results back
SlackInstant alerts for escalation-category items
Google SheetsTriage log for auditing tags, overrides, and response times

Inputs you need first

Step-by-step build

  1. Define the tag taxonomy

    Write down three tag groups in a Google Sheet. Intent: 6 to 10 categories that match how your team already works, plus 'other'. Urgency: low, normal, high, escalate. Sentiment: positive, neutral, negative. Add one sentence per tag describing when it applies. Keep the list short; a taxonomy with 30 intents produces inconsistent tagging from people and agents alike. Mirror the same tags in your helpdesk so the agent writes values that already exist instead of inventing new ones.

  2. Write explicit escalation triggers

    List the conditions that always go to a person immediately: any mention of injury, fire, allergic reaction, or product safety; words like lawyer, attorney, lawsuit, or regulator; chargeback, dispute, or bank reversal notices; threats or harassment; and requests to delete personal data. These items get urgency 'escalate', a Slack alert to the named owner, and no drafted reply. Treat this list as a hard rule the agent cannot override, and review it quarterly with whoever owns legal and finance questions.

  3. Prepare the policy library

    Collect your help center articles and policies into one folder, one document per topic, each with a title, a last-updated date, and a URL if public. Remove outdated versions. The agent will be instructed to cite only from this library by title or URL. If a policy is ambiguous, for example 'returns accepted within a reasonable time', fix the wording before launch, because the agent will repeat whatever ambiguity it finds.

  4. Connect intake with Zapier or Make

    Create a workflow triggered by new tickets in your helpdesk and new review notifications. Pass the agent the message text, channel, customer name, order number if present, and prior messages in the thread. Strip payment card details and full addresses before sending. For reviews, include the star rating and platform. Start on a free or entry tier of your automation tool with a single trigger; add channels once the first one works.

  5. Build the classification and draft step

    Send each item to the agent with the system prompt below plus the relevant policy documents. Require a structured output: intent, urgency, sentiment, a one-line summary, suggested assignee group, cited policy titles, a confidence level, and a draft reply. If no policy applies, the agent must leave the draft empty and write 'needs human answer'. Write the draft as an internal note, never as a sent reply.

  6. Route and alert

    Write tags and assignment back to the helpdesk. Route billing to whoever handles payments, returns to the fulfillment lead, product questions to the product owner. Post escalate-level items to a private Slack channel with the ticket link and summary. Log every item in the Google Sheet with columns for timestamp, ticket ID, tags, confidence, cited policy, and a blank 'human override' column.

  7. Handle public reviews separately

    For reviews, the agent drafts a short, courteous response that acknowledges the experience, avoids admitting fault or discussing order details publicly, and invites the customer to contact support privately. Drafts go to an approval queue, either a Sheet tab or a Slack thread, with columns for platform, rating, review text, draft, approver, and status. Nothing is posted until a named person marks it approved and posts it themselves or via an approved integration.

  8. Test with historical tickets

    Run 50 to 100 past tickets through the workflow in a test view or sandbox where nothing is written to live tickets. Compare agent tags with the correct tags in your sample and note disagreements. Check that every escalation-trigger example was caught. Read every draft for accuracy against policy. Adjust tag descriptions and prompt wording, then re-run. Do not go live until escalation triggers are caught every time in your test set.

  9. Go live in shadow mode first

    For the first one to two weeks, let the agent tag and draft while your team works the queue normally and records overrides in the log. Then let agents' tags drive routing, while drafts stay as internal notes that a person edits and sends. Announce clearly to the team that the agent cannot send, refund, or close tickets.

  10. Maintain weekly

    Each week, review the override column: which tags were changed most, which drafts were rewritten heavily, and whether any urgent item was mislabeled. Update the policy library whenever a policy changes, the same day. Add new intent categories only when 'other' grows large. Re-run the historical test set after any prompt or policy change.

Example agent instructions

Adapt this to your business. Placeholders in braces are filled by your automation tool.

You are a support triage assistant for {business_name}. You classify and draft; you never send messages, issue refunds, offer credits, or promise outcomes. For the message below, return: intent (one of {intent_list}), urgency (low, normal, high, escalate), sentiment (positive, neutral, negative), a one-sentence summary, suggested team, cited policy titles, confidence (high, medium, low), and a draft reply. Rules: If the message mentions injury, safety, a lawyer or legal action, a chargeback or bank dispute, threats, or a data deletion request, set urgency to escalate, leave the draft empty, and explain why in the summary. Only state facts found in the policy documents provided; cite each by title. If no policy answers the question, write 'needs human answer' instead of a draft. Do not invent order details, dates, tracking numbers, or exceptions. For public reviews, keep the draft under 80 words, do not discuss order specifics, and invite the customer to contact {support_email}. Write in a warm, plain tone. Channel: {channel}. Customer: {customer_name}. Thread: {ticket_text}. Policies: {policy_docs}.

Human sign-off points

Guardrails

Expected time saved

Estimate: if your team handles about 150 tickets and reviews per week and spends around 3 minutes each reading, tagging, routing, and looking up the policy before writing, triage alone is roughly 7 to 8 hours per week. Pre-tagged items with a policy-grounded draft might cut that sorting and lookup time substantially, though editing drafts still takes time. Results depend on volume, policy clarity, and how often drafts need rewriting.

How to tell it's working

FAQ

Can the agent just send the easy replies automatically?

This recipe deliberately keeps a person on every send. Even simple order-status replies can be wrong if tracking data is stale or the thread has context the agent misread. Once you have weeks of logs showing drafts in one narrow category rarely need edits, you can consider auto-sending that category, but start with human sends and revisit only with evidence from your own override log.

What if a customer asks for something our policy does not cover?

The agent is instructed to return 'needs human answer' instead of improvising. That ticket goes to a person who decides. If the same uncovered question keeps appearing, that is a signal to write a new help center article or policy section, add it to the library, and let the agent cite it from then on.

Is it safe to send customer messages to an AI model?

Review your model provider's data handling terms and choose a business plan whose terms fit your needs. Strip payment card numbers and full addresses before sending, limit the agent's helpdesk permissions, and tell customers in your privacy policy that you use service providers to help process support requests. If you have specific regulatory obligations, get professional advice before connecting.

Want this set up for you?

Build it yourself with the steps above, or tell us what you need. We scope the work with you and send a written quote before anything starts.

Request a quote

Sources

Written by Agents Lookup. Official pages for the tools this recipe uses: