Automation Quick Wins Finder

---
name: automation-quick-wins
description: "Analyzes a sample of recent support conversations from a Rippit workspace to find topics an AI/bot agent could automate by adding Q&A or help-article entries. For each conversation it extracts a normalized topic plus redacted summaries of what the customer asked and what the agent answered, then scores each one for automation fit, likely blockers, and resolution confidence. It clusters topics into families, applies a conservative tiering rule that separates true quick wins from cases still needing a human check, and flags wording inconsistencies or group-specific answers that would make a single Q&A entry wrong. Outputs two CSVs — candidate Q&A cases and a ranked list of quick-win topic families — plus a short written brief. Use when a user wants to know what their automated agent could handle, asks for automation quick wins, Q&A candidates, or deflection scope."
---
# Automation Quick Wins Finder

Finds the topics an automated agent could resolve quickly by analysing a sample of recent support conversations and judging which ones a generic Q&A entry would fully answer.

## First run — make it yours

Before asking anything, LOOK at what the user actually has:
1. Call `list_data_sources` and list what is connected.
2. Identify the conversation table and `describe_table` it. Note which columns could serve as: the conversation identifier, a timestamp, a grouping field (brand / business unit / region), and a topic-category field if one exists.
3. If there is no conversation-level table or no usable identifier column, say so plainly and ask how to proceed rather than running into a dead end.

THEN ask one question at a time, leading with what you found and proposing a default:

1. **{{id_file}}** — "I can read conversations from your workspace. Do you have a file of conversation IDs to analyze, or should I pull a recent sample directly from the table I found?"
2. **{{data_source}}** — "Your workspace has <source>, with a conversation table <table> whose <column> column holds the IDs — use that as the source?"
3. **{{id_column}}** — "Which column matches your conversation IDs — <column> looks right, confirm?" (IDs are usually time-ordered strings; keep as strings.)
4. **{{sample_size}}** — "How large a sample, and in how many chunks? A common default is the newest 5,000 IDs, deduped and sorted newest-first, split into 5 chunks of 1,000."
5. **{{grouping_field}}** — "Want results broken out by a grouping field like brand or business unit? <column> looks like it holds that — use it?"
6. **{{fit_categories}}** — "The skill sorts each case into automation-fit categories. Defaults: easy_qa, needs_account_data, needs_agent_action, needs_judgment_or_risk, not_a_support_question, unclear. Use these or your own?"
7. **{{blocker_categories}}** — "What blocker categories for non-automatable cases? Defaults cover account-specific data, backoffice action, verification, payment issue, policy/legal, discretion, complaint/emotional, tech bug, other."
8. **{{excluded_categories}}** — "Which fit categories should be excluded from quick-win tiers entirely regardless of answer quality (e.g. legal/risk/complaint cases)?"
9. **{{topic_families}}** — "The skill clusters topics into families via ordered regex rules. Start from a default family list for your domain, or give me your common topics?"

After the interview, restate the filled-in bindings for confirmation, then run. On later runs, reuse these bindings unless the user asks to change them.

## Inputs and defaults (state the assumption, do not block on questions)
- ID source: if {{id_file}} is given, dedupe and sort DESC (time-ordered IDs → desc = newest first). Default sample = {{sample_size}} (e.g. newest 5,000 split into 5 chunks of 1,000 → 5 worksheets).
- Rippit source: `list_data_sources`, then use {{data_source}}. ID column is {{id_column}} (keep IDs as strings). Re-check the id with `list_data_sources`.
- Coverage probe first: `read_table` with {{id_column}} IN [every ~250th ID], reading the ID and timestamp columns. Expect most to match; the very newest IDs may be unsynced. Report rows loaded per worksheet (e.g. 959/1000), do not treat as failure.

## Steps
1. Write ids_1..N.json (JSON arrays of strings) to the scratchpad and a shared brief file with the enrichment spec below.
2. Spawn one sub-agent per chunk IN ONE MESSAGE (general-purpose). Each: `create_worksheet` (tableId = {{data_source}} table, where {{id_column}} IN [chunk ids], titled "Automation Quick Wins - chunk N", no rowLimit/sampleRows/columnIds), confirm row count is stable, `enrich_worksheet` ONCE with all 7 columns, loop `get_enrich_status` until done:true, aggregate counts, and return worksheetId, rows loaded, column ids, fit distribution. Tell them NOT to download export URLs and NOT to page rows by hand.
3. Merge in the main session: for each worksheet call `read_table` with ALL rows (limit = chunk size) and only the needed columnIds ({{id_column}}, the timestamp, {{grouping_field}}, any topic-category column, plus the 7 enrichment columns). A full read spills to a file on disk; load with Python (json) and map column ids to labels via the `columns` array. Do NOT use filtered reads: small results come back inline and flood the context.
4. Analyse with pandas (see tiers and families), write two CSVs to /mnt/user-data/outputs/ and deliver with SendUserFile.
5. Verify: row totals per sheet, easy_qa count equals the sum of per-sheet aggregate counts, ID uniqueness, no emails/URLs in summaries, spot-check ~8 random rows.

## Enrichment columns (identical on every worksheet)
1. qa_topic (string): short normalized topic label, 2-6 words, Title Case, English; reuse wording; 'No Real Question' if nothing was asked.
2. customer_question (string): 1-2 sentence English summary of what the customer asked; strip names, emails, usernames, account numbers, exact amounts; 'No customer question' if none.
3. agent_answer (string): 1-2 sentence summary of what the human agent answered/did; same redaction; 'No agent reply' if only bot/none.
4. fin_qa_fit (enum over {{fit_categories}}, plus unclear): classify the automation fit. The 'easy' category applies ONLY if ALL true: a general how-it-works question (policy, rules, steps, timeframes, requirements, where to find something); the agent answer is generic and word-for-word valid for any customer; no account lookup / balance / transaction / eligibility / verification-status check; no backoffice action; no risk/legal/complaint element. The remaining categories cover: depends on this customer's data with no action taken; agent had to perform an action (adjust an account, change a setting, process/cancel a request, update details, escalate); judgment/risk cases (legal, fraud, complaints, angry/vulnerable, discretionary); and not-a-support-question (spam, wrong channel, business inquiry, empty).
5. fin_blocker (enum over {{blocker_categories}} plus none): 'none' only if the fit is the 'easy' category.
6. answer_is_generic (boolean): agent answer has nothing specific to this account.
7. fin_confidence (integer 1-5): confidence a bot with a Q&A entry resolves it fully (5 certain, 3 caveats, 1 needs human).

## Tiers for the final CSV
- Tier A: fin_qa_fit = the 'easy' category. Keep rows even when agent_answer is 'No agent reply' (flag agent_answer_available = no).
- Tier B: fit in (needs_account_data, needs_agent_action equivalents from {{fit_categories}}) AND answer_is_generic = yes AND an agent answer exists AND not a no-real-question row. Label as 'needs a check'.
- Exclude any category in {{excluded_categories}} even if generic.
- Do NOT use fin_confidence alone as a filter: it is loose (many rows score 4-5 including non-questions and account-specific cases).

CSV 1 (candidates) columns: qa_tier, conversation_id, date, group, topic_category, topic_family, topic_what_is_it_about, what_customer_asked, what_agent_answered, agent_answer_available, answer_is_generic, ai_confidence_1_5, blocker, source_worksheet. Write with utf-8-sig.

CSV 2 (ranked topics): rank, topic_family, conversations_in_sample, pct_of_sample, easy_A, generic_answer_B, candidates_total, candidates_pct_of_family, avg_confidence_candidates, suggested_qa_draft, caveat. Sort by candidates_total.

## Topic families
raw qa_topic labels fragment badly (often hundreds of distinct values). Cluster with ordered regex rules on qa_topic, falling back to customer_question, using the families in {{topic_families}}. Put specific rules before broad ones (broad keyword stems match too much). Families are approximate; say so in the answer.

## Suggested Q&A drafts
Only draft text for families with several consistent agent answers, built from what agents actually said, marked as drafts to verify against policy. Always look for and report: (a) wording inconsistencies between agents (e.g. a timeframe quoted differently; a multiplier quoted differently), (b) group-specific content (e.g. a brand without a given program), because a single Q&A entry would otherwise be wrong for some groups in {{grouping_field}}.

## Pitfalls learned
- Rippit export_worksheet URLs need a logged-in session; curl returns an HTML app page. Read rows back with `read_table` instead and mention the worksheets live in Rippit.
- `read_table` results under ~50KB come back inline and fill the context; full reads spill to disk. Plan for the spill and process with Python.
- Any AI-summary column may be null for some rows; do not rely on it.
- 1-2 rows per 1,000 may come back 'Processing failed' with null confidence; exclude them and say so.
- Enriched topic wording is not normalized across sheets; merge families after combining, not per sheet.
- The strict 'easy' tier is conservative (often ~5%). Large pools (e.g. status-check chats) mostly need an account lookup; report the stretch number separately and offer a follow-up pass instead of folding it into the tiers.

## Final answer format
Short prose: assumptions (sample definition, rows loaded), the ranked quick wins (top ~6 with counts), the decisions needed (inconsistent wording, group-specific content), problems encountered, a plain confidence statement (high for tier A, medium for tier B, family boundaries approximate), then the two files.