Amazon Ticket Labeling: Strands Decider vs Sonnet 4.6
Compare Claude Sonnet 4.6 with the small Strands Decider model for sorting tickets in an internal Amazon system into 20 categories, by cost and response time.
Overview
Two Amazon engineers, writing with the Strands Agents team that makes Strands Decider, describe replacing Claude Sonnet 4.6 with that small decision model for labeling tickets in one of Amazon's internal ticketing systems. A decision model returns a probability for each option instead of writing text, so the list of labels becomes the options of one question. The post covers how the switch was tested, what it saved, and one surprise: reordering the keys of the same JSON request moved the probabilities by up to 0.19, so requests are now sent with sorted keys.
What a task looks like
Each task is one ticket, with its comments, and one question: which of 20 categories fits, or none? Each category is written as a one- or two-sentence criterion. Sonnet answered in JSON from a long system prompt. Strands Decider returns a probability for each option and a confidence, and a label is applied only above a tuned threshold. Long tickets lose their oldest comments to fit its 4,096-token window.
How it’s scored
The post reports average time per call, tokens per ticket and cost per ticket at published list prices. The two models were compared on live traffic by judging every disagreement. The post gives no accuracy figures or labeling method. It describes quality only in words: Strands Decider performed as well as Sonnet, and the E4B model beat it on a separate set of tickets.
- Released
- Nov 9, 2026
- Size
- 20 categories plus none. The number of tickets compared is not stated.
- Versions
- Strands Decider 2B was tried first. The E4B build on Gemma 4 replaced it after scoring clearly better on a test set of tickets.
- Access
- The tickets are internal and not published. Strands Decider's code is public on GitHub, and its weights are on Hugging Face.
Summarized by Stubbi from the public sources listed below. Stubbi has not run this benchmark.
Explore the original evidence
Explore the original evaluation, its methods, and reported findings.
Open evalCoverage
One internal Amazon ticketing system. Tickets include customer escalations, capacity requests, automated alarms and security findings, each labeled with one of 20 categories or none. The post compares production Claude Sonnet 4.6 with Strands Decider 2B and its E4B build on Gemma 4. It does not say how many tickets were tested.
Methodology
Each category was rewritten as a one- or two-sentence criterion and sent as the options of one choice question, plus none. A label is applied only above a tuned confidence threshold. A configuration switch ran Sonnet alone, both models on the same tickets, or Strands Decider alone, and the team judged every disagreement on live traffic. Reported averages: 4,215 ms and about $22,200 per million tickets for Sonnet, against 897 ms and about $500 for E4B on a SageMaker ml.g5.xlarge instance, at us-east-1 list prices and excluding Lambda.