JeffyBench
Compare text classifiers and language models on sentiment, news topics and banking requests as the number of possible labels rises from two to 77.
Explore the original evidence
Explore the original evaluation, its methods, and reported findings.
Open evalCoverage
Nico Brenner compares his Jeffy classifier with prompted language models, decision models and trained classifiers on SST-2, AG News and Banking77. The evaluation reports accuracy and latency.
Methodology
The October 2026 write-up uses 100 held-out examples per task for the prompted comparison. Its corrected classifier results use the full evaluation splits. SetFit uses 20 training examples per class. The methods differ in training data and evaluation size. The October 26 correction revised the FastText and SetFit baselines and withdrew the Qwen3-32B result.