Benchmark
TraceTrial-Bench
Separate an agent’s task performance from the reliability of the software harness running it.
Explore the original evidence
Explore the original evaluation, its methods, and reported findings.
Open evalCoverage
TraceLite AI and the paper’s 23 authors built 95 tasks spanning context, execution, governance, lifecycle, observability, tools and verification.
Methodology
Execution traces and environment checks support separate task and harness assessments. Evidence review then examines possible failure causes. Coverage is uneven, with one verification task. Selected executions and varying runtime conditions limit causal and repeatability claims. Nomos is a separate community experiment; paper v1 is not a benchmark edition.