Benchmark
LongMemEval
Test whether assistants remember, update, and reason about information across conversations.
Explore the original evidence
Explore the original evaluation, its methods, and reported findings.
Open evalCoverage
500 questions covering extraction, multi-session reasoning, knowledge updates, temporal reasoning, and abstention; S and M settings use different history lengths.
Methodology
Give an assistant a timestamped conversation history and ask questions whose answers depend on that history. Evaluate its answers, including whether it declines questions with no supporting information, using the official protocol.