Benchmark
RemoteExploitBench
Test whether a security agent can demonstrate a known vulnerability against a remotely accessible application in a controlled environment.
Explore the original evidence
Explore the original evaluation, its methods, and reported findings.
Open evalCoverage
OpenAI built this evaluation for remote exploitation. Its examples include controlled deployments of GitLab, PostgreSQL and Jenkins.
Methodology
The task is to make a known exploit work against the remote target. This differs from searching for unknown flaws in a web application or stepping outside the assigned target. Model access policies and refusals affect the reported outcomes. The announcement does not specify a public task release or numbered edition.