We introduce ScreenBench, a benchmark of AI agents’ performance on compound-screening analysis tasks that contain hidden errors. The ScreenBench dataset comprises 182 tasks built from real cell-viability screening plates and their dosing records, into which we plant one to three undisclosed errors.
Each task asks for a routine result about compound activity, identity, or measurement location. Answering correctly requires the agent to notice the errors and correct for them. A score of 1 means every graded answer is correct, and 0 means no better than simple shortcuts like copying values from the records. Gemini 4 Argon scored 0.49, Opus 5.5 scored 0.47, and GPT 6.1 Sol scored 0.12. When the task brief lists the possible kinds of error, all three agents scored between 0.94 and 0.96.
There are seven kinds of error, drawn below. In each plate with an error, the dark wells stop lining up with the rings. The edge effect is a decoy that doesn’t change any answer. We tried to make each error about as strong as the failures we’ve seen in these screens.
The seven kinds of error
One made-up 384-well plate. Rings mark compounds the records say should kill cells; dark wells are low signal.
high signal (cells alive)low signal (hit)expected hit, from the records
As recordedHits line up with the rings
RotationReadout turned 180°
Line swapTwo rows or columns exchanged
Well orderWells laid out in the wrong order
Plate mix-upAnother plate’s readout
Source exchangeOutlined wells got another source plate’s compounds
Failed regionA patch loses signal
Edge effectDecoy: changes no answer
In the real tasks we plant errors into real readouts.
In a high-throughput screen, every hit call depends on the plate map, the record of which compound went into which assay well. Barcode tracking and transfer logs make errors in that chain rare, but not impossible.
A readout filed under the wrong plate barcode can still pass quality control, because Z′ and similar statistics measure how well the assay worked, not whether each well is matched to the right compound. Hit calling on that data can be arithmetically flawless and still credit activity to the wrong compound, so inactive compounds become hits and real actives disappear.
Experienced screening teams guard against this before trusting a plate. They check that the controls sit where the plate map puts them and that known actives read out where expected. We wanted to know whether AI agents make the same checks without being told to.
The tasks come from eight of our unpublished cell-viability screens, 467 plates in total. Each task gives the agent eight plates of readouts, the dosing records, the library map and each compound’s results in related screens. We planted between one and three errors in each task and didn’t mention them in the instructions.
Example (made-up numbers)
Records
Compound C went into well A01 of plate P01. A01 reads 12% of the plate median, so C looks like a strong hit.
Data
The plate’s readout was saved rotated 180°. C’s real well is P24, in the opposite corner, which reads 96%. C is inactive. The first two plates in the figure above show what a rotation looks like.
How you’d notice
The controls are on the wrong side, and compounds that were hits in earlier screens only match low-signal wells after you turn the plate around.
Each task asks about every compound in the screen, about 2,400 of them, but we grade only 8 to 12 answers, and the agent doesn’t know which. We call these graded items. Since we planted the errors, we know the right answer to each one and can grade automatically. Most of the score comes from items that an error changed, so an agent gets partial credit for each error it handles. Every task also has a guided version that is identical except for one extra paragraph listing the kinds of error to look for.
Results
We ran three agents once on each task, each in its own company’s coding tool, without web access. A script that just copies the records scores 0, because it is one of the shortcuts that set the zero point. Unscaled, it would get 0.27, mostly from items that no error touches.
Told what to look for, every agent scores above 0.9
Mean task score over 182 tasks. 0 is the best shortcut answer, 1 a full solve.
UnpromptedTold what to look for
One run per task. *GPT 6.1 Sol was used to select 95 of the tasks, which can lower its score; on the other 87, Opus scores 0.45, Gemini 0.44 and GPT 6.1 Sol 0.13.
Everything the agents need to fix the errors is in the data. Unless someone tells them what to look for, they mostly don’t check.
62 tasks get past all three agents
Each column is one task and each square one unprompted attempt. Tasks are sorted by the best score any agent reached, hardest first.
solvedpartial creditno credit (no better than the best shortcut)
One run per task, so any single square is noisy.
Gemini takes far longer
We gave Gemini six hours per task and the others one hour. Most of Gemini’s runs went through a subscription, so some of that time may be rate limiting. If we count Gemini’s attempts over an hour as zero, its score drops to 0.30.
Similar score, sixteen times slower
Share of attempts finished by each time. Solid lines: unprompted; dashed: told what to look for.
Gemini 4 Argon, median 53.8 minOpus 5.5, 3.4 minGPT 6.1 Sol, 2.3 min
Line swaps and well-order errors get missed
For most kinds of error, at least one agent gets nearly every affected item right. For line swaps, where two rows or columns trade places, and for wrong well order, no agent gets more than 27% right.
Two kinds of error get past every agent
Accuracy on each kind of error, when not told what to look for.
GPT 6.1 SolOpus 5.5Gemini 4 Argon
We think the two are missed for different reasons. A line swap moves only two rows or two columns, fewer than one well in eight, so the plate as a whole still lines up with the records. A well-order error scrambles nearly every compound well, so the plate stops matching earlier screens, but the controls stay where they should be. The plate passes a control check, and the mismatch is easy to put down to biology. In one task, Opus did exactly that and never tried a different well order.
Most wrong answers come straight from the records
When an agent gets an affected item wrong, it has usually reported the value from the records. That’s true of 90% of Opus’s wrong answers on these items, 79% of GPT 6.1 Sol’s and 58% of Gemini’s. Opus writes about a possible swap, rotation or other plate error in 167 of its 182 runs, yet still scores below 0.5 on 77 of them.
Work with us
We’re keeping the tasks and answer keys private. We can build more tasks like these from our lab data. Get in touch if you want evaluation access or training data.
Full methods, results for each kind of error and per-task scores are in the preprint.