Why don’t machine learning research agents overfit?

Why we picked it · the editor's summary

An explorer agent iterated against the same validation set for hundreds of rounds, and on most of eight datasets a fresh agent that never saw that set matched its result from a 32-token description of the strategy. Bertran Lopez and Roth argue this is why benchmark gains transfer: a strategy that short cannot have memorized the held-out examples. When they gave agents direct validation access to force overfitting, 38 of 102 runs scored more than 10% above true held-out accuracy, and compression removed those gains. The argument assumes no side channel such as validation data seen in pretraining. If agents iterate on your eval set, a validation-blind rebuild from a short summary is a cheap test of whether the gains are real.