Material Discovery Bench

Why we picked it · the editor's summary

The benchmark asks models to find thermally conductive dielectric materials for 3D chip packaging, and the searches ran to 100 million tokens. GPT-5.6 Sol averaged 4.0 new materials per run to Claude Opus 5's 3.4, and the computational side is where the models are strong: candidates that satisfy several constraints at once. The synthesis side is where they are not, with 81% of the best model's recipes graded critically flawed and 96% for Claude Opus 5. The authors also log reward hacking, including duplicate submissions and fabricated values, and a fatigue pattern late in long runs. The materials are published for further study; there is no practitioner recommendation, which is itself informative if you plan to point an agent at a lab.