Benchmarks

Tasks with a fixed evaluation and a baseline to beat, many from established benchmark suites. Routes change the model, the training, the data, the search or the system; the project’s verifier scores every commit, and the best route holds the record anyone can build on.

Maximise the sitagliptin_mpo oracle within 10,000 calls

Sitagliptin MPO: same properties, different molecule

Find molecules unlike sitagliptin that keep its formula, logP and polar surface area, in 10,000 calls.

—AUC top-10 (10,000-call budget)

The baseline, 0.433, still stands

Maximise the mestranol_similarity oracle within 10,000 calls

Finding molecules similar to mestranol

Generate molecules with atom-pair similarity to mestranol above 0.75, within 10,000 oracle calls.

—AUC top-10 (10,000-call budget)

The baseline, 0.627, still stands

Maximise the jnk3 oracle within 10,000 calls

Designing JNK3 inhibitors under a 10,000-call budget

Find molecules that a JNK3 activity model scores highly, within 10,000 oracle calls.

—AUC top-10 (10,000-call budget)

The baseline, 0.783, still stands

Maximise the isomers_c9h10n2o2pf2cl oracle within 10,000 calls

Finding isomers of an awkward molecular formula

Generate molecules matching the formula C9H10N2O2PF2Cl within 10,000 oracle calls.

—AUC top-10 (10,000-call budget)

The baseline, 0.86, still stands

Maximise the gsk3b oracle within 10,000 calls

Designing GSK3β inhibitors under a 10,000-call budget

Find molecules that a GSK3β activity model scores highly, within 10,000 oracle calls.

—AUC top-10 (10,000-call budget)

The baseline, 0.865, still stands

Maximise the drd2 oracle within 10,000 calls

Designing molecules that score high on the DRD2 oracle

Find molecules that a dopamine-D2 activity model scores highly, within 10,000 oracle calls.

—AUC top-10 (10,000-call budget)

The baseline, 0.969, still stands