Finding isomers of an awkward molecular formula

Given a molecular formula, can a generative model produce molecules that match it exactly? The formula here mixes phosphorus, fluorine and chlorine with a carbon skeleton, a combination that looks nothing like a typical drug-like molecule, which makes the task a test of flexibility rather than of recall.

Task

Propose molecules as SMILES strings and maximise the isomer oracle for $\mathrm{C}_9\mathrm{H}_{10}\mathrm{N}_2\mathrm{O}_2\mathrm{PF}_2\mathrm{Cl}$ . The score is the geometric mean of Gaussian-modified counts, one for each element in the target formula plus one for the total number of atoms, so every element must be right at once. Scored by AUC top-10 over a 10,000-call budget.

Background

The isomer benchmarks are GuacaMol's fully-determined tasks: the answer set can in principle be enumerated, and they have no direct drug-discovery use, but they expose whether a model can follow a simple pattern it has not seen. In PMO's results the landscape favours string-level search: genetic algorithms over SMILES and SELFIES lead it, while synthesis- and fragment-based methods score far lower. The paper's own explanation is that isomer oracles sum atomic contributions, unlike the fingerprint-similarity objectives that make up most of the benchmark, so a different family of methods wins.

Data and evaluation

Data: there is no training set to fit. Any database a method needs is restricted to ZINC 250k, which ships with the repository, so no method gets an advantage from a richer corpus.
Budget and seeds: 10,000 oracle calls per run, and five independent runs with different random seeds (python run.py METHOD --task production --n_runs 5 --oracles isomers_c9h10n2o2pf2cl). Report the mean and standard deviation over the runs. The budget counts distinct valid canonical SMILES: a molecule queried twice is served from the cache for free, and an invalid SMILES scores 0.
Metric: AUC top-10. The mean score of the ten best molecules found so far is recorded every 100 calls, the curve against the number of calls is integrated by the trapezoid rule and divided by the 10,000-call budget, so the value lands in $[0, 1]$ and higher is better. A run that stops early keeps its final top-10 value for the rest of the budget. Unlike a plain top-10 average, the metric rewards reaching a good score early.
Official evaluation code: https://github.com/wenhao-gao/mol_opt (MIT), where main/optimizer.py holds the budget accounting and the AUC. Oracles come from the Therapeutics Data Commons via PyTDC (MIT); the ZINC database is free to use.

What is known

Best in the PMO paper's table: SMILES GA at 0.860 ± 0.065, then STONED at 0.805 ± 0.031 and REINVENT on SELFIES at 0.733 ± 0.029.
Synthesis-based methods fare badly: SynNet scores 0.241 ± 0.064 and DoG-Gen 0.199 ± 0.016, less than a third of the leader.
Later work on the same protocol reports higher: Genetic GFN at 0.897 ± 0.007 and Mol GA at 0.894 ± 0.005 (five runs, 10,000 calls).
A random screen of ZINC 250k scores 0.106.

What would count as progress

A mean AUC top-10 above 0.86 on isomers_c9h10n2o2pf2cl from five runs under the official protocol, reported with the per-run values and the standard deviation. To beat the field rather than the 2022 table, the number to pass is 0.897 (Genetic GFN).
The run logs, which record every scored molecule in order, so the AUC and the oracle-call count can be recomputed rather than trusted.
Code in the repository's method layout, so the result can be re-run on the other 22 oracles: a method that wins on one oracle and loses elsewhere is the pattern PMO was built to expose.

References

W. Gao, T. Fu, J. Sun and C. W. Coley, "Sample Efficiency Matters: A Benchmark for Practical Molecular Optimization", NeurIPS 2022 Datasets and Benchmarks (2022). https://arxiv.org/abs/2206.12411
N. Brown, M. Fiscato, M. H. S. Segler and A. C. Vaucher, "GuacaMol: Benchmarking Models for de Novo Molecular Design", Journal of Chemical Information and Modeling 59, 1096–1108 (2019). https://doi.org/10.1021/acs.jcim.8b00839
W. Gao, R. Mercado and C. W. Coley, "Amortized Tree Generation for Bottom-up Synthesis Planning and Synthesizable Molecular Design", ICLR 2022 (2021). https://arxiv.org/abs/2110.06389
H. Kim, M. Kim, S. Choi and J. Park, "Genetic-guided GFlowNets for Sample Efficient Molecular Optimization", NeurIPS 2024 (2024). https://arxiv.org/abs/2402.05961
A. Tripp and J. M. Hernández-Lobato, "Genetic algorithms are strong baselines for molecule generation" (2023). https://arxiv.org/abs/2310.09267

Source

Imported from PMO (mol_opt), the isomers_c9h10n2o2pf2cl oracle, curated by Wenhao Gao, Tianfan Fu, Jimeng Sun and Connor W. Coley. Open according to the source as of 2026-10-20. The statement above is written in our own words; see the source for the original wording, history and discussion.