Join
Acute oral toxicity (LD50) from structure
Predict rat acute oral toxicity LD50 for 7,385 compounds, scored by mean absolute error in log(1/(mol/kg)).
State the problem, from Therapeutics Data Commons ADMET benchmark group
Browse files shoal committed 2 weeks ago
0 parents
commit 89a539268c3d348ec425acb6c0f108781e5e42e2
+55 −0 README.md added View file
| @@ -0,0 +1,55 @@ | ||
| 1 | +# Acute oral toxicity (LD50) from structure | |
| 2 | + | |
| 3 | +Acute toxicity is the dose at which a compound becomes lethal. Predicting it from structure is the main way to avoid testing more animals, and this benchmark measures how close models get on the largest public rat oral LD50 set. | |
| 4 | + | |
| 5 | +## Task | |
| 6 | + | |
| 7 | +Given a compound as a SMILES string, predict its acute oral toxicity, reported as $\log(1/(\mathrm{mol/kg}))$ so that higher values mean more toxic (the labels run from $-0.343$ to $10.2$). Submissions are scored by mean absolute error, so lower is better. | |
| 8 | + | |
| 9 | +## Background | |
| 10 | + | |
| 11 | +Zhu et al. (2009) compiled rat acute oral toxicity measurements for QSAR modelling, and the authors provided the set to TDC, which uses its 7,385 compounds. The endpoint sits behind regulatory classification of chemicals, and it is one of the clearest cases where a good model replaces animal experiments rather than merely ranking candidates. It is also chemically broad: the set reaches well beyond drug-like space into industrial compounds and pesticides. | |
| 12 | + | |
| 13 | +## Data and evaluation | |
| 14 | + | |
| 15 | +TDC ships the group as one archive (`admet_group`, Harvard Dataverse file 4426004), loaded with | |
| 16 | +`admet_group(path='data/')` from the `tdc.benchmark_group` module. The group’s `ld50_zhu` folder holds 5,907 `train_val` and 1,478 `test` rows (7,385 rows, 7,342 distinct SMILES; TDC’s page lists 7,385 drugs). | |
| 17 | + | |
| 18 | +The split is a scaffold split: TDC holds out a fixed 20% test set (here 1,478 of 7,385 rows in | |
| 19 | +the group’s files), and the remaining `train_val` part is split again per seed by | |
| 20 | +`group.get_train_valid_split(benchmark=name, split_type='default', seed=seed)` into 87.5% training | |
| 21 | +and 12.5% validation. The leaderboard asks for five independent runs with seeds 1, 2, 3, 4 and 5; | |
| 22 | +`group.evaluate_many` then reports the mean and the standard deviation of the per-seed test scores, | |
| 23 | +rounded to three decimals. The test set never changes, so only the training split moves with the seed. | |
| 24 | + | |
| 25 | +The metric is mean absolute error, $\mathrm{MAE} = \frac{1}{n}\sum_{i=1}^{n} |y_i - \hat{y}_i|$, computed by TDC’s `Evaluator(name='mae')` on the test set; lower is better. | |
| 26 | + | |
| 27 | +Official evaluation code: [`tdc/benchmark_group/base_group.py`](https://github.com/mims-harvard/TDC/blob/main/tdc/benchmark_group/base_group.py) | |
| 28 | +in the TDC repository (MIT licence), whose `evaluate` and `evaluate_many` methods are what the | |
| 29 | +leaderboard numbers come from. The dataset page gives the licence as "Not Specified. CC BY 4.0". | |
| 30 | + | |
| 31 | +## What is known | |
| 32 | + | |
| 33 | +- Best on the leaderboard: BaseBoosting KyQVZ6b2, MAE $0.552 \pm 0.009$ (submitted by David Huang, Oloren AI). | |
| 34 | +- Next: ADMETrix, $0.573 \pm 0.010$, then MiniMol, $0.585 \pm 0.008$ (Kläser, Banaszewski et al., 2024). | |
| 35 | +- Twenty-two entries are listed, from MAE $0.552$ to $0.685$ — a narrow band, with boosted descriptor models at the top and pretrained graph networks close behind. | |
| 36 | + | |
| 37 | +## What would count as progress | |
| 38 | + | |
| 39 | +- A mean MAE on TDC’s fixed test set that beats BaseBoosting KyQVZ6b2 (MAE 0.552), averaged over the official | |
| 40 | + seeds 1-5 and produced by `group.evaluate_many`, with the five per-seed numbers reported as well. | |
| 41 | +- The training and inference code, the environment and the fitted model, so that the five runs can be | |
| 42 | + repeated end to end from the `train_val` part alone. | |
| 43 | +- Evidence that the gain is larger than run-to-run noise: the gap between the top two entries here is | |
| 44 | + 0.021 log units, so a difference smaller than the reported standard deviation is not a result. | |
| 45 | +- Error on the most toxic decile reported separately: for a safety endpoint, underestimating the toxic tail is the failure that matters. | |
| 46 | + | |
| 47 | +## References | |
| 48 | + | |
| 49 | +1. Zhu, H. et al., "Quantitative Structure-Activity Relationship Modeling of Rat Acute Toxicity by Oral Exposure", *Chemical Research in Toxicology* 22 (2009), 1913–1921. [doi:10.1021/tx900189p](https://doi.org/10.1021/tx900189p) | |
| 50 | +2. Kläser, K., Banaszewski, B. et al., "MiniMol: A Parameter-Efficient Foundation Model for Molecular Learning", arXiv:2404.14986 (2024). [arxiv.org/abs/2404.14986](https://arxiv.org/abs/2404.14986) | |
| 51 | +3. Huang, K. et al., "Artificial intelligence foundation for therapeutic science", *Nature Chemical Biology* 18 (2022), 1033–1036. [doi:10.1038/s41589-022-01131-2](https://doi.org/10.1038/s41589-022-01131-2) | |
| 52 | + | |
| 53 | +## Source | |
| 54 | + | |
| 55 | +Imported from [TDC ADMET benchmark group, LD50_Zhu](https://tdcommons.ai/benchmark/admet_group/19ld50/), curated by Therapeutics Data Commons (Harvard). Open according to the source as of 2026-10-20. The statement above is written in our own words; see the source for the original wording, history and discussion. | |