加入
CYP3A4 inhibition from structure
Classify whether a compound inhibits the CYP3A4 enzyme, scored by AUPRC over 12,328 compounds.
shoal 2周前提交
0 个父提交
提交 971bb25d4eae4702222a9a22866ccb66dcb87510
+55 −0 README.md 新增 查看文件
| @@ -0,0 +1,55 @@ | ||
| 1 | +# CYP3A4 inhibition from structure | |
| 2 | + | |
| 3 | +CYP3A4 metabolises a large share of marketed drugs, so a compound that blocks it can change how every other drug a patient takes is cleared. This benchmark asks whether a compound is an inhibitor, from its structure. | |
| 4 | + | |
| 5 | +## Task | |
| 6 | + | |
| 7 | +Given a compound as a SMILES string, predict the binary CYP3A4 inhibition label. Submissions are scored by the area under the precision-recall curve (AUPRC), which TDC prefers to AUROC when positives are the minority class; higher is better. | |
| 8 | + | |
| 9 | +## Background | |
| 10 | + | |
| 11 | +Veith et al. (2009) screened tens of thousands of compounds against five cytochrome P450 isozymes in a quantitative high-throughput assay; TDC turns the CYP3A4 arm into a 12,328-compound classification benchmark. Inhibition of this enzyme is the usual mechanism behind drug-drug interactions, which is why it is tested early. With over twelve thousand labelled compounds from a single assay, it is one of the few ADMET endpoints where the data, not the label noise, is the limiting factor. | |
| 12 | + | |
| 13 | +## Data and evaluation | |
| 14 | + | |
| 15 | +TDC ships the group as one archive (`admet_group`, Harvard Dataverse file 4426004), loaded with | |
| 16 | +`admet_group(path='data/')` from the `tdc.benchmark_group` module. The group’s `cyp3a4_veith` folder holds 9,861 `train_val` and 2,467 `test` rows (12,328 distinct SMILES, matching TDC’s page); 41.5% of labels are positive, and 1,082 of the 2,467 test compounds are. | |
| 17 | + | |
| 18 | +The split is a scaffold split: TDC holds out a fixed 20% test set (here 2,467 of 12,328 rows in | |
| 19 | +the group’s files), and the remaining `train_val` part is split again per seed by | |
| 20 | +`group.get_train_valid_split(benchmark=name, split_type='default', seed=seed)` into 87.5% training | |
| 21 | +and 12.5% validation. The leaderboard asks for five independent runs with seeds 1, 2, 3, 4 and 5; | |
| 22 | +`group.evaluate_many` then reports the mean and the standard deviation of the per-seed test scores, | |
| 23 | +rounded to three decimals. The test set never changes, so only the training split moves with the seed. | |
| 24 | + | |
| 25 | +The metric is AUPRC, the area under the precision-recall curve, computed by TDC’s `Evaluator(name='pr-auc')`; higher is better. TDC uses it in place of AUROC where the positive class is the minority, because it is the more demanding summary there. Submissions pass continuous scores. | |
| 26 | + | |
| 27 | +Official evaluation code: [`tdc/benchmark_group/base_group.py`](https://github.com/mims-harvard/TDC/blob/main/tdc/benchmark_group/base_group.py) | |
| 28 | +in the TDC repository (MIT licence), whose `evaluate` and `evaluate_many` methods are what the | |
| 29 | +leaderboard numbers come from. The dataset page gives the licence as CC BY 4.0. | |
| 30 | + | |
| 31 | +## What is known | |
| 32 | + | |
| 33 | +- Best on the leaderboard: MapLight + GNN, AUPRC $0.916 \pm 0.000$, fingerprints and descriptors with graph isomorphism network features (Notwell and Wood, 2023). | |
| 34 | +- Next: ContextPred, $0.904 \pm 0.002$, and AttrMasking, $0.902 \pm 0.002$, both pretrained graph networks (Hu et al., 2020). | |
| 35 | +- Nineteen entries are listed, from AUPRC $0.916$ down to $0.654$. The spread is wider than on the small endpoints in this group, so differences here are easier to believe. | |
| 36 | + | |
| 37 | +## What would count as progress | |
| 38 | + | |
| 39 | +- A mean AUPRC on TDC’s fixed test set that beats MapLight + GNN (AUPRC 0.916), averaged over the official | |
| 40 | + seeds 1-5 and produced by `group.evaluate_many`, with the five per-seed numbers reported as well. | |
| 41 | +- The training and inference code, the environment and the fitted model, so that the five runs can be | |
| 42 | + repeated end to end from the `train_val` part alone. | |
| 43 | +- Evidence that the gain is larger than run-to-run noise: the gap between the top two entries here is | |
| 44 | + 0.012 AUPRC, so a difference smaller than the reported standard deviation is not a result. | |
| 45 | + | |
| 46 | +## References | |
| 47 | + | |
| 48 | +1. Veith, H. et al., "Comprehensive characterization of cytochrome P450 isozyme selectivity across chemical libraries", *Nature Biotechnology* 27 (2009), 1050–1055. [doi:10.1038/nbt.1581](https://doi.org/10.1038/nbt.1581) | |
| 49 | +2. Notwell, J. H. and Wood, M. W., "ADMET property prediction through combinations of molecular fingerprints", arXiv:2310.00174 (2023). [arxiv.org/abs/2310.00174](https://arxiv.org/abs/2310.00174) | |
| 50 | +3. Hu, W., Liu, B., Gomes, J., Zitnik, M., Liang, P., Pande, V. and Leskovec, J., "Strategies for Pre-training Graph Neural Networks", *ICLR* (2020), arXiv:1905.12265. [arxiv.org/abs/1905.12265](https://arxiv.org/abs/1905.12265) | |
| 51 | +4. Huang, K. et al., "Artificial intelligence foundation for therapeutic science", *Nature Chemical Biology* 18 (2022), 1033–1036. [doi:10.1038/s41589-022-01131-2](https://doi.org/10.1038/s41589-022-01131-2) | |
| 52 | + | |
| 53 | +## Source | |
| 54 | + | |
| 55 | +Imported from [TDC ADMET benchmark group, CYP3A4_Veith](https://tdcommons.ai/benchmark/admet_group/12cyp3a4i/), curated by Therapeutics Data Commons (Harvard). Open according to the source as of 2026-10-20. The statement above is written in our own words; see the source for the original wording, history and discussion. | |