# Predicting DFT band gaps from crystal structure Band gap decides whether a crystal is useful as a semiconductor, a transparent conductor or a photocatalyst, and computing it costs a DFT run per material. This task asks for the computed gap from the structure, on 106,113 Materials Project entries. ## Task Given a crystal structure, predict the PBE band gap in electronvolts as reported by the Materials Project. Scoring is the mean absolute error over the five fixed folds. More than 40% of the entries are metals with zero gap, so the target is a spike at zero plus a broad tail. ## Background The labels are PBE gaps from the Materials Project, filtered by Matbench to entries within 150 meV of the hull and without noble gases. It is the second-largest task in the suite, large enough that graph networks clearly beat descriptor-based models. The task is also a reminder of a limit: PBE systematically underestimates measured gaps, so a model that predicts these labels perfectly still would not predict experiment, which is what the suite's `matbench_expt_gap` task is for. ## Data and evaluation - Data: `matbench_mp_gap.json.gz` from MPContribs-ML (106,113 records, each a pymatgen Structure and a PBE gap in eV), at or via `task.load()` in the `matbench` package; the gaps are Materials Project calculations. - Split: five folds fixed in `matbench_v0.1_validation.json`, built with scikit-learn's `KFold(n_splits=5, shuffle=True, random_state=18012019)`. Train on each fold's training set, tune inside it only, predict its test set, report the mean. Changing the folds invalidates the comparison. - Metric: mean absolute error in eV over the five test folds, lower is better. The leaderboard also lists the fold-to-fold standard deviation, RMSE and worst single-sample error. - Official evaluation code: the `matbench` package (MIT), ; the task's `record(fold, predictions)`, as in `mb.matbench_mp_gap.record(...)` on a `MatbenchBenchmark` `mb`, scores a fold and `mb.to_file(...)` writes the submission file. ## What is known - Top of the task leaderboard: coGN, mean MAE 0.1559 eV (standard deviation 0.0017). - Runners-up: DeeperGATGNN 0.1694, coNGN 0.1697 and ALIGNN 0.1861 eV. - A dummy regressor scores 1.3272 eV against a mean absolute deviation of 1.3271. - The leaders' worst-case errors stay around 7 eV, so some compounds are still missed by several electronvolts. - The leaderboard's repository has merged no submission since 20 January 2024; newer ones wait as open [pull requests](https://github.com/materialsproject/matbench/pulls), so the official numbers may lag what has been achieved. ## What would count as progress - A mean MAE below 0.1559 eV on the five fixed folds, beating coGN, submitted as the `matbench` benchmark file so every fold's predictions can be re-scored. - Code and a pinned environment that reproduce the run from the fixed folds. - Per-fold numbers, not just the mean, so that a gain can be compared with the leader fold by fold on the same folds; a small gain should hold on each fold or over repeated training runs. Reproducing the current leader counts too, since the board has not moved since 2024. ## References 1. A. Jain et al., "The Materials Project: A materials genome approach to accelerating materials innovation", *APL Materials* **1**, 011002 (2013). 2. A. Dunn, Q. Wang, A. Ganose, D. Dopp and A. Jain, "Benchmarking materials property prediction methods: the Matbench test set and Automatminer reference algorithm", *npj Computational Materials* **6**, 138 (2020). 3. R. Ruff, P. Reiser, J. Stühmer and P. Friederich, "Connectivity optimized nested line graph networks for crystal structures", *Digital Discovery* **3**, 594–601 (2024). ## Source Imported from [Matbench v0.1 matbench_mp_gap](https://matbench.materialsproject.org/Leaderboards%20Per-Task/matbench_v0.1_matbench_mp_gap/), curated by Materials Project (Dunn et al.). Open according to the source as of 2026-10-20. The statement above is written in our own words; see the source for the original wording, history and discussion.