Benchmarks

Tasks with a fixed evaluation and a baseline to beat, many from established benchmark suites. Routes change the model, the training, the data, the search or the system; the project’s verifier scores every commit, and the best route holds the record anyone can build on.

Predict steel yield strength from composition

Predicting steel yield strength from composition

Predict the yield strength of a steel in MPa from its chemical formula alone, over 312 measured alloys.

—MAE (MPa)

The baseline, 79.9468, still stands

Predict the highest optical phonon peak frequency

Predicting the highest phonon peak frequency of a crystal

Predict the frequency of the highest optical phonon peak of a crystal, in 1/cm, from its structure.

—MAE (cm^-1)

The baseline, 28.7606, still stands

Predict perovskite cell formation energy from structure

Predicting perovskite formation energy from structure

Predict the DFT heat of formation of a 5-atom perovskite cell in eV from its structure, over 18,928 cells.

—MAE (eV/unit cell)

The baseline, 0.0269, still stands

Predict PBE band gap from crystal structure

Predicting DFT band gaps from crystal structure

Predict the PBE band gap of a crystal in eV from its structure, over 106,113 Materials Project entries.

—MAE (eV)

The baseline, 0.1559, still stands

Predict formation energy from crystal structure

Predicting formation energy from crystal structure

Predict the DFT formation energy of a crystal in eV/atom from its structure, over 132,752 entries.

—MAE (eV/atom)

The baseline, 0.017, still stands

Predict log10 shear modulus from crystal structure

Predicting the shear modulus of a crystal

Predict the base-10 logarithm of the DFT Voigt-Reuss-Hill shear modulus of a crystal from its structure.

—MAE (log10 GPa)

The baseline, 0.067, still stands

Predict exfoliation energy from crystal structure

Predicting exfoliation energy of 2D materials

Predict the DFT exfoliation energy of a layered crystal in meV/atom from its structure, over 636 materials.

—MAE (meV/atom)

The baseline, 33.1918, still stands

Classify bulk metallic glass formation from composition

Predicting bulk metallic glass formation from composition

Classify whether an alloy composition forms a full bulk metallic glass, over 5,680 reported ternary alloys.

—ROC-AUC

The baseline, 0.9603, still stands

Predict experimental band gap from composition

Predicting experimental band gaps from a chemical formula

Predict the measured band gap of an inorganic solid in eV from its composition alone, over 4,604 compounds.

—MAE (eV)

The baseline, 0.2865, still stands

Predict refractive index from crystal structure

Predicting refractive index from crystal structure

Predict the computed refractive index of a crystal from its structure, over 4,764 Materials Project entries.

—MAE (refractive index, unitless)

The baseline, 0.2711, still stands