Denoising single-cell RNA-seq counts

Single-cell RNA-seq captures only a small fraction of the mRNA molecules in each cell, so the observed counts are a noisy, sparse sample of true expression. This benchmark asks a method to estimate the true expression from those counts, and scores it without a ground truth by holding out a random part of the molecules.

Task

A method receives a cell-by-gene matrix of UMI counts, the training part of a molecule-level split. It must return a denoised matrix of the same shape: its estimate of each gene's expression in each cell. The estimate is compared with the held-out test molecules from the same cells.

Background

Denoising, often called imputation in the single-cell literature, means estimating the true expression level of each gene in each cell, and it is hard to evaluate because that true level is never observed. Batson, Royer and Webber (2019) proposed molecular cross-validation (MCV): split the observed molecules of every cell at random into a training and a test set, denoise the training set, and measure how well it predicts the test set. They showed that this tracks accuracy against a ground truth. The Open Problems task, by Wesley Lewis, Scott Gigante and Robrecht Cannoodt, uses MCV to compare methods such as MAGIC, ALRA, DCA and kNN smoothing.

Data and evaluation

• Data: three datasets in release v1.0.0: 1k peripheral blood mononuclear cells (10x Genomics, v3 chemistry), human pancreas (inDrop) and Tabula Muris Senis lung (10x). They are downloaded by the task's dataset loaders from public sources.
• Split: the molecules of every cell are split at random, 90% to training and 10% to test, with molecular_cross_validation.util.split_molecules and seed 0; genes with no training counts are dropped.
• Metric: two losses. The mean-squared error is taken after normalising both matrices to 10,000 counts per cell and applying $\log(1+x)$. The Poisson loss is the negative log-likelihood of the test counts given the denoised values, rescaled by the ratio of total test to total training counts. Lower is better for both. Each is rescaled per dataset between the no-denoising control (0) and the perfect-denoising control (1), clipped to $[0, 1]$, and the overall score is the mean over the 3 × 2 pairs, with failed runs counted as 0. The leaderboard shows it to two decimals; methods with several parameter sets are listed by their best and worst set.
• Code: release v1.0.0 lives in the legacy openproblems repository; the task now continues in openproblems-bio/task_denoising.

What is known

On the v1.0.0 leaderboard (as of 2026-10-20):

• MAGIC leads at 0.63 (0.6302 before rounding, recomputed from the published results) with its best parameter set (0.41 with its worst), followed by ALRA at 0.49.
• DCA (0.08), iterative kNN smoothing (0.07) and kNN smoothing (0.06) barely improve on leaving the data alone.
• A pre-release (v1.1.0-rc1) re-runs the task on 17 datasets in the new framework. There kNN smoothing leads at 0.13, MAGIC scores 0.10 and the no-denoising control 0.09, so on broader data no method is far from doing nothing. Scores between releases are not comparable.

What would count as progress

• An overall score above 0.6302 on the v1.0.0 datasets and split, computed with the release's evaluation code, with code that reproduces it.
• A method that clearly beats the no-denoising control on the 17-dataset benchmark in task_denoising, where the best method now scores 0.13.
• Gains on both losses, the mean-squared error and the Poisson loss, rather than on one of them.

References

1. Batson J, Royer L, Webber J. Molecular cross-validation for single-cell RNA-seq. bioRxiv 786269 (2019). doi:10.1101/786269
2. van Dijk D, Sharma R, Nainys J, et al. Recovering gene interactions from single-cell data using data diffusion. Cell 174, 716–729 (2018). doi:10.1016/j.cell.2018.05.061
3. Luecken MD, Gigante S, Burkhardt DB, Cannoodt R, et al. Defining and benchmarking open problems in single-cell analysis. Nature Biotechnology 43, 1035–1040 (2025). doi:10.1038/s41587-025-02694-w

Source

Imported from Open Problems in Single-Cell Analysis: Denoising (denoising), curated by the Open Problems in Single-Cell Analysis consortium. Open according to the source as of 2026-10-20 (leaderboard release v1.0.0). The statement above is written in our own words; see the source for the original wording, history and discussion.