scPertEval — Evaluation Protocols for Perturbation Sequencing#
scPertEval is a command-line tool for experimenting with and sharing reference implementations of evaluation protocols in single-cell perturbation studies.
It is introduced in Towards Principled Evaluation of Single-Cell Perturbation Prediction Models [Schäfer et al., 2026] by Philipp S. L. Schäfer, Kendall A. Reid, Zach Boldyga, Ekin D. Aksu, Hugo Hakem, and Julio Saez-Rodriguez — please cite it if you use this package.
Evaluating predictions across a dataset’s perturbations reduces to a single question: how different is one group of cells from another? To answer this, an evaluation protocol is defined: a specific formulation of a metric, along with some representation of the perturbation data fed to the metric. However, there are a multitude of possibilities — many already reflected in the literature — and it can be challenging to compare and contrast protocols across the field and ultimately choose the right approach for a given dataset and problem space.
scPertEval renders each protocol as a short, readable building block to run, read, reuse, and contribute back — a place for collaboration and alignment in the field. The same catalog of protocols backs three commands:
score— score a model’s predictions against ground truth, one metric value per perturbation (see Scoring predictions).calibrate— calibrate a protocol against built-in positive/negative controls, reporting the Dynamic Range Fraction (DRF) and Bound Discrimination Score (BDS) — how well it separates real signal from an uninformative baseline (see Calibration).de— export per-gene differential expression to HDF5.
Quick start#
pip install scperteval
scperteval calibrate data/wessels23.h5ad -p all --de-method t-test
Get scPertEval installed and set up your development environment.
Learn how to run protocols, interpret scores, and explore the building blocks.
Step-by-step notebooks: CLI walkthrough, Python API, and extending the tool.
Full reference for the Python API.
Browse the source code, open issues, or contribute a pull request.
Citation#
If you use scPertEval, please cite [Schäfer et al., 2026].
@unpublished{Schafer_2026,
author = {Schäfer, Philipp S. L. and Reid, Kendall A. and Boldyga, Zach
and Aksu, Ekin Deniz and Hakem, Hugo and Saez-Rodriguez, Julio},
title = {Towards a Principled Evaluation of Single-Cell Perturbation
Response Prediction Models},
note = {In preparation},
year = {2026},
}