arboreto
Infer gene regulatory networks (GRNs) from gene expression data using scalable algorithms (GRNBoost2, GENIE3). Use when analyzing transcriptomics data (bulk RNA-seq, single-cell RNA-seq) to identify transcription factor-target gene relationships and regulatory interactions. Supports distributed computation for large-scale datasets.
Install this skill
Works with Claude Code, Codex, Cursor, Gemini CLI, GitHub Copilot, OpenCode, and 70+ other harnesses.
npx skills add KalarisLabs/research-agent-skills --skill arboretouvx research-agent-skills install arboretoDocumentation & Instructions
Arboreto
Overview
Arboreto is a Python library from Aerts Lab for inferring gene regulatory networks (GRNs) from gene expression data. It parallelizes tree-based ensemble regression (GRNBoost2, GENIE3) with Dask across local cores or remote clusters.
Core capability: Identify which transcription factors (TFs) regulate which target genes based on expression patterns across observations (cells, samples, conditions).
Upstream: PyPI 0.1.6 (2021-02-09, latest). Docs: arboreto.readthedocs.io. Primary downstream consumer: pySCENIC.
Quick Start
Install arboreto:
uv pip install arboreto
Basic GRN inference:
import pandas as pd
from arboreto.algo import grnboost2
if __name__ == '__main__':
# Load expression data (genes as columns)
expression_matrix = pd.read_csv('expression_data.tsv', sep='\t')
# Infer regulatory network
network = grnboost2(expression_data=expression_matrix)
# Save results (TF, target, importance)
network.to_csv('network.tsv', sep='\t', index=False, header=False)
Critical: Always use if __name__ == '__main__': guard because Dask spawns new processes.
Core Capabilities
1. Basic GRN Inference
For standard GRN inference workflows including:
- Input data preparation (Pandas DataFrame or NumPy array)
- Running inference with GRNBoost2 or GENIE3
- Filtering by transcription factors
- Output format and interpretation
See: references/basic_inference.md
Use the ready-to-run script: scripts/basic_grn_inference.py for standard inference tasks:
python scripts/basic_grn_inference.py expression_data.tsv output_network.tsv --tf-file tfs.txt --seed 777 --limit 5000
2. Algorithm Selection
Arboreto provides two algorithms:
GRNBoost2 (Recommended):
- Fast gradient boosting-based inference
- Optimized for large datasets (10k+ observations)
- Default choice for most analyses
GENIE3:
- Random Forest-based inference
- Original multiple regression approach
- Use for comparison or validation
Quick comparison:
from arboreto.algo import grnboost2, genie3
# Fast, recommended
network_grnboost = grnboost2(expression_data=matrix)
# Classic algorithm
network_genie3 = genie3(expression_data=matrix)
For detailed algorithm comparison, parameters, and selection guidance: references/algorithms.md
3. Distributed Computing
Scale inference from local multi-core to cluster environments:
Local (default) - Uses all available cores automatically:
network = grnboost2(expression_data=matrix)
Custom local client - Control resources:
from distributed import LocalCluster, Client
local_cluster = LocalCluster(n_workers=10, memory_limit='8GB')
client = Client(local_cluster)
network = grnboost2(expression_data=matrix, client_or_address=client)
client.close()
local_cluster.close()
Cluster computing - Connect to remote Dask scheduler:
from distributed import Client
client = Client('tcp://scheduler:8786')
network = grnboost2(expression_data=matrix, client_or_address=client)
For cluster setup, performance optimization, and large-scale workflows: references/distributed_computing.md
Installation
uv pip install arboreto
Conda (Bioconda):
conda install -c bioconda arboreto
Dependencies (from upstream requirements.txt): dask[complete], distributed, numpy, pandas, scikit-learn, scipy
Input formats: pandas DataFrame, dense numpy.ndarray, or sparse scipy.sparse.csc_matrix (rows = observations, columns = genes). For array/matrix inputs, pass gene_names explicitly.
Common Use Cases
Single-Cell RNA-seq Analysis
import pandas as pd
from arboreto.algo import grnboost2
if __name__ == '__main__':
# Load single-cell expression matrix (cells x genes)
sc_data = pd.read_csv('scrna_counts.tsv', sep='\t')
# Infer cell-type-specific regulatory network
network = grnboost2(expression_data=sc_data, seed=42)
# Filter high-confidence links
high_confidence = network[network['importance'] > 0.5]
high_confidence.to_csv('grn_high_confidence.tsv', sep='\t', index=False)
Bulk RNA-seq with TF Filtering
from arboreto.utils import load_tf_names
from arboreto.algo import grnboost2
if __name__ == '__main__':
# Load data
expression_data = pd.read_csv('rnaseq_tpm.tsv', sep='\t')
tf_names = load_tf_names('human_tfs.txt')
# Infer with TF restriction
network = grnboost2(
expression_data=expression_data,
tf_names=tf_names,
seed=123
)
network.to_csv('tf_target_network.tsv', sep='\t', index=False)
Comparative Analysis (Multiple Conditions)
from arboreto.algo import grnboost2
if __name__ == '__main__':
# Infer networks for different conditions
conditions = ['control', 'treatment_24h', 'treatment_48h']
for condition in conditions:
data = pd.read_csv(f'{condition}_expression.tsv', sep='\t')
network = grnboost2(expression_data=data, seed=42)
network.to_csv(f'{condition}_network.tsv', sep='\t', index=False)
Output Interpretation
Arboreto returns a DataFrame with regulatory links:
| Column | Description |
|---|---|
TF |
Transcription factor (regulator) |
target |
Target gene |
importance |
Regulatory importance score (higher = stronger) |
Filtering strategy:
limit=Nat inference time (return top N links globally)- Post-hoc importance threshold (e.g., > 0.5)
- Top links per target via
groupby('target') - Statistical significance testing (permutation tests, external tools)
Integration with pySCENIC
Arboreto powers the GRN inference step in pySCENIC. pySCENIC 0.11+ passes sparse expression matrices to grnboost2 / genie3; pySCENIC 0.12+ defaults to arboreto_with_multiprocessing.py (no Dask) for compatibility — use standalone arboreto when you need Dask scaling.
# Standalone: infer co-expression modules before pySCENIC cisTarget pruning
from arboreto.algo import grnboost2
network = grnboost2(expression_data=expression_df, tf_names=tf_list, limit=5000)
# Downstream: pySCENIC ctx pruning, regulon definition, AUCell (see pySCENIC docs)
Convert AnnData to a DataFrame for arboreto directly:
expression_df = adata.to_df() # cells x genes
Reproducibility
Always set a seed for reproducible results:
network = grnboost2(expression_data=matrix, seed=777)
Run multiple seeds for robustness analysis:
from distributed import LocalCluster, Client
if __name__ == '__main__':
client = Client(LocalCluster())
seeds = [42, 123, 777]
networks = []
for seed in seeds:
net = grnboost2(expression_data=matrix, client_or_address=client, seed=seed)
networks.append(net)
# Consensus: links recurring across runs (example: mean importance per TF-target pair)
import pandas as pd
combined = pd.concat(networks)
consensus = (
combined.groupby(['TF', 'target'], as_index=False)['importance']
.mean()
.query('importance > 0.5')
)
Troubleshooting
Memory errors: Reduce dataset size by filtering low-variance genes or use distributed computing
Slow performance: Use GRNBoost2 instead of GENIE3, enable distributed client, filter TF list
Dask errors: Ensure if __name__ == '__main__': guard is present in scripts (required on Windows/macOS with spawn-based multiprocessing)
Empty results: Check data format (genes as columns), verify TF names match column names in the expression matrix
Sparse data: Use scipy.sparse.csc_matrix and pass matching gene_names; supported since arboreto 0.1.6 / pySCENIC 0.11
Agent operating procedure
- Check the environment. Confirm tool versions, the reference genome/annotation build and the input formats (FASTQ, BAM, VCF, h5ad).
- Pin down the inputs. Confirm formats, identifiers and parameters from the data or the user. Ask rather than guess any value that changes the result.
- Run a small version first. Run the pipeline on a small subset (one sample, one chromosome, a few thousand cells) first.
- Execute the full task using the instructions and references above.
- Validate the result. Check QC metrics, sample identities, genome build consistency and batch effects before interpreting results.
- Report. State what was run (versions, commands, parameters), what was checked, and what is still uncertain.
| If this happens | Do this |
|---|---|
| Genome builds or identifiers do not match between inputs | Stop and harmonize (liftover, ID mapping) before continuing. |
| A function, flag or endpoint in these instructions is missing in the installed version | Check the installed version's own documentation (help(), --help, official docs), adapt, and tell the user. Never invent an API. |
| A required input, identifier or parameter is ambiguous | Ask the user, or state the assumption explicitly before running. |
Integrity rules
- Never fabricate results, parameters, identifiers, citations or statistics. If something cannot be run or verified, say so plainly.
- Do not interpret biological significance beyond what the statistics support; report multiple-testing correction.
- Treat version-specific details here as possibly outdated: confirm them against the official documentation for the installed version.
- Ask before actions that cost money, consume shared GPUs or cloud quota, touch personal or patient data, or cannot be undone.
Related skills
cellxgene-census: Query the CZ CELLxGENE Census programmatically for versioned public single-cell and spatial transcriptomics data.pydeseq2: Runs differential expression analysis on bulk RNA-seq count data with PyDESeq2, the Python port of DESeq2.bulk-rnaseq: End-to-end bulk RNA-seq orchestrator — takes raw FASTQ reads through QC and trimming (FastQC, fastp/Trim Galore), alignment and quantificat…
Trigger Prompts & Evaluation Corpus
These evaluation prompts test that the agent correctly selects arboreto for relevant tasks and leaves it out for near misses.
Should trigger (3)
- should trigger
“How do I infer a TF-target network from my expression matrix with GRNBoost2?”
expects Shows grnboost2 usage on a pandas DataFrame with rows as observations and genes as columns, optionally passing a tf_names list. Includes the `if __name__ == '__main__':` guard, sets a seed, and explains the output columns TF, target and importance.
- should trigger
“I have a 12,000-cell scRNA-seq dataset in pbmc.h5ad and a list of human TFs in allTFs_hg38.txt. I ran grnboost2 in a script on Windows and got a Dask error: 'An attempt has been made to start a new process before the current process has finished its bootstrapping phase.' How do I fix this and run it properly?”
expects Identifies the missing `if __name__ == '__main__':` guard as the cause (Windows spawn-based multiprocessing) and shows how to wrap the code. Converts AnnData to a DataFrame (or sparse csc_matrix with gene_names), loads the TF list, sets a seed, tests on a small subset first, and optionally configures a local Dask Client.
- should trigger
“I want to figure out which transcription factors are likely driving gene expression changes across my 200 bulk liver samples. I'd like a ranked list of regulator-to-target gene links I can then feed into a downstream regulon analysis. What should I do?”
expects Recommends co-expression-based regulatory network inference with arboreto (GRNBoost2, optionally GENIE3 for comparison) restricted to a TF list. Explains filtering by importance or top links per target, running multiple seeds for robustness, and that the importance scores are not statistical significance. Mentions handing results to pySCENIC.
Near misses (2)
- near miss
“I have bulk RNA-seq raw count tables for treated vs control mice. Which genes are significantly differentially expressed, with FDR correction and a volcano plot?”
expects Should use pydeseq2 for differential expression with a design formula, Wald test, Benjamini-Hochberg FDR and a volcano plot. It is not GRN inference, so arboreto should not be used.
- near miss
“Pull all the T cells from the CZ CELLxGENE Census for human lung and give me the expression data for these 50 genes.”
expects Should use cellxgene-census to query Census metadata and gene expression slices. No regulatory network inference is requested, so arboreto does not apply.