Curation (Python)#

raven-toolbox objects in raven_toolbox.curation.

Functions#

Function	Summary
`batch_curate`	Add or update metabolites, reactions and genes from data tables.
`batch_curate_from_tsv`	File-path convenience wrapper for `batch_curate`.
`CurationResult`	Record of what `batch_curate` added and updated.
`download_genome_data`	Download the GFF3 annotation and protein FASTA for an NCBI genome assembly.
`get_gene_data`	Build a gene-mapping table from NCBI genome annotation files.
`parse_gff_gene_table`	Parse a GFF3 file into a gene-protein mapping table.
`process_protein_fasta_file`	Rename a protein FASTA's headers using a gene mapping table.
`rename_model_genes`	Rename `model`'s genes in place using a mapping table.
`RenameGenesResult`	Result of a gene-renaming pass.

Reference#

batch_curate#

batch_curate(model: cobra.Model, *, mets_df: pd.DataFrame | None=None, genes_df: pd.DataFrame | None=None, rxns_df: pd.DataFrame | None=None, rxns_coeffs_df: pd.DataFrame | None=None, met_id_prefix: str='M_', rxn_id_prefix: str='R_', overwrite_with_empty: bool=True)

Add or update metabolites, reactions and genes from data tables.

Each table is optional. rxns_df and rxns_coeffs_df must be provided together (one describes the per-reaction attributes, the other carries the stoichiometric coefficients).

Parameters

model: Model to curate in place.
mets_df, genes_df, rxns_df, rxns_coeffs_df: Tables matching the schema described in raven_toolbox.curation.
met_id_prefix, rxn_id_prefix: Prefixes for new metabolite and reaction ids (for example s_ and r_ for yeast-GEM, M_ and R_ for the cobrapy and BiGG default). New entity ids are formed by finding the largest existing zero-padded suffix matching the prefix and incrementing from there.
overwrite_with_empty: How an empty cell is applied to an entity that already exists in the model, as curateModelFromTables's overwriteWithEmpty. With True (default) a column present in the table overwrites the stored value, so an empty cell clears it: formula, gene short name, reaction name, subsystem, EC code, notes and InChI become empty. For charge and confidence score an empty cell becomes 0 when at least one cell of the column has a value, and a column without any value is not written. For a MIRIAM column an empty cell removes the namespace from the existing entity when at least one existing entity of the table has a value in that column. With False an empty cell leaves the stored value as it is. Independent of this option, an empty cell never changes a gene rule or a lower or upper bound (no bound given). A new entity takes the cells that have a value; an empty charge or confidence score is 0 when the column has a value in some row of the table. Must be a bool or numpy.bool_, otherwise TypeError is raised.

`Numeric cells (charge, confidence score, lower and upper bound, reversibility, ``rxnIdx``` :

and coefficient) are read as RAVEN reads them: an empty cell or ``NaN`` (any case, with an :

optional sign) is not given, and other text must be a plain decimal number or ``Inf``. A :

comma, a hexadecimal number or any expression raises :class:ValueError, and nothing is :

evaluated. ``true`` and ``false`` in any case are 1 and 0 in the lower bound, upper bound and :

```rev`` cells and are not numbers in the other columns. A cell is trimmed as MATLAB's` :

```strtrimtrims it, after dropping the backspaces at its start astextscan`` does; the` :

non-breaking space, NUL and the next line character stay in the text. :

A charge must be a finite whole number (``2``, ``-3``, ``2.0`` and ``2e0`` are accepted, :

```0.5, 2.5andInfraise :class:`ValueError`; nothing is rounded). Arev`` cell` :

must be 0, 1, ``true`` or ``false`` (any other number raises, although the value is not :

otherwise used). A ``rxnIdx`` cell must be a finite whole number of at least 1; an index :

appears in at most one row of the reaction table (the coefficient table repeats it, one row :

per metabolite). As in RAVEN, the two reaction tables are matched on index and name, and an :

index used by two rows of ``rxns_df`` raises :class:ValueError, which names the index and :

the data rows. The index is compared by value (``1``, ``1.0`` and ``1e0`` are the same). :

A coefficient must be a finite number other than zero; two rows of one reaction for the :

same metabolite add their coefficients. A row of a reaction table without any content is :

ignored (in the coefficient table the content is the index, reaction name, metabolite name :

and compartment); a row with content needs a ``rxnIdx`` when the table has that column, :

and a coefficient row needs a coefficient. A metabolite row needs a compartment. A :

reaction of one table without a row in the other raises :class:ValueError. :

An empty MIRIAM column removes no annotation. :

A lower bound above the upper bound (also against the stored bound of an existing :

reaction, since both bounds are set together) raises :class:ValueError. RAVEN's :

```curateModelFromTableswrites such bounds as given andsolveLP`` rejects the model when` :

it is run ("Invalid set of bounds for reaction(s): ..."); a cobra reaction cannot hold them. :

The following checks run before the model is touched, so a table that fails leaves the :

model unchanged: :

* the required columns; :

* the metabolites of the coefficient rows; :

* the bounds (several rows that update one reaction are checked one after the other); :

* a gene listed twice in ``genes_df``; :

* the gene rules; :

* the numeric cells, the charges, the ``rev`` cells and the ``rxnIdx`` cells described: above.

Errors that cobra raises while the tables are applied are not checked beforehand, and the :

model may then be partly changed. :

A ``grRules`` cell must be a well-formed gene rule, in the grammar of RAVEN's :

```parseGrRule: genes joined by andororin any case or by&, &&, |`` and` :

```||``, with balanced parentheses. A rule with an operator that lacks an operand, empty` :

parentheses or two genes without an operator between them raises :class:ValueError, as it :

raises ``RAVEN:badInput`` there. The rule of a new reaction that names a gene which neither :

the model nor ``genes_df`` has also raises :class:ValueError; RAVEN's ``addRxns`` is :

called without permission to add genes and stops with an error of its own. The rule of an :

`existing reaction may name a gene the model does not have. A gene id that ``genes_df``` :

lists more than once raises :class:ValueError, whether the model has the gene or not. :

The reaction gets the rule as this grammar reads it, so ``G1&G2`` is one gene, and it stays :

one gene through :func:~raven_toolbox.io.write_yaml_model and :

:func:~raven_toolbox.io.read_yaml_model and through cobra's SBML files. cobra's JSON :

reader reads the rule with cobra's own grammar and splits ``G1&G2`` into ``G1`` and ``G2``. :

A reaction table with a header and no row is empty: when both reaction tables are empty :

nothing happens, and a coefficient row without a reaction row (or the reverse) raises :

:class:ValueError. :

Returns

CurationResult: A summary of the changes.

batch_curate_from_tsv#

File-path convenience wrapper for batch_curate.

Each path is optional. A table file is read as RAVEN's curateModelFromTables reads it: the first line is the header (also when it is blank, which leaves no usable column) and its names are trimmed and lose one pair of surrounding double quotes; a column with an empty header is ignored; when two columns match the same core name the first one is used and the others are ignored, and a repeated name of another column gets .1, .2 ... appended (RAVEN uses both columns). The rest is tab-delimited text with every cell kept as text (NA, null or None stay text and 22 stays 22), a double quote around a cell, spaces before a cell dropped, a trailing tab on a row ignored, and a row with more non-empty cells than the header has columns refused with ValueError (RAVEN wraps the extra cell into a new row). A line without any text is skipped; RAVEN makes an empty row from a line that holds only "" or from a row that ends in two or more tabs.

Parameters

model : cobra.Model: Model to curate in place.
mets_tsv : str or Path or None: Path of the metabolite table. Default None, which skips the metabolite table.
genes_tsv : str or Path or None: Path of the gene table. Default None, which skips the gene table.
rxns_tsv : str or Path or None: Path of the reaction table. It must be given together with rxns_coeffs_tsv, otherwise batch_curate raises ValueError. Default None.
rxns_coeffs_tsv : str or Path or None: Path of the table with the stoichiometric coefficients of the reactions. It must be given together with rxns_tsv. Default None.
met_id_prefix : str: Prefix of the ids of new metabolites, as in batch_curate. Default "M_".
rxn_id_prefix : str: Prefix of the ids of new reactions, as in batch_curate. Default "R_".
overwrite_with_empty : bool: How an empty cell is applied to an entity that already exists in the model, as in batch_curate. Default True, which lets an empty cell clear the stored value. With False an empty cell leaves the stored value as it is. A value that is not a bool or numpy.bool_ raises TypeError.

CurationResult#

Record of what batch_curate added and updated.

Each list holds the cobra ids of the affected entities, in the order they were processed.

download_genome_data#

download_genome_data(accession: str, *, output_dir: str | Path='.', verbose: bool=True)

Download the GFF3 annotation and protein FASTA for an NCBI genome assembly.

Retrieves both files in one archive via the NCBI Datasets v2 API. When both files are already present in output_dir, nothing is fetched; deleting them forces a new download.

Parameters

accession:: NCBI genome assembly accession, for example "GCF_000002595.2". Both RefSeq (GCF_) and GenBank (GCA_) prefixes are accepted (case-insensitively).
output_dir:: Directory to save the downloaded files in (default: current directory).
verbose:: Print download progress.

Returns

(gff_file, faa_file):: Paths to the downloaded GFF3 annotation and protein FASTA files.

Raises

ValueError: If accession does not start with GCF_ or GCA_.
RuntimeError: If the download or the extraction fails, or the archive lacks the GFF3 or the protein FASTA file.

get_gene_data#

get_gene_data(accession_or_path: str | Path, *, output_file: str | Path | None=None, download_dir: str | Path='.')

Build a gene-mapping table from NCBI genome annotation files.

Parameters

accession_or_path:: An NCBI assembly accession (downloaded with download_genome_data if not already present in download_dir), or a path to a local GFF3 file.
output_file:: If given, also save the table as a tab-delimited file.
download_dir:: Directory to download to when accession_or_path is an accession.

Returns

pandas.DataFrame: See parse_gff_gene_table.

Raises

ValueError: If accession_or_path is neither an existing file nor a recognised NCBI assembly accession.

parse_gff_gene_table#

parse_gff_gene_table(gff_path: str | Path)

Parse a GFF3 file into a gene-protein mapping table.

Extracts gene and CDS pairs. The protein accession is taken from each CDS's protein_id (or, failing that, a Dbxref=GenBank:... entry), and the owning gene is resolved through the Parent chain (CDS -> mRNA -> gene for eukaryotes, CDS -> gene for prokaryotes), so the GenBank_protein column matches a protein FASTA's headers.

Parameters

gff_path : str or Path: Path of an uncompressed GFF3 file, read as UTF-8. Comment lines (starting with #) and lines with fewer than 9 tab-separated fields are skipped.

Returns

pandas.DataFrame: Columns locus_tag, old_locus_tag, GeneID, gene_name, GenBank_protein, UniProt. Deduplicated (a CDS repeats once per exon) on every column except UniProt, the same key as getGeneData.m.

process_protein_fasta_file#

process_protein_fasta_file(faa_file: str | Path, gene_table: pd.DataFrame | str | Path, header_col: str, *, output_dir: str | Path='.')

Rename a protein FASTA's headers using a gene mapping table.

Replaces each sequence header with the value from header_col, matched via the GenBank_protein accession present in the original FASTA header (its first whitespace-delimited token). Sequences whose accession is not found in gene_table keep their original header.

Parameters

faa_file:: Path to the protein FASTA file (.faa) to process.
gene_table:: A DataFrame, or the path of a tab-delimited file that holds one (for example from raven_toolbox.curation.get_gene_data). It needs the columns "GenBank_protein" and header_col.
header_col:: Column whose values replace each FASTA header (for example "locus_tag").
output_dir:: Directory the processed FASTA is saved in. The output file name is the original base name with "_processed" appended before the extension.

Returns

Path: Path to the written processed FASTA file.

Raises

FileNotFoundError: If faa_file does not exist.
ValueError: If gene_table is missing "GenBank_protein" or header_col.

Notes

A row is skipped when its GenBank_protein is empty or missing. An empty or missing header_col value does not skip the row: the header of the matching sequence becomes an empty string, as in processProteinFastaFile.m.

rename_model_genes#

rename_model_genes(model: cobra.Model, gene_table: pd.DataFrame | str | Path, from_col: str, to_col: str)

Rename model's genes in place using a mapping table.

Parameters

model:: Model to rename genes in.
gene_table:: A DataFrame, or the path of a tab-delimited file that holds one.
from_col:: Column whose values match the ids currently in model.genes.
to_col:: Column whose values replace them.

Returns

RenameGenesResult :

Raises

ValueError: If from_col or to_col is not a column of gene_table.

RenameGenesResult#

Result of a gene-renaming pass.

Parameters

renamed:: Number of model genes for which a mapping was found and applied.
unmapped:: Model gene ids with no entry in from_col (or an empty to_col), left unchanged. Sorted, unique.