pyModelBlocks Module Index¶

Complete alphabetic API for all public classes and methods in this package, grouped by module.

mb.core.general.core module¶

class mb.core.general.core.Arg(key, dtype=<class 'str'>, positional=False, default=None, descr=None)[source]¶

Bases: object

Object representing an argument to a data transform (positional or keyword). Arguments parameterize all aspects of the transform except the input data, which should be computed as prerequisites in the dependency graph. Positional arguments are treated as obligatory, keyword arguments are treated as optional.

Prm key:	`str`; name of the argument
Prm dtype:	`type`; data type of the argument
Prm positional:	`bool`; whether the argument is positional
Prm default:	default value for the argument

class mb.core.general.core.ExternalResource[source]¶

Bases: mb.core.general.core.StaticResource

Abstract base class for dependency to external resource.

class mb.core.general.core.FailureSet[source]¶

Bases: set

add(other)[source]¶

Add an element to a set.

This has no effect if the element is already present.

class mb.core.general.core.MBFailure(path, cls)[source]¶

Bases: mb.core.general.core.MBType

Class to represent build failures.

Syntax:

(<DIR>/)

class mb.core.general.core.MBType(path)[source]¶

Bases: object

Abstract base class for ModelBlocks types.

class mb.core.general.core.ParamFile(path)[source]¶

Bases: mb.core.general.core.MBType

File containing configuration parameters for building targets.

Prerequisites:

(DIR/)<NAME>.<TYPE>prm.ini

Syntax:

(<DIR>/)prm.ini

class mb.core.general.core.Repo[source]¶

Bases: mb.core.general.core.ExternalResource

A corpus of naturalistic stories meant to contain varied, low-frequency syntactic constructions. There are a variety of annotations and psycholinguistic measures available for the stories.

URL: ` <>`_

Syntax:

.

class mb.core.general.core.StaticResource(path)[source]¶

Bases: mb.core.general.core.MBType

Abstract base class for dependency to a static resource.

Syntax:

.

class mb.core.general.core.SuccessSet[source]¶

Bases: set

add(other)[source]¶

Add an element to a set.

This has no effect if the element is already present.

mb.core.general.core.decrement_delimiters(s)[source]¶

Decrease the depth of all delimiters in s.

Prm s:	`str`; the input string
Returns:	`str`; s with shallower delimiters (shifted towards the beginning of DELIM)

mb.core.general.core.increment_delimiters(s)[source]¶

Increase the depth of all delimiters in s.

Prm s:	`str`; the input string
Returns:	`str`; s with deeper delimiters (shifted towards the end of DELIM)

mb.core.general.table module¶

class mb.core.general.table.EvMeasures(path)[source]¶

Bases: mb.core.general.core.MBType

Abstract base class for event measures (evmeasures) types.

class mb.core.general.table.EvMeasuresMerged(path)[source]¶

Bases: mb.core.general.table.EvMeasures

Merge of evmeasures with itemmeasures

Prerequisites:

ItemMeasures
(DIR/)<CORPUS>.evmeasures

Syntax:

(<DIR>/)<ItemMeasures>.evmeasures

class mb.core.general.table.ItemMeasures(path)[source]¶

Bases: mb.core.general.core.MBType

Abstract base class for item-by-item measures (itemmeasures) types.

class mb.core.general.table.ItemmeasuresConcat(path)[source]¶

Bases: mb.core.general.table.ItemMeasures

Itemmeasures from (column) concatenation of tables

Prerequisites:

ItemMeasures (repeatable)

Syntax:

(<DIR>/)(<SHARED_PRE>).<ItemMeasures>(.<ItemMeasures>)*.(<SHARED_POST>).concat.itemmeasures

class mb.core.general.table.ItemmeasuresRolled(path)[source]¶

Bases: mb.core.general.table.ItemMeasures

Itemmeasures rolled from tokmeasures

Prerequisites:

TokMeasures
(DIR/)<LINEITEMS>.lineitems

Syntax:

(<DIR>/)<TokMeasures>.itemmeasures

class mb.core.general.table.PrdMeasures(path)[source]¶

Bases: mb.core.general.core.MBType

Abstract base class for predictor measures (prdmeasures) types.

class mb.core.general.table.ResMeasures(path)[source]¶

Bases: mb.core.general.core.MBType

Abstract base class for response measures (resmeasures) types.

class mb.core.general.table.ResMeasuresReg(path)[source]¶

Bases: mb.core.general.table.ResMeasures

resmeasures for regression analysis

Prerequisites:

EvMeasures
prm/<CENS-PARAMS>.partprm.ini
prm/<CENS-PARAMS>.censprm.ini

Syntax:

(<DIR>/)<EvMeasures>.<CENS_PARAMS_FILE>-<PART_PARAMS_FILE>-<PARTITION_NAME>.resmeasures

class mb.core.general.table.TokMeasures(path)[source]¶

Bases: mb.core.general.core.MBType

Abstract base class for token-by-token measures (tokmeasures) types.

class mb.core.general.table.TokMeasuresDLT(path)[source]¶

Bases: mb.core.general.table.TokMeasures

Compute DLT (integration cost) measures from linetrees

Prerequisites:

(<DIR>/)<LineToks>.gold.linetrees
scripts/dlt.py

Syntax:

(<DIR>/)<GoldLineTrees>.dlt.tokmeasures

mb.core.general.text module¶

class mb.core.general.text.LineItems(path)[source]¶

Bases: mb.core.general.core.MBType

Abstract base class for lineitems types.

class mb.core.general.text.LineToks(path)[source]¶

Bases: mb.core.general.core.MBType

Abstract base class for linetoks types.

class mb.core.general.text.LineToksDelim(path)[source]¶

Bases: mb.core.general.text.LineToks

Add “!ARTICLE” delimiter to start of linetoks, if not already present

Prerequisites:

LineToks

Syntax:

(<DIR>/)<LineToks>.delim.linetoks

class mb.core.general.text.LineToksMorphed(path)[source]¶

Bases: mb.core.general.text.LineToks

Run morfessor over linetoks

Prerequisites:

LineToks

Syntax:

(<DIR>/)<LineToks>.morph.linetoks

class mb.core.general.text.LineToksReversed(path)[source]¶

Bases: mb.core.general.text.LineToks

Reverse linetoks

Prerequisites:

LineToks

Syntax:

(<DIR>/)<LineToks>.rev.linetoks

mb.core.regression.regression module¶

mb.core.regression.lmer module¶

class mb.core.regression.lmer.PredictionExecutableLMER(path)[source]¶

Bases: mb.core.regression.core.PredictionExecutable

Exectuable for prediction from a linear mixed-effects (LMER) regression model

Prerequisites:

scripts/predict-lmer.sh
scripts/lmer-tools.R
scripts/predict-lmer.R

Syntax:

(<DIR>/)predict-lmer

class mb.core.regression.lmer.PredictionLMER(path)[source]¶

Bases: mb.core.regression.core.Prediction

Predict from linear mixed-effects (LMER) regression

Prerequisites:

Regression
bin/predict-lmer
(DIR/)<EVMEASURES>.evmeasures(DIR/)<RESMEASURES>.resmeasures

Syntax:

(<DIR>/)<Regression>.<CENS_PARAMS_FILE>-<PART_PARAMS_FILE>-<TRAIN_PARTITION_NAME>-<MODEL_CONFIG_FILE>-<PREDICTORS>-<EVAL_PARTITION_NAME>-lmer.pred

class mb.core.regression.lmer.RegressionExecutableLMER(path)[source]¶

Bases: mb.core.regression.core.RegressionExecutable

Exectuable for fitting a linear mixed-effects (LMER) regression model

Prerequisites:

scripts/regress-lmer.sh
scripts/lmer-tools.R
scripts/regress-lmer.R

Syntax:

(<DIR>/)regress-lmer

class mb.core.regression.lmer.RegressionLMER(path)[source]¶

Bases: mb.core.regression.core.Regression

Run linear mixed-effects (LMER) regression

Prerequisites:

EvMeasures
bin/regress-lmer
(DIR/)<RESMEASURES>.resmeasuresprm/<NAME>.lmerprm.ini

Syntax:

(<DIR>/)<EvMeasures>.<CENS_PARAMS_FILE>-<PART_PARAMS_FILE>-<TRAIN_PARTITION_NAME>-<MODEL_CONFIG_FILE>-<PREDICTORS>-lmer.reg

class mb.core.regression.lmer.SignifExecutableLRT(path)[source]¶

Bases: mb.core.regression.core.SignifExecutable

Exectuable for computing likelihood ratio test (LRT)

Prerequisites:

scripts/signif-lrt.sh
scripts/lmer-tools.R
scripts/signif-lrt.R
scripts/signif.py

Syntax:

(<DIR>/)signif-lrt

class mb.core.regression.lmer.SignifLRT(path)[source]¶

Bases: mb.core.regression.core.Signif

Likelihood ratio significance test(s). Only available for LMER models.

Prerequisites:

(<DIR>/)<Regression>.<CENS_PARAMS_FILE>-<PART_PARAMS_FILE>-<TRAIN_PARTITION_NAME>-<MODEL_CONFIG_FILE>-<PREDICTORS>-<EVAL_PARTITION_NAME>-lmer.pred
bin/signif-lrt
(DIR/)<RESMEASURES>.resmeasuresprm/<NAME>.<REG>prm.ini

Syntax:

(<DIR>/)<PredictionLMER>.<CENS_PARAMS_FILE>-<PART_PARAMS_FILE>-<TRAIN_PARTITION_NAME>-<MODEL_CONFIG_FILE>-<PREDICTORS>-<EVAL_PARTITION_NAME>-<REGRESSION_TYPE>-lrt.sig

mb.core.regression.cdr module¶

class mb.core.regression.cdr.PredictionCDR(path)[source]¶

Bases: mb.core.regression.core.Prediction

Predict from continuous-time deconvolutional regression (LMER)

Prerequisites:

Regression
bin/predict-cdr
(DIR/)<EVMEASURES>.evmeasures(DIR/)<RESMEASURES>.resmeasures

Syntax:

(<DIR>/)<Regression>.<CENS_PARAMS_FILE>-<PART_PARAMS_FILE>-<TRAIN_PARTITION_NAME>-<MODEL_CONFIG_FILE>-<PREDICTORS>-<EVAL_PARTITION_NAME>-cdr.pred

class mb.core.regression.cdr.PredictionExecutableCDR(path)[source]¶

Bases: mb.core.regression.core.PredictionExecutable

Exectuable for prediction from a continuous-time deconvolutional regression (CDR) model

External resources:

/home/compling/cdr

Prerequisites:

scripts/predict-cdr.sh
/home/compling/cdr

Syntax:

(<DIR>/)predict-cdr

class mb.core.regression.cdr.RegressionCDR(path)[source]¶

Bases: mb.core.regression.core.Regression

Run continuous-time deconvolutional regression (LMER)

Prerequisites:

EvMeasures
bin/regress-cdr
(DIR/)<RESMEASURES>.resmeasuresprm/<NAME>.cdrprm.ini

Syntax:

(<DIR>/)<EvMeasures>.<CENS_PARAMS_FILE>-<PART_PARAMS_FILE>-<TRAIN_PARTITION_NAME>-<MODEL_CONFIG_FILE>-<PREDICTORS>-cdr.reg

class mb.core.regression.cdr.RegressionExecutableCDR(path)[source]¶

Bases: mb.core.regression.core.RegressionExecutable

Exectuable for fitting a continuous-time deconvolutional regression (CDR) model

External resources:

/home/compling/cdr

Prerequisites:

scripts/regress-cdr.sh
/home/compling/cdr

Syntax:

(<DIR>/)regress-cdr

mb.external_resources.cdr module¶

class mb.external_resources.cdr.CDRRepo[source]¶

Bases: mb.core.general.core.ExternalResource

A repository for estimating temporally diffuse effects from time series data.

URL: https://github.com/coryshain/cdr

Syntax:

/home/compling/cdr

mb.external_resources.natstor module¶

class mb.external_resources.natstor.EvMeasuresNatStor(path)[source]¶

Bases: mb.core.general.table.EvMeasures

Natural Stories base evmesaures

External resources:

/home/corpora/original/english/naturalstories/naturalstories_RTS/processed_RTs_MB.tsv

Prerequisites:

/home/corpora/original/english/naturalstories/naturalstories_RTS/processed_RTs_MB.tsv
(DIR/)naturalstories.mfields.lineitems

Syntax:

(<DIR>/)naturalstories.evmeasures

class mb.external_resources.natstor.ItemMeasuresNatstor(path)[source]¶

Bases: mb.core.general.table.ItemMeasures

Natural Stories base itemmeasures

External resources:

/home/corpora/original/english/naturalstories/naturalstories_RTS/all_stories.tok

Prerequisites:

/home/corpora/original/english/naturalstories/naturalstories_RTS/all_stories.tok
(DIR/)naturalstories.lineitems

Syntax:

(<DIR>/)naturalstories.itemmeasures

class mb.external_resources.natstor.ItemMeasuresNatstorMergeFields(path)[source]¶

Bases: mb.core.general.table.ItemMeasures

Natural Stories itemmeasures augmented with fields needed to merge with experimental measures (evmeasures)

External resources:

/home/corpora/original/english/naturalstories/naturalstories_RTS/all_stories.tok

Prerequisites:

/home/corpora/original/english/naturalstories/naturalstories_RTS/all_stories.tok
(DIR/)naturalstories.lineitems

Syntax:

(<DIR>/)naturalstories.mfields.itemmeasures

class mb.external_resources.natstor.ItemMeasuresNatstorTime(path)[source]¶

Bases: mb.core.general.table.ItemMeasures

Natural Stories timestamp itemmeasures

External resources:

/home/corpora/original/english/naturalstories/audio

Prerequisites:

(<DIR>/)naturalstories.itemmeasures
/home/corpora/original/english/naturalstories/audio

Syntax:

(<DIR>/)<ItemMeasuresNatstor>naturalstories.t.itemmeasures

class mb.external_resources.natstor.LineItemsNatstor(path)[source]¶

Bases: mb.core.general.text.LineItems

Natural Stories lineitems

External resources:

/home/corpora/original/english/naturalstories/naturalstories_RTS/all_stories.tok

Prerequisites:

/home/corpora/original/english/naturalstories/naturalstories_RTS/all_stories.tok
(DIR/)naturalstories.linetoks

Syntax:

(<DIR>/)naturalstories.lineitems

class mb.external_resources.natstor.LineToksNatstor(path)[source]¶

Bases: mb.core.general.text.LineToks

PTB Tokenized sentences (linetoks) for the Natural Stories corpus

Prerequisites:

(DIR/)naturalstories.ptb.linetrees

Syntax:

(<DIR>/)naturalstories.linetoks

class mb.external_resources.natstor.LineTreesNatstorPTB(path)[source]¶

Bases: mb.core.general.tree.LineTrees

Hand-annotated parse trees for the Natural Stories corpus.

External resources:

/home/corpora/original/english/naturalstories/parses/penn/all-parses.txt.penn

Prerequisites:

/home/corpora/original/english/naturalstories/parses/penn/all-parses.txt.penn
scripts/editabletrees2linetrees.pl

Syntax:

(<DIR>/)naturalstories.ptb.linetrees

class mb.external_resources.natstor.LineTreesNatstorPennSource[source]¶

Bases: mb.core.general.core.ExternalResource

Source Penn Treebank style hand-corrected parses for the Natural Stories corpus

External resources:

/home/corpora/original/english/naturalstories

Prerequisites:

/home/corpora/original/english/naturalstories

Syntax:

/home/corpora/original/english/naturalstories/parses/penn/all-parses.txt.penn

PARENT_RESOURCE¶: alias of NatstorRepo

class mb.external_resources.natstor.NatstorAudioSource[source]¶

Bases: mb.core.general.core.ExternalResource

Natural Stories audio source

External resources:

/home/corpora/original/english/naturalstories

Prerequisites:

/home/corpora/original/english/naturalstories

Syntax:

/home/corpora/original/english/naturalstories/audio

PARENT_RESOURCE¶: alias of NatstorRepo

class mb.external_resources.natstor.NatstorProcessRTs[source]¶

Bases: mb.core.general.core.ExternalResource

Forked processing script for Natural Stories RT data

External resources:

/home/corpora/original/english/naturalstories

Prerequisites:

/home/corpora/original/english/naturalstories
scripts/process_ns_mb.R

Syntax:

/home/corpora/original/english/naturalstories/naturalstories_RTS/process_ns_mb.R

PARENT_RESOURCE¶: alias of NatstorRepo

class mb.external_resources.natstor.NatstorRTSource[source]¶

Bases: mb.core.general.core.ExternalResource

Natural Stories source reading times

External resources:

/home/corpora/original/english/naturalstories
/home/corpora/original/english/naturalstories/naturalstories_RTS/process_ns_mb.R

Prerequisites:

/home/corpora/original/english/naturalstories
/home/corpora/original/english/naturalstories/naturalstories_RTS/process_ns_mb.R

Syntax:

/home/corpora/original/english/naturalstories/naturalstories_RTS/processed_RTs_MB.tsv

PARENT_RESOURCE¶: alias of NatstorRepo

class mb.external_resources.natstor.NatstorRepo[source]¶

Bases: mb.core.general.core.Repo

A corpus of naturalistic stories meant to contain varied, low-frequency syntactic constructions. There are a variety of annotations and psycholinguistic measures available for the stories.

URL: https://github.com/languageMIT/naturalstories

Syntax:

/home/corpora/original/english/naturalstories

class mb.external_resources.natstor.NatstorTokSource[source]¶

Bases: mb.core.general.core.ExternalResource

Natural Stories source tokenization

External resources:

/home/corpora/original/english/naturalstories

Prerequisites:

/home/corpora/original/english/naturalstories

Syntax:

/home/corpora/original/english/naturalstories/naturalstories_RTS/all_stories.tok

PARENT_RESOURCE¶: alias of NatstorRepo

mb.external_resources.ptb module¶

class mb.external_resources.ptb.LineTreesPTB(path)[source]¶

Bases: mb.core.general.tree.LineTrees

Linetrees from a section of the Penn Treebank (PTB)

class mb.external_resources.ptb.PTBSection[source]¶

Bases: mb.core.general.core.ExternalResource

Section of the Penn Treebank (PTB) parse source

class mb.external_resources.ptb.PTBSections(path)[source]¶

Bases: mb.core.general.tree.LineTrees

Concatenated PTB sections starting with START and ending with END

Syntax:

(<DIR>/){wsj,swbd,brown}<START>to<END>.linetrees

class mb.external_resources.ptb.PennTreebankRepo[source]¶

Bases: mb.core.general.core.Repo

One million words of 1989 Wall Street Journal material annotated in Treebank II style. A small sample of ATIS-3 material annotated in Treebank II style. Switchboard tagged, dysfluency-annotated, and parsed text. A fully tagged version of the Brown Corpus. Brown parsed text.

URL: https://catalog.ldc.upenn.edu/ldc99t42

Syntax:

/home/corpora/original/english/penn_treebank_3