Model Diagnostics#

Several classes and/or routines are available to help with examining and diagnosing model fits.

class sails.diags.ModelDiagnostics[source]#

Class to store and display LinearModel diagnostic information. Several diagnostic criteria are computed including:

R_square (R_square): the percentage of variance explained in each channel
Stability Index (SI): Indicator of the stability of the MVAR parameters (SI<1 indicates a stable model)
Stability Ratio (SR): A stronger test of stability computed from the ratio of the largest eigenvalue of A to all others.
Durbin-Watson (DW): A test of autocorrelation in residuals of the model fit. Values should be close to 2 indicating no autocorrelation, values close to 0 indicate a positive autocorrelation and 4 and negative autocorrelation.
Log Likelihood (LL): The log-likelihood of the model.
Akaike’s Information Criterion (AIC): An indication of the model ‘quality’, lower values indicate a more accurate, less complex model.
Bayesian Information Criterion (BIC): An indication of the model ‘quality’, lower values indicate a more accurate, less complex model.
Percent Consistency (PC): Indicates how well a model captures the auto and cross correlation in a time-series. Only computed if compute_pc is passed to the relevant function.

classmethod combine_diag_list(diags)[source]#

Helper function for combining diagnostics from a list of ModelDiagnostics instances for easy comparison and visualisation.

Parameters:: diags (list of ModelDiagnostics instances) – The ModelDiagnostics to concatenate
Return type:: sails ModelDiagnostics instance

classmethod compute(model, data, compute_pc=False)[source]#

Classmethod for computing a set of model diagnostics from a fitted model applied to a time-series dataset.

Parameters:

model (sails LinearModel class) – A fitted linear model
data (ndarray) – A 3d time-series of size [nchannels x nsamples x ntrials]
compute_pc (bool) – Flag indicating whether to compute the percent consistency, this can be time-consuming for large datasets (Default=False).

Return type:

sails ModelDiagnostics instance

summary(all_models=True)[source]#: Print the ModelDiagnostics in a pre-formatted table

class sails.diags.DelayDiagnostics[source]#

Class which computes the mutual information as a function of lag from zero lag to the first zero crossing of the autocorrelation function.

classmethod delay_search(data, maxdelay, step, sample_rate, constant_window=True)[source]#

Compute MI as a function of lag from zero lag to the first zero crossing of the autocorrelation function

Parameters:

data (numpy.ndarray) – array of [signals, samples, trials]
maxdelay (int) – maximum delay to consider
step (int) – step to increment the delay by
sample_rate (float) – sample rate of data
constant_window (bool) – Flag indicating that the same number of datapoints should be included at each delay. Default is True

Returns:

Populated object containing diagnostics for each value in delay

Return type:

DelayDiagnostics

sails.modelfit.get_residuals(data, parameters, delay_vect, backwards=False, mode='valid')[source]#

This is a helper function for computing the residuals of a dataset after the MVAR predictions have been removed.

Parameters:

data (ndarray) – Data to compute the residuals from, of size [nchannels x nsampes x nrealisations]
parameters (ndarray) – MVAR parameter matrix, of size [nchannels x nchannels x model order]
delay_vect (ndarray) – Vector of lag indices corresponding to the third dimension of the parameter matrix
backwards (bool) – Flag indicating whether the forwards or backwards parameters havebeeen passed (Default value = False)
mode ({'valid','full_nan','full'}) – Options for excluding or replacing residuals which do not have a full model prediction ie the third sample of an order 5 model. ‘valid’ removes samples without a full model prediction, ‘full_nan’ returns resids of the same size as the data with nans replacing excluded samples and ‘full’ returns resids keeping non-full model samples in place.

Returns:

Residual data, of size [nchannels x nsamples x nrealisations]

Return type:

ndarray