Model Diagnostics

Several classes and/or routines are available to help with examining and diagnosing model fits.

class sails.diags.ModelDiagnostics[source]

Class to store and display LinearModel diagnostic information. Several diagnostic criteria are computed including:

R_square (R_square)
the percentage of variance explained in each channel
Stability Index (SI)
Indicator of the stability of the MVAR parameters (SI<1 indicates a stable model)
Stability Ratio (SR)
A stronger test of stability computed from the ratio of the largest eigenvalue of A to all others.
Durbin-Watson (DW)
A test of autocorrelation in residuals of the model fit. Values should be close to 2 indicating no autocorrelation, values close to 0 indicate a positive autocorrelation and 4 and negative autocorrelation.
Log Likelihood (LL)
The log-likelihood of the model.
Akaike’s Information Criterion (AIC)
An indication of the model ‘quality’, lower values indicate a more accurate, less complex model.
Bayesian Information Criterion (BIC)
An indication of the model ‘quality’, lower values indicate a more accurate, less complex model.
Percent Consistency (PC)
Indicates how well a model captures the auto and cross correlation in a time-series. Only computed if compute_pc is passed to the relevant function.
classmethod combine_diag_list(diags)[source]

Helper function for combining diagnostics from a list of ModelDiagnostics instances for easy comparison and visualisation.

Parameters:diags (list of ModelDiagnostics instances) – The ModelDiagnostics to concatenate
Returns:
Return type:sails ModelDiagnostics instance
classmethod compute(model, data, compute_pc=False)[source]

Classmethod for computing a set of model diagnostics from a fitted model applied to a time-series dataset.

Parameters:
  • model (sails LinearModel class) – A fitted linear model
  • data (ndarray) – A 3d time-series of size [nchannels x nsamples x ntrials]
  • compute_pc (bool) – Flag indicating whether to compute the percent consistency, this can be time-consuming for large datasets (Default=False).
Returns:
Return type:

sails ModelDiagnostics instance
summary(all_models=True)[source]

Print the ModelDiagnostics in a pre-formatted table

class sails.diags.DelayDiagnostics[source]

Class which computes the mutual information as a function of lag from zero lag to the first zero crossing of the autocorrelation function.

Compute MI as a function of lag from zero lag to the first zero crossing of the autocorrelation function

Parameters:
  • data (numpy.ndarray) – array of [signals, samples, trials]
  • maxdelay (int) – maximum delay to consider
  • step (int) – step to increment the delay by
  • sample_rate (float) – sample rate of data
  • constant_window (bool) – Flag indicating that the same number of datapoints should be included at each delay. Default is True
Returns:

Populated object containing diagnostics for each value in delay
Return type:

DelayDiagnostics
sails.modelfit.get_residuals(data, parameters, delay_vect, backwards=False, mode='valid')[source]

This is a helper function for computing the residuals of a dataset after the MVAR predictions have been removed.

Parameters:
  • data (ndarray) – Data to compute the residuals from, of size [nchannels x nsampes x nrealisations]
  • parameters (ndarray) – MVAR parameter matrix, of size [nchannels x nchannels x model order]
  • delay_vect (ndarray) – Vector of lag indices corresponding to the third dimension of the parameter matrix
  • backwards (bool) – Flag indicating whether the forwards or backwards parameters havebeeen passed (Default value = False)
  • mode ({'valid','full_nan','full'}) – Options for excluding or replacing residuals which do not have a full model prediction ie the third sample of an order 5 model. ‘valid’ removes samples without a full model prediction, ‘full_nan’ returns resids of the same size as the data with nans replacing excluded samples and ‘full’ returns resids keeping non-full model samples in place.
Returns:

Residual data, of size [nchannels x nsamples x nrealisations]
Return type:

ndarray