Model Diagnostics¶
Several classes and/or routines are available to help with examining and diagnosing model fits.

class
sails.diags.
ModelDiagnostics
[source]¶ Class to store and display LinearModel diagnostic information. Several diagnostic criteria are computed including:
 R_square (
R_square
)  the percentage of variance explained in each channel
 Stability Index (
SI
)  Indicator of the stability of the MVAR parameters (SI<1 indicates a stable model)
 Stability Ratio (
SR
)  A stronger test of stability computed from the ratio of the largest eigenvalue of A to all others.
 DurbinWatson (
DW
)  A test of autocorrelation in residuals of the model fit. Values should be close to 2 indicating no autocorrelation, values close to 0 indicate a positive autocorrelation and 4 and negative autocorrelation.
 Log Likelihood (
LL
)  The loglikelihood of the model.
 Akaike’s Information Criterion (
AIC
)  An indication of the model ‘quality’, lower values indicate a more accurate, less complex model.
 Bayesian Information Criterion (
BIC
)  An indication of the model ‘quality’, lower values indicate a more accurate, less complex model.
 Percent Consistency (
PC
)  Indicates how well a model captures the auto and cross correlation in a
timeseries. Only computed if
compute_pc
is passed to the relevant function.

classmethod
combine_diag_list
(diags)[source]¶ Helper function for combining diagnostics from a list of ModelDiagnostics instances for easy comparison and visualisation.
Parameters: diags (list of ModelDiagnostics instances) – The ModelDiagnostics to concatenate Returns: Return type: sails ModelDiagnostics instance

classmethod
compute
(model, data, compute_pc=False)[source]¶ Classmethod for computing a set of model diagnostics from a fitted model applied to a timeseries dataset.
Parameters:  model (sails LinearModel class) – A fitted linear model
 data (ndarray) – A 3d timeseries of size [nchannels x nsamples x ntrials]
 compute_pc (bool) – Flag indicating whether to compute the percent consistency, this can be timeconsuming for large datasets (Default=False).
Returns: Return type: sails ModelDiagnostics instance
 R_square (

class
sails.diags.
DelayDiagnostics
[source]¶ Class which computes the mutual information as a function of lag from zero lag to the first zero crossing of the autocorrelation function.

classmethod
delay_search
(data, maxdelay, step, sample_rate, constant_window=True)[source]¶ Compute MI as a function of lag from zero lag to the first zero crossing of the autocorrelation function
Parameters:  data (numpy.ndarray) – array of [signals, samples, trials]
 maxdelay (int) – maximum delay to consider
 step (int) – step to increment the delay by
 sample_rate (float) – sample rate of data
 constant_window (bool) – Flag indicating that the same number of datapoints should be included at each delay. Default is True
Returns: Populated object containing diagnostics for each value in delay
Return type:

classmethod

sails.modelfit.
get_residuals
(data, parameters, delay_vect, backwards=False, mode='valid')[source]¶ This is a helper function for computing the residuals of a dataset after the MVAR predictions have been removed.
Parameters:  data (ndarray) – Data to compute the residuals from, of size [nchannels x nsampes x nrealisations]
 parameters (ndarray) – MVAR parameter matrix, of size [nchannels x nchannels x model order]
 delay_vect (ndarray) – Vector of lag indices corresponding to the third dimension of the parameter matrix
 backwards (bool) – Flag indicating whether the forwards or backwards parameters havebeeen passed (Default value = False)
 mode ({'valid','full_nan','full'}) – Options for excluding or replacing residuals which do not have a full model prediction ie the third sample of an order 5 model. ‘valid’ removes samples without a full model prediction, ‘full_nan’ returns resids of the same size as the data with nans replacing excluded samples and ‘full’ returns resids keeping nonfull model samples in place.
Returns: Residual data, of size [nchannels x nsamples x nrealisations]
Return type: ndarray