Using CoSMoMVPA multiple-comparison correction

Rationale

A previous exercise showed how to generate a whole-brain map. How do we know which features (voxels) show a significant effect?

A few approaches for whole-brain multiple-comparison methods have been proposed in the literature, but are not used in CoSMoMVPA:

  • Bonferroni correction; typically too conservative.

  • FDR: can lead to invalid inferences.

  • Parametric cluster-based methods (SPM, AFNI, FSL): can be too liberal.

  • Fixed-treshold cluster-based approach: requires choosing an uncorrected (feature-wise) threshold.

In CoSMoMVPA we supply an implementation for Threshold-Free Cluster Enhancement ([SN09]). Optionally this can also be used with first-level null-data ([SCT13]), although that is not done in this exercise.

Exercise with fMRI data

Note: Since data in the ak6 dataset is not in MNI space, we cannot do group analysis. This exercise therefore considers data from one subject using data in ten chunks corresponding to ten runs. However, the appraoch can also be used (in exactly the same way) for group analysis, where each chunk corresponds to one subject.

In this exercise, load data from ten runs from the ak6 datasets. Compute, for each chunk, the effect of stimulus presentation versus baseline. Then, using cosmo cluster neighborhood and cosmo montecarlo cluster stat, compute a TFCE zscore map corrected for multiple comparisons.

Extra exercise: do the same analysis for primates versus insects.

Template: run multiple comparison correction skl

Check your answers here: run multiple comparison correction / Matlab output: run_multiple_comparison_correction

Exercise with time-generalization data

In the MEG datasets coming with CoSMoMVPA there is only data from one participant. For this exercise we split the data in ten parts and treat each as coming from one pseudo-participant.

After splitting the data in ten parts, run the time generalization measure on each part, looking for pairs of time points from which the data shows category information. After the necessary transpositions run cosmo_montecarlo_cluster_stat to obtain a group map corrected for multiple comparisons.

Template: run meeg time generalization mcc skl

Check your answers here: run meeg time generalization mcc / Matlab output: run_run_meeg_time_generalization_mcc