run bad double dipping analysis
Matlab output: run_bad_double_dipping_analysis
%% Double dipping
% Warning: this exercise shows the *bad* practice of double dipping
% (also known as circular analysis). You must never, ever use
% results double dipping to interpret results for a real analysis that you
% would publish.
% compute number of samples
% set targets
% allocate space for output
% generate random gaussian train data of size nsamples x nfeatures
% assign the result to a variable 'train_data'
% for the double dipping test data, assign 'double_dipping_test_data'
% to be the same as the training data.
% *** WARNING ***
% For real data analyses (that you would publish in a paper) you
% must never do double dipping analysis - its results are invalid
% for the independent data, generate random gaussian data (of the
% same size as train_data) and assign to a variable
% compute class labels predictions for both test sets using
% cosmo_classify_lda. Store the predictions in
% 'double_dipping_pred' and 'independent_pred', respectively
% compute classification accuracies
% store accuracies in the iter-th row of the 'accuracies' matrix
% show histogram