Please use this identifier to cite or link to this item: https://hdl.handle.net/10316/48016
Title: Permutations of functional magnetic resonance imaging classification may not be normally distributed
Authors: Al-Rawi, Mohammed S. 
Freitas, Adelaide 
Duarte, João V. 
Cunha, Joao P. 
Castelo-Branco, Miguel 
Keywords: Permutation testing; Normality; Central limit theorem; Classification analysis; Anderson–Darling test
Issue Date: Dec-2017
Publisher: SAGE Publications
Project: info:eu-repo/grantAgreement/FCT/SFRH/SFRH/BD/69735/2010/PT 
info:eu-repo/grantAgreement/FCT/PEst/C/SAU/3282/2013 
info:eu-repo/grantAgreement/FCT/5876/147206/PT 
Serial title, monograph or event: Statistical Methods in Medical Research
Volume: 26
Issue: 6
Abstract: A fundamental question that often occurs in statistical tests is the normality of distributions. Countless distributions exist in science and life, but one distribution that is obtained via permutations, usually referred to as permutation distribution, is interesting. Although a permutation distribution should behave in accord with the central limit theorem, if both the independence condition and the identical distribution condition are fulfilled, no studies have corroborated this concurrence in functional magnetic resonance imaging data. In this work, we used Anderson-Darling test to evaluate the accordance level of permutation distributions of classification accuracies to normality expected under central limit theorem. A simulation study has been carried out using functional magnetic resonance imaging data collected, while human subjects responded to visual stimulation paradigms. Two scrambling schemes are evaluated: the first based on permuting both the training and the testing sets and the second on permuting only the testing set. The results showed that, while a normal distribution does not adequately fit to permutation distributions most of the times, it tends to be quite well acceptable when mean classification accuracies averaged over a set of different classifiers is considered. The results also showed that permutation distributions can be probabilistically affected by performing motion correction to functional magnetic resonance imaging data, and thus may weaken the approximation of permutation distributions to a normal law. Such findings, however, have no relation to univariate/univoxel analysis of functional magnetic resonance imaging data. Overall, the results revealed a strong dependence across the folds of cross-validation and across functional magnetic resonance imaging runs and that may hinder the reliability of using cross-validation. The obtained p-values and the drawn confidence level intervals exhibited beyond doubt that different permutation schemes may beget different permutation distributions as well as different levels of accord with central limit theorem. We also found that different permutation schemes can lead to different permutation distributions and that may lead to different assessment of the statistical significance of classification accuracy.
URI: https://hdl.handle.net/10316/48016
ISSN: 0962-2802
eISSN 1477-0334
DOI: 10.1177/0962280215601707
Rights: embargoedAccess
Appears in Collections:I&D CIBIT - Artigos em Revistas Internacionais

Show full item record

SCOPUSTM   
Citations

1
checked on Mar 25, 2024

WEB OF SCIENCETM
Citations 20

1
checked on Mar 2, 2024

Page view(s) 20

776
checked on Mar 26, 2024

Download(s)

356
checked on Mar 26, 2024

Google ScholarTM

Check

Altmetric

Altmetric


This item is licensed under a Creative Commons License Creative Commons