Article Link

Abstract 

Background: In fMRI decoding, temporal embedding of brain spatial features allows the incorporation of brain activity dynamics into the multivariate pattern classification process, and provides enriched information about stimulus-specific response patterns and potentially improved prediction accuracy.

New method: This study investigates the possibility of enhancing the classification performance by exploring temporal embedding, to identify the optimum combination of the spatiotemporal features based on their classification performance. We investigated the importance of spatiotemporal feature selection using a slow event-related design adapted from the classic Haxby study. Data were collected using a multiband fMRI sequence with temporal resolution of 0.568 seconds.

Comparison with existing methods: A wide range of spatiotemporal observations were created as various combinations of spatiotemporal features. Using both random forest, and support vector machine classifiers, prediction accuracies for these combinations were then compared with the single spatial multivariate pattern approach that uses only a single temporal observation.

Results: Our findings showed that on average spatiotemporal feature selection improved prediction accuracy. Moreover, the random forest algorithm outperformed the support vector machine and benefitted from temporal information to a greater extent.

Conclusions: As expected, the most influential temporal durations were found to be around the peak of the hemodynamic response function, a few seconds after the stimuli onset until ~4 seconds after the peak of the hemodynamic response function. The superiority of spatiotemporal feature selection over single time-point spatial approaches invites future work to design optimal approaches that incorporate spatiotemporal dependencies into feature selection for decoding.

Keywords: fMRI, Multi-Variate Pattern Analysis, Spatiotemporal Feature Selection, Multi-band EPI, Random Forest, Support Vector Machine

Figure 1. Graphical illustration of data processing. The top row indicates the preprocessing steps employed in this study. Spatiotemporal embedding is shown in the dotted box, which involves concatenating the outcome of fMRI data feature selection (voxels inside red squares are symbolic representation of bilateral IT after feature selection), for fMRI volumes `a` through to `b`. Each 3D cube is a symbolic fMRI volume. The bottom row illustrates the learning steps.