| dc.description.abstract | This study used multimodal sequences to explore whether individuals can integrate and connect stimuli across and within modality. In the current study, we used two behavioral experiments and one event-related potentials (ERPs) study to discuss and further explore how the stimuli were integrated across modalities and within modality. In Experiment 1, the procedure of Kemény and Lukács (2019) was followed, and an implicit test that excluded serial responses was added to examine whether the participants have a non-motor learning representation in the multimodal sequence. Same as the previous study, we observed the multimodal sequential learning effect and the reaction times for stimuli were affected by the cross-modal random sequence. In addition, we found different results from previous study that was both auditory and visual unimodal sequence revealed sequential learning effect, but the unimodal learning effect in auditory stimuli still larger than in visual stimuli. The results in the additional implicit test which was without the instructions of serial key pressings to the stimuli found that participants cannot perform a higher correct response rate than the pure- guessing probability. Therefore, we inferred that the sequential learning is mainly based on motor learning.
Experiment 2 used lexical decision task to examine the duration of the priming effect for visual and auditory words, and conduct correlation analysis with the results of sequential learning effects. The results indicated that the priming effect for the visual and auditory words lasted until the interval of three stimuli, and this result was consistent to the unimodal sequential learning effect in both auditory and visual modalities. Experiment 3 contained the ERP to discuss whether there is an effect of N400 when the sequence fragments in the implicit test was incongruent with the previously learned multimodal sequence structure, and examining whether the stimuli
ii
were integrated across modalities for participants. It was found that only when the modality and the identity of stimulus were incongruent, the greater N400 effect can be found. Further, the ERP results indicated that a negative going waveform was found during the late time window of 400 to 800 milliseconds, when the modality was incongruent (regardless of whether the identity of stimulus is congruent or not). Given the results above, individuals might integrate the stimulus across modalities when the sensory information is intermixed, and at the same time, there will also be a connection between the non-adjacent stimuli within modality. | en_US |