Нейрокогнитивные механизмы обработки морфосинтаксической информации: мультимодальные исследования тема диссертации и автореферата по ВАК РФ 00.00.00, кандидат наук Алексеева Мария Алексеевна

• Алексеева Мария Алексеевна
• кандидат наук
• 2024

Оглавление диссертации кандидат наук Алексеева Мария Алексеевна

TABLE OF CONTENTS

Publications and approbation of results

Introduction

General description of the research problem

Automatic morphosyntactic processing and general methodology

Research questions, goals, and hypotheses

Main theoretical and practical novelty of the present research

Key aspects to be defended at Viva Voce

The practical relevance of research outcomes presented by the degree-seeking candidate

Research description

Research Question

Research Question

Conclusion

Acknowledgments

References

Appendices

Appendix A. Article "Neurophysiological correlates of automatic integration of voice

and gender information during grammatical processing"

Appendix B. Article "Нейрофизиологические корреляты автоматической обработки

нулевой морфемы: данные вызванных потенциалов"

Appendix C. Article "Inflectional zero morphology - Linguistic myth or neurocognitive

reality?"

Appendix D. Dissertation text in Russian (перевод диссертации на русский язык)

Введение диссертации (часть автореферата) на тему «Нейрокогнитивные механизмы обработки морфосинтаксической информации: мультимодальные исследования»

Introduction General description of the research problem

In spite of extensive research, language faculty is one of the least understood functions of the human brain. Therefore, language-cognition interface remains a central question of a long-standing debate about the nature of linguistic processing (Jackendoff, 2003). This general research agenda translates into more specific research questions. One of such questions concerns the degree of functional and neuroanatomical interplay between language and general cognitive processes, such as memory or attentional control. Another important and related question is the interplay between processing linguistic information per se and different extralinguistic speaker-related features, for example gender (Knosche et al., 2002).

Human language is characterized by a rich system of syntactic/grammatical rules that govern the combinatorial use of constituent elements. It was demonstrated that the neurocognitive system underlying (morpho)syntactic processing is relatively independent of other cognitive systems, having a high degree of automaticity (Hahne & Friederici, 1999; Pinker, 2003; Pulvermuller et al., 2008). Research has shown, for example, that morphosyntactic processing can still be successfully accomplished by the brain even if the attention is diverted away from the linguistic input. It is true at least for English (Pulvermuller et al., 2008), Finnish (Shtyrov et al., 2003), and French (Hanna et al., 2014). Recently, similar research has been conducted using Russian (Alekseeva et al., 2024) further extending a largely universal nature of this phenomenon.

Commencing arguably before the non-automatic (top-down controlled) processes of parsing lexico-semantic information and combining overall sentence information including morphology, semantics and syntax, morphosyntactic processing is believed to include an automatic level as well as a further attention-dependent one (Friederici, 2002). The former type of processing takes place independently of focused attention. Experimentally, this can be probed using a special passive non-attend paradigm. Moreover, by employing carefully selected and controlled specific language materials,

this paradigm could be used to acquire novel knowledge about particular morphological components and their processing inside as well as outside the language system. One such example is the processing of voice-specific parameters that can interact with the language's grammatical system and can be modulated by morphology. In my PhD research, I take early morphosyntactic processing as the starting point in order to examine two key processing levels within the morphosyntactic field: (1) the extralinguistic level where the processing of the morphosyntactic gender information is based on the speaker's voice features, and (2) the linguistic level, where the processing is based on the presence of specific morphological components - inflectional affixes. The latter level focuses on the investigation of a specific morphological component - the so-called zero morpheme. Both of these levels provide useful insights into morphosyntactic word processing. On the one hand, in my research I examine the potential inclusion of additional extralinguistic information. On the other hand, I explore a particularly controversial (both linguistically and cognitively) type of a null constituent - the zero morpheme, whose neurocognitive correlates are still poorly understood. Both of these research directions will be discussed in more details below.

Automatic morphosyntactic processing and general methodology

The temporal dynamics of written and spoken language comprehension has been typically explored using techniques such as electroencephalography (EEG) and magnetoencephalography (MEG), thus making it possible to extract important neurocognitive information regarding specific neurolinguistic processes with a highly precise temporal and spatial resolution. These methods have allowed to isolate distinct components of event-related potentials (ERPs in EEG) or fields (ERFs in MEG) that could be associated with different stages of (morpho)syntactic processing. Of these, the most studied ERP/ERF components are the Early Left-Anterior Negativity or ELAN (Neville et al., 1991), the Left Anterior Negativity or LAN (Friederici et al., 1993), and P600, a later positive shift (Osterhout & Holcomb, 1992). Besides the timing information,

the use of EEG/MEG topographic and source analyses capabilities can provide relevant information regarding the cortical structures involved.

The first stage of this processing is known to be accompanied by the ELAN component with a peak around 120-200 ms after the critical event onset (e.g., a grammatical/syntactic/morphosyntactic violation). It owes its name to its comparably early latency and a left frontal distribution (Friederici et al., 1993; Hahne & Friederici, 1999; Neville et al., 1991), although it has also been found to be elicited bilaterally (Hahne & Jescheniak, 2001; Knosche et al., 1999). Usually, it is related to syntactic violations during early and automatic analysis of word and sentence composition. For example, Friederici and colleagues showed that ELAN is closely related to phrase structure errors (Friederici, 1995; Friederici et al., 1993). Such errors can be seen during a mismatch/disagreement between the words in a phrase (for example, *Der Freund wurde im hesucht/The friend was in the visited; Friederici et al., 1993). Such violations make it impossible to build a felicitous local phrase structure.

Others, however, have suggested that the ELAN effect may be an artifact (Steinhauer & Drury, 2012), as its onset is relatively early and may not be related to morphosyntactic processing per se. For example, lexical access has been argued to take at least ~200 ms to commence (Allopenna et al., 1998; Sereno & Rayner, 2003); hence it is problematic to clearly specify the nature of morphosyntactic processes preceding it. At the same time, several studies have shown that lexical and semantic processing can indeed start much earlier and can even take place without focussed attention on the linguistic input (Pulvermuller & Shtyrov, 2006), suggesting a high degree of automaticity of early language processing. One explanation for this rapid and automatic activation suggests that language comprehension system in essence predicts possible incoming elements (including morphosyntactic ones) as early as possible based on the characteristics of previously encountered sentences (Fonteneau, 2013), which may explain the very early onset of the ELAN. These predictions could be related to the basic morphosyntactic processes, which start with the preprocessing of the syntactic structure and continue with semantic information processing (Friederici, 2002; Friederici & Weissenborn, 2007).

Later morphosyntactic components do not correspond to purely automatic processes and can be linked to additional top-down attention-related stages of comprehension and a range of other processes, potentially running in parallel/cascade fashion with the automatic ones. I will therefore discuss them only briefly. The second stage of morphosyntactic processing is revealed in the LAN component, which lasts approximately from 300 ms to 500 ms and has a similar left anterior distribution. Whilst having a certain degree of automaticity (Gunter et al., 1997, 2000), it is still largely connected to the more controlled processing stages. For example, the LAN is usually referred to the morphosyntactic difficulties within the overall sentence structure (Friederici, 1995; Molinaro et al., 2015; Osterhout & Mobley, 1995) as well as to the detection of phrase-agreement violations (Barber & Carreiras, 2005; Bornkessel & Schlesewsky, 2006). The arguably final processing stage is reflected by the P600 complex. This is a later (after 500 ms) positive shift whose maximum amplitudes are registered over the posterior scalp regions in EEG. It is linked to the global-level syntactic integration, reanalysis, and repair (Friederici, 2002; Hagoort et al., 1993; Kaan & Swaab, 2003; Osterhout & Mobley, 1995).

Considering a largely predictive nature of syntactic dependencies, parsing of phrases, and morphological decomposition, it is important to use a paradigm that could address the automatic aspects of morphosyntactic processing both within and without the language system. Here, I implemented the protocol of passive auditory presentation typically used in language studies where auditorily presented experimental stimuli are accompanied by a concurrent visual task, to which participants are instructed to pay attention while ignoring the auditory linguistic input (for example, morphosyntactic study by (Hasting et al., 2007).

In line with the typical ERP approach, I used a counterbalanced design whereby the same gender-marked verbs appeared in syntactically or extralinguistically correct and incorrect pronoun-verb phrases using an inflectionally rich language - Russian. Crucially, I also included a further, context-free condition - comprising the same gender-marked verbforms but without the preceding subject that could pre-activate the gender feature. This approach capitalises on a morphosyntactic priming theory (Oltra-Massuet et al.,

2017), according to which there is a priming element in the phrase processing that pre-activates (i.e., primes) the processing of the next word's morphosyntactic features. More specifically, it was demonstrated that the construction of a pre-processed grammatical template based on a certain pronoun feature (here, gender) enhances activation of the ELAN component (Hasting et al., 2007; Leminen et al., 2013; Pulvermuller et al., 2008; Pulvermuller & Shtyrov, 2003, 2006). As detailed below, this methodological design could address research questions related to the following two levels of morphosyntactic processing: (1) the processing of morphosyntactic gender information extracted from the speaker's voice, and (2) putative existence and activation of zero morpheme representations in the brain.

Research questions, goals, and hypotheses

Research Question 1

Within Research Question 1 (Alekseeva et al., 2022a), I investigated how and when linguistic and extralinguistic gender information is integrated during sentence processing. The speaker's voice provides reliable and quick information that supports sentence comprehension (Knosche et al., 2002). For example, the speaker's apparent gender can be recognized from their voice (Apicella & Feinberg, 2009; Hodges-Simeon et al., 2011; Junger et al., 2013; Li et al., 2014; Mullennix et al., 1995; van Dommelen & Moxness, 1995). Furthermore, many languages mark the grammatical gender explicitly, adding another source of gender information and further complicating the interaction between the speaker's gender and the grammatical gender of a given language constituent. Moreover, sentence comprehension takes into account both the listener's gender and that of the speaker (Junger et al., 2013). A specific voice-gender effect has been documented in ERPs as an early (at around 150-250 ms) increase of the amplitude when hearing the opposite gender voices, compared to gender-matched stimuli (Casado & Brunelliere, 2016).

Other ERP studies have shown the modulation of the N400 and/or P600 components related to the inconsistencies between the semantic meaning and the speaker's representation (Bornkessel-Schlesewsky et al., 2013; Lattner & Friederici, 2003; Van Berkum et al., 2008). While the P600 component, previously shown to be of predominantly syntactic nature, typically accompanies the violations of the stereotypical role nouns, the semantically related N400 may result from semantic-pragmatic incongruences that occur when the listener encounters words/phrases that are not semantically felicitous given the speaker's apparent identity. Given the sparsity of these findings and the lack of compatibility between them, the questions of how and when the speaker's gender information is integrated by the listener's language system during the comprehension process remain largely unanswered.

Taking together the timing of early morphosyntactic processing discussed in the introduction and the findings of early speaker-listener gender interaction suggested by

11

some of the previous ERP studies mentioned above, I hypothesized that it could be the case that these are simultaneous processes that are parallel in nature and/or tightly integrated.

Research question 2

My Research Question 2 (Алексеева и др., 2022b; Alekseeva et al., 2022c) refers to one of the most problematic issues in language processing - the disconnect between theoretical linguistic frameworks and their putative neurobiological correlates. Linguists often use highly abstract and derived models that include abstract constituents and theoretical constructs largely without any evidence of their neurocognitive basis. One of such elements is the so-called "null constituent" - that is, one without any overt graphemic or phonological representation (Bally, 1932). As a category, it includes sentential constituents with semantics (or pragmatics) like ellipsis, wh- and np-movements, which were showed to have neurocognitive representations (see Carlson & Harris, 2018; Sekerina et al., 2019; Лауринавичюте et al., 2015). However, the most problematic element in this abstract category is the zero morpheme, which does not have any semantics but projects only a rather abstract syntactic function (for example, verbal number agreement: they give-0pL, he give-sso). Due to the lack of any overt representation, there is a significant controversy around zero morpheme's nature and its neurocognitive reality - in terms of individual languages and universalist theories of grammar.

Any study of the zero morpheme is complicated by several pitfalls. First, it is the inability to separate zero-morpheme processing from other morphological units and thus document unequivocal evidence of zero-morpheme syntactic function (as opposed to simply detecting the absence of graphemically or phonologically represented element; Fodor, 2013). In addition, a controversial logic of including the zero in grammar as a morpheme makes it difficult to operationalize it (see main linguistic theories that tackled the zero-morpheme problem: Anderson, 1992; Aronoff, 1993; Beard, 1995; Chomsky, 2014; Halle et al., 1993; Oltra Massuet, 1999; Wunderlich & Fabri, 1995).

Also, the issues of processing and storing of zero morphemes remain unclear. On the one hand, a zero-marked form (e.g. "cat-0sG') can be stored as a whole (see full-form storage and dual-system approaches in Bybee, 1995 and Pinker, 2015). This entails the inability to separate zeroes from the rest of the morphemes presented in the wordform overtly. On the other hand, any evidence of decomposition taking place would reflect its neurocognitive representation as a distinct morpheme (see decomposition approach in Taft, 2004).

The limited existing literature on the topic diverges in the results and their interpretations. For example, Vasilyeva (2016), using a visual lexical decision task, found shorter reaction times for zero-marked forms compared to the overt ones, while Gor and colleagues (2017), using the same task in the auditory modality, did not find any significant differences in reaction times for these two types of inflectional morphemes. At the brain level, wordforms with zero morphemes were studied using functional magnetic resonance imaging (fMRI) using language production with a visual lexical decision task (Sahin et al., 2006). These authors found significant activation in Broca's area for zero morpheme processing and interpreted it as evidence of its neurocognitive reality.

Put together, these inconclusive and fragmented results do not provide sufficient evidence of zero-morpheme processing. Faster reaction times for zeroes can in fact be argued to be the evidence of the whole-form storage and thus no zero-morpheme processing. In turn, similar reaction times for zero and overt morphemes can stem from decomposition for both types of morphemes, similar processing routes or just and equal time for parsing overt morphemes and gaps in the input stream. Moreover, behavioral tasks (such as lexical decision task) and fMRI do not address neural processes directly, and do not provide fine-grain time resolution accurate enough to address the level of highly automatic early morphological processing in the highly transient process of language comprehension and, therefore, may not be sensitive enough to disentangle the processing of zero morphemes from the neighboring units (most importantly, word stem).

The use of EEG, on the other hand, offers excellent temporal resolution for tracking linguistic processes. Furthermore, a passive non-attended approach to recording brain responses allows to focus on attention-independent processing stages/types, allowing to estimate putative automatic morphosyntactic processes. In addition, a grammatically paradigm conventionally applied in neurolinguistic ERP research, which involves counterbalanced presentation of grammatically/morphosyntactically felicitous and incongruent combinations of words/morphemes, allows to manipulate not as much the zero morpheme as a covert form per se but rather its syntactic function while successfully controlling for several variables.

To make use of this approach, I operationalized zero-morpheme constituent in a gender agreement paradigm and investigated its ability to be predicted or primed by specific linguistic information, tested through a specific ERP component (i.e., ELAN), whose patterns I argue could offer direct evidence of its processing. According to the pre-activation or predictive coding accounts (Friston, 2005), a reduction in the corresponding ERP's amplitude during the inflectional morpheme's presentation should be observed when an overt gender marker (for example, feminine in Russian) is pre-activated by the preceding feminine pronoun (Fonteneau, 2013; Pulvermuller & Shtyrov, 2003; Shtyrov et al., 2003), whereas a fully-fledged morpheme activation is expected in the absence of such a pre-activation resulting in a larger response by comparison.

Main theoretical and practical novelty of the present research

Main theoretical novelty

Our research provides a better understanding of morphosyntactic processing of morphological units. It also goes beyond the level of morphology and language itself -by considering more abstract categories as speaker-related information, i.e., voice-based gender characteristics. Thus, Research Question 1 provides insight into the temporal order of the linguistic and extralinguistic information processing, and it also allows us to consider it in a non-linear way. Research Question 2 helps gain a better understanding of the zero-morpheme nature by looking at it from a variety of angles, but also to demonstrate its existence at the neurocognitive level. In addition, this study serves as a foundation for future research into zero morphemes, which can be extended to other cases both in Russian and in other languages as well as to other types of null constituents.

Main methodological and practical novelty

• The present passive auditory presentation protocol with counterbalanced presentation of (in)congruent pronoun-verb phrases, combined with EEG recording, allows to examine the mechanisms involved in the integration of speaker's extralinguistic (here, gender) information into the neurolinguistic processing, as well as to provide direct evidence of the reality of specific linguistic elements (such as zero morpheme) in a timely-precise manner.

• The obtained findings regarding the automaticity and earliness of speaker-specific information carried by voice properties into language comprehension is not only useful for the understanding of the unfolding of these processes at neural level, but it may also have implications for the educational purposes in relation to learning both native and second language.

• The outputs of the zero-morpheme research can be used for recommendations for developing language grammar materials for educational purposes with objective explanation of the zero-morpheme basis and usage.

• The EEG/MEG protocol with passive auditory presentation with visual distraction and eye-tracking morphosyntactic priming protocol can be used to investigate the level of automaticity of morphosyntactic processing in L2 Russian learners. Furthermore, it can serve as one of the assessment tools in language proficiency tests for learners of Russian as a foreign language as well as to be expanded to other inflectionally rich languages. Building the bridge between fundamental and applied sciences -integrating objective experimental approaches from psychology and neuroscience into the study of fundamental issues in linguistics (or, potentially, other sciences and humanities) provides the opportunity to address a range of cross-disciplinary gaps in neurocognitive and neurolinguistic knowledge and related fields.

Key aspects to be defended at Viva Voce

Research Question 1:

1. Morphosyntactic gender and speaker's apparent gender information are processed by the brain relatively early after the information is available in the auditory input, and in a largely automatic fashion.

2. These early automatic processes of parsing morphosyntactic information and voice gender characteristics from the auditory stream occur in parallel and the speaker's gender information is integrated into the morphosyntactic processing.

Research Question 2:

3. Zero morpheme processing is evidenced at the neurocognitive level.

4. Processing of zero morphemes is separate from the processing of the rest of the word, similar to the overt equivalents, and adheres to the principles of full decomposition.

The practical relevance of research outcomes presented by the degree-seeking candidate

The reliability of the results is grounded in using a recognized research methodology and appropriate statistical tools that correspond to the research problem, as well as in extensive theoretical basis.

The theoretical and methodological framework of the study was established based on internationally recognized works in the field of study, following all standards of cognitive neuroscience.

Research description

Research Question 1

To test the hypotheses of Research Question 1, I used the abovementioned protocol of passive auditory presentation with a morphosyntactic paradigm which included gender subject-verb (dis)agreement in Russian with balanced acoustic, psycholinguistic, and voice-related properties of the stimuli. Ten 5-phoneme Russian verbforms were used in single-word condition and, in combination with first-person singular pronoun, in phrase conditions. I used two voices - male and female - to construct either congruent or incongruent conditions according to the grammatical gender of the verbform (e.g., "h aynxn/I walkedMAsc' presented by a male or a female speaker). Thus, I directly accessed the integration of voice-related information (speaker's apparent gender taken from voice characteristics) into automatic morphosyntactic processing. Using Voice Reader Home 15 (Linguatec), I synthesized all the experimental material, which differed only in their voice characteristics after normalization manipulations. The constructed verbforms and phrases were repeated 20 times, thus resulting in 400 trials per participant. They were divided into 4 blocks and presented simultaneously with a silent video as a distractor task. Participants completed a questionnaire assessing their memory of the distractor video content. Electrophysiological signatures of morphosyntactic processing were recorded via 128-electrodes EEG system.

37 right-handed Russian native speakers (17 males, age range 19-32 years, mean age = 22.55, SD = 3.1) took part in the experiment. All of them had no history of neurological or psychiatric disorders, had normal or corrected-to-normal vision and hearing. Participants signed the consent form before taking part in the experiment. The research was approved by the HSE University Psychology Department Ethics Committee, Moscow, Russia.

Main analysis followed a 2 (Voice/Gender Congruence: congruent vs. incongruent) x 8 (electrode lines: F, FFC, FC, FCC, C, CCP, CP and CPP) x 8 (channels in a line across both hemispheres excluding central vertical line of channels)

rmANOVA design, which returned two significant results at ~ 150 ms and ~400 ms after the verbal past-tense affix. This affix is essentially a gender marker which could agree or disagree with the speaker's voice. First, rmANOVA in the 130-170 ms time window showed a significant three-way interaction (F49,1764 = 2.634, p = .006, n2 =

0.068). Post-hoc analyses registered more negative responses for incongruent stimuli at the fronto-central, central, and centro-posterior regions in both hemispheres (Fi,36= 5.620, p = .023, n2 = 0.135) with the strongest effect over the central cites (see Figure

1, channel C1).

Averaged ERP waveforms and topographic maps for Congruent and Incongruent conditions

Note. Grey shaded areas highlight early and late Voice Congruency effects at 150 ms and 400 ms after verbal suffix on representative scalp sites (C1 and FC4, respectively), compatible with ELAN and N400 components, respectively.

The observed incongruency effect in the 130-170 ms window is very similar to the well-known ELAN-like response, previously linked to early syntactic parsing and morphosyntactic processing. These findings are in line with existing demonstrations of the ELAN component for grammatical gender incongruencies (Hasting et al., 2007; Leminen et al., 2013; Pulvermuller et al., 2008; Pulvermuller & Shtyrov, 2003, 2006; Shtyrov et al., 2011). The crucial difference here is that in the current study this early effect was elicited not by grammatical but by voice gender incongruencies. This, to our knowledge, is the first demonstration of a rapid nature of extralinguistic feature's integration into morphosyntactic processing with a high degree of automaticity.

Second, in the 350-450 ms time window an N400 component was observed, also with a significant three-way interaction between all factors (F49,1764 = 2.072, p = .024, n2 = 0.054). Post-hoc analyses showed the strongest congruency effect at the right fronto-central and central region (F1,36 = 4.239, p = .047, n2 = 0.105) with more negative responses for the incongruent condition (see Figure 1 channel FC4). The observed N400 component in the 350-450 ms time window is in line with previous studies and likely reflects the morphosyntactic processing and inclusion of semantics (Hanulikova & Carreiras, 2015). It was previously considered to be the first step of integration of extralinguistic information (Hanulikova & Carreiras, 2015; Van Berkum et al., 2008). However, as our results suggest, this appears to be already the secondary stage of this integration which is initiated much earlier, at the ELAN latency.

Thus, by implementing a passive non-attended design I detected two intervals -at 150 ms and 400 ms - when voice-related gender processing and general morphosyntactic processing appear to take place in the brain. These processes work in parallel showing a high degree of interaction and integration. Moreover, the rapid onset of the first stage (reflected in the ELAN component) and its appearance in the absence of focused attention on the spoken input suggest a high degree of automaticity of this integration. More generally, these results demonstrate the usefulness of the present experimental approach not only for addressing purely linguistic processes but also for investigating the contribution of extralinguistic factors into language comprehension.

Research Question 2

To test the Research Question 2 hypotheses the same passive design with visual distraction was used as in the Research Question 1. Experimental materials included standalone auditory synthesized past-tense verbs in Russian (for example, "купил/boughtMAsc') and phrases with the same verbs preceded by singular third-person pronouns which grammatically agreed or disagreed with the verbforms in gender (for example, "он купил-0 - она купила - *она купил-0 / he boughtMASc - she boughtFEM - *she boughtMASc'), spoken in female and male voices. Masculine verbforms, devoid of overt gender marker ('- a' in feminine forms), thus putatively incorporated zero morpheme '-0' (here, putative zero masculine inflectional marker), whereas the feminine forms contained the overt inflectional affix '-a', explicitly denoting feminine gender.

Brain activity was recorded in twenty-three healthy right-handed native Russian speakers (age range 18-30 years, mean age = 20.25±2.5, 11 females) using a 128-channel active-electrode EEG system. All participants had no history of neurological or psychiatric disorders, had normal or corrected-to-normal vision and hearing. Participants signed the consent form before taking part in the experiment. The research was approved by the HSE University Psychology Department Ethics Committee, Moscow, Russia.

Cluster-based random permutation analysis showed statistically significant cluster of differences between grammatically correct and incorrect zero-marked phrases at 216-308 ms time interval (p = 0.022). In line with previous research, more negative response was observed for grammatically incorrect items in the fronto-central part with a shift to the left hemisphere, which is compatible with the modulation of the ELAN component (Halmetoja, 2021; Kimppa et al., 2019; Kubota et al., 2018; Leminen et al., 2013; Pulvermuller & Shtyrov, 2003). The association of this component with early automatic morphosyntactic processing and its appearance here specifically for the incorrect zero-inflection stimuli (* она купил-0 / *she boughtMASc)

indicate the same type of activation processes for this zero as for the overt form and their interactions with preceding context. Thus, this result demonstrates a rapid and automatic activation of a zero-morpheme's representation. In a later time window (316-420 ms), a significant cluster was also found for the zero-marked conditions (p = 0.018). In line with previous research, this effect was observed at the central and frontocentral sites, which is compatible with the modulation of the LAN component, indicating further morphosyntactic processing (Friederici, 2002; Tanner & Van Hell, 2014).

Averaged ERP waveforms and topographic maps for Correct and Incorrect conditions with zero-marked verbforms

■100 o 200 400 600 soo Cluster analysis (31fr^20 ms): p = .018

Note. Grey shaded areas highlight Grammatically effects in 216-308 ms and 316-420 ms after verbal suffix time windows on representative scalp sites, compatible with ELAN and LAN components, respectively.

The overtly-marked condition did not show any significant amplitude change in the early time window, thus indicating the absence of early automatic morphosyntactic processing for feminine overt forms (note that we found one for masculine zero forms, see above). The later processing stage was, however, still observed in 308-476 ms time window (p = 0.026). In line with previous research, it had central and fronto-central localization with a shift to the left hemisphere (Friederici, 2002; Popov et al., 2020; Tanner & Van Hell, 2014). The observed difference in the onset of the morphosyntactic processing could in principle also be taken as evidence of easier processing of zero forms, stored as a whole-form without separation of the stem and an inflectional zero marker, whereas morphosyntactic processing of overtly-marked forms require more time to process the acoustic/phonetic component and/or the selection of the corresponding marker (Pinker, 2015), leading to a late onset of the morphosyntactic processing. However, this assumption does not account for additional morpheme-to-morpheme processing for masculine zero-marked forms as it does for overt feminine marking (Flick et al., 2018; Gwilliams, 2020; Leminen et al., 2019).

Averaged ERP waveforms and topographic maps for Correct and Incorrect conditions with overtly-marked verbforms

Note. Grey shaded area highlights Grammaticality effect in 308-417 ms after verbal suffix time window on representative scalp sites, compatible with LAN component.

So, both types of morphemes demonstrate a medium-latency ERP index of morphosyntactic processing reflected by the LAN component, showing a similarity at this secondary processing stage. Together with the early stage demonstrated for the zero-marked forms above, my findings provide support for the neurocognitive reality of zero morpheme as a distinct linguistic entity.

A range of other potential issues related to zero-morpheme existence are still in need of further investigation, such as the universality of this category both for a given language (shared or distinctive in different morphological functions, parts of speech) and across languages (comparison between inflectionally poor and rich languages, questioning whether, e.g., the latter have zero morphemes and the former do not). In addition to using a passive presentation approach described above, I suggest other potential ways to address such further questions. Considering the morphosyntactic priming theory, one can manipulate the predictability specifically - by using the primes specifying a certain value of syntactic function and zero and overtly-marked targets (see Dahan et al., 2000 - for eye-tracking). According to the morphosyntactic priming theory, one should need additional time to decide which value zero marker is represented in the target based on the corresponding context. This design could be employed with different types of verbal, adjectival, and noun inflections. This will help to further investigate the issue of zero morpheme representations as well as to address further questions regarding their specific features within and across languages.

Заключение диссертации по теме «Другие cпециальности», Алексеева Мария Алексеевна

Conclusion

My PhD research constitutes an important step towards a better understanding of the brain mechanisms of language comprehension. Its key contributions to the field are in revealing new aspects of extralinguistic information processing and its integration with the linguistic information, and in addressing the nature of the so-called zero morpheme, a long-debated theoretical construct in linguistics, whose reality in the brain has avoided detection until now. First, our findings provide valuable evidence regarding the timescale of language comprehension, particularly regarding the integration of the voice features with morphosyntactic processing. Second, the novel experimental protocol used in my research allows to experimentally address the zero-morpheme problem from a different perspective and to collect direct evidence of its neurocognitive nature as well its early processing signatures at the very first, automatic steps of morphosyntactic analysis. Future research including the protocols proposed in this work will be able to investigate neurobiological and behavioral signatures of zero morpheme and verify the uniqueness or universality of zero-morpheme functioning and representation within and across languages.

Список литературы диссертационного исследования кандидат наук Алексеева Мария Алексеевна, 2024 год

References

Alekseeva, M., Myachykov, A., Bermudez-Margaretto, B., & Shtyrov, Y. (2022a). Neurophysiological correlates of automatic integration of voice and gender information during grammatical processing. Scientific Reports, 12(1), 1-10. https://doi.org/10.1038/s41598-022-14478-2

Alekseeva, M., Myachykov, A., & Shtyrov, Y. (2022c). Inflectional zero morphology -Linguistic myth or neurocognitive reality? Frontiers in Psychology, 13: 1015435 https://doi.org/10.3389/fpsyg.2022.1015435

Alekseeva, M., Myachykov, A., Bermudez Margaretto, B., & Shtyrov, Y. (2024).

Morphosyntactic prediction in automatic neural processing of spoken language: EEG evidence. Brain Research, 148949. https://doi.org/10.1016/j.brainres.2024.148949

Allopenna, P. D., Magnuson, J. S., & Tanenhaus, M. K. (1998). Tracking the Time Course of Spoken Word Recognition Using Eye Movements: Evidence for Continuous Mapping Models. Journal of Memory and Language, 38(4), 419-439. https://doi.org/10.1006/jmla.1997.2558

Anderson, S. R. (1992). A-morphous morphology (Issue 62). Cambridge University Press.

Apicella, C. L., & Feinberg, D. R. (2009). Voice pitch alters mate-choice-relevant perception in hunter-gatherers. Proceedings of the Royal Society B: Biological Sciences, 276(1659), 1077-1082. https://doi.org/10.1098/rspb.2008.1542

Aronoff, M. (1993). Morphology by itself: Stems and inflectional classes (Vol. 22). MIT press.

Bally, C. (1932). Linguistique générale et Linguistique française, par Charles Bally. Impr. des Presses universitaires le France.

Barber, H., & Carreiras, M. (2005). Grammatical gender and number agreement in Spanish ERP. Journal of Cognitive Neuroscience, 17(1), 137-153. https://doi.org/10.1162/0898929052880101

Beard, R. (1995). Lexeme-morpheme base morphology: a general theory of inflection and word formation. Suny Press.

Bornkessel, I., & Schlesewsky, M. (2006). The extended argument dependency model: A neurocognitive approach to sentence comprehension across languages. Psychological Review, 113(4), 787-821. https://doi.org/10.1037/0033-295X.113A787

Bornkessel-Schlesewsky, I., Krauspenhaar, S., & Schlesewsky, M. (2013). Yes, You Can? A Speaker's Potency to Act upon His Words Orchestrates Early Neural Responses to Message-Level Meaning. PLoS ONE, 8(7).

https://doi.org/10.1371/journal.pone.0069173

Bybee, J. (1995). Regular morphology and the lexicon. Language and Cognitive Processes, 10(5), 425-455. https://doi.org/10.1080/01690969508407111

Carlson, K., & Harris, J. A. (2018). Zero-Adjective contrast in much-less ellipsis: the advantage for parallel syntax. Language, Cognition andNeuroscience, 33(1), 77-97. https://doi.org/10.1080/23273798.2017.1366530

Casado, A., & Brunelliere, A. (2016). The influence of sex information into spoken words: a mismatch negativity (MMN) study. Brain Research, 1650, 73-83. https://doi.org/10.1016/j.brainres.2016.08.039

Chomsky, N. (2014). Aspects of the Theory of Syntax (Issue 11). MIT press.

Dahan, D., Swingley, D., Tanenhaus, M. K., & Magnuson, J. S. (2000). Linguistic Gender and Spoken-Word Recognition in French. Journal of Memory and Language, 42(4), 465-480. https://doi.org/10.1006/jmla.1999.2688

Flick, G., Oseki, Y., Kaczmarek, A. R., Al Kaabi, M., Marantz, A., & Pylkkanen, L. (2018). Building words and phrases in the left temporal lobe. Cortex, 106, 213-236.

Fodor, J. D. (2013). Processing empty categories: A question of visibility. In Cognitive models of speech processing (pp. 351-400). Psychology Press.

Fonteneau, E. (2013). Structural syntactic prediction measured with ELAN: Evidence from ERPs. Neuroscience Letters, 534(1), 211-216. https://doi.org/10.1016/j .neulet.2012.11.055

Friederici, A. D. (1995a). The time course of syntactic activation during language processing: A model based on neuropsychological and neurophysiological data. In Brain and Language (Vol. 50, Issue 3, pp. 259-281). https://doi.org/10.1006/brln.1995.1048

Friederici, A. D. (1995b). The time course of syntactic activation during language processing: A model based on neuropsychological and neurophysiological data. In Brain and Language (Vol. 50, Issue 3, pp. 259-281). https://doi.org/10.1006/brln.1995.1048

Friederici, A. D. (2002a). Towards a neural basis of auditory sentence processing. Trends in Cognitive Sciences, 6(2), 78-84. https://doi.org/10.1016/S1364-6613(00)01839-8

Friederici, A. D. (2002b). Towards a neural basis of auditory sentence processing. Trends in Cognitive Sciences, 6(2), 78-84. https://doi.org/10.1016/S1364-6613(00)01839-8

Friederici, A. D., Pfeifer, E., & Hahne, A. (1993). Event-related brain potentials during natural speech processing: Effects of semantic, morphological and syntactic violations. Cognitive Brain Research, 1(3), 183-192. https://doi.org/10.1016/0926-6410(93)90026-2

Friederici, A. D., & Weissenborn, J. (2007). Mapping sentence form onto meaning: The syntax-semantic interface. Brain Research, 1146(1), 50-58. https://doi.org/10.1016lj.brainres.2006.08.038

Friston, K. (2005). A theory of cortical responses. Philosophical Transactions of the Royal Society B: Biological Sciences, 360(1456), 815-836. https://doi.org/10.1098/rstb.2005.1622

Gor, K., Chrabaszcz, A., & Cook, S. (2017). Processing of native and nonnative inflected words: Beyond affix stripping. Journal of Memory and Language, 93, 315-332.

Gunter, T. C., Friederici, A. D., & Schriefers, H. (2000). Syntactic Gender and Semantic Expectancy. Journal of Cognitive Neuroscience, 12(4). https://doi.org/10.1162/089892900562336

Gunter, T. C., Stowe, L. A., & Mulder, G. (1997). When syntax meets semantics. Psychophysiology, 34(6), 660-676. https://doi.org/10.1111/j.1469-8986.1997.tb02142.x

Gwilliams, L. (2020). How the brain composes morphemes into meaning. Philosophical Transactions of the Royal Society B, 375(1791), 20190311.

Hagoort, P., Brown, C., & Groothusen, J. (1993). The syntactic positive shift (sps) as an erp measure of syntactic processing. Language and Cognitive Processes, 8(4), 439483. https://doi.org/10.1080/01690969308407585

Hahne, A., & Friederici, A. D. (1999). Electrophysiological Evidence for Two Steps in Syntactic Analysis: Early Automatic and Late Controlled Processes. Journal of Cognitive Neuroscience, 11 (2), 194-205. https://doi.org/10.1162/089892999563328

Hahne, A., & Jescheniak, J. D. (2001). What's left if the Jabberwock gets the semantics? An ERP investigation into semantic and syntactic processes during auditory sentence comprehension. Cognitive Brain Research, 11 (2), 199-212. https://doi.org/10.1016/S0926-6410(00)00071-9

Halle, M., Marantz, A., Hale, K., & Keyser, S. J. (1993). Distributed morphology and the pieces of inflection. 111-176.

Halmetoja, S. (2021). The processing of inflected spoken words in mono-and bilinguals: an event-related potential study. University of Helsinki.

Hanna, J., Mejias, S., Schelstraete, M. A., Pulvermuller, F., Shtyrov, Y., & van der Lely, H. K. J. (2014). Early activation of Broca's area in grammar processing as revealed by the syntactic mismatch negativity and distributed source analysis. Cognitive Neuroscience, 5(2), 66-76. https://doi.org/10.1080/17588928.2013.860087

Hanulikova, A., & Carreiras, M. (2015). Electrophysiology of subject-verb agreement mediated by speakers' gender. Frontiers in Psychology, 6. https://doi.org/10.3389/fpsyg.2015.01396

Hasting, A. S., Kotz, S. A., & Friederici, A. D. (2007a). Setting the stage for automatic syntax processing: The mismatch negativity as an indicator of syntactic priming.

34

Journal of Cognitive Neuroscience, 19(3), 386-400. https://doi.Org/10.1162/jocn.2007.19.3.386

Hasting, A. S., Kotz, S. A., & Friederici, A. D. (2007b). Setting the stage for automatic syntax processing: The mismatch negativity as an indicator of syntactic priming. Journal of Cognitive Neuroscience, 19(3), 386-400. https://doi.org/10.1162/jocn.2007.19.3.386

Hodges-Simeon, C. R., Gaulin, S. J. C., & Puts, D. A. (2011). Voice Correlates of Mating Success in Men: Examining "Contests" Versus "Mate Choice" Modes of Sexual Selection. Archives of Sexual Behavior, 40(3), 551-557. https://doi.org/10.1007/s10508-010-9625-0

Jackendoff, R. (2002). Foundations of Language: Brain, Meaning, Grammar, Evolution. Oxford University Press UK.

Jackendoff, R. (2003). Foundations of Language: Brain, Meaning, Grammar, Evolution. Behavioral and Brain Sciences, 26(6), 651-665.

https://doi.org/10.1017/S0140525X03000153

Junger, J., Pauly, K., Bröhr, S., Birkholz, P., Neuschaefer-Rube, C., Kohler, C., Schneider, F., Derntl, B., & Habel, U. (2013). Sex matters: Neural correlates of voice gender perception. Neurolmage, 79, 275-287.

https://doi.org/10.1016Zj.neuroimage.2013.04.105

Kaan, E., & Swaab, T. Y. (2003). Repair, revision, and complexity in syntactic analysis: An electrophysiological differentiation. Journal of Cognitive Neuroscience, 15(1), 98-110. https://doi.org/10.1162/089892903321107855

Kimppa, L., Shtyrov, Y., Hut, S. C. A., Hedlund, L., Leminen, M., & Leminen, A. (2019). Acquisition of L2 morphology by adult language learners. Cortex, 116, 74-90.

Knösche, T. R., Lattner, S., Maess, B., Schauer, M., & Friederici, A. D. (2002). Early Parallel Processing of Auditory Word and Voice Information. Neurolmage, 17(3), 1493-1503. https://doi.org/10.1006/nimg.2002.1262

Knosche, T. R., MaeB, B., & Friederici, A. D. (1999). Processing of syntactic information monitored by brain surface current density mapping based on MEG. Brain Topography, 12(2), 75-87. https://doi.org/10.1023/A:1023442426799

Kubota, M., Ono, Y., Ishiyama, A., Zouridakis, G., & Papanicolaou, A. C. (2018). Magnetoencephalography Reveals Mismatch Field Enhancement from Unexpected Syntactic Category Errors in English Sentences. Neuroscience Letters, 662, 195-204. https://doi.org/10.1016/j.neulet.2017.07.051

Lattner, S., & Friederici, A. D. (2003). Talker's voice and gender stereotype in human auditory sentence processing - evidence from event-related brain potentials. Neuroscience Letters, 339(3), 191-194. https://doi.org/10.1016/S0304-3940(03)00027-2

Leminen, A., Leminen, M., Kujala, T., & Shtyrov, Y. (2013). Neural dynamics of inflectional and derivational morphology processing in the human brain. Cortex, 49(10), 2758-2771. https://doi.org/10.1016/j.cortex.2013.08.007

Leminen, A., Smolka, E., Dunabeitia, J. A., & Pliatsikas, C. (2019). Morphological processing in the brain: The good (inflection), the bad (derivation) and the ugly (compounding). Cortex, 116, 4-44.

Li, Y., Gu, F., Zhang, X., Yang, L., Chen, L., Wei, Z., Zha, R., Wang, Y., Li, X., Zhou, Y., & Zhang, X. (2014). Cerebral Activity to Opposite-Sex Voices Reflected by Event-Related Potentials. PLoS ONE, 9(4), e94976.

https://doi.org/10.1371/journal.pone.0094976

Molinaro, N., Barber, H. A., Caffarra, S., & Carreiras, M. (2015). On the left anterior negativity (LAN): The case of morphosyntactic agreement: A Reply to Tanner etal. Cortex, 66, 156-159. https://doi.org/10.1016/j.cortex.2014.06.009

Mullennix, J. W., Johnson, K. A., Topcu-Durgun, M., & Farnsworth, L. M. (1995). The perceptual representation of voice gender. The Journal of the Acoustical Society of America, 98(6), 3080-3095. https://doi.org/10.1121/L413832

Neville, H., Nicol, J. L., Barss, A., Forster, K. I., & Garrett, M. F. (1991). Syntactically based sentence processing classes: Evidence from event-related brain potentials. Journal of Cognitive Neuroscience, 3(2), 151-165.

https://doi.org/10.1162/jocn.1991.3.2.151

Oltra Massuet, M. I. (1999). On the notion of theme vowel: A new approach to Catalan verbal morphology. Massachusetts Institute of Technology.

Oltra-Massuet, I., Sharpe, V., Neophytou, K., & Marantz, A. (2017). Syntactic Priming As a Test of Argument Structure: A Self-paced Reading Experiment. Frontiers in Psychology, 8. https://doi.org/10.3389/fpsyg.2017.01311

Osterhout, L., & Holcomb, P. J. (1992). Event-related brain potentials elicited by syntactic anomaly. Journal of Memory and Language, 31(6), 785-806. https://doi.org/10.1016/0749-596X(92)90039-Z

Osterhout, L., & Mobley, L. A. (1995). Event-Related Brain Potentials Elicited by Failure to Agree. In Journal of Memory and Language (Vol. 34, Issue 6, pp. 739-773). https://doi.org/10.1006/jmla.1995.1033

Pinker, S. (2015). Words and rules: The ingredients of language. Basic Books.

Pinker, S. (2003). The language instinct. How the mind creates language. Penguin uK.

Popov, S., Miceli, G., Curcic-Blake, B., & Bastiaanse, R. (2020). The role of semantics and repair processes in article-noun gender disagreement in Italian: An ERP study.

Brain and Language, 206, 104787.

Pulvermuller, F., & Shtyrov, Y. (2003). Automatic processing of grammar in the human brain as revealed by the mismatch negativity. Neurolmage, 20(1), 159-172. https://doi.org/10.1016/S1053-8119(03)00261-1

Pulvermuller, F., & Shtyrov, Y. (2006). Language outside the focus of attention: The mismatch negativity as a tool for studying higher cognitive processes. Progress in Neurobiology, 79(1), 49-71. https://doi.org/10.1016/j.pneurobio.2006.04.004

Pulvermuller, F., Shtyrov, Y., Hasting, A. S., & Carlyon, R. P. (2008). Syntax as a reflex: Neurophysiological evidence for early automaticity of grammatical processing. Brain and Language, 104(3), 244-253. https://doi.org/10.1016/j.bandl.2007.05.002

Sahin, N. T., Pinker, S., & Halgren, E. (2006). Abstract grammatical processing of nouns and verbs in Broca's area: evidence from fMRI. Cortex, 42(4), 540-562.

Sekerina, I. A., Laurinavichyute, A. K., & Dragoy, O. (2019). What Eye Movements Can and Cannot Tell Us About Wh-Movement and Scrambling (pp. 147-165). https://doi.org/10.1007/978-3-030-01563-3_8

Sereno, S. C., & Rayner, K. (2003). Measuring word recognition in reading: Eye movements and event-related potentials. Trends in Cognitive Sciences, 7(11), 489493. https://doi.org/10.1016/j.tics.2003.09.010

Shtyrov, Y., Kimppa, L., Pulvermuller, F., & Kujala, T. (2011). Event-related potentials reflecting the frequency of unattended spoken words: A neuronal index of connection strength in lexical memory circuits? Neurolmage, 55(2), 658-668. https://doi.org/10.1016Zj.neuroimage.2010.12.002

Shtyrov, Y., Pulvermuller, F., Naatanen, R., & Ilmoniemi, R. J. (2003). Grammar processing outside the focus of attention: an MEG study. Journal of Cognitive Neuroscience, 15(8), 1195-1206. https://doi.org/10.1162/089892903322598148

Steinhauer, K., & Drury, J. E. (2012). On the early left-anterior negativity (ELAN) in syntax studies. Brain and Language, 120(2), 135-162. https://doi.org/10.1016Aj.bandl.2011.07.001

Taft, M. (2004). Morphological Decomposition and the Reverse Base Frequency Effect.

The Quarterly Journal of Experimental Psychology Section A, 57(4), 745-765. https://doi.org/10.1080/02724980343000477

Tanner, D., & Van Hell, J. G. (2014). ERPs reveal individual differences in morphosyntactic processing. Neuropsychologia, 56, 289-301.

Van Berkum, J. J. A., van den Brink, D., Tesink, C. M. J. Y., Kos, M., & Hagoort, P. (2008). The Neural Integration of Speaker and Message. Journal of Cognitive Neuroscience, 20(4), 580-591. https://doi.org/10.1162/jocn.2008.20054

van Dommelen, W. A., & Moxness, B. H. (1995). Acoustic Parameters in Speaker Height and Weight Identification: Sex-Specific Behaviour. Language and Speech, 38(3), 267-287. https://doi.org/10.1177/002383099503800304

Vasilyeva, M. (2016). Russian case inflection: Processing costs and benefits. Advances in Formal Slavic Linguistics 2016, 427.

Wunderlich, D., & Fabri, R. (1995). Minimalist morphology: An approach to inflection.

Zeitschrift Für Sprachwissenschaft, 14(2), 236-294.

Алексеева М. А., Мячиков А. В., Штыров Ю. Ю. (2022b). Нейрофизиологические корреляты автоматической обработки нулевой морфемы: данные вызванных потенциалов. Журнал высшей нервной деятельности им. И.П. Павлова, 72(5), 666-677. https://doi.org/10.31857/S0044467722050033

Лауринавичюте, А. К., Драгой, О. В., Иванова, М. В., Купцова, С. В., & Уличева, А. С. (2015). Психологическая нереальность синтаксических следов. Вопросы Языкознания, 1, 102-110.