{"id":101,"date":"2021-10-28T16:21:10","date_gmt":"2021-10-28T16:21:10","guid":{"rendered":"https:\/\/openbooks.library.unt.edu\/sourcetoanalysis\/?post_type=chapter&#038;p=101"},"modified":"2024-03-03T22:02:22","modified_gmt":"2024-03-03T22:02:22","slug":"chapter-5","status":"publish","type":"chapter","link":"https:\/\/openbooks.library.unt.edu\/sourcetoanalysis\/chapter\/chapter-5\/","title":{"raw":"Creating an  Annotation Tool-Kit","rendered":"Creating an  Annotation Tool-Kit"},"content":{"raw":"&nbsp;\r\n<div class=\"textbox textbox--learning-objectives\"><header class=\"textbox__header\">\r\n<p class=\"textbox__title\">Learning Objectives:<\/p>\r\n\r\n<\/header><\/div>\r\n<div class=\"textbox\">\r\n<ol>\r\n \t<li>Identify previous sources on lexeme- and morpheme-level glossing<\/li>\r\n \t<li>Create an annotation tool kit for the language<\/li>\r\n \t<li>Learn to build an excel spreadsheet for Sheet Swiper<\/li>\r\n \t<li>Learn about hierarchical annotation<\/li>\r\n<\/ol>\r\n.\r\n\r\n<\/div>\r\n&nbsp;\r\n<h2>5.1 Previous research<\/h2>\r\nInterlinear Glossed Texts (IGT) are used to turn transcribed and translated recordings into data that are usable by communities, linguists, and computer scientists. The amount of effort spent on creating IGT is in an inverse relationship with the amount of available resources there are in the language of analysis. This means that the more grammatical descriptions, dictionary materials, and linguistic analyses you consider in your preparation, the faster and less labor-intensive the process of preparing IGT will be.\r\n\r\nAn IGT annotation is typically a bundle of annotation:\r\n\r\n&nbsp;\r\n\r\nLine 1. tx\/ Transcription (sometimes called the baseline)\r\n\r\nLine 2. mr\/ Morphological representation (break the word down into its morphological parts, i.e. affixes and roots)\r\n\r\nLine 3 mg\/ Morpheme gloss (the meaning of each morpheme)\r\n\r\nLine 4 wg\/ Word gloss\r\n\r\nLine 5 ft\/ Free translation\r\n\r\nLine 6 Note\/\r\n\r\n&nbsp;\r\n\r\nAnd illustrated in the following Manipuri example (Chelliah 1997, p. 225)\r\n\r\ntx\/ \u00a0 \u00a0 \u00a0 \u00a0 m\u00e1n\u0259 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 ap\u0259l \u00a0 \u00a0 \u00a0 \u00a0 \u010d\u00e1l\u0259g\u00ed\r\n\r\nmr\/ \u00a0 \u00a0 \u00a0 m\u00e1 -n\u0259 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 ap\u0259l \u00a0 \u00a0 \u00a0 \u00a0\u010d\u00e1 -l\u0259k -\u00ed\r\n\r\nmg\/ \u00a0 \u00a0 \u00a0he =IS:FOC \u00a0 apple \u00a0 \u00a0 \u00a0eat -AM:VEN -SMOD:DECL\r\n\r\nwg\/ \u00a0 \u00a0 \u00a0 he \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0apple \u00a0 \u00a0 \u00a0 eating\r\n\r\nft\/ 'When he came here he was eating an apple.'\r\n\r\n&nbsp;\r\n\r\nWe will be reviewing the use of a common annotation program called FLEx which uses a Lexicon created by the manual input of data. One way of growing the lexicon is by adding words to the lexicon as you gloss a text - as you translate a text, you provide glosses for the words and morphemes in the clause and these are added to the lexicon. \u00a0The next time that word or morpheme appears, the program will find the stored gloss in the Lexicon and provide that annotation automatically. The richer the Lexicon, the faster the annotation.\r\n<div class=\"textbox\">Discussion:\u00a0 Deconstruct the small IGT sample from Manipuri provided above. For example, discuss what is meant by a dash, what is meant by the equal sign, why some glosses are in capital letters and others in lower case. Why is there a period in 3.POS? What information about the language can be gleaned from this example? What other questions might you investigate based on this example?<\/div>\r\nBased on our experience in creating IGT, we suggest that, before you begin your annotation project, collect as much information from previous resources as you can. When no previous research has been done on the language you\u2019re working on, look to surrounding and related languages. You may also want to contact researchers who have worked on these languages for their unpublished works as well. Previous research you find will help you:\r\n<ul>\r\n \t<li>Develop familiarity with the language and\/or language families so you can more readily recognize constituent boundaries and structure<\/li>\r\n \t<li>Identify relevant areas of typological interest to locate areas more complicated to translate<\/li>\r\n \t<li>Preload your lexicon with content words, function words, and, if possible, bound morphemes to support faster automatic glossing<\/li>\r\n \t<li>Better understand the phonology and orthography so that you can provide a consistent transcription or baseline for annotation<\/li>\r\n \t<li>Think through abbreviation conventions for morphemes and grammatical categories that will be needed<\/li>\r\n \t<li>Determine the closed categories of functional word and morphemes. \u00a0Identify some of their members. \u00a0For example, semantic role marking would a closed category and a member of that category would be agentive role.<\/li>\r\n<\/ul>\r\n<div class=\"textbox\">\r\n<h4>Discussion<\/h4>\r\nIs there a linguistics researcher that has done work with your community? Who has these materials, and can you get access to them? Is the language described the same variety as yours? Do you find previous materials to be an accurate representation of your language?\r\n\r\n<\/div>\r\n&nbsp;\r\n<h2>5.2 Types of Documents<\/h2>\r\nHere are some types of documents you may be able to find and their potential usefulness to your project:\r\n<h3>Word lists and dictionaries<\/h3>\r\nThe data from these texts can be formatted for input into a FLEx Lexicon to automatically populate your glosses. The more words that are inputted into your FLEx Lexicon to begin with, the fewer you have to manually add.\r\n\r\nThe most accessible word list files are spreadsheets, which can be modified for import into the FLEx Lexicon. See below for demonstration of\u00a0 this process. If the dictionary or word list you have access to is not in an appropriate format, try:\r\n<ul>\r\n \t<li>contacting the researchers who collected the data to see if you can get access to the list in a more usable format.<\/li>\r\n \t<li>reach out to linguists or computer scientists for help in scraping word list data from digital files<\/li>\r\n \t<li>manually enter lexical data into a spreadsheet (which may or may not be reasonable depending on the amount of data)<\/li>\r\n \t<li>use the dictionary as a reference during glossing, only inputting relevant words.<\/li>\r\n<\/ul>\r\n&nbsp;\r\n<h3>Transcribed, translated, and\/or glossed texts<\/h3>\r\nExisting texts offer opportunities to gain familiarity with the language. The value of these texts increases dramatically when they are glossed. From previous glossed texts, you can pull morphemes with their glosses to include to your FLEx Lexicon. You can also look for multiple instances of the same morpheme to notice similarities and differences in usage between contexts or inconsistencies in glossing. The inconsistencies may reveal the contested spelling of a word or morpheme, polysemy (multiple meanings), different preferences of different authors, and so on. Annotation will go faster if spelling is consistent. \u00a0Check the CoRSAL website for \u00a0an example of an spelling standardization program to help with the annotation process.\r\n\r\n&nbsp;\r\n<h3>Linguistic analyses<\/h3>\r\nAny existing linguistic analysis of a language represents a step forward for your IGT project. The analyst benefits by developing a deeper understanding of the language they are working on, and existing analyses such as morpheme glosses can be incorporated into the FLEx Lexicon, offering some shortcuts for developing your IGT conventions.\r\n<h3><\/h3>\r\n<h3>Using previous research: An example from Hakha Lai<\/h3>\r\nAs part of a collaborative US National Science Foundation Project between the University of North Texas and Indiana University, Bloomington (#2031052), our team members collected interviews from Hakha Lai (HL) speakers to explore how COVID-19 health-related information is received and understood by this population. This resulted in hours of HL spoken recordings. These were transcribed, and a rough clause-level translation was created by HL-speaking students using SayMore. In order to further translate the material, noting word and morpheme glosses, we studied all existing HL materials we could acquire. These included:\r\n<ul>\r\n \t<li>Language examples in published articles, especially useful were appendices, e.g., the appendix on Roengpitya, Rungpat (1997) Glottal stop and glottalization in Lai (connected speech) in Linguistics of the Tibeto-Burman Area 20.2<\/li>\r\n \t<li>Unpublished partially glossed texts (through direct correspondences with several researchers)<\/li>\r\n \t<li>An existing Lai-English dictionary (<a href=\"http:\/\/www.chin-dictionary.com\/\">www.chin-dictionary.com<\/a>)<\/li>\r\n \t<li>A existing revised, incomplete Lai-English dictionary (through direct correspondence with a researcher)<\/li>\r\n<\/ul>\r\nIn order to include this previous research in our FLEx Lexicon, we created, then merged the following two spreadsheets:\r\n\r\n1) Translation data from the word lists\r\n\r\n2) Grammatical glosses from existing texts and linguistic research. Since we had access to several HL texts, a word list, and previous research, we were able to put together a list of 210 morphemes with preliminary grammatical glosses based on these materials. Here is the information we pulled for each:\r\n<ol>\r\n \t<li><strong>Morpheme <\/strong>Spellings-What spellings are used in the text for the morpheme? Are there discrepancies?<\/li>\r\n \t<li><strong>HL<\/strong> <strong>Gloss<\/strong>-What glosses are used for each morpheme? Are there multiple glosses used for the same morpheme? What do these glosses mean?<\/li>\r\n \t<li><strong>Placement<\/strong>-Where does the morpheme show up? Is it part of a verb phrase or noun phrase or something else?<\/li>\r\n<\/ol>\r\nFrom here, we worked to develop a preliminary gloss for each morpheme to be imported in the FLEx Lexicon import. At that point, our team members began glossing the many interviews using the enriched Lexicon. The increased speed at which we were able to annotate was notable, with many morphemes and lexemes automatically glossed by FLEx.\r\n<div><\/div>\r\n<div>To learn more about the process of preloading your Lexicon with existing word and morpheme lists, you might watch <a href=\"https:\/\/unt.zoom.us\/rec\/share\/nTSJwEaz_Sc78QakK-6pr-cIRu6fIfrNgWc6CqrlbhwrU7qfksO9ldsEysTPkeUd.QujqBe9pKVbRqTjH?startTime=1634833024000\">this video<\/a> by our team member Marty Heaton and read the <a href=\"https:\/\/corsal.unt.edu\/56-using-flex-2\">step-by-step instructions<\/a> by team member Ben Hull.<\/div>\r\n<div class=\"_Entity _EType_OWALinkPreview _EId_OWALinkPreview _EReadonly_1\">\r\n<div id=\"LPBorder_GTaHR0cHM6Ly9zb2Z0d2FyZS5zaWwub3JnL3NoZWV0c3dpcGVyLw..\" class=\"LPBorder115192\"><\/div>\r\n<\/div>\r\n<div>\r\n<div>\r\n<div class=\"textbox shaded\">\r\n<div>Discussion:<\/div>\r\n<div>What are some existing word lists in your language? Are these in digital format? Is there agreement on what to call the bound morphemes? Do the abbreviations make sense to you? (This would be question for both non-specialists and linguists).<\/div>\r\n<\/div>\r\n<\/div>\r\n<\/div>\r\n<h2>5.4 What to gloss?<\/h2>\r\nNative speakers will often have an easier time coming up with translations of content words. The meaning of units smaller than words, like the form -ed in the English word <em>walked<\/em> can be hidden even to speakers, \u00a0They know the meaning for sure, but it is deep in their unconscious ability to speak the language. It may be hard for speakers to identify and gloss these without additional help. \u00a0In the remainder of this chapter we will talk about this idea of additional help in annotation.\r\n\r\nThe practice of creating a gloss for a morpheme first requires a decision of whether to gloss based on grammatical function, such as aspect (TNS for tense) or specific iteration of that function such as FUT \u2018future.\u2019 Again, based on our personal experiences in glossing texts from a variety of Tibeto-Burman languages, we know that, often, as we are glossing, we understand the function or grammatical category of a morpheme well ahead of the specific semantics. Here are some additional scenarios where glossing is a challenge.\r\n<div class=\"textbox textbox--examples\"><header class=\"textbox__header\">\r\n<p class=\"textbox__title\">Morphemes that require description<\/p>\r\n\r\n<\/header>\r\n<div class=\"textbox__content\">\r\n\r\n\u201cThis word means that the action was <em>performed earlier today<\/em>\u201d\r\n\r\n<\/div>\r\n<\/div>\r\n<div class=\"textbox textbox--examples\"><header class=\"textbox__header\">\r\n<p class=\"textbox__title\">Morphemes that don\u2019t change the meaning of the lexical word.<\/p>\r\n\r\n<\/header>\r\n<div class=\"textbox__content\">\r\n\r\n\u201cThat word attached to <em>eat<\/em> means <em>the process of<\/em> <em>eating<\/em>\u201d\r\n\r\n<\/div>\r\n<\/div>\r\n<div class=\"textbox textbox--examples\"><header class=\"textbox__header\">\r\n<p class=\"textbox__title\">Morphemes that don\u2019t mean the same thing when split from the lexical word.<\/p>\r\n\r\n<\/header>\r\n<div class=\"textbox__content\">\r\n\r\n\u201cThis word means that the tiger is a woman, but it doesn\u2019t mean <em>woman<\/em> by itself.\u201d\r\n\r\n<\/div>\r\n<\/div>\r\n<div class=\"textbox textbox--examples\"><header class=\"textbox__header\">\r\n<p class=\"textbox__title\">Morphemes with no clear meaning<\/p>\r\n\r\n<\/header>\r\n<div class=\"textbox__content\">\r\n\r\n\u201cI don\u2019t know what that word means.\u201d\r\n\r\n<\/div>\r\n<\/div>\r\n<p style=\"text-align: left\">You may know what the morpheme is relaying but not exactly what to call it or which category of morphology it fits into (like the first example, where we know it means something about having performed an activity earlier in the day but perhaps not being sure if it falls in the \u00a0tense or aspect category or something else). \u00a0Conversely, you may strongly suspect the category of the morpheme but not know exactly what it means. \u00a0For example, morpheme may clearly be expressing aspect but is not clearly ongoing or sporadic. \u00a0We propose that your annotation can have two parts: the part you know and, the part you need to further research. We are calling this a hierarchical way of annotating, on the left (usually the thing you know) is the super category. \u00a0This will be the general functional or semantic category of the morpheme an on the right is what you don't know, the sub category or specific semantics of the morpheme.<\/p>\r\n&nbsp;\r\n<h3>Scenario: French<\/h3>\r\nWe were offered the free translation \u201cJean could go\u201d and we asked a few questions to a native speaker, allowing us to associate English translations with each word in the French sentence.\r\n\r\n&nbsp;\r\n\r\nJean \u00a0\u00a0\u00a0 pourrais \u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 aller.\r\n\r\nJean\u00a0\u00a0\u00a0\u00a0 could\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 go\r\n\r\n\u2018Jean could go.\u2019\r\n\r\nWhile the name <em>Jean<\/em> and the verb <em>aller<\/em> \u2018go\u2019 offer more straightforward translations, <em>pourrais<\/em> is more problematic, since it is glossed with the English modal verb \u2018could\u2019 which has several possible interpretations that may or may not line up with the French word including:\r\n<ul>\r\n \t<li>Ability: Jean <em>could<\/em> go to the store (before the accident).<\/li>\r\n \t<li>Possibility: Jean <em>could<\/em> go to the store (on his way home, if he thinks of it).<\/li>\r\n<\/ul>\r\nFurthermore, a native speaker might tell us that 'pourrais':\r\n<ul>\r\n \t<li>has other forms including \u2018pouvoir\u2019 and \u2018peux\u2019 (indicating that the word has several possible forms)<\/li>\r\n \t<li>may also be translated as <em>might<\/em>, <em>can<\/em>, or <em>may <\/em>(indicating that its translation into English is contextual).<\/li>\r\n<\/ul>\r\nSo, how do you gloss it? Here are some logical options:\r\n\r\n<strong>Mark it with &lt;?&gt; and deal with it later.<\/strong> This is the easiest option, but it ignores any information collected about the morpheme in the process of glossing the text. Since it ignores collected information and moves on, it allows the glosser to cover more ground faster.\r\n\r\n<strong>Translate it with a similar word in the glossing language<\/strong> (like \u2018might\u2019 or \u2018could\u2019). The advantage of this option over the former is that it keeps track of some information discovered in the process of glossing. This option is good for lexical morphemes, but bad for grammatical ones as:\r\n<ul>\r\n \t<li>It is imprecise (<em>might<\/em> can express possibility or permission)<\/li>\r\n \t<li>It assumes more analysis exists than has been done<\/li>\r\n \t<li>It assumes translatability of grammatical morphemes between languages that may not have accurate correspondences.<\/li>\r\n \t<li>It also may lose information about the syntactic position of the morpheme since words with modal meanings may be expressed in a number of ways in a language not limited to a certain syntactic position (adverbials or verbs with complement clauses).<\/li>\r\n \t<li>It makes the text less searchable, since someone looking to do analysis on modals would have to use a set of English modals (could, would, should, must, can, etc.) to find the data to begin analysis. In other words, it is not tagged in a way that facilitates research.<\/li>\r\n<\/ul>\r\nThis option has some of the advantages of (2) in that it keeps information gained from the process of glossing. It is more precise (\u201cthis morpheme expresses possibility\u201d) than giving it a loose translation, but it suffers from the same searchability problem. If morphemes are marked with whatever the researcher feels is the closest analysis, how do you search to find all things with like distribution to further analyze the data?\r\n\r\n<strong>Mark it as a \u2018modal\u2019 or \u2018verbal auxiliary\u2019 element<\/strong>. This option offers a broad category label and is more searchable, but loses more nuanced information about semantics gleaned from the process of glossing.\r\n\r\nThe following chapter considers how to deal with this situation using IGT Workflow.","rendered":"<p>&nbsp;<\/p>\n<div class=\"textbox textbox--learning-objectives\">\n<header class=\"textbox__header\">\n<p class=\"textbox__title\">Learning Objectives:<\/p>\n<\/header>\n<\/div>\n<div class=\"textbox\">\n<ol>\n<li>Identify previous sources on lexeme- and morpheme-level glossing<\/li>\n<li>Create an annotation tool kit for the language<\/li>\n<li>Learn to build an excel spreadsheet for Sheet Swiper<\/li>\n<li>Learn about hierarchical annotation<\/li>\n<\/ol>\n<p>.<\/p>\n<\/div>\n<p>&nbsp;<\/p>\n<h2>5.1 Previous research<\/h2>\n<p>Interlinear Glossed Texts (IGT) are used to turn transcribed and translated recordings into data that are usable by communities, linguists, and computer scientists. The amount of effort spent on creating IGT is in an inverse relationship with the amount of available resources there are in the language of analysis. This means that the more grammatical descriptions, dictionary materials, and linguistic analyses you consider in your preparation, the faster and less labor-intensive the process of preparing IGT will be.<\/p>\n<p>An IGT annotation is typically a bundle of annotation:<\/p>\n<p>&nbsp;<\/p>\n<p>Line 1. tx\/ Transcription (sometimes called the baseline)<\/p>\n<p>Line 2. mr\/ Morphological representation (break the word down into its morphological parts, i.e. affixes and roots)<\/p>\n<p>Line 3 mg\/ Morpheme gloss (the meaning of each morpheme)<\/p>\n<p>Line 4 wg\/ Word gloss<\/p>\n<p>Line 5 ft\/ Free translation<\/p>\n<p>Line 6 Note\/<\/p>\n<p>&nbsp;<\/p>\n<p>And illustrated in the following Manipuri example (Chelliah 1997, p. 225)<\/p>\n<p>tx\/ \u00a0 \u00a0 \u00a0 \u00a0 m\u00e1n\u0259 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 ap\u0259l \u00a0 \u00a0 \u00a0 \u00a0 \u010d\u00e1l\u0259g\u00ed<\/p>\n<p>mr\/ \u00a0 \u00a0 \u00a0 m\u00e1 -n\u0259 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 ap\u0259l \u00a0 \u00a0 \u00a0 \u00a0\u010d\u00e1 -l\u0259k -\u00ed<\/p>\n<p>mg\/ \u00a0 \u00a0 \u00a0he =IS:FOC \u00a0 apple \u00a0 \u00a0 \u00a0eat -AM:VEN -SMOD:DECL<\/p>\n<p>wg\/ \u00a0 \u00a0 \u00a0 he \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0apple \u00a0 \u00a0 \u00a0 eating<\/p>\n<p>ft\/ 'When he came here he was eating an apple.'<\/p>\n<p>&nbsp;<\/p>\n<p>We will be reviewing the use of a common annotation program called FLEx which uses a Lexicon created by the manual input of data. One way of growing the lexicon is by adding words to the lexicon as you gloss a text - as you translate a text, you provide glosses for the words and morphemes in the clause and these are added to the lexicon. \u00a0The next time that word or morpheme appears, the program will find the stored gloss in the Lexicon and provide that annotation automatically. The richer the Lexicon, the faster the annotation.<\/p>\n<div class=\"textbox\">Discussion:\u00a0 Deconstruct the small IGT sample from Manipuri provided above. For example, discuss what is meant by a dash, what is meant by the equal sign, why some glosses are in capital letters and others in lower case. Why is there a period in 3.POS? What information about the language can be gleaned from this example? What other questions might you investigate based on this example?<\/div>\n<p>Based on our experience in creating IGT, we suggest that, before you begin your annotation project, collect as much information from previous resources as you can. When no previous research has been done on the language you\u2019re working on, look to surrounding and related languages. You may also want to contact researchers who have worked on these languages for their unpublished works as well. Previous research you find will help you:<\/p>\n<ul>\n<li>Develop familiarity with the language and\/or language families so you can more readily recognize constituent boundaries and structure<\/li>\n<li>Identify relevant areas of typological interest to locate areas more complicated to translate<\/li>\n<li>Preload your lexicon with content words, function words, and, if possible, bound morphemes to support faster automatic glossing<\/li>\n<li>Better understand the phonology and orthography so that you can provide a consistent transcription or baseline for annotation<\/li>\n<li>Think through abbreviation conventions for morphemes and grammatical categories that will be needed<\/li>\n<li>Determine the closed categories of functional word and morphemes. \u00a0Identify some of their members. \u00a0For example, semantic role marking would a closed category and a member of that category would be agentive role.<\/li>\n<\/ul>\n<div class=\"textbox\">\n<h4>Discussion<\/h4>\n<p>Is there a linguistics researcher that has done work with your community? Who has these materials, and can you get access to them? Is the language described the same variety as yours? Do you find previous materials to be an accurate representation of your language?<\/p>\n<\/div>\n<p>&nbsp;<\/p>\n<h2>5.2 Types of Documents<\/h2>\n<p>Here are some types of documents you may be able to find and their potential usefulness to your project:<\/p>\n<h3>Word lists and dictionaries<\/h3>\n<p>The data from these texts can be formatted for input into a FLEx Lexicon to automatically populate your glosses. The more words that are inputted into your FLEx Lexicon to begin with, the fewer you have to manually add.<\/p>\n<p>The most accessible word list files are spreadsheets, which can be modified for import into the FLEx Lexicon. See below for demonstration of\u00a0 this process. If the dictionary or word list you have access to is not in an appropriate format, try:<\/p>\n<ul>\n<li>contacting the researchers who collected the data to see if you can get access to the list in a more usable format.<\/li>\n<li>reach out to linguists or computer scientists for help in scraping word list data from digital files<\/li>\n<li>manually enter lexical data into a spreadsheet (which may or may not be reasonable depending on the amount of data)<\/li>\n<li>use the dictionary as a reference during glossing, only inputting relevant words.<\/li>\n<\/ul>\n<p>&nbsp;<\/p>\n<h3>Transcribed, translated, and\/or glossed texts<\/h3>\n<p>Existing texts offer opportunities to gain familiarity with the language. The value of these texts increases dramatically when they are glossed. From previous glossed texts, you can pull morphemes with their glosses to include to your FLEx Lexicon. You can also look for multiple instances of the same morpheme to notice similarities and differences in usage between contexts or inconsistencies in glossing. The inconsistencies may reveal the contested spelling of a word or morpheme, polysemy (multiple meanings), different preferences of different authors, and so on. Annotation will go faster if spelling is consistent. \u00a0Check the CoRSAL website for \u00a0an example of an spelling standardization program to help with the annotation process.<\/p>\n<p>&nbsp;<\/p>\n<h3>Linguistic analyses<\/h3>\n<p>Any existing linguistic analysis of a language represents a step forward for your IGT project. The analyst benefits by developing a deeper understanding of the language they are working on, and existing analyses such as morpheme glosses can be incorporated into the FLEx Lexicon, offering some shortcuts for developing your IGT conventions.<\/p>\n<h3><\/h3>\n<h3>Using previous research: An example from Hakha Lai<\/h3>\n<p>As part of a collaborative US National Science Foundation Project between the University of North Texas and Indiana University, Bloomington (#2031052), our team members collected interviews from Hakha Lai (HL) speakers to explore how COVID-19 health-related information is received and understood by this population. This resulted in hours of HL spoken recordings. These were transcribed, and a rough clause-level translation was created by HL-speaking students using SayMore. In order to further translate the material, noting word and morpheme glosses, we studied all existing HL materials we could acquire. These included:<\/p>\n<ul>\n<li>Language examples in published articles, especially useful were appendices, e.g., the appendix on Roengpitya, Rungpat (1997) Glottal stop and glottalization in Lai (connected speech) in Linguistics of the Tibeto-Burman Area 20.2<\/li>\n<li>Unpublished partially glossed texts (through direct correspondences with several researchers)<\/li>\n<li>An existing Lai-English dictionary (<a href=\"http:\/\/www.chin-dictionary.com\/\">www.chin-dictionary.com<\/a>)<\/li>\n<li>A existing revised, incomplete Lai-English dictionary (through direct correspondence with a researcher)<\/li>\n<\/ul>\n<p>In order to include this previous research in our FLEx Lexicon, we created, then merged the following two spreadsheets:<\/p>\n<p>1) Translation data from the word lists<\/p>\n<p>2) Grammatical glosses from existing texts and linguistic research. Since we had access to several HL texts, a word list, and previous research, we were able to put together a list of 210 morphemes with preliminary grammatical glosses based on these materials. Here is the information we pulled for each:<\/p>\n<ol>\n<li><strong>Morpheme <\/strong>Spellings-What spellings are used in the text for the morpheme? Are there discrepancies?<\/li>\n<li><strong>HL<\/strong> <strong>Gloss<\/strong>-What glosses are used for each morpheme? Are there multiple glosses used for the same morpheme? What do these glosses mean?<\/li>\n<li><strong>Placement<\/strong>-Where does the morpheme show up? Is it part of a verb phrase or noun phrase or something else?<\/li>\n<\/ol>\n<p>From here, we worked to develop a preliminary gloss for each morpheme to be imported in the FLEx Lexicon import. At that point, our team members began glossing the many interviews using the enriched Lexicon. The increased speed at which we were able to annotate was notable, with many morphemes and lexemes automatically glossed by FLEx.<\/p>\n<div><\/div>\n<div>To learn more about the process of preloading your Lexicon with existing word and morpheme lists, you might watch <a href=\"https:\/\/unt.zoom.us\/rec\/share\/nTSJwEaz_Sc78QakK-6pr-cIRu6fIfrNgWc6CqrlbhwrU7qfksO9ldsEysTPkeUd.QujqBe9pKVbRqTjH?startTime=1634833024000\">this video<\/a> by our team member Marty Heaton and read the <a href=\"https:\/\/corsal.unt.edu\/56-using-flex-2\">step-by-step instructions<\/a> by team member Ben Hull.<\/div>\n<div class=\"_Entity _EType_OWALinkPreview _EId_OWALinkPreview _EReadonly_1\">\n<div id=\"LPBorder_GTaHR0cHM6Ly9zb2Z0d2FyZS5zaWwub3JnL3NoZWV0c3dpcGVyLw..\" class=\"LPBorder115192\"><\/div>\n<\/div>\n<div>\n<div>\n<div class=\"textbox shaded\">\n<div>Discussion:<\/div>\n<div>What are some existing word lists in your language? Are these in digital format? Is there agreement on what to call the bound morphemes? Do the abbreviations make sense to you? (This would be question for both non-specialists and linguists).<\/div>\n<\/div>\n<\/div>\n<\/div>\n<h2>5.4 What to gloss?<\/h2>\n<p>Native speakers will often have an easier time coming up with translations of content words. The meaning of units smaller than words, like the form -ed in the English word <em>walked<\/em> can be hidden even to speakers, \u00a0They know the meaning for sure, but it is deep in their unconscious ability to speak the language. It may be hard for speakers to identify and gloss these without additional help. \u00a0In the remainder of this chapter we will talk about this idea of additional help in annotation.<\/p>\n<p>The practice of creating a gloss for a morpheme first requires a decision of whether to gloss based on grammatical function, such as aspect (TNS for tense) or specific iteration of that function such as FUT \u2018future.\u2019 Again, based on our personal experiences in glossing texts from a variety of Tibeto-Burman languages, we know that, often, as we are glossing, we understand the function or grammatical category of a morpheme well ahead of the specific semantics. Here are some additional scenarios where glossing is a challenge.<\/p>\n<div class=\"textbox textbox--examples\">\n<header class=\"textbox__header\">\n<p class=\"textbox__title\">Morphemes that require description<\/p>\n<\/header>\n<div class=\"textbox__content\">\n<p>\u201cThis word means that the action was <em>performed earlier today<\/em>\u201d<\/p>\n<\/div>\n<\/div>\n<div class=\"textbox textbox--examples\">\n<header class=\"textbox__header\">\n<p class=\"textbox__title\">Morphemes that don\u2019t change the meaning of the lexical word.<\/p>\n<\/header>\n<div class=\"textbox__content\">\n<p>\u201cThat word attached to <em>eat<\/em> means <em>the process of<\/em> <em>eating<\/em>\u201d<\/p>\n<\/div>\n<\/div>\n<div class=\"textbox textbox--examples\">\n<header class=\"textbox__header\">\n<p class=\"textbox__title\">Morphemes that don\u2019t mean the same thing when split from the lexical word.<\/p>\n<\/header>\n<div class=\"textbox__content\">\n<p>\u201cThis word means that the tiger is a woman, but it doesn\u2019t mean <em>woman<\/em> by itself.\u201d<\/p>\n<\/div>\n<\/div>\n<div class=\"textbox textbox--examples\">\n<header class=\"textbox__header\">\n<p class=\"textbox__title\">Morphemes with no clear meaning<\/p>\n<\/header>\n<div class=\"textbox__content\">\n<p>\u201cI don\u2019t know what that word means.\u201d<\/p>\n<\/div>\n<\/div>\n<p style=\"text-align: left\">You may know what the morpheme is relaying but not exactly what to call it or which category of morphology it fits into (like the first example, where we know it means something about having performed an activity earlier in the day but perhaps not being sure if it falls in the \u00a0tense or aspect category or something else). \u00a0Conversely, you may strongly suspect the category of the morpheme but not know exactly what it means. \u00a0For example, morpheme may clearly be expressing aspect but is not clearly ongoing or sporadic. \u00a0We propose that your annotation can have two parts: the part you know and, the part you need to further research. We are calling this a hierarchical way of annotating, on the left (usually the thing you know) is the super category. \u00a0This will be the general functional or semantic category of the morpheme an on the right is what you don't know, the sub category or specific semantics of the morpheme.<\/p>\n<p>&nbsp;<\/p>\n<h3>Scenario: French<\/h3>\n<p>We were offered the free translation \u201cJean could go\u201d and we asked a few questions to a native speaker, allowing us to associate English translations with each word in the French sentence.<\/p>\n<p>&nbsp;<\/p>\n<p>Jean \u00a0\u00a0\u00a0 pourrais \u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 aller.<\/p>\n<p>Jean\u00a0\u00a0\u00a0\u00a0 could\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 go<\/p>\n<p>\u2018Jean could go.\u2019<\/p>\n<p>While the name <em>Jean<\/em> and the verb <em>aller<\/em> \u2018go\u2019 offer more straightforward translations, <em>pourrais<\/em> is more problematic, since it is glossed with the English modal verb \u2018could\u2019 which has several possible interpretations that may or may not line up with the French word including:<\/p>\n<ul>\n<li>Ability: Jean <em>could<\/em> go to the store (before the accident).<\/li>\n<li>Possibility: Jean <em>could<\/em> go to the store (on his way home, if he thinks of it).<\/li>\n<\/ul>\n<p>Furthermore, a native speaker might tell us that 'pourrais':<\/p>\n<ul>\n<li>has other forms including \u2018pouvoir\u2019 and \u2018peux\u2019 (indicating that the word has several possible forms)<\/li>\n<li>may also be translated as <em>might<\/em>, <em>can<\/em>, or <em>may <\/em>(indicating that its translation into English is contextual).<\/li>\n<\/ul>\n<p>So, how do you gloss it? Here are some logical options:<\/p>\n<p><strong>Mark it with &lt;?&gt; and deal with it later.<\/strong> This is the easiest option, but it ignores any information collected about the morpheme in the process of glossing the text. Since it ignores collected information and moves on, it allows the glosser to cover more ground faster.<\/p>\n<p><strong>Translate it with a similar word in the glossing language<\/strong> (like \u2018might\u2019 or \u2018could\u2019). The advantage of this option over the former is that it keeps track of some information discovered in the process of glossing. This option is good for lexical morphemes, but bad for grammatical ones as:<\/p>\n<ul>\n<li>It is imprecise (<em>might<\/em> can express possibility or permission)<\/li>\n<li>It assumes more analysis exists than has been done<\/li>\n<li>It assumes translatability of grammatical morphemes between languages that may not have accurate correspondences.<\/li>\n<li>It also may lose information about the syntactic position of the morpheme since words with modal meanings may be expressed in a number of ways in a language not limited to a certain syntactic position (adverbials or verbs with complement clauses).<\/li>\n<li>It makes the text less searchable, since someone looking to do analysis on modals would have to use a set of English modals (could, would, should, must, can, etc.) to find the data to begin analysis. In other words, it is not tagged in a way that facilitates research.<\/li>\n<\/ul>\n<p>This option has some of the advantages of (2) in that it keeps information gained from the process of glossing. It is more precise (\u201cthis morpheme expresses possibility\u201d) than giving it a loose translation, but it suffers from the same searchability problem. If morphemes are marked with whatever the researcher feels is the closest analysis, how do you search to find all things with like distribution to further analyze the data?<\/p>\n<p><strong>Mark it as a \u2018modal\u2019 or \u2018verbal auxiliary\u2019 element<\/strong>. This option offers a broad category label and is more searchable, but loses more nuanced information about semantics gleaned from the process of glossing.<\/p>\n<p>The following chapter considers how to deal with this situation using IGT Workflow.<\/p>\n","protected":false},"author":19,"menu_order":5,"template":"","meta":{"_monsterinsights_skip_tracking":false,"_monsterinsights_sitenote_active":false,"_monsterinsights_sitenote_note":"","_monsterinsights_sitenote_category":0,"pb_show_title":"on","pb_short_title":"","pb_subtitle":"","pb_authors":[],"pb_section_license":""},"chapter-type":[],"contributor":[],"license":[],"class_list":["post-101","chapter","type-chapter","status-publish","hentry"],"part":3,"_links":{"self":[{"href":"https:\/\/openbooks.library.unt.edu\/sourcetoanalysis\/wp-json\/pressbooks\/v2\/chapters\/101","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/openbooks.library.unt.edu\/sourcetoanalysis\/wp-json\/pressbooks\/v2\/chapters"}],"about":[{"href":"https:\/\/openbooks.library.unt.edu\/sourcetoanalysis\/wp-json\/wp\/v2\/types\/chapter"}],"author":[{"embeddable":true,"href":"https:\/\/openbooks.library.unt.edu\/sourcetoanalysis\/wp-json\/wp\/v2\/users\/19"}],"version-history":[{"count":29,"href":"https:\/\/openbooks.library.unt.edu\/sourcetoanalysis\/wp-json\/pressbooks\/v2\/chapters\/101\/revisions"}],"predecessor-version":[{"id":323,"href":"https:\/\/openbooks.library.unt.edu\/sourcetoanalysis\/wp-json\/pressbooks\/v2\/chapters\/101\/revisions\/323"}],"part":[{"href":"https:\/\/openbooks.library.unt.edu\/sourcetoanalysis\/wp-json\/pressbooks\/v2\/parts\/3"}],"metadata":[{"href":"https:\/\/openbooks.library.unt.edu\/sourcetoanalysis\/wp-json\/pressbooks\/v2\/chapters\/101\/metadata\/"}],"wp:attachment":[{"href":"https:\/\/openbooks.library.unt.edu\/sourcetoanalysis\/wp-json\/wp\/v2\/media?parent=101"}],"wp:term":[{"taxonomy":"chapter-type","embeddable":true,"href":"https:\/\/openbooks.library.unt.edu\/sourcetoanalysis\/wp-json\/pressbooks\/v2\/chapter-type?post=101"},{"taxonomy":"contributor","embeddable":true,"href":"https:\/\/openbooks.library.unt.edu\/sourcetoanalysis\/wp-json\/wp\/v2\/contributor?post=101"},{"taxonomy":"license","embeddable":true,"href":"https:\/\/openbooks.library.unt.edu\/sourcetoanalysis\/wp-json\/wp\/v2\/license?post=101"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}