Eesti lastekeele korpuse morfoloogilise märgendamise kitsaskohtadest

Abstract

Artikli eesmärk on anda ülevaade sellest, mis raskendab avalikult kättesaadava eesti lastekeele korpuse automaatset morfoloogilist märgendamist ning anda soovitusi, kuidas tulevaste korpuste märgendamist ja standardiseerimist analüüsi tarbeks paremaks muuta. Analüüsisime korpust kirjakeelele mõeldud morfoloogiaanalüsaatori abil. Automaatse analüüsi järel vaatasime, kui suur osa sõnadest sai analüüsi või jäi analüsaatorile tundmatuks nii alamkorpuste kui lapse- ja hoidjakeele lõikes. Selgus, et analüüsi saanud sõnade osakaal igas alamkorpuses varieerus hoidjakeeles 94–98% ja lastekeeles 57–96% vahel. Suurt rolli mängib lindistuste üleskirjutamisviis: hoidjakeelt kirjutatakse üles kirjakeelele sarnaselt, kuid lastekeeles lähtutakse kuuldeortograafiast. Kõik alamkorpused küll järgivad CHILDES-i transkriptsioonisüsteemi ettekirjutusi, ent iga alamkorpus on koostatud erinevaid eesmärke silmas pidades ja on erineva transkribeerimisstiiliga, millest järjepidevalt kinni ei peeta. Märgenduse tulemust hindasime käsitsi läbivaatamise teel. Artiklis toome välja, millised olid nii tundmatuks jäänud kui vale analüüsi saanud sõnade sagedasemad probleemid ja pakume võimalikke lahendusi.

"Issues in morphological annotation of the Estonian child language corpus"

This article presents the results of an initial attempt to automatically annotate the currently existing, publicly available Estonian child language corpus morphologically. CLAN software is not suitable for morphological analysis of Estonian, but Estonian language technology resources are available for written language and can be adapted to spoken language and specific genres. The automatic parser provided annotation for 92–98% of words in the child-directed speech and 57–96% of the child speech, with the results for child speech varying across corpora. A manual analysis was also conducted of words which were automatically annotated in a random selection of transcriptions from each corpus. Across corpora, 63–96% of annotated words were correctly annotated. Reasons for the variation are discussed, and obstacles to automatic annotation are identified at various levels.

First, the corpora have been collected and transcribed with various goals and according to differing principles, hence the style and detail of transcription vary greatly across the corpora. Second, even within a single corpus, discrepancies appear in coding which need to be uniformly resolved in order to ensure accurate morphological annotation. Finally, for flagging non-standard or idiosyncratic forms, the implementation of metacodes available for use in the child language corpora would greatly assist the task of automatic morphological parsing. For each corpus, a user dictionary adapted to the particular genre and the particular corpus would need to be developed, including proper names and idiosyncratic words. The marking of errors is a crucial area which needs to be standardised in order to enable automatic annotation. Additionally, five groups of words which received inaccurate annotation were identified, and suggestions are made for transcription of child language corpora in order to ease the task of morphological annotation in the future.