Lecture: Error annotation by means of the Scope – Substance Error Taxonomy

Master data
Title: Error annotation by means of the Scope – Substance Error Taxonomy
Description:

Since the beginning of error analysis as an applied linguistics discipline a variety of error taxonomies have been proposed. Broadly speaking, they are based either on linguistic categories or on surface-structure descriptions (Dobric and Sigott 2014; Sigott, Cesnik and Dobric 2016; Pibal and Sigott, forthcoming). One of the problems inherent in these taxonomies is the high degree of subjectivity and the resulting lack of annotator agreement. The Scope – Substance error taxonomy takes an alternative approach to error description by using the concepts of scope and substance to describe errors. In principle, this distinction was already suggested by Lennon (1991), who used ‘extent’ to refer to scope and ‘domain’ to refer to substance. Scope refers to the linguistic and extralinguistic context that needs to be taken into account for an error to be noticed whereas substance designates the size of the linguistic structure that needs to be changed in order for the error to disappear. Levels of scope and substance are described in terms of word, phrase, clause, sentence and text (Quirk et al. 1985). This enables errors to be described in terms of fourteen combinations of scope and substance. We contend that this approach to error description should leave less room for inividual interpretation because the model of grammatical analysis that it is based on is made explicit

In order to investigate annotator agreement reached on the basis of the taxonomy, a pilot study with trained student annotators was conducted. Annotator agreement was expressed in terms of Error Location Density Indices developed for this purpose. The pilot study has shown that while the approach has potential, annotator agreement was still low. This was attributed to a lack of detail in the instructions provided for annotators. Consequently, the guidelines for the application of the taxonomy were refined. They now contain instructions for setting up an authoritative reconstruction of the learner text by applying the principle of minimal correction, instructions for dealing with unitary constituency, nested error, multi-level error and multiple error. The refined guidelines also contain instructions for dealing with missing constituents and superfluous constituents. Moreover, guidelines for dealing with punctuation errors have been formulated.

Currently, a second pilot study of the taxonomy including the refined guidelines is in progress. The same learner texts will be used. In addition to a student annotator group a second annotator group will be recruited from university language teaching staff in order to investigate the effect of differences in language proficiency. The results will be available later in the year and will be reported in the presentation.

Keywords: error annotation, learner corpora, inter-annotator agreement, error location density index, second language writing
Keywords:
Type: Poster presentation
Homepage: http://lcr2017.eurac.edu/
Event: 4th Learner Corpus Research Conference (Bolzano)
Date: 06.10.2017
lecture status:

Assignment
Organisation Address
Fakultät für Kultur- und Bildungswissenschaften
  Institut für Anglistik und Amerikanistik
Universitätstraße 65 - 67
9020 Klagenfurt
Austria
   anglistik@aau.at
https://www.aau.at/en/english/
To organisation
Universitätstraße 65 - 67
AT - 9020  Klagenfurt
Universität Klagenfurt
  Zentraler Informatikdienst (ZID)
Universitätsstr. 65-67
A-9020 Klagenfurt
Austria
  +43 463 2700 9600
   zid@aau.at
http://zid.aau.at
To organisation
Universitätsstr. 65-67
AT - A-9020  Klagenfurt

Categorisation
Subject areas
  • 602007 - Applied linguistics
Research Cluster
  • Educational research
Focus of lecture
  • Science to Science (Quality indicator: n.a.)
Classification raster of the assigned organisational units:
  • No classification raster available for the assigned organisational units.
Group of participants
  • Mainly international
Published?
  • Yes
working groups
  • Angewandte Linguistik

Cooperations
No partner organisations selected