Stammdaten

Titel: The SD error taxonomy and agreement. A validation study
Beschreibung:

Every kind of an attempt of classification which is based on human discretion is inherently flawed in terms of objectivity and agreement. Classifying anything related to language is the most extreme example of such inherent problematic nature for several reasons: there is no fixed correct ‘finite’ version of a language performance to be set as a benchmark; there is a problem of comparing various classification attempts; and there are no fully reliable statistic measurements which successfully address these issues of inconsistency in language analysis .There is no set value for most of language competences as an acceptable phrase, clause or a sentence, relaying the same semantic content, can have theoretically an unlimited number of variations. If, for example, in error analysis a rater analyses a given performance comparing it with the variety of the acceptable or correct form she or he might have in mind, it might produce a completely different error analysis from someone else doing the same but with a different version of the correct or acceptable form even if the norm they have as means of evaluation to be referred to is fully agreed upon. The paper explores the extent to which a unification of the procedure of error analysis can improve on this unreliability of classification in language analysis and focuses on the measurement of inter-annotator agreement. The study explores the effects a standardized error taxonomy (SD Error Taxonomy) and a standardized process of error analysis can have on the overall result of error analysis among multiple trained raters. The results show that even though full agreement can hardly be expected in language /error) analysis, even in highly controlled conditions of a pilot project, employing a unified error taxonomy and error analysis procedure together with extensive rater training produces higher agreement and results in more validity and reliability.

Schlagworte:
Typ: Angemeldeter Vortrag
Homepage: -
Veranstaltung: CALK 2014 (Klagenfurt)
Datum: 26.09.2014

Zuordnung

Organisation Adresse
Fakultät für Kulturwissenschaften
 
Institut für Anglistik und Amerikanistik
Universitätstraße 65 - 67
9020  Klagenfurt
Österreich
   anglistik@aau.at
https://www.aau.at/en/english/
zur Organisation
Universitätstraße 65 - 67
AT - 9020  Klagenfurt
Universität Klagenfurt
 
Zentraler Informatikdienst (ZID)
Universitätsstr. 65-67
A-9020  Klagenfurt
Österreich
  +43 463 2700 9600
   zid@aau.at
http://zid.aau.at
zur Organisation
Universitätsstr. 65-67
AT - A-9020  Klagenfurt

Kategorisierung

Sachgebiete
  • 602008 - Anglistik
  • 503011 - Fachdidaktik Geisteswissenschaften
Forschungscluster
  • Bildungsforschung
Vortragsfokus
  • Science to Science (Qualitätsindikator: n.a.)
Klassifikationsraster der zugeordneten Organisationseinheiten:
  • Für die zugeordneten Organisationseinheiten sind keine Klassifikationsraster vorhanden
TeilnehmerInnenkreis
  • Überwiegend international
Publiziert?
  • Nein
Arbeitsgruppen Keine Arbeitsgruppe ausgewählt

Kooperationen

Keine Kooperationspartner ausgewählt