Developing a c-test to measure language ability as an alternative to a skills-based test

Ingrid Sarapuu, Ene Alas

Abstract


The article investigates the properties of a c-test and its ability to measure test-takers’ overall English language proficiency in the Estonian context. For this purpose, prior research concerning c-test validity and reliability are consulted, and the c- test’s assets with relation to a skills-based proficiency evaluation instrument are sought. The article then discusses the process of developing a c-test following the procedure recommended by Grotjahn (1987) and Raatz and Klein-Braley (2002), and piloting it among Estonian gymnasium students who simultaneously took the skills-based national examination in the English language. Statistical analysis displays very strong correlations between the c-test results and those of the national examination, as well as with teacher evaluation of the test subjects’ proficiency, substantiating c-test’s viability as an economical language ability measure in contexts where quick appraisal of the respective ability is required. The study disclosed implications for language proficiency assessment practices as well as for the process of c-test development.

C-test keeleoskuse mõõtmiseks osaoskustesti alternatiivina

Artiklis vaadeldakse keelepädevust kui konstrukti ja näidatakse, kuidas konstrukti erinev defineerimine on viinud keeleoskuse testide erineva ülesehituseni, lähtudes sellest, kas keelt nähakse osaoskuste kogumina või ühtse jagamatu oskusena. Vaadeldes keelt jagamatu oskusena on osaoskustel põhinevate testide kõrvale tekkinud teisi alternatiivseid testi-tüüpe, millest üheks on c-test. Artiklis analüüsitakse c-testi omadusi ning püütakse selgusele jõuda, kas sellist testitüüpi on võimalik kasutada üldise inglise keele pädevuse hindamiseks Eesti kontekstis. Lähtudes Grotjahni (1987) ning Raatzi ja Klein-Braley (2002) mudelist, vaadeldakse kõigepealt c-testi koostamise ja administreerimise protsessi gümnaasiumi kontekstis paralleelselt riigieksamiga. C-testi katsetamise tulemusi võrreldakse seejärel samade gümnaasiumiõpilaste poolt sooritatud osaoskustel põhineva keelepädevustesti tulemustega ning õpilaste õpetajapoolsete pädevushinnangutega. Uurimusest selgub, et kuigi c-testil on olulisi puudusi, on c-testi korrelatsiooniindeksid nii osaoskustel põhineva pädevustestiga kui ka õpetajapoolsete hinnangutega väga kõrged, lubades pidada c-testi usaldusväärseks keeleoskuse hindamise instrumendiks, mille peamisteks eelisteks on testi koostamise suhteline kiirus ja madal administreerimiskulu.


Keywords


language assessment, overall language ability, test development, validity, reliability, national examination

Full Text:

PDF

References


Bachman, Lyle 1990. Fundamental Considerations in Language Testing. Oxford, UK: Oxford University Press.

Bachman, Lyle; Palmer, Adrian 1996. Language Testing in Practice. Oxford, UK: Oxford University Press.

Canale, Michael; Swain, Merrill 1980. Theoretical bases of communicative approaches to language teaching and testing. – Applied Linguistics, 1 (1), 1–47. http://dx.doi.org/10.1093/applin/1.1.1

Chapelle, Carol A.; Abraham, Roberta G. 1990. Cloze method: what difference does it make? – Language Testing, 7 (2), 121–146. http://dx.doi.org/10.1177/026553229000700201

Coleman, James A. 1994. Profiling the advanced language learner: The c-test in British further and higher education. – Rüdiger Grotjahn (Ed.), Der C-Test. Theoretische Grundlagen und Praktische Anwendungen, Vol. 2. Bochum: Brockmeyer, 217–237.

Coleman, James A. 1996. A comparative survey of the proficiency and progress of language learners in British universities. – Rüdiger Grotjahn (Ed.), Der C-Test. Theoretische Grundlagen und praktische Anwendungen, Vol. 3. Bochum: Brockmeyer, 367–399.

Eckes, Thomas; Grotjahn, Rüdiger 2006. A closer look at the construct validity of c-tests. – Language Testing, 23 (3), 290–325. http://dx.doi.org/10.1191/0265532206lt330oa

Fulcher, Glenn 2010. Practical Language Testing. Hodder Education.

Grotjahn, Rüdiger; Stemmer, Brigitte 2002. C-tests and language processing. – J. Coleman, R. Grotjahn, U. Raatz (Eds.), University Language Testing and the C-Test. Bochum: AKS-Verlag, 115–130.

Grotjahn, Rüdiger 1987. How to construct and evaluate a c-test: A discussion of some problems and some statistical analyses. – R. Grotjahn, C. Klein-Braley, D. K. Stevenson (Eds.), Taking Their Measure: The Validity and Validation of Language Test. Bochum: Brockmeyer, 219–253.

Gümnaasiumi riiklik õppekava. http://www.oppekava.ee/index.php/G%C3%BCmnaasiumi_riiklik_%C3%B5ppekava (10.3.2016).

Harsch, Claudia 2014. General language proficiency revisited: Current and future issues. – Language Assessment Quarterly, 11 (2), 152–169. http://dx.doi.org/10.1080/15434303.2014.902059

Hulstijn, Jan H. 2007.The shaky ground beneath the CEFR: quantitative and qualitative dimensions of language proficiency. – Modern Language Journal, 91 (4), 663–667. http://dx.doi.org/10.1111/j.1540-4781.2007.00627_5.x

Hulstijn, Jan H. 2015. Language Proficiency in Native and Non-Native Speakers: Theory and Research. John Benjamins Publishing Company. http://dx.doi.org/10.1075/lllt.41

Hymes, Dell H. 1972. On communicative competence. – J. B. Pride, J. Holmes (Eds.), Sociolinguistics. Selected Readings. Harmondsworth: Penguin, 269–293.

Inglise keele riigieksami eristuskiri. http://www.innove.ee/UserFiles/erituskirjad_2015/RE%20inglise%20keel%20eristuskiri%202015.pdf (3.4.2015).

Klein-Braley, Christine 1996. Towards a theory of c-test processing. – R. Grotjahn (Ed.), Der C-Test. Theoretische Grundlagen und Praktische Anwendungen, Vol. 3. Bochum: Brockmeyer, 23–94.

Klein-Braley, Christine 2002. Psycholinguistics of c-test taking. – J. Coleman, R. Grotjahn, U. Raatz (Eds.), University Language Testing and the C-Test. Bochum: AKS-Verlag, 131–142.

Klein-Braley, Christine; Raatz, Ulrike 1984. A survey of research on the c-test. – Language Testing, 1 (2), 134–146. http://dx.doi.org/10.1177/026553228400100202

Koller, Gerhard; Zahn, Rosemary 1996. Computer based construction and evaluation of C-tests. – R. Grotjahn (Ed.), Der C-Test. Theoretische Grundlagen und praktische Anwendungen, Vol. 3. Bochum: Brockmeyer, 401–418.

Köberl, Johann; Sigott, Günther 1994. Adjusting c-test difficulty in German. – R. Grotjahn (Ed.), Der C-Test. Theoretische Grundlagen und Praktische Anwendungen, Vol. 2. Bochum: Brockmeyer, 179–192.

Linneman, Markus; Wilbert, Jürgen 2010. The C-Test: A valid instrument for screening language skills and reading comprehension of children with learning problems? – R. Grotjahn (Ed.), The C-Test: Contributions from Current Research. Frankfurt a.M.: Lang, 113–124.

Oller, John W. 1979. Language Tests at School: A Pragmatic Approach. London: Longman.

PSPP https: //www.gnu.org/software/pspp/ (15.12.2015).

Purpura, James E. 2008. Assessing communicative language ability: Models and their components. – Elana Shohamy, Nancy H. Hornberger (Eds.), Language Testing and Assessment. Encyclopedia of Language and Education. Springer, 53–68.

Raatz, Ulrich; Klein-Braley, Christine 2002. Introduction to language testing and to c-tests. – J. Coleman, R. Grotjahn, U. Raatz (Eds.), University Language Testing and the C-Test. Bochum: AKS-Verlag, 75–91.

Riigieksamite materjalid 2013. http: //www.innove.ee/et/riigieksamid/riigieksamite-materjalid/riigieksamite-materjalid-2013 (15.12.2015).

Sigott, Günther; Köberl, Johann 1996. Deletion patterns and c-test difficulty across languages. – R. Grotjahn (Ed.), Der C-Test. Theoretische Grundlagen und praktische Anwendungen, Vol. 3. Bochum: Brockmeyer, 159–172.

Sigott, Günther 2004. Towards Identifying the C-Test Construct. Peter Lang Verlag.

Wilmes, Carsten 2007. Validation of a German Language Placement Test Based on a Modified C-test Procedure. Dissertation.




DOI: http://dx.doi.org/10.5128/ERYa12.14

Refbacks

  • There are currently no refbacks.


Copyright (c) 2016 Ingrid Sarapuu, Ene Alas

Creative Commons License
This work is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License.

ISSN 1736-2563 (print)
ISSN 2228-0677 (online)
DOI 10.5128/ERYa.1736-2563