This study explores the variation within native English-speaking teachers (NESTs) when evaluating English learners’ essays in terms of their teaching experiences. It was assumed that those who had been teaching at the same institute for more than five years (experienced NESTs) would understand better what their students wanted to convey, with deeper understandings of Korean culture and possibly the different writing styles based on Korean language, resulting in more lenient evaluating patterns, than those who just came to Korea to teach English (new NESTs). Interestingly, in contrast to the previous studies, the experienced NESTs evaluated more severely than the new NESTs. Additionally, based on the previous studies comparing NESTs with non-native English-speaking teachers (NNESTs), the experienced group demonstrated similarities to NNESTs after more than five years of assimilation in contrast to the new NESTs. This was a conclusion reached by the result that the new NESTs regarded content more substantially than the experienced NESTs. Lastly, unlike the previous studies, the NESTs’ perceived-difficulty played a more important role in grading than their perceived-importance regarding the three criteria (grammar, content, and vocabulary). The paper elaborates the details of the results and suggests a new approach to reduce the rater-biases even before rater-training is discussed.
Keywords: Experienced NESTs New NESTs NNESTs EFL writing performance Rater-biasesBrown, J. D. (1991). Do English and ESL faculties rate writing samples differently? TESOL Quarterly, 25, 587–603.
Brown, A. (1995). The effect of rater variables in the development of an occupation-specific language performance test. Language Testing, 12(1), 1–15.
Chalboub-Deville, M., & Wigglesworth, G. (2005). Rater judgment and English language speaking proficiency. World Englishes, 24(3), 383–391.
Cherubini, L. (2009). Reconciling the tensions of new teachers’ socialization into school culture: A review of the research. Issues in Educational Research, 19(2), 83–99.
Eckes, T. (2012). Operational rater types in writing assessment: Linking rater cognition to rater behavior. Language Assessment Quarterly, 9, 270–292.
Hyland, K., & Anan, E. (2006). Teachers’ perceptions of error: The effects of first language and experience. System, 34(4), 509–519.
Knoch, U., Read, J., & von Randow, J. (2007). Re-training writing raters online: How does it compare with face-to-face training? Assessing Writing, 12, 26–43.
Kobayashi, T. (1992). Native and nonnative reactions to ESL compositions. TESOL Quarterly, 26, 81–112.
Kondo-Brown, K. (2002). A FACETS analysis of rater bias in measuring Japanese second language writing performance. Language Testing, 19(1), 3–31.
Korean Immigration Service (2013). Statistics annual report on foreigners. Resource document. http://www.index.go.kr/egams/stts/jsp/potal/stts/PO_STTS_IdxMain.jsp?idx_cd=2756. Accessed December 13, 2013.
Lee, H. (2009). Native and nonnative rater behavior in grading Korean students’ English essays. Asia Pacific Education Review, 10(3), 387–397.
Lev-Ari, S., & Keysar, B. (2012). Less-detailed representation of non-native language: Why non-native speakers’ stories seem more vague. Discourse Process, 49(7), 523–538.
Lynch, B. K., & McNamara, T. F. (1998). Using G-theory and many-facet Rasch measurement in the development of performance assessments of the ESL speaking skills of immigrants. Language Testing, 15(2), 158–180.
McNamara, T. F. (1996). Measuring second language performance. London: Longman.
Min, C., Kim, J., Rha, K., & Kang, H. (2014). Setting up assessment indicators for analyzing the effectiveness of the native English teacher policy of Korea English teacher training institutes. The Journal of Korea Elementary Education, 25(3), 93–110.
Myford, C. M., & Wolfe, E. W. (2003). Detecting and measuring rater effects using many-facet Rasch measurement: Part I. Journal of Applied Measurement, 4(4), 386–422.
Na, H., Lee, M., & Lee, E. (2010). A study on the way of teaching L2 culture and their social, cultural adaptability of NESTs in Korea: Based on how long they stayed in Korea. Statistical Consultation, 23(1), 13–23.
Schaefer, E. (2008). Rater bias patterns in EFL writing assessment. Language Testing, 25(4), 465–493.
Scollon, R., & Scollon, S. (1995). Intercultural communication: A discourse approach. Oxford: Blackwell.
Shi, L. (2001). Native- and nonnative-speaking EFL teachers’ evaluation of Chinese students’ English writing. Language Testing, 18(3), 303–325.
Shin, Y. (2010). A FACETS analysis of rater characteristics and rater bias in measuring L2 writing performance. English Language & Literature Teaching, 16(1), 123–142.
Song, B., & Caruso, I. (1996). Do English and ESL faculty differ in evaluating the essays of native English-speaking and ESL students? Journal of Second Language Writing, 5(2), 163–182.
Weigle, S. C. (1998). Using FACETS to model rater training effects. Language Testing, 15(2), 263–287.
Wigglesworth, G. (1993). Exploring bias analysis as a tool for improving rater consistency in assessing oral interaction. Language Testing, 10(3), 305–335.
Winke, P., Gass, S., & Myford, C. M. (2012). Raters’ L2 background as a potential source of bias in rating oral performance. Language Testing, 30, 231–252.