Textbooks are an essential component in the classroom, making it extremely crucial to regularly evaluate textbooks used in schools so that their pedagogical contributions towards the teaching and learning processes can be guaranteed. The study aimed to identify and analyze the cognitive levels of questions available in Secondary Three All About English textbooks, which have been used in Singapore since 2013. The objective of carrying out the analysis was to determine the overall cognitive rigour of questions, using a matrix superimposing two taxonomies: Revised Bloom’s Taxonomy and Webb’s Domain-of-Knowledge (DOK) model. The sample of this study consisted of questions from the Express and Normal Technical (NT) English language textbooks published by Hodder Education, where 346 questions were analyzed in total. The cognitive rigour matrix was used in the classification of the questions. The results revealed that in the Express level textbook, most questions were within the cell [2,3] (35.9%), consisting of Understand and “Strategic Thinking” questions. In contrary, for the Normal Technical level textbook, most questions were within the cell [5,4] (17.6%), consisting of Evaluate and Extended Thinking” questions. Overall, there was a good mix of Lower-order thinking skill (LOTS) and Higher-order thinking skill (HOTS) questions. However, not all cognitive levels were clearly varied, with an overemphasis on Understand and Evaluate questions, neglecting Remember and Apply questions. In light of the results, it is recommended that coursebook writers should incorporate questions of varied cognitive demands. We recommend that teachers should regularly evaluate the effectiveness of school textbooks so that there is closer alignment between the curriculum and the textbooks that teachers use in the classroom.
Keywords: Cognitive rigour, coursebook analysis, coursebook evaluation, types of questionsAdli, N., & Mahmoudi, A. (2017). Reading comprehension questions in EFL textbooks and learners’ levels. Theory and Practice in Language Studies, 7(7), 590-595.
Anderson, L.W., & Krathwohl, D.R. (2001). A taxonomy for learning, teaching, and assessing: A revision of Bloom’s taxonomy of educational objectives. New York: Longman.
Armbruster, B., & Ostertag, J. (1993). Questions in elementary science and social studies textbooks. In B.K. Britton, A. Woodward, & M.R. Binkley (Eds.), Learning from textbooks: Theory and practice (pp. 69-94). Hillsdale, NJ, US: Lawrence Erlbaum Associates, Inc.
Assaly, I.R., & Smadi, O.M. (2015). Using Bloom’s Taxonomy to evaluate the cognitive levels of master class textbook’s questions. English Language Teaching, 8(5), 100- 110.
Berelson, B. (1952). Content analysis in communication research. New York, NY, US: Free Press.
Bloom, B.S. (Ed.), Englehart, M.D., Furst, E.J., Hill, W.H., & Krathwohl, D.R. (1956). Taxonomy of educational objectives, Handbook I: The cognitive domain. New York: David McKay Co Inc.
Blosser, P.E. (1991). How to ask the right questions. Washington, DC: National Science Teachers Association.
Chin, C. (2004). Questioning students in ways that encourage thinking. Teaching Science: The Journal of the Australian Science Teachers Association, 50(4), 16-21.
Chinoda, A.M. (1982). An Analysis of Questions in Selected High School Social Studies Textbooks Used in Zimbabwe (Doctoral dissertation). George Peabody College for Teachers of Vanderbilt University. Coleman, V.M. (2011). National Music Education Standards and Adherence to Bloom’s Revised Taxonomy (Doctoral dissertation). Walden University, US.
Cotton, K. (1988). Classroom questioning. School Improvement Research Series, 5, 1-22.
Curriculum Planning and Development Division. (2008). English language syllabus 2010. Ministry of Education, Singapore.
Feng, Z. (2013). Using teacher questions to enhance EFL students’ critical thinking ability. Journal of Curriculum and Teaching, 2(2), 147-153.
Forehand, M. (2010). Bloom’s taxonomy. Emerging Perspectives on Learning, Teaching, and Technology, 41(4), 47-56.
Freahat, N.M., & Smadi, O.M. (2014). Lower-order and higher-order reading questions in secondary and university level EFL textbooks in Jordan. Theory and Practice in Language Studies, 4(9), 1804-1813. Hess, K.K. (2004). Applying Webb’s depth-of-knowledge (DOK) levels in reading. [online] Retrieved from: http://www.nciea.org/publications/DOKreading_KH08.pdf
Hess, K.K. (2005). Applying Webb’s Depth-of-Knowledge (DOK) levels in social studies. Retrieved from http://www.nciea.org/publications/DOKsocialstudies_KH08.pdf
Hess, K.K. (2006). Applying Webb’s Depth-of-Knowledge (DOK) levels in science.Retrieved from http://www.nciea.org/publications/DOKscience_KH08.pdf
Hess, K.K., Carlock, D., Jones, B., & Walkup, J.R. (2009). What exactly do “fewer, clearer, and higher standards” really look like in the classroom? Using a cognitive rigour matrix to analyze curriculum, plan lessons, and implement assessments. Retrieved from https://ccsso.confex.com/ccsso/2010/webprogram/Handout/Session1381/cognitive%20 rigour%20paper_9%2030%2009%20_2_.pdf
Igbaria, A.K. (2013). A content analysis of the WH-questions in the EFL textbook of “Horizons”. International Education Studies, 6(7), 200-224.
Joffe, H., & Yardley, L. (2004). Content and thematic analysis. In D.F. Marks, & L. Yardley (Eds.), Research methods for clinical and health psychology (pp. 56-68). London, UK: Sage.
Krippendorff, K. (2018). Content analysis: An introduction to its methodology. USA: Sage Publications, Inc. Larsen-Freeman, D. (2000). Techniques and principles in language teaching. New York: Oxford University Press.
Leen, C.C., Hong, H., Kwan, F.F.H., & Ying, T.W. (2014). Creative and critical thinking in Singapore schools. Office of Education Research, National Institute of Education, Nanyang Technological University. Singapore: National Institute of Education, Centre for Educational Research.
Lim, S.C. (2002). Developments in the English language syllabuses in Singapore. Asia Pacific Journal of Education, 22(2), 81-95.
Lin, B. (2003). English in Singapore: An insider’s perspective of syllabus renewal through a genre-based approach. RELC Journal, 34(2), 223-246.
Mee, C.Y. (2002). English language teaching in Singapore. Asia Pacific Journal of Education, 22(2), 65-80. Moo, S.N. (1997). Teachers’ perceptions of normal technical students’ motivation and classroom behaviour. In A.S.C., Chang, S.C. Goh, S.N. Moo, & A.Y. Chen (Eds.), Report on motivation and classroom behaviour of normal technical students. Singapore: National Institute of Education, Centre for Educational Research.
Parker, M., & Hurry, J. (2007). Teachers’ use of questioning and modelling comprehension skills in primary classrooms. Educational Review, 59(3), 299-314.
Riazi, M., & Mosallanejad, N. (2010). Evaluation of learning objectives in Iranian highschool and pre-university English textbooks using Bloom’s taxonomy. TESL-EJ: The Electronic Journal for English as a Second Language, 13(4), 1-16.
Stanny, C. (2016). Reevaluating Bloom’s taxonomy: What measurable verbs can and cannot say about student learning. Education Sciences, 6(4), 37.
Sunggingwati, D. (2003). Reading questions of junior high school English textbooks. Bahasa Dan Seni, 31(1), 84-105.
Tarman, B., & Kuran, B. (2015). Examination of the cognitive level of questions in social studies textbooks and the views of teachers based on Bloom’s taxonomy. Educational Sciences: Theory and Practice, 15(1), 213-222.
Teo, C.L. (2002). A Wider Choice or a Wiser Choice: A Comparative Analysis of the New English Textbooks for Primary 1 and 2 Pupils in Singapore (Master’s thesis). Singapore: SEAMEO RELC. Wagner, T. (2008). Even our “best” schools are failing to prepare students for 21st-century careers and citizenship. Educational Leadership, 66(2), 20-25.
Webb, N.L. (1997). Criteria for alignment of expectations and assessments in Mathematics and Science education. Research Monograph 6. National Institute for Science Education, Madison, WI.
Webb, N.L. (2002). Depth of Knowledge Levels for Four Content Areas. Unpublished manuscript. Wisconsin Center for Education Research, University of WisconsinMadison, Madison, WI. Retrieved from http://ossucurr.pbworks.com/w/file/ fetch/49691156/ Norm%20web% 20dok%20by%20 subject%20area.pdf
Wixson, K.K. (1983). Postreading question-answer interactions and children’s learning from text. Journal of Educational Psychology, 75(3), 413-423.
Zohar, A., & Dori, Y.J. (2003). Higher-order thinking skills and low-achieving students: Are they mutually exclusive?. The Journal of the Learning Sciences, 12(2), 145-181.