12th September 2017

The culture of knowing: the dangers of assessing product over process

By Tom Garside

In high-schools the world over, there is a common tendency for teachers and their students to put a disproportionate focus on results; weekly assessments, end-of-term tests, mock exams, final exam results… there always seems to be another test or assessment looming on the horizon. A real and present danger of this situation, however, is that the focus on the assessed product can come at the expense of the important processes which underlie learning, especially in the field of foreign languages.

Unlike most curriculum subjects, languages are learnt through process of trial and error, reflection and application of both skills and systems, with very few quick fixes or shortcuts. Yes, memory and retrieval of learnt information is important to any learning, but language cannot be remembered for a test with any degree of depth. Not everyone who studies in this way, no matter how many hours a night are spent poring over word lists and grammar rules, will pass a language exam. However, anyone who has gained a depth, breadth and flexibility with language, through application, interaction and language practice can pass a test which is graded to their level of study, while getting more sleep and undergoing less stress. We are all already expert language users; it just depends how we choose to grow the language (whether first, second, third or ‘other’) that we are given. Language tests and examinations should be seen as part of a wider purpose for the language we teach, rather than the be all and end all of a course of study at any level.

Negative washback, the negative effects of a final assessment which are felt during the course of study itself, can have real and lasting effects on language performance in students. One symptom of the washback from this product-oriented view of language education is that disproportionate amounts of time are spent on fixed forms of language: reading and writing. Paper skills are seen as more predictable, structural and teachable, as the level of variation in texts on paper is low. Time can be spent looking at the language and content provided, and there is ample opportunity for dictionary work and other support while reading. Compare this to the ways in which most English-language media are processed outside the classroom, particularly in the form of listening (to movies, music and online media), and a definite shortfall is noticeable in the skills needed to survive in English outside of the teaching environment. The skills of listening flexibly, decoding extended messages, recognising shifts in topic and opinion and dealing with unknown vocabulary are necessary for both reading and listening, and would definitely be of benefit to students sitting a language exam that they have never seen or heard before, so why are these skills typically underdeveloped in test-centred cultures of education?

Defining the culture of knowing

In my experience, educators are often insecure about teaching using tasks to which there is no ‘right answer’. The skills above are interpretive, rather than empirical; they enable learners to understand attitudes, opinions and messages as a whole, rather than focusing on testable details about information, so cannot be as easily quantified as a set of true/ false or multiple choice questions, despite the fact that they represent a more authentic view of how we actually use language in real life. This tendency of educators to opt for easily quantifiable data from their learners, rather than seeing the wider communicative package, further reinforces a culture of education where ‘knowing the answer’ takes precedence over explaining how and why information can be understood, by rewarding proscriptively ‘correct’ answers to restricted test questions. This in turn precludes learners from reflecting on how they are learning in favour of what they know. This ‘culture of knowing’ produces a vicious cycle which can prevent free, generative use of language rather than upgrading and improving it.

In a real high-school setting, with the demands of large classes, restrictive syllabus requirements and resource limitations, it is understandable that quantifiable outcomes are important for teachers, schools and education authorities alike. Yes, it is essential to have an objective assessment system that encompasses the range of abilities and ages that are being tested. However, allowing the standardised nature of a testing system dictate the ways in which lessons are developed and delivered will never produce the most effective all-round results for the learners. An effective course of language study will prepare students not just for the test situation, but for any situation where language is used, whether it is found on the test or not.

So what are some holistic solutions for teachers, allowing greater all-round language development? How can we keep motivation levels high and still ensure high-quality results in end-of-course assessments?

Towards a holistic view of language preparation

Here are some guiding principles for language education that will to some degree get teachers and students thinking beyond ‘the right answer’ and encourage a more generative, wider-reaching view of their language. Focusing on process rather than product can give your students the tools they need to do the linguistic jobs required of them in and out of the classroom, in tests and more importantly, in their future lives.

1) Approach skills and systems differently

Remember that language skills (reading, writing, speaking and listening) involve very different ways of processing language as compared to language systems work (vocabulary, grammar and pronunciation). Use systems-focused lessons to teach patterns of language rather than individual examples, and have students practice applying the patterns you present to them. This can be achieved by asking students to work inductively with groups of examples, identifying patterns and testing them out in their own words, for example:

If production, information, performance and maintenance are nouns, which of the following words are nouns? Romance, extraction, distract, collection, compliance, inflate.
Now name 3 more nouns that follow each of the 2 spelling patterns.

Follow systems-based presentations with skills work, looking at how taught language is used in context. For listening and reading tasks, don’t just focus on comprehension, but investigate the grammar and vocabulary used by the speaker / writer. Referring to the systems you have taught reinforces the students’ understanding of language in different situations. Ask students to deliver spoken or written responses to texts using the language they read or hear, to integrate taught forms into their own messages.

2) Ask for justification for student answers

When a student offers the answer to a question from a language task, don’t just accept it instantly as right or wrong – ask them for justification:

Why did you get that answer?

Where did you find that answer in the text?

Or ask them for the method they used to find the answer

How did you find that answer?

What other words in the sentence / text / recording gave you that idea?

This will bring the process of decoding information from tasks into the students’ study routines. Challenging at first, the more they are asked to reason their responses, the easier it will become. In addition, if they know that you are going to ask them, they are more likely to prepare a comment on their processes independently.

3) Give learners the tools for the jobs you ask them to do

How sure are you that your students know how best to address the tasks and activities you give them? If you cannot answer that question with confidence, teach them directly how to read / listen / complete the sentences you give them. This is possible with careful instructions before handing out tasks in class, or by directly teaching the strategies you want them to use, for example a nice way of teaching skim-reading for a first look at a text (useful for contextualising exam reading):

i) Show the students a text of about half a page in length, and tell them that they will read it from beginning to end in… 10 seconds.

ii) Hand out the texts face down on students’ desks and tell them not to turn over until you say so

iii) Instruct students to start with their eye on the first word of the text, and when you say ‘go’, to move their eyes across the page WITHOUT reading every single word, until their eye lands on the final word of the page (give them this word if you like) at 10 seconds exactly.

iv) Instruct students to turn over their texts and say ‘GO!’. Count to ten, and tell them to turn over their texts. Be strict about this, and don’t worry if some students don’t look like they’ve finished.

v) Nominate students to give you any word or phrase that they can remember from the text, and display them to the board. They will almost definitely give you a list of nouns, verbs and adjectives which relate to the topic of the text. If no words are forthcoming, repeat from stage iv, for another 10 seconds, until you are happy all students have got some key words.

vi) As an extension, or for stronger skim readers, ask them to remember where in the text each word appeared (beginning, middle or end), and they will develop a better idea of each word’s position in the text if they need to find it quickly later on.

By performing this skim training regularly, skimming can be incorporated as a reading strategy into any text you work with in class, and makes for more informed exam reading in assessments.

Even for simple grammar or vocabulary gapfill tasks, a quick reference to key words in each sentence (time phrases, number of people, other contextual clues…) will bring the process of answering into the students’ toolkit for more informed, process-based work.

4) Use exam materials to teach exam processes

For exam preparation classes, a good way to get students familiar with the mechanics of the exam is to make them test-setters for the day. Give groups of students past exam papers (or sections, for longer exams), but with the questions removed. Ask students to read the texts, listen to the recording or look at the test prompts and write questions in the style of the exam they are preparing for. Pass the newly written exam sections around to the different groups, and see how easy, difficult or impossible the questions are. After the time is up, get feedback on the quality of the questions, and how well they fit the exam type they are preparing for. Choose the most authentic questions from each group and use it as a model exam.

This process of rethinking texts and language questions from an examiner’s perspective gets students into the processes behind the questions, and a surprising amount of analytical language work goes in to the work of preparing realistic questions for their section of the exam.

5) Make students play devil’s advocate

We are all so used to giving our own, real opinions on topics that we can lose sight of how well we express ourselves – only ever talking about your real opinion gets repetitive pretty quickly and risks losing the spark of an interesting idea or passionate argument. Engage students in the argumentative process by giving them unconventional opinions to argue. More thought will go into the process of constructing an argument for hats to be compulsory on weekends, or cellphones to be banned for the under-16s, than would be put into a standard, boring argument about school uniforms or less homework. An off-the-wall topic focuses students away from the subject, and onto the argumentative process which needs to be strong for any task answer to be successful.

Overall, just because your students are preparing for an important exam, it doesn’t mean that their study has to be dry, repetitive and time-consuming. A good language user has confidence in any language setting, so get your students thinking about the systems and processes that underlie their language abilities, and they will have the broader tools to more than cope with the relatively narrow set of skills tested by a typical language assessment.

How involved are you in your students' learning processes? How much do your students understand their learning strategies? Join Tom on the 2nd of October for an in depth CPD workshop at our training centre in Hong Kong: Product vs Process in ESOL exam classes.

Tom Garside, EfA’s former Director of Teacher Training, has 18 years of teaching and training experience in Europe, New Zealand and China. He holds a degree in Linguistics and French, Cambridge CELTA and DELTA qualifications, a Post-Graduate Diploma in TESOL and an MATESOL. He has trained teachers in Europe, as part of the European Union Comenius teacher development project, provided initial training for the Trinity CertTESOL and provides in-service training for native and non-native-speaker teachers in a wide range of teaching situations.  He is the author of the essential CertTESOL course supplement, Tesol: A Gateway Guide for Teachers of English.