This is the third and final part of my short series reflecting on my mid-term exams. Timely too, seeing as I’m writing the final exams this week. If you’re interested, you can read about my reflections on my written exams here, and the feedback I collected on my exams here. In general I’m much happier with my skills as a speaking examiner, but in the student feedback that I collected there was still room for improvement. This post takes a bit of an experiential direction, beginning by looking at what I do, then what I think/thought about it, and finishing my trying to make some changes for this round of exams.
What did I do last time?
I’m going to deal with this in two parts, my method and my scoring system. In fact, I’m mostly concentrating on the scoring system, as it seems the most appropriate tool for helping generate the performances that I would like from my students. Nevertheless, I’ll start with my method.
My mid-term speaking exams were 20 minute conversations between groups of four people. These groups were randomly selected a few days before the exam. Students could choose any or all of the four topics that we had studied in the half semester, and could prepare what they wanted to say, although they were discouraged from memorizing long passages of text.
You can read my full scoring system on the second page of the document below. The language used is necessarily simplistic in order that students can understand it, but this is perhaps a problem when it comes to judging fine-grained differences in performance.
My scoring system scores students on five traits, each scored from one to five (yes, students get a full 20% just for showing up). A score of three represents a pass in each trait. Half marks are possible. The five traits are as follows:
- Difficulty & Interest
Difficulty and interest requires the student to use more complex language and to talk about interesting things within the topic. Participation asks the student to play a full role in the conversation. Fluency requires them to speak at a comfortable speed, with no big hesitations. Understanding is that of the teacher, but more importantly their peers too. Effort is my attempt to motivate students of both higher and lower ability coming in to the course, by challenging them to exceed my expectations.
What do/did I think about it?
The slash in the title above refers to the fact that I scribbled down a few reflections during my speaking exams last time. Other insights are coming from thinking about the exams as I write this. My method, I think, is fairly suitable. It gives enough freedom to students to express themselves and is in keeping with the fairly fluency based nature of the class. Also, a four way conversation is a more challenging proposition than one between two people, and students have to work a little harder to stay involved and follow what is going on. There is also the efficiency saving of only explaining to five groups per class, rather than ten if I did it in pairs. As time is limited, this is a very practical reason to test in larger groups.
All this means that there is fairly little that I want to change. The only thing that I wonder about is changing the number of topics, and their specificity. Four topics, between four people in a twenty minute exam leaves about one minute 15 seconds per person, per topic. In general, testing in class aimed toward being able to speak for a two and a half minutes per person, per topic. Thus one change that I would like to make is to limit groups to two topics, and also to make them more specific. Last time I had very loose topic prompts (eg. Favourite foods, shopping style and stories). This time I’d like to tighten them up a little bit, for example: “My ambition and what I have done to achieve it” for the personal background module. I’d also like to increase the spontaneity a bit by selecting a topic randomly. This might also require a change to the scoring system to reflect this.
Thinking about scoring systems, it’s clear to me that mine needs a bit of work. The main thing is that it perhaps doesn’t reflect clearly exactly what kind of performance I was looking for. This is due to the lack of a clearly defined construct, a project which I never quite got around to finishing properly. Nevertheless, I have tried to briefly outline a construct below. These are the kind of things that students should perhaps be able to do in their exams. This is based largely on grading notes from my last exam.
Students should be able to talk in a reasonable amount of detail about 2 topics as part of a twenty minute conversation, making the conversation interesting through a variety of opinions (backed up if possible), personal stories and unusual information/facts. Students should be able to organize the conversation into short turns rather than long monologues, and be able to both claim and relinquish the floor when appropriate. The conversation should be relatively spontaneous. All speech should be understandable (to both peers and teacher) and fluent (defined as a steady rate of speech with minimal hesitation and restarts). Accuracy in grammar, word choice, syntax and pronunciation is not important unless it hinders understanding, but errors that were explicitly discussed in class should be avoided. Some attempt to (correctly) use language from class is preferable, but long memorized passages are not. No Korean language, aside from names, is permitted.
Looking at the scoring system linked above, I can see several places in which it does not match the construct and needs to be changed. The first is the slightly odd category of difficulty and interest. This is a bit counter-intuitive because, as anyone who’s ever attended one of my conference presentations will tell you, it’s perfectly possible to say something very boring using difficult language, and of course the other way around. Clearly this needs to be split out. Looking back at my notes, my way of judging difficulty seems to be to note instances of target language use. Therefore, it makes sense to split this out into its own category (more on this later). This leaves us with the rather subjective category of “interest”. Again, I went back to my notes on this one, and found that the performances I scored highly tended to contain interesting stories, unusual information and strong opinions. This goes some way to making things less subjective, but much more importantly, gives the students a guide to how they can score top marks.
Another category that requires a little tweaking is participation. I’d like to include turn length, questions, turn management and amount said into a slightly updated rubric. The idea behind this is to make the conversations a bit more spontaneous and conversation-like, and avoid a problem I encountered occasionally last time of students essentially going round the table delivering monologues.
The fluency and understanding categories are largely fine as they are, though I want to add restarts into the fluency section. That just leaves me with the final section, effort. I like this section, as it gives the lower level speakers in my class something to aim for. I don’t like grading on ability only, as despite level testing it can vary quite widely in my classes. Again though, I’d like to be able to give students a little more guidance on how they might do it. This is a place where attempts to use target language can be recognized, along with not memorizing long pieces of language and speaking spontaneously. I could also try to recognize humour here. Finally, some recognition of shy students participating confidently would be good, as this is something that I have tried to encourage throughout the semester.
This just leaves a further section for penalty points. Given this is an English exam, speaking Korean except for names is not allowed and must be made clear. Also, long diversions from the topic should also be penalized as I am trying to get students to show what they learned in class. Finally, I think I want to punish errors that we have talked about in class, as these too are evidence of (not) learning.
What am I going to do about it?
When I make my exam guide on Wednesday, I’m going to do the following things:
- Allow students to choose one topic in advance for the exam, and give them one of the other three in the exam.
- Make the topics much more specific and relevant to class content.
- Make the first category interest, defined as opinions, stories and interesting facts.
- Add questions, turn management and turn length into the participation section.
- Add restarts into the fluency section.
- Write some notes in the “Effort” section, explaining to students how they can get better scores through spontaneous speech, humour and confidence.
- Explain clearly the penalty points system.
These exam reflections have been pretty long, so thanks for reading this far. I’m interested in any ways which you think I could further improve this system, and also in how you do your own speaking exams. If you want to read more you might even want to check out @alexswalsh‘s post on his speaking exams.