16 weeks, roughly 5 hours of class time in each. Throw in a couple of presentations and a magazine making project, as well as exams, entrance tests and university festivals, and it doesn’t leave a lot of time for learning something as large as a language. Nevertheless, we grab our textbooks and have a go – and while we do so we also try and order ourselves for the dishing out of grades or levels. Basically the two problems I imagine that many teachers with some autonomy grapple with: what to teach, and how to assess it. In this post I’m going to set the background for creating class constructs that go some way to tackling this problem.
Construct is a term drawn from assessment literature, and is a more or less a statement of what the test author believes they are testing, how they should test it, and what the results might look like. As an example, a construct for the TOEFL exam would be a definition of the English ability required to take a higher education course, perhaps in terms of vocabulary size, grammatical knowledge, skills (summarizing, note taking), functions, knowledge of genres and many other things. It would also include the kind of tasks that the authors felt would test these, and what acceptable and unacceptable performances looked like. All of this is realized in the test that is actually taken, and the rating scales, scoring and the final grade. Therefore, if you score a full 120 on the TOEFL IBT, you can congratulate yourself on being the embodiment of what ETS (the makers of TOEFL) think academic English is.
“Teaching to the test” gets a bit of a bad rep, especially in Korea where anything that isn’t an academic reading passage is ruthlessly cast aside. It feels a bit dirty to be honest, like you’re being cowed by the man – encouraging your students to chase letter and number grades over actually learning anything useful, or teaching test-taking strategies rather than language. If the test is crap (TOEIC, the Korean university entrance exam) then this is abundantly true, but if the test is good, then surely this can be a good thing (these two situations tend to be called negative and positive washback respectively). For a short course test such as mine, which is aimed at measuring learning, control of the design should play a large part in deciding what should be learned (though we know that this is not an exact science), and so a construct not only defines the construction of a test, but in this case the construction of the whole course.
But why exactly is this useful? Firstly, going back to the opening sentence, time is short, English is not only big but constantly shifting. With hundreds of thousands of words, not to mention fixed phrases, as well as countless combinations of functions, domains of use, registers and skills, pinning English down to something teachable is constant source of frustration and argument in journal articles, blogs and at conferences. General English courses (in the form of books) try to tread the most middling, inoffensive and general line, in order not to upset anyone into not buying them. However, this means they also tend to miss out anything culturally specific, potentially insulting or simply left-field. Having a construct allows you to cut out the irrelevant stuff and focus on what your students (and you!) really want and need. In my case, students can translate about 3000 single words in English, and have a pretty decent reading level. Their grammar is OK if they can write it out first, but spoken interaction is often conducted in single words at the beginning of the course. They also have very little knowledge outside the academic register. I’ve talked a lot about this already (and will again), but safe to say that concentrating on speaking skills almost exclusively is a good bet.
The second advantage that I can see for developing a construct for the class is that if you want the exam to dictate teaching, you theoretically should write the exam first. The problem of course, is that it’s difficult to write an exam based on content that you haven’t taught yet, especially if your course is based a lot on lexis that arises from what students say, rather than being planned in advance. A construct for the class provides a nice straight ledge for aligning one’s ducks on, and if teaching and testing are conducted with reference to it then the two should reflect and reinforce each other. This hopefully will help me to tackle two problems that I’ve encountered in previous semesters – difficulty in writing exams that accurately reflect what we have done in class, and also the fact that in feedback I tend to score low on questions about students understanding my goals. As an extra idea, there would of course be nothing to stop you designing a construct in collaboration with your students.
So what goes into designing a construct? I’m going to finish this post by examining in a little more detail the kind of thinking that one might need to do, and presenting the questions that might need to be answered. In doing this I’m drawing heavily on the work of Sari Luoma (2004) on speaking assessment, though these considerations could easily be adapted to other assessment concepts.
A construct links the theoretical with the more concrete (though of course this is still within a context of a test, which itself is often a prediction of how a testee would fare in the real world). Part of this is defining the context of the test – institution, purpose, takers and backgrounds, the tester and the plans for score use. While the theoretical definition for speaking might be the same for young children, teenagers and young adults might be similar, the ways of eliciting speech (task type, topic) will be very different, so context here is extremely important.
A construct should have some indication of the length and frequency of the assessment, as well as the tasks required to elicit it and the methods used to score it. This helps keep things practical (no sense in having hour long one on one speaking tests when you teach 200 students) as well as, in the case of my class constructs, meaning that class activities can mirror testing activities.
What are you actually going to try to teach and test here? The more specific you can be here the better, so you might want to think about sub-skills, grammatical structures and vocabulary ranges, rather than something general like speaking. You should also consider what a good, average and bad performance might look like in these terms. All of this will help greatly in designing rating scales and rating performances.
Models and Frameworks
What’s even better is if you can relate the thinking above to reading that you’ve done in the area. An example of this might be Hymes’s SPEAKING framework. This gives you a base to work from in terms of teaching and learning.
A Construct Definition
Finally, you should attempt to summarize all of the thinking above into a neat little paragraph like the one below:
The aim of this test (class) is to assess (teach/improve) the examinees’ ability to express their ideas in English, take their interlocutor’s contributions into account and make use of them in the discussion, and collaborate in the creation of interaction. Social appropriateness is not assessed (taught) explicitly. (Luoma 2004: 121).
So that is roughly what a construct design process looks like. In the next post or two I’m going to have a go at it myself. In the meantime I’d be interested to know your views on whether this is a sensible approach. Are there any downsides to working this way? Am I consigning my students to a life of exam hell? Any argument very much welcomed below the line.
Luoma, S. (2004). Assessing speaking. Ernst Klett Sprachen.