Tag Archives: Constructs

Class constructs: creating my own (part 1)

I blogged previously about the possibility of creating a construct for a short term class in order to keep teaching and testing in line with one another. There is also the advantage that your construct can be shared with students as a form of class goal, and activities can be justified to students in terms of it (especially if they are of the less fun variety). As a brief recap, a construct is a short statement of what you will teach and test, how you will go about it and the expected results and standards. In this post, I will document the first part of the process of creating my own construct.

At the end of the last post I looked at 4 areas that need to be considered in creating a construct. These were:

  • Assessment (& teaching) context (Students, institution, geographical location, purpose and score use and tester).
  • Assessment (& teaching) procedures (What students are expected to do in class and exams)
  • Construct definition (What do you mean by the terms used to describe your class – what is “English”, “Conversation” or “Speaking” for this class?)
  • Models and Frameworks (How can you justify the above with reference to clever people or yourself?)

In this post I will try to outline my thoughts on the first two areas.

Assessment and Teaching Context

A good place to start here is asking who my students are. In my case this also covers a lot of the geographical and institutional factors. Beautiful and unique snowflakes that they of course are, my lot do form quite a usefully homogenous group in two ways. Firstly, they are all Korean and are products of the educational culture here, and secondly they are all students at a polytechnic university. This allows me to make some guiding assumptions:

  • Their English education will have been largely reading and listening focused, and grammar and vocabulary will often have been decontextualized and almost always depersonalized. If they have encountered speaking they have not been especially successful in learning it. I’d venture to say that they have generally learned English as an academic subject rather than a language.
  • They are not taking English as a major, and so they are unlikely to be learning it out of a love for the subject (though this is possible). They are more likely to be learning it out of long-term pragmatic value, but in the short-term their grade is the most important factor. Their future careers are more likely to require practical, rather than perfect, English.

In terms of assessment purpose and score use, one or two things are worth considering. Firstly, I’m aiming to assess achievement not proficiency. In other words, someone who makes a great effort and improves from 0 to intermediate should theoretically score higher than an initially high-intermediate speaker who improves little. Secondly, assessment is not only in terms of exams, but performed continuously over the term through participation, quizzes, projects and 1:1 conversation. The scores have a very narrow use, which is assigning grades for the term. However these grades may dictate scholarships, so it is important that they accurately reflect effort and achievement.

One final consideration is who the assessor is. For the most part it is me, but I do feel that student views should play a part in assessment as well, especially in something as subjective as participation. I think allowing students to play a part in scoring themselves and others also helps to motivate them, as well as keeping complaints down at final grading time.

Assessment and Teaching Procedures

In assessing and teaching the course I want to take the notion of “conversation” as literally as possible.  By this I mean that the aim of the course will be to develop the ability to hold medium length conversations in English on a few topics, and we will learn to do this by having short conversations throughout the course, which will serve as a framework for practicing useful lexis, conversational skills and strategies and a little bit of grammar.

Given this aim it makes sense for the mid-term and final speaking exams to take the form of conversations. This will form the principal drive for the course, and students will be expected to apply what they have learned during class in the exams. The length of the exam is important, as it should be sufficient to pose a real challenge to students (or at least appear to).

Also significant is the number of participants. This is a really interesting question that I am still working on puzzling out. My preference in the past has been for 4 person speaking assessments. I believe that they pose a greater degree of challenge in terms of organizing turns and dealing with multiple inputs. They’re also practically much easier to arrange and going back to the length, I think that a 25 minute 4 person exam sounds more difficult than a 12.5 minute group conversation. The potential downside to this is that a lot of my classwork is done in pairs, though there is nothing to say that I couldn’t up group size over the course of a semester.

Another thing to figure out is the role of written exams. It is institutionally mandated that 50% of my mid-term and final exams is a written paper. What, then, is the role of writing in conversation? Listening might provide some of those marks, perhaps choosing the right answer to a question. The discrimination of similar sounds could also be included.  I also think that common errors that we point out in class should have a role. Finally, vocabulary and lexis in the form of gap fills will be important, as well as subtler shades of meaning that we talked about in class that simply won’t come up in a speaking exam. As far as possible, I would like to avoid grammar transformation exercises and reading passages. 

All of this and I’m only really through talking about final assessments. Ongoing assessments (quizzes and participation scores) should also be generally conversation based, and reflect the effort made to actually have conversations, on the basis that conversational skills cover a wide range of areas, and are probably subject to individual variation. It’s developing an individual ability to have conversations that I am most interested in during this course. Partly this can be taught directly in terms of strategies and language  but partly this is something that you figure out for yourself by getting involved. The course needs to both offer opportunities to do this and reward them when they are taken.

To bring this post to a conclusion, as I am already over my self-imposed 1,000 word guideline, my teaching and assessment aims should be to improve speaking as this is the area in which my students need most improvement. A conversation based approach gives an opportunity for personalizing the language as well as providing a reasonably well defined structure for assessment (see the next post). Conversation must form the basis for ongoing and final assessment of achievement on the course, with an emphasis on fluency and communication skills rather than accuracy (or complexity especially). The ability to deal with small group work is thought to be important, as is the ability to function in English speaking environments for a slightly longer duration.

In the next post I’m going to tackle my description of conversation. I hope you’ll be there to read it. In the meantime if you have comments, questions or suggestions, please leave them below the line.

Cheers,

Alex

Class constructs: an introduction.

16 weeks, roughly 5 hours of class time in each. Throw in a couple of presentations and a magazine making project, as well as exams, entrance tests and university festivals, and it doesn’t leave a lot of time for learning something as large as a language. Nevertheless, we grab our textbooks and have a go – and while we do so we also try and order ourselves for the dishing out of grades or levels. Basically the two problems I imagine that many teachers with some autonomy grapple with: what to teach, and how to assess it. In this post I’m going to set the background for creating class constructs that go some way to tackling this problem.

Construct is a term drawn from assessment literature, and is a more or less a statement of what the test author believes they are testing, how they should test it, and what the results might look like. As an example, a construct for the TOEFL exam would be a definition of the English ability required to take a higher education course, perhaps in terms of vocabulary size, grammatical knowledge, skills (summarizing, note taking), functions, knowledge of genres and many other things. It would also include the kind of tasks that the authors felt would test these, and what acceptable and unacceptable performances looked like. All of this is realized in the test that is actually taken, and the rating scales, scoring and the final grade. Therefore, if you score a full 120 on the TOEFL IBT, you can congratulate yourself on being the embodiment of what ETS (the makers of TOEFL) think academic English is.

“Teaching to the test” gets a bit of a bad rep, especially in Korea where anything that isn’t an academic reading passage is ruthlessly cast aside. It feels a bit dirty to be honest, like you’re being cowed by the man – encouraging your students to chase letter and number grades over actually learning anything useful, or teaching test-taking strategies rather than language. If the test is crap (TOEIC, the Korean university entrance exam) then this is abundantly true, but if the test is good, then surely this can be a good thing (these two situations tend to be called negative and positive washback respectively). For a short course test such as mine, which is aimed at measuring learning, control of the design should play a large part in deciding what should be learned (though we know that this is not an exact science), and so a construct not only defines the construction of a test, but in this case the construction of the whole course.

But why exactly is this useful? Firstly, going back to the opening sentence, time is short, English is not only big but constantly shifting. With hundreds of thousands of words, not to mention fixed phrases, as well as countless combinations of functions, domains of use, registers and skills, pinning English down to something teachable is constant source of frustration and argument in journal articles, blogs and at conferences. General English courses (in the form of books) try to tread the most middling, inoffensive and general line, in order not to upset anyone into not buying them. However, this means they also tend to miss out anything culturally specific, potentially insulting or simply left-field. Having a construct allows you to cut out the irrelevant stuff and focus on what your students (and you!) really want and need. In my case, students can translate about 3000 single words in English, and have a pretty decent reading level. Their grammar is OK if they can write it out first, but spoken interaction is often conducted in single words at the beginning of the course. They also have very little knowledge outside the academic register. I’ve talked a lot about this already (and will again), but safe to say that concentrating on speaking skills almost exclusively is a good  bet.

The second advantage that I can see for developing a construct for the class is that if you want the exam to dictate teaching, you theoretically should write the exam first. The problem of course, is that it’s difficult to write an exam based on content that you haven’t taught yet, especially if your course is based a lot on lexis that arises from what students say, rather than being planned in advance. A construct for the class provides a nice straight ledge for aligning one’s ducks on, and if teaching and testing are conducted with reference to it then the two should reflect and reinforce each other. This hopefully will help me to tackle two problems that I’ve encountered in previous semesters – difficulty in writing exams that accurately reflect what we have done in class, and also the fact that in feedback I tend to score low on questions about students understanding my goals. As an extra idea, there would of course be nothing to stop you designing a construct in collaboration with your students.

So what goes into designing a construct? I’m going to finish this post by examining in a little more detail the kind of thinking that one might need to do, and presenting the questions that might need to be answered. In doing this I’m drawing heavily on the work of Sari Luoma (2004) on speaking assessment, though these considerations could easily be adapted to other assessment concepts.

Assessment Context

A construct links the theoretical with the more concrete (though of course this is still within a context of a test, which itself is often a prediction of how a testee would fare in the real world). Part of this is defining the context of the test – institution, purpose, takers and backgrounds, the tester and the plans for score use. While the theoretical definition for speaking might be the same for young children, teenagers and young adults might be similar, the ways of eliciting speech (task type, topic) will be very different, so context here is extremely important.

Assessment Procedures

A construct should have some indication of the length and frequency of the assessment, as well as the tasks required to elicit it and the methods used to score it. This helps keep things practical (no sense in having hour long one on one speaking tests when you teach 200 students) as well as, in the case of my class constructs, meaning that class activities can mirror testing activities.

Construct Definition

What are you actually going to try to teach and test here? The more specific you can be here the better, so you might want to think about sub-skills, grammatical structures and vocabulary ranges, rather than something general like speaking. You should also consider what a good, average and bad performance might look like in these terms. All of this will help greatly in designing rating scales and rating performances.

Models and Frameworks

What’s even better is if you can relate the thinking above to reading that you’ve done in the area. An example of this might be Hymes’s SPEAKING framework. This gives you a base to work from in terms of teaching and learning.

A Construct Definition

Finally, you should attempt to summarize all of the thinking above into a neat little paragraph like the one below:

The aim of this test (class) is to assess (teach/improve) the examinees’ ability  to express their ideas in English, take their interlocutor’s contributions into account and make use of them in the discussion, and collaborate in the creation of interaction. Social appropriateness is not assessed (taught) explicitly. (Luoma 2004: 121).

So that is roughly what a construct design process looks like. In the next post or two I’m going to have a go at it myself. In the meantime I’d be interested to know your views on whether this is a sensible approach. Are there any downsides to working this way? Am I consigning my students to a life of exam hell? Any argument very much welcomed below the line.

Cheers,

Alex

Reference

Luoma, S. (2004). Assessing speaking. Ernst Klett Sprachen.