Reading #12

Ockey, G. J. (2009).
Developments and Challenges in the Use of Computer-Based Testing for Assessing Second Language Ability
Modern Language Journal, 93(Focus Issue), 836-847.

Ockey argues that Computer based testing has failed to realize its anticipated potential. Describe and discuss on his reasons for his view, and tell why you either agree or disagree with him.

Cummins, P. W. & Davesne, C. L. (2009).
Using Electronic Portfolios for Second Language Assessment
Modern Language Journal, 93(Focus Issue), 848-867.

Cummins & Davesne offer an alternative to CBT with electronic portfolios. Comment on some of the ideas from this article that you'd be interested in trying out in your own classroom.

>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>

Re. Ockey, Developments and Challenges in the Use of Computer-Based Testing

In this article, Ockey discusses the range of challenges and limitations associated with CBT, and gives two primary reasons for his conclusion that CBT testing has failed to live up to its anticipated potential. First, we have still not found a reliable way to develop computer-based tests that employ more authentic task types. Second, computers are still unable to reliably measure a test-takers actual feeling and meaning in constructed response items, especially regarding speaking and writing.

I agree with Ockey’s reservations, because I don’t have enough faith in computer programs to believe they can adequately assess constructed response types such as essays or speech. Computer programs can be developed to measure a wide range of features of speech and writing, but one thing that computers can certainly never measure is how well people are communicating and expressing themselves.

Although many types of CBT tests may often be nearly 100% reliable, their overal validity is highly debateable. Tests such as the PhonePass/Versant tests and Automated Essay-Scoring (AES) systems tests, in which computers measure speaking or writing ability, have statistically been shown to give scores that correlate to a high degree with scores given by real human raters.

However, Ockey has two main criticisms: first, that the inauthentic nature of the assessment items (such as asking test-takers to rearrange words to form sentences) may potentially have a negative washback effect, and second, that it is impossible for a computer analysis to determine how much sense the speaker or the writer is actually making.

>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>

Re. Cummins & Davesne, Using Electronic Portfolios

A general observation regarding the use of portfolios (digital or otherwise)… as a means of assessment, I think there are many reasons why portfolios are probably the most holistic way of assessing a student’s progress. The best thing about this kind of assessment is that it puts more responsibility on the student-- not just to get a good score on 3 or 4 tests, but to engage in a certain amount of planning and organization, and to reflect on how each element contributes to the overall purpose of a particular course. It can help give the learner the motivation to engage in the kind of incremental preparation that reinforces a more meaningful learning experience. It is also a means of assessment that allows for a greater degree of individual choice and expression.

Of all of the methods of EP assessment examined in this article, I think the one that I would find most useful would be the Langauge Dossier, because it can contain such a variety of evidence of the learner’s progress-- video of speaking tasks, written essays, student-produced interviews, Powerpoint presentation, etc. Also because it would be easy for classmates to share and learn from each others work via a class blog.