Reading #12

Ockey, G. J. (2009).
Developments and Challenges in the Use of Computer-Based Testing for Assessing Second Language Ability
Modern Language Journal, 93(Focus Issue), 836-847.

Ockey argues that Computer based testing has failed to realize its anticipated potential. Describe and discuss on his reasons for his view, and tell why you either agree or disagree with him.

Cummins, P. W. & Davesne, C. L. (2009).
Using Electronic Portfolios for Second Language Assessment
Modern Language Journal, 93(Focus Issue), 848-867.

Cummins & Davesne offer an alternative to CBT with electronic portfolios. Comment on some of the ideas from this article that you'd be interested in trying out in your own classroom.

>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>

Re. Ockey, Developments and Challenges in the Use of Computer-Based Testing

In this article, Ockey discusses the range of challenges and limitations associated with CBT, and gives two primary reasons for his conclusion that CBT testing has failed to live up to its anticipated potential. First, we have still not found a reliable way to develop computer-based tests that employ more authentic task types. Second, computers are still unable to reliably measure a test-takers actual feeling and meaning in constructed response items, especially regarding speaking and writing.

I agree with Ockey’s reservations, because I don’t have enough faith in computer programs to believe they can adequately assess constructed response types such as essays or speech. Computer programs can be developed to measure a wide range of features of speech and writing, but one thing that computers can certainly never measure is how well people are communicating and expressing themselves.

Although many types of CBT tests may often be nearly 100% reliable, their overal validity is highly debateable. Tests such as the PhonePass/Versant tests and Automated Essay-Scoring (AES) systems tests, in which computers measure speaking or writing ability, have statistically been shown to give scores that correlate to a high degree with scores given by real human raters.

However, Ockey has two main criticisms: first, that the inauthentic nature of the assessment items (such as asking test-takers to rearrange words to form sentences) may potentially have a negative washback effect, and second, that it is impossible for a computer analysis to determine how much sense the speaker or the writer is actually making.

>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>

Re. Cummins & Davesne, Using Electronic Portfolios

A general observation regarding the use of portfolios (digital or otherwise)… as a means of assessment, I think there are many reasons why portfolios are probably the most holistic way of assessing a student’s progress. The best thing about this kind of assessment is that it puts more responsibility on the student-- not just to get a good score on 3 or 4 tests, but to engage in a certain amount of planning and organization, and to reflect on how each element contributes to the overall purpose of a particular course. It can help give the learner the motivation to engage in the kind of incremental preparation that reinforces a more meaningful learning experience. It is also a means of assessment that allows for a greater degree of individual choice and expression.

Of all of the methods of EP assessment examined in this article, I think the one that I would find most useful would be the Langauge Dossier, because it can contain such a variety of evidence of the learner’s progress-- video of speaking tasks, written essays, student-produced interviews, Powerpoint presentation, etc. Also because it would be easy for classmates to share and learn from each others work via a class blog.

Reading 11

Thursday, April 1, 2010
Reading Prompt #11
RICHARDSON Ch. 6. The Social Web: Learning Together
Warschauer, M., Knobel, M., & Stone, L. (2004).
>>>>>>>>>>>>>>>>>>>>

Share an idea of two about how you could use the ideas from Richardson with ESL students.

Re. Richardson, Chapter 6.

Regarding Twitter, Diigo and del.icio.us-- even after reading this chapter, I still don’t understand enough about how they work, so I must admit that I am unable to even imagine what concrete use they may serve in language classroom.

Somewhere in the middle of the chapter, as I was feeling more and more confused, I started wondering if this wasn’t going beyond the point of reason-- going a bit too far in terms of ‘piling on’ the digital technology. But then, I may just be too old and too slow. I suppose kids nowadays have a bigger appetite and aptitude for adapting to all the latest trends and applications, and I’m sure a skillful teacher will know how to selectively adopt some of these applications that actually contribute to effective learning.

>>>>>>>>>>>>>>>>>>

Warschauer, M., Knobel, M., & Stone, L. (2004).
Technology and equity in schooling: Deconstructing the Digital Divide. Educational Policy, 18(4), 562-588.

What is the "digital divide?' And why do Warschauer et al argue that this term may longer be applicable. What are the issues they found regarding differences in technology use with ELLs and other low socioeconomic students. What are some ways you could address this in your own schools or programs?

Warschauer argues that the term “digital divide” is less applicable today because the problem of unequal access to technology is far too complex a problem to be understood simply by counting how many students have access to computers or the internet. Simply comparing access to technology leads to misleading results.

In fact, as Warschaur points out, this question of access to technology is every bit as complex as any other of the many factors contributing to inequality in education. Regarding technology in schools, the central question is how technology is integrated into the curriculum. Other contributing factors include how well the systems are maintained.

One of the points that Warschauer makes repeatedly is that schools need to stop seeing mastery of various applications as an end in itself, and give more attention to helping students learn to use them for actual learning and research.

Reading #10

RICHARDSON Ch. 7. Fun with Flickr: Creating, Publishing, and Using Images Online
BLACKBOARD - Kern, R. (2006). Perspectives on technology in learning and teaching languages. TESOL Quarterly, 40(1), 183-210.

After reading Richardson, what ideas do you have for using images and programs such as Flickr in the language learning classroom?

Kern gives a broad overview of issues in CALL, with specific examples from three areas and implications for teaching and research. Select and share your own thoughts and opinions on one or more issues that Kern raises.

Re. Flickr
RICHARDSON Ch. 7. Fun with Flickr: Creating, Publishing, and Using Images Online

As Richardson points out, some of the best features of a site like Flickr are the different ways in which images (and videos) can be organized. They can be indexed with tags, put into sets and slideshows, or organized together with other users’ images into groups with specific themes.

One advantage of flickr is that it might be considerably faster and easier for younger students to learn how to use, compared with, for example, video-casting software.

I think one good (and pretty obvious) way to try to begin using flickr in a K-12 class (and I believe it is similar to some of the other activities we have deiscussed in class) might be to have students bring their cameras on a field trip (to the zoo, museum, etc.), and then in groups students can prepare and upload a slideshow of images, which they can then show in class and narrate for their classmates. Another project could be to have students make a step-by-step “how to” presentation using a set of images. For example, they could interview a beekeeper, and get enough images of the process to make a short presentation on beekeeping which they could show and narrate for the class. Or the instructor might have them interview their parents or grandparents, and prepare a similar “how to” on the steps in making a pie, barbequeing a goat, etc.--the key, of course, is choosing a direction or topic that students will think is interesting and also fun to share with classmates.

Re. Kern
Kern, R. (2006). Perspectives on technology in learning and teaching languages. TESOL Quarterly, 40(1)

I think it was this paragraph that I thought was most thought-provoking in this article:

“Does multimedia authoring improve learners’ language use in terms
of accuracy, fluency, and appropriateness in offline contexts? We don’t
know. But the value of such projects may be found elsewhere. Nelson, for
example, is not looking at language learning in the traditional sense of
acquisition of morphosyntax or vocabulary, or even academic writing.
Rather, he is looking at learners’ acquisition of a metacommunicative
ability to reflect broadly on signifying practices and specifically on
textualization, considering language as just one dimension of semiosis.” (p. 197)

There are many details that can be measured when attempting to evaluate the effectiveness of various types of CALL tools, media, methods or approaches. I think that among the more difficult variables to measure are the effects in terms of learners’ attitudes and motivation. I was thinking specifically about the long-distance tele-collaboration that Kern was describing. When looking at the effectiveness of tele-collaboration, then, I think that one must keep in mind that a lot of the potential benefits might be in raising learners’ level of cultural awareness, in sparking their interest and curiousity in the culture associated with the target language, and in helping remind learners that language-learning has concrete and real-life applications. So the difficulty for evaluation, I think, is that the biggest potentially-positive effects are the hardest to measure.

Reading Prompt #9 (March 25th)

Reading Prompt #9 (March 25th)

Barbierie, F. (2005). What is Corpus Linguistics?
Conrad, S. (2000). Will Corpus Linguistics Revolutionize Grammar Teaching in the 21st Century? TESOL Quarterly, 34(3), 548-560.

What is Corpus Linguistics? And why are some language teachers so excited about it? What applications can you envision for your current or future classroom?

I think what I understand from Conrad’s “integration of grammar and vocabulary” is that, instead of teaching students what may or may not be grammatically possible (correct or incorrect), it may be much more effective to focus their learning on those lexical and grammatical combinations that have been shown (by corpus linguistics) to be preferable (more natural) to native speakers.

I like the idea that corpus-influenced teaching materials can help streamline the teaching of certain language features, giving students the boiled-down essentials of the high-frequency grammatical patterns that will be the most useful to them in everyday situations.

As far as classroom applications of corpus linguistics-based research, I agree with Conrad that probably the greatest impact will be on making teaching materials more practical and effective.

As for which applications I can see for my own teaching, I think that teachers should take advantage of current corpus-linguistic research to help them use existing materials more effectively.

Also, as more and more classrooms have internet access, it has become fast and easy to check concordances when questions arise regarding particular grammar patterns, collocations, etc.

One thing I would like to know more about is the details of exactly how corpora are assembled, how choices are made on which sources to include (or exclude), and how those choices influence the analyses people make based on these corpora. For example, are sources from ALL users of English (native/ non-native/ all of the various Englishes) going into the same corpus? How does that influence the analyses that are based on that corpora?

Readings, March 9th

Re. Levy, Chapter 4.

In learning about all of the different forms of CMC interaction, it often occurs to me that it is easy to think of ways in which it can provide learning opportunites for language learners who are already highly-motivated and highly-independent learners. It is a lot harder to think of ways that the forms of CMC discussed in this chapter (synchornous and asynchronous text communication) could actually be used effectively for normal students in normal classroom environments.

If students can be motivated to keep up with it regularly, I think that chat can be a good way to practice for students at any level of proficiency. E-mail can work best for students wanting to improve formal writing skills.

But I think the bigger question is how students can be motivated to actually use these forms of CMC regularly enough to get meaningful practice.

I text-chat in my 3rd language (Mandarin) now and then, but not specifically for the purpose of practicing reading and writing. However, I can definitely see how it could be a very helpful language-learning activity if done regularly enough.

One thing I like about the text-chat format is that it is easy to save a copy of the text for later reference (like checking unfamiliar words or phrases in the dictionary). Also, the usage is much more like spoken language than other things I might try to use for reading practice.

But I think for it to effective and enjoyable in the long-term, one would have to be pretty lucky in finding a language-learner who is at a similar level of proficiency, and who has a similar level of motivation, so that both people are willing to make a regular thing of it.

Re. Sauro, (2009).

This study undertook to find out whether recasting or metalinguistic prompting was the more effective form of corrective feedback given via synchronous text chat (SCMC).

The relative effectiveness of these two forms of corrective feedback on students' production of grammatical writing was evaluated indirectly, by measuring the students’ (receptive) skills on “acceptability judgement” tests.

The article also suggests that corrective feedback given via SCMC is substantially different in a number of ways than feedback given face-to-face, and refers to previous comparison studies involving face-to-face and CMC feedback. Some of the limitations identified with face-to-face feedback are: more time stress, more ambiguity, and more dependence on short-term memory capability. Text chat gives learners more time to consider their response, and even to look back several times at the textual corrective feedback. For these reasons, it is suggested that SCMC feedback can be more effective than face-to-face feedback in terms of “elicted uptake”: "It is these limitations of what are otherwise effective properties of recasts delivered during face-to-face interaction that put SCMC in the form of text chat at an advantage for encoding recats in ways that facilitate cognitive comparison." (page 100)

Re. Implications:

I suppose these findings can have some practical value for designers of CALL software, but I am having trouble envisioning just what sort of interactive activities that might involve.

That’s because in this study, a single student is interacting with a single instructor. And I think we already know that one-on-one tutoring can be an effective means of language learning, whether that is done face-to-face or via CMC.

The study shows that corrective feedback which includes metalinguistic prompts is more effective that recasts, but I don't see what significance that finding might have in conventional ESL/EFL educational settings, beyond the significance for instructors who actually engage in one-on-one syncronous text chat with their students. I don't know, but I doubt that there are very many ESL/EFL instructors who are able to spend much time doing that.

I guess what I don’t see is how this interaction model (and the finding from this particular study) could be adapted to something on a larger scale than just one-on-one.

Could a software program be written to give this kind of feedback automatically, so that a student is interacting with a computer instead of with a real person? Or could one instructor give this kind of feedback to a dozen students simultaneously?

Reading presentation, March 4th

Help Options and Multimedia Listening:
Students' Use of Subtitles and the Transcript
Maja Grgurovic and Volker Hegelheimer

http://docs.google.com/leaf?id=0B_Q93EAojk9qYWJjZDc2NGEtMjYzMS00Y2E1LTljNzQtNzZhMjc0YzgyZGM3&hl=en

Reading #7 (March 4th)

LEVY – Ch. 7 Practice

In the Listening section, I can agree that audio-video-conferencing-activities would seem to have a huge potential in giving students authentic listening (and speaking) practice, but since I have never actually seen it done, I would have to say that I have a difficult time visualizing exactly how such classroom activities it could be structured effectively.

I don’t agree completely with Levy’s ideas about the CMC. He states several times that “asynchronous forms of CMC are potentially better-suited to grammatical development…” (p.186).

Yes, synchronous CMC (chat) dialog is more like spoken communication than formal writing, but I think that the grammatical knowledge that learners build (and internalize) though extensive speaking (or CMC chatting) will reinforce their sense of what grammatical forms are appropriate even in formal writing.

CMC chat can provide immediate feedback, and for learners at early stages, the way they write is more like the way they speak.

Of course, if it is an course for advanced learners specifically focused on academic writing, then synchronous chat will not be the best form of practice.

Grgurović, M. & Hegelheimer, V. (2007). Help
Options and Multimedia Listening: Students' Use of Subtitles and the Transcript. Language Learning & Technology, 11(1), 45-66.

All the while I was reading this study, the question that kept occurring to me is this:

Is the activity that the authors designed specifically for this study actually an effective way to practice and reinforce listening skills?

Or was it designed solely to answer the very narrow question of whether students find using subtitles or transcripts more effective in the case of comprehension breakdown?

If it is the first purpose, then that makes a lot more sense to me. However, if they see it as an effective listening practice for ELLs, then I would have a lot of questions to ask, for example:

If it is supposed to be a listening activity, then why are the comprehension questions given t the students in writing??

If it is supposed to be a listening activity, why are students sent directly to a text-based form of support (if they answer incorrectly) without first giving them some modified form of audio-visual (listening) input to help them comprehend better?

Many ELLs have difficulty in improving their speaking and listening precisely because they are in the habit of relying too heavily on reading. If given the choice, these students may rely 100% on the text-based support, and ignore the audio…which kind of turns a “listening” activity into just another reading activity.

I think if I was designing (or testing) this kind of multimedia activity, I would also like to see at least one more level added to the activity. In the case of a wrong answer to the comprehension question at the end of each segment, instead of sending the user straight to a text-based form of support, I think it would be good to add at least one (and maybe two) steps to the process in the form of audio support. For example, if they answer incorrectly after the first viewing, let them:

1. see the video segment again (without textual support, and without the option
to pause or rewind), and then have them attempt the comprehension
question again. If they still answer incorrectly, then have them…

2. see the video segment once more (again without textual support), but this
time give them the option to pause or replay the segment before
attempting the comprehension question again.