|
Articles Conference Reviews |
200766ZeffSession 6.6: Closing the Loop in Cyberspace: Online Learning Environments as Sites for Assessing Learning and Teaching
Bill Condon (Washington State University), Ron Balthazor and Christy Desmet (Univeristy of Georgia), Mike Palmquist (Colorado State University), Fred Kemp (Texas Tech University) How is assessment being used to evaluate teaching and learning with technology? This was the fundamental question behind the session “Closing the Loop in Cyberspace: Online Learning Environments as Sites for Assessing Learning and Teaching” facilitated by Bill Condon of Washington State University. Condon invited luminaries in the field with well developed writing programs that use technology: Ron Balthazor and Christy Desmet from University of Georgia, Mike Palmquist from Colorado State University, and Fred Kemp from Texas Tech University. The session involved each ruminating on how their programs use assessment to better understand how technology and online learning environments help students write better and how the use of technology is helping improve the teaching of writing followed by lively Q&A. The first speakers were Ron Balthazor and Christy Desmet from the University of Georgia who discussed their ongoing research of “Emma,” their open source homegrown paper grading and commenting tool. Because Emma allows them to do all paper commenting and grading through the same system, they have the opportunity to collect and study all the embedded meta-data. They are currently involved in three ongoing assessment studies. The first is examining the correlation between library instruction and student citation practice. What they have found is that library instruction alone showed no improvement. The improvement came when the instruction was done in conjunction with good research prompts. The second study is examining the most frequently marked errors in student papers. The results are proving to be very similar to the findings of Andrea and Karen Lunsford who are redoing the 1988 Connors and Lunsford research on the top writing errors of college students. Balthazor and Desmet discussed that the benefit of their research on error is that it has helped teachers identify the top errors they are seeing in student writing and then do just in time mini lessons on those specific issues. The final and largest study being done with the Emma data is on revision. They are looking at 450 pairs of before and after papers to examine improvement. In the initial stage of the study, they’ve found that they saw the most improvement from revision from students in the middle. The best writers and the weakest writers showed the least improvement after revision. They are now looking more closely at 20 students who improved the most and who improved the least to determine the causes. The second presentation was by Mike Palmquist from Colorado State University. Technology has been an integral part of writing instruction and assistance at Colorado State University for over 10 years with the movement of their online writing center to the web in 1996. In 1999 they did some studies on the impact of computers on writing that changed the way they looked at their online writing resources. One finding of the study was that teaching writing in a computer lab changed the dynamics from one of lecture to a more experiential learning environment where the students could ask questions, had resources at their fingertips, could share drafts with others, and work on their writing online in the lab. In other words, teaching writing in a computer lab was more like an art studio class rather than an art history class. These findings led the team at Colorado State University to restructure what they were doing online to one that captured the spirit of a studio experience. Today, their writing studio is a showcase of what can be done to assist writers online and has over 42000 users worldwide with 20-30 schools using the associated course management system. The popularity and effectiveness of their system started to get the writing@colorado team to think about assessment in a more serious way. They decided to build assessment into new features such as their eportfolio and wiki. Because the writing studio is database driven, they can slice and dice the data in many ways. Questions such as what features were being used and what were not and why continue to interest them and direct how the writing studio is updated and expanded. One study they are doing is evaluating the impact course management systems are having on teaching and learning. For the first stage of the study, they are looking at how new GTAs and new instructors use computer classrooms versus traditional classroom. The next stage of the study will be to see what happens when these new GTAs and instructors receive mentoring in the use of more advanced course management tools. The final presentation was by Fred Kemp from Texas Tech University (TTU). Kemp opened with an overview of what they’re doing at TTU with their TOPICS course management system and ICON (Interactive Composition Online). They developed these systems as a way to offer quality and consistent composition instruction when using graduate student teachers with vastly different expertise and teaching experience. In developing the system the TTU team examined how a writing course could be refigured so that the classroom experience supported the online work and not the other way around. The result was reducing class-time, increasing the number of assignments, and making the grading anonymous through a grading pool. At TTU all composition courses use the same assignments and papers are graded anonymously by a pool of graders. This allows the papers to be graded rather than the student. It also insures consistency in grading. Today, they serve 3000 students a semester and this system allows the program to retain the knowledge and continue learning even with rotating faculty. In terms of assessment, by putting everything online, TTU is able to capture important decisions while they are being made. In other words, they built assessment into the process itself. Their system has 264 fields that collect data. They now have 5 years of data from 20,000 students and about one quarter of a million documents. This represents 6.5 million fields of data. No one has ever tried to use all this data. Some of the findings thus far have focused on student and instructor behavior. For example, they have found that instructors have bias toward certain genres. This system is not without its critics. The most common is that it abdicates the learning to technology and reduces the role of faculty. The TTU team argues that the system actually insures consistency, allows the program to retain the knowledge and continue learning even with rotating faculty, and makes and that the real basis of the criticism is faculty concern about losing authority. The findings are showing that though faculty may be uncomfortable about losing authority, this approach of shifting authority from the faculty to a more studio approach better supports the student writer. The session concluded with time for Q&A. After a lively exchange about assessment projects at different programs that are researching how smart and strategic use of technology is improving the teaching and learning of writing, the theme of the session became clear: Assessment is not only a necessary part of program administration, but good assessment is good research and vice versa.
Comments? |