Selecting Webtexts for Analysis

The method of gathering data for analyzing the scholarly value of online texts involved the selection of a representative group of online texts. The parameters of the analysis include a focus on webtexts published within the first ten years of Kairos’s existence as a scholarly online journal in the subfield of Computers and Writing, beginning with Issue 1.1 published in Spring 1996 and ending with Issue 10.2 published in Spring 2006. The archives of this time period include an overwhelming volume of work with 22 issues and more than 300 webtexts. For the time period under study, Kairos organized publications into five main sections: CoverWeb, Features, Praxis, Interviews, and Reviews. Two of these sections—CoverWeb and Features—were rich for analysis in that the publications organized within these sections focused primarily on original research and interpretation of texts and data, and therefore shared the most similarities with a traditional scholarly journal publication.

A natural and perhaps ideal group of webtexts was identified through an annual award associated with the journal titled, the “Kairos Best Webtext Award.” This set was selected based on the assumption that webtexts chosen as the “best” would reveal the highest standards for the kind of scholarship that Kairos claims to publish. Additionally, because this award has been presented annually since the inception of the journal, it provides a set of texts that could be analyzed for trends over time, thereby reflecting the technological evolution of the medium. The Kairos Best Webtext Award recognizes the outstanding webtexts of each year of publication and is determined by nominations for any “publicly accessible” webtext. Select Kairos staff and board members review the nominated webtexts and choose one winner and one or two finalists each year. The nominated webtexts must meet specific award criteria.

The award was developed in conjunction with the debut of the journal. The first Kairos Best Webtext Award was presented at the 1997 Computers and Writing Conference to recognize webtexts created and published online in 1996-1997. At the date of this analysis, the Awards page listed 25 webtexts—nine winners (one for each year of the award up through 2006) and 16 finalists; usually two finalists were chosen per year with the exception of the 2000-01 and 2005-06 award periods. Out of these 25 texts, 16 texts were published in Kairos and nine texts were published elsewhere: two each were published in Pre/Text, Enculturation, and Computers & Composition Online, and three were self-published projects. Because this study focuses on texts published in Kairos, these nine texts were automatically eliminated from the sample.

Additionally, three of the Kairos webtexts that require a different and more complicated assessment approach were also excluded from the sample. These three webtexts resemble edited compilations in which individual contributions by multiple authors comprise a collaborative webtext, and each deserves a separate analysis from the more typical stand-alone webtexts that dominate the type of text published within this journal. They include from earliest published to latest published: “Hypertext Reflections: Exploring the Rhetoric, Poetic, and Pragmatics of Hypertext” by Mike Palmquist, et.al., published in 2.2, Spring 1997; “Computers and Writing 2000,” by John Barber, et.al., published in 5.2, Fall 2000; and “Violence of Text: An Online Academic Publishing Exercise,” by Adrian Miles, et.al., published in 8.1, Spring 2003.

The 13 remaining Kairos-published “best” webtexts became the first data set for analysis. This longitudinal sampling includes at least one webtext from each year that Kairos has been published with the exception of 1998-2001; winners and finalists during this time period were published in journals other than Kairos.

The 13 selected webtexts, along with a set of more recent webtexts—all CoverWeb and Feature webtexts published in Kairos between 2004 and 2006—were instrumental in determining parameters for constructing the assessment tool and are currently under review as part of the larger study to determine the nature of online scholarship.