The method of gathering data for analyzing the scholarly value of online texts involved the selection of a representative group of online texts. The parameters of the analysis include a focus on webtexts published within
the first ten years of Kairos’s existence as a scholarly online
journal in the subfield of Computers and Writing, beginning with Issue 1.1
published in Spring 1996 and ending with Issue 10.2 published in Spring 2006.
The archives of this time period include an overwhelming volume of work with
22 issues and more than 300 webtexts. For the time period under study, Kairos organized publications into
five main sections: CoverWeb, Features, Praxis, Interviews,
and Reviews. Two of these sections—CoverWeb and Features—were
rich for analysis in that the publications organized within these sections
focused primarily on original research and interpretation of texts and data,
and therefore shared the most similarities with a traditional scholarly journal
A natural and perhaps ideal group of webtexts was identified
through an annual award associated with the journal titled, the “Kairos Best
Webtext Award.” This set was selected based on the assumption that
webtexts chosen as the “best” would reveal the highest standards
for the kind of scholarship that Kairos claims to publish. Additionally,
because this award has been presented annually since the inception of the
journal, it provides a set of texts that could be analyzed for trends over
time, thereby reflecting the technological evolution of the medium. The Kairos Best
Webtext Award recognizes the outstanding webtexts of each year of publication
and is determined by nominations for any “publicly accessible” webtext.
Select Kairos staff and board members review the nominated webtexts
and choose one winner and one or two finalists each year. The nominated webtexts
must meet specific award
The award was developed in conjunction with the debut of the journal. The
first Kairos Best Webtext Award was presented at the 1997 Computers
and Writing Conference to recognize webtexts created and published online
in 1996-1997. At the date of this analysis, the Awards page listed 25 webtexts—nine
winners (one for each year of the award up through 2006) and 16 finalists;
usually two finalists were chosen per year with the exception of the 2000-01
and 2005-06 award periods. Out of these 25 texts, 16 texts were published
in Kairos and nine texts were published elsewhere: two each were
published in Pre/Text, Enculturation, and Computers & Composition
Online, and three were self-published projects. Because this study focuses
on texts published in Kairos, these nine texts were automatically
eliminated from the sample.
Additionally, three of the Kairos webtexts that
require a different and more complicated assessment approach were also
excluded from the sample. These three webtexts resemble edited compilations
in which individual contributions by multiple authors comprise a collaborative
webtext, and each deserves a separate analysis from the more typical stand-alone
webtexts that dominate the type of text published within this journal.
They include from earliest published to latest published: “Hypertext Reflections: Exploring the
Rhetoric, Poetic, and Pragmatics of Hypertext” by Mike Palmquist, et.al.,
published in 2.2, Spring 1997; “Computers and Writing 2000,” by
John Barber, et.al., published in 5.2, Fall 2000; and “Violence of
Text: An Online Academic Publishing Exercise,” by Adrian Miles, et.al.,
published in 8.1, Spring 2003.
The 13 remaining Kairos-published “best” webtexts
became the first data set for analysis. This longitudinal sampling includes
at least one webtext from each year that Kairos has been published with the
exception of 1998-2001; winners and finalists during this time period were
published in journals other than Kairos.
The 13 selected webtexts, along with a set of more recent webtexts—all CoverWeb and Feature webtexts published in Kairos between 2004 and 2006—were instrumental in determining parameters for constructing the assessment tool and are currently under review as part of the larger study to determine the nature of online scholarship.