Criteria for Traditional Print-Based Scholarship

An influential study on the assessment of scholarship within the academy, “Scholarship Assessed: Evaluation of the Professorate,” offers one model for judging scholarly performance based on a universally-applicable set of standards. Charles Glassick, Mary Huber, and Gene Maeroff (1997) compiled information from several sources—including interviews with editors of 31 scholarly journals and 58 university presses—regarding the criteria used to determine the scholarly merit of manuscripts, proposals, submissions and more. Among the multiple lists and guidelines, the authors discovered that scholarly activity is guided by six shared themes: clear goals, adequate preparation, appropriate methods, significant results, effective presentation, and reflective critique. These six standards offer a conceptual framework for identifying and evaluating print-based scholarship. The authors incorporate prompts for each standard represented in their model as follows:

Clear goals: Does the scholar state the basic purposes of his or her work clearly? Does the scholar define objectives that are realistic and achievable? Does the scholar identify important questions in the field?

Adequate Preparation: Does the scholar show an understanding of existing scholarship in the field? Does the scholar bring the necessary skills to his or her work? Does the scholar bring together the resources necessary to move the project forward?

Appropriate Methods: Does the scholar use methods appropriate to the goals? Does the scholar apply effectively the methods selected? Does the scholar modify procedures in response to changing circumstances?

Significant Results: Does the scholar achieve the goals? Does the scholar’s work add consequentially to the field? Does the scholar’s work open additional areas for further exploration?

Effective Presentation: Does the scholar use a suitable style and effective organization to present his or her work? Does the scholar use appropriate forums for communicating work to its intended audiences? Does the scholar present his or her message with clarity and integrity?

Reflective Critique: Does the scholar critically evaluate his or her own work? Does the scholar bring an appropriate breadth of evidence to his or her critique? Does the scholar use evaluation to improve the quality of future work? (p. 36)

This heuristic reflects the expectations inherent in traditional scholarly publications. Specifically, journal articles are expected to contain an argumentative claim (establishing clear goals), a review of the literature (establishing adequate preparation), a statement of methodology (appropriate methods), and so forth. Scholarly arguments are expected to make a substantial contribution to the field and follow a standard linear arrangement of key parts (introduction, concession/refutation, conclusion), with each part achieving certain goals (for example, the introduction should establish the claim and the context of the argument).

Additional sources to consider for references to print-based standards of scholarship—and ones that are most relevant to this study of online scholarly articles—include prominent print journals in the sub-field of Computers and Writing. Patricia Webb Peterson (2002) observed that a journal designates “acceptable scholarship” based on the identification of a list of preferred topics, formats, and styles within its submission guidelines. The well-regarded print journal Computers and Composition, for example, includes the following “Editing Philosophy and Profile” in its “Guidelines for Editors and Authors:”

Not only do the editors of the journal look for articles that have sound theoretical and/or pedagogical bases, but they strive to publish articles that in their very writing demonstrate the high-quality writing the discipline teaches. This is generally accomplished through a coherent organization, well-developed arguments, well-written sentences, and accurate documentation. Authors should introduce subject matter within the context of those interested in computers and composition, using terms and cultural references that either are commonly understood within our international community or are carefully explicated within the article itself….Because the journal has primarily an academic audience, it is generally scholarly and more formal than magazines; yet, it strives to avoid a preachy or labored tone.

These guidelines establish a number of scholarly standards for the texts published—and thereby deemed “scholarly”—within the journal: arguments should be grounded in theory or pedagogy; the writing itself should be clear, logical, coherent, grammatically correct—all the well-known conventions associated with traditional definitions of effective print writing; documentation should be incorporated in a fair and consistent manner; terminology should be familiar and appropriate to the audience or otherwise defined within the context of the argument; and the tone should be formal.

While the submission guidelines of reputable journals establish explicit standards of scholarship, it is through an analysis of actual texts published within the journal that scholars can find more implicit standards. In her analysis of the rhetorical presentation of a text published in Computers and Composition, for example, Peterson (2002) identified several formatting, typographic and stylistic conventions that are seemingly transparent to scholars and which, she asserted, are representative of a majority of print-based scholarly arguments:

Peterson asserted that these “formatting” conventions contribute to the scholarly ethos of a text in that they “emphasize scholarly tradition constructed by/through our learned expectations of print journals.” In other words, they establish expectations for readers in ways that mark these texts automatically as scholarship.

A majority of these standards are echoed in the field’s guides to scholarly publishing. Specifically, Joseph Gibaldi’s MLA Style Manual and Guide to Scholarly Publishing (2003) established the qualities that distinguish all effective scholarly prose:

Effective scholarly writing, then, depends on clarity and readability as well as on content. The organization and development of ideas, unity and coherence of presentation, and fitness of sentence structure, grammar, and diction are all essential considerations, as is the correctness of the mechanics of writing—capitalization, punctuation, spelling, and so on. (p. 64)

Here Gibaldi identifies the interdependence of form and content. Both must work together to produce an effective scholarly argument. He argues that attention to these formal and stylistic conventions helps writers establish scholarly ethos:

In a large field such as ours, adherence to these codes allows your writing to be taken seriously, whether by referees who decide the publication of your work or by readers whom you ultimately hope to convince with your evidence and arguments but who are otherwise unacquainted with you. Indeed, it is through the confines imposed by a commonly acknowledged set of practices that readers can judge the competence of your methods and the individuality of what you offer. (p. xvi)