DVDs, Placement Validity, and Efficacious Deliberation
The primary function of the University of Arizona Writing Program's (UAWP's) orientation videos is to advise students about their first-year writing placement. For years, the University of Arizona's orientation team had pressured the writing program to simplify its placement procedures to reduce scheduling time; this request, combined with the development of new placement models, led program writing specialist Ty Bouldin to develop, in 2002, a placement formula based on multiple indicators of student ability, as explained in the above clip. Yet Bouldin, noting the limitations of these measures, also created an advising and appeal process that asked students to self-assess the accuracy of their placement, to engage in dialogue with advisors if they had further questions or if they wished to change their placement, and to submit a portfolio of their writing if they believed they would be served better by a more advanced class.
The UAWP's approach thereby draws from several different placement models.
- The initial formula provides a more complex profile of students than a single-measure placement. Nonetheless, it relies on objective data to accommodate the pragmatics of large-scale placement in a major university. Because of the limitations of such measures, the UAWP considers this profile to yield a pre-placement only.
- The model relies heavily on students' own judgment and self-knowledge in ways that are informed by (though not identical to) directed self-placement (Royers and Gilles 1998), as we discuss further below.
- The model offers one-on-one advising to students who ask for it. This advising may be followed by a multi-genre portfolio assessment. In those cases where students do submit a portfolio, the UAWP accepts that placement decisions have high content and systemic validity (Royer and Gilles 2002, pp. 268-269). When coupled with a cover letter addressed to the dialogic advising context, we find portfolios also address personal and social measures such as motivation and workload to enhance predictive validity.
Only a small percentage of students reach the point of submitting a portfolio. At an earlier stage, our placement validity relies, first, on whether students can adequately self-assess whether they should accept their pre-placement and, second, on whether students with concerns are willing to engage advisors in dialogue.
There are recognised barriers to engaging students in these ways. In their pioneering approach to directed-self placement, Daniel Royer and Roger Gilles (1998; 2002) ask students to choose which first-year writing class to take after reading a brochure and after hearing a talk, leaving the responsibility for subsequent success or failure on the shoulders of the student, rather than on the various artifacts (such as test scores) that a program "reads" in order to place a student. This process has been critiqued because it may over-project students' self-efficacy for assessing their writing according to the cultural practices of an institution they have not yet joined (Reynolds 2003; Blakesley 2003).
Recognizing that students may not have full self-efficacy in this context, the orientation videos help move the model toward what Pamela Bedore and Deborah Rossen-Knill (2004) call informed self-placement, where responsibility for placement is shared with a high degree of transparency. Bedore and Rossen-Knill ask whether, when given a choice of placement, incoming students receive that choice in the way a writing program hopes they will. Do students have enough information about different course offerings to make a good decision? Have students reflected on their writing practices sufficiently to engage in unassisted deliberation? Most essentially, will new university students approach faculty when they need additional assistance?
Some of the UAWP's early advising experiences illustrated how difficult it can be to invite students into effective dialogue. In 2004, for example, the UAWP worked with the Office of Orientation to create an advising web site that provided information and asked students to reflect on their experience in order to customize advice for students. Yet there was substantial evidence that the majority of students did not take advantage of this advising, and students came to orientation ill prepared for further dialogue about their placement, despite available on-site advisors. From 2005 onwards, however, digital video has helped fill this gap. Videos shown at orientation give students the course information they need, delivered through images that have proven more effective than lectures or print texts at getting students to reflect on their writing practices and their pre-placements.
The videos rely on multiple strategies to encourage advising dialogues where appropriate. To begin, they work to cast doubt on the authority of the pre-placement. As seen in the above clip, UAWP instructor and summer placement advisor Shawn Steinhart looks directly at the camera (and hence the student audience) and explains the factors contributing to pre-placements. Then Steinhart walks off-screen to the right and immediately appears again from the left, more tightly framed, with a noticeable, even jarring, shift in the background. These strategies combine to attract attention as Steinhart starts to disarticulate explicitly any assumptions that the pre-placement serves as the last word on students' preparedness for university writing. Steinhart continues to look directly at students and challenges them to consider the types of knowledge they might bring to a dialogue about placement, thereby preparing the ground for productive face-to-face dialogue with on-scene advisors.
The overall result is enhanced placement validity.