Does Quantity Equal Quality? The Relationship between Length of Response and Scores on the SAT Essay
Authors
-
The College Board
Jennifer L. Kobrin -
The College Board Hui Deng
-
The College Board Emily J. Shaw
Abstract
This study was designed to address two frequent criticisms of the SAT essay -- that essay length is the best predictor of scores, and that there is an advantage in using more "sophisticated" examples as opposed to personal experience. The study was based on 2,820 essays from the first three administrations of the new SAT. Each essay was coded for number of words, number of paragraphs, whether or not the response included first-person, and whether or not the response went to the second page. Analyses included descriptive statistics and group comparisons on the essay response features, correlations between essay length and scores, and hierarchical multiple regression to examine the contribution of each essay feature variable to the prediction of essay scores. The number of words in the essay explained 39% of the variance of essay scores. Whether or not the essay reached the second page explained an additional 1.5%, and whether or not the essay was written in first person explained an additional 1.1% . An examination of these features potentially affecting SAT essay scores is essential to maintain that the SAT writing section promotes valid interpretations of students' writing skills. The research described in this paper may benefit other testing programs that include essay assessments. The careful analysis of response features and the identification of potential construct-irrelevant features in essay assessments are important for evaluating the content and construct validity of writing assessments.Downloads
Published
How to Cite
Issue
Section
References
Breland, H.M., Danos, D.O., Kahn, H.D., Kubota, M.Y., & Sudlow, M.W. (1991). A study of gender and performance on Advanced Placement history examinations (College Board Research Rep.No. 91-4). New York: College Entrance Examination Board.
Breland, H.M., & Jones, R.J. (1988). Remote scoring of essays (College Board Research Rep.88-3). New York: College Entrance Examination Board.
Cioffi, F.L. (2005, May 20). Argumentation in a culture of discord. The Chronicle of Higher Education, p. B6.
Cohen, J. (1988). Statistical power analysis for the behavioral sciences (2nd ed.). Hillsdale, NJ: Lawrence Erlbaum Associates.
College Board. (2004). ScoreWrite: A guide to preparing for the new SAT essay. New York:College Entrance Examination Board.
College Board (in press). Technical Manual for the SAT Reasoning Test. E. Kimmel (Ed.). New York: College Entrance Examination Board.
Hoover, E. (2005, April 22). And on the 11th day, the readers rested. The Chronicle of Higher Education, p. A39.
Katzman, J., Lutz, A., & Olson, E. (2004, March). Would Shakespeare get into Swarthmore? The Atlantic Monthly, pp. 97-100.
Nunnally, T.E. (1991). Breaking the five-paragraph-theme barrier. The English Journal, 80, 67-71.
Perelman, L. (2005, May 29). New SAT: Write longly, badly, and prosper. Los Angeles Times, p.M5.
Powers, D.E., Fowles, M.E., Farnum, M., & Ramsey, P. (1994). Will they think less of my handwritten essay if others word process theirs? Effects on essay scores of intermingling handwritten and word-processed essays. Journal of Educational Measurement, 31, 220-233.
Setoodeh, R. (2005, March 14). Penmanship, the newest SAT worry. Newsweek, p. 48.
Skutches, G. (2005, July 24). New SAT essay format is “teaching to the test,†not teaching writing, critical thinking [Opinion]. The Morning Call, p. D1.
Thomas, P. (2004). The negative impact of testing writing skills. Educational Leadership, 62, 76-79.
Wesley, K. (2000). The ill effects of the five paragraph theme. English Journal, 90, 57-60.
Winerip, M. (2005, May 4). SAT essay test rewards length and ignores errors of fact. New York Times, p. B9.