ES EN
Vol. 20. Núm. 2. 2014. Páginas 99-108

A case study in principled assessment design: Designing assessments to measureand support the development of argumentative reading and writing skills

[Un estudio de casos en el dise√Īo de la evaluaci√≥n centrada en principios: dise√Īode evaluaciones para medir e impulsar el desarrollo de la argumentaci√≥n enhabilidades de lecto-escritura]

Paul Deane1 , Yi Song1 1Educational Testing Serv., Princeton, New Jersey, USA

Abstract

This paper presents a principled approach to assessment design in which major design decisions arestructured to support teaching and learning. This approach, developed as part of a long-term researchinitiative at ETS, Cognitively Based Assessments of, for and as Learning (CBAL), draws upon the learning andcognitive science literatures to create richly-structured assessments that simultaneously measure criticalcomponent skills and model effective strategies for applying those skills to complex performance tasks. Toillustrate our approach, we focus on an important literacy practice: argumentation. Our model seeks tomeasure qualitative shifts in the development of critical argumentation skills by postulating argumentationlearning progressions informed by the developmental literature. These learning progressions play a criticalrole in guiding assessment design decisions (selecting targeted skills, developing items to measure thoseskills, and determining task sequences) and may have the potential to support teachers' instructionaldecisions that effectively scaffold the development of students' argumentation skills.

Resumen

Este artículo aborda la cuestión de cómo diseñar de manera fundamentada una evaluación donde las principalesdecisiones se toman con el fin de apoyar el proceso de enseñanza-aprendizaje. Este trabajo ha sidodesarrollado como parte de un extenso proyecto de investigación en el ETS -evaluación cognitiva de, por ypara el aprendizaje (CBAL en su acrónimo inglés)- y se nutre de la literatura previa sobre cognición yaprendizaje para crear evaluaciones con una estructura muy elaborada que, de forma simultánea, midenhabilidades críticas y modelan estrategias eficaces para aplicar esas habilidades a tareas complejas de resolver.Para ilustrar este marco de trabajo, nos centramos en una importante práctica relacionada con lalectura y la escritura: la argumentación. Nuestro modelo trata de medir cambios cualitativos en el desarrollode habilidades críticas de argumentación, postulando una progresión de aprendizaje para la argumentacióntomada de la literatura especializada. Las progresiones de aprendizaje juegan un papel decisivo a lahora de tomar decisiones relativas al diseño de la evaluación (seleccionar las habilidades básicas, elaborarpreguntas para medir esas habilidades y determinar la secuencia de las tareas) y pueden también contribuira que los profesores tomen decisiones relativas a la instrucción que sirvan para estructurar de formaefectiva el desarrollo de la capacidad de argumentar de sus estudiantes. 

References
Aristotle, 1939
Aristotle (1939). Topics (E. S. Forster, Loeb, Trans.). Classical Library. Cambridge, MA: Harvard University Press.
Bennett, 2010
R.E. Bennett
Cognitively based assessment of, for, and as learning: A preliminary theory of action for summative and formative assessment
Measurement: Interdisciplinary Research and Perspectives
8
2010
70-91
Bracewell et al., 1978
J.R. Bracewell
M. Scardamalia
C. Bereiter
The development of audience awareness in writing
Resources in education
1978
154-433
Brem and Rips, 2000
S. Brem
L. Rips
Evidence and explanation in informal argument
Cognitive Science
24
2000
573-604
Bright, 1995
R. Bright
Writing instruction in the intermediate grades. What is said, What is done, What is understood
1995
Butler and Britt, 2011
J.A. Butler
A.M. Britt
Investigating instruction for improving revision of argumentative essays
Written Communication
28
2011
70-96
Chambliss, 1995
M.J. Chambliss
Text cues and strategies successful readers use to construct the gist of lengthy written arguments
Reading Research Quarterly
30
1995
778-807
Clark and Delia, 1976
R. Clark
J. Delia
The development of al persuasive skills in childhood and early adolescence
Child Develoment
47
1976
1008-14
Coirier et al., 1999
P. Coirier
J.E.B. Andriessen
L. Chanquoy
From planning to translating: The specificity of argumentative writing
Foundations of argumentative text processing
Amsterdam University Press
Amsterdam, The Netherlands
1999
1-28
Corcoran et al., 2009
T. Corcoran
F.A. Mosher
A. Rogat
Learning progressions in science: An evidence-based approach to reform
2009
Council of Chief State, 2010
Council of Chief State School Officers & National Governors Association (2010). Common Core State Standards for English language arts and literacy in history/social studies, science, and technical subjects. Washington, DC: Author. Retrieved from www.corestandards.org/the-standards/ELA-Literacy.
De La Paz and Graham, 1997a
S. De La Paz
S. Graham
Strategy instruction in planning: Effects on the writing performance and behavior of students with learning disabilities
Exceptional Children
63
1997
167-81
Deane, 2011
P. Deane
Writing assessment and cognition, (ETS Research Report No. 11-14)
2011
DiPardo and Freedman, 1988
A. DiPardo
S.W. Freedman
Peer response groups in the writing classroom: Theoretic foundations and new directions
Review of Educational Research
58
1988
119-49
Eisenberg and Garvey, 1981
A. Eisenberg
C. Garvey
Children's use of verbal strategies in resolving conflicts
Discourse Processes
4
1981
149-70
Erduran et al., 2004
S. Erduran
S. Simon
J. Osborne
TAPping into argumentation: Developments in the application of Toulmin's argument pattern for studying science discourse
Science Education
88
2004
915-33
Ferretti et al., 2009
R.P. Ferretti
W.E. Lewis
S. Andrews-Weckerly
Do goals affect the structure of students’ argumentative writing strategies?
Journal of Educational Psychology
101
2009
577-89
Ferretti et al., 2000
R.P. Ferretti
C.A. MacArthur
N.S. Dowdy
The effects of an elaborated goal on the persuasive writing of students with learning disabilities and their normally achieving peers
Journal of Educational Psychology
93
2000
694-702
Flower et al., 1986
L. Flower
J.R. Hayes
L. Carey
K.A. Schriver
J. Stratman
Detection, diagnosis, and the strategies of revision
College Composition and Communication
37
1986
16-55
Gorin et al., 2014
J.S. Gorin
T.P. O’Reilly
J.P. Sabatini
Y. Song
P. Deane
Measurement: Facilitating the goal of literacy
Teaching Reading and Writing: Improving Instruction and Student Achievement
Paul H. Brookes Publishing Co
Baltimore, MD
2014
119-28
Graham and Harris, 1989
S. Graham
K.R. Harris
Improving learning disabled students’ skills at composing essays: Self-instructional strategy training
Exceptional Children
56
1989
201-14
Graham and Hebert, 2010
S. Graham
M.A. Hebert
Writing to read: Evidence for how writing can improve reading (A Carnegie Corporation Time to Act Report). Washington
DC: Alliance for Excellent Education
2010
Graham et al., 1995
S. Graham
C.A. MacArthur
S.S. Schwarts
The effects of goal setting and procedural facilitation on the revising behavior and writing performance of students with writing and learning problems
Journal of Educational Psychology
87
1995
230-40
Hayes and Flower, 1980
J.R. Hayes
L.S. Flower
Identifying the organization of writing processes
Cognitive process in writing
Lawrence Erlbaum Associates
Hillsdale, NJ
1980
3-30
Heritage, 2008
Heritage, M. (2008). Learning progressions: Supporting instruction and formative assessment Retrieved from h t t p & # 5 8 ; & # 4 7 ; & # 4 7 ; w w w & # 4 6 ; k 1 2 & # 4 6 ; w a & # 4 6 ; u s & # 4 7 ; a s s e s s m e n t & # 4 7 ; C l a s s roomAssessmentIntegration/pubdocs/FASTLearningProgressions.pdf.
Hillocks, 2002
G. Hillocks Jr.
The testing trap: How state writing assessments control learning
2002
Kinsler, 1990
K. Kinsler
Structured peer collaboration: Teaching essay revision to college students needing writing remediation
Cognition and Instruction
7
1990
303-21
Klaczynski, 2000
P. Klaczynski
Motivated scientific reasoning biases, epistemological beliefs, and theory polarization: A two-process approach to adolescent cognition
Child Development
71
2000
1347-66
Knudson, 1992
R.E. Knudson
The development of written argumentation: An analysis and comparison of argumentative writing at four grade levels
Child Study Journal
22
1992
167-84
Kuhn, 1991
Kuhn, D. (1991). The skills of argument. Cambridge, England: Cambridge University Press. Kuhn, D., Katz, J.B., & Dean, D., Jr. (2004). Developing reason. Thinking & Reasoning, 10, 197-219.
Kuhn et al., 1997
D. Kuhn
V. Shaw
M. Felton
Effects of dyadic interaction on argumentive reasoning. Cognition and Instruction
15
1997
287-315
Kuhn and Udell, 2003
D. Kuhn
D. Udell
The development of argument skill
Child Development
74
2003
1245-60
Kuhn et al., 2013
D. Kuhn
N. Zillmer
A. Crowell
J. Zavala
Developing Norms of Argumentation: Metacognitive
Epistemological, and Social Dimensions of Developing Argumentive Competence. Cognition and Instruction
31
2013
456-96
Larson et al., 2004
M. Larson
M.A. Britt
A.A. Larson
Disfluencies in comprehending argumentative texts
Reading Psychology
25
2004
205-24
Leit√£o, 2003
S. Leit√£o
Evaluating and ing counterarguments: Studies of children's rhetorical awareness
Written Communication
20
2003
269-306
Matsuhashi and Gordon, 1985
A. Matsuhashi
E. Gordon
Revision, addition, and the power of unseen text
The acquisition of written language: Response and revision
Ablex
Norwood, NJ
1985
236-49
McCann, 1989
T.M. McCann
Student argumentative writing: knowledge and ability at three grade levels
Research in the Teaching of English
23
1989
62-76
Mislevy et al., 2003
R.J. Mislevy
L.S. Steinberg
R.G. Almond
On the structure of educational assessments
Measurement: Interdisciplinary Research and Perspectives
1
2003
3-67
Nation's Center, 2008
Nation's Center for Education Statistics. (2008). The nation's report card: Writing 2007 (U.S. Department of Education Publication No. NCES 2008-468). Washington, DC: U.S. Government Printing Office. Retrieved from h t t p & # 5 8 ; & # 4 7 ; & # 4 7 ; n c e s & # 4 6 ;ed.gov/nationsreportcard/pdf/main2007/2008468.pdf.
Newell et al., 2011
G.E. Newell
R. Beach
J. Smith
J. Van Der Heide
Teaching and learning argumentative reading and writing: A review of research
Reading Research Quarterly
46
2011
273-304
Nussbaum and Kardash, 2005
M.E. Nussbaum
C.M. Kardash
The effects of goal instructions and text on the generation of counterarguments during writing
Journal of Educational Psychology
97
2005
157-69
O’Reilly and Sheehan, 2014
O’Reilly and Sabatini, O’Reilly, T., & Sabatini, J. (in press). Reading for Understanding: How performance moderators and scenarios impact assessment design (ETS Research Report). Princeton, NJ: Educational Testing Service.
O’Reilly and Sheehan, 2009
O’Reilly, T., & Sheehan, K.M. (2009). Cognitively Based Assessment of, for and as Learning: A Framework for Assessing Reading Competency (ETS Research Report RR-09-26). Princeton, NJ: Educational Testing Service.
Pellegrino, 2014
J.W. Pellegrino
Assessment as a Positive Influence on 21st Century Teaching and Learning: A Systems Approach to Progress
Psicología Educativa
20
2014
65-77
Perkins et al., 1991
D.N. Perkins
M. Farady
B. Bushey
Everyday reasoning and the roots of intelligence
Informal reasoning and education
Erlbaum
Hillsdale, NJ
1991
83-106
Persky et al., 2003
Persky, H.R., Daane, M. C, & Jin, Y. (2003). The nation's report card: Writing 2002 (U.S. Department of Education Publication No. NCES 2003-529). Washington, DC: U.S. Government Printing Office.
Reznitskaya et al., 2001
A. Reznitskaya
R.C. Anderson
B. McNurlen
K. Nguyen-Jahiel
A. Archodiou
S. Kim
Influence of oral discussion on written argument
Discourse Processes
32
2001
155-75
Scardamalia and Bereiter, 1986
M. Scardamalia
C. Bereiter
Research on written composition
Handbook of Research on Teaching
Macmillan
New York
1986
778-803
Sexton et al., 1998
M. Sexton
K.R. Harris
S. Graham
Self-regulated strategy development and the writing process: Effects on essay writing and attributions
Exceptional Children
64
1998
295-311
Shanahan, 1990
Shanahan, T. (Ed.) (1990). Reading and writing together : New perspectives for the classroom. Norwood, MA: Christopher-Gordon.
Shavelson, 2009
Shavelson, R. (2009). Reflections on learning progressions . Paper presented at the Learning Progressions in Science (LeaPS) Conference, Iowa City, IA.
Sheehan and O’Reilly, 2011
Sheehan, K.M., & O’Reilly, T. (2011). The CBAL Reading Assessment: An Approach for Balancing Measurement and Learning Goals (ETS Research Report RR-11-21). Princeton, NJ: Educational Testing Service.
Simon and Holyoak, 2002
D. Simon
K.J. Holyoak
Structural dynamics of cognition: From consistency theories to constraint satisfaction. Personality and Social Psychology
6
2002
283-94
Stein and Miller, 1993
N.L. Stein
C.A. Miller
A theory of argumentative understanding: Relationships among position preference, judgments of goodness, memory, and reasoning
Argumentation
7
1993
183-204
Van Eemeren and Grootendorst, 1992
F.H. Van Eemeren
R. Grootendorst
Argumentation, communication, and fallacies: A pragma-dialectical perspective
1992
Van Eemeren et al., 1996
F.H. Van Eemeren
R. Grootendorst
F.S. Henkemans
Fundamentals of argumentation theory: A handbook of historical backgrounds and contemporary developments
1996
Van Rijn et al., 2014
P.W. Van Rijn
E.A. Graf
P. Deane
Empirical Recovery of Argumentation Learning Progressions in Scenario-Based Assessments of English Language Arts
Psicología Educativa
20
2014
109-15
Vygotsky, 1978
L.S. Vygotsky
Mind in society: The development of higher psychological processes
1978
Walton, 1992
D. Walton
Plausible argument in everyday conversation
1992
Zieky, 2014
M. Zieky
An introduction to the use of evidence-centered design in test development
Psicología Educativa
20
2014
79-87

Copyright © 2018. Colegio Oficial de Psicólogos de Madrid

© Copyright 2018. Colegio Oficial de Psicólogos de Madrid ContactPrivacy PolicyCookies Policy

We use our own and third­party cookies. The data we compile is analysed to improve the website and to offer more personalized services. By continuing to browse, you are agreeing to our use of cookies. For more information, see our cookies policy

Aceptar