SEO Ed Digest
|
Vol. 4, Issue 3
March 2007
Bringing urban P-16 education resources to policymakers, parents, advocates, and district and school staff in the District of Columbia
Research on DC Schools
National Lessons Learned
New Ideas
The State Education Office does not endorse the views expressed in the resources and reports contained in the SEO Ed Digest.
|
|
This issue of the SEO Ed Digest covers recent and background research on value-added models and value-added accountability systems. A value-added model means that student growth will be measured annually. That is, a student would be tested at the beginning of the school year and at the end of the school year. The school would then be able to measure how much a student learned in that year. Measuring annual student gains and tying these gains to school and teacher performance is value-added accountability. This topic is especially relevant because the reauthorization of No Child Left Behind (NCLB) is scheduled for later this year. The Commission on No Child Left Behind, a bipartisan, independent effort dedicated to improving NCLB, came out with their report in mid-February and recommended that teachers be evaluated annually based on progress in the test scores of their students. The panel also recommended that if teachers did not show student progress after two years, the teacher would begin professional development. After seven years, if students still do not make positive gains, the teacher would be prevented from teaching in a school that receives federal poverty aid.
Articles
Reports
Articles
Panel Urges Progress Goals for Teachers (February 13, 2007)
http://www.washingtonpost.com/wp-dyn/content/article/2007/02/13/AR2007021300893.html
What Makes a Teacher Effective? (February 14, 2007)
http://www.usatoday.com/news/education/2007-02-13-effective-teachers_x.htm
Reports
Beyond NCLB: Fulfilling the Promise to Our Nation’s Children (February 2007)
http://www.aspeninstitute.org/site/c.huLWJeMRKpH/b.938015/k.40DA/Commission_on_No_Child_Left_Behind.htm
In February 2006, 15 leaders in education came together to form the Commission on No Child Left Behind (NCLB), a bipartisan, independent effort dedicated to improving NCLB. The Commission traveled across the country, listening to the experiences of students, educators, parents, administrators, state and district officials, experts and policymakers. Their work uncovered shortcomings in both the implementation of the statute and in some tenets of the law itself. They believe that to do better, the law must be dramatically improved, and this report outlines specific and actionable recommendations for establishing a high-achieving education system. One of their recommendations is to require all teachers to be Highly Qualified Effective Teachers (HQET)—teachers who demonstrate effectiveness in the classroom. Under HQET, states would be required to put in place systems for measuring the learning gains of a teacher’s students through a “value-added” methodology, using three years of student achievement data, as well as principal evaluations or teacher peer reviews. The new HQET measure will, for the first time, trigger guaranteed, quality professional development for teachers who need it most. Those who are not initially successful in producing measurable learning gains in the classroom must be given access to effective professional development to help them succeed. Those who are unable to demonstrate effectiveness in the classroom after receiving support for a reasonable period of time should no longer teach those students most in need of help.
Commentary: Aligning the System: The Case for Linking Teacher Pay to Student Learning (March 2006)
http://www.cgp.upenn.edu/pdf/Aligning%20the%20System.pdf
In this commentary, the authors argue that if America is to remain a stable, middle-class society, public schools must be transformed. In today’s fiercely competitive global economy, where high wages demand advanced education and skills, the salaries of teachers can no longer be paid without regard to results in student learning. This disconnect between means and ends no longer serves teachers, students, or the nation. Over the next decade, according to the National Commission on Teaching and America’s Future, 2 million of the nation’s 3.4 million teachers are expected to leave the profession, substantially more from attrition than from retirement. If compensation differentiation is not made to permit highly talented individuals to rise through the ranks much faster, it will be impossible to stem this tide. Nor will the best and the brightest individuals be attracted to the profession or induce more teachers to take on the hardest assignments. Further, the authors argue that Statewide data from Tennessee demonstrate that while longevity drives pay increases to retirement, average instructional effectiveness plummets in the last third of teachers’ careers. It also does not make sense to pay teachers the same salary when empirical evidence shows significant variation in the quality of instruction across classrooms. Linking compensation to student learning results requires a new unionism. But given a fair system with appropriate supports, teachers should welcome linking their pay to student performance.
Education Next: Up Against the Wall (2002)
http://www.hoover.org/publications/ednext/3398961.html#
Family background decisively shapes student achievement. As a result, simplistic accountability systems that dole out rewards or sanctions based on test scores run the risk of punishing schools and teachers for problems beyond their control. One solution to this problem would be to attempt to measure the achievement gains that a school or teacher elicits by subtracting their latest test scores from the previous year’s. These gains are less susceptible to the home influences than the simple level of achievement at one point in time. Thus, by measuring gains, we can pinpoint the “value” that a school has “added” to its students’ educational experience. Value-added analysis, while a promising innovation, suffers from various statistical shortcomings. Errors in measurement could lead to schools and teachers being rewarded or sanctioned wrongfully. The question is whether the benefits of value-added analysis outweigh the flaws. Education Next explores the debate with four reports. Dale Ballou details the pitfalls in “Sizing Up Test Scores”; Anita A Summers trusts the testing experts in “Expert Measures”; Jay P. Greene looks to the practice of private firms in “The Business Model”; and Donald R. McAdams reminds us not to let the perfect be the enemy of the good in “Enemy of the Good.”
Evaluating Value-Added Models for Teacher Accountability (2003)
http://www.rand.org/pubs/monographs/2004/RAND_MG158.pdf
Value-added modeling (VAM) to estimate school and teacher effects is currently of considerable interest to researchers and policymakers. Recent reports suggest that VAM demonstrates the importance of teachers as a source of variance in student outcomes. Policymakers see VAM as a possible component of education reform through improved teacher evaluations or as part of test-based accountability. They are particularly intrigued by VAM because of the view that its complex statistical techniques can provide estimates of the effects of teachers and schools that are not distorted by the powerful effects of such noneducational factors as family background. Although VAM holds great promise, it also raises many fundamental and complex issues. Some of these issues may appear arcane, but the reasonableness of the findings of VAM studies depends on them. If these issues are not adequately addressed, VAM is likely to misjudge the effectiveness of teachers and schools and could produce incorrect generalizations about their characteristics, thus hampering systematic efforts to improve education. Unfortunately, investigation and discussion of the issues raised by the use of VAM in education have been fragmented and limited. In addition, much of the discussion is unpublished, and the practical import of these concerns when VAM is applied to student achievement remains largely unclarified. In this monograph, the authors clarify the primary questions raised by the use of VAM for measuring teacher effects, review the most important recent applications of VAM, and discuss a variety of the most important statistical and measurement issues that might affect the validity of VAM inferences. Although the document focuses on measures of teacher effectiveness, many of the points discussed here also apply to measures of school effects.
While Massachusetts has a sophisticated, even complex, state accountability system, the current system does not allow the Commonwealth to follow individual students' academic trajectory toward proficiency over time. The purpose of this paper is to propose that Massachusetts' accountability plan for schools and districts include a value-added component. To explore how better use of student achievement data could enhance the current system of accountability and school improvement, the Rennie Center convened a diverse group – representing teachers' unions, parents, school committees, superintendents, principals, and other education experts – and consulted with them over several months in preparation of this report. The group focused its attention on the measurement, over time, of student learning gains. They concluded that such a system will enhance: the state's capacity to make fair judgments about school effectiveness; teachers' capacity to provide focused learning support for students; parents' understanding of their children's academic growth in school; and administrators' and local policymakers' decisions about how to improve educational programs. This paper presents the Rennie Center's conclusions about how and why Massachusetts should supplement its current accountability system with value-added analysis.
Policymakers’ Guide to Growth Models for School Accountability: How Do Accountability Models Differ? (2005)
http://www.ccsso.org/publications/details.cfm?PublicationID=287
This paper addresses many questions education leaders may have about the differences between status models and growth models. Both status models and growth models used for school accountability are defined and described. Additionally, a type of status model (the improvement model) and a type of growth model (the value-added model) are discussed in the paper. In addition, the paper provides more specific information about current research and practices regarding the different models including purposes and structures of status and growth models for accountability with the focus on school as the unit of analysis; advantages and disadvantages of the different models in relation to purposes of an accountability system; challenges in implementation of growth models and value-added models (VAMs) and description of resource requirements; policy questions that may be addressed prior to state policymakers choosing to use a growth model for education accountability; and considerations in deciding to combine a growth model with a status model, such as the AYP reporting requirements under NCLB. Also, the paper addresses the potential use of growth models by states for school accountability and the possibility of adding a growth model to existing systems to provide additional information about educational performance of schools and groups of students.
The Promise of Value-Added Testing (November 2002)
http://www.ppionline.org/documents/Value_Added_Testing.pdf
This paper examines both the promise and the problems of value-added testing. It shows that there are still a number of issues that need to be addressed, so one should be cautious in its application. It should be used in combination with other approaches. Nevertheless, the authors argue that the potential benefits of value-added testing are significant. Using a value-added testing regime in addition to the standard one could yield at least three important benefits: 1) It would make the Elementary and Secondary Education Act more effective by providing a more accurate picture of which schools, school districts, and states are and are not making progress; 2) it would generate objective measures of teacher quality that could be used to improve teaching; and 3) it would lend itself more readily to evaluating school reform programs.
The Real Value of Teachers: If Good Teachers Matter, Why Don’t We Act Like It? (Winter 2004)
http://www2.edtrust.org/NR/rdonlyres/5704CBA6-CE12-46D0-A852-D2E2B4638885/0/Spring04.pdf
In this report, the author shows how value-added data reveals the tremendous impact teachers make on student learning. The authors show how states, districts and schools are using this information to: measure the effectiveness of teacher preparation programs; measure the effects of different models of professional development; and identify highly effective teachers using value-added data. Two primary goals can then be accomplished from these measures: an increase in the overall number of effective teachers, which includes improving the effectiveness of teachers currently in the classroom; and getting more effective teachers into the classrooms of the low-income children who rely on them the most for their learning.
Reporting School Quality in Standards-Based Accountability Systems (Spring 2001)
http://www.ecs.org/html/offsite.asp?document=http%3A%2F%2Fcresst96%2Ecse%2Eucla%2Eedu%2Fproducts%2Fnewsletters%2Fpolicybriefnl%2Epdf
As more schools move towards using performance-based standards to measure accountability, the actual percentages of students with improved learning can be difficult to interpret. This report reviews the advantages and disadvantages of using different methods for defining progress, including longitudinal, quasi-longitudinal and value-added assessment methods. In order to minimize the potential negative effects of reporting and ensure proper interpretation of results, this report suggests that states emphasize school improvement rather than current performance level, report margins of error for results, and evaluate the validity of results and trends through more than just one test score, using a variety of indicators.
Teacher Effects as a Measure of Teacher Effectiveness: Construct Validity Considerations in TVAAS (Tennessee Value-Added Assessment System) (April 2001)
http://www.ecs.org/clearinghouse/32/77/3277.pdf
Measurement experts at the University of Colorado provide a technical critique of the TVAAS as a valid measure of teacher effectiveness. The authors raise several questions about the methods used and how well they identify teacher effectiveness. A particular concern with TVAAS is that it doesn’t consider that high-achieving students are often assigned to highly effective teachers, so that the assessment of teacher effectiveness is magnified. The researchers argue that, since teaching quality in this model is defined solely in terms of raising student achievement, the model does not distinguish between good teachers and teachers who are teaching to the test.
Using Student Progress To Evaluate Teachers: A Primer on Value-Added Models (September 2005)
http://www.ets.org/Media/Research/pdf/PICVAM.pdf
The quantitative evaluation of teachers based on an analysis of the test score gains of their students is an exciting prospect that has gained many proponents in recent years. Such evaluations employ a class of statistical procedures called “value-added models” (VAMs). These models require data that track individual students’ academic growth over several years and different subjects in order to estimate the contributions that teachers make to that growth. Despite the enthusiasm these models have generated among many policymakers, several technical reviews of VAMs have revealed a number of serious concerns. Indeed, the implementation of such models and the proposed uses of the results raise a host of practical, technical, and even philosophical issues. This report is intended to serve as a layperson’s guide to those issues, aiding interested parties in their deliberations on the appropriate uses of a powerful statistical tool. It counsels caution and the need to carry out due diligence before enshrining such procedures into law, especially if high stakes are attached to the results.
The Value of Value-Added Analysis (January 2003)
http://www.nsba.org/site/docs/12000/11966.pdf
As states and districts explore new ways to assess the quality of schools and support their continuous improvement, value-added analysis of standardized test-score data has emerged as an attractive complement to traditional means of measuring school effectiveness. By factoring out non-school related influences on student achievement gains, the value-added approach lets state policy makers and local education authorities gauge the impact of education policies on student learning. Modern accountability systems that incorporate value-added analysis provide an objective means for assessing the effectiveness of teachers, structuring individualized plans for professional development, evaluating new education programs and curricula, and driving instructional improvement in the classroom. This Policy Research Brief explores the emerging role of value-added analysis in shaping these accountability systems.
Back to OSSE Ed Digest Main