The Promise of Next Generation Systems of Assessments

Applying multiple sources of information about students, including test scores, for varied purposes and ownership of learning

A recent article in Education Week carried this provocative headline: “Is it time to kill annual testing?” The commentary by Stephen Sawchuck, the publication’s associate editor, was yet another indicator of the rising sense within the educational community that student testing and assessment need to be rethought at a more fundamental level.

Many educators know what they oppose, but far fewer are able to specify what should take the place of large-scale standardized exams. The answer may lie in what I term a next generation system of assessments built around multiple sources of information about students that are used for multiple purposes.

Next generation systems of assessments need not consist entirely or even mostly of more test data. Instead, they will seek to gather, analyze, organize and then put into use a variety of data sources designed to gain the greater insight into students that is needed to improve instruction and to increase student self-knowledge and ownership of learning.

Many new tools and techniques are being developed that will help facilitate systems of assessments. Artificial intelligence methods to cheaply and efficiently score student writing continue to make progress. Many new item types that can be added to existing tests are being developed. Measures of social and emotional learning have been devised, field-tested and found to be effective. Classroom-based performance assessments graded with high-quality scoring guides are gaining in popularity. Student demonstrations and projects are becoming more common as are student self-reports and surveys.

Additionally, observations of student learning by teachers and administrators that yield great insight are unobtrusive and do not take away from instructional time. Information on key behaviors such as attendance and discipline help identify students in need of assistance.

As interesting and useful as any one of these measures might be, the answer lies not in focusing exclusively on any one of them but in combining, integrating and using results from this wider range of data sources.

Multiple-Measure Profiles

Districts and schools are finding ways to organize this information in the form of student profiles. A profile is a means to capture who a student is in more depth and complexity to gain greater insight into the student. Data from profiles can help educators spot students in need of attention, serve to guide the development of those making good progress and accelerate those ready to do so, and help focus instruction. When aggregated across all students, profiles can be used to track key performance indicators at the school and district levels and inform school improvement efforts as well.

The most important potential purpose of the profile, though, is to enable students to take more control over their education. In a next generation assessment world, students increasingly will use profiles to gain insight into their own strengths and areas in need of improvement, to reflect on the choices they are making and to lay out a path for their future.

The profile enables them to build self-knowledge, self-awareness and ownership of their learning. These skills are essential for success in a knowledge economy that will require individuals to self-assess and add new skills to enter and progress through a career successfully.

A profile can contain scores from standardized tests of academic content knowledge, course grades, results from classroom-based assessments, along with an array of demographic information, student self-reports and self-evaluations, parental survey responses and teacher observations of student learning skills.

Profiling Graduates

The Mastery Transcript Consortium, for example, is a group of 230 member schools that are creating a new transcript based on three core design concepts: (1) organize student results around performance areas rather than academic disciplines; (2) define mastery levels for foundational graduation requirements and advanced work; and (3) use these levels to signify mastery of specific skills, knowledge or habits of mind, as defined by each high school. The goal is to move away from grades, which can be untethered from actual student knowledge and skills, and toward better, more reliable and valid measures of what students actually know and can do. Achieving this goal will require a much wider range of learning experiences and assessments.

Fairfax County Public Schools in Virginia has developed a Portrait of a Graduate, which defines a richer, deeper set of competencies that Fairfax schools seek to cultivate in their students. POG competencies include more complex learner goals such as collaborator, creative and critical thinker, and goal-directed and resilient individual.

In addition to traditional test data, the Portrait of a Graduate may include student-generated data such as daily reflection on learning, results from student-led conferences and capstone projects that require community engagement. By contributing to the POG, students can better articulate and reflect on what they’re learning.

Andrea Hand, Fairfax school district’s manager of best practices in teaching and learning, observes that she has seen an impact in both teaching practices and student learning. “POG grows, values and affirms a diverse skill set that enhances student engagements as a ‘through line’ for learning across pre-K-12.” An elementary school resource teacher and coach in the school district noted these Portrait of a Graduate attributes “help students be able to use content or knowledge in real-life scenarios.”

Multiple measures can be a key tool for school improvement as well. Adam Carter, chief academic officer of Summit Public Schools, a national charter school operator based in Redwood City, Calif., explains how a multiple-measures approach helped his schools close the achievement gap for English language learners. Although ELLs had lower test scores, the school did not simply assume the answer was to pull them out of class for remediation. Instead, all the information on these students was compiled and analyzed. This included test scores, measures of performance and progress, and student self-reports on attitudes toward learning.

After analyzing the totality of data, educators in the Summit schools found that ELLs differed from similar lower-performing students most significantly on their mindset. As a result, the school implemented a program to change student mindset toward achievement. A year later, half of the gap between ELLs and non-ELLs had been closed at all Summit schools.

Data Dashboards

Many school districts are adopting data dashboards. These are generally used for determining progress on key performance indicators and setting improvement goals at the district and school levels.

Early warning systems, a targeted form of dashboard, are used primarily to identify and support students who are falling behind, particularly in the 9th grade. These systems combine elements of a profile such as attendance, course grades, credits earned and progress toward meeting graduation requirements to identify students not on track to graduate.

The Spokane, Wash., Public Schools created an early warning system in 2012 that combines information on a range of indicators and makes a prediction of the probability the student will graduate. Spokane’s system combines measures of absences, discipline problems and poor academic performance. Each child has a dropout risk factor calculated on the basis of these indicators. The factor is expressed as a probability, and it is updated regularly. Students with the highest probability of not graduating are identified and provided support with interventions designed to change behaviors and increase their likelihood of graduating.

Several tools are available that give educators the capability to create dashboards that include multiple measures. For example, Panorama Education has a commercial product that integrates information on academics, attendance, behavior and social-emotional learning. The Panorama Student Success system draws data from existing databases that contain grades, attendance and other district-collected information and combines these details with student self-reports on social-emotional skills such as grit, growth mindset, self-management, social awareness and self-efficacy. Additional areas where students can rate their behaviors and attitudes are learning strategies, classroom effort, social perspective taking, subject-specific self-efficacy and emotion regulation.

Implementing a System

Educators at schools and districts taking on the challenge of multiple measures and complex student profiles point to a series of key implementation issues that must be addressed to maximize the probability of success:

 DEVELOP ASSESSMENT LITERACY AND DATA ANALYTICS SKILLS AT ALL LEVELS. Educators and administrators must understand how the multiple-measure system works and how to interpret the information it generates. At least one person on each staff needs to be sufficiently versed in data analytics to help translate results to the entire staff when necessary. Principals in particular need to be able to go beyond the surface of data dashboards to conduct root cause analyses.

 BE OPEN-MINDED. Some schools have created their own assessment systems from scratch. Others have adopted frameworks designed by organizations such as the Stanford Center for Assessment, Learning and Equity, and yet others have adopted existing systems such as Advanced Placement’s Capstone Seminar program, International Baccalaureate’s Diploma Programme or Cambridge Assessment International Programmes and Qualifications and exams. Many schools have built their own social-emotional learning instruments using frameworks provided by the Collaborative for Academic, Social, and Emotional Learning, widely known as CASEL.

 AVOID HIGH-STAKES USES OF LOW-STAKES ASSESSMENT. The CORE districts in California were careful not to attach high stakes to new measures, such as school climate and social-emotional learning, early on in the process of implementing them. Over time, if the research justifies it, they may attach appropriate stakes to such information but only after obtaining buy-in from key constituents.

 DO NOT UNDERESTIMATE THE ENERGY AND COMMITMENT NEEDED TO INSTITUTIONALIZE A SYSTEM OF ASSESSMENTS. Anecdotal reports from some schools in projects especially designed to create next generation systems of assessments report fatigue and challenges sustaining implementation when leadership changes occur. This type of change requires deep implementation that extends beyond a group of true believers or innovators to all the constituents in the school community.

Moving Ahead

Building multiple-measure systems of assessments can be challenging in a time when many within and outside of schools are calling for less, not more, data on student performance. However, the future clearly lies in integrated data systems that provide insights into individual students and simultaneously facilitate continuous organizational improvement. Educators interested in devising systems that capture a range of data to inform learning and instructional improvement will want to begin by observing what their leading-edge peers are doing.

Creating data-driven schools that generate more and better information about students will be challenging. However, this can lead to a system that empowers learners and teachers and that supports continuous improvement. The result will be students who are better prepared to succeed in the dynamic, ever-changing world they enter.