A Smart Approach to Greater Insights and Impact in Education
At the Bill & Melinda Gates Foundation, we talk a lot about data. Data allows us to measure things we care about — like how many cases of polio there are in the world this year as opposed to 10 years ago, or what percentage of students are graduating high school prepared for college-level work.
Using data, we don’t just get an accurate picture of what’s happening, we can also begin to understand what’s happening and why. That is critical, because we cannot reliably improve education until we know what programs and interventions work to improve learning and economic outcomes. That goes for policy makers, teachers and administrators, and families. We depend on researchers to bring to light insights about what works and make those efforts accessible to the public.
In that spirit, we at the Gates Foundation recently spent a day learning about some of the different approaches being used across the country to connect data systems and apply research in ways that both guarantee privacy and offer insights that can inform both policies and practices.
One example of an institution that does that well is the Tennessee Education Research Alliance, or TERA.
TERA is a research-practice partnership between the Tennessee Department of Education and Vanderbilt University, the first such statewide partnership focused on state-level K-12 education policy. The partners are working together to create an expanding body of knowledge that directly impacts Tennessee’s K-12 school improvement strategies.
Their current research priorities are to reimagine state support for teachers’ professional learning, drive improvement at low-performing schools, and strengthen Tennessee’s education labor market. That requires data — looking at items such as attendance, behavior, course-taking and course completion, assessments and graduation rates.
“We’re asking questions that are relevant to the Department of Education’s work — things that they otherwise might not have known or known what to do about,” says Erin O’Hara, executive director of TERA.
For example, TERA Senior Fellow Gary Henry and colleagues looked at two types of reform efforts in Tennessee schools, the Achievement School District and the Innovation Zone schools. Both were composed of K-12 schools whose performance was in the bottom 5 percent of schools statewide. In the Achievement School District, the state took over the schools and reconstituted them in a new district with new methods and practices. In the Innovation Zones, a different intervention was applied, with the schools staying within their school district and receiving more money and greater autonomy to work on their particular problems and turn performance around.
Henry and his colleagues found that over a period of six years, standardized test results in three subjects improved markedly in the Innovation Zone schools compared to schools in the bottom 5 percent that did not receive an intervention. But the schools in the state-run Achievement School District did not outperform peer schools. The researchers also found that the Innovation Zone schools had more consistent programming and had less teacher and principal turnover. Now the researchers are looking at the degree to which that teacher and principal turnover affected performance.
Over time, TERA hopes to examine which groups of students performed best under those circumstances. Research will be able to show, for example, the rate at which those students in the Innovation Zone schools graduated and went on to college or other postsecondary programs.
To do that, however, the researchers will need to track student performance over time and follow students’ individual journeys by accessing different databases that contain student-level data over multiple years.
Fortunately, the state of Tennessee has prioritized and invested in what is known as a longitudinal data system. Longitudinal data systems link databases across the education continuum — from preschool through education after high school and into the workforce. By linking to data maintained by the state university system, for example, TERA researchers can learn how prepared for college-level work the students were. And by connecting to workforce and income data, they can see how students’ long-term prospects were affected or improved after the intervention in the Innovation Zone schools.
Some policymakers worry that using student level data over multiple years and institutions can place student privacy at risk. But Erin O’Hara, TERA’s executive director, says that research organizations like TERA place a premium on protecting student-level data and that the types of nuanced research that her group is performing wouldn’t be possible using only aggregated data. “Just using publicly available data is not going to give you enough information about the ‘so what’ or ‘what’s next’ for this group of kids or these types of teachers,” she says. “Being able to look at students at the individual level is really critical to doing work that is more constructive.”
As importantly, TERA is rigorous about implementing data protections, she says, making sure only researchers approved by the Tennessee Department of Education can access student-level data, that the data is not personally identifiable, and that reporting on outcomes is done only at levels that don’t allow for individual identification of a given student in any way.
Making decisions about what approaches will help improve student outcomes and close achievement gaps — particularly for Black, Latinx and low-income students — requires being able to capture and apply data-driven insights. We’re well into an age where data is helping us understand problems and identify effective solutions across sectors — from “smart cities” using data to more effectively design trash collection routes to figuring out what you might like to watch on Netflix tonight. Now we must address the many barriers that impede robust education research, creating an exponential change in the use of education and economic data for greater insights and impact.
