The unlikely champion for testing kids around the world on empathy and creativity

Andreas Schleicher vs. robots

Andreas Schleicher is a German data scientist—tall and precise with a grey mustache and a steely gaze. The head of the education division at the Organisation for Economic Cooperation and Development (OECD), he gives off an impression of determined focus. That’s useful, considering that he’s on a mission to change the way countries around the world teach their children.

Society, according to Schleicher, is preparing for the future of work all wrong. We’re scared that human jobs will be replaced by robots. But we’re still teaching kids to think like machines.

“What we know is that the kinds of things that are easy to teach, and maybe easy to test, are precisely the kinds of things that are easy to digitize and to automate,” Schleicher said at the LearnIt conference in London in January. It’s fairly easy to teach and test math, for example—but robots happen to be pretty good at math, too. Kids can, however, imagine, create, question, and collaborate in ways that robots cannot (at least, not yet). These are the skills that Schleicher wants the world’s education systems to emphasize. ”The advent of AI should push us to think harder of what makes us human,” he said, adding that if we are not careful, the world will be educating “second-class robots and not first-class humans.”

He is far from the first person to raise the issue. Speaking at the World Economic Forum in 2018, Jack Ma, the former head of Chinese e-commerce giant Alibaba, argued that unless educators focus on teaching the skills that are uniquely human—independent thinking, teamwork, and caring for others—kids don’t stand a chance. “If we do not change the way we teach our children, in 30 years we will be in big trouble,” he said.

Schleicher is uniquely positioned to drive that change, thanks to the powerful lever he oversees: The Programme for International Student Assessment (PISA), a test now taken by 500,000 kids in 79 countries every three years, the results of which are studied by educators the way soccer fans obsess over the World Cup draw.

For more than two decades, he has fixated on what skills tests can accurately assess—and, more importantly, what skills we want to measure in the first place. Like Ma and many education reformers, he believes that the industrial model of education, which shuttles kids through an assembly line of subjects and then asks them to regurgitate the information on tests, needs to change.

His solution: Change the test, and maybe the world’s priorities will change, too.

Treasure what you measure

There’s a well-known saying in education that you treasure what you measure. Schleicher’s mission seems to be to try and invert that: To change what you treasure, change what you measure.

PISA started as an assessment of math, science and reading. But since 1997, Schleicher has been waging an increasingly vocal revolution to move the focus from a more content-oriented academic test toward a test that measures a range of skills, mindsets, and competencies, including empathy and creativity. “We have to acknowledge that PISA is, at the moment, a partial picture of what is important,” he said in an interview. “That’s a criticism I take seriously.”

In recent years, Schleicher has pushed for the design and development of new, optional tests to complement PISA. (The PISA governing board votes on which to take up.) These include tests focusing on problem-solving, collaborative problem-solving, and, this year, so-called “global competencies,” such as open-mindedness and the desire to make the world a better place. In 2021, it will tackle creative thinking, trying to find ways to assess, and have students assess, flexibility in thinking and habits of creativity such as being inquisitive and persistent.

The scope of the change he is driving is enormous: not-so-subtly nudging the world’s educational systems to move away from a single-minded focus on subjects like science, reading and math and toward more complex, interdisciplinary skills and mindsets. Some think the time for such change is long overdue; others that Schleicher has overextended both himself and the OECD.

Which is where his unflappability comes in handy. “I am not afraid of thinking in decades in these things,” he said. “If you want to achieve something really good in this space, that’s what you have to put into this.”

A brief history of the world’s biggest test

The first PISA test was given to 32 countries in 2000, and the results (pdf) were reported in 2001. Unsurprisingly, Korea and Japan—countries that place a huge cultural premium on education—performed well. The US, which has a huge, complex educational system, was in the middle. Meanwhile, Finland, a country that most global experts had never even considered, became the field’s overnight darling when it ranked in the top five countries for math and science, and took first place in reading. With no high-stakes tests, no homework, and virtually no private schools, Finnish students topped the charts. (The country’s secret sauce? Teachers who are competitively selected, get frequent training and feedback with others, and are given huge amounts of freedom.)

For the first time, the world could compare systems. Prior to PISA, data was the mostly the purview of each country. (Two international tests, the Progress in International Reading Literacy Study (PIRLS), and the Trends in International Mathematics and Science Study (TIMMs), predated PISA, but neither had its influence.) Fernando Reimers, head of the Global Education Innovation Initiative at Harvard University, said that before PISA, “We were all just in the dark.”

Schleicher loves to use the example of Brazil, which once boasted about its world-class system—until it came out at the bottom of the pack. While the OECD was shining a bright light on who was really learning, the goal, according to Schleicher, was not to shame low-performing countries, but to mobilize government leaders to care about teaching and learning as much as they cared about employment numbers or GDP.

“Why are girls reporting lower levels of life satisfaction than boys?”

PISA data have provided remarkable insights—way beyond which countries do best at math, reading, and science. For example, a well-being study quantified which countries’ students are the most motivated, the most stressed-out, and the happiest, as measured by students self-reporting life satisfaction. It could not prove any causation between well-being and achievement. Some countries did well on the academic parts of the test and were happy, like Switzerland and the Netherlands. Some countries performed well on PISA but reported being miserable (Korea). Others, like Mexico, were delightfully happy but low scorers.

Schleicher says the variation of the data was itself surprising and revealing, and pointed to a strong connection between student well-being and feeling supported by teachers. Kids in Singapore, the US, and the UK were among the most anxious, even if they were well-prepared for a test; Estonians, Finns and Swiss students, less so. Girls were consistently more anxious than boys, and wanted better grades.

Gabriela Ramos, chief of staff at the OECD, said that such gender disparities are a grave concern. “Why are girls reporting lower levels of life satisfaction than boys?” she asked. “Can we do something about it and not only be concerned about it?”

The report also tried to unpack the critical connection between students’ well-being and sense of belonging and their academic achievement. “If you feel good, you learn better,” said Ramos. “It’s not about promoting high achievement or not; it’s how you do it with the other sets of skills we need to develop in children.”

Other insights from PISA include the fact that China performs well on PISA despite having typical class sizes of more than 50 kids. This leads Schleicher to conclude that class size is irrelevant to learning: What matters most is the quality of teachers, who need to be intellectually challenged, trusted, and have room for professional growth.

Then there’s France, which scores poorly on sense of belonging: Poor, immigrant kids report among the highest levels of alienation of any participating country. France also has a problem with extremism. Schleicher is not saying there’s causation. But it’s a data point to consider.

Ideally, the PISA data can be used to shape policy decisions. McKinsey has applied machine learning to PISA data to explore whether students learn better in classrooms that are teacher-directed or student-led. And the OECD has used PISA data to explore which kinds of systems deliver the best educational outcomes for the world’s poorest students.

The evolution of testing

In 1997, Schleicher convened a group, led by Switzerland and with input from other OECD countries, to discuss not what could be tested for, but what should be tested for. Typically for the OECD, the project had a very boring name: Definition and Selection of Competencies (DeSeCo).

The idea was to move beyond thinking about education in OECD terms—as a means to improve productivity and minimize unemployment—toward thinking about it more broadly, as a way to make societies more cohesive, democratic, and directed toward minimizing inequality. PISA as it stood was good and fine, but “students’ success in life depends on a much wider range of competencies,” the report (pdf) noted when it eventually came out in 2003.

The group identified three areas to explore: relational, or how we get along with others; self, how we regulate our emotions and motivate ourselves, and content, what schools need to teach. The report read:

“The development and maintenance of human and social capital represents an important factor for societies to not only generate prosperity, social cohesion, and peace, but first and foremost to manage the challenges and tensions of an increasingly interdependent, changing, and conflictual world.”

The tests evolved accordingly. To some reformers, the addition of new testing areas signals a welcome focus on critical mindsets and knowledge—the so-called 21st century skills many tout today. When Giancarlo Brotto, a global education strategist for Smarttech.com, said at LearnIt that his 10-year goal for his kid’s learning was, “I want my son to come home complaining about his empathy mark,” the crowd cheered.

“I want my son to come home complaining about his empathy mark.”

Douglas Archibald is the director of Whole Education, a network of schools which focuses on trying to help kids succeed academically, but also develop a broader set of  skills and qualities. He credits Schleicher for dramatically elevating the discourse about what the future of learning should look like. But he is doubtful that the scope of change Schleicher is now tackling is feasible at the level it matters most: across larger education systems.

“There is no one else bringing together the people in charge of these education systems to seriously think about how their systems are future-proofed,” including the question of which skills and mindsets we need most, he said. But having spent years been inside such systems, Archibald says he is skeptical. “The actual trying-to-make-that-change is hard.”

The countries that are succeeding in developing skills like empathy and communication—Singapore, Finland, certain Canadian states—have relatively small educational systems, Archibald said. But scale creates new challenges. In England, for example, the school inspection body (OFSTED) has acknowledged that developing creativity and resilience has been difficult due to pressure from the government for schools and students to report good tests results.

But Schleicher is unfazed by the scope of the challenge. “I have been told so many times this is completely crazy,” he said, “that this could never be done.”

An army of critics 

Many welcome the change in PISA’s priorities. ”It is a great thing for OECD to help focus the mind of policy makers on educating the whole child and not just the academic child,” said Jim Knight, a former UK education minister. “If we continue with our fetishization of solely the academic, we won’t be equipping our economy with the talent it needs for the future range of occupations that create value in the future.”

But as PISA has gained prominence, so has the criticism surrounding it. Many say that it’s led countries to teach to the test, with education ministers scrambling to best one another rather than thoughtfully considering how to improve their systems. Some criticize its sampling techniques, including how students in Shanghai are accounted for. Other high-profile critics lambast the OECD for partnering with for-profit education companies like Pearson, which reap the rewards of more and more tests for a range of subjects.

Some critics are also questioning whether the OECD has gone too far in its quest to create shiny new tests every few years. In 2014, more than 100 academics around the world called for a moratorium on PISA testing, citing just the problem Schleicher claims to be trying to fix: An over-reliance on testing and a tendency to recommend Band-Aid solutions for complex problems.  They wrote:

“The new Pisa regime, with its continuous cycle of global testing, harms our children and impoverishes our classrooms, as it inevitably involves more and longer batteries of multiple-choice testing, more scripted ‘vendor’-made lessons, and less autonomy for teachers.”

There’s also the question of whether PISA is simply overextended. Mark Schneider, director of the US Department of Education’s Institute of Education Science, said the organization should focus on keeping up with changes in how math, reading and science are assessed—not wade so far into things like global competencies. “They should be tending to their knitting,” he said, “making sure the test they have is evolving to take into account new ideas and concepts” in its core testing subjects, rather than expanding into so many “innovative” domains (the OECD’s response is here).

“If you do not have a north star, perhaps you limit your vision.”

Tom Loveless from the Brookings Institute agrees that PISA is trying to do too much. “Policy relevance has become PISA’s overarching objective, which has led to excessive and unfounded policy recommendations,” he wrote in 2014.

Since PISA’s tests are optional, it’s also unclear whether the newer tests focusing on skills beyond academic ones will have much impact. The US, for example, opted not to take the “global competencies assessment,” along with a whopping 40 out of 79 countries.

But Schleicher remains set on the idea that only by setting goals around things like independent thinking can countries make a plan to equip students with the skills they need. “If you do not have a north star, perhaps you limit your vision,” he said.

Building a better test

Schleicher claims that he always meant for the test to be about what mattered—motivation, social, and emotional learning. But when PISA started, countries would have never bought into such a test, and the metrics did not exist.

Both have changed.

The realization that kids are incredibly stressed-out has taken hold. The science of learning has uncovered how we learn—the critical role of knowledge and content, but also our need for meaning and agency, to have a say in what we learn and how we go about it. How safe a child feels also impacts how they learn, as does how you discipline them, pronounce their name, and make assumptions about their performance.

At the same time, technology—including AI—is making it possible to measure far more than what a child knows about math or science. “AI can be so powerful,” Schleicher said. “It can actually help us diagnose student learning—personalizing learning—making it more engaging.”

For example, technology enables testers to see not just whether students get a problem right or wrong, but how they solved it. This will give the OECD insights about how people learn, which is more powerful information than simply finding out what students know. “That’s the kind of information that’s going to help students learn better and teachers teach better and schools to become more effective,” Schleicher said. He offered the example of one classroom he saw in China. Children wrote Chinese characters on a scanner, the results of which were interpreted by an iPad by the child’s side, giving immediate feedback. The teacher received all the data on a dashboard and could see which kids were taking longer than others or struggling with the task.

“It’s becoming easier to measure the previously unmeasurable because technology allows us to capture a richer array of data,” said Knight, the former UK education minister. “We can capture by observation how we think people are doing—how peers think we are doing—to create a richer data set.”

The question is whether testing for these things creates unintended consequences. PISA did. As Pasi Sahlberg, a widely respected Finnish education expert explains in the Washington Post : “The test has had a towering influence on national educational policies. Many countries whose performance has fallen have gone into ‘PISA shock,’ and shifted direction dramatically to raise their standings and their standards.”

“We know expectations about achievement influences achievement. What happens when we have expectations about being?”

To some, the idea of policymakers going into shock about student well-being sounds like a welcome change. But it can also feel a bit like science fiction in the classroom, with student data used to track not just who is failing to meet academic standards, but social and emotional and values-laden ones, too. We know expectations about academics influences achievement. What happens when we have expectations about being? It’s not hard to picture a world in which wealthy parents hire tutors to help kids master empathy, and competitive students jockey to be top-ranked creatives.

What’s clear for now is that the OECD, at the hands of Schleicher, has amassed a treasure trove of data that can be used, like all data, for good or for ill.

Schleicher is currently knee-deep in Education 2030, a project to “help countries find answers to what knowledge, skills, attitudes and values are needed for today’s students to thrive and shape their world, as well as how instructional systems can effectively develop them.” He’s continuing to shape the conversation about what education should be while also elevating the need to have the conversation in the first place. That is incontrovertibly a good thing. Everyone should care where education is going and be involved in how it changes, including parents and students themselves. Indeed, a recent addition to the mix has been student views: as part of the effort to define what education should look like in 2030, students offered feedback on what they want their own education to look like.

Schleicher said that when the students first turned up at the very conservative offices at the OECD, people questioned what kids were doing in the building. Then the Education 2030 group, made up of academics and policy makers from the 36 member countries as well as Brazil, started to listen—and realized that kids had a lot to say about their own education.

“If we want to build agency, we have to use and rely on the agency of children,” he said.

Empowering the uniquely human

Over the past 20 years, and particularly the last 10, the case for reforming education has exploded. Neuroscience has shown us how malleable brains are, particularly in early childhood and our teenage years, suggesting there are crucial windows of opportunity for learning.

These discoveries are coming at the same time that political polarization is on the rise, with right-wing nationalist groups gaining popularity in the US and Europe, Brexit in Britain, and the election of Donald Trump. In this climate, a focus on values and communication feels both welcome and necessary.

For Schleicher, that is precisely why we need to measure things like values and empathy—not just how much we know about photosynthesis. People have the “capacity to imagine and build things of intrinsic positive worth,” he says. Unlike robots, they can manage between black and white, integrate knowledge, and apply it situationally. But the thing that’s most unique about humans, he says, is “our capacity to take responsibility, to mobilize our cognitive, and social and emotional resources to do something that is of benefit to society.”

Can we design a test for that? Not surprisingly, Schleicher says yes. “Assessment can narrow your perspective or broaden it. It depends on how you use it.”

Knight concurs—if we think about testing more broadly. “In order to successfully develop the kinds of things [Schleicher] is talking about, we have to move away from thinking the end game for everything in school is a test,” he said. Schleicher said who knows what the future will hold: assessment by 3D camera, or tools and techniques from the arts, such as assessment by exhibition or observation.

Archibald likens the evolution of the test to a pilot’s dashboard, which currently has a few dials: academic standards, as well as a few social and emotional ones. The future Schleicher wants involves a way more complicated dashboard. “There’s a whole lot of dials and they are very subjective and quite complex to develop and assess at a system level,” he says. Teachers will need to know how to use those dials to help students, parents will have to be convinced of the need, and schools heads and politicians will need buy in.

Schleicher believes it’s worth trying. “We still have an education system that is a relic of the industrial age,” he said. He can readily cite places where he sees change already underway: Japan, he says, changed its rote based learning system by removing 30% of the content in its curriculum to encourage deeper learning and added a greater emphasis on creativity.

“You might think a country like the UK values creativity, and Japan is more compliance based, but reality looks the reverse,” he said. “Memorization is more dominant in UK classrooms than many classrooms in Asia.”

Schleicher says the world is already changing education for the better. But he is fairly pessimistic about the pace and breadth of that change.

“This is a great moment to make education a more goal-oriented activity,” he said. “Do I see that at scale? I’m not sure.”

“The faster the world changes, the more people cling to what they know best.”