OECD tests

How socioeconomic background makes a difference in education outcomes

Educational disadvantage is a significant factor in students’ educational outcomes. In Australia there is a staggering level of inequality between outcomes for students from high socioeconomic background and those from low socioeconomic background. Even attending a school with a high or low average socioeconomic background can make a difference to how a student will perform educationally.

So we know socioeconomic background makes a difference. I am interested in how and why it makes a difference.

The OECD sees educational disadvantage as a lack of access to quality education and a lack of positive environment for learning experiences at school and at home. In Australia it can result in gaps of approximately three years of schooling. We hear about these gaps when mainstream media, usually with a sensationalist spin, publish the results of national or international standardised tests, such as PISA, the OECD Programme for International Student Assessment.

Such media coverage is always simplistic and is not much help if we want to understand the ways in which socioeconomic differences result in different educational outcomes of reading, maths and science literacy.

PISA’s attempts to explain the how and why

PISA publishes its PISA context assessment framework to supplement its regular international PISA testing of reading, maths and science.  The idea is to help us understand students’ background, home and schooling contexts and how these contexts relate to students’ PISA test scores.

This framework includes students’ backgrounds, processes at schools, students’ motivation, interests and beliefs, career aspirations, general attitudes and behaviours, and their dispositions to problem solving and collaborative learning.

Although the framework does try to fill some gaps of information for us, these are just snapshots rather than an analysis of the impact of students’ background characteristics on their participation in these processes, or whether the educational system, schooling processes and classroom practices may favour certain groups over others.

I believe they do not capture how and why these contextual aspects lead to different engagement and performance in school nor the students’ developmental processes underlying the PISA test scores. In other words, they do not help to shed light on how and why some students perform better than others.

As Indian economist and philosopher Amartya Sen put it, knowing the “inequality of what” is important to improve equality.  We should shift our focus from measuring schooling contexts and processes, with the assumptions that they are positive for all students, to understanding how these aspects shape students’ opportunities and participation in school.

In order to truly understand what is happening with inequality I believe we have to recognise the implicit social relationships and social structures in the schooling processes that position students in different vantage points.  I have taken an analytical approach to look at what can be done at the student and school level.

I want to take you there, to look at what the PISA team is saying and to add my own comments and ideas.

What PISA says about socioeconomic background and my reactions

What Pisa says about students’ family background

PISA 2015 says that students’ socioeconomic background contributes to positive academic performance. More educated parents are able to provide

  • a richer set of learning opportunities at home
  • more access to written materials for reading and other resources that engage their child’s curiosity
  • engagement in discussions and cultural experiences at home which contribute to their children’s PISA reading achievement
  •  high expectations for their children’s academic performance and interest in their schoolwork which lead to parental participation in school and
  • additional tuition for their children out of school.

My reaction

These measures suggest that economic capital provides material resources that give students the means to achieve educational achievements. Associated with economic capital is familial capital, that is, parents’ interest and expectations shape students’ attitudes and aspirations in ways that align with schools’ interest and expectations.

I believe what is important here is this alignment of values and expectations between the school, parents and the students that enable students to take part effectively in the schooling processes. It is about being accustomed to similar communication and learning cultures at home.  So it is less about the schooling environment, and more about students finding the school environment to be an extension of home life experience and thus they are able to align with the school norms.

When principals, teachers, parents and students value learning and learning practices in similar ways, there appears to be more parental interactions with schools. However parental contact is not always an indicator of those shared values about learning.

In fact, PISA also found that across education systems globally, more parents from lower socioeconomic schools participate in more school-related activities than parents of children who attend advantaged schools, and performance of students tend to be lower for those that attend schools with higher level of parental contact with schools.

This means that we must look at the school context and interactions between parents and schools to understand the nature of engagement and effects on student participation.  

What PISA says about student ethnicity

PISA 2015 reports that students from ethnic backgrounds on average perform worse than those with English as first language. However, students from ethnic background who are in top quartile of socioeconomic status performed better than their counterparts whose first language is English.

PISA also concludes that educational aspirations correlate with career aspiration and vary between different ethnic groups with students in the higher socioeconomic quartile reporting higher career aspirations.

My reaction

PISA results indicate that linguistic diversity impacts educational performance in nuanced ways. Students from ethnic backgrounds have linguistic capital that can be a resource for learning. The extent that they can mobilise this resource in schools depends on the linguistic skills and knowledge of teachers, other students, and whether the curriculum and teaching practices promote linguistic diversity.

If students are not able to share their linguistic skills or if these skills are not appreciated, they can encounter barriers in the classroom, particularly if teachers do not have adaptive teaching skills required to deal with comprehension difficulties and likely cultural differences.

Ethnicity intersects with socioeconomic background so it is too simplistic to suggest that students from ethnic backgrounds will not do well in schools. The issue should not be ethnicity itself but the contexts in which ethnicity enables or limits students’ opportunities and participation in schooling practices.

Recognising that educational performance is not symptomatic of students’ ethnic or socioeconomic backgrounds in singularity is important. Learning outcomes vary as students try to mobilise their linguistic capital within the classrooms and school communities.

The effects of linguistic capital also amplify or alleviate impacts of economic capital. If some students are more aligned with certain ways of teaching and learning and that is what the school prefers and expects, then those students are favoured over those that may not have such linguistic or literacy dispositions.

Educational inequality actually arises from teachers and schools’ lack of recognition of students’ diverse linguistic dispositions in their teaching practices and implementation of the curriculum, even if they do so with good intention.

What PISA says about policies to improve educational inequality

PISA 2015 has called for Australian policy makers to address students and schools with lower socioeconomic background to improve their educational performance. The policy debate tends to revolve around issues of school funding to improve access and participation for students from disadvantaged backgrounds.

My reaction

Providing additional economic resources is important but may not always reduce educational inequality. This is because educational inequalities that appear in socioeconomic or cultural differences actually carry broader social and cultural processes into educational systems, schools and classrooms. Students, parents, teachers and principals are placed in and have to operate within these processes.

While the PISA context assessment framework recognises that socioeconomic and schooling contexts impact learning outcomes, its snapshot measurements of these contexts do not shed light on how and why these contexts impact teachers and students. and in turn unequal student access to and participation in schools.

We need to delve beneath PISA’s proxies for contexts to understand how students engage with teachers and their peers and how their own individual characteristics or family upbringing may lead to positive or negative relationships within these interactions. For example, if the school recognises students’ linguistic and cultural diversity and permits their representation in the learning curriculum or other school activities, these contexts can promote collaboration between students and teachers, school leaders and teachers and parents and school.

Understanding how student-teacher and family-school relations shape different educational values and appreciation for certain teaching practices is important. For example, while PISA finds that inquiry-based learning is positive for learning outcomes, it does not explain how and why, for whom and in which situations this mode of teaching is effective. Such a linear assumption about teaching and learning does not account for the marginalisation of those who might not have the disposition for this type of learning environment.

Reducing inequality needs more than just access to resources

Thus, while PISA points to the need to address inequality by addressing economic resources, I believe there is a clear case to go beyond this.  We need to deeply understand students’ “real” opportunities within our systems of education. I believe we need to look more closely at what students can reasonably do (or not do) with those resources given their backgrounds and situations.

Resources are important, but just because a school has a wide variety of resources doesn’t mean all of its students will benefit from those equally.

I am arguing that policy attention to improve educational inequality should place student agency and diversity at the forefront, rather than focussing on resources with the assumption that all students will be able to access them in similar ways with similar outcomes.

More in my paper Capital and capabilities in education: Re-examining Australia’s 2015 PISA performance and context assessment framework

Lien Pham is a Lecturer in the Graduate Research School, University of Technology Sydney. Her research interests are international education and development, political participation in non-democracies, language and identity, and Vietnam studies. She has collaborated in research projects about political participation in non-democracies, and international education practices in Australia. She has also consulted for various NSW government agencies in public policies research and evaluations, and multilateral organisations including UNESCO Bangkok on educational policy reforms. Lien can be found on Twitter @LienPha42919006

Why condemning international tests is a distraction, and what we really should be worried about

Governments all around the world seem to be influenced by the international rankings of students by the Organisation for Economic Co-operation and Development (OECD). Australia is no different. Famously, in 2012 our then prime minister, Julia Gillard, set a goal for our students to rank in the world’s “top 5 by 2025”

Increasingly, educators have been questioning the validity of these rankings and asking why policy makers pay so much attention to them. One critique getting a lot of attention is by Stanford University’s Professor Martin Carnoy who concluded recently the OECD’s Program for International Student Assessment (PISA) rankings are misleading.

According to Carnoy they are misleading because of (1) differences between countries in students’ access to resources (2) the validity of using PISA results for international comparisons, (3) larger than acknowledged test score errors and (4) the Shanghai educational system scores, held up as a model for the rest of the world, are based on a subset of students that is not representative of the Shanghai student population as a whole. Carnoy was speaking specifically about the PISA results and the US, but his opinions would also be valid for Australia.

Tests do distort policy and practice. Most educators, and many parents, these days understand that. There is no evidence that standardised tests, or any league tables generated as a result, improve student achievement!

But there are other lessons aimed at the US coming from PISA rankings that perhaps the US, and other countries such as Australia, should be paying attention to. How many times does it have to be pointed out that “socio-economic disadvantage has a notable impact on student performance”? Also PISA 2009 asserted local funding of education exacerbates inequality and “may be the single most important factor for the US.” But it seems impossible for Americans to come to grips with these findings.

Meanwhile here in Australia the divide in schooling is growing. The too widespread entrenched inequality of educational outcomes and opportunities are further exacerbated by economic trends. Tom Bentley and Ciannon Cazaley from Victoria University’s Mitchell Institute say “As a result, there is a mismatch between the learning needs of students and schools, and the current capabilities of education systems”.

I contend that, while I agree there is much about OECD rankings that are wrong, condemning these international tests is a distraction.

Yes there are problems with PISA

Of the four reasons given by Carnoy as to why the PISA scores are misleading the first two, about different access and validity, can be simply agreed with.

But PISA Reports are not just analyses of test scores! Conclusions drawn from test data and information supplied by school principals agree with many other reports. Using test scores to construct league tables frequently ignores variances attached to the means and media analyses and political responses often show less than adequate understanding of basic statistics or of the education literature. Carnoy also details the unreliable way in which the Shanghai sample was constructed by PISA.

The important point is this: were the results from Shanghai, Japan or Singapore to be ignored, the conclusions which can be reasonably drawn from the remaining data are still a challenge to the practices adopted by US and other countries participating in PISA.

Indeed ignoring international tests altogether would still leave the conclusions from other studies which show the positive contribution to school student achievement by school leadership which supports teacher quality, demanding curriculum, rich experiences, attention to those having difficulty, socioeconomic background and diversity of the class and community relations.

Carnoy recommends changes in the interpretation of results of international tests: relating changes over time to family academic resource (FAR) factors. As well micro-data made available to allow further analysis, independent academics appointed to PISA’s decision-making board. And last, policymakers should focus more on differences in achievement gains between the US states, rather than between the US and other countries.

 

Bad education reform in the US is not the result of reacting to OECD rankings

The practices of educational administrators, politicians and corporate donors to education reform in the US are not based on international test scores and analyses. They are derived from ideological beliefs about “the market” and how people behave. The private sector is considered good, public sector bad, government intrusion into the community is excessive and inhibits self-reliance. The beliefs are financial incentives drive superior performance, competition improves quality and choice (of school) is a democratic right.

In this scenario of course teachers’ unions are seen as very bad and that their influence needs to be removed. Rich donors, whose donations cost public money because of tax offsets (as pointed out by Diane Ravitch in her comprehensive criticism of US education policy, “The Death and Life of the American School System”) relentlessly pursue that objective through campaigns for elections to school boards.

None of these beliefs are based on valid research any more than is climate change denialism or Bishop Usher’s views on the origin of the Earth. To blame PISA is at best a distraction.

The main response by US authorities in recent times to assertions about student achievement has been the creation of more charter schools, independently run though publicly, and privately, financed. Recently the Common Core State Standards Initiative which makes testing easier has dominated reform. All as part of “Race to the Top” endorsed by President Obama and overseen by Education Secretary Arne Duncan. The failure of charter schools has been well demonstrated, David Zyngier describing them as a flawed idea and there is a revolt against Common Core Standards led by Diane Ravitch.

In most countries educational performance by students attending independent schools is due mainly to student selection and retention. When controlled for socioeconomic factors any superior performance compared with students from public schools disappears. More money spent on those schools delivers relatively little extra gain. The very important University of Queensland study by Son Nghiem and colleagues revealed that what mattered was a suite of factors relating to the home environment.

 

Inequality and education

Carnoy admits the importance of the home environment and the resources made available by parents. He points out education analysts in the US “pay close attention to the level and trends of test scores and their relationship with socioeconomic factors, that is inequality”. Indeed they do: David Berliner, in his Letter to the President, Diane Ravitch, Jean Anyon and many others have shown policies and practices in the US do not address poverty’s impact: US practices aren’t like those of successful countries.

Finnish teachers would not stay long in US schools according to Finnish expert Pasi Sahlberg in “What if Finland’s great teachers taught in U.S. schools?”.

 

Education Achievement and Economic Performance

Carnoy points to the study by the Hoover Institution’s Eric Hanushek and others which argued that the average national math test scores are the single best predictor of national economic growth, at least for the years 1960-2010. The poor US student test scores therefor threaten US economic superiority!

The seeming contradiction arising from the economic success of the US (as measured by GDP and similar metrics) and Nobel prizes seem to deny this prospect. Diane Ravitch argued there was no relationship between test scores and economic productivity. Researcher and writer on education reform Norman Eng pointed out that learning in school was narrow, detached and contrived whereas work, especially highly skilled jobs, was active, cross disciplinary and “out-of-the-box”.

There is indeed a poor relationship between test scores and wages, and rewarding teachers on the basis of their student’s scores likely misdirects resources according to Nobel prize-winner James Heckman and colleagues.

But there is more. Michael Teitelbaum’s said his research shows claims of looming problems due to a shortage of quality graduates in STEM subjects is a myth. US high tech companies employ large numbers of foreign students with US earned PhDs on H-1 visas who tend to be locked into jobs at lower salaries. They aren’t smarter than available US workers and do not bring talents not otherwise available. They are just cheaper! They register fewer patents and their PhDs are from less prestigious universities.

In any event, as reviews by Simon Marginson, now of University College London, and Director of American Studies at Columbia University Andrew Delbanco, have recently pointed out, too often universities do not enrol students on merit but choose them from more prestigious colleges: access is decreasing as quality of teaching declines and fees increase. Fewer graduates means higher salaries and widening economic divergence.

 

Putting PISA in its place

International programs like PISA are a snapshot of achievement in tests by a sample of 15 year olds to questions about their comprehension of writing, reading, mathematics and science. Why not accept them for that! Americans obsess over their ranking in the world and use any excuse to deny the contrary: consider health care. Just as well they don’t play Rugby Union.

Advantaged families in many countries also obsess over test scores achieved by the schools their children attend or might attend and spend huge sums of money moving house to be near “better” schools. They devote time to pushing their children to complete excessive amounts of homework and engage in so much other activity that they have little time for simply developing the ability to form relationships with others.

Standardised tests pander to the myth of accountability. They diminish the enhancement of creativity and give undue weight to a few subjects notwithstanding the importance of those. But privileging literacy and numeracy marginalises other skills. Large numbers of people who had difficulty with basic subjects or even the school experience in general have gone on to be successful in science, the arts and other domains. Some were dyslexic and some autistic, but often were discriminated against at school.

 

Here’s what we really should be worried about

The US is a country where education researchers contribute high-quality research, which most US policy makers ignore. What is also ignored is early childhood, a time of far greater importance than school years, a time when inequality has most impact on families and therefore on children. The implications continue to be ignored in the US, to an extent in Australia.

In Australia, privileging independent schools wastes taxpayer money, disadvantaged children in poor suburbs and country areas suffer poorly supported teachers and inadequate resources. So do Indigenous children who in the Northern Territory are subject to Direct Instruction, their own languages are marginalised and boarding schools are pitched as the solution. Inequality increases.

Meanwhile Prime Minister Turnbull wonders whether we can afford to fully implement the Gonski reforms whilst asserting that Australia is a fair country, and Opposition leader Shorten notes weekend penalty rates mean parents can afford to send their children to private schools!

 

des180907Des Griffin is Gerard Krefft Fellow at the Australian Museum, Sydney where he was director from 1976 to 1998. He graduated from Victoria University of Wellington and the University of Tasmania in marine biology. He is interested in museums and arts organisations, the environment and science, organisational dynamics, especially leadership and governance and in education. He was founding president of Museums Australia, the single association representing museum people from 1993 to 1996. He was appointed a Member of the Order of Australia (AM) in 1990 and elected a Fellow of the Royal Society of New South Wales in June 2014. He writes at www.desgriffin.com.

Des is the author of Education Reform: The Unwinding of Intelligence and Creativity (Springer, 2014). The book and individual chapters can be downloaded from the Springer site which contains abstracts of the chapters. The book can be purchased from the site or from booksellers such as Fishpond.