RMIT University

Love this: Creativity Can Be Measured – in Diverse Ways. What we can learn from PISA

The latest on PISA Creative Thinking results:

Kylie Murphy: PISA results show thinking can be cultivated. Australian teachers are doing that better than most others

Dan Harris (part one): Fourth in the whole world but the government doesn’t care

Now, read on!

The good news is that the just-released PISA Creative Thinking test reveals new ways of assessing creativity. Now we must decided how Australia might apply these methods and findings.

What PISA did differently: two important methodological testing innovations

1. The assessment includes new, interactive item-types based on a visual design tool. For the first time in PISA, some items required students to produce a visual artefact, rather than construct a written response or choose the correct answer.

2. The assessment only includes open-ended tasks with no single solution but multiple correct responses. That demands more complex scoring methods, based on rubrics and sample responses. The collection and analysis of responses of many students around the world informs those rubrics and responses.  

This is good news for educators who believe in the dangers of a ‘single right answer’ approach to learning and assessment. Getting away from the exclusive use of written numerical and/or narrative responses and using visual design tools is also a great step forward. The test’s attention to the power of creative thinking to address complex social problems is also a welcome focus of the assessment and its results.

Fig 1: PISA Creative Thinking test domains

Student beliefs

This important data set aligns with international best practice creativity research. It reflects longstanding reports by both students and teachers that any area of enquiry can benefit from creative approaches and creative risk-taking. PISA’s test reports that “around 8 out of 10 students (OECD average) believe that it is possible to be creative in nearly any subject”,  But many students did not hold positive beliefs about their own creativity or ability to improve. This seemingly contradictory finding is consistent with my own research in which perceptions of creativity in compulsory education are expanding beyond the arts and into all subject areas, but self-confidence continues to lag. .

School environment

PISA’s focus on the whole-school environment is encouragingwelcome, but in stark contrast to the other areas of enquiry, this one is minimal and draws on limited data to make incomplete recommendations. It says, in part:  

·            “Classroom pedagogies can make a difference. Across OECD countries, between 60-70% of students reported that their teachers value their creativity, that they encourage them to come up with original answers, and that they are given a chance to express their ideas in school.” Australia has long produced robust, empirical data specific to our national context that advocates as much, but has not been taken up by policy makers.

·            “Participating in school activities such as art, drama, creative writing or programming classes regularly (once a week) is associated with better performance in creative thinking than doing so infrequently or every day.” I’m mystified as to why arts activities (note that they stop short of advocating for direct instruction of arts skills, but that’s another battle) are associated with improved creative thinking at weekly rather than daily engagement. Or, for that matter, why such measurements are helpful. Are Education Minister Jason Clare and his advisors really debating whether to mandate weekly versus daily arts activities? His three 2023 education reviews hardly seem to suggest as much.

Lastly, PISA recommends that improving creative thinking across whole school systems “consistently and effectively…requires educators, curriculum developers and assessment designers to have a shared understanding of what creative thinking is, how students can develop creative thinking skills, and how their progress can be measured”.

Again, our calls for this detailed work to effectively underpin our National Curriculum’s Creative Thinking General Capability have gone unanswered – or more recently answered with a mandate for direct instruction and phonics. 

Will the weight of the international PISA recommendations effect greater change?

Key Insights but will they become Key Actions?

In summing up, the report links high performance in creative thinking with performance in the PISA core domains. Yet some countries and economies performed relatively better than expected in creative thinking, given their students’ mathematics, science and reading performance. Students in Australia, Canada, Finland, and New Zealand demonstrated a “large overall relative strength in creative thinking together with high mean performance.” These findings certainly don’t seem to concur with the current government narrative about Australia’s international test scores falling. Despite an overall decline since the tests began in 2000, Australia’s PISA scores in maths, science and reading have remained about the same since 2017, a part of the narrative always left out. Surely creativity is a far more important workplace skill now than it was 24 years ago, a fact similarly left out of current debates. 

According to the PISA summary, high-performing systems in creative thinking have often implemented at least two of the following four concrete approaches to supporting the development of creativity and creative thinking in education:

1. Embedding creativity and/or creative thinking throughout the curriculum.

2. Supporting educators to recognise, develop and evaluate creative thinking by defining learning progressions or rubrics. 

3. Creating opportunities in the curriculum for students to engage in creative and/or interdisciplinary work. 

4. Encouraging accountability through monitoring and evaluation. 

We need a more joined-up approach

While Australia can boast a Creative Thinking General Capability in our National Curriculum, it is hardly present ‘throughout’ the curriculum as point number one above recommends. Points 2-4 offer clear ways of improving how we capacitate teachers and students for creativity. My own research has long advocated for a more joined-up, creative ecological approach to fostering creativity in schools, one which takes into consideration place, people, processes, product, and policies (both internal and external). The PISA recommendations limit its systems approach to curriculum, pedagogy and assessments – in other words, only products and processes. This leaves a long way to go toward integrating subjects, transforming places and spaces for the digital revolution and crucial person-to-person collaborations, as well as recognising the centrality of place to learning, as our First Nations colleagues have long advised us to do.

Still confused about creativity?

Finally: “Students in many countries/economies report that they do not find learning or engaging in creative work at school particularly enjoyable.” Might this be because teachers – and students – are still largely confused about what creativity and creative thinking are? Are teachers and students confused about how to ‘do’ creativity in school contexts? I doubt students would say the same about creativity on TikTok, or decorating their room, or sports participation, where creativity and its learning potential are inherent. Teachers have long been calling for more help on assessing creativity, but students in these PISA results are calling for the content to be more creative. Australia’s education sector needs to evolve past the obsession with creativity assessment, and – while not perfect – the PISA test suggests some ways forward. 

The Australian Council for Educational Research manages PISA in Australia and will release a national report later in the year, with results for states and territories and additional demographic characteristics.  Beginning in 2025, though, ACER will take over the administration of the entire suite of PISA tests, a great coup for Australia. In doing so, Australia will be in a perfect position to advance PISA’s aim of “providing internationally comparable data on students’ competencies that have clear implications for education policies and pedagogies”.

Now all we need is the political will to make creativity central to that work.

Daniel X. Harris is a professor at RMIT and a leading international scholar in creativity, diversity and social change. They were most recently an Australian Research Council Future Fellow, DECRA Fellow, RMIT Vice Chancellor’s Primary Research Fellow, and are currently research professor in the School of Education, RMIT University, and Director of Creative Agency research lab: www.creativeresearchhub.com.

Fourth in the whole world! Yet the government doesn’t care

Since PISA released its first creative thinking test results last week, there has been a flurry of commentary both formal and informal among educators and education researchers. 

The report, called Creative Minds, Creative Schools, ranks Australia 4th out of a total 81 participating countries, with Singapore topping the list at number 1 in all areas including literacy, numeracy and creative thinking. That’s sweet revenge for the city-state Apple co-founder Steve Wozniak once called ‘uncreative’ .

In the decade since then, Singapore has shown itself to be a leader in both direct instruction and creative innovation, a trend now making global headlines due to PISA. But is Australia listening? And will we similarly be able to pivot from the 2023 juggernaut of ‘return to phonics’ and direct instruction, toward a more nuanced approach to education that incorporates both approaches?

How is this included in the curriculum

Broad findings of the test are widely available, including yesterday’s post here by Kylie Murphy. But the findings have not yet been sufficiently unpacked in relation to the ample amount of Australia-specific empirical data and scholarship already available. There are some familiar findings here: the PISA Executive Summary definition that “indices of imagination and adventurousness, openness to intellect, curiosity, perspective taking and persistence are positively associated with creative thinking performance” is something most ‘creative skills and capacities’ lists and studies (including mine) have identified over years. 

The more pointed question remains: where and how are these indices included in the Australian Curriculum in ways that are actionable by teachers overburdened with literacy, numeracy and a constant prioritising of STEM curriculum?

What’s creativity got to do with it

The PISA Creative Thinking test results not only provide scores in a range of task types, but also correlation against scores in reading, science and mathematics skills. Together, they provide an interesting relational snapshot between what has traditionally been considered ‘core’ content for learners, and creative thinking, now a recognised 21st century skill alongside critical thinking, collaboration and communication. These assessment results show that “academic excellence is not a prerequisite for excellence in creative thinking”. This will come as no surprise to most educators. While some students excel in ‘academic’ ways of thinking and doing, not all do – a difference long documented as a poor indicator of success in work and life.

What we do know – and what PISA results reinforce –  is that test results, including creative thinking here, often correlate to socio-economic status: “Students with higher socio-economic status performed better in creative thinking, with advantaged students scoring around 9.5 points higher than their disadvantaged peers on average across the OECD.” Where is the government attention to these statistics, in the constant rhetoric about falling test scores?

Interestingly though, “the strength of the association between socio-economic status and performance is weaker in creative thinking than it is for mathematics, reading and science,” a powerful rationale for the levelling power of giving more priority to skills and capacities like creative thinking. In Australia and just five other countries, “more than 88% of students demonstrated a baseline level of creative thinking proficiency (Level 3), meaning they can think of appropriate ideas for a range of tasks and begin to suggest original ideas for familiar problems (OECD average 78%)”. That’a result Australia should be proud of and keen to build upon in both social equity respects as well as the increasingly outmoded ATAR obsession.

Different types of creative thinking tasks show different aptitudes

While the rankings show which countries scored highly overall, the test also highlighted variations in types or applications of creativity. These results show what Australian students do well, in our unique creative contexts and cultural orientations. It also provides an opportunity for us to understand how we can make the most of them. The risk, of course, is that the data are used for blunt comparison, a deficit-approach that often drives ‘moral panic’ responses around fear of ‘slipping’ in international rankings, and short-term stop-gap solutions. For the 2022 results, students in Singapore were the most successful across several task types, especially social problem-solving tasks. Students in Korea were the most successful in scientific problem-solving contexts and evaluate and improve ideas tasks. Students in Portugal performed the most successfully in visual expression tasks.

Such results offer an exciting opportunity to reflect as a national education sector on how we might aspire to raising aptitude in multiple tasks, for example, rather than simply ‘beating’ other countries in overall results.

Gender and equity gaps

The report makes a point of how comprehensively those identified as girls outperformed those identified as boys in creative thinking. “In no country or economy did boys outperform girls in creative thinking, with girls scoring 3 points higher in creative thinking on average across the OECD,” and in all type of creative tasks. 

If participating nations were to use the data to fund “Get More Boys into Creativity” campaigns, as they do with girls in STEM, the utility of a binary gender analysis would be clearer. Unfortunately, the numbers don’t carry through university and workplace trends: A recent analysis of female-identified versus male-identified creative university graduates and early-career employees does not correlate to the strong performance by female-identified 15 year olds. It shows female-identified creatives at both adult stages consistently fall behind their male-identified counterparts.

A welcome measure

Overall, the PISA Creative Thinking test results are a welcome international measure to complement the literacy, numeracy and science tests. Thus far, there has been no comment from government on Australia’s fantastic 4th in the world result – in stark contrast to the ongoing failure narrative of falling test scores. Australian students need to be well-rounded and best prepared for the jobs of the future by the end of their secondary schooling. That’s why our teacher preparation programs at RMIT University’s School of Education, I’m sure like the vast majority of other schools, ensure that all students receive training in all the basics that our new teachers and students need to excel in 21st century life, at the centre of which is creativity. 

Daniel X. Harris is a professor at RMIT and a leading international scholar in creativity, diversity and social change. They were most recently an Australian Research Council Future Fellow, DECRA Fellow, RMIT Vice Chancellor’s Primary Research Fellow, and are currently research professor in the School of Education, RMIT University, and Director of Creative Agency research lab: www.creativeresearchhub.com.

Scholarships for teaching students are great – but will they really diversify the profession now?

Australia is in the midst of a teacher shortage, and with 35% of teachers considering leaving the workforce before they reach retirement age, the problem may get worse before it gets better. This means we need to increase the number of teachers graduating from university teaching degrees. The full set of data for 2024 university applicants isn’t available yet, but UAC data suggests that applications to study teaching degrees at universities are trending downwards

One of the strategies to address the teacher shortage is the new Federal Government scholarships to encourage more people to undertake teaching degrees. While hoping to attract more people to teaching overall, the scholarships target groups under-represented in the profession, with scholarships available for First Nations peoples, people for whom English is an additional language/dialect, people with disabilities, people from regional, rural or remote locations, and people from low socio-economic backgrounds. Currently, the level of diversity in the student population in Australian schools far exceeds the diversity of the teachers, with the majority of teachers being from monolingual, White-Anglo and middle-class backgrounds, and more likely to be born in Australia than the general population.

Benefits of a diverse teacher workforce

Research also tells us that a diverse teaching population has a positive impact on student learning outcomes and engagement in schooling. Students perceive schools as more inclusive and welcoming environments when they see teachers who have similar racial and ethnic backgrounds. Based on teachers’ own experiences as culturally and linguistically diverse students, they can better understand their students’ cultural practices and beliefs and how they grow as learners. As insiders to the experiences of racism, they are valuable in the fight for educational and social justice. They make significant contributions to their school communities, due to their distinct experiences and their ability to offer students a different worldview, as well as becoming cross-cultural mentors for their mainstream colleagues.

But will these scholarships work to diversify the teaching profession?

There is no doubt that these scholarships will be attractive for some promising teacher candidates who would otherwise face greater challenges juggling study with their work, health needs and caring responsibilities. There is potential for the pool of students studying teaching to be widened because of the availability of such scholarships,  which would be a positive outcome.

However, financial support during their studies isn’t going to provide everything these students need to have a successful career in teaching. For example, our research has found that teachers from culturally, linguistically and racially diverse backgrounds (we use the acronym CLRD) experience higher levels of isolation, exclusion and racism in their workplaces. CLRD teachers can experience discrimination on the basis of skin colour, accent, dress and even food. Teachers have told us:

“At times, my faculty  would have lunch together in the staff room. It would have been nice to be told about this, even just to be polite, but it did make me feel very left out.”

“Teachers from Anglo background speak to you in a condescending way, belittle you, question your knowledge and qualifications, and there’s definitely a hierarchy where they consider themselves better than you.”

Forced to conceal their true identity

While there isn’t explicit evidence to connect these experiences to racism, every CLRD teacher who participated in our research shared a story like this. Teachers from CLRD backgrounds often feel forced to conceal their true identity to try and fit in, and it means that they’re less likely to stay in the profession and thrive in their careers.

In addition, most CLRD teachers described additional labour they were expected to undertake because of their race, language or cultural background. Some teachers were happy to do this work to help their students, but many commented that this was labour they did not see their white counterparts being asked to do.

Further, when it comes to scholarships, it’s vital that recipients successfully complete their ITE programs. Some teacher candidates from equity groups may require additional academic support from their university, and may not complete their programs without that help. Some universities do a great job of providing this support, but it takes extra resources. How students will be supported needs to be a part of the discussion.

So will these scholarships keep new teachers from leaving the profession?

The financial support may help teacher candidates from equity groups to take the leap into university studies, but it’s not a single solution to teacher retention.  Teachers on these scholarships are required to teach in public schools for a period equal to the length of their studies – two or four years. But to create a sustainable pipeline of teachers, we need them to stay longer than that, and based on our research there are other barriers that need to be addressed. Support from school leadership teams is essential, as is a united front on the part of the school, to reject racism and discrimination. Schools and leadership teams must genuinely see cultural and linguistic diversity as a positive attribute, rather than a deficit. Cash incentives during their studies isn’t going to be enough of a drawcard to stay in a harmful work environment.

From left to right: Dr Rachael Dwyer (she/her) is a Lecturer in Curriculum and Pedagogy at the University of the Sunshine Coast. Her scholarship is focused on creating social change, through decolonizing, arts-based approaches to teaching, advocacy and research, and sharing her scholarship in ways that impact policy and practice. Dr Rachael Jacobs (she/her) is a Senior Lecturer in Creative Arts Education at Western Sydney University and a former secondary school teacher. Her research interests include assessment in the arts, language acquisition through the arts and decolonised approaches to embodied learning. Professor Catherine Manathunga (she/her) is an historian who draws together expertise in historical, sociological and cultural studies research to bring an innovative perspective to educational research, particularly focusing on the higher education sector. She has worked for over 32 years in universities throughout Australia and Aotearoa New Zealand. Professor Daniel Harris (they/them) is an Australian Research Council Future Fellow in the School of Education, RMIT University, Melbourne, Australia, and Co-Director of Creative Agency research lab: www.creativeresearchhub.com. They are an international expert in creativity studies, creative methods, affect theory and autoethnography. They are committed to the power of collaborative creative practice and social justice research to inform social change.  Dr Jing Qi (she/her) is Manager of Community Languages Teacher Education Program in the School of Global, Urban and Social Sciences at RMIT. Jing draws together experiences in multilingual, transcultural, and technological studies in her current educational research projects in the areas of teacher education, international education and teacher education.