University of New England

research impact

How research-based news articles (like this one) accelerate research impact

Research impact, , ,

Translating research findings into practice or policy change is notoriously slow despite the time, effort and funding invested in research. In my peer reviewed journal article and presentation about research-based news articles, I give a step-by-step guide on how to write effectively for these research news sites to create impact and accelerate knowledge translation (also called research translation).

I also argue that while our institutions benefit and encourage us to engage in such research translation, they should recognise the time it takes to write and publish for these sites in our workloads. Further, institutions need to ensure their employment and promotion systems reward the efforts required for this type of research translation and stakeholder engagement. If these systems do not keep up, institutions risk reducing the potential impact of their research as researchers juggle their time.

Why bother with news articles?

Researchers have many demands within their institutions. Any investment in time to write research-based news articles (RBNAs) needs to be justified with important reasons. Firstly, in education, the translation of research into practice has been debated for a long time, with a large lag in uptake due to poor access to research findings and the high workloads of our target educators.

Secondly, many of our stakeholders do not necessarily have access to peer-reviewed papers. They are often exhausted from supporting children’s and student’s learning. For example, in my area of wellbeing research, my stakeholders are regional, rural and remote educators, support workers and parents. They are all busy groups of people.

Thirdly, although policymakers might have access to research libraries, they are also time-poor, wading through an increasing number of peer-reviewed publications.

Fourthly, in an information-rich environment, it is difficult for researchers to cut through the noise and have their research read, understood and put into practice.

Fifthly, RBNAs allow researchers to link their peer-reviewed publications. That ensures stakeholders who want more information are able to easily access their work.

Lastly, excellent research occurs in our universities and research institutes. But it is often only partially used because it is only accessed by other academics. Translating knowledge through RBNAs is one way to reduce such waste.

Research impact: benefits of RBNAs

There are many benefits of publishing RBNAs. This format allows researchers to summarise their research into snack-sized, easily digestible articles of around 600-1200 words available to the general public. Also, professionals working in the field might use the findings to inform their practice and decisions or increase their understanding and awareness of issues impacting their work. Researchers benefit by having a wider audience engage with their research, either by reading the RBNA or clicking on hyperlinks to their other research outputs. These metrics can be tracked using Altmetrics.which can be reported in funding, job and promotion applications as proof of stakeholder engagement and community service. Additionally, the researcher’s work is more likely to be noticed by media outlets, which might request further articles or interviews. This engagement further increases stakeholder and public engagement.

Understanding how RBNAs work

In my journal article, I use a new framework. It shows how RBNAs work and how researchers adapt their skills to write them using news values. Using a fishing analogy, shown in Figure 1, I explain the differences between RBNAs and writing opinion pieces in a newspaper.

Figure 1: Framework to explore RBNAs (Source: Rogers, 2024)

RBNAs are based on your research, using the platform of your institution as an authority and vantage point. Your academic knowledge, experience and passion are used as a fisherperson uses their knowledge to hunt fish.

Your research data and project become the fishing rod, skilfully moved and positioned to create impact. Importantly, the fishing line is stretched and adapted to accommodate the fish and conditions, just as you need to expand and adapt your writing style for different news sites. These articles are not mini essays, so this requires a definite shift in your style, language and tone.

The fishing hook is the engaging and practical part of your research. This can be tricky for researchers to identify because they might find all parts of their research interesting. Most readers will not share your fascination with theories and methodology. Working with your institution’s media and communication officers can be a big help here.

The most important part of your article is the bait. How will you lure your readers to your article? The easiest way to do this is to use news values that journalists use as shown in Figure 2 . 

Figure 2: News values (adapted from Harcup & O’Neill [2017], and Parks [2019]).

How to write, publish and disseminate RBNAs

Work with your media and communication team in a professional way. Let them know about your research and identify stakeholders. They are skilful at finding news sites for your particular area. For my early childhood education and family wellbeing research, these news sites include EduResearch Matters, The Sector, EducationHQ, The Conversation, Partyline, Women’s Agenda, The Spoke, and my own institution’s UNE School of Education Research Newsletter.

When drafting an RBNA or pitching an idea to an editor, frontload a one-sentence summary of your findings and place it in the first paragraph. (As an example, scroll up to look at the second sentence of this current RBNA). The first paragraph, headline and lead image need to work together to grab the audience’s attention.

This technique differs

This writing technique is quite different to an academic article or a mystery novel. The reader does not have to wade through to the end to find the punchline. Your style will need to change depending on the news site, so read some articles from your targeted site.

Use simple, everyday English without jargon and clearly explain technical terms. Use sub-headings and images to guide the reader. Your media and communications officers can read through drafts, and offer suggested edits.

Ensure you work with these colleagues and the news site editor collaboratively. Respect their journalism skills, and remember they are knowledgeable in their field. They are experts in style, tone, images, and importantly, what readers will (and will not) engage with. They know how to adapt your research to fit with news values.

Your content knowledge, combined with their journalistic expertise, can be a match made in heaven, provided you are willing to learn from them. When this happens, your stakeholders win.

To disseminate your RBNA widely, work with your media and communications team to do this through social media. Republish your article to other relevant news sites when this option is available. Learn from colleagues who have large social media followings by watching and imitating what they do. Be sure to tag your research colleagues, partners and funders when posting a link to your article.

Research impact – Challenges for academics

The Australian Universities Accord Final Report says there is a need to expand ‘government support for research translation’. But academics work in an increasingly time-pressured environment. They face increasing administrative pressures due to managerial-inspired systems and software that encourage research record keeping and compliance over innovation, creativity, stakeholder engagement and actual research.

Recognising the challenges, early career researchers, Granek and Nakash, explain:

As junior academics in vulnerable (i.e., pre-tenure) positions, we are well aware of the fact that it is easier to answer the question of why do [knowledge translation] KT than how to do KT given the very real academic constraints … the reality of a neoliberal academic climate that rewards publications and grants at the expense of the time and energy spent on the other kinds of KT initiatives … cannot be ignored. We work in a particular sociopolitical context that values some kinds of knowledge over others.

While institutions are often good at listing these activities in organisational narratives, they need to support this work in a practical way. Such activities must be valued and acknowledged in academic systems and workload agreements.

It’s time to value all the work researchers do to ensure our whole society benefits from our research.

Dr Marg Rogers is a senior lecturer in early childhood education at UNE and a postdoctoral fellow at the Manna Institute.

Budget 2024: These early childhood educators love kids. But love won’t pay the bills

Early childhood education and care, , , ,

This is the second in a series of posts on the 2024 Budget. Today: early childhood care and education by the University of New England’s Marg Rogers, postdoctoral fellow at the Manna Institute. Yesterday: school funding by Curtin University’s Matthew P. Sinclair, a lecturer in education policy. Monday: higher education by the University of Melbourne’s Abigail Payne, director of the Melbourne Institute.

The 2024 Federal Budget has included new and continuing early childhood initiatives to support educators and relieve the workforce crisis. That said, there are no rainbows or new pots of gold in easy reach for educators. Here’s what is new, and what remains the same.

New: Paid practicum placements (from July 2024)

Educators are eligible for a payment of $319.50 a week for practicum placements outside of their own workplace. Educators have to do many practicums as they work through certificate, diploma and degree qualifications. In this past this has led to educators taking annual leave from their job to do their placement, or worse, unpaid hours creating ‘prac poverty’.

New: Lower indexation rates for HECS-HELP student debts (retrospective adjustment from 2023)

Rather than rely on the rate of inflation to set the indexation amount on student loans, the Government will now rely on lower and more predictable figures. The indexation rate, (similar to interest), will be either the Wage Price Index or the Consumer Price Index, whichever is lower on any given year. 

This initiative will bring relief to early childhood educators who are trained, or training to be degree qualified and have a HECS-HELP student loan. Educators are often especially affected as they generally take longer to pay these debts off because:

  • they have low wages, so the rate of compulsory repayments is lower;
  • as 92% of the workforce are women, they are often taking career breaks to be the primary carers of children and relatives; and
  • they often work part time due to their caring responsibilities, meaning there are often years where they pay little to none of the debt.

These debts often fester for years, increasing educators’ levels of poverty, and reducing their ability to apply for a home loan. Thus, HECS-HELP schemes are an outdated and very sexist policy designed in the 1980s, largely by men who had little understanding of the impact it would have on women.

Continuation: Wage increases (when the Fair Work Commission completes its processes)

Probably the most disappointing part of the budget for educators is there is no increase in wages until the Fair Work Commission has completed its Annual Wage Review and Gender Pay Equity Research exercise. 

While this is important work, it will not help educators in the middle of a cost of living crisis who are leaving their jobs because they can’t afford to stay. It is important to note here that degree qualified early childhood educators receive about 20% less than school teachers with the same qualifications. 

That said, the Government has committed $30 million over 2024-25 to the processes of paying educators more once the decision is finalised. 

Continuation: Incentive System payments (2024-25)

Apprentices, trainees and employees benefit from Phase II of the Incentive System for priority skill areas, such as early education. In the second phase, educators on a traineeship could receive between $3000-5000 as a bonus over the two years. Additionally, sign on incentives payments of $4000-5000 are available for early childhood services to attract staff. 

This might help to attract some new educators in a time of high employment. This bonus is especially needed in regional, rural and remote regions, and low-income metropolitan suburbs who live in ‘childcare deserts’. This is where three or more families are competing for one space within a service. In some areas, it is 20 or more families, with parents waiting for many years to access early learning for their child. 

General initiatives: New supports impacting early childhood educators

As part of the general taxpaying population, educators will benefit from the stage 3 tax reforms. It will improve their take-home pay as they earn less than $146,000. Additionally, because 92% of the workforce in early education are women, some will be eligible for these new measures, depending on their circumstances. These include:

  1. domestic violence payment of $5000 for those fleeing an abusive relationship (continuation of a pre-existing scheme);
  2. improved funding in crisis and transitional accommodation (new funding); and 
  3. superannuation added to Commonwealth paid parent leave (from 2025). 

The verdict

In some ways, this is a disappointing budget for educators. Their wages have been effectively reduced from low to unsustainable during a time of high inflation and a cost-of-living crisis. This will mean more educators will leave, if they cannot wait for the lengthy Fair Work Commission’s processes. 

Many have already left. They are enjoying higher wages in other sectors, such as the Aged Care Sector, whose wages were adjusted previously.

This high level of attrition negatively impacts:

  1. children (who need secure, caring relationships to support their learning, and access to important developmental screening);
  2. parents (especially women, who cannot work when they have no access to early learning);
  3. family wellbeing (reduced family income increases household stress);
  4. communities (who are losing young families when they cannot access early learning, and who cannot attract workers from other sectors to the area), and 
  5. the economy (as there are less taxpayers when parents cannot access early education for their children). 

Unfortunately, passion for children’s education does not pay the bills and provide a sustainable economic wage. We need real reform in early childhood education so our children, parents, families, communities and country thrive.

Dr Marg Rogers is a senior lecturer in early childhood education at UNE and a postdoctoral fellow at the Manna Institute.

What happens now to children and families after these horrors?

mental health, , , ,

In the aftermath of the horrors of the attacks in Bondi and Wakeley, many community members have been involved in or witnessed traumatic events. These can  impact mental health and family life, what we call events which cause moral injury. 

Our team has co-created resources to support children who grow up in families where a parent has a moral injury. As Anzac Day approaches, it is also relevant to consider defence, veteran and first responder (service) families.

What is moral injury? 

Moral injury (MI) is a deep wounding of the soul. It is the social, psychological and spiritual response when something or someone has gone beyond the limits of an individual’s deeply held values and beliefs. This can include events where vulnerable members of society are affected. 

In countries like Australia, members of the public seldom witness such extreme events. That’s in contrast to service members who are more frequently exposed to trauma. As such, they are more likely to have a moral injury than the general population.

Moral injury can also be caused by abuse or betrayal by individuals and organisations. For example, a child might be abused by an adult who should be protecting them. Similarly, an organisation might say they will support staff members who injure themselves at work but fail to do so. 

Sometimes injuries can be compounded. For example, a police officer might experience an injury when they witness the mistreatment of a community member. When they report it, however, they might be betrayed, demoted or ostracised by management or colleagues. 

Interestingly, moral injury is not yet considered a psychological condition. However, it can lead to mental health conditions such as Post Traumatic Stress Disorder (PTSD) and depression. It should be noted that all of us feel upset or shaken at times by what we experience or see in everyday life. This does not mean we will develop a moral injury, because it is caused by a deep wound, generally from very traumatic events.

What does moral injury feel like? 

Those with a moral injury feel a deep sense of shame and betrayal and experience feelings of unworthiness or dirtiness after seeing or being part of such events. They might think they could have done more, despite the impossible choices they might have faced in an emergency.

Those with a moral injury can withdraw from their family members and friends because of these feelings. They might think they are unworthy of loving relationships. They may even fear contaminating their loved ones because they feel guilty for what they have done or failed to do.

How does moral injury impact family life? 

Despite their best efforts to shield them, a parent’s moral injury can negatively impact children’s and teenager’s family life and mental health. Children generally misinterpret their parent’s withdrawal as rejection. They can blame themselves for their parent’s behaviour and even the moral injury.

Also, children might be exposed to their parent’s aggressive risk-taking behaviours. The parent can be over protective because of the danger they themselves have been exposed to. Children’s world view is often impacted by their parent’s. Children and teenagers might also start to see the world as a dangerous place, or that those in authority, or government departments and organisations cannot be trusted.

Adding to these challenges is the availability of mental health services for all family members, especially for those in regional, rural and remote communities. The Australian Bureau of Statistics found 30% of defence members and 50% of veterans live in these communities. So do many first responders. Therefore, there is a need for online resources and support for these families.

Our research and resources

Research showed a lack of resources to support this group of children and their families. Our Child and Family Resilience team worked with Australian and international research partners from Canada and the UK to address this need. We gathered the voices of adult children and spouses of veterans and first responders with a moral injury. And we also collected stories from support workers and clinicians who support those with a moral injury. 

We used these narratives to co-create free, online research-based resources to support children with a parent who has a moral injury. This includes a research-based storybook to support children’s understanding of their parent’s behaviour and develop coping strategies. The storybook has research-based information in the prologue and epilogue to assist educators, parents and support workers to understand what these children experience. 

Accompanying the storybook is a research-based module for parents to build their capacity to assist these children. We are also co-creating a module for support workers and educators.

Who is the storybook for? 

It should be noted that the book is not suitable for group or classroom readings; rather it is only for children who are already experiencing these issues at home. It is designed for one-to-one reading with a child and their parent, school counsellor, support worker, or educator. 

Bibliotherapy provides a non-pharmaceutical intervention to improve an individual’s mental health through reading, reflecting and discussing books to improve understanding. In this way, storybooks provide children with an opportunity to empathise with the characters and practice their emotional responses safely. The book is designed as a springboard for discussing the story and what the child is experiencing at home. 

Stakeholder feedback 

Our online survey provided feedback on the suitability of the resource. The participants were stakeholders, including service personnel, their families, and those supporting them. They provided helpful feedback to help us improve the book, along with comments such as these:

 “…I am currently still processing the injuries… I have share[d] it with …my children…(now adults) … they hurt from my actions or inactions, they become wounded children”.

”My children are grown, however, this would have been a very helpful resource for us”.

“Real words to start the conversation”.

“So sad this book wasn’t around when [my partner’s] kids and granddaughter were younger, realising what a difference it may have made if they could have understood what was happening with him.”

“This wee book provides the clearest explanation yet of the origins and initial steps toward explaining and solving a highly complex problem that (as veterans) my husband and I have been grappling with for the past 62 years”.

What next? 

The book is also being piloted with UK families through the Kings College of Military Health Research. Our team will adapt resources from feedback by July’s end to create a final copy that will be released online.

We wish we lived in a world where moral injury and mental health disorders are non-existent. In the meantime, our team needs further funding to co-create more free research-based bibliotherapy resources for children impacted by their parent’s occupations.

Marg Rogers is a senior lecturer in early childhood education at UNE. She is a postdoctoral fellow within the Manna Institute, building place-based research capacity to improve the mental health of regional, rural and remote Australia. She researches marginalised voices within families and education, especially in regional, rural and remote communities. Specifically, she researches ways to support the wellbeing of defence, veteran, first responder and remote worker families and early childhood educators.

Toddlers and teens: the news educators and parents need right now

Early childhood education and care, social and emotional competence, , , , , ,

Among educators and parents, the most often complained about age groups are toddlers and teens. Physically, socially, emotionally and cognitively there are many similarities in these developmental ages. Understanding these similarities can reduce frustrations and help us better connect with them.

In this two-part series, we explore the physical, social and emotional similarities. In the second article (published tomorrow), we will explore the cognitive similarities, share tips on building positive relationships, and provide ways to address their mental health and wellbeing.

What are the similarities?

Physical

This is a time of rapid physical growth for both age groups. Brains are struggling to keep up, causing what might seem like clumsiness and frequent accidents as they learn how to move and be in their rapidly changing bodies. They might not know their own strength and accidently break something or hurt someone as they test shifting limits. It is important to avoid overreacting and attaching a purpose behind these actions as there may be none. Letting them know you are upset and that you do not want them doing that again is okay, but try to leave it there.

This rapid growth means both toddlers and teenagers need loads of sleep. This can be tricky for teenagers who like to stay up late, then struggle with morning routines and learning activities. Additionally, gaming, streaming and social media means there is more to occupy them in the evenings. Parents are often unpopular if they take devices off children at bedtime, but it might mean a big difference enabling them to get the physical rest they need.

Emotional

Both ages are times of opposites. One minute children seem to be clingy and wanting attention and support, then the next they are pushing you away, expressing their opinions, and saying ‘No! I can do it’, snarling or grunting. They are still very needy at all times, despite the bravado. 

Learning to step back and allowing them some freedom is important, but letting them know you are there whenever they need you is vital. The saying ‘Children need your love when they least deserve it’ is very true. Teach them that if they want to do something themselves, or have time to themselves that it is okay, but that they need to express this wish in a way that is not hurtful. Providing example sentences can help them choose appropriate words.

Social

Socially, children are still learning what is acceptable, what will elicit a response, and how to navigate relationships. Emotionally, they are more likely to find rejection heartbreaking because they are forging their identity. Feeling rejected for toddlers might look like someone not sharing their toys, or pushing them over. For teens it is far more complex, and involves feeling liked and belonging within friendship or sub cultural groups. 

To be mentally healthy all humans need to feel a sense of belonging. We need an identity that locates us safely in groups of others. For toddlers those groups include the family and possibly the educators and peer group in their early childhood setting. For teenagers the importance of the family group declines (but doesn’t disappear) as they seek their place in a range of different peer groups in both the face-to-face and virtual worlds they inhabit. Learning who we are in these groups is often a function of how the group reacts to us, and children need a secure base of caring relationships. This supports them to manage the turbulent emotions that come with learning that not everyone in the world will like them or want to be with them.

Regarding identity

In regards to identity, toddlers are realising they are separate to their primary caregiver, and teenagers are forging their identity as a young person separate from their parents. At both ages, egos are very fragile, so it is important parents provide a place where they can feel safe and secure within their own home, away from the hurdy gurdy of friendships. Ideally, the family environment creates a safe basis from which children can reach out into the world and develop their own identities within their own groups. If there is not a safe environment at home, other spaces might help provide some support, such as libraries, extra-curricular groups and clubs. 

Teenagers are now old enough to realise what people say and what they mean can be different. This new skill means they often believe people are thinking the worst of them, despite the reality that people are not thinking about them at all. It is important to point out to teenagers that it is a time where they are more likely to be self-conscious, but the reality is most people are not thinking about anything but themselves or the task at hand.

For both age groups, having time alone at home is important as this time gives them the space to process their experiences and reinforce for themselves just who they are. For teens, this means times where they are not on social media. They might complain, but it is good for them to relax and not always be socially available. Time in the family unit is also important as it reinforces the relationships that make home a safe place.

Looking after yourself

Overall, it is challenging educating and parenting these age groups, so finding another trusted and experienced educator or parent to chat to is vital for your own wellbeing. It is normal for educators and parents of toddlers and teenagers to feel exhausted, challenged and exasperated at times. It is essential to recognise your own limits. It is not selfish to desire time alone to recharge batteries to enable you to cope with the next challenge thrown your way. Nor is it selfish to reach out for help when those difficulties feel overwhelming. Looking after yourself is vital for the long haul.

Marg Rogers is a senior lecturer in early childhood education. She researches marginalised voices within families and education especially in regional, rural and remote communities. Marg is a postdoctoral fellow within the Commonwealth Funded Manna Institute.

Margaret Sims is a professor in early childhood education and care and has worked in the areas of family support and disabilities for many years. She researches in the areas of professionalism in early childhood and higher education, families, disabilities, social justice and families from CaLD backgrounds. She is an honorary professor at Macquarie University.

NAPLAN: Where have we come from – where to from here?

NAPLAN, , ,

With the shift to a new reporting system and the advice from ACARA that the NAPLAN measurement scale and time series have been reset, now is as good a time as any to rethink what useful insights can be gleaned from a national assessment program.

The 2023 national NAPLAN results were released last week, accompanied by more than the usual fanfare, and an overabundance of misleading news stories. Altering the NAPLAN reporting from ten bands to four proficiency levels, thereby reducing the number of categories students’ results fall into, has caused a reasonable amount of confusion amongst public commentators, and many excuses to again proclaim the demise of the Australian education system. 

Moving NAPLAN to Term 1, with all tests online (except Year 3 writing) seems to have had only minimal impact on the turnaround of results.

The delay between the assessments and the results has been a limitation to the usefulness of the data for schools since NAPLAN began. Added to this, there are compelling arguments that NAPLAN is not a good individual student assessment, shouldn’t be used as an individual diagnostic test, and is probably too far removed from classroom learning to be used as a reliable indicator of which specific teaching methods should be preferred. 

But if NAPLAN isn’t good for identifying individual students’ strengths and weaknesses, thereby informing teacher practices, what is it good for?

My view is that NAPLAN is uniquely powerful in its capacity to track population achievement patterns over time, and can provide good insights into how basic skills develop from childhood through to adolescence. However, it’s important that the methods used to analyse longitudinal data are evaluated and interrogated to ensure that conclusions drawn from these types of analyses are robust and defensible.

Australian governments are increasingly interested in students’ progress at school, rather than just their performance at any one time-point. The second Gonski review (2018) was titled Through Growth to Achievement. In a similar vein, the Alice Springs (Mparntwe) Education Declaration (2019) signed by all state, territory and federal education ministers, argued,

“Literacy and numeracy remain critical and must also be assessed to ensure learning growth is understood, tracked and further supported” (p.13, my italics)

Tracking progress over time should provide information about where students start and how fast they progress, and ideally, allow  insights into whether policy changes at the system or state level have any influence on students’ growth.

However, mandating a population assessment designed to track student growth, does not always translate to consistent information or clear policy directions – particularly when there are so many stakeholders determined to interpret NAPLAN results via their own lens.

One recent example of contradictory information arising from NAPLAN, relates to whether students who start with poor literacy and numeracy results in Year 3 fall further behind as they progress through school. This phenomenon is known as the Matthew Effect. Notwithstanding widespread perceptions that underachieving students make less progress on their literacy and numeracy over their school years compared with higher achieving students, our new research found no evidence of Matthew Effects in NAPLAN data from NSW and Victoria.

In fact, we found the opposite pattern. Students who started with the poorest NAPLAN reading comprehension and numeracy test results in Year 3 had the fastest growth to Year 9. Students who started with the highest achievement largely maintained their position but made less progress.

Our results are opposite to those of an influential Grattan Institute Report published in 2016. This report used NAPLAN data from Victoria and showed that the gap in ‘years of learning’ widened over time. Importantly, this report applied a transformation to NAPLAN data before mapping growth overall, and comparing the achievement of different groups of students.

After the data transformation the Grattan Report found,  

“Low achieving students fall ever further back. Low achievers in Year 3 are an extra year behind high achievers by Year 9. They are two years eight months behind in Year 3, and three years eight months behind by Year 9.” (p.2)

How do we reconcile this finding with our research? My conclusion is that these opposing findings are essentially due to different data analysis decisions.

Without the transformation of data applied in the Grattan Report, the variance in NAPLAN scale scores at the population level decreases between Year 3 and Year 9. This means that there’s less difference between the lowest and highest achieving students in NAPLAN scores by Year 9. Reducing variance over time can be a feature of horizontally-equated Rasch-scaled assessments – and it is a limitation of our research, noted in the paper.

There are other limitations of NAPLAN scores outlined in the Grattan Technical report. These were appropriately acknowledged in the analytic strategy of our paper and include, modelling the decelerating growth curves, accounting for problems with missing data, allowing for heterogeneity in starting point and rate of progress, modelling measurement error, and so on. The latent growth model analytic design that we used is very suited to examining research questions about development, and the type of data generated by NAPLAN assessments.

In my view, the nature of the Rasch scores generated by the NAPLAN testing process does not require a score transformation to model growth in population samples. Rasch scaled scores do not need to be transformed into ‘years of progress’ – and indeed doing so may only muddy the waters.

For example, I don’t think it makes sense to say that a child is at a Year 1 level in reading comprehension based on NAPLAN because the skills that comprise literacy are theoretically different at Year 1 compared with Year 3. We already make a pretty strong assumption with NAPLAN that the tests measure the same theoretical construct from Year 3 to Year 9. Extrapolating outside these boundaries is not something I would recommend.

Nonetheless, the key takeaway from the Grattan report, that “Low achieving students fall ever further back” (p.2) has had far reaching implications. Governments rely on this information when defining the scope of educational reviews (of which there are many), and making recommendations about such things as teacher training (which they do periodically). Indeed, the method proposed by the Grattan report was that used by a recent Productivity Commission report, which subsequently influenced several Federal government education reviews. Other researchers use the data transformation in their own research, when they could use the original scores and interpret standard deviations for group-based comparisons.

Recommendations that are so important at a policy level should really be underpinned by robustly defended data analysis choices. Unfortunately the limitations of an analytic strategy can often be lost because stakeholders want takeaway points not statistical debates. What this example shows is that data analysis decisions can (annoyingly) lead to opposing conclusions about important topics.

Where to from here

Regardless of which interpretation is closer to the reality, NAPLAN 2023 represents something of a new beginning for national assessments in Australia. The key change is that from 2023 the time series for NAPLAN will be reset. This means that schools and states technically should not be comparing this year’s results with previous years. 

The transformation to computer adaptive assessments is also now complete. Ideally this should ensure more precision in assessing the achievement of students at the both ends of the distribution – a limitation of the original paper-based tests. 

Whether the growth patterns observed in the old NAPLAN will remain in the new iteration is not clear: we’ll have to wait until 2029 to replicate our research, when the 2023 Year 3s are in Year 9.  

Sally Larsen is a Lecturer in Learning, Teaching and Inclusive Education at the University of New England. Her research is in the area of reading and maths development across the primary and early secondary school years in Australia, including investigating patterns of growth in NAPLAN assessment data. She is interested in educational measurement and quantitative methods in social and educational research. You can find her on Twitter @SallyLars_27