Deakin University

Student engagement data: what does it actually mean?

student engagement, , ,

With the many distractions facing students today – laptops, smartphones and social media, just to name a few – it is not surprising that teachers want to keep track of what their students are doing in and outside of class. EdTech companies promise teachers to track their students’ engagement automatically and in real time. They often offer visually appealing dashboard with easy interpretable graphs. But what do these tools actually measure? And what can we read from those dashboards? 

Engagement data, as I call them in a recently published article, can be found in Learning Management Systems (LMS), such as Moodle and Canvas. They can also be found in learning platforms commonly used in primary and secondary education, such as Education Perfect and MathSpace. Even online library Epic! offers engagement data. Engagement data should be seen as any sort of metrics that claim to say something about students’ on-task behaviour, their interaction with a platform or any predictive analysis based on these interactions. Engagement data differs from performance data, which is all about results, however, sometimes engagement data is correlated with performance data to ‘assess’ risk.

There are, roughly, five different types of engagement data. They are time-on-task data, task completion data, participation data, technical data and biometric data.

Time on task

Time-on-task data is based on the time students have spent on the platform. It is a common element in most LMS. Whether or not this is a good indicator of students’ engagement is questionable. Having a browser window or application open without being involved in learning activities, would not be seen by many teachers as a sign of deep engagement. But on the platforms’ dashboards this is presented as such. 

Engagement data on Education Perfect (source: help.educationperfect.com)

What got done

Task completion data seems to be a more straightforward metric. It actually gives teachers an indication of how many tasks their students have completed. However, this depends very much on how task completion is framed. Epic!, for instance, gives an overview of how many books students have ‘read’. However, students could have just clicked through the pages without having actually read something. Another problem with task completion data is the sheer number of tasks a student has completed comes to stand for a student’s level of engagement. This is especially the case on platforms where the number of tasks is potentially unlimited, such as in Education Perfect, where the system automatically generates new tasks based on students’ performances. Following this logic, the most engaged student is the student who has completed the most tasks. 

A screenshot of student engagement data on the library platform Epic

How reading engagement is measured on Epic (source: getepic.com)

Both contribution data and technical data are more common in the world of social media and online forums than in educational contexts. The first measures the number of posts by an individual student. Students discussing the learning content, then, is seen as an additional indicator of their level of engagement, rather than merely completing learning activities. The LMS Brightspace is a particularly interesting case, as it puts learning data in a sociogram, linking the most popular contributors with each other. It is an idea of ‘social learning’ that has more in common with the way in which we are chasing likes on Instagram and TikTok.

Fig. 4

Contribution data on Brightspace/D2L (source: community.d2l.com) 

Are students users?

Technical data are rather peculiar metrics. They merely describe technical interactions with the platform, such as the number of times specific content has been viewed. These data seem to have very little to do with what is generally understood as engagement by educators and scholars. However, in the world of social media these data are very important. They determine the extent to which content can be monetised. This is called user engagement. But are students users? Increasingly so. More education is delivered through commercial platforms profiting from increased use of licences sold to school or individual students.  

Biometric data is not common on the platforms that are currently used in Australian classrooms. But several experimental studies have looked at tracking students’ engagement by measuring brainwaves or by tracking their eye-movement. Influential organisations such as the OECD even promote these techniques to keep students engaged in the digital world. Apparently, it becomes increasingly normalised to monitor students’ bodies. I have been a teacher myself. I do understand teaching involves some surveillance and control. But as a colleagues of mine once said, can we ask from students to be engaged all the time?   And do we always need to know when they are not? 

The biggest problem

Perhaps the biggest problem with all these types of engagement data is that it frames engagement as something that is per definition measurable. Indeed, anything that cannot be measured cannot be put on a dashboard or be used by algorithms. This ‘technological’ idea of engagement excludes other elements of engagement that are put forward by scholars, such as the emotional and more cognitive dimensions. Engagement data do not show if a student enjoys school, for instance. They also do not show if a student is motivated to do another task or if they are resilient enough to deal with setbacks.  

Educational platforms, then, present a very narrow idea of what engagement is about. The question is: does this affect teachers?  Does it change their perception of what engagement is? So far, little research has been conducted on this topic. With this article I hope to create more awareness on the issue. Let us all reflect what kind of engagement we want to see in our students. Let’s critically reflect on the metrics of engagement put forward by digital learning platforms. 

Chris Zomer is a research fellow for the ARC Centre of Excellence for the Digital Child. His research interests include the datafication of learning, gamified learning platforms and the use of technology in education more broadly. For his doctoral thesis, he investigated how gamified learning applications reshape ideas, understandings and enactments of student engagement in a private girl school with an ‘academic’ student population.

Evidence based practice - Community of practice

How we can challenge oppression from the ground up

community of practice, evidence-based practices, , ,

The use of “Evidence-Based-Practice” (EBP) as a discourse harms teachers. I’m not the first person to say that here. Nicole Bunker, in a previous post, describes the landscape of Australian education as awash with the dominance of the “what works” agenda. 

She says that the all-encompassing desire to impose EBP has become an oppressive force upon teachers. It promotes a narrow base of evidence in relation to “what works”. It removes teachers from the position to make judgments of what is best in their contexts. And it obscures the structural problems that perpetuate inequity in Australian schools. This is something that teachers need to push back against.

I want to add to the discussion surrounding this issue for teachers. I’d like to propose communities of practice (CoPs) can serve as an important opportunity for teachers to challenge the oppression of EBP. It can also be a means of supporting teachers to reclaim their voice and agency in education.

Where we find ourselves

Due to the complexity of the Australian education landscape, teachers are often left to navigate diverse and conflicting educational ideologies about Australian schooling. Yet, when EBP is applied in education this fact is ignored in favour of narrow ideas of education. EBP relies on ideological perspectives that see the role of the teacher as one that has a causal effect in student learning. This view is uncritically accepted by many. The tendency for policymakers over the past 20 years has been to follow this logic by applying top-down reform to teaching and initial teacher education (perceived as inputs). These are then assumed to achieve increases in various quantitative measures of student success (viewed as the expected outputs).

Ideas about “what works” in education always get caught up in ideologies of how schools should function and how teachers ought to teach. This is rarely acknowledged. As a result, EBP becomes more of a legitimation tool to enforce reform and discourage critique rather than move the project of Australian education forward to a more desirable future. We cannot expect that “what works” in one school community will inevitably achieve the same result in another. Teachers therefore need to be given opportunities to engage with evidence that considers its appropriateness to their context. By ignoring the importance of context, EBP ideologies ultimately limit the ability for teachers to engage in this important work of navigating and challenging “evidence”, as policy continues to favour “top-down” mandate approaches.

It attempts to redefine what it means to be a teacher

EBP also attempts to redefine the very nature of what it means to be a teacher. It places itself within a paradigm that wants to claim teaching as an effective and efficient profession. That is unrealistic. We need not – indeed we can’t – see the teacher as wholly efficient or effective. There are many aspects of education that are neither effective nor efficient, but are still valued. The move to support young Australians in understanding consent and respectful relationships is just one example of this.

What counts as evidence is highly contested. It therefore needs to be considered in light of the group or organisation citing it. This is especially true now, as the push for “what works” in education becomes increasingly driven by vested interests. Ultimately, the uptake of evidence depends upon the ideological perspectives of teachers, school leaders and the wider community in which schooling takes place. Professional development that allows opportunities for teachers to thoughtfully consider research evidence (including EBP) and evaluate its worth in relation to context affirms the authority that each school has to meet the needs of their communities.

We need to remind ourselves that it is not evidence that will move education forward, but the current and future decisions of teachers and school leaders. 

Why we need to centre on teachers

Although it has long disappeared in the media cycle, teacher supply in Australia remains at crisis level. The focus of our federal government has been on recruitment. In the meantime, state governments continue to pile on the mandates ignoring the messages this sends to our current teachers.

If educational discourse continues to treat teachers as simply obedient implementers of somebody else’s EBP, we will continue to lose many passionate and powerful teachers. 

What we need – maybe now more than ever – is to find ways of empowering teachers to enact intentional practice that supports the purposes and aims of education in their communities.

Creating new futures

My research is interested in how a community of practice (CoP) model can be used to provide a space for teachers to explore and challenge the ideologies that currently impact on their teaching practice, including (but not limited to) that of EBP.

A CoP consists of a group of teachers that join together regularly around a common concern. They learn how to improve their practice as a result of their interactions with one another. CoPs respect teachers as public intellectuals, who engage with one another to discern from “what works” in their context in tandem with (as opposed to being dictated to by) evidence.

Along with supporting teacher retention, belonging and agency, CoPs are a powerful opportunity for teachers to reclaim their voice and ownership of their practice, through the interrogation of the ideologies that impact on their work, including those of EBP.

EBP favours “top-down” approaches to educational problems and displaces questions of purpose with questions of process and effectiveness. CoPs provide an important counter-practice to EBP, which is “bottom-up” in its approach and allows space for teachers to critique and wrestle with EBP in light of the ideologies that they – along with their colleagues and broader school community – hold about education. CoPs allow for teachers to be heard, in a climate where teacher concerns are at best ignored, or at its worst, silenced.

My research will investigate the extent to which providing space for this kind of reflective practice might make a difference in the lives of teachers currently working in Australian schools. 

Interested in taking action?

I am currently looking for teachers who are interested in participating in a CoP to explore and reflect on the various ideologies impacting on their work in 2025. 

Are you, or someone you know, interested in participating in this project? I have included details of the project at the end of this post to consider.

It’s about time we prioritised spaces for teachers to critically engage with the ideologies that seek to claim education on their behalf. 

This project seeks to do just that.

Are you a critically reflected teacher?

Are you a teacher who thinks deeply and critically about your practice? Or is this something you have never really had an opportunity to do but would like to engage in with others who think the same way?  

I am seeking a group of committed critically reflective teachers, who are eager to experience what kind of transformational impact individual and collective critical thinking can have on their practice. Where you perceive yourself on the path of becoming “critically reflective” is unimportant. What is important is that you have a desire to think deeply about your practice! 

In 2025, as part of my research project exploring critical thinking and teacher agency, participants will have the opportunity to join together in a community with other like-minded teachers, exploring the ideological nature of education and their work as teachers. Participation in this study will involve approximately 7 hours of commitment over a period of around 18 months. Participation will involve dialogue and reflection upon the various ideological impacts of teacher work in various Zoom conference meetings and through an asynchronous private chat group, followed by an individual interview at the conclusion of the project. I am interested in your personal experiences and opinions, not in information about specific schools and their practices. 

For many of you who have either listened to my podcast segment, Ideology in Education, on the TER Podcast, or have read my posts on my Substack, The Interruption, you will know that this is something I am deeply passionate about and believe to be truly important for all teachers. 

So it doesn’t matter whether you:

  • teach primary or secondary,  
  • have been a teacher for 20 years or have just started, 
  • work in the government, independent or catholic systems, 
  • are on-going, part-time or casual.  

If you’re a registered teacher currently teaching in Australia, you can get involved! 

If this seems like something you would be interested in being involved in or have further questions, please don’t hesitate to reach out to me via email at tdmahone@deakin.edu.au and I will get back to you with further information about participating in the project.  

Know of anyone who might be interested? If so, feel free to forward this information on (as is) via email or social media! 

This study has received Deakin University ethics approval (reference number: HAE-24-046). 

Tom Mahoney is a teacher and educator of secondary VCE mathematics and psychology students. He is currently completing a PhD in Educational Philosophy part time through Deakin University. His research explores the influence of dominant educational ideologies on teacher subjectivity. Tom is on LinkedIn.

Explicit teaching mandate – a pushback now is critical

explicit instruction, ,

Today, NSW teachers will spend  their professional learning session focused on explicit teaching, also known as explicit instruction. 

The NSW Education Department Secretary Murat Dizdar told the ABC: “On day one, term two, which is a school development day, right across 2,200 schools, we will be undertaking explicit teaching learning, in every single school in New South Wales.”

Excessive focus on explicit methods will have side effects and could lead to students not meeting curriculum expectations.

A pushback is critical – explicit teaching is not a magic bullet, nor should it be the single pedagogy in any classroom. Definitions of explicit teaching vary, as do its implementations. The approach to explicit teaching and its effectiveness will depend on the discipline and specific focus in question.

 Learning is complex: Multiple pedagogical approaches are needed

We all agree that teaching and learning are critically important but complex. Teachers are focused on improving student learning. However, in Australia the 3 yearly PISA results over the last 2 decades show a decline in 15-year-old students’ ability to apply their reading, scientific and mathematical knowledge and skills to solve real-life problems.  PISA focuses on the capacity of students to analyse, reason and communicate ideas effectively, to continue learning throughout life, and become successful in the workplace.  One of the highest ranked countries in PISA has mathematical problem solving at the centre of their curriculum framework. In Singapore teachers are highly valued 

Those pushing explicit instruction,do not recognise that the literature doesn’t support its use in mathematics education. It’s either commentary or uses literature focused on research outside the field of mathematics education (e.g., literacy in the early years) and is not drawing on other mathematics education research literature. Other research is in very specific situations, such as students with some specific disability, or where the ‘thing’ being learned is very narrow.

The language used to describe various pedagogical approaches from general to specific matters. Advocates of explicit instruction or explicit teaching often state this should be the main (or only) approach used by teachers and often incorrectly infer it is the only evidence-based approach. Definitions of explicit teaching vary, as do its implementations. Importantly, the approach to explicit teaching and its effectiveness will depend on the discipline and specific focus in question. 

Comparing the pair

Explicit teaching is typically described as teacher-centred. A lesson based on this ideology begins with the teacher presenting their understanding of the lesson focus, followed by an explanation of important ideas, and a demonstration of how to do  particular examples. Students then work on similar ‘tasks’ with teacher support reducing over time as students demonstrate they are able to achieve success independently. Such lessons conclude with the teacher highlighting the important ideas from the lesson. 

Alternative approaches, where students investigate or inquire into mathematical and real-world problems  are typically described as student-centred. A lesson based on this ideology typically begins by considering a real-world situation or mathematical context that demands exploration and application of prior mathematical and/or real-world knowledge and problem-solving processes. As is often the case in social settings (including workplaces), students are encouraged to work on the task both independently and in small groups. The skilful teacher then draws on their planning and observations of students’ learning to orchestrate discussion whereby key ideas and thinking strategies are shared and evaluated by the class. This too, is explicit teaching… but the enactment allows for greater student agency and voice. This interactive, cyclical process might be repeated several times as students are supported to solve the problem.

Is it simply a matter of “Teachers, choose your pedagogy!”?

No. Australia is a low-equity education system. This means our classrooms are highly diverse. The idea that there is one best way to teach all students is not evidence-based and warrants scrutiny. Making judgments on how to teach students well relies on professional knowledge of the school, the students, the curriculum, and the real-world contexts that are important for students to learn about. Planning for student learning, and teaching effectively in the moment, are skills that teachers develop through their initial teaching qualification(s) and practice over the course of their careers. A skilful teacher will adopt a balance of teacher and student-centred approaches, depending on what the learning focus for the day calls for. 

Teaching and learning is complex. Thus, there is no one way for teachers to act in every classroom irrespective ot school type (e.g., mainstream, special education), Year level (F-12 and beyond), discipline in focus (e.g., mathematics), time of year, and even time in a lesson sequence or unit of work.

Once ‘something’ is learned it can be challenging to consider how to best teach that ‘something’ to others. This is why teachers have discipline knowledge, pedagogical knowledge (both general and specific to each discipline they teach) and curricula knowledge. We should value teachers and their knowledge of teaching, initially developed in their University degrees, and developed further as they teach and engage in professional learning – especially that specific to the specific subject and year levels in focus.

How are education systems responding to the debate?

In 2017, the Victorian Government published the High Impact Teaching Strategies, commonly referred to as the HITS. These are based on the work of Hattie’s (2009) meta-analysis of over 800 studies, his 2012 book and work from Marzano (2017). A meta-analysis is a synthesis of many different studies across levels of schooling (early childhood, primary, secondary and tertiary), types of schooling (e.g., mainstream schools, special education) and discipline areas (e.g., English, Mathematics). Hattie’s approach thus aggregates findings from many studies together. This ‘averaging’ approach can be criticised but the top ten strategies are unsurprisingly part of every teacher’s set of competencies. 

Explicit teaching (following Hattie, 2009) is one of the 10 high impact teaching  strategies or instructional practices presented. An argument is made that all 10 HITS: Setting goals, structuring lessons, explicit teaching, worked examples, collaborative learning, multiple exposures, questioning, feedback, metacognitive strategies and differentiated teaching should be part of a teacher’s practice. Some of these practices are described using different terminology elsewhere. Importantly, the HITS are seen as being used alongside other effective strategies by teachers. 

However, in different jurisdictions explicit teaching is presented as ‘all encompassing’  or all central to other more specific strategies including questioning, feedback, connections.

Questioning

If we think about questioning – an essential pedagogical approach in every discipline and Year level, and which all teachers would aim and plan to be effective – different questions have different purposes. The importance a teacher gives to the students’ response can vary greatly. Most secondary mathematics pre-service students would read an article such as Questioning our patterns of questioning to develop an understanding of different patterns of interactions (initiation-response-feedback, funnelling or focusing). In planning and in-the-moment in the lesson, a teacher selects the interaction type depending on the specific focus for learning at that point in the lesson: mainly providing feedback (IRF), or funnelling students to use a specific strategy, or helping students’ articulate their current thinking. Teachers ask important planned questions and respond to student input in ways related to the learning focus.

Aiming for methods that make sense

Any discussion about teaching must be specific to what is intended to be learned by students. Otherwise too much is open to interpretation.

We should be aiming for methods that are understood and make sense to students – these won’t be forgotten in the longer term. Teaching needs to focus on learning opportunities that persist beyond the short term.

Those who expect learning to be evident immediately do not understand what it means to learn or to understand. Learning  is an ongoing process.

Two examples from within mathematics education are included here. Anthony and Hunter’s (2009) review of the characteristics of effective teaching of mathematics discussed explicit language instruction and explicit strategies for communicating mathematics (explaining and justifying) but did not report evidence for explicit teaching as effective teaching of mathematics. Discussing research-informed strategies for teaching mathematics,  Sullivan notes that if explicit instruction is taken to be “drill-orientated approaches, with the teacher doing most of the talking” and mathematical thinking, then this is not conducive to student engagement nor motivation to learn. 

If we look at the curriculum teachers are implementing, it is very clear in the Australian curriculum, both recent and current, that explicit instruction alone will not provide opportunities for students to meet the expectations of the general capabilities, cross-curriculum priorities, nor of specific disciplines (especially mathematics).

The first aim

According to the Australian Curriculum V9.0, the first aim of Mathematics is to: “ensure that students become confident, proficient and effective users and communicators of mathematics, who can investigate, represent and interpret situations in their personal and work lives, think critically, and make choices as active, engaged, numerate citizens.”

This cannot be achieved without students engaged in decision-making about their own learning. Equally, the proficiencies and processes that underpin the mathematics  curriculum cannot be learned solely via explicit instruction.

The school classroom, the people ‘doing mathematics’ should be the learners, not the teachers, hence the term ‘student-centred’. Teachers do their mathematics in preparation for class. Mathematics teachers need to use varied pedagogies, both planned and in the moment.

Irrespective of definitions, teachers plan for effective teaching and have specific learning goals in mind. As a lesson unfolds, teachers make decisions – based on their planning – and use a variety of pedagogical strategies to maximise learning opportunities for all students. All teachers have the learning at the centre of their planning. In the classroom, the teacher should be empowered to make decisions about pedagogy based on their teachers education, prior classroom experiences, the curriculum, and professional learning (especially that focused on knowing how students learn particular ideas in a discipline.

Complex and nuanced

Teaching and learning is complex and nuanced. Thus, there is no one way for teachers to act in every classroom irrespective ot school type (e.g., mainstream, special education), Year level (F-12 and beyond), discipline in focus (e.g., mathematics), time of year, and even time in a lesson sequence or unit of work.

Once ‘something’ is learned it can be challenging to consider how to best teach that ‘something’ to others. This is why teachers have discipline knowledge, pedagogical knowledge (both general and specific to each discipline they teach) and curricula knowledge. 

We should value teachers and their knowledge of teaching, initially developed in their University degrees, and developed further as they teach and engage in professional learning – especially that specific to the specific subject and year levels in focus.

Dr Jill Brown is an Associate Professor in Mathematics Education at Deakin University. She has been working in teacher education for two decades with preservice and inservice secondary, primary and early childhood teachers of mathematics.Jill is internationally recognised for her research in the field of mathematics education. She has an impressive list of publications that focus on mathematical modelling, the teaching and learning of functions, and the use of digital technologies by teachers and students.

Does the new AI Framework serve schools or edtech?

artificial intelligence, Artificial Intelligence in schools, , , , ,

On 30 November, 2023, the Australian federal government released its Australian Framework for Generative AI in Schools. This is an important step forward. It provides much-needed advice for schools following the November 2022 release of ChatGPT, a technological product capable of creating human-like text and other content. This Framework has undergone several rounds of consultation across the education sector. The Framework does important work in acknowledging opportunities while also foregrounding the importance of human wellbeing, privacy, security and safety.

Out of date already? 

However, in this fast-moving space, the policy may already be out of date. Following early enthusiasm (despite a ban in many schools), the hype around generative AI in education is shifting. As experts in generative AI in education,researching it for some years now, we have moved to a much more cautious stance. A recent UNESCO article stated that “AI must be kept in check in schools”. The challenges in using generative AI safely and ethically, for human flourishing, are increasingly becoming apparent.

Some questions and suggestions

In this article, we suggest some of the ways that the policy already needs to be updated and improved to better reflect emerging understandings of generative AI’s threats and limitations. With a 12-month review cycle, teachers may find the Framework provides less policy support than hoped. We also wonder to what extent the educational technology industry’s influence has affected the tone of this policy work.

What is the Framework?

The Framework addresses six “core principles” of generative AI in education: Teaching and Learning; Human and Social Wellbeing; Transparency; Fairness, Accountability; and Privacy, Security and Safety. It provides guiding statements under each concept. However, some of these concepts are much less straightforward than the Framework suggests.

Problems with generative AI

Over time, users have become increasingly aware that generative AI does not provide reliable information. It is inherently biased, through the biased material it has “read” in its training. It is prone to data leaks and malfunctions. Its workings cannot be readily perceived or understood by its own makers and vendors; it is therefore not transparent. It is the subject of global claims of copyright infringement in its development and use. It is vulnerable to power and broadband outages, suggesting the dangers of developing reliance on it for composing content.

Impossible expectations

The Framework may therefore have expectations of schools and teachers that are impossible to fulfil. It suggests schools and teachers can use tools that are inherently flawed, biased, mysterious and insecure, in ways that are sound, un-biased, transparent and ethical. If teachers feel their heads are spinning on reading the Framework, it is not surprising! Creators of the Framework need to interrogate their own assumptions, for example that “safe” and “high quality” generative AI exists, and who these assumptions serve.

As a policy document, the Framework also puts an extraordinary onus on schools and teachers to do high-stakes work for which they may not be qualified (such as conducting risk assessments of algorithms), or that they do not have time or funding to complete. The latter include designing appropriate learning experiences, revising assessments, consulting with communities, learning about and applying intellectual property rights and copyright law and becoming expert in the use of generative AI. It is not clear how this can possibly be achieved within existing workloads, and when the nature and ethics of generative AI are complex and contested.

What needs to change in the next iteration?

  1. A better definition: At the outset, the definition of generative AI needs to acknowledge that it is, in most cases, a proprietary tool that may involve the extraction of school and student data. 
  2. A more honest stance on generative AI: As a tool, generative AI is deeply flawed. As computer scientist Deborah Raji says, experts need to stop talking about it “as if it works”. The Framework misunderstands that generative AI is always biased, in that it is trained on limited datasets and with motivated “guardrails” created largely by white, male and United States-based developers. For example, a current version of ChatGPT does not speak in or use Australian First Nations words, for valid reasons related to the integrity of cultural knowledges. However, this indicates the whiteness of its “voice” and the problems inherent in requiring students to use or rely on this “voice”. The “potential” bias mentioned in the Framework would be better framed as “inevitable”. Policy also needs to acknowledge that generative AI is already creating profound harms, for example to children, to students, and to climate through its unsustainable environmental impacts.
  3. A more honest stance on edtech and the digital divide: A recent UNESCO report has confirmed there is little evidence of any improvement to learning from the use of digital technology in classrooms over decades. The use of technology does not automatically improve teaching and learning. This honest stance also needs to acknowledge that there is an existing digital divide related to basic technological access (to hardware, software and connectivity) that means that students will not have equitable experiences of generative AI from the outset.
  4. Evidence: Education is meant to be evidence-informed. Given there is little research that demonstrates the benefits of generative AI use in education, but research does show the harms of algorithms, policymakers and educators should proceed with caution. Schools need support to develop processes and procedures to monitor and evaluate the use of generative AI by both staff and students. This should not be a form of surveillance, but rather take the form of teacher-led action research, to provide future high-quality and deeply contextual evidence. 
  5. Locating policy in existing research: This policy has missed an opportunity to connect to extensive policy, theory, research and practice around digital literacies since the 1990s, especially in English and literacy education, so that all disciplines could benefit from this. The policy has similarly missed an opportunity to foreground how digital AI-literacies need to be embedded across the curriculum, supported by relevant existing Frameworks, such as the Literacy in 3D model (developed for cross curricular work), with its focus on operational, cultural and critical dimensions of any technological literacy. Another key concept from digital literacies is the need to learn “with” and “about” generative AI. Education policy needs to reference educational concepts, principles and issues, also including automated essay scoring, learning styles, personalised learning, machine instruction and so on, with a glossary of terms.
  6. Acknowledging the known dangers of bots: It would also be useful for policy to be framed by long-standing research that demonstrates the dangers of chatbots, and their compelling capacity to shut down human creativity and criticality and suggest ways to mitigate these effects from the outset. This is particularly important given the threats to democracy posed by misinformation and disinformation generated at scale by humans using generative AI. 
  7. Teacher transparency: All use of generative AI in schools needs to be disclosed. The use of generative AI by staff in the preparation of teaching materials and the planning of lessons needs to be disclosed to management, peers, students and families. The Framework seems to focus on students and their activities, whereas “academic integrity” needs to be modelled first by teachers and school leaders. Trust and investment in meaningful communication depend on readers knowing the sources of content, or cynicism may result. This disclosure is also necessary to monitor and manage the threat to teacher professionalism through the replacement of teacher intellectual labour by generative AI.
  8. Stronger acknowledgement of teacher expertise: Teachers are experts in more than just subject matter. They are expert in the pedagogical content knowledge of their disciplines, or how to teach those disciplines. They are also expert in their contexts, and in their students’ needs. Policy needs to support education in countering the rhetoric of edtech that teachers need to be removed or replaced by generative AI and remain only in support roles. The complex profession of teaching, based in relationality and community, needs to be elevated, not relegated to “knowing stuff about content”. 
  9. Leadership around ethical assessment: OpenAI made a clear statement in 2023 that generative AI should not be used for summative assessment, and that this should be done by humans. It is unfortunate the Australian government did not reinforce this advice at a national policy level, to uphold the rights of students and protect the intellectual labour of teachers.
  10. More detail: While acknowledging this is a high-level policy document and Framework, we call for more detail to assist the implementation of policy in schools. Given the aim of “defining what safe, ethical and responsible use of generative AI should look like” the document would benefit from more detail; a related  education document from the US runs to 67 pages.

A radical policy imagination

At the 2023 Australian Association for Research in Education (AARE) conference, Jane Kenway encouraged participants to develop radical research imaginations. The extraordinary impacts of generative AI require a radical policy imagination, rather than timid or bland statements balancing opportunities and threats. It is increasingly clear that the threats cannot readily be dealt with by schools. The recent thoughts of UNESCO’s Assistant Director-General for Education on generative AI are sobering.

A significant part of this policy imagination needs to find the financial and other resources to support slow and safe implementation. It also needs to acknowledge, at the highest possible level, that if you identify as female, if you are a First Nations Australian, indeed, if you are anything other than white, male, affluent, able-bodied, heterosexual and compliant with multiple other norms of “mainstream” society, it is highly likely that generative AI does not speak for you. Policy must define a role for schools in developing students who can shape a more just future generative AI, not just use existing tools effectively.

Who is in charge . . . and who benefits?

Policy needs to enable and elevate the work of teachers and education researchers around generative AI, and the work of the education discipline overall, to contribute to raising the status of teachers. We look forward to some of the above suggestions being taken up in future iterations of the Framework. We also hope that all future work in this area will be led by teachers, not merely involve consultation with them. This includes the forthcoming work by Education Services Australia on evaluating generative AI tools. We trust that no staff or consultants on that project will have any links whatsoever to the edtech, or broader technology industries. This is the kind of detail that may help the general public decide exactly who educational policy serves.

Generative AI was not used at any stage in the writing of this article.

The header image was definitely produced using Generative AI.

An edited and shorter version of this piece appeared in The Conversation.

Lucinda McKnight is an Australian Research Council Senior Research Fellow in the Research for Educational Impact Centre at Deakin University, undertaking a national study into the teaching of writing with generative AI. Leon Furze is a PhD Candidate at Deakin University studying the implications of Generative Artificial Intelligence in education, particularly for teachers of writing. Leon blogs about Generative AI, reading and writing.

Disability: Let’s adjust learning design now for everyone

learning design, students with disability, Universal design for learning (UDL), , , , , ,

Bob Dylan’s classic Subterranean Homesick Blues goes:  “You don’t need a weatherman to know which way the wind blows.” Anyone teaching or working in higher education knows the number of students with disabilities is growing. The number and proportion of students disclosing disability has grown every year since data was first reported nationally in 1996.

The surge in the number of students disclosing their disabilities is the result of many influences. Reporting of disability status in student systems is on the rise for reasons including improved processes and a greater willingness of students to disclose. We also know that both incidences and reporting of some forms of disability – notably mental health conditions – are on the increase in broader society.

Are students with disabilities overrepresented in higher education?

The Universities Accord discussion paper presented data showing students with disabilities are now overrepresented in higher education. The reality is more complex and goes to the heart of how students with disability are defined and counted in higher education. 

We recently explored this in our article ‘Three decades of misrecognition: Defining people with disability in Australian higher education policy”. We want to use this blog to highlight opportunities to improve the learning environment and graduate outcomes for university students with disability.

First-year retention and success rates, and degree-completion rates for students with disabilities, remain well below those of other students. Almost two out of every three students complete their degree within six years, compared to around one out of every two students with disability.

Recognition-redistribution paradox

Universities are aware of this and have worked for many years to provide support to these students. But one unintended consequence of their efforts has been the creation of what has elsewhere been called the ‘recognition-redistribution’ paradox. In the context of disability, recognition means positively highlighting, or celebrating, what it means to be disabled. Redistribution, on the other hand, means acknowledging the disadvantage experienced by persons with disabilities when they encounter social and structural barriers, or even outright discrimination.

Consequently, this leads universities simultaneously saying to students with disabilities “we don’t define you by your disability” and “we can offer you support – but only if we define you by your disability”.

One reason for this paradox is, perversely, located in an important key protection for persons with disability that is found in both the Disability Discrimination Act (1992) and the Disability Standards for Education (2005). This is the notion of the ‘reasonable adjustment’.

What is a reasonable adjustment?

A reasonable adjustment is an action taken by an institution to ensure that a student with disability can participate in education free from direct or indirect discrimination. 

These adjustments may include extra exam time, modifying the curriculum or presenting information in different formats. But to gain access to a reasonable adjustment, the student must a) identify as disabled, b) acknowledge a ‘deficiency’ and c) have their disability medicalised by a health professional.

One recommendation arising from the recent Disability Royal Commission is to remove the word “reasonable” from reasonable adjustment. This would be an important step, as it would effectively reverse the burden of proof from the student to the institution.Yet the paradox would remain. In addition, legal entitlement to a reasonable adjustment is restricted. Students who do not identify with disability, but who need some form of flexibility for health-related reasons, are thus ‘disabled’ by institutional processes if they request or are granted a reasonable adjustment.

In the future, support provision for students with disabilities will be thrust into the spotlight, for several reasons. As discussed above, general awareness around disability cultures is improving and with it, improved commitments to affirming the rights of persons with disabilities.

Universities must improve their outreach

Yet if we want higher education to achieve the ambitious growth targets proposed in the recent Australian Universities Accord Interim Report, universities will need to improve their outreach and engagement with groups of students historically under-represented in higher education. 

This includes students with disabilities.

How universities support these students need to shift – dramatically. It cannot put greater pressure on universities’ disability support offices.

A universal design for learning

The fundamental approach to disability support needs to move from the primacy of the reasonable adjustment to inaccessible curriculum to principles of universal design for learning (UDL) that reduces the need for adjustments by design.

UDL is an approach to teaching and learning that uses a variety of methods and approaches to teaching to remove unnecessary barriers to learning. Rather than just offering one way of students engaging with the curriculum, and demonstrating their understanding, UDL is about flexibility and adjustment to suit a variety of learners.

System wide implementation of UDL will reduce the burden on students disclosing and substantiating their disability to be eligible for negotiated changes to inaccessible curriculum.

This is a key issue at the heart of our recent paper where we argue current reporting mechanisms may not be fit for purpose. Personal information such as disability status should only be collected if there is a direct benefit to the student and/or a wider benefit in terms of institutional understanding and support for these students.

Ultimately, UDL challenges a university to reconsider almost every aspect of their operation, including:

·         Attitudes of all staff towards students with disabilities.

·         The development and promotion of polices to support students with disabilities.

·         Creation of a fully inclusive physical/built environment.

·         How information – both academic and non-academic – is communicated within the institution.

·         What software and hardware technologies are provided for students, and what types brought by students can be supported.

·         Wider social inclusion, including extra-curricular activities.

This is not to say that systemic adoption of UDL will completely replace the use of reasonable adjustments. It cannot fully resolve the recognition-redistribution paradox. 

But it can significantly improve the quality of the educational experience for literally thousands of students, both with and without disability.

From left to right: Tim Pitman is an associate professor at Curtin University, researching higher education policy and widening access and participation for groups of students historically under-represented in higher education, including those from low-socio economic backgrounds, Indigenous persons, people with disability, people from non-English speaking backgrounds and people from regional and remote parts of Australia. Matt Brett is Director of Academic Governance and Standards at Deakin University where he has oversight for academic governance, academic policy, course approvals, equity reporting, institutional research and surveys, quality assurance, and quality reviews. He is a Child of Deaf Adults (CODA)  and began his career in higher education as a sign language interpreter. He has a sustained and multi-dimensional impact on student equity.  Katie Ellis is a professor and director of the Centre for Culture and Technology at Curtin University  where she conducts disability led research into socially just digital futures. She also co-chairs Universities Enable.

Refugee Week: Why universities could – and how they should – offer refuge

equity in higher education, higher education, higher education policy, refugee students, , , , , , ,

Every year, a fraction of the world’s forcibly-displaced people get the opportunity to resettle in one of the main refugee-resettling countries, including Australia.  Refugees escape war and violence and search for a place to rebuild their lives. Access to and success in higher education supports refugee integration. However, while access to higher education is around 40 per cent on average globally, among refugees, it is only six per cent.

There is much universities can do to address this challenge! 

This week (June 18 to June 24) is international Refugee Week and its theme is Finding Freedom. Freedom is more than the absence of suffering and persecution; genuine freedom entails having the opportunities to be and do what one has reason to value. For refugees, having real freedom means being able to make their own decisions, engage meaningfully in society, and achieve their goals and aspirations. 

In this piece we reflect on education as a means of freedom and the role of universities in helping refugees find freedom.

Globally, universities engage in humanitarian work in many ways. Universities, as public goods, can facilitate integration opportunities through their role in society. Firstly, as sites of higher learning, universities can offer hope and pathways to individual, community development and tools for economic participation and future nation-rebuilding. Secondly, as key brokers between students and professions — through liaison with community, employers, and professional associations — universities can push for more postgraduate opportunities and shift employer and societal attitudes towards more positive welcome for forced migrants. Thirdly, universities have a role to play in creating more durable solutions to refugee resettlement through the development of educational migration refugee pathways.

Universities Can and Should Play a Bigger Role in Supporting Refugees

In a recent book, entitled The Good University, sociologist Raewyn Connell highlights five key features of a good university. For Connell, a good university is democratic, engaged, truthful, creative, and sustainable,“fully present for the society” that supports it. An engaged university is a responsive and responsible university. A good university produces socially relevant knowledge for addressing pervasive issues (e.g. environmental catastrophe and humanitarian crises). An engaged university deals with difficult societal issues such as injustice, racism, domination, and exploitation.

A good university is not simply an economic machinery; it does not aspire just to contribute to knowledge economy. A good (and engaged) university is committed to building a knowledge society that is just, caring, democratic, and sustainable. 

In our collective response to humanitarian crises, universities have three critical roles to play. The most common strand of engagement concerns widening access to teaching and learning in higher education. Universities can offer special consideration to admit forcibly displaced people, including offering online access to courses to people in displacement contexts, such as this example from the University of Leicester in the UK. Many universities in Australia and internationally also offer financial assistance in the form of scholarships.  

The second form of humanitarian response is research and training. Universities generate knowledge on causes, consequences, and potential solutions of humanitarian crisis. Humanitarian training focuses on equipping leaders in emergencies with evidence-based knowledge and skills. Examples include the Harvard Humanitarian Initiative and Deakin University’s Centre for Humanitarian Leadership.

The third example of university responses to humanitarian crisis is advocacy. University staff and students in the Global North engage in awareness raising activities and campaigns. The efforts range from mobilising financial resources and engaging in public consultation to organizing seminars and panel discussions on humanitarian issues. National examples of coordinated advocacy include the Universities of Sanctuary movement in the UK and the Welcoming University initiative in Australia. The Refugee Education Special Interest Group is an example of a grassroots activism network in Australia that works to advocate for better educational opportunities and outcomes for students from forced migration backgrounds. At the institutional level, the Centre for Social Justice and Inclusion at UTS is an excellent example of a university that is leveraging its resources to advocate at the local level, as well as using its networks to amplify its own and other advocates’ messages nationally.

Policy Invisibility of Refugees

Policy invisibility is a major issue in Australia. Despite being a signatory to major global refugee-related initiatives, including the Refugee Convention (1951) and the New York Declaration for Refugees and Migrants (2016), Australia has failed to ensure that refugees are consistently included in educational policies. Major national inclusion initiatives (for example, the Multicultural Access and Equity Guide and the Alice Spring Educational Declaration) recognise refugees as targets of policy action. However, when it comes to the higher education sector, refugees are invisible. They are subsumed under other equity groups such as Non-English Speaking Background or low Socio-Economic status group. None of these grouping recognise unique educational needs of refugees.  Policy invisibility at sectoral level means, universities have little or no financial incentives to support students with forced migration backgrounds. 

What can be done

In a report to the Commonwealth government, Peter Shergold and colleagues stressed: “Investing in refugees, investing in Australia”.  That is true.  High educational attainment enables refugees to actively participate in the economic, social, and cultural lives of the host society.  It supports integration. Conversely, low educational attainment means a loss of human capital, which in turn may diminish national economic productivity and competitiveness. This is particularly the case, given the majority of refugees are young and eager to rebuild their lives. 

In their journeys to, through, and out of higher education, refugees and asylum seekers in Australia can face many challenges associated with English language proficiency, navigational resources, and ongoing academic support. 

Facing similar challenges, the German government managed to enrol tens of thousands of refugees into higher education by (a) funding an independent agent that could assess educational levels and qualifications of refugees, (b) supporting refugees to study in special academic preparatory colleges, and (c) providing funding to universities  enable them to provide ongoing academic support to refugee students.

We believe we can learn a lesson from the coordinated approach to refugee education in Germany. This requires policy recognition as a formally identified equity cohort; it necessitates sustained ‘Welcoming Refugees Universities’ coordination; and it demands a greater shared responsibility between students, staff, institutions, and governments to make sure that the challenges we have been writing about for nearly 10 years become action points, rather than points of perennial concern. 

What matters is that educational opportunities help refugees find freedom. The importance of freedom and education for refugees cannot be overstated. For refugees, freedom means more than just the absence of physical confinement. It also means the ability to live a life of dignity and autonomy. Education is a key enabler of this kind of freedom. 

A free and fair society should ensure that all qualified members have access to quality and relevant higher education. By providing refugees with the knowledge and skills they need to thrive, education empowers them to build a better future for themselves, and their families. 

From left to right: Dr Tebeje Molla is a Senior Lecturer and ARC Future Fellow in the School of Education at Deakin University, Australia. Dr Sally Baker is a Senior Lecturer in the School of Education at UNSW. The Hon. Prof. Verity Firth is Pro Vice-Chancellor (Social Justice & Inclusion) and Executive Director, Centre for Social Justice and Inclusion at UTS.


Have we lost trust in science?

COVID-19 and schools, HASS, Science, STEM education, ,

Trust in Science, Society, and the Australian State: A Crisis in the Making?

“The return to school has no room for anti-vax sentiment or vaccine hesitancy,” NSW Education Minister Sarah Mitchell told the Sydney Morning Herald. The questioning and the loss of trust in science has been brought into full force by the Covid-19 pandemic and has entered public discourse and all spheres of private and communal life.

Vaccines have been splashed across headlines in Australia, with rapid uptake amongst the latest added cohort tempered by vocal, policed anti-vaccination protests. This questioning of science and trust in scientists poses a much larger question – How are science, society, and the state interacting? Why do understandings of science continue to be limited to ‘STEM’ disciplines rather than all systemic forms of knowledge? And how do educators and researchers respond to such concerns in a higher education context that has been severely hampered by the epidemic whilst its work becomes more visible, and sometimes more important than ever? 

A recent symposium held online by Deakin University’s Centre for Research for Educational Impact (REDI), and the Science and Society Network (SSN) sought to spotlight such questions. ‘Science, Society and the Australian State: A Crisis in the Making?’ ran across three days with four panels and a roundtable. Both vaccine uptake and the climate crisis were common threads across the event. Mitchell’s statement about the incongruence of vaccine hesitancy and open classrooms begs for debate from university leadership and institutional representatives, but crucially requires inter-disciplinary perspectives and a view across philanthropy, private sector, government policy and media.

Scepticism and loss of trust in science was a major thread throughout the symposium. Professor Emerita, Raewyn Connell suggested that the proliferation of knowledge may be attributed to its commodification. However, dealing with this knowledge was the key question – How do we communicate across disciplines and avoid miscommunication in a sea of knowledge? Brian Schmidt, Nobel Laureate and Vice-Chancellor, Australian National University commented on the rise of the internet and how we now face a drowning out the voices of the academics and experts. He asserted that in the educational context students can be taught how to understand this information and learn how to act critically and ethically. 

With the proliferation of knowledge claims there is a tendency to deprive science of its context, nuance, and complexity. When this happens the ability to appropriate or simply deny scientific knowledge becomes much easier. The rise of misinformation and conspiracy theories available online and on social media compete with knowledge claims from experts. Alexandra Roginski from Deakin University spoke about her research on ‘Conspirituality’ and how deep social rifts have been created through questioning the seriousness of Covid-19. Roginski provoked us to consider the historical precedence of misrepresenting science and attacking of scientific consensus.  These are not new phenomena and have a history within the campaigns of the tobacco lobby. From a public facing perspective, epidemiologist Catherine Bennett from Deakin University discussed her role in communicating epidemiology expertise with the media. She stressed the importance of starting conversations and figuring out what information people have. She clarified that some people unknowingly spread false information while others may do so on purpose. A sympathetic and informed position can enable dialogues which convey enough complexity to explain key ideas and communicate effectively. 

As the symposium continued, the role of the educational institution became a key focus. Centring on the curriculum, Mark Rose from Deakin University discussed the impact science and policy on indigenous Australians. He advocated for a curriculum that balances the many facets of modern Australia and equips children to deal with competing worldviews, allowing a nimbleness of mind to see different perspectives, yet remain anchored at the same time. Focusing on the university, Tamson Pietsch (University of Technology Sydney) used a framework of ‘publics’ as she acknowledged that universities are deeply dynamic and political in nature. She placed climate change at the centre of the issues around which publics must come together to bring demands on governments and institutions. Student and broader educational publics can act to meet the challenges facing universities, the educational sector and, to an extent, help shape society at large. 

Finally, it became clear that tackling eroding trust in science is not just a job for ‘the scientists’, although it is often presented as such. The latest reform to higher education illustrates how STEM values have been favoured above critical thinking and creativity offered by the humanities. However, as Joy Damousi from Australian Catholic University explained, the collaborative skillsets of both STEM and HASS are required to tackle global challenges. The so called ‘divide’ between the two areas of knowledge is complicated and nuanced. Glenn Withers, Professor of Economics, Australian National University argued that there is value in all forms of knowledge. In particular, he suggested that holistic knowledge offered by indigenous knowledge systems may provide a way forward to work across disciplines and should be developed and institutionalised as an additional learned academy. 

With the threats of climate change and Covid-19 playing upon our collective psyche, grappling with the issues of science, society and the Australian state is a critical task. So where do we go from here? The symposium debating the nexus of science, society and state has helped advance a set of critical debates. Tackling vaccine hesitancy and climate change is an interdisciplinary task, which requires educators and researchers to think critically and creatively. Educators, researchers, and institutions play an important role in forming publics and taking a stance. The role of nuanced, yet clear communication is required to spread accurate knowledges against a plethora of information to prevent misunderstandings and gain trust. Finally, taking different knowledges into account, such as indigenous knowledges, may offer us a way forward in facilitating holistic and interdisciplinary work to take on the developing challenges we collectively face. 

Natasha Rooney (Deakin University) is a PhD Candidate at the Alfred Deakin Institute of Globalisation and Citizenship, Deakin University. Her research is on the circulation of epigenetic and postgenomic models of life in the Global South.