Multimodal texts surround us. What are they? How can we use them in our teaching?

 The ways in which we communicate with each other in today’s world are wide ranging. We live in a time where politicians tweet national policy announcements, a YouTuber can have 75 million subscribers from around the world, and pre-teen children communicate using images on Instagram. It seems strange then, that assessment practices in schools largely remain focused on traditional written texts such as essays and reports. These texts often involve only language mode despite there being other modes that can be effectively used to express meaning. By other modes I mean communication including things like images, sounds, signs and gestures. When a text uses two or more modes we call it a multimodal text.

I have been researching how teachers use and teach multimodal texts and I believe Australia needs to update the way we understand multimodality in our schools and how we assess our students across the curriculum.

What are modes and what is multimodality?

A mode is a socially and culturally-shaped resource for meaning-making. Modes include not only language but gesture, image, sound and space. In fact, digital platforms such as Instagram and Snapchat rely on communication through pictures. Instagram is a photo and video sharing app and Snapchat is a messaging app that lets users exchange pictures and videos, called snaps, that are meant to disappear after they are viewed.

Other forms of communication might include ensembles of modes such as movement and sound. How these are orchestrated determines the intent and main message. A film for example contains a character’s voice and gestures as well as the space in which they move, not to mention the film score. Another example could be a sculpture that uses visual and spatial conventions. This is known as multimodality and it is important that teachers in schools teach all of the modes so all students can succeed.

 My research has revealed that teachers understand multimodal texts, defined as a combination of two or more communication modes by ACARA, as texts only involving digital technologies. But they can be much more than this. Cultural theorists understand texts as cultural artefacts—meaning even an ancient ceramic pot can be considered a multimodal text as it shares knowledge and meaning for a viewer through its shape, markings on it as well as the compositional material. Therefore, multimodal texts are more than just iMovie trailers.

How to plan for and teach authentic multimodal texts

 It is important that students can effectively communicate through both oral and written language, not just within the subject of English, but in other subjects as well. The Australian curriculum now expects students to be able to comprehend and compose multimodal texts in curriculum areas such as science, history and the arts.

Planning for teaching that supports students’ understanding and knowledge of literacy demands in texts that use a range of modes is critical for student to be able to compose multimodal texts effectively. These literacy demands include the codes and conventions associated with each mode.

When students learn about the codes and conventions, their meta-language (the way they can talk about the codes and conventions) improves and this transfers to their compositions.

Knowing how to create cohesive multimodal texts includes knowing how the modes link and talk to each other. For example, visual image and text in children’s pictures books work together to share the narrative. Curriculum planning therefore needs to ensure inclusion of content about a range of texts in each curriculum area as well as how different modes are used in these texts.

Effective teaching methods are also needed to teach critical multimodal literacies. These approaches to teaching relate to both the comprehension and composition of multimodal texts. Models of teaching need to be age-appropriate and address students’ personal, social and cultural needs. They also need to support students in being able to read texts critically through inferential comprehension.

How students can be effective readers and producers of multimodal texts

Students need to be able to read multimodal texts. A proficient reader of multimodal texts will more likely be a more effective composer of multimodal texts.

Here is my list of what I believe proficient readers of multimodal texts should be able to do.

  •  Use prior and new knowledge to share and understand meaning through all the modes
  • Make connections between their own experience and others
  • Develop metalanguage related to all of the modes
  • Draw appropriate and diverse conclusions from a range of texts
  • Make predictions through reading, testing and revising
  • Create different texts as a result of interpreting other texts
  • Critique what multimodal texts they read.

Once students are able to analyse and talk about multimodal texts they can then more effectively create their own.

Assessment related to composing multimodal texts

Varying assessment and options for students is empowering and makes them at the centre of learning. Multimodal texts and assessment therefore can be the catalyst in creating authentic and engaging assessment tasks that students write for purpose and with a particular audience in mind.

“Different modes demand different intellectual work from pupils and this work ‘fills up’ the concepts to be learnt in different ways. The range of representational resources made available through visual communication (spatial relations, colour, etc.), for example, enable the expression of kinds of meaning that would be difficult, or perhaps impossible, in language (Jewitt et al., 2000)”(p. 84).

It is therefore critical that educators consider ways in which to vary their assessment to meet the needs of their students. Referred to as ‘multimodal reshaping’ teachers can offer students a range of options in terms of assessment that address the same criteria. It ensures students have a choice and voice and can therefore express their meaning via a range of modes and ensembles of modes.

For example, if students are required to present an argument for or against climate change they could do so through embodiment (dance or drama), an artwork, a sequence of photographs, a newspaper report, a blog or a 3D model such as a diorama.

 What are some examples of multimodal texts across the curriculum?

English Stories that include text and images, newspaper articles, photographs, memes, comic strips, dramatisations
Science Scientific illustrations or animations, 3D models, ‘How To’ guides with pictures, museum exhibits
History Biographical portraits, dioramas, replicas of primary sources, pamphlets, posters, mobiles
The arts Sculptures, performances – dramatic and musical, soundscapes, choreographed works, advertisements

My research with teachers and students into the use and teaching of multimodal texts continues. In the meantime, I hope we can see assessment practices in schools capture more effectively some of the wide-ranging ways we communicate with each other in the world today.

Georgina Barton is Associate Professor (Literacies and Pedagogy) in the School of Teacher Education and Early Childhood at the University of Southern Queensland. She was a teacher in schools for over 20 years with experience as an Acting Principal,lead literacy intervention teacher and Head of Department, the Arts. She also spent time in South India teaching English. She has over 80 publications in the areas of literacy, multiliteracies, multimodalities, the arts and culturally and linguistically diverse contexts including internationalisation. She is currently involved in an Australian Research Council grant led by Professor Mary Ryan that explores effective and reflexive approaches to teaching writing. She is a Fellow of the Australian Literacy Educators’ Association and was the Conference Chair of the Australian Association for Research in Education (AARE)2017-2018. Georgina is on Twitter at @BartonGeorgina