By Sari Pietikäinen, Alexandra Jaffe, Helen Kelly-Holmes, Nik Coupland
Sociolinguistics from the Periphery "presents a fascinating book about change: shifting political, economic and cultural conditions; ephemeral, sometimes even seasonal, multilingualism; and altered imaginaries for minority and indigenous languages and their users"
“Working with Multimodality” links the field of ‘New Literacy Studies’ to multimodality by trying to consider how literacy practices can be better understood within wider domains. In other words, New Literacy Studies and multimodality show how meaning-making processes exploit multiple modes in order to create texts that are modally complex. Yet, to date, very few studies have examined multimodal composition outside educational contexts. Therefore, while concentrating on professional contexts involving multimodal creation, the author attempts to offer a literacy framework for both education and training, suggesting “a need to think far more progressively about what literacy might mean in the future with digital and media convergence” (p. 2). Rowsell’s belief is that working with modes allows the development of higher levels of abstraction and universalization across discipline-specific practices. As suggested by Halliday (1978), Rowsell defines a mode as the textual product resulting from the “cultural shaping of a material” (p. 3). Naturally, given the fact that a multimodal text is composed of more than one mode, it is essential for the author to disclose how modes work. Indeed, meaning-making processes can be transmodal (i.e. when the elements within a text create a whole, e.g., films, where meaning is provided by both visual and sound modes), intermodal (i.e. when elements within a text create a link between modes, although they may exist independently from one another, while at the same time cross-referencing, e.g., illustrations), or intramodal (i.e. where more than one element jointly creates meaning, e.g., the clothing industry, where colour combines with particular textiles in order to create a more pronounced effect). The relationship between modes is dependant on the mode-assembling process, which takes into consideration which, amongst the available modes, is the most suitable or has the greatest aptness (Kress 2010) with regard to text creation. This process transcends time and determines a transformation, or transduction, of the modes under consideration.
As said above, meaning-making processes in literacy education have seldom been investigated. The gap that the author would like to bridge within literacy education is therefore closely related to the way in which we can work on the creation of complex multimodal texts. Indeed, ‘new’ literacies, such as sound, images and hypertexts, exist alongside ‘old’ literacies, such as space, dance, movement and textiles which, in the author’s opinion, should be taken into consideration in literacy education policies, since learning with modality means converging both the social and subjective elements of meaning-making, which is a process that is always culturally-based and socially-inherent (p.4), and which we must bear in mind in any attempt to explain what literacy may mean if linked to today’s fast-paced digital and media convergence.
The analysis, based on an ethno-methodological approach, focuses on nine different modes: film, sound, visual, interface, videogames, space, movement, word, and textiles. Each is examined as a case study and dealt with in nine different chapters, starting with film, which embraces several other modes, and then moving on to all the others, each of which is connected to the previous one. The ethnographic means by which such modes are examined consist of thirty open-structured interviews with creators, who range from web designers and film producers to textile designers and dancers, and from song writers to videogame creators and advertisers. The investigation takes a look at storytelling as a way of creating meaning. What the author actually does is look at the social practices involved in modal production in order to show that design and creation are more important than the final product itself. The analysis of case studies across modes outside the context of educational pedagogy is, according to the author, extremely interesting, as it allows the elaboration of a framework for modal learning. Multiple modes are thus brought together to form an integrated theory of multimodality. The questions at the end of each chapter provide food for thought, while key-points are dealt with by the author’s comments on each case study presented.
Chapter one: Film
In this chapter, consideration is given to representation rather than communication. As Kress (2010) claims, representation concerns the addresser, whereas communication focuses on the recipient. The storytelling emerging during interviews with Tobias Wiegand (an animator), Robin Benger (a documentary director) and Rebecca Birch (a film producer) shows how professionals understand (and reproduce) the representation of a perceptual world. As they all explain, various modes, such as words, music, space, and movement, are grouped together in films in a synaesthic way so as to represent ideas, emotions and perceptual worlds. As such, films require a great deal of planning. Educational movie-making projects require the development of skills concerning the transformation of a topic into a narrative which needs to be developed in order to provide verbal, visual and sound coherence.
Chapter two: Sound
Music is not only appreciated from a cultural viewpoint, but is also synaesthically-constructed, as it involves different sounds, colours, emotions, stories and meaning. This chapter examines sound as a way of conveying and interpreting meanings, as revealed through interviews with David Murphy (a composer), and Paul Chivers (a mixer-composer). In their opinion, meaning results from either the combination of chords, or the linking of lyrics to melodies, or even a remix of old music, so as to create new compositions. Such combinations may become a part of literacy, since, for example, music can be the equivalent of words, genres, and registers.
Chapter three: Visual
According to Rowsell, visual elements are linked to creativity, subjectivity and intuition. This is clearly demonstrated through interviews with Ben Hodson (an illustrator), Bany Mendi (a director), and Lee Edward Födi (a writer-illustrator). Although visual elements are at the core of most texts, this mode is seldom incorporated into language. This is a point to be taken into consideration because today we live in an increasingly design-orientated world. Therefore, the analysis and production of visual techniques should be included in educational projects, as they require “thinking and expressing in images what is often beyond linguistic capabilities” (p. 45). Indeed, the visual mode is supplementary to other meaning-creating modes, as it enriches the interpretation offered by a text.
Chapter four: Interface
Interface refers to “the face of digital environments” (p. 60), and as such, it is the way in which content is displayed. Here, the focus is on interface design, with particular regard to aesthetics-driven, user-friendly, mobile and networked interfaces. This is a necessity, considering that people, in particular, youth, ‘consume’ and produce digital media very quickly. The emphasis, therefore, is on the way in which technologies converge to create texts whose interface represents an ideal combination of function and aesthetics. The results, as seen in interviews with Lisa Murphy (a web director), Adrian Thiessen and Kristen Nater (a media president and vice-president), and Joe Delisco (the icloud creator), aim to encourage both educators and students to develop meta-awareness of web-search engine keywords, as well as inspire web-design, with the latter being based on the idea that web-texts are read by following an F-pattern (according to which a text is first read on the left column and then on the rows on the right; cf. p. 66) rather than the traditional Z-reading path (according to which a text is read from left to right, from top to bottom, line by line; cf. Kress and van Leeuwen, 2006).
Chapter five: Videogames
Rowsell regards videogames as problem-solving tools, as they expect players to “strategize, communicate, interpret context, solve problems, analyze characters, possess hand/eye coordination, have patience, understand semiotic tools, use their spatial sense” (p. 79), which could be easily applied to the context of literacy learning, from which students could benefit greatly, considering that the skills pertaining to videogames are the same as those used in modern and digital communication systems. Indeed, this is what emerges from interviews with David Elton (a videogame creator), and Kevin Kee (a videogame creator in the field of mobile phone technology).
Chapter six: Space
The importance of space in relation to place is the key element of interviews with three architects, Anthony Robins, David Parker and Ana Lakoseljak. Space and place have a cultural as well as a subjective meaning, and in literacy learning, such a mode should be taken into consideration. Designing space according to its social role is an important achievement in terms of recognition of how space is used, thus requiring a meaningful fruition of space.
Chapter seven: Movement
The mode of movement is the main focus here, in particular, with regard to dance as a form of communication. Unlike all the other modes, which can be experienced without a performer, movement is the only one where a performer is required in “the midst of practice to communicate” (p. 110). The analysis of this mode is offered through a report of interviews with Karin Kain (a ballet dancer), Glenys McQueen-Fuentes (a ballet teacher), and Derek Metz (an actor). The lesson to be learned here is that movement, be it formalized (e.g. ballet), or free (e.g. expressive movement and dance), allows people to vent their own personality traits. In literacy projects, movement allows transmodal analysis and design, since it lets students express their own interests, either in music or more visual forms.
Chapter eight: Word
Although language is the primary mode of communication, words can be limiting if individuals rely only on oral expression. Words need to be supplemented by other modes in order to allow effective communication. Such an analysis is based on four case studies, i.e., interviews with Gary Bonilla (Creative Director for Nestlé), Grant Lefleche (a journalist), Kari-Lynn Winters (an author of books for children), and Gail Bowen (a playwright). In all cases, words are seen as visual tools, which, together with either static or moving images, can amplify and emphasize the basic meaning of a message. By aiding students in their attempt to understand what word best conveys meaning, and where a particular lexical choice does not fulfil communication objectives, and should therefore be excluded in favour of other modes, students will be able, on their own, to decide which modes are the most suitable to communicate information or ideas.
Chapter nine: Textiles
The use of textiles is a mode which has very rarely been analysed in the field of education. However, according to Rowsell, literacy teachers can include this mode in classroom activities because textiles, by encompassing both design and technology, fashion and imagination, and business and economics, offer practical projects, problem-based learning and practical demonstrations. Case studies regarding such exploitation of the textile mode in literacy classes are offered through interviews with fashion creators Trish Ewanika and Michelle Vanderheyden.
Overall, the book is a valid introduction to literacy, as it follows a practical rather than a theoretical approach. This places “Working with Multimodality” among the most authoritative textbooks on New Literacy that are currently available A key point of the book is that it is written in a clear and user-friendly style, and definitions of terminology are provided wherever necessary. Because of its characteristics, “Working with Multimodality” is an invaluable resource for teachers, trainers and students. It is suitable as further reading in a course on literacy, particularly if the readership is comprised of students with a very basic or limited linguistic background. It is very useful for linguistics students and would-be language teachers and students involved in communications studies courses. Further, individual chapters may be used as integrated material for courses on literacy education, multimodality and applied linguistics.
Although all chapters of the book are compelling, I would like to highlight the part of the fourth chapter that deals specifically with web interface. Indeed, this is a new mode of communication whereby there is no clear-cut distinction between the text producer and the text consumer; the recipient of the message is the ‘prosumer’, i.e., s/he is simultaneously the text producer and text consumer. Readers of this mode are editors because they are enabled to enter, edit and manipulate web-text while visiting a website. In addition, to the best of my knowledge, no other books in linguistics have ever emphasized the F-reading pattern characterizing web-texts. Overall, interface may create interaction, and may facilitate communication, and this chapter helps educators who are trying to develop discourse meta-awareness in students.
Nevertheless, there are still a few criticisms to be made. Apart from a typo on p. 56 (‘illustratorr’ instead of ‘illustrator’), in my opinion, there is some ambiguity in the text where two terms are being defined: the concepts of ‘emic’ and ‘etic’ are mentioned on page 25, but when the term ‘emic’ first appears (p. 9), the reader is left to his or her own devices with regard to the interpretation of its meaning. Similarly, although on page 15 the author provides a clear definition of ‘mode’ by referring to the one provided by Kress, this is totally missing on page 3 under the paragraph “Modes” (where it would be more useful). Unfortunately, the author also made a few oversights: Kress (2010), who is quoted throughout the textbook, is not listed in the bibliography; the same is true for Rowsell (2012), and Whorf (1929). Such oversights are a pity, especially when considering the overall value of the volume.
Regardless of the minor criticisms above, “Working with Multimodality” is a valid aid for teachers, advanced students and linguists wishing to have a better understanding of the relationship between literacy and education in our modern digital world.
Halliday, Michael 1978. Language as Social Semiotic. London: Edward Arnold.
Kress, Gunther 2010. Multimodality: A Social Semiotic Approach to Contemporary Communication. London: Routledge.
Kress, Gunther and van Leeuwen, Teo (2006). Reading Images: The Grammar of Visual Design. New York: Routledge.
ABOUT THE REVIEWER:
STEFANIA M. MACI is a researcher of English Language and Translation at the University of Bergamo, Italy, where she teaches English linguistic courses at graduate and undergraduate level. She is member of CERLIS (Centro di Ricerca sui Linguaggi Specialistici, coordinated by Prof. Maurizio Gotti), CLAVIER (The Corpus and Language Variation in English Research Group), BAAL (British Association of Applied Linguistics), and AIA (Associazione Italiana di Anglistica). Her research is focussed on the analysis of the English language in academic contexts, with particular regard to the analysis of English in National and Professional Contexts. Amongst her recent publications are: “Glocal Features of In-flight magazines” (2012), “Arbitration in action: the display of arbitrators' neutrality in witness hearings” (2012); “The Discussion Section of Medical Research Articles: A Cross Cultural Perspective” (2012); “Fast-Track Publications: The Genre of Medical Research letters” (2012); “The Genre of Medical Conference Posters” (2012); “Poster Makers Should Think as Much about Show Business as Science. The Case of Medical Posters in a Diachronic Perspective” (2012); and the monographs “Tourism Discourse: professional, promotional, digital voices” (2013); “The Language of Tourism” (2010), and “The Linguistic Design of Mary Magdalene” (2008).