LANGUAGE IN INDIA

Social Semiotics as a Tool for Visual Literacy

Kumaran Rajandran, M.A.

Abstract

This paper deals with the use of social semiotics to enhance visual literacy. Literacy is no longer confined to reading and writing and its meaning has expanded to include other modes that convey information. Among these modes are visuals that are frequently produced and manipulated by those in advertizing.

Visuals play many roles and can help strengthen the desirability of products or services. To better exploit visuals, designers can improve their visual literacy with a tool to help them create visuals that influence consumers to acquire products or services. This paper forwards social semiotics as a tool to use in creating such visuals. Its use emphasizes the interpretation inherent in visual literacy. Social semiotics helps advertisers imply the meanings they desire in visuals by creatively using the many elements that constitute visuals like angle or field of vision. It enables them to work smarter and increases the quality of their visual literacy. This potential is exemplified by a brief analysis of two visuals from commercial bank brochures.

Introduction

Reading and writing have come to define literacy. Being literate has often meant being able to read and write that conferred high social prestige to those who had these abilities. Yet, communication is more than just reading and writing. Language is but one mode in communication that coexists with other modes. For example, facial and bodily gestures accompany spoken language and visuals accompany written language. These extra modes contribute to multimodal communication where more than one mode is used to communicate. While reading and writing received a lot of focus, these other modes were ignored or became marginal. This did not reflect communication as it is in real life.

This paper intends to look at how social semiotics could help improve visual literacy. It focuses on advertizing as it is an industry that exploits visuals often. Improving visual literacy in this work environment might aid designers in advertizing to design better visuals that together with language create better advertisements. This ability of social semiotics to improve visual literacy is exemplified through an analysis of two visuals from commercial bank brochures.

Visual Literacy

Literacy refers to the ability to gain knowledge (Mayer, 2000, p. 363). The use of the word visual in visual literacy is far from precise as not only visuals but even written language is seen with the eyes. However, convention has set visual literacy to refer only to visuals. The term visual literacy was coined by John Debes in 1969 (Branch, 2000, p. 381) and as visual literacy covers many areas, many definitions for it have been forwarded, as noted by Messaris & Moriarty (2005) and Branch (2000). These definitions identify two points, namely the ability to understand visuals and the ability to create visuals (Messaris & Moriarty, 2005, p. 482). This is akin to reading and writing in language literacy (Messaris & Moriarty, 2005, p. 482).

To be visually literate, people must be able to make sense of visuals and be enabled to make sense with visuals (Mayer, 2000, p. 364). As in reading and writing, visual literacy can and must be taught and learnt as it helps people enrich their cognition and critical thinking (Messaris & Moriarty, 2005, pp. 482-483). Visually literate people should know how to decipher visuals but also how to create visuals. To achieve this, visual literacy deserves as much study, experience and understanding as other literacies (Ryan & Conover, 2004, p. 7).

Visuals and Advertizing

Visuals are varied and consist of drawings, paintings, pictures and include charts, diagrams, graphs and much more in any surface like paper or digital from any source from pencil to paint to electronic and can be moving or non-moving, interactive or non-interactive. Ryan & Conover (2004, p. 19) mention that people are visually predisposed and Oakland (Cited in Mestre et al, 1999, p. 35) writes that sight contributes to 75% of learning that gives it an undeniable importance to people.

As such, visuals have much potential and their role in communication is in theory unlimited. Barthes (Cited in Kress & van Leeuwen, 1996) was among the earliest to recognize the role of visuals in communication but believed their meaning is set by language because visuals are very fluid and with many possible meanings that are only stabilized by language. Yet, Kress & van Leeuwen (1996) question this idea. They believe that both modes are connected but independently bring meaning to communication, as each is organized and structured differently.

Utilizing visuals brings many benefits. They invoke the three dimensionality sensed in the perception of objects that makes them suitable for depicting objects in space and their physical features (Stöckl, 2004, p. 17). Visuals are easier to memorize as they tap into emotions to provide direct sensory input that attracts people to them, as proven by psychological experiments (Stöckl, 2004, p. 17). These benefits do not decrease the importance of language. Both language and visuals are interwined and to produce and understand one mode presupposes the other mode due to their close ties in communication and human cognition (Stöckl, 2004, p. 18). Both modes work together to create holistic texts (Eckrammer, 2004, Stöckl, 2004).

Visuals are becoming more prominent in communication (Fairclough, 1989, Mayer, 2000, Messaris & Moriarty, 2005, van Leeuwen, 2005). This is reflected in advertizing and as early as 1989 Fairclough (1989) mentioned that visuals were becoming frequent in advertisements. This reverses the earlier trend where language dominated advertizing (Williams cited in Rampley, 2005, p. 15). Early advertisements used language to sell but advertisements after the 1920s started using visuals that complemented the language used and from the 1960s visuals in advertisements became more cryptic that invited consumers to decipher what the visuals meant in relation to the language used (Leiss, Kline & Jhally, 1990, p. 208).

Lippmann (Cited in Lester, 2003, p. 71) says visuals are the surest way to convey an idea that is supported by Wilcox & Nolte (1997). They cite Wanta whose research discovered that people perceive articles with visuals as more important than those without visuals. If this is so for articles, then it might also be true for advertisements. Advertisers have taken advantage of visuals as they need to use the trends in society to advertize their products or services so consumers notice their advertisements. Visuals have many benefits to advertisers but mainly they elicit emotions in consumers besides providing an explicit or implicit link between the products or services advertized and their potential for consumers (Messaris, 1997).

To create such visuals, designers need to have a high level of visual literacy because they need to use their artistic resources to design visuals that interest consumers. They have to exploit these artistic resources to ensure the meaning they intend is the meaning deciphered by consumers. As such, creating visuals becomes more than just aesthetics because designing them is not done in vacuum. This need not make it haphazard. Instead, it is using the artistic resources designers have with market knowledge, such as knowledge of consumers, their needs and wants and their culture to result in good visuals to advertize products or services.

To design visuals without market knowledge risks visuals not achieving their aim or worst, rub consumers the wrong way. It also wastes the resources spent, be it time, energy or money, in designing them. Designers are faced with two complementary tasks in designing visuals. They not only need to know how visuals are created but also how visuals are interpreted that requires them to have a rather high level of visual literacy. While they are trained to create visuals, they might not be trained to interpret visuals. This is where social semiotics helps.

Social Semiotics

The origin of social semiotics is semiotics. Semiotics derives from the Greek word semeion meaning sign (Barton & Hudson, 1997, p. 202). Semiotics has been utilized to study many phenomenons like language, sound or visuals because each phenomenon consists of many signs that are manifested differently. The major contributor to semiotics is Saussure. He saw a sign as having two components- the signified and the signifier. While the signified was the thing itself, the signifier was used to refer to the thing (Barton & Hudson, 1997, p. 203).

For example, the word boy has as its signified a real boy whose signifier is the word boy spelt b-o-y. Saussure writes that the relation between the signified and the signifier is arbitrary. The word boy is in no intrinsic way the best word to refer to a young male human that is known differently in different languages, like budak lelaki in Malay or muchacho in Spanish. Signs are abstract and arbitrary but people have to submit to such signs that have been fixed by convention and practice to communicate. For Saussure, the word boy- the signifier- can only refer to a young male human- the signified- and not other things.

Social semiotics begins from this basic notion to study signs also but in the context of specific social practices and situations (van Leeuwen, 2005, p. xi). Signs become semiotic resources that are used to communicate (van Leeuwen, 2005, p. 3), among which are visuals. The weight of context must not be forgotten as different contexts employ visuals for different purposes in different ways. These social practices and situations influence the way signs or semiotic resources are utilized. For example, architectural diagrams might simplify a building's components and use black and white colors but clothing advertisements might show each curve and crease of a gown in vivid colors as their purposes and end users differ.

Designers and consumers can benefit from social semiotics. Social semiotics is commonly employed to deconstruct visuals but it has not been employed much to construct visuals. Consumers could deconstruct visuals to discover their meanings. They try to find out what explicit and implicit meanings advertisers wish to convey through visuals. Yet, why should social semiotics not be used for constructing visuals? Designers can exploit the many visual elements to convey the meaning they desire to consumers through visuals. This is done with market knowledge as visuals might not always be interpreted in the same way in distinct contexts. Market knowledge is inevitable in designing visuals that speak to consumers so they are read in the manner designers intend them to be read.

Benefits of Social Semiotics

Social semiotics is useful to advertizing because a lot of research has been done to discover and develop what is known about visuals. Research by Kress & van Leeuwen (2001, 1996), Eckrammer (2004), Stöckl (2004) and van Leeuwen (2005) have contributed to understanding how visuals are used, whether alone or in the company of other modes. Such research is multidisciplinary because social semioticians do not look exclusively at visuals in one field but in many fields. This results in methods pertinent to visuals from different fields. For example, Kress & van Leeuwen (2001, 1996) employed their method of visual analysis to visuals from diverse sources such as books, compact disks, magazines, newspapers and web pages. This ensures their method and the interpretation based on it are reliable and valid.

Another benefit of social semiotics is that it clearly relates form to function. It uses aesthetic guidelines that also function as meaning cues (Messaris & Moriarty, 2005, p. 487). For example, the eye level angle can be created by manipulating the camera but its use means characters in the visuals are social equals of viewers of the visuals (Kress & van Leeuwen, 1996).

This form-function link lets designers create the visuals they need with the artistic resources they have. Social semiotics helps designers predict how visuals are interpreted and demands designers to be critical when using their artistic resources. This could improve their visual literacy as designing visuals does not become merely aesthetic or haphazard but a critical and informed activity sharpened by designers' abilities to understand and create visuals.

Yet, semiotics and by extension social semiotics have been guilty of using elaborate concepts, theories and vocabulary that alienate those outside it (Rose, 2007, p. 104). It also consists of diverse methods and designers must be aware of the strengths and weaknesses of the method they chose to interpret visuals. Besides that, different people might interpret visuals differently, using the same method or different methods (Rose, 2007).

Social semiotics tries to circumvent this by basing itself on context and it is not abstract but concrete. Its methods and the interpretations based on these methods are discovered and developed based on the use of visuals in different but real life contexts. Social semiotics admits that visuals are dynamic and are reflexive of their context. Such contextualized information helps designers create visuals that best target consumers.

However, social semiotics does not give static and universal interpretations. It recognizes that change is inherent in society and hence in the way it creates and understands visuals. Designers must not look to social semiotics as providing them rules to design visuals but as providing them guidance to do so. It does not intend to replace artistic resources but becomes a valuable tool for designers because it informs them how images might be interpreted in context. Designers can and should be encouraged to use visuals creatively and individually to result in new and unique visuals. They must not be bound by social semiotics until it limits this potential.

Social Semiotics in Practice

Two visuals from two commercial bank brochures are now analyzed using social semiotics. The approach used is from Kress & van Leeuwen (1996). Their method recognizes the importance of communication through visuals. It is meant only for visuals and considers the many elements in visuals that imply meaning. This is not available in Chandler's (2007) method of semiotic analysis that can be used for any mode and is not as detailed and systematic as that of Kress & van Leeuwen (1996).

Although Kress & van Leeuwen (1996) write that visuals have three types of relations, this paper only considers their interactive interactions. Interactive interactions are the social relations implied between characters in visuals and viewers of visuals through angle, image act and field of vision (Kress & van Leeuwen, 1996). Knowing the interactive interactions permits designers to design visuals that can imply different relations between the product or service advertized and consumers.

Each element consists of different components that if exploited differently can imply different social relations between characters and viewers. These relations are implied because they do not exist in real life but are given the illusion of existing through these three elements. Angle, image act and field of vision do not contain any intrinsic meaning but they are given meaning by people who use them in different contexts. This goes back to the importance of context in determining the meaning of semiotic resources. They are not permanent and can change in time provided that people adapt to such change and technology permits these elements to be used differently.

There are three types of angle, namely the high angle, the low angle and the eye level angle, depending on the height of the camera. The higher or lower the angle, the wider the power distance between characters and viewers. When characters look down on viewers via the low angle, it implies that they are in a higher physical position and by extension they have more power than viewers (Kress & van Leeuwen, 1996, p. 146). It puts characters in position to order viewers as characters are the superiors in this interaction.

The opposite occurs when characters look up to viewers via the high angle. Only the eye level angle does not involve any power difference between characters and viewers as both are situated as if they are facing each other as social equals (Kress & van Leeuwen, 1996, p. 146). It must be remembered that these three angles are not discrete components but exist on a continuum where the camera can create different degrees of the same angle.

Image act depicts the imaginary relations between characters and viewers (Kress & van Leeuwen, 1996, p. 122) and there are two, namely demand and offer. Demand visuals portray characters establishing contact through gaze and gesture(s) with viewers. Characters seem to look at or point to viewers, as if they know viewers look at them. Offer visuals portray characters not establishing contact with viewers. Viewers can see characters but characters can not see them. Demand visuals engage characters and viewers but offer visuals disengage them.

Field of vision is the type of shot taken, depending on the camera's distance from characters (Kress & van Leeuwen, 1996). Like angle, shots exist on a continuum but Hall (Cited in Kress & van Leeuwen, 1996, p. 131) identifies six shots in relation to the human figure.

The closer the camera is to characters, the closer the implied social ties among characters and viewers. At intimate distance the face or head is seen, at close personal distance the head and shoulders are seen, at far personal distance the person is seen from the waist up, at close social distance the whole person is seen, at far social distance the whole person with some space around him or her is seen and at public distance the torsos of at least four or five persons are seen (Hall cited in Kress & van Leeuwen, 1996, p. 131). These distances hint at the level of intimacy between characters and viewers based on their physical distance from each other. For example, a mother and a child can come close enough to hug but not a president and a citizen who can only come close enough to shake hands.

What interactive interactions are created and maintained by the visuals from the two commercial bank brochures? The two visuals are named V1 and V2 for ease of reference.

Figure 1: V1

Appendix 2

Figure 2: V2

Both are present on brochures for deposit products from two different Malaysian commercial banks. V1 has three human characters in it- one is a male and two are females. They seem to be Chinese and are wearing rather formal clothing. The characters sit slightly sideways, next to each other. The first female character is smiling and is holding a mobile phone in her right hand and the female and male characters behind her are also smiling. In front of them is a table with three glasses, a file and a plate and behind them is a white wall. The first female character stares into space but her companions seem to look at her mobile phone or the table. All characters are seen from the waist up. V2 has four human characters in it- two are males and two are females. They seem to be multiracial and are wearing smart casual clothing. They are smiling and sit facing each other around a round table that has cups on it and behind them is a white wall. All characters are seen from the waist up.

Both V1 and V2 use the eye level angle. This implies characters are social equals of viewers. Viewers are to identify with characters as the latter are presented as people like the former. As both are social equals, they must belong to the same socioeconomic group. They should then have the same needs and wants. This angle increases viewers' identification with characters and the scenes depicted. It invites viewers to see how people like them are enjoying due to using the product. As the product is non-tangible, viewers cannot see it per se but they can see the results of acquiring it. The eye level angle also implies the scenes in V1 and V2 are likely to be experienced by viewers in their daily lives. The visuals create a link between product acquisition and enjoyment with friends. As people like viewers seem to benefit from the product, then viewers might be motivated to acquire the product so they can realize such scenes in their own lives.

Both visuals use the offer image act. Characters are in a world of their own and they do not acknowledge viewers' presence. They seem to say 'We offer you this product so you can be like us, relaxing among friends'. This matches the offer in language also as it offers the features and benefits of the product to consumers. In this way, both modes help sustain the promotional value of the brochure. Offer visuals might look as if they are more distant or impersonal than demand visuals but in a way, they help make the product more desirable. The facial and bodily gestures strengthen the offer because in V1 and V2 characters are smiling and sit relaxed among friends. By not acknowledging viewers and being in their own world, characters erect a barrier between both of them, between those who have and do not have the product.

V1 and V2 use the far personal distance. Kress & van Leeuwen (1996, p. 130) write that that this is the distance where subjects of personal interest and involvement are discussed. It is ideal for commercial bank brochures as deposits are of personal interest to most people that need their involvement to grow. As the human figure is seen from the waist up, characters and viewers can touch fingers if they both extend their arms (Kress & van Leeuwen, 1996, p. 130) that is the distance between friends who are talking face to face. Characters are portrayed as friends of viewers and this position empowers them to advice viewers to acquire the product. As friends often have their best interest at heart, the product gains credibility because if it was not good, then characters as friends would not be shown relaxing in the visuals. But this is not so. They can relax as the product takes care of their finances. By implicature, the product must be of value that motivates characters to promote it to viewers.

The interactive interactions desire to make characters and viewers closer. Viewers are made to believe characters are their friends through the far personal distance who have the same concerns as they do through the eye level angle. Why is this personal relation important? Banks have been considered as distant and impersonal entities that lack a human face. Being in the service industry, they lack tangible products that differenciate one commercial bank from another commercial bank (Lane, King & Russell, 2005). As brochures are among the first points of contact between commercial banks and consumers, they can be used to give a good impression of the banks. The visuals are the most noticeable part of the brochures as they occupy about 2/3 of the brochures' cover. These visuals give a human face literally and metaphorically to the product.

Human characters humanize the product and show the non-monetary results of acquiring it by attaching an emotional value, as in V1 and V2 that employ a group of friends socializing. By identifying with characters, viewers are made to believe that even they can be like characters once they acquire the product. Banks also cannot possibly meet each consumer. By personalizing the visuals, they imply the type of relation they (want to) have with their customers. The visuals become their proxies to establish good relations with consumers. They contribute to give an idea of the bank as understanding the demands of customers in the visuals and by extension, of consumers viewing the visuals. V1 and V2 socialize consumers into their role as satisfied customers even before they obtain the product. Their central message is the desireability of the product and the approachability of the bank.

Knowing the message the visuals wish to convey could help designers as they design visuals. By including the elements of interactive interactions, they know which form matches which function. Together with other elements like color and context, they can create visuals that are not only pleasing to the eye but also with the meanings they intend. This goes towards improving their visual literacy as they can now exploit the visual elements better for advertizing purposes. This helps designers improve the quality of their work and contributes to increasing their value in an organization.

Conclusion

This paper has forwarded social semiotics as a tool to improve visual literacy in advertizing. Although social semiotics can help in design, it must not be depended on solely. Designers must bear in mind that social semiotics is but a tool for design that should not hamper their creativity and individuality. Instead, it should be used in tandem with these capabilities. They should tailor their artistic resources to be more sensitive as to how, why and what visuals say in line with what they know about consumers. Adopting such an approach contributes to improving their visual literacy. This sensitivity is pertinent if designers want to create visuals that transmit the meaning they intend to consumers. This, then, requires an investment in education so they can better match their artistic resources with the meanings they want to convey.

Colophon

References

Barton, Edwin J. & Hudson, Glenda A. (1997). A Contemporary Guide to Literary Terms. Boston: Houghton Mifflin.

Branch, Robert Maribe. (2000). A Taxonomy of Visual Literacy. In Ann Watts Pailliotet & Peter B. Mosenthal (Eds.), Reconceptualizing Literacy in the Media Age (pp. 377-402). Advances in Reading/Language Research, Volume 7. Stamford: Jai Press Inc.

Chandler, Dan. (2007). Semiotics: The Basics. London: Routledge.

Eckkrammer, Eva Martha. (2004). Drawing on theories of inter-semiotic layering to analyse multimodality in medical self-counselling texts and hypertexts. In Eija Ventola, Cassily Charles & Martin Kaltenbacher (Eds.), Perspectives on Multimodality (pp. 211-226). Document Design Companion Series, Volume 6. Amsterdam: John Benjamins Publishing Company.

Fairclough, Norman. (1989). Language and Power. Language in Social Life Series. London: Longman.

Kress, Gunther & van Leeuwen, Theo. (1996). Reading Images. London: Routledge.

________. (2001). Multimodal Discourse: The Modes and Media of Contemporary Communication. London: Arnold.

Lane, W. Ronald, King, Karen Whitehill & Russell, J. Thomas. (2005). Kleppner's Advertising Procedure. Sixteenth Edition. New Jersey: Pearson Prentice Hall.

Leiss, William, Kline, Stephen & Jhally, Sut. (1990). Social Communication in Advertising. Second Edition. London: Routledge.

Lester, Paul Martin. (2003). Visual Communication. Third Edition. Belmont: Thomson.

Mayer, Richard E. (2000). The Challenge of Multimedia Literacy. In Ann Watts Pailliotet & Peter B. Mosenthal (Eds.), Reconceptualizing Literacy in the Media Age (pp. 363-376). Advances in Reading/Language Research, Volume 7. Stamford: Jai Press Inc.

Messaris, Paul. (1997). Visual Persuasion: The Role of Images in Advertising. Thousand Oaks: Sage Publications.

Messaris, Paul & Moriarty, Sandra. (2005). Visual Literacy Theory. In Ken Smith, Sandra Moriarty, Gretchen Barbatsis & Keith Kenney (Eds.), Handbook of Visual Communication (pp. 481-502). Mahwah: Lawrence Erlbaum Associates, Publishers.

Mestre, Michel, Stainer, Alan, Stainer, Lorice & Strom, Bill. (1999). Visual communications- the Japanese experience. Corporate Communications, 5:1, 34-41.

Rampley, Matthew. (2005). Visual Culture and the Meaning of Culture. In Matthew Rampley (Ed.), Exploring Visual Culture (pp. 5-17). Edinburgh: Edinburgh University Press.

Rose, Gillain. (2007). Visual Methodologies. Second Edition. London: Sage Publications.

Ryan, William & Conover, Theodore. (2004). Graphic Communications Today. Fourth Edition. New York: Thomson Delmar Learning.

Stöckl, Hartmut. (2004). In between modes. In Eija Ventola, Cassily Charles & Martin Kaltenbacher (Eds.), Perspectives on Multimodality (pp. 9-30). Document Design Companion Series, Volume 6. Amsterdam: John Benjamins Publishing Company.

van Leeuwen, Theo. (2005). Introducing Social Semiotics. Oxon: Routledge.

PLEASE CLICK HERE TO READ THE ARTICLE IN PRINTER-FRIENDLY VERSION.

Language of Mass Media: A Study Based on Malayalam Broadcasts - A Doctoral Dissertation | Resisting Change through Individual Heroism - Chinua Achebe's Things Fall Apart | Social Semiotics as a Tool for Visual Literacy | Mastering Tenses Creatively | History of Growth and Reforms of British Military Administration in India, 1848-1949 | Communication and Inarticulation -
Symbols and Images in the Plays of Eugene O'Neill | The Impact of Gender on Proficiency, Attitude and Social Class of Pre-University Students in Mysore within the Framework of Learners' Multilingualism | HOME PAGE of September 2008 Issue | HOME PAGE | CONTACT EDITOR