Online Magazine of the Visual Narrative – ISSN 1780-678X
Issue 1. Cognitive Narratology
The cognitivist approach to film in the light of systemic-functional theory: a changing of the guards?
Author: Sofie De Grauwe
Abstract (E): For several years now, Bordwell and others in the cognitivist film approach have been criticising the rule of ‘Grand Theory’, i.e. psychoanalytic and culturalist film theory. They claim that the cognitivist approach is better qualified to study film. In this article, however, it is claimed that the systemic-functional approach is a better alternative. In their reaction to ‘Grand Theory’, the cognitivists reject any general theory of film, and favour empirical research. This reaction is too extreme. In the systemic-functional theory, the combination of a top-down and a bottom-up approach is considered necessary for theorising. In this article, the foundations of systemic-functional analysis of film are defended, i.e. the claim that film is a semiotic system. On the basis of this argument, the cognitive approach is criticised. It is hoped that the present article may start a dialectical discussion between the cognitivist and the systemic-functional approach, to determine which is a better alternative to ‘Grand Theory’.
Keywords: Film theory, Cognitivism, Systemic-functional theory, Semiotic system
In this article, I will discuss the metatheoretical foundations of the systemic-functional analysis of film, and present the theoretical framework developed in the systemic-functional approach. Systemic-functional theory will be critically compared with the cognitivist approach to film.
In the cognitivist approach, a general theory of film is rejected in favour of empirical research. Systemic-functional analysis, on the other hand, combines a top-down and a bottom-up approach.
The cognitivist approach is advocated by, among others, David Bordwell. He is a film theorist, historian and critic. In the book Post-Theory, edited and introduced by Bordwell and Noel Carroll, Bordwell elaborates on the cognitivist concepts he treated in his book Making Meaning. In the cognitivist approach, the focus is on cognitive processes, human actions and perceptions.
The systemic-functional analysis is based on the Systemic Functional Linguistic (SFL) theory of Halliday. This is a linguistic theory, in which it is claimed that the language system does not work according to meaningless, purely formal rules. Instead, it is functionally motivated. In addition, the language system is not static, but dynamic. It is subject to change: as a social and historical construction, it changes with social and historical circumstances.
The application of this linguistic theory to the non-linguistic mode of film is based on the (substantiated) thesis that film is a semiotic system. On the basis of this thesis, it is possible to investigate whether and in which aspects the linguistic system as a semiotic system is comparable to the semiotic system of film.
In the first section, the cognitivist approach as presented in the book Post-Theory will be presented. As this article cannot treat the cognitivist ideas of both Post-Theory and Making Meaning in some depth, the latter is only marginally referred to. In the second section, the semiotic nature of film will be discussed, and the cognitivist stance will be critically analysed from a semiotic viewpoint. Finally, in the third section, the systemic-functional approach to film will be presented.
1. Cognitivist approach to film
Post-Theory presents the defence of a cognitivist approach to film. In two introductory essays, David Bordwell and Noel Carroll position the cognitivist approach against contemporary film theory. In the cognitivist approach, the focus is on general cognitive processes, human actions and perceptions. Bordwell and Carroll call for a more rigorous theorising, based on empirical research and not encompassed within a general film theory. The rest of the book consists of articles presenting this kind of theorising in connection with a small-scale problem with regard to film.
In this section, I will restrict myself to presenting the cognitivist approach. In section 2, it will be subjected to a critical analysis.
In Post-Theory, the cognitivist approach is situated against what is called the ‘Grand Theory’ of the two major contemporary approaches in film studies: ‘subject-position theory’ (or psychoanalytic film theory) and ‘culturalism’. Instead of the top-down approach of Grand Theory, or Theory in short, Bordwell and Carroll advocate a bottom-up approach.
Bordwell and Carroll have various objections to a top-down approach. Both refrain from a general theory out of fear of the supposed limiting and doctrinal character of one single unified theory. Carroll claims theory should be dialectical, i.e. hypotheses should be continually exposed to intense criticism. Carroll concedes that there should be ‘different levels of generality and abstraction’ (Bordwell & Carroll (B&C) 1996: 39) in film theorising, but they should not be subsumed under a ‘single general theory’ (B&C 1996: 39) with ‘presuppositions about either the nature or function of cinema’ (B&C 1996: 39), as this would involve a ‘totalizing system’ and a ‘monolithic conception of film theory’ (B&C 1996: 41). In addition, the specificity of individual films would be ignored in a general theory: ‘every film comes out of the standard-issue sausage machine, looking and smelling the same’ (B&C 1996: 43). Carroll also rejects a general theory out of fear of essentialist film theory. With this, he means film theory which is ‘committed to medium specificity in such a way that whatever counts as theorising about film must be connected to features of the medium that are thought to be uniquely or essentially cinematic’ (B&C 1996: 39). Carroll considers such demands of theoretical purity as limiting.
Both Bordwell and Carroll object to the kind of theorising as conducted in Grand Theory. It is ‘allegorical’ and ‘associational’ (B&C 1996: 23) instead of using causal co
A bottom-up approach would not carry all these shortcomings. It would encourage a thorough study of particular phenomena, without the doctrinal constraints of a general theory. An empirical approach would not exclude, but instead, paradoxically, promote rigorous film theorising, ‘so long as it involves the production of generalizations or general explanations or general taxonomies and concepts about film practice’ (B&C 1996: 39). Although there should be different levels of generality and abstraction in film theorising, the most general level, i.e. a general theory, should be postponed, according to Carroll, because ‘we do not yet know enough’ (B&C 1996: 58).
Bordwell and Peterson both object to semiotic and (structuralist) linguistic approaches to film. In structuralist linguistics, borrowed by Grand Theory, langue is defined as a ‘closed structure’ (B&C 1996: 17). In a semiotic discourse, film is considered analogous to language. This analogy could go quite far, which provoked accusations of ‘linguistic imperialism’. Bordwell, for example, attacks Hall’s extreme linguistic analogy. According to Hall, ‘”televisual discourse” is subject to all the complex formal ‘rules’ by which language signifies”‘ (B&C 1996: 18). Bordwell claims that this kind of extreme analogy prevents the study of the medium-specific elements of film, i.e. their aural and visual aspects. He asks for a defence of the claim that film is ‘plausibly analogous to language’ (B&C 1996: 18). In the next section, I will try to do this.
Both Peterson and Bordwell object to a code-based model of communication, in which messages are encoded and decoded on the basis of ‘a system of conventions shared by the users of the code’ (B&C 1996: 112). Peterson opts for a method in which ‘reasoning and problem solving are not cut to a single universal pattern, but are context-bound’ (B&C 1996: 119).
A last objection to a semiotic discourse pertains to the supposedly arbitrary convention of the sign. Peterson claims that in semiotic theory the link between signifier and signified arbitrary and conventional is. This explains why the affinity for conventional art is inherent in semiotic theory. The latter would have great difficulty in studying avant-garde. According to Bordwell, a great difference between verbal language and film lies in the easiness of their learning. Visual ‘codes’ are much easier to learn than linguistic ‘codes’, because the visual signs (if there are such things) are motivated, while linguistic signs are supposed to be arbitrary. This difference makes cinematic codes ‘significantly different from the codes governing language-based sign systems’ (B&C 1996: 95).
2. The semiotic nature of film language
In this section, the semiotic nature of film language will be studied. Firstly, I will investigate whether the criteria for semiotic systems can be applied to film, its visual mode in particular (section 2.1). This will be done on the basis of Paul Thibault’s Re-reading Saussure (1997). Thibault is a linguist working in the systemic-functional tradition, which started in the eighties under the impulse of Michael Halliday. In his book Re-reading Saussure, he presents (among others) two criteria for the assessment of semiotic systems. In section 2.2, the claims of the cognitive approach to film are evaluated from the systemic-functional perspective, presented in section 2.1.
2.1 Criteria for semiotic systems
To assess the semiotic nature of film language, two criteria can be used. These are stratification and instantiation or schematicity.
In a semiotic system, signs are established through a signifying relation between their internal strata. These strata are the signified and the signifier, themselves internally stratified. As extensive study has been done on the stratification of the linguistic sign, I will first present stratification in the linguistic system, and then investigate if such a stratification can be found in film, more specifically the visual mode.
Figure 1 displays Thibault’s (1997: 230) analysis of the linguistic sign and its relation with thought and sound. Perhaps an adjustment could be made to this representation (see Figure 2), in which the level of the phonic chain would be subdivided into the stratum of the phonic chain (cf. phonetics, or, more generally, Hjelmslev’s expression-substance) and the stratum of the delimited phonological units (cf. phonology, or Hjelmslev’s expression-form). Apart from making this level parallel to the upper level of the signified, it would make clearer the constructive effort on the side of phonology and its systemic basis. This will become more evident in the following discussion of Figure 2.
linguistically construed thought-substance
linguistically construed thought-substance
linguistically construed thought-substance
signified 1 signified 2 signified 3
signifier 1 signifier 2 signifier 3
linguistically construed thought-substance
Figure 2 displays the constructed nature of both signified and signifier. These two are constructed through a two-way relation with each other. On the one hand, the signifier, which is construed through the interaction of phonic chain and delimited phonological units, symbolically construes the level of the signified. In this construction, functional role relations interact with delimited grammatical units to produce the signified. On the other hand, the signified, construed through the interaction of these functional relations and grammatical units, symbolically construes the signifier, through the interaction of phonic chain and delimited phonological units. These processes are carried out simultaneously in the construction of the sign. The result of this signification process is twofold. The linguistic sign is bordered by thought on the side of the signified, and by sound on the side of the signifier. Through the interaction of the signified and amorphic thought in the signifying process, a linguistically construed thought substance is construed. In a parallel way, a linguistically construed phonological substance is construed through the interaction of physical-material sound and the signifier.
The same strata can b
signifier (visual-graphic images)
Figure 3: Tristratal organization of the visual-graphic sign (experiental meaning only)
On the level of the signified, grammatical classes realise semantic functions. Such grammatical classes are vectors and volumes. Vectors are formed on the basis of oblique lines, which convey a sense of directionality. These lines may be formed by gazes, objects (like arms) etc. Vectors semantically realise processes, representing a kind of action. However, not all processes are construed as vectors. Static, conceptual processes (as opposed to dynamic, narrative processes) are realised by juxtaposition.
These processes connect participants. Participants are semantic roles, such as Actor, Goal, etc. They are grammatically construed by volumes, which are formed on the basis of distinct entities in the visual composition.
As in the linguistic system, there is a two-way relation between signified and signifier. The signified cannot exist independently of the signifier. The visual image, here used in a sense parallel to the linguistic acoustic image or signifier, is necessary for the construction of meaning. Without the visual units established by the visual image, no grammatical units can be established, e.g. there can be no vectors if there are no lines.
Similarly, the signifier cannot exist independently of the signified. There is no reason to construct units with meaning-making potential (signifiers), if they are not used to make meaning (through their combination with the level of the signified). For example, if there are no vectors, there is no reason to establish lines on the basis of a continuum.
Both signifier and signified are meaning-making potentials, symbolically construed in and through the other. Only through this simultaneous two-way construal relation, can signs be formed, and thus meaning be made.
A semiotic system consists of different levels of specificity. At the most abstract level, one finds schemata. The most specific level is the level of the instances. In between, there is a graded series of levels of specificity. Again, I will first present a discussion of schematicity in the linguistic system, after which its presence in film will be studied.
In Table 1 (Thibault 1997: 72), Thibault’s representation of the different schematicity levels of the linguistic system is rendered. Langue1 is the most abstract level, it is the level of ‘schematic categories’. It is the level of pure values, which are negatively defined through differences. The two orders of difference, i.e. the orders of signified and signifier, are not yet cross-coupled in the signification process.
Langue2 is the level of ‘prototypical instances of forms’, in the sense of ‘regular models’ (one may argue that Thibault’s terminology here is not consistent with his theory. ‘Categories’ seems more suitable for the level of langue2, while ‘instances’ is more commonly used at the level of parole. Despite this problem, I will continue to employ Thibault’s terminology.). Forms are relatively positively defined through opposition and contrast with each other in syntagmatic and associative groups. The structuring principles of the syntagmatic and associative relations make possible the cross-coupling of signified and signifier. This is so because through these rel
Langue3 is the level of the typical uses of the forms according to the text type. On this level the typical co-patterning of forms and the cross-coupling with other semiotic domains are studied.
In such a multi-layered system, the meaning of an instance is dependent on the schema it corresponds to. It is determined on the basis of synchronic identity with the schema, i.e. the instance is judged to belong to a certain linguistic type on the basis of ‘stable and invariant features which distinguish this unit from others in the same language system’ (Thibault 1997: 306). These features are independent of situational material and semantic variations.
The schema may be defined as ‘both sufficiently general and sufficiently abstract to be able to define properties and attributes which are shared by the prototypical members of the category in question’ (Thibault 1997: 308 I have added the word ‘prototypical’ here to reconcile Thibault’s definition with the notion of prototypicality). Prototypical instantiations of schematic categories are defined by Thibault as ‘fully conform[ing] to all of the criteria established by the higher-order schema[ta]’ (Thibault 1997: 308). Non-prototypical instantiations can be defined as partially conforming to all the criteria or (fully or partially) conforming only to some criteria (this definition has been inspired by Thibault). This partial conformity is a matter of degree.
Thibault emphasises that schemata and prototypical instantiations cannot be equated, they belong to different levels of specificity. Being the most abstract, schemata belong to langue1, while the more specific prototypical instantiations belong to langue2. As the latter refer to ‘typical grammatical patterns in the language system’ (Thibault 1997: 311), which are established through both syntagmatic and paradigmatic or associative relations, these prototypical instantiations must be situated at the level of langue2.
The principle of schematicity is also present in film. In the visual system, too, there are higher-order visual categories, which are instantiated in prototypical and non-prototypical instances. One can design an example in which the role of syntagmatic and paradigmatic relations in the construction of meaning becomes clear. This example is presented in Table 2.
In this figure, terms like transaction and non-transaction are situated on the level of the visual langue1. Transaction and non-transaction are comparable with respectively linguistic transitivity and intransitivity. In a transaction, an action (in a general sense) of a participant is ‘carried through’ to another participant. In a non-transaction, the action is not carried through.
On the level of the visual langue2, more and less prototypical instantiations of these schematic terms can be identified, such as [Actor-process-Goal] (The signs ‘[‘ and ‘]’ denote that the roles and processes they encompass are combined in one image ). These instantiations are the result of the combination of syntagmatic and paradigmatic instantiations of the schematic terms. A prototypical instantiation of ‘transaction’ along the syntagmatic axe is [Agent-process-Affected], as opposed to the non-prototypical absolute transaction [Agent-process-(implied Affected)]. The combination of the former with the prototypical action results in the already mentioned [Actor-process-Goal]. In between the ‘explicit’ transactions and the absolute transactions, instantiations such as [Actor-process]-[Goal] can be situated. In these constructions, an image of an absolute transaction (with e.g. a character pointing a gun) is followed by an image of the Goal (e.g. the character whom the gun is pointed at).
As said before, the role of the syntagmatic and paradigmatic relations becomes clear in this example. Meaningful patterns are construed by combining signs, i.e. by forming syntagmatic relations between them. This combination is possible through the association of these signs with other, similar signs and their combinations, i.e. through their paradigmatic relations. For example, the interpretation of an image in which a character points a gun at another character as being an instance of the syntagmatic combination [Actor-process-Goal] is only possible through the association of the characters and the process (and the relation between these two) with similar characters and processes and the relations between the latter two. Only then can the categories of Actor and Goal be constructed, and can they be combined into meaningful patterns.
2.2. Critical Analysis of the Cognitivist Approach
In this section, a critical discussion of the cognitivist approach to film will be presented on the basis of systemic-functional theory. Firstly, the advantages of the combination of a top-down and a bottom-up are presented. Secondly, the postulation of an internal system in film is defended. Finally, the open-endedness of the systemic-functional system is discussed.
2.2.1 Top-Down Vs. Bottom-Up
The postulation of an internal system of film (as in a systemic-functional approach) implies a top-down approach. However, this is combined with a bottom-up approach, as empirical research is necessary to confirm the claims made by systemic-functional theory. In this section, the advantages of the combination of a top-down and a bottom-up approach are presented.
Unlike the cognitivist approach, systemic-functional theory advocates the combination of a top-down and a bottom-up approach. Both are necessary to correct and enrich each other, so that the top-down approach does not become doctrinal, and the bottom-up approach does not keep working with the same questions.
The systemic-functional theory shows that a general theory is possible that is not essentialist nor doctrinal. In systemic-functional theory, both differences and similarities between film and other media are investigated. Secondly, the systemic-functional system is an open, rather than a closed and limiting system, liable to change and adaptable in its uses. Bordwell is right in asserting that empiricism makes possible a thorough study of particular phenomena. To exclude the most abstract and general layers in theorising, however, is also limiting (instead of pluralistic). It is on these levels that the deeper connections between particular phenomena and the nature of film can be investigated, which, according to me, is also an interesting object of study, which should certainly not be excluded from theorising. On this level, a top-down approach can be very useful. At the same time, the latter should be checked with empirical facts, so that criticism and empirical research are not excluded when studying general theory.
This means that Carroll is right when he says that film theorising should follow the dialectical model. He does not, however, go far enough in his conclusions. The dialectical model is also applicable to general theory, and it should be applied to general theory, to prevent it from becoming doct
Apart from opening an interesting theoretical domain of abstract questions, general theory is also beneficial for empirical research. What makes theories interesting, is that they can be challenged, so that theory can progress. This does not only apply to small-scale theories, but also to general theories. If you refuse to formulate a general theory, you continue to do research on the basis of the previous general theory. This is conceded by Carroll, when he says that the cognitivist approach tries to answer questions formulated by psychoanalytic film theory. Only by formulating a new general theory, can theorising be offered a new perspective.
Systemic-functional theory satisfies the requirements of theorising as formulated by Carroll and Bordwell. In SFT, theory is checked with the empirical facts of films. Theorising occurs on the basis of logical instead of associational reasoning, it is precise, it does not use bricolage. This shows that not all general theories are vague and doctrinal. (it is not because one general theory was limiting that you should be afraid of every general theory)
Carroll’s argument that ‘we do not yet know enough’ (B&C 1996: 58) to formulate a general theory is weak. We never know enough, theoretical hypotheses are a challenge to overcome our ignorance. New hypotheses present new perspectives to investigate, and overhaul through investigation if necessary. ‘Perhaps one day’ (B&C 1996: 58) we will be able to formulate a general theory, claims Carroll. However, theories are not formulated when all or a lot of the answers are found, but they are used in order to find the answers. It can be added that Carroll refers to science as an example to follow in film theorising. Yet, science always had theories, they were not postponed, even if they did not know enough or did not have all the answers. In addition, Carroll does not seem to show much hesitation in borrowing general theories from other fields.
2.2.2 The Systemic-Functional ‘System’
Bordwell and Peterson reproach semiotic approaches with what can be called ‘linguistic imperialism’, i.e. the (complete) projection of the linguistic system onto film. In this section, the postulation of an internal system of film are defended, and its advantages pointed out.
In part 2.1, we have seen that film can be regarded as a semiotic system, as it fulfils the criteria for semiotic systems. Therefore, a semiotic approach in which various semiotic systems are studied both in their particularity and in their similarities with other semiotic systems, appears to be justified. It is true that, in systemic-functional theory, the linguistic system is taken as the starting point for the construction of a theory of the cinematic system. This is so because the linguistic system is the semiotic system best studied. As both language and film are semiotic systems, the results of the study of the linguistic system should have at least some relevance for the study of the cinematic system. However, this does not mean that the linguistic system is uncritically and blindly projected onto the cinematic system. In a systemic-functional approach to film, attention is paid to both meaning relations which are comparable to linguistic meaning relations, and meaning relations which are particular to the cinematic mode. In this way, the particularity of the cinematic mode is respected, and there is no expansion from one mode to the other, such as linguistic imperialism. Therefore, the reproach of linguistic imperialism does not apply to systemic-functional theory.
The consequences of the rejection of an internal system are firstly the division of form and meaning, and secondly the neglect of the filmmaker, and of the communicative function of film in general.
I will not elaborate too much on the division between form and meaning, because this element is not so prominent in Post-Theory as it is in Making Meaning. As an internal system is rejected in the cognitivist approach, film meaning has to be constructed by resorting to external domains such as ‘anthropology, psychology, linguistics, and aesthetics’ (Bordwell 1989: 272). As a result, film form and meaning are separated. This can be seen, for instance, in Bordwell’s ‘historical poetics’, his alternative to contemporary film theory, in which he separates the analysis of form from the analysis of meaning. It is also shown in Carroll’s distinction between formal theories, in which cinematic forms and structures are studied, and social theories of film. In systemic-functional theory, cinematic forms and structures are not studied in a formal theory. Form and function cannot be separated, as they complement each other to form meaningful signs (see above), and cinematic forms and structures are studied in conjunction with their meaning.
A second consequence of the rejection of an internal system of film is the rejection, or at least the neglect, of its communicative function. This can be explained as follows. As an inherent film system is rejected, film is perceived as essentially unordered, and recourse has to be taken to external domains to construct film meaning. As a result, the focus is redirected from film to these external domains. However, the link between the communicators is established by film. By directing the focus away from film, it is also directed away from its communicative function.
However, film is not just a given reality, which would justify a one-way approach in which only interpretation is studied. Film is always constructed by the filmmaker. Therefore, a two-way approach is necessary, which takes into account both production and reception.
In Post-Theory, the code model is rejected by Bordwell and Peterson for the wrong kinds of reasons. In systemic-functional theory, the code model is rejected as well, but this does not imply a rejection of film’s communicative function. The code model is considered wrong in its comparison of communication to telephone transmission: a message, encoded by the sender on one side, is sent through a wire to be decoded by the receiver at the other side. In the systemic-functional approach, on the other hand, communication is realised by agents jointly making meanings in and through available resources and conventions. Thus, the rejection of the code model implies the rejection of one kind of representation of communication, not of communication in se.
2.2.3 Open-Endedness of the System
Many of the objections raised against the use of an internal system in interpretation, amount to a criticism of the supposed ‘closedness’ of the system. This is the case, for example, in Carroll’s claim that, when a system is used, ‘every film comes out of the standard-issue sausage machine, looking and smelling the same’ (B&C 1996: 43). However, this critique does not apply to the system as it is conceived by systemic-functional theory. In this theory, the system is not a closed entity. It is open-ended, as it is socially and historically constructed, which makes it liable to change. A distinction must be made between the system and the individual uses of the system. The system is a meaning resource, employed by individuals in their individual way in the construction of meaning. Thus, the concrete instances which are the result of this construction, are not determined by the system. There is a measure of creating: individuals are left some freedom in the construction of meaning. As regards film, this means that meaning is not controlled by the filmmaker nor the viewer. Just as the filmmaker uses the system in his/her individual way to construct meaning, the viewer uses the system in his/her individual way to construct an interpretation.
The open-endedness is inherent in the system through its inherent associative relations and prototypicality. As syntagmatic relations tend to present ‘an ordered succe
However, the socio-historical relativity of the system does not imply absolute relativity of either the values or the signs of the system. Within the system, values and signs are not random, they are defined by the system. It is not because the system is relative with regard to the socio-historical circumstances, that its being can be relativised.
The open-endedness of the system implies the presence of some principle of chaos in the system. Thus, as claimed by Thibault (1997), system implies both order and chaos. Both principles are necessary for the production of meaning. Firstly, as meaning can only be produced when patterns can be discerned, a principle of order is necessary to produce meaning. Secondly, the production of meaning is only possible if these patterns, rather than becoming rigid, remain adaptable to situations. Therefore, a principle of chaos is necessary too. As they comprise both these principles, systems are excellently suited for the production of meaning.
This account shows that in their reference to linguistics Bordwell and Peterson use terms and notions which are obsolete in contemporary linguistics. Langue is no longer considered a closed system, the rigid code model has been replaced by the notion of open communication systems (see above). The link between signifier and signified in the linguistic sign is considered motivated in systemic-functional linguistic theory, so that the difference in motivation of the sign in verbal language and film is a matter of gradation, not a clear dividing line. In his article ‘Convention, Construction and Cinematic Vision’, Bordwell investigates the problem of the origin of the ‘shot-reverse shot’ sequence in cinema. After exploring the existing positions, viz. natural origin or arbitrary convention, he comes to the conclusion that it is a motivated convention. Unfortunately, Bordwell did not take into account the concept of ‘motivated sign’, so that he had to go through the whole line of reasoning he displays in his article to arrive at his conclusion.
3. The systemic-functional approach to film
In this section, the systemic-functional approach to film is presented in relation to Thibault’s langue2 and langue3. Firstly, I will develop the systemic-functional framework at the level of langue2. For this purpose, Kress & Van Leeuwen’s (K&VL 1996) systemic-functional theory of the visual image will be applied to the moving image. Then, this theory is used to subject Bordwell’s theory to a critical analysis (section 3.1). In section 3.2, the systemic-functional framework is developed at the level of langue3. Both genre and intermodal relations are studied.
3.1 Systemic-Functional Framework for the Visual Mode of Film
The systemic-functional theory of the visual image as developed by K&VL is based on Halliday’s systemic-functional theory. According to Halliday, semiotic modes have three metafunctions: the ideational, interpersonal and textual metafunction.
Firstly, semiotic systems have to represent objects (or represented participants) and relations between them. This is their ideational metafunction. There are two kinds of ideational relations, narrative and conceptual relations. Narrative processes are dynamic processes, directly involving one or two participants. In the visual image, they are represented by vectors. These emanate from volumes, which are semantically construed as participants. They may be directed to other participants, in two-participant constructions. Such processes may be actions, in which an Actor performs an action, possibly directed to a Goal. Other narrative processes are reactions, speech processes, mental processes.
Conceptual representations represent participants in their stable essence. There are three different kinds of conceptual processes, in the visual image, these are classificational, analytical and symbolic processes. As they are static processes, they are not grammatically construed by (dynamic) vectors, but by juxtaposition.
Secondly, semiotic systems have to project social relations between interactive participants (the producer of the (complex) sign and the receiver of it) and between interactive and represented participants. This is their interpersonal metafunction. Because there is a disjunction between the context of production and of reception in film, the social relations between interactive participants are represented instead of enacted. The producer communicates through a (represented) substitute ‘I’ or a character, because he cannot be present. Therefore the viewers are addressed in such a way that they do not have to respond.
There are two systems used at the interpersonal level: ‘interaction’ and modality. The interactional system deals with elements such as social distance and subjective attitude. In this system, relations of power, involvement, social distance etc. are established between the viewer and the represented participants through camera angle, size of frame, etc.
In the visual mode, only one kind of modality can be discerned, viz. epistemic modality. This kind of modality is concerned with the ‘truth value or credibility of [.] statements about the world’ (K&VL 1996: 160). As we will see in chapter 4, the verbal mode can realise two kinds of modality, viz. epistemic and deontic modality. The latter is concerned with the ‘degree of obligatoriness of a proposition’ (Davidse 1995: 9.1).
Finally, semiotic systems have the capacity to form texts. Texts are complexes of signs, in which the signs cohere internally and with the context. This is their textual metafunction. In the visual image, the textual function or composition integrates the represented and interactive elements into a meaningful whole on the basis of three interrelated systems: information value, salience and framing.
In the verbal and the auditive mode, these functions work at a diachronic or temporal level. In the film image, these three levels can work both spatially and temporally. Firstly, the processes and participants of the ideational level can be represented spatially, in one, static image, or they can be represented temporally, through movement of the camera or through a juxtaposition of shots. Secondly, the mood system of the interpersonal level can be construed spatially by elements such as angle, or size of frame, and/or temporally, through movement of the camera (which can construe the meaning of involvement). Similarly, the modality system can not only be construed synchronically, by colours and light, but also by camera movement. Finally, textual meanings can be construed both spatially and temporally, through camera movement and juxtaposition of shots, and the rhythm which is created by these temporal devices.
3.2 Genre and Intersemiotic Complementarity
Genre, in the general meaning of text type, can be situated on the level of langue3 (see Table 1). This is the level of typical instantiations of forms, their typical uses determined by text type. In langue3, two levels can be distinguished. Firstly, there is the level of the typical co-patternings of the forms. We can situate these typical co-patternings on the three metafunctional levels, the ideational, the interpersonal and the textual level. These ideational, interpersonal and textual co-patternings are then integrated on the textual level on the basis of structural formula. The latter are ‘well-defined configurations of the elements of structure of a text’ (Davidse 1996: Part I, 37
Genre can then be seen as a prototypical configuration involving the ideational, interpersonal and textual level. By including the ideational, interpersonal and textual level in the definition of genres, a system of genres is possible. The prototypicality of the definitions, and the stress on the social and historical nature of the definitions, would ensure the open-endedness of the system.
The second level of langue3 is the level of the typical cross-couplings of the forms with other semiotic modalities. In his article on page-based multimodality, Royce (1998) claims that the different semiotic systems in a multimodal text work together, and semantically complement each other to constitute a coherent and meaningful whole (I can agree with Royce’s claim in that the different modes in a multimodal text mostly work together to form a coherent semantic unity. However, they do not always succeed, as two modes can contradict each other on the ideational level. According to K&VL, the different interests of different social groups can be expressed in contrasting encodings in a multimodal text. Thus, the verbal component can have another meaning than the visual text. The analytical framework developed by Royce can be used to identify these contradictions. Bordwell (1980), too, considers film as an ‘art of temporal process’, and points to the importance of rhythm.
The relations between the different modes in this whole are not just conjunctive relations. The modes constitute synergistic relations with each other, i.e. the total meaning resulting from the combination of the elements, is more than the sum of the meanings of the elements. In this connection, Royce has coined the term intersemiotic complementarity to designate the complementary relationship between the modes.
As film is a multimodal medium, in which the visual, verbal and auditive mode work together to establish a meaningful whole, this framework can also be applied to it. In this framework, the complementarity of the semiotic modes is analysed on the ideational, the interpersonal and the textual level.
According to K&VL, the different semiotic modes are integrated on the basis of an overarching code. In (static) visual representations, this is the code of spatial composition. In speech, music, dance, and we can add film, this overarching code is rhythm, the code of temporal composition (Bordwell (1980), too, considers film as an ‘art of temporal process’, and points to the importance of rhythm.). We can situate this code on the level of langue3. On the basis of this code, the ideational, interpersonal and textual levels of the different modes are integrated into a meaningful whole.
In this article, the cognitivist approach to film has been criticised on the basis of systemic-functional theory. In its reaction to Grand Theory, the cognitivist approach reject a general theory of film in favour of dialectical theorising based on empirical research. Semiotic approaches are rejected because of their projection of a closed linguistic system onto film.
In their reaction to Grand Theory, however, the cognitivists show to be too radical. The systemic-functional theory, on the other hand, presents a better alternative to Grand Theory. It shows that the rejection of Grand Theory does not necessarily imply a rejection of general theory. In fact, it is claimed that the combination of a top-down and a bottom-up approach is necessary for theorising, as these two approaches correct and enrich each other. In addition, the postulation of film as an open-ended internal semiotic system, which is studied both in its particularity and in its relations with other semiotic systems, counters accusations of linguistic imperialism. The systemic-functional claim about the nature of film can be motivated by Thibault’s notion of semiotic systems. I have shown that the two characteristics of semiotic systems, viz. stratification and schematicity, are present in film language.
The systemic-functional approach to film has been presented in relation to Thibault’s distinction between langue2 and langue3. Firstly, the systemic-functional framework has been elaborated at the level of langue2 through an application of Kress and Van Leeuwen’s theory of the static visual image to the moving image. The three metafunctional levels out of which meaning is constructed can also be recognised in film. It has been demonstrated how the three metafunctions can be realised both spatially and temporally. As there is no hierarchic relation between the metafunctional levels, it is possible to take into account all elements of film language.
Secondly, the systemic-functional framework has been elaborated at the level of langue3. At the level of intermodal relations, Royce’s concept of intersemiotic complementarity has been applied to film. The different semiotic modes of film are integrated at the three metafunctional levels on the basis of the overarching code of rhythm. At the level of interfilmic relations, genre has been defined in terms of the prototypical organisation of ideational, interpersonal and textual elements, according to text type. By taking into account the three levels of meaning making, and their prototypical configurations, (prototypical) definitions of text types can be constructed.
Bordwell and Carroll both complain about the lack of dialectical response of Grand Theory to new developments. According to Bordwell, Grand Theory ignores the ‘most important contemporary developments in linguistic theory’ (B&C 1996: 22), meaning Chomsky’s theories. He claims that ‘the silence is plainly strategic’. In addition, Carroll complains about the lack of dialectical response of psychoanalytic theory to the cognitivist approach. However, Carroll does not display any dialectical response of the cognitivists to systemic-functional theory, and Bordwell ignores the newest development in linguistic theory, viz. systemic-functional linguistic theory. One wonders whether this silence may be strategic too.
Bordwell, D. (1980) ‘The Musical Analogy’ in Altman, R. (ed.) Cinema/Sound. New Haven: Yale French Studies
Bordwell, D. (1989) Making Meaning. Inference and Rhetoric in the Interpretation of Cinema . Cambridge (Mass.)/London (England): Harvard University Press
Bordwell, D. & Carroll, N. (ed.) (1996) Post-Theory. Reconstructing Film Studies . Wisconsin/London: The University of Wisconsin Press
Davidse, K. (1996) Text Grammar: Course notes (1996-1997) Katholieke Universiteit Leuven
Kress, G. & Van Leeuwen, T. (K&VL) (1996) Reading Images. The Grammar of Visual Design . London/New York: Routledge
Royce, T.D. (1998) ‘Synergy on the Page: Exploring intersemiotic complementarity in page-based multimodal text’, typescript, Tokyo: Columbia University
Thibault, Paul J. (1997) Re-reading Saussure. The Dynamics of Signs in Social Life . London/New York: Routledge
This site is optimized for Netscape 6 and higher
site design: Sara Roegiers @ Maerlantcentrum