Steve Oswald, Didier Maillat & Louis de Saussure

25Deceptive and uncooperative verbal communication

Abstract: In this chapter, we address verbal communication in non-cooperative contexts, especially deceptive, ideological and fallacious communication. We first review the main traditions that have studied this issue: ancient Rhetoric, contemporary Argumentation Theory, Communication Science, Critical Discourse Analysis, and Pragmatics, from Grice to Cognitive linguistics. We dedicate particular attention to contemporary post-Gricean Pragmatics and to its links with research in Psychology on cognitive heuristics. The notion of epistemic vigilance is addressed in the section dedicated to applications, where the social aspects of deceptive verbal communication are also developed.

Keywords: deception, fallacies, persuasion, heuristics, critical discourse analysis, argumentation theory, rhetoric, epistemic vigilance

1Introduction

The phenomenon of deceptive and uncooperative communication is among those phenomena that call for interdisciplinary inquiry. Owing to its complexity – it is psychologically, sociologically, politically, ethically and linguistically significant (among some of its relevant dimensions) –, its study involves a variety of research questions which can be approached from a number of different disciplines and frameworks. Since we will be concerned with the linguistic and discursive aspects of deception, the focus of this chapter will be on deceptive and uncooperative verbal communication. Its purpose is targeted at reviewing and discussing five fields of scientific inquiry which have been used to approach deceptive verbal communication, namely linguistic pragmatics, communication studies, (critical) discourse analysis, argumentation theory and cognitive pragmatics; the discussion will also deal with the requirements an explanatory account of how verbal deception works should fulfil. To the extent that we will focus on communicative settings, we will furthermore be led to evoke both social and cognitive aspects of deception.

Linguistic pragmatics, within the strand originating in the works of Herbert Paul Grice, is particularly relevant to the study of deceptive communication, mainly because of Grice’s clearly delineated cooperative principle (1989 [1975]) which is taken to regulate how people communicate with each other and make sense of verbal utterances. Resorting to his work seems a priori a sound enterprise, to the extent that once cooperation is defined, what non-cooperation amounts to can ipso facto also be identified, which can accordingly constitute a first step in an attempt to characterise verbal deception.

Within Communication Science, deception has attracted a great deal of attention over the past four decades (see Levine & Kim 2010 for a comprehensive overview). Questions such as the definition of deception and its relationship to lying, deception prevalence, deception motives and deception detection have given rise to abundant studies, sometimes informed by neuroscience, which have in turn contributed a great deal to our understanding of the reasons people deceive others and their (very) relative ability to spot deceivers.

The study of the social and political implications of verbal deception, notably in terms of discrimination and ideology (re)production/propagation, has traditionally been the province of Critical Discourse Analysis (henceforth CDA), whose origin can be traced back to the movement referred to as Critical Linguistics and which was initiated in the 1970s by the seminal work of Roger Fowler, Gunther Kress, Bob Hodge and Tony Trew (Fowler et al. 1979). Although the focus of this research programme is not verbal deception proper (to the exception of van Dijk’s (2006) paper on manipulation), an underlying assumption of CDA research is that language is the vehicle of ideology, which is covertly communicated and contributes to enforcing and reproducing actual power relations. In this sense, CDA nourishes socio-political ambitions and takes linguistic theory as a means to debunk and expose how ideology is (sometimes explicitly, but most of the time implicitly) verbally conveyed in discourse. CDA has recently undertaken a cognitive turn by incorporating cognitive considerations about language processing (notably through the study of metaphor and of its persuasive power, see e.g., Chilton 1994, 2005; Hart 2010) and is therefore nowadays informed and enriched by psychologically-grounded models of human cognition.

Verbal deception oftentimes takes the form of fallacious argumentation, by managing to get addressees to accept insufficiently or ill-evidenced conclusions. While the Western roots of the study of argumentation lie within Aristotle’s work and still continue to inspire generations of contemporary researchers, a number of theories tackling the distinction between sound and fallacious argumentation are nowadays available; their normative input is of particular relevance to any approach to verbal deception interested in the study of how people manage to mislead their audiences through argumentative means.

A fifth, more contemporary, area of research around deceptive communication is the cognitive pragmatic approach. Scholars working within this framework take a cognitive perspective on verbal information processing to argue that deception constrains verbal comprehension so as to divert the targets’ attention from mobilising information that would allow them to identify the deceptive intent. In other words, this perspective seeks to precisely characterise what it means cognitively for information to be foregrounded or backgrounded. The main theoretical resource researchers in this paradigm make use of is Sperber & Wilson’s Relevance Theory.

The remainder of this chapter will be divided in 3 sections: section 2 will spell out the general features of each of these five approaches and their respective contribution to the study of deceptive verbal communication; Section 3 will evoke some of the applications of research on verbal deception; the conclusion will summarise the main points of the chapter and mention further directions of research.

2Studying deceptive and uncooperative verbal communication

2.1Linguistic Pragmatics: Grice’s model

The Oxonian philosopher of language H. P. Grice is considered to be one of the founders of pragmatics, i.e. the scientific study of language in use, of contextual meaning and understanding. Originally interested in accounting for how people are routinely able to understand each other without much difficulty despite the fact that sentences usually express literally much less or different things than what is actually conveyed, Grice was mostly interested by implicit meaning and offered a mixed explanation – conventional and cognitive-intentional – as to how humans grasp implicit meaning. His research was seminal for other scholars who further tackled semantic underdeterminacy, i.e. the intrinsic ambiguity of sentences across contexts, and who offered a variety of elaborations, either by assuming a default level of interpretation (they are referred to as ‘neo-Griceans’, such as Horn and Levinson) or by reducing the apparatus to a basic cognitive principle (these are referred to as ‘post-Griceans’ and include scholars such as Sperber & Wilson and Carston; see Carston 2002 for a detailed discussion).

Grice developed an account of communicative rationality postulating that language users adopt linguistically cooperative behaviour. His cooperative principle (“Make your conversational contribution such as is required, at the stage at which it occurs, by the accepted purpose or direction of the talk in which you are engaged”, Grice [1975] 1989: 26) requires interlocutors to observe a number of principles called maxims: Quantity, Quality, Relation and Manner, which enjoin speakers to respectively provide as much information as required, to be truthful and not to assert that for which they lack evidence, to be relevant and to be perspicuous. When speakers observe these maxims (or overtly fail to observe them for the purpose of triggering a specific implicit meaning) they are deemed to be cooperative communicators, thereby ensuring a “maximally effective exchange of information” (Grice 1989: 28). Conversational participants are said to expect each other to comply with these principles and this is taken to account for the possibility of verbal communication, both explicit and implicit.

Communication, under Grice’s terms, is defined as intentional. The mere existence of implicit meaning is perhaps the clearest indication that language users, in the process of comprehension, do much more than simply decode the literal meaning of the words that were uttered by the speaker and that they actually process them in a more elaborate way. Take for instance example (1):

(1)	A:	Where is Brian?
	B:	I just saw his car parked on the driveway in front of his house.

(2)	Brian is at home

In (1), A formulates a question to find out about Brian’s whereabouts. B, instead of giving a concise answer in the form of a precise location, tells A where Brian’s car is. While literally unrelated to the question (the question is about Brian, not about his car), B’s answer leads A to infer (2), namely that Brian is at home, by inviting A to mobilise background information, among which the fact that when someone’s car is parked in front of the house the person is likely to be at home. A is quite effortlessly able to infer this implicit meaning; despite superficial irrelevance in B’s answer, the assumption that B is cooperative and produces information that is consistent with the cooperative principle and the conversational maxims allows A to infer (2), an implicit part of meaning which Grice named implicature.108 This goes on to show that addressees in fact aim at recovering the intentions with which their interlocutors formulated their utterances in order to calculate the meaning of the latter. They are thus fully capable of distinguishing irony, metaphor and many other types of implicit meaning from the semantic content of the expressions used to convey them. The derivation of meaning can consequently be characterised as an inferential process which takes the literal meaning of the sentence as input and considers its implications in the present talk exchange, assuming that the speaker is cooperative, that is, for example, that what she says in the end fulfils the hearer’s expectations in this respect. Thus, the conclusion (2) comes up directly as a result of the CP-motivated assumption that in the end, the contribution of the speaker is indeed related to the topic of the conversation despite appearances.

A typical example involving the flouting of a maxim is that of figures of speech. Metaphors and ironic utterances, for example, are literally false and can consequently be described as flouts of the maxim of Quality. Yet, just as in the example above, cooperation obtains through the inference of implicit meanings and there is no deception going on. However, when a maxim is not overtly, but covertly, violated, deception and uncooperativeness arise. While originally designed to capture issues of meaning, the cooperative principle and the conversational maxims have in fact often been extended to capture behavioural cooperation in communication as well, so much so that Grice-inspired cooperative principles have been brought forth to tackle communicative behaviours such as humour (see Raskin 1985: 102–103) or politeness (see Leech 1983), for instance, but also deceptive communication, even though Grice’s main objective was only to explore meaning within its linguistic bounds and to provide a rational theory of communication, construed as a principled means of exchanging information.

Interestingly, when characterising cooperation, Grice mentions in passing a situation straightforwardly interpretable in terms of deception: listing four different ways of failing to fulfil a maxim, he mentions the following as the first possible case: “He [the speaker] may quietly and unostentatiously violate a maxim; if so, in some cases he will be liable to mislead” (Grice 1989: 30). A first characterisation of deception is thus available in terms of non-cooperation: deceptive speakers are those who deliberately fail to observe the conversational maxims without making it manifest to their audience and with a view to mislead them. In other words, deceptive speakers covertly fail to (i) provide the amount of information required, (ii) be truthful, (iii) be relevant and/ or (iv) be perspicuous. This also means that deceptive messages can be expected to be deficient or lacking maxim-wise.109 It is precisely to the empirical testing of this assumption that some researchers in Communication Science have devoted their efforts as we shall see next.

2.2Communication Science: Information Manipulation Theory

The predominantly North-American research tradition of Communication Science (CS) has studied deceptive communication for more than four decades. Studies in the field have explored many different facets of the phenomenon over the years and as a consequence our current understanding of deception is complex and detailed. Obviously one of the first issues deception research has been concerned with is linked to the identification of a clear and stable definition, and most studies in the field accordingly explicitly formulate their own understanding of the phenomenon, which supposes some measure of philosophical inquiry.110 Because human interaction can either be cooperative or uncooperative, the second issue scholars have been concerned with has been deception prevalence, i.e. the extent to which deception is frequent in human interaction. On this issue, studies usually converge towards the conclusion that deception is less frequent than non-deception (see e.g., Serota & Levine 2008; DePaul et al. 1996): we are certainly likely to come across deceptive messages on a daily basis, but non-deceptive messages are still the majority, which is inherently predicted by Grice’s generalisation. A third area of inquiry targets the reasons behind deception; consensus has more or less been reached around the idea that people deceive when truthfulness and honesty involve too many obstacles in order to reach a given goal (see Levine et al. 2008). A fourth (and vast) subfield of deception research deals with deception detection; results from empirical testing reveal that people do only slightly better than a coin toss when it comes to identifying deceptive speakers and messages (see Bond & DePaulo 2006). More generally, it seems, according to Levine & Kim (2010), that deception detection research informs us more on the deceptive message and on its source than on people’s actual ability to spot deception, which is probably a consequence of the nature of the experimental research designs used (Levine & Kim, ibid.).

CS studies of deception usually consist in the submission of questionnaires to panels of respondents – most of the time North American undergraduates; the data collected through these surveys is compiled and analysed so as to draw generalisations. In the study of deception detection, for instance, experiments would typically consist in the submission of a range of deceptive and non-deceptive utterances to respondents, who then have to judge whether the statements are truthful or not. In the study of deception motives, respondents are typically asked to react to a range of given scenarios manipulated to trigger either deceptive or non-deceptive reactions; subjects’ (deceptive or non-deceptive) responses then show whether in scenarios affording motives for deception they are likely to produce deceptive statements or not and allows the researchers to assess the relationship between the production of deceptive messages and reasons for doing so.

One particular approach to deception within CS, Information Manipulation Theory (McCornack and McCornack et al. 1992, henceforth IMT), is specifically designed to assess linguistic and pragmatic features of deception (i.e. its verbal features). The preliminary observations made in the previous section point to the possibility of an account of verbal deception couched in Gricean pragmatics: first of all, Grice’s model is designed to deal with intentional communication, which is the case of deception (see section 3). Second, when dealing with failures to observe the maxims, Grice admitted himself that some failures might be unostentatious, and that as a consequence the speaker, in those cases, might be misleading (i.e. deceptive). McCornack and colleagues precisely tried to verify that maxim violation was at the core of deceptive messages. As such, IMT is a theory which addresses the features of deceptive messages as covert deviations from conversational standards and exploits what would be the ‘uncooperative’ counterpart of the Gricean model of communication as cooperation.111

By considering that messages may be manipulated along four informational dimensions to fulfil deceptive goals, McCornack (1992) attempts to interface Gricean pragmatics with previous research on deception. Several deception researchers had already identified and focused independently on these dimensions: Ekman (1985), Metts (1989) and Turner et al. (1975) attended to the amount of information given as well as its adequacy to truth (which, in Gricean terms, corresponds to the focus of the maxims of Quantity and Quality respectively), Bavelas et al. (1990) to the way messages are formulated (i.e. maxim of Manner), and Turner et al. (1975) to matters of relevance (i.e. maxim of Relation). McCornack observes that this previous research fits nicely with the Gricean maxims. Underlying his work is the assumption that deceptive messages are “unique from other forms of discourse in that they involve deviations from what can be considered rational and cooperative conversational behaviour” (1992: 2). Unlike previous research which yielded problematic taxonomies of deception types, IMT provides a consistent framework allowing to group deceptive messages under one (or more) violation(s) of the finite set of four conversational maxims identified by Grice.

In order to consolidate its main claims, IMT relied on empirical testing. The study carried out by McCornack and his colleagues first gathered a number of deceit-eliciting scenarios, which were then edited in order to avoid ambiguity and to get rid of potential biases. A limited number of scenarios was then submitted to undergraduate students, who were instructed to supply messages in response to one of the potentially lie-eliciting situations. Their task was to write down what they would have said in such situations. The results were analysed as actual deceptive data, and showed that the responses indeed contained violations of the conversational maxims, which was taken to confirm the initial hypothesis: deception seems to operate on the manipulation of information along the dimensions of Quantity, Quality, Manner and Relation.

IMT faces three types of criticisms, both from a methodological and a theoretical perspective. First, the methodology behind IMT can be questioned on the grounds of the lack of naturally occurring deception. The scenarios respondents had to react to in the studies were artificially constructed, and therefore we have no guarantee that the data collected corresponds to spontaneous deceptive communicative behaviours outside the lab (see Galasiński 2000): IMT has analysed “elicited ‘deceptive’ messages – that is, messages that participants consider to be deceptive, and this does not mean that they would actually use them in discourse in a natural setting” (Galasiński 2000: 33, author’s italics). A second problem related to IMT’s construal of deception is that it fails to capture all instances of deceptive communication: we can in principle think of examples where addressees are misled by relevant, truthful, manner- and quantity-appropriate statements. These occur for instance when political and media advisers release statements in a context where they are not likely to be picked up. The ‘Jo Moore scandal’ is such an example: on September 11, 2011, a few hours after the terrorist attacks on the United States, Jo Moore, a special adviser to the British Secretary of State, circulated a memo suggesting that the time was right to “bury bad news”,112 hoping to release potentially damaging information in a context where probably no one would have paid attention to it. Interestingly, such a public announcement would not need to violate any conversational maxim; its deceptive nature resides in the fact that it is released in a context in which its recipients are likely to fail to find it relevant in its own right, since their resources are likely to be mobilised to process information directly in relation to an overwhelming event such as 9/11 (see Maillat & Oswald (2011: 77–78) for a discussion). Therefore, while (probably) most cases of deceptive communication involve covert violations of the conversational maxims, some of them do not, and this means that they cannot be captured by IMT. A third limit of IMT is its inability to distinguish intentional from unintentional maxim violation. Since it does not provide criteria meant to distinguish overt from covert maxim violation, it remains unclear how IMT can disentangle cases of intentional maxim violation from mere errors which naturally occur in conversation, for instance errors resulting from misrepresentations of the common ground shared by the interlocutors. Contravention to one or more conversational maxims is only an indicator of a potential manipulation.

Even though IMT falls short of providing a full-fledged plausible account of deception, the significance of its results open the possibility of accounts based not on Gricean maxims alone but, rather, on more recent approaches elaborated in the continuity of Grice’s work. Such an attempt is that of Cognitive pragmatics (section 2.5).

2.3Critical Discourse Analysis

As mentioned earlier, CDA emerged as a research programme meant to expose how speakers’ linguistic choices may encode, reproduce and propagate ideologies and thereby enforce social inequality and discrimination. From its inception, CDA has drawn on linguistic theory – and on Halliday’s Systemic Functional Grammar in particular (see Halliday 1973, 1985; Halliday & Hasan 1985) – to explore the assumption that, to the extent that the richness of the linguistic system allows us to select from different formulations to express the same conceptual content, linguistic choices are ideologically relevant. A classic example of the representational possibilities offered by the passive voice constitutes a (now classical) illustration of this claim (see Trew 1979): the semantic agent of a clause, realised in subject position in active sentences, can be omitted in the passive voice without this affecting the grammaticality and interpretability of the sentence.113 The point is that in political contexts with high stakes, speakers might take advantage of these syntactico-semantic possibilities to obfuscate information. As discussed by Trew, a newspaper that writes ‘Eleven African were shot dead and fifteen wounded’ (instead of ‘The police shot eleven Africans dead and wounded fifteen others’) might be liable for obfuscating the information that it is the police who directly caused the death of these eleven people by firing their guns (see Trew 1979: 34). By opting for the passive voice, which does not require explicit mention of the semantic agent of the predicate, the newspaper might be charged with reducing the perceived responsibility of the police in the violent events. In turn, this can be interpreted as a ‘pro-police’ ideological choice.

Since CDA is not – and does not claim to be – a unified theory, it is by nature open to interdisciplinary inquiry (even if, as highlighted by Chilton 2005, it has traditionally been only selectively interdisciplinary). This is why CDA draws from many approaches in the Humanities without having the pretension of integrating them all into a consistent and systematic theoretical framework. Relaxing epistemological constraints thus encourages the integration of numerous trends in linguistic and communication research. Multidisciplinarity in CDA is found in the diversity of methodologies and theoretical approaches employed. Over the years, recourse to fields of research as varied as Conversational Analysis, Corpus Linguistics, Content Analysis, Discourse Analysis of several traditions, Ethnography of Communication, Systemic-Functional Grammar, Text Linguistics, Rhetoric, Argumentation Theory, Pragmatics, Cognitive Linguistics, Frame Semantics, Cognitive (or Mental) Models Theory, Script Theory, Semiotics, Social Semiotics, among others, has contributed to develop and enrich research in CDA. Indeed, as CDA is not driven by a unified theoretical goal (such as providing a theoretical model of human communication), but rather by the prime concern about “the discourse dimensions of power abuse and the injustice and inequality that result from it” (van Dijk 1993: 252), it follows that the theoretical apparatus it employs is “chosen or elaborated as a function of [its] relevance for the realization of such a socio-political goal” (ibid.). Consequently, CDA has borrowed an array of analytical and conceptual instruments from all these disciplines, which it has used to deconstruct the way ideology functions – most of the times covertly – in discourse. The result of such an integrative endeavour is the elaboration of a multifarious and heterogeneous toolkit for linguistic analysis guided by a utility criterion.114

CDA often hovers over the issue of verbal deception without explicitly discussing it. Yet, most of CDA research is devoted to the study of covert influence in language, which is said to contribute to power abuse and social inequality. Van Dijk (2006) nevertheless devotes an entire article to manipulation, in which he defends a triangulated (social, discursive and cognitive) approach designed to tackle the phenomenon in a comprehensive way. Attention is thus paid to (i) its social features: manipulation implies power and illegitimate power abuse between social actors and “it only makes sense to speak of manipulation, […] when speakers or writers are manipulating others in their role as a member of a dominant collectivity” (van Dijk 2006: 364); (ii) its cognitive features: “manipulation always involves a form of mental manipulation” (ibid.: 360); (iii) its communicative features: manipulation […] takes place by text and talk” (ibid.: 360). In its social dimension, manipulation is said to exploit group membership, hierarchical roles assigned by institutional and professional positions, material or symbolic resources defining the power of groups and their members. Typically, professors, or politicians, by virtue of their institutional positions and their privileged access to information and public discourse, are more powerful than pupils or voters respectively, and thus might exploit that power to manipulate the latter. While van Dijk does acknowledge that the powerless may also manipulate the powerful, his focus is politically and socially constrained, which is why he is interested in the wider picture: according to him, it makes sense to speak of manipulation as “it is illegitimate in a democratic society, because it (re)produces, or may reproduce, inequality” (van Dijk 2006: 363–364). From a cognitive perspective, he recognises, as we will see further in section 2.5, that deception is not special: “it makes use of very general properties of discourse processing” and consists in “illegitimate hindering or biasing of the process of discourse comprehension” (ibid.: 366). While cognitive considerations do inform us on how manipulation works, they do not tell us why manipulation is perceived as deceitful; this in turn legitimates, in the view of CDA, the need for a social dimension in the study of deception. Finally, the discursive dimension of van Dijk’s take on manipulation is related to the sort of linguistically informed analysis carried out in traditional and mainstream CDA: it consists in listing and describing different structures and types of linguistic constructions which can be strategically used to ideologically influence representations.

One of CDA’s main contributions to the study of verbal deception lies in its identification of the discursive strategies language users resort to in order to convey ideological representations of the world which are deemed to be desirable and well suited.115 Among those, we find referential, predicative and legitimising strategies. Referential strategies are used to represent the world, social actors in particular, in a way that is compatible with the speaker’s ideology and interests. Predicative strategies are meant to endow representations of social actors with evaluative social values (typically, these strategies are used to positively represent in-groupness and to negatively represent out-groupness). Legitimising strategies comprise the list of different means by which the speaker – or their discourse – displays epistemic reliability, credibility or social legitimacy.

Even if mainstream CDA does not make any strong claim as to the deterministic potential of these strategies, it seems reasonable to assume that many instances of verbal deception can make use of these strategies in a covert way; that is, deceivers can reasonably be expected to rely on these strategies without drawing attention to the fact that they do. Overall, the fundamental contribution of CDA to the study of verbal deception is thus of an illustrative nature and is to be found in the vast amount of systematic corpus studies drawing our attention to the way these strategies are realised in discourse.

2.4Argumentation Theory

Building on Aristotle’s foundational distinctions between logic, dialectic and rhetoric, scholars in Argumentation Theory (see this volume, chapter 13) have explored over the years the multiple facets of argumentation, including its relationship with persuasion and deception. Although deception can be approached quite straightforwardly through the rhetorical notion of speaker ethos, to the extent that in order to convince someone it is often enough to get them to deem you trustworthy, competent and benevolent, mainstream research in argumentation has also focused on the connection between deception and fallacious argumentation. The adoption of this perspective, we surmise, follows from the tacit but seemingly widespread consideration, among argumentation scholars, that deception is “above all a question of means, and not of ends” (Nettel & Roque 2012: 58). As a result, research on the deceptive character of some argumentative moves is foremost concerned with the way arguments can become deceptive, i.e. taken as a means to attain further goals such as the adoption of beliefs or specific behaviour.

The standard treatment of fallacies defines a fallacy as an argument “that seems to be valid but is not so” (Hamblin 1970: 12), and, perhaps more objectively, as an argument in which the conclusion does not follow from the premises. However, this is not yet equivalent to considering that a fallacy is necessarily deceptive. Fallacies can indeed be used to deceive, but they are sometimes unintentionally committed, and, as such, cannot be deemed to be necessarily deceptive. To take but a very simple example, consider George W. Bush’s famous argument to legitimise the so-called war on Terror following the 9/11 attacks:

(3)	“Either you are with us, or you are with the terrorists.”116

While (3) clearly lays the grounds of the fallacy known as the false dilemma, and while we do not question that this is most probably the type of reasoning underlying Bush’s utterance, we cannot ascertain that he is trying to deceive us from the mere presence of a fallacious argument: for one, he could be making a simple reasoning mistake, and an error of this kind is by no means direct evidence of any sort of deceptive intention. It could also be the case that he actually sincerely, though mistakenly, believes that the world is Manichean and that there are only two possible stances on the issue at hand. If such is the case, then the example should not be treated as deceptive either, because under these circumstances George W. Bush is not even aware that he is making a reasoning mistake and that the way he supports his standpoint is built on fallacious grounds.

The non-necessarily deceptive character of fallacious argumentation has been recognised by argumentation scholars, even if an explicit stance on the issue is seldom voiced. Tindale is among those who acknowledge that fallacies may not be systematically deceptive, when he warns that “we should not take deception to be part of the definition of fallacy” (2007: 15). Walton similarly considers that “a fallacious argument can be deceptive by appearing to be a better argument of its kind than it really is” (2010: 159, notice the modal ‘can’), adding in the same paper that fallacies “are arguments that work as deceptive stratagems” (2010: 179). We contend that the treacherous nature of fallacies – referring here to their misleading effect on their addressees – should not be equated with the speaker’s intent to deceive.

Despite the problems involved in systematically assessing the relationship between deception and fallacious arguments, the study of argumentation provides relevant insights for an inquiry into deceptive strategies, to the extent that deception often has to rely on justification in order to go through. Since successful – i.e. persuasive or convincing – fallacies need to remain unspotted in order to fulfil their role, a better knowledge of how they work can inform us on the workings of the deceptive mechanisms at play in faulty justification. The typology of fallacies that has been elaborated since Aristotle117 provides us with valuable tools to analyse the verbal manifestation of inferential processes that appear to be faulty in some respect. This means that thanks to argumentation theory we can analyse in depth cases of deception in which fallacious argumentation is used, so as to identify with precision where the ‘trick’ lies in the faulty inference: some fallacies indeed play on the fact that the evidence provided is (covertly) irrelevant to the issue at stake (this is typical of the fallacies termed red herrings where the speaker tries to divert the addressee’s attention from one argument or standpoint onto another unrelated one), some others rely on defective causal relationships (such as the post hoc ergo propter hoc fallacy in which a subsequent event is presented as having been caused by a prior event by virtue of their temporal succession, or the cum hoc ergo propter hoc fallacy, which relies on the temporal and/ or spatial co-occurrence of two events to attempt to justify a causal relation between them); others play on the credibility of the source of information (ad hominem, ad populum and ad verecundiam, for instance, see Maillat 2013 and Oswald & Hart 2013), and so on.

The study of fallacies is thus relevant to the study of deception because it targets a process many times involved in deception. This also means that the study of fallacies can be assessed in cognitive terms, given its focus on inference. Moreover, it calls for an in-depth consideration of two important dimensions that are relevant to fallacy-dependent deception, namely its production and its reception. The two questions that can be assessed within a cognitively-grounded framework of fallacious deception are the following: why do people commit fallacies? Why do people fall prey to fallacies? The latter in fact amounts to studying why fallacies can at all be effective arguments.

Within argumentation studies, the first question has started to be tackled in connection to the issue of cognitive biases (see Walton 2010; Jackson 1996; Correia 2011, forth.): people are deemed to commit fallacies as they follow cost-effective cognitive shortcuts and biases that allow for quick, fast and frugal inferences to take place. To give but one example borrowed from Correia (2011), the cognitive illusion known as the focusing illusion (Schkade & Kahneman 1998), which accounts for the propensity people have to make category judgements based only on a small (non-representative) subset of elements, can plausibly be considered to be the cognitive mechanism at play behind the fallacy known as the hasty generalisation, in which a speaker draws a general conclusion from evidence that does not warrant it.

Probably one of the biggest advantages of resorting to a cognitive framework in order to account for deceptive fallacious communication lies in its ability to equally account for inferential work at the reception end. Thus, a cognitive account would capture the addressee’s failure to spot a fallacy presumably through its propensity to draw attention to information sets that are not detrimental to its success, i.e. to draw the addressee’s attention away from critical information. Here, again, the management of information accessibility can be accounted for in terms of the cognitive psychological features of our fallible and bias-driven way of processing information.

2.5The Cognitive Pragmatic approach

A way to envisage bridging the gaps between these main trends while taking advantage of their richness is to adopt a cognitive pragmatic perspective, as Maillat & Oswald (2009, 2011), Maillat (2013), Oswald (2010, 2011, 2014), Saussure (2005, 2013, 2014), Lewiński & Oswald (2013), Oswald & Lewiński (2014), Oswald & Hart (2013) do. Sperber & Wilson’s (1995) Relevance Theory is not originally designed to address deception in communication but to explain meaning, i.e. verbal understanding, by resorting to a cognitive principle of efficiency. The theory builds on Gricean pragmatics although it leaves aside the normative aspects of the theory (the maxims) and focuses instead on the notion of intention recognition. Relevance theory is inscribed within a broad approach to cognition, knowledge, communication and human behaviour which gives rise to a general anthropology of human cultures based on cognitive assumptions (Sperber 1985, 1990, 1997, 2001, 2007, 2011; see also this volume, chapter 6). Researchers in these trends have recently come to address the issue of arguing, deceiving and spotting deception. The most notable contribution in this respect is the development of the notion of epistemic vigilance (see work by Sperber et al. 2010) and its specification in terms of argumentation through the argumentative theory of reasoning (Mercier & Sperber 2009, 2011; Mercier 2012), which provides a general cognitive framework in which these phenomena may be construed. In another cognitive framework, Dessalles suggests that individuals resort to argumentation in order to “advertise one’s ability to detect lies and errors” (Dessalles 2011: 76; see also this volume, chapter 11) with the aim, ultimately, to control and defend their leadership abilities.

The references within cognitive pragmatics already mentioned at the beginning of the previous paragraph take a decisively linguistic approach to communication that is directly relevant to the discussion at hand. They put forward the idea that the success of deceptive attempts in communication is directly related to the degree of cognitive accessibility of certain contextual assumptions and consider Relevance theory to be an ideally-suited theoretical framework to account for this assumption. In this framework, deception is considered to be an intentional phenomenon mostly due to its covert nature: deceivers cannot conceivably hide something they are not aware of. Furthermore, deception is seen as operating at the level of verbal information processing, by constraining the addressee’s access to critical information. This means that during the comprehension process there are constraints at play that secure the inaccessibility or dismissal of pieces of information that could compromise the success of the deceptive attempt in the cognitive system of the addressee. A cognitive account of deception will then rely on a precise characterisation of how verbal information is processed during the comprehension procedure in order to identify the different types of cognitive constraints that partake in the success of verbal deception.

In a series of publications on the topic, Maillat, Oswald and Saussure have started to explore the possibilities of pragmatic research to consider the links between understanding and believing, through an examination of the cognitive operations at play when language users process verbal information in argumentative and deceptive communicative contexts. A model of verbal deception was thus elaborated by focusing on the role of the construction of context which lies at the heart of relevance-theoretic pragmatics (the Context Selection Constraint model, see Maillat & Oswald 2009, 2011). The main idea behind this model is that the successful deceiver tries to make sure that the contextual information against which a target deceptive utterance is processed is devoid of any critical, contradicting information; in other words, successful deception plans and exploits the addressee’s failure to assess as relevant any information set that could defeat the deceptive utterance.

In order to account for such a constraint, it is necessary to assess (i) how verbal information is understood, and, more specifically, (ii) how information can be backgrounded and/or foregrounded in order to be perceived as irrelevant or relevant respectively. Relevance Theory (Sperber & Wilson 1986, 1995; Wilson & Sperber 2012; Carston 2002; henceforth RT), as a cognitively grounded theory of communication, proposes a theoretical framework equipped with criteria that determine information selection. According to the theory, which construes communication as an evolutionarily advantageous means to secure a more accurate representation of the world, verbal information is processed in communication following a principle of cognitive efficiency named cognitive principle of relevance. In the process of recovering the meaning intended by the speaker – which often involves recovering many more contents than those that are explicitly uttered by the speaker, as we discussed in section 2.1 – the addressee will select in priority the most accessible assumptions, in terms of processing effort, and at the same time those that yield more cognitive effects, the latter being usually defined in terms of epistemic advantages: a given assumption will be deemed relevant if it allows the cognitive system to benefit from new reliable information, to get rid of inaccurate old information or to strengthen information that was not previously fully confirmed. In a nutshell, RT postulates that relevant information is information that is easy to process and at the same time information that is useful to the cognitive system in terms of its reliability and informativeness. In this perspective, understanding what a speaker means is therefore the result of working out the best effort/ effect ratio by following a path of least effort when selecting the information set against which an utterance is being interpreted. These two parameters, called extent conditions of relevance (Sperber & Wilson 1995: 125) constitute under this view the pivotal mechanisms that deception exploits.

If deceptive communication can be construed as a phenomenon that operates constraints on the selection of information during the comprehension process, it can ipso facto be described as a twofold mechanism: on the one hand it tries to make sure that every information set that is mobilised in the process is compatible and coherent with the target deceptive utterance, while on the other it strives to keep critical information sets that would defeat the deceptive attempt concealed (i.e. so as to leave them unprocessed or to get the cognitive system of the addressee to dismiss them). Under this view, deception works simultaneously as a weakening and a strengthening constraint. This opens up the possibility of studying specific wordings and linguistic constructions that precisely trigger such constraints. Examples of such analyses are gradually becoming available as research within this framework is carried out; phenomena such as metaphors (Oswald & Rihs 2013), flashbulb contexts (Maillat & Oswald 2009), the straw man fallacy (Oswald & Lewiński 2014, Lewiński & Oswald 2013), source-related fallacies (Oswald & Hart 2013), the ad populum fallacy (Maillat 2013), presuppositions (Saussure 2013) have been interpreted from this perspective, and the range of linguistic phenomena at the heart of deceptive uses of language can be expected to be extended further, as all of them can be thought of as enforcing strengthening and weakening strategies meant to constrain the salience of certain contextual assumptions.

An additional component of this research programme lies in its compatibility with more general cognitive research on biases and heuristics. To begin with, the comprehension procedure described above can itself be construed as a heuristic (Wilson & Sperber 2004: 259): understanding is a fallible process, if only because most of the time the addressee is responsible for mobilising (i.e. selecting) the intended contextual assumptions in order to make sense of the linguistic material that has been uttered by the speaker – and he can always fail in this selection. In this sense, deceptive communication can be characterised as an asymmetric process in which the speaker constrains the addressee’s selection of contextual assumptions as the latter engages in the comprehension process, thereby deliberately trying to hinder his selection of critical information. This is possible by virtue of the heuristic nature of comprehension, which is an important property of the relevance-theoretic cognitive account of human communication, as it integrates the very fallibility of the system. A heuristic-driven cognitive process is prone to making errors since fast and frugal heuristics, although they provide resource-optimal means of deriving new knowledge, are not, by definition, exhaustive or systematic. As a result, our cognitive system can – and in fact, will – err at times. While this comment is true of the comprehension process we detailed above, it also applies to other cognitive heuristics that influence the cognitive operations needed whenever we modify our cognitive environment, i.e. our representation of the world.

This should not be regarded as a defect in the system though, as Tversky & Kahneman (1974), or more recently Gigerenzer (2008) have shown: heuristics are the results of an evolutionary drive in optimising cognitive efficiency as they offer the best balance between speedy derivation of new knowledge and costly inferential thorough evaluation processes. As such they offer fast and reasonably robust means of acquiring new knowledge at a fraction of the cognitive cost.

From a cognitive perspective, proponents of the cognitive pragmatic approach argue that deception can often be seen as exploiting these heuristics, as they correspond to specific instances when the cognitive system takes shortcuts. The first of these heuristics to be taken advantage of by deceptive strategies is the very comprehension procedure: as seen above, interpretation is a non-exhaustive, context-dependent representation of the intended meaning, which can miss out critical information sets. On a second level, the literature in cognitive psychology has identified many other such heuristics and biases which allow us to predict the type of cognitive processes that deceptive uses of language are likely to target. Thus, a wealth of research has been carried out on cognitive biases and heuristics since the 1960s (see e.g., Tversky & Kahneman 1974; Gigerenzer 2008), detailing how our cognitive system tends to privilege fast and economical processes over reflective ones, thereby giving prevalence to cognitive illusions. These illusions correspond to three types of cognitive errors: errors in judgement, reasoning and memory (see Pohl 2004 for an overview). For instance, the anchoring effect affects the selection of contextual assumption held in long-term memory, as a subject’s recall is biased towards a known target value (Tversky & Kahneman 1974; Mussweiler et al. 2004). The labeling effect describes the bias induced on long-term memories by the wording chosen to describe those memories. The validity effect describes the tendency for repeated statements to be judged as more and more valid across iterations (Hascher et al. 1977; Hackett Renner 2004).

As in the case of the validity effect, many such cognitive biases are certainly part of the mechanisms targeted by verbal deception, and as a result, counterparts to cognitive biases can be found in the body of argumentative fallacies that have been identified through history (see e.g., Correia 2011, forth.). In the case of the validity effect, Maillat (2013) shows how it underlies the heavy usage of repetition in propaganda. Maillat (2013), Maillat & Oswald (2011) establish a link between the mere-exposure effect and the ad populum fallacy, while Saussure (2013) tentatively links the accommodation of discursive presuppositions with the exploitation of the confirmation bias.

A cognitive framework such as this one is also related to the experimental research being carried out in the tradition of psychology of reasoning (e.g., Wason 1966; Johnson-Laird 2006; Evans & Over 1996, who have shown how humans are prone to reasoning errors), developmental psychology on the role of contextual factors in credal attitudes (Harris et al. 2012) and more broadly in cognitive psychology and pragmatics. Work carried out by anthropologists and philosophers around the notion of epistemic vigilance (Sperber et al. 2010), addressing the cognitive bases of trust and consistency checking, is typically relevant to the research on deception and uncooperative verbal communication, although not specific to verbal communication. Interestingly, this very solid grounding of the cognitive pragmatic approach to deception in a number of experimental traditions ensures the testability of its theoretical claims, while also providing an independently motivated cross-examination of some of its central arguments.

3Applications

Research on the mechanisms at work in intentional deception and manipulation in verbal communication is probably the core research of (critical) discourse analysis as a discipline. However, one important distinction between ordinary (cooperative) and deceptive communication needs to be taken into account. In both cases, of course, we deal with intentional communication. In both cases the speaker intends to make information available to the addressee. In both cases, the latter processes the discourse so that he entertains assumptions about the meaning intentions of the speaker, and these assumptions are, in the end, the meaning actually conveyed by the speech acts. Yet, that the speaker is cooperative in verbal communication is taken as a precondition, or a presumption, before anything like information processing can take place. That people observe the cooperative principle, to shape this in Gricean terms, that they conform to the presumption of relevance (the presumption communicated by any utterance that it is relevant and thus worth processing), as Sperber & Wilson (1995) put it, is an automatic starting point for understanding the message, and even in deceptive communication the speaker claims to observe these principles without which communication would simply not occur. But a manipulative or deceptive attempt supposes a second-order intention about informational, behavioural, perlocutionary effects that the message conveyed is supposed to trigger in the addressee. In short: ‘normal’ benevolent communication can be studied without wondering what the speaker has in mind: we can study the meanings and actual effects of utterances on the addressees (which we analysts are too if only by intuition) without having to think about the actual intentions of speakers; after all, meaning intentions are intended to be detected by the addressee. With manipulation or deception, as discussed before, it is the opposite: assessing with certainty that there is manipulation going on and not an error of some sort, i.e. assessing that there is an intention to mislead, is out of reach for discourse analysis of course. Hence scholars tend to address deception only indirectly as a type of problematic, or fallacious, influence.

Addressing the mechanisms of deception in verbal communication is therefore not really different from addressing the mechanisms of fallacious persuasion. Fallacious persuasion occurs in propagandas of various kinds but of course typically in discourses related to politics and economy, in fanatic and dogmatic discourses of various kinds including sectarian discourses, but also in more subtle settings with all sorts of degrees of persuasiveness where questionable interventions are interwoven with more acceptable ones. Such discourses are very frequent and there are many studies about them in various frameworks. The more traditional way of addressing the issue of persuasiveness and manipulation is to observe how the considered speech/ discourse develops and represents social roles associated both with groups and with ideologies within relations of dominance, power and exclusion (with notions such as in-/out-groups for example). Today, this approach, mostly descriptive, is complemented with the type of cognitive approaches described in the preceding section. Besides basic linguistic facts such as passivisation and nominalisation, a number of semantic and pragmatic-contextual features are identified in the literature as playing a major role in persuading or, as Herman and Chomsky (1988) put it, in ’manufacturing consent’.

The role of the lexicon is identified in the literature on deception as crucial in that, when appropriately contextualized, it can lead to problematic inferences because of the many connotations that a lexical choice can involve. Euphemisms, when used in institutional settings, can be a trace of a persuasive attempt through minimizing or even blocking the effects of some action or the undesirability of some situation; consider for example the expressions collateral damage or friendly fire. This is of course not to say that euphemisms are by nature persuasive or deceptive, otherwise in the end any figure of speech would be so too. But in given contexts, there is little doubt that they are: a damage is not treated as a tragedy, even though on the ground a true tragedy has of course happened. Allott (2005) studies cases such as the use of the word democracy in Bush’s speeches related to the invasion in Iraq, and suggests that the word only gets processed shallowly in order to maintain the validity of the expectation that the speaker is being relevant (or cooperative); democracy is a connotatively loaded word and thus gives rise to positive feelings and favours consent even if the word gets deprived of a true content, notably because its semantic emptiness gets unnoticed. The wide research tradition initiated by Lakoff & Johnson (1980) and further elaborated by Fauconnier & Turner (2003) or linguists like Langacker, usually termed ‘Cognitive linguistics’, takes a close look at the lexicon and spots recurrent metaphorical associations that words trigger, which, they suggest, anchor on schematic associations that are fundamental to our cognition of the outer world. Usually these relations are not problematic in any serious way; an example of such associations is love is a journey which appears as underlying utterances such as Our relationship is at a crossroads or That love affair was a rough ride. In other cases however, as extensively discussed (see e.g., Hart 2010), such associations have some persuasive effects in triggering access to parasitic representations (as Chilton 2005 shows with the classical example of Hitler’s Mein Kampf). More generally, many facts of semantic nature play a crucial role in how the representation is going to be dealt with in terms of acceptability, but the context in which the utterances is processed is crucial and, as Maillat & Oswald (2009, 2011) suggest, the context itself can be manufactured in a way in which some assumptions, even very accessible ones in the abstract, are put aside during the very process of comprehension. Typical jokes and riddles make use of such effects, as when asking the interlocutor if he finds it frightening that New Year’s Eve next year will be on Friday 13th (Oswald 2010; Saussure 2013). Starting from the idea that interlocutors are confident that the speaker is cooperative or relevant, the addressee will tend to find an interpretation which maintains this presumption and sometimes he will fail to take obvious pieces of information into account in order to satisfy this expectation of relevance or cooperation (for example that 31st December cannot possibly be on 13th). Saussure (2013) makes a similar case about the Swiss vote on banning minarets, arguing that simply raising the question calls for recruiting odd assumptions about the potential danger of minarets; he argues that such effects arise on the basis of undue accommodations of presuppositions. A number of other semantic and pragmatic facts directly influence the inferences drawn by the audience but also the epistemic acceptability of the propositions, such as information about the origin of the belief that they carry (hearsay, inference, perception ...), a notion called evidentiality which is becoming more and more fashionable in the domain of persuasiveness in discourse (see the recent issue of Discourse Studies specifically dedicated to this, 2011).

Needless to say, trying to spot non cooperative, persuasive, manipulative attempts is also central to media studies. Besides more traditional ways of analysing discourse, i.e. in terms of relations of power and in-/out-groups, a well-established paradigm of research in cognitive psychology is beginning to attract more and more attention from other scholars including those working on media. Media studies is now a field which crucially includes studies on the regularity of typical effects triggered in the media on the perception of the world and how the way information is shaped has a direct and predictable influence. Thus, scholars working within Communication Science have looked at the media-related counterparts of the cognitive biases mentioned in the previous section. Sundar (2007), Flanagin & Metzger (2007), or more recently Metzger & Flanagin (2013), look at the parameters affecting credibility and authoritativeness in online media. Sundar investigates the impact that interactivity has on information assessment and constructs an elaborate model that integrates dozens of so-called media effects which constrain the way information is interpreted (Sundar 2008). While this type of research does not focus primarily on deceptive uses it is very well equipped to explain the type of strategies that manipulators could resort to in order to achieve credibility, authoritativeness, etc. Interestingly, Communication Science is also responsible for starting a new chapter in this line of research as scholars have started investigating cultural variation in the kind of cognitively-induced effects discussed in this chapter. Hornikx et al. (2013), for instance, show how what would constitute an ad verecundiam fallacy (argument of authority) in Holland, would fail to trigger the same effect in Germany.

While these recent, detailed discourse-analytical applications of fundamental research on deception are nowadays at the centre of much scholarly work across disciplines, nevertheless it must be highlighted that these efforts are not new and that the relationships between language, beliefs, as well as social parameters were already the main issues underlying Viktor Klemperer’s own philological work, which set out to examine how the Nazi regime modified the German language in order to fulfil its ideological needs (see Klemperer 2006 [1947]). Purely linguistic processes such as neologisms, prefixes or lexical narrowing, and their interaction with macro strategies of repetition, censorship and media control, were among the mechanisms Klemperer extensively discussed to understand how the regime managed to spread its influence in communicative strategies, with a strong focus on the changes systematically implemented on the German lexicon (see for instance Klemperer’s insightful analysis of the concepts ‘fanatic’ and ‘hero’). It is directly in the continuity of these observations that contemporary discourse analysis, informed by cognitive science and communication science, can proceed to expose the pervasiveness of deception and other types of illegitimate communicative influence.

From a macro perspective, the directions of research mentioned above should be complemented with research about propaganda and how information spreads at the scale of a whole population. Work on epistemic vigilance (Sperber et al. 2010) has already started to look at how cultural and institutional constraints could be put in place in order to reduce the risks of misinformation and deception on larger communicative scales, far beyond interpersonal communication. Notably, two directions of research seem to arise naturally once we take this question into account. First, there is the issue of explaining why cultural information is relevant to some communities, and why and how it spreads. Although it has been investigated within cognitive science, this question will also need to take input from communication science, social psychology and media studies, as these are concerned with the way information spreads (see for instance Bangerter & Heath 2004 on the Mozart effect). Second, research on propaganda and the transmission of (problematic) cultural information also needs to specify how misinformation manages to overcome the epistemic filters already in place at an institutional level (peer reviewing in academic circles, editorial control in the media, reputation indexes online, etc.). There is much to be gained, in this perspective, from the interaction between the different disciplines.

4Conclusion

As it focuses on linguistic features of deception, the review of research on deceptive communication provided here falls short of covering other, non-linguistic, relevant aspects of scientific inquiry on the subject. Many linguistic, pragmatic and cognitive parameters involved in deception have been discussed herein, but para-linguistic aspects of information control remain to be detailed. Notably, from the perspective of communication at a population scale, it goes without saying that information control and information circulation strategies play a central role in the propagation of ideology and of deceptive messages. Censorship, for one, illustrates this kind of para-linguistic control. This is what happened in Nazi Germany since 1933, when the regime managed to achieve total press control (see e.g., Wilke’s (2005) insightful analysis). Whereas media control has been claimed to be more difficult to achieve in democracies because coercion is not an acceptable option in such societies (but see Herman & Chomsky 1988), this observation fuelled early research in critical linguistics (see section 2.3) such as Fowler’s (1991) Language in the news, who argued that news making is biased, despite the alleged independence of the press. This and many other issues touching upon how information is materially allowed to circulate are of crucial interest to complement linguistic research on deception.

If we now conclude by getting back to the interpersonal features of deceptive communication, we need to highlight that the social dimensions that play a role in the reproduction of ideologies are active only because there exists linguistic means to convey information, contexts in which information is processed, and inferences drawn by individuals – those people, individualised or not, to whom the communication is directed. All these factors have to be known so that the underlying mechanisms not only of verbal communication, but in particular of biased, non-cooperative, manipulative communication can be grasped. If a systematic one-to-one relation can be theoretically motivated, and experimentally tested, between a specific verbal stimulus, a set of verbal (or non-verbal) attempts at providing salience (foregrounding) to certain contextual assumptions or, on the contrary, at lowering it (backgrounding) and attested sub-optimal comprehension processes on the part of the addressee, compared to a control condition in which no such foregrounding-backgrounding constraint occurs, then the language sciences and the study of verbal communication – pragmatics – would definitely have something interesting to contribute to our understanding of deception and uncooperative communication in the real world.

References

Allot, Nicholas. 2005. The role of misused concepts in manufacturing consent: a cognitive account. In Louis de Saussure & Peter Schulz (eds.), Manipulation and Ideologies in the Twentieth Century: Discourse, Language, Mind, 147–168. Amsterdam/Philadelphia: John Benjamins.

Bangerter, Adrian & Chip Heath. 2004. The Mozart effect: Tracking the evolution of a scientific legend. British Journal of Social Psychology 43. 605–623.

Bavelas, Janet Beavin, Alex Black, Nicole Chovil & Jennifer Mullett. 1990. Equivocal communication. Newbury Park, CA: Sage.

Bond, Charles F. & Bella M. DePaulo. 2006. Accuracy of deception judgments. Review of Personality and Social Psychology 10. 214–234.

Carston, Robyn. 2002. Thoughts and utterances. The pragmatics of explicit communication. Oxford: Blackwell.

Chilton, Paul. 1994. Analysing political discourse: Theory and practice. London: Routledge.

Chilton, Paul. 2005a. Manipulation, memes and metaphors: the case of Mein Kampf. In Louis de Saussure & Peter Schulz (eds.), Manipulation and Ideologies in the Twentieth Century: Discourse, language, mind, 15–43. Amsterdam: Benjamins.

Chilton, Paul. 2005b. Missing links in mainstream CDA: Modules, blends and the critical instinct. In Ruth Wodak & Paul Chilton (eds.), A new agenda in (Critical) Discourse Analysis: Theory, methodology and interdisciplinarity, 19–53. Amsterdam: John Benjamins.

Copi, Irving M. & Carl Cohen. 1994. Introduction to logic, 9th ed. New York: Macmillan.

Correia, Vasco. 2011. Biases and fallacies: The role of motivated irrationality in fallacious reasoning. Cogency 3(1). 107–126.

Correia, Vasco. 2014. Biased argumentation and critical thinking. In Thierry Herman & Steve Oswald. (eds.), Rhetoric and cognition. Theoretical perspectives and persuasive strategies. Bern: Peter Lang.

DePaulo, Bella M., Deborah A. Kashy, Susan E. Kirkendol, Melissa M. Wyer & Jennifer A. Epstein. 1996. Lying in everyday life. Journal of Personality and Social Psychology 70. 979–995.

Dessalles, Jean-Louis. 2011. Reasoning as a lie detection device (Commentary on Mercier & Sperber:’Why do humans reason? Arguments for an argumentative theory’). Behavioral and Brain Sciences 34(2). 76–77.

Dijk, Teun van. 1993. Principles of critical discourse analysis. Discourse & Society 4(2). 249–283.

Dijk, Teun van. 2006. Discourse and manipulation. Discourse & Society 17(2). 359–383.

Ekman, Paul. 1985. Telling lies: clues to deceit in the marketplace, politics, and marriage. New York: Norton.

Evans, Jonathon St. B. T. & David E. Over. 1996. Rationality and reasoning. Hove: Psychology Press.

Fauconnier, Gilles & Mark Turner. 2003. The way we think: Conceptual blending and the mind’s hidden complexities. New York: Basic Books.

Flanagin, Andrew J. & Miriam J. Metzger. 2007. The role of site features, user attributes, and information verification behaviors on the perceived credibility of web-based information. New Media & Society 9(2). 319–342.

Fowler, Roger. 1991. Language in the News: Discourse and Ideology in the Press. New York: Routledge.

Fowler, Roger, Bob Hodge, Gunther Kress & Tony Trew. 1979. Language and Control. London: Routledge and Kegan Paul.

Galasiński, Dariusz. 2000. The language of deception. A discourse analytical study. 2000. The language of deception. A discourse analytical study. London: Sage.

Gigerenzer, Gerd. 2008. Why heuristics work. Perspectives on Psychological Science 3(1). 20–29.

Grice, H. Paul. 1989. Studies in the way of words. Cambridge, MA: Harvard University Press.

Grice, H. Paul. 1975. Logic and conversation. In Paul Cole & Jerry L. Morgan (eds) Syntax and Semantics 3 (Speech Acts), 41–58. New York: Academic Press.

Halliday, Michael A. K. 1973. Explorations in the functions of language. London: Edward Arnold.

Halliday, Michael A. K. & Ruqaiya Hasa. 1985. Language, context, and text: Aspects of language in a social-semiotic perspective. Victoria: Deakin University Press.

Halliday, Michael A. K. 1985. An introduction to functional grammar. London: Arnold.

Hamblin, Charles. 1970. Fallacies. London: Methuen.

Hansen, Hans V. & Robert C. Pinto. 1995. Fallacies. Classical and contemporary readings. University Park, PA: Pennsylvania State University Press.

Harris, Paul L., Kathleen H. Corriveau, Elisabeth S. Pasquini, Melissa Koenig, Maria Fusaro & Fabrice Clément. 2012. Credulity and the development of selective trust in early childhood. In Michael Beran, Johannes Brandl, Josef Perner & Joëlle Proust (eds.), Foundations of Metacognition, 193–210. New York: Oxford University Press.

Hart, Christopher. 2010. Critical Discourse Analysis and Cognitive Science: New Perspectives on Immigration Discourse. Basingstoke: Palgrave.

Hackett Renner, Catherine. 2004. Validity effect. In Rüdiger Pohl (ed.), Cognitive illusions. A handbook on fallacies and biases in thinking, judgement and memory, 201–213. Hove & New York: Psychology Press.

Hasher, Lynn, David Goldstein & Thomas Toppino. 1977. Frequency and the conference of referential validity. Journal of Verbal Learning and Verbal Behavior 16. 107–112.

Herman, Edward & Noam Chomsky. 1988. Manufacturing Consent: The Political Economy of the Mass Media. New York: Pantheon.

Hornikx, Jos & Margje ter Haar. 2013. Evidence quality and persuasiveness: Germans are not sensitive to the quality of statistical evidence. Journal of Cognition and Culture 13(5). 483–501.

Jackson, Sally. 1996. Fallacies and heuristics. In Johan van Benthem, Frans H. van Eemeren, Rob Grootendorst & Frank Veltman (eds.), 101–114. Logic and argumentation. Amsterdam: Royal Netherlands Academy of Arts and Sciences.

Jacobs, Scott, Edwin J. Dawson & Dale Brashers. 1996. Information manipulation theory: a replication and assessment. Communication monographs 63(1). 70–82.

Johnson-Laird, Philip N. 2006. How we reason. Oxford: Oxford University Press.

Klemperer, Victor. 2006 [1947]. Lingua tertii imperii. The language of the third Reich. London: Continuum.

Lakoff, George & Mark Johnson. 1980. Metaphors we live by. Chicago: University of Chicago Press.

Leech, Geoffrey. 1983. Principles of Pragmatics. London: Longman.

Levine, Timothy R. & Rachel K. Kim. 2010. Some considerations for a new theory of deceptive communication. In Matthew Knapp & Mark. McGlone (eds.), The Interplay of Truth and Deception, 16–34. New York: Routledge.

Levine, Timothy R., Rachel K. Kim & Lauren M. Hamel. 2010. People lie for a reason: Three experiments documenting the principle of veracity. Communication Research Reports 27(4). 271–285.

Lewiński, Marcin & Steve Oswald. 2013. When and how do we deal with straw men? A normative and cognitive pragmatic account. Journal of Pragmatics 59(Part B). 164–177.

Maillat, Didier. 2013. ‘Constraining context selection: On the pragmatic inevitability of manipulation. Journal of Pragmatics 59(Part B). 190–199.

Maillat, Didier & Steve Oswald. 2011. Constraining context: a pragmatic account of cognitive manipulation. In Christopher Hart (ed.), Critical discourse studies in context and cognition, 65–80. Amsterdam: John Benjamins.

Maillat, Didier & Steve Oswald. 2009. Defining manipulative discourse: the pragmatics of cognitive illusions. International Review of Pragmatics 1(2). 348–370.

McCornack, Steven A., Timothy R. Levine, Kathleen A. Solowczuk, Helen I. Torres & Dedra M. Campbell. 1992. When the alteration of information is viewed as deception: An empirical test of information manipulation theory. Communication Monographs 59. 17–29.

McCornack, Steven A. 1992. Information Manipulation Theory. Communication Monograph. 59. 1–16.

Mercier, Hugo & Dan Sperber. 2009. Intuitive and reflective inferences. In Jonathan Evans & Keith Frankish (eds.), In Two Minds, 149–170. New York: Oxford University Press.

Mercier, Hugo & Dan Sperber. 2011. Why do humans reason? Arguments for an argumentative theory. Behavioral and Brain Science 34. 57–111.

Mercier, Hugo. 2012. Looking for arguments. Argumentation 26(3). 305–324.

Metts. Sandra. 1989. An exploratory investigation of deception in close relationships: a consequence of violated expectations. Journal of social and personal relationships 6. 159–179.

Flanagin, Andrew J. & Miriam J. Metzger. 2013. Credibility and trust of information in online environments: The use of cognitive heuristics. Journal of Pragmatics 59(Part B). 210–220.

Mussweiler, Thomas, Englich Birte & Fritz Strack. 2004. Anchoring effect. In Rüdiger Pohl (ed.), Cognitive illusions. A handbook on fallacies and biases in thinking, judgement and memory, 183–200. Hove & New York: Psychology Press.

Nettel, Ana Laura & Georges Roque. 2012. Persuasive argumentation versus manipulation. Argumentation 26. 55–69.

Oswald, Steve. 2010. Pragmatics of uncooperative and manipulative communication. Neuchâtel: University of Neuchâtel dissertation

Oswald, Steve. 2011. From interpretation to consent: Arguments, beliefs and meaning. Discourse Studies 13(6). 806–814.

Oswald, Steve. 2014. It is easy to miss something you are not looking for: A pragmatic account of covert communicative influence for (Critical) Discourse Analysis. In Chris Hart & Piotr Cap (eds.). Contemporary critical discourse studies, 97–120. London: Bloomsbury.

Oswald, Steve & Christopher Hart. 2013. Trust based on bias: cognitive constraints on source-related fallacies. In Dima Mohammed & Marcin Lewiński (eds.), Proceedings of the 10th OSSA Conference, Virtues of argumentation.

Oswald, Steve & Marcin Lewiński. 2014. Pragmatics, cognitive heuristics and the straw man fallacy. In Thierry Herman & Steven Oswald (eds.), Rhetoric and cognition. Theoretical perspectives and persuasive strategies, 313–343. Bern: Peter Lang.

Pohl, Rüdiger (ed.). 2004. Cognitive illusions. A handbook on fallacies and biases in thinking, judgement and memory. Hove & New York: Psychology Press.

Raskin, Victor. 1985. Semantic mechanisms of humor. Dordrecht: Kluwer.

Saussure, Louis de. 2005. Manipulation and cognitive pragmatics: Preliminary hypotheses. In Louis de Saussure & Peter Schulz (eds.), Manipulation and Ideologies in the Twentieth Century: Discourse, language, mind, 113–146. Amsterdam: Benjamins.

Saussure, Louis de. 2013. Background relevance. Journal of Pragmatics 59(Part B). 178–189.

Saussure, Louis de. 2014. Présuppositions discursives, assertion d’arrière-plan et persuasion. In Thierry Herman & Steven Oswald (eds.), Rhetoric and cognition. Theoretical perspectives and persuasive strategies, 279–311. Bern: Peter Lang.

Schkade, Daniel & David Kahneman. 1998. Does living in California make people happy?. American Psychological Society 9(5). 340–346.

Serota, Kim & Timothy Levine. 2001. The prevalence of deception in American life. Michigan State University.

Sperber, Dan & Deirdre Wilson. 1995 [1986]. Relevance. Communication and cognition, 2nd ed. Oxford: Blackwell.

Sperber, Dan, Fabrice Clément, Christophe Heintz, Olivier Mascaro, Hugo Mercier, Gloria Origgi & Deirdre Wilson. 2010. Epistemic vigilance. Mind and Language 25(4). 359–393.

Sperber, Dan. 1985. On Anthropological Knowledge: Three Essays. Cambridge: Cambridge University Press.

Sperber, Dan. 1990. The epidemiology of beliefs. In Collin Fraser & George Gaskell (eds.), The social psychology of widespread beliefs, 25–43. Oxford: Clarendon Press.

Sperber, Dan. 1997. Individualisme méthodologique et cognitivisme. In Raymond Boudon, Alban Bouvier & François Chazel (eds.), Cognition et sciences sociales, 123–136. Paris: Presses Universitaires de France.

Sperber, Dan. 2001. Conceptual tools for a natural science of society and culture (Radcliffe-Brown Lecture in Social Anthopology 1999. Proceedings of the British Academy 111. 297–317.

Sperber, Dan. 2007. Rudiments d’un programme naturaliste. In Michel Wieviorka (ed.), Les Sciences Sociales en Mutation, 257–264. Auxerre: Editions Sciences Humaines.

Sperber, Dan. 2011. A naturalistic ontology for mechanistic explanations in the social sciences. In Pierre Demeulenaere (ed.), Analytical sociology and social mechanisms, 64–77. Cambridge: Cambridge University Press.

Sundar, Shyam S. 2007. Social psychology of interactivity in human-website interaction. In Adam N. Joinson, Katelyn Y. A. McKenna, Tom Postmes & Ulf-Dietrich Reips (eds.), The Oxford Handbook of Internet Psychology, 89–104. Oxford: Oxford University Press.

Sundar, Shyan S. 2008. The MAIN model: A heuristic approach to understanding technology effects on credibility. In Miriam J. Metzger & Andrew J. Flanagin (eds.), Digital media, youth, and credibility, 72–100. Cambridge, MA: The MIT Press.

Tindale, Christopher. 2007. Fallacies and Argument Appraisal, Cambridge: Cambridge University

Trew, Tony. 1979. Theory and ideology at work. In Robert R. Fowler, Gunther Kress & Tony Trew (eds.), Language and Control, 94–116. London: Routledge & Kegan Paul.

Turner, Ronny E., Charles Edgley & Glen Olmstead. 1975. Information control in conversations: honesty is not always the best policiy. Kansas Journal of Sociology 11. 69–89.

Tversky, Amos & Daniel Kahneman. 1974. Judgment under Uncertainty: Heuristics and Biases. Science 185(4157). 1124–1131.

Wason, Peter C. 1966. Reasoning. In Brian M. Foss (ed.), New horizons in psychology, 135–157. Harmondsworth: Penguin.

Walton, Douglas. 2010. Why fallacies appear to be better arguments than they are. Informal Logic 30(2). 159–184.

Wilke, Jürgen. 2005. Press instructions as a tool to manipulate the public under the German Nazi government – with an eye towards the German Democratic Republic. In Louis de Saussure & Peter Schulz (eds.), Manipulation and Ideologies in the Twentieth Century: Discourse, Language, Mind, 275–303. Amsterdam/Philadelphia: John Benjamins.

Wilson, Deirdre & Dan Sperber. 2004. Relevance Theory. UCL Working Papers in Linguistics 14. 249–287.

Wilson, Deirdre & Dan Sperber. 2012. Meaning and relevance. Cambridge: Cambidge University