EVALUATION and EVALUATORS
In general, evaluating an argumentative discourse involves making a reasoned positive or negative « value judgment » about it. Like any other argumentative activity, evaluation can be misleading or well-founded, regardless of the quality of the discourse it approves or condemns. To avoid being arbitrary, evaluators must specify their methods, criteria, and scales of reference. As with any other argument they must be open to criticism.
1. Dimensions of Evaluation
1.1 Scales of Evaluation
Arguments can be evaluated on the basis of different types of scales
An efficiency scale — The best argument is the most effective one, that is, the one that best orients its recipients to thesis it defends, or the action it advocates.
A scale of logical-scientific validity — On such a scale, good arguments are valid deductions, that start from true premises and carry their truth to their conclusion according to valid rules and methods.
Invalid arguments are misleading. Effective arguments may also be misleading. Indeed, effective arguments are systematically suspected of being misleading.
Conversely, a valid argument may be ineffective. For example, “P, therefore P” is a valid deductive inference, yet it has no persuasive power. Ordinary arguments often disguise this kind of truism by using two different formulations of the same proposition P “P, therefore (paraphrase of P)”, see vicious circle.
Trains never leave before their scheduled departure time. Taken to the letter, the following is not a valid justification for the delay: « Because of the delay, the train will not leave on time. » Considering the context and the meaning of the speaker’s intent, the information is clear « because of the accumulated delay before arrival, etc. The station agents are not responsible for the delay.
1.2 Binary and Stepwise Evaluation
— Binary evaluation classifies arguments as either valid (good, accepted) or invalid (bad, rejected). This type of evaluation adheres to the rules and criteria of formal logic.
It requires translating the argument from ordinary language into a logical language. This logical characterization, which is considered to express the argument’s essence, is then evaluated. Finally, this logical evaluation is transferred to the original discourse, see connectives.
— Gradual evaluation positions the argument on a scale, ranging from bad to good.
In practice, the evaluation criteria depend on the quality of the set of critical questions used for the evaluation of the argument scheme under consideration.
The argument under evaluation is then examined based on each condition. Since these critical questions may be quite heterogeneous, the overall evaluation may only be a precarious synthesis of the results of these different operations.
2. The Diagnosis of Fallacy
The imputation of fallacy is an adversarial procedure by which a discourse is condemned, rejected, or disqualified. According to the principle of « no execution without representation, » the accused arguer has the right to defend their argument. Discussions about the fallacious nature of argumentation are open-ended and their conclusions are subject to challenge and correction. Like any other arguments, these discussions are possibly fallacious themselves. In any case, meta-argumentative disputes about the evaluation of arguments evaluation provide interesting data for argument analysis.
Evaluators
Who evaluates the arguments? Hamblin gives a clear answer to this question: the logician is not the arbiter of the argument or dispute (Hamblin 1970, pp. 244-245).
Consider, now, the position of the onlooker and, particularly, that of the logician, who is interested in analyzing and, perhaps, passing judgment on what transpires. If he says “Smith’s premises are true” or “Jones argument is invalid”, he is taking part in the dialogue exactly as if he were a participant in it ; but, unless he is in fact engaged in a second-order dialogue with other onlookers, his formulation says no more than the formulation “I accept Smith’s premises” or “I disapprove of Jones’s argument”. Logicians are, of course, allowed to express their sentiments but there is something repugnant about the idea that Logic is a vehicle for the expression of the logician’s own judgments of acceptance and rejection of statements and arguments. The logician does not stand above and outside practical argumentation or, necessarily, pass judgment on it. He is not a judge or a court of appeal, and there is no such judge or court: he is, at best, a trained advocate. It follows that it is not the logician’s particular job to declare the truth of any statement or the validity of any argument.
While we are using a legal metaphor it might be worthwhile to draw an analogy from legal precedent. If a complaint is made by a member of some civil association such as a club or a public company, that the officials or management have failed to observe some of the association’s rules or some part of its constitution, the courts will, in general, refuse to handle it. In effect the plaintiff will be told: “Take your complaint back to the association itself. You have all the powers you need to call public meetings, move rescission motions, vote the managers out of office. We shall intervene on your behalf only if there is an offence such as a fraud.” The logician’s attitude to actual argument should be something like this.
The diagnosis of fallacious speech operates on a meta-argumentative level, but this second level does not transcend the dialog under scrutiny, it remains an integral part of the argumentative game. In other words, the judgment « this argument is fallacious » works in the same way as any other ordinary refutation, whether it is made by a participant (ordinary use of the word fallacy) or by an analyst, who then acts as a participant. One must then speak of an ad fallaciam argument.
In a letter to Edmond Schérer, the economist Léon Walras refers to a controversy between Edmond Scherer himself and Adolph Guéroult:
I take […] your [= Scherer’s] study of December 30 to the point where you […] clearly and plainly address the more general considerations about the divergence between his [Guéroult’s] opinions and yours.
“Perfectibility, you say is a modern idea, one of those that best indicates the distance between the old world and the new world. It bears its own self-evidence, so that its opponents are but a few sophists or a few misanthropes. It has become into the general law of intelligence. But perfectibility must not be confused, as M. Guéroult sometimes seems to do, with the possibility of perfection. This confusion is not merely a matter of words; for those who understand the scope of the questions, it marks the dividing point between two systems, liberalism and socialism. Socialism, reduced to its principle, is nothing other than the belief in the possible perfection of society and the effort to realize this state.”
This is clear and precise. Mr. Guéroult and you agree up to a certain point: for both of you, humanity advances and does not retreat; the law of the development and organization of society is a law of progress and not of decadence. Beyond these limits, you part company, you think that society is only perfectible, while M. Guéroult, for his part, thinks that society, will sooner or later, will be perfect; you are a liberal, M. Guéroult is a socialist. Perfectibility or perfection, liberalism or socialism, such is the alternative and the question that is raised. (Léon Walras, [“Socialism and liberalism”], [1863][1])
In this final paragraph, Walras summarizes the debate between Schérer and Guéroult. According to Walras, Schérer argues that Guéroult deduces from the possibility of the perfectible–point on which they agree–the possibility of the perfect (point on which they disagree). The passage « things are perfectible, therefore they can be made perfect » is a typical argument based on derived words.
Walras, speaking for Schérer, does not consider this inference to be a sophism (he does not attribute to Guéroult the intention of misleading his readers), a fallacy of “confusion”, not even an error. He makes a factual observation. Walras does not take an external logical evaluative point of view, he speaks as a politico-economic analyst who emphasizes this point as a subissue of the larger debate, “Liberalism or Socialism?”
For a laissez-faire approach in argumentation
Arguments are conducted within specific domains by communities of speakers. These communities correspond to what Hamblin calls “civil associations”, having their own interests, programs, ways of thinking and rules for deliberation and action. In these domains, logicians as such do not have the substantial expertise required. This observation lies at the heart of “critical liberalism”, which advocates a laissez-faire attitude toward argumentation.
From this point of view, what of evaluation? First, the data to be considered for the evaluation should not be limited to the one isolated argument under scrutiny, but consists of a well-defined selection of contradictory discourses developed around the same issue, see fallacy 1.
Second, following Hamblin, the evaluation of the arguments is entrusted to the “civic association” which is defined as the community interested in the outcome of the issue being evaluated.
Consequently, the evaluation process can be empirically documented and critiqued on three levels:
First, the evaluation practices of the participants, such as such as concessions, objections, refutations and counter-discourses in general.
Second, the emergence of a specific ordinary critical metalanguage, such as accusations of fallacy, misplaced authority, irrelevance, emotionality, amalgamation and impugned motives, etc. (Doury 2000).
Third, evaluations carried out by the specialists in the field. This level, which includes scientific expertise, and is the ultimate level of evaluation. Scientists routinely evaluate their colleagues’ discourses and fallacies; historians evaluate other historians’ fallacies (Fisher 1970), and teachers and students evaluate each other’s arguments.
These activities are all « [meta-argumentative] », as opposed to « ground-level argumentations » (Finocchiaro, 2013, p. 1). If the intervention is useful and desired, argument analysis specialists can intervene at all levels. As Hamblin has explained, the specialist’s role and deontological position is that of a « well-trained advocate. » Specialists can evaluate all the arguments in the world, adopting the stance of a participant analyst and evaluator. They work under a double constraint of externality (observer) / internality (participant), which is well known in ethnomethodology.
When their presence is required, they can be useful in court as juris-logicians or juris-linguists, that is, as advisors to a party or the judge, but certainly not as substitutes for the latter.
Argumentative discourse is evaluative and critical in itself; scientific evaluation is a process of argumentative expansion and deepening of the issue. There is no super-evaluator who can put an end to the critical process by providing a final, conclusive, evaluation silencing all other participants.
[1] Quoted from Léon Walras L. (1896). “Socialisme et libéralisme”. In Études d’économie sociale – Théorie de la répartition de la richesse sociale Lausanne: Rouge & Paris: Pichon. P. 4.
[« Socialism and liberalism. » In Social economics studies – Theory of social wealth distribution.]