Fle.am

While working with the LiDA 101 course, I was introduced to the concept of ‘source evaluation’, which, in my opinion, does not hold for any kind of research but historical.

By extension, the Robert Harris' CARS checklist for source evaluation, which was recommended by the course authors, is a rather misbegotten attempt at formalising the important qualities of the documents due to several reasons. Moreover, this attempt is directly misleading.

First, such qualities as ‘Credibility’, ‘Accuracy’, ‘Reliability’ and ‘Support’ cannot be easily quantified, and their very definitions will be subject too many a bias so to pronounce them even as a rule-of-thumb framework. We shall discuss this question In a little more detailed fashion a few paragraphs below.

Second, the underlying criteria for the ‘Accuracy’ part are not just nebulous, they are outright incalculable at best, and they are reinforcing bias through group-think at worst. This leads to the whole part of the framework being just a waste of time. Take this as an example:

Some work is timeless, like the classic novels and stories, or like the thought provoking philosophical work of Aristotle and Plato. Other work has a limited useful life because of advances in the discipline (psychological theory, for example), and some work is outdated very quickly (such as technology news).

This very definition of so-called Timelessness shows not only a certain group bias (for the sake of illustration, if I were a writer in a Jim Crow laws era, I would pronounce the works of Ku Klux Clan founders as ‘timeless’, because there once was a rather universal consensus on the racial question in the USA, especially in the ex-confederate states. On the other hand, when considering the works of anti-racists ‘timeless’, would not we just bow to the modern taste?).

The other criterion, namely one of Comprehensiveness cannot prevent from personal biases either. Comprehensive according to whom? If according to the researcher – here one's biases bloom. According to his tutor – good luck with importing someone else's point of view into one's creative work. According to the academic consensus – see the paragraph above and the previous sentence. (And don't forget that the ‘textbook definition’ is not always the ‘academic consensus’ due to the time lag.)

Third, one, of course, cannot – and I dare say shouldn't be – fully impartial. The point is to understand the biases and the positions of the authors and indicate your own ones to the public. So such check-lists are, to my mind, more dangerous than useful in trying to assess the information – on-line or off-line, for they create a false sense of security: if the author's hidden agenda matches your preference, you will simply let it through.

It leaves a question – then how we should evaluate the information in which we live and breathe? How to prevent being duped by fake news and on-line manipulators? Well, there is no easy answer, no check-list (because as soon as you have a set of fixed criteria, someone can push his agenda through). So think, just think, dammit, don't be afraid of making a mistake, and don't be afraid of admitting one while being beaten at a dispute! Critical thinking is about arriving at a reasonable conclusion without sticking to the Ultimate Truth, but it is also about the culture of discussion and the ability to change.