Oct 14, 2014 | By

Arguing about Evidence

Our simple exploration of the concept of rationality has so far dealt primarily with the last two parts of the three part model for rationality: choosing the optimal outcome and reasoning logically. In neither case have we found clarity. Choice requires valuation, which engages our emotional faculties – and emotions are notoriously difficult to integrate with conscious thought, as so much remains hidden. And while reasoning logically and consistently may be feasible, the complete truth will never be accessible – and the universe itself does not appear to be consistent.

infinte spiralInfinite regress.

The first, and in this case, last step in our model of rationality is the question of evidence. The issues here are no less intractable. The simplest way of defining evidence would be to say that evidence is knowledge, and knowledge only includes true statements or “facts” about the world. If we can base our rational thinking on facts, then we can have confidence in the validity of our rationality. However, as pointed out in an earlier post on  knowledge, this standard is not feasible. Facts are not as easily determined as we might wish. Proving that any given statement about the world is true (proving a fact) requires an infinite regress of justifications – short of the infinite, a “consensus” is often possible, but there always remains some small but finite probability that any given statement is not factually correct. Significantly, consensus is impossible for many of the most interesting potential statements about the world – those that have preoccupied philosophers and theologians for thousands of years.

Putting aside this theoretical challenge, there are significant practical problems with evidence as well. Most of the time, even in the case of simple decisions, the facts that are accessible to us in making a decision are quite limited. My wife and I are planning a trip to Zion National Park – we have reviewed some literature and done some internet searches, but what we have examined is a tiny fraction of the available information. Reviewing all the available information would be infeasible as it would take more time than the trip itself!  Additionally, we are often faced with conflicting information. In the case of our Zion trip, if we check reviews from other travelers for tips we will find differing opinions. For any given decision problem, we have to decide how much investigation and review is enough – our evidence will never be complete.

zion caveImage from Zion National Park.

Deciding how much data is enough is critically important in evaluating scientific data, and there are sophisticated statistical methods for testing the validity of drawing a given conclusion from a given data set. However, there are many opportunities for errors, bias and even, unfortunately, fraud – while the peer review process in science is intended to police these issues, it is not foolproof, and an increasing number of flawed studies have been reported in recent decades.

A different evidentiary problem arises with more complex decisions. As explained in the post More Data is Not Enough, many questions are subject to varying degrees of uncertainty that serve as an insurmountable barrier to prediction. When looking historically, we may never be able to determine whether a particular past event was random or caused – and we correspondingly cannot predict the future in a given complex system even if we had unlimited access to data. Given that it is so notoriously difficult to forecast the weather, it should not be a surprise that predicting climate change is tough – particularly since the flap of a single butterfly could, according to chaos theory, affect the trajectory of a storm 1,000 miles away.

There is an additional limitation with respect to evidence that is often referred to with the aphorism: “The absence of evidence is not evidence of absence.” The idea (quote attributed to Carl Sagan) is that there may be a fact for which we have no evidence. For example, while we yet have no evidence of extraterrestrial life, it is clearly possible, perhaps even likely, that such life exists, given the ubiquitous nature of life on earth and the predicted existence of millions of earth-like planets. Technically, of course, the “absence of evidence” for a statement does provide some evidence that the statement may be false – but what is important is that the absence of evidence cannot be used as a proof that the statement is false. As a consequence, it is impossible to rule out, beyond any doubt, the possible truth of any statement that is not directly contradicted by evidence. It was once widely believed to be true that all swans were white and none were black – until black swans were discovered in Australia.

black swan

All of these tangled theoretical issues with evidence still leaves us facing the biggest disputes about evidence – how much weight do we give to specific pieces of evidence in our rational considerations – and are there some evidentiary claims that we reject entirely, for example by giving them a weighting of 0.

Join the Discussion

Why ask?