Wednesday, May 03, 2006

A view of evidence

In the last month I've discussed various aspects of the evidence-based debate. I'd now like to present some of my ideas on the subject. As I've suggested previously, the definition of evidence seems to lie at the heart of the dispute. But before I attempt to define evidence, I think it's useful to consider the word itself.

Evidence is the noun form of the adjective evident, ultimately deriving from the Latin evidens from ex- (out, forth) + videre (to see). The Oxford English Dictionary gives the following definitions for evident: "1. a. Conspicuous b. Obvious to the sight. 2. Clear to the understanding or the judgement; obvious, plain. 3. Indubitable, certain, conclusive. —1653." And the primary definition of evidence is simply "The quality or condition of being evident."

So a very literal interpretation would take evidence to be what can be seen, or "shown forth". It seems natural to extend this beyond vision to sense perceptions in general. Further metaphorical extensions bring us to the notion of clarity, certainty, and conclusiveness. And already the problem becomes apparent: we have moved from sense perceptions to things that might be considered similarly certain and conclusive. Like what? Deductive arguments? Accepted theory? Long experience? Expert opinions? Religious precepts?

Let's consider each of these in turn, starting with deductive arguments. A valid deductive argument is water-tight—provided that its premises are true. But how do we establish them? Perhaps we can depend on accepted theory. But history is littered with theories that were once universally accepted, but are now discredited or superseded. Of course there's always long experience. Experience is the cumulative product of personal practice and observation. But it is notoriously subject to selection bias (and perhaps other biases too). And when it comes to rare events, no amount of experience is sufficient. Can expert opinion step into the breach? The opinion of an expert represents a synthesis of many different sources of information, usually carried out over many years. While there may be many good reasons to trust an expert (such as his or her qualifications, intelligence, experience, and good standing in the community), well-meaning experts have been spectacularly wrong any number of times. Ultimately trusting expert opinion is an act of faith. Which nicely brings us to religious precepts as a source of certainty. These are perhaps the original "self-evident" truths—that is, to the believer, but perhaps to nobody else.

It is, of course, true that sense perceptions can be misleading too. A classic example is the straight stick that appears bent when placed in a glass of water due to the refraction of light. Any number of other illusions and hallucinations make interpretation of sense perceptions a thorny philosophical problem—one of the fundamental challenges in epistemology. Without wishing to minimize these, I will set them aside, by simply asserting that we know the external world through our sense perceptions. They are the only raw materials at our disposal, and they are the closest we can get to certainty about the external world. Evidence is observational.

But that doesn't mean that observation is all there is to evidence. For instance, a scientific experiment involves action (manipulating or controlling conditions) as well as observation. But if there's no observation, there's no evidence. To consider a particular example, this means that opinion doesn't count as evidence, but observation of opinion does! That is:
  • Opinion about X isn't evidence about X.
But
  • Observation of opinion about X is evidence about opinion about X.
I certainly don't mean to imply that opinion is worthless—that would be a bit self-defeating, now wouldn't it?—but it just can't be counted as evidence. In any case, regardless of what I or anyone else has to say, opinion will always guide us.

Ultimately, I think that opinion is most convincing when it is backed up—whether by deductive argument, theory, experience, evidence, or some combination of these. And I hope that my opinions have some of these supports.

While I haven't yet given a definition of evidence, I have presented what I think is a crucial qualification: evidence is observational. But there's a lot more to it than that!
Bookmark and Share

7 Comments:

Anonymous Mohammed-TA said...

Nick your post is much thought provoking indeed. I am not sure if I have grasped all of what you share here. So that I might learn from further discussions, in response I express my confusions:

When I am sitting in front of you and talking to you, you most likely know for sure that I exist. What is the evidence for this certainty?
Unless you become me, you cannot directly know the reality that is "Me". But your sensory perception is so strong that it removes all doubts that I do not exist. In other words, its your sensory perceptions of myself that lead you to know that I am real as opposed to the self of me that is real. Your perception is the evidence (a marker?) of reality. Then am I correct in concluding that evidence is actually an indirect representation of reality, but is not reality? Since representation must be true for it to be called evidence (otherwise it is nothing but assumed evidence), then may be evidence is some testimony or witness (in truth) of reality – something I have said before in a previous comment. Since it has to be "in truth", it assumes clarity and imparts certainty and assumes the meaning expressed in the Oxford dictionary.

You wrote "well-meaning experts have been spectacularly wrong any number of times", and I do agree with you mostly. But if I am not making a mistake here, you also seem to suggest that expert opinion is not observational. And since it is not observational, it is weak and is NOT evidence.

Here is a hypo:

A community of blind people has never known the green colour of grass. A unique person in this community appears who can really see. This person observes that grass is green, and so he opines. His opinion is evidence based and is observational. But this is not verifiable by other members of the community because they do not have this observational capacity.
Is it not possible, then, that some amongst us could evolve uniquely in a way that they acquire observational capacities (more wisdom, superior vision, deeper insights, better judgments, etc.) that are unique only to them? If yes, even if in theory, then perhaps communities could wrongly reject opinions, at least in some cases, when they have failed to verify those in observations of their own undertakings. And may be opinions based on such unique observational potentials are actually verifiable --- all is that is needed is acquisition of similar observational abilities.

10:41 PM, May 05, 2006  
Blogger Nick Barrowman said...

Mohammed,

I've strayed into philosophy, and have no doubt that I'm running roughshod over some exquisitely delicate flowers of knowledge, the product of a long history of careful thought about sense perceptions, reality, and knowledge. Perhaps the best I can hope for is to not make too much of a mess!

Then am I correct in concluding that evidence is actually an indirect representation of reality, but is not reality?

I think that's reasonably close to my view.

Since representation must be true for it to be called evidence (otherwise it is nothing but assumed evidence) ...

I'm not sure I agree. First, it's not clear to me what a "true" representation is. Second, I think that evidence can be misleading for a number of reasons, including the play of chance (i.e. bad luck).

... then may be evidence is some testimony or witness (in truth) of reality – something I have said before in a previous comment.

Yes—although I didn't touch on it here, I gave your previous comment some thought as I was putting this post together. In fact, I wrote a paragraph discussing the legal notion of evidence (and the special role of witnesses and testimony), and what light it might shed on a broader conception of evidence, but in the end I decided not to include it. Partly this was because I felt rather ignorant of the legal precepts. But it was also because the notion of a guarantor of evidence raised still more questions. I think you're right that it's a very important dimension.

... if I am not making a mistake here, you also seem to suggest that expert opinion is not observational.

That's correct. When I say "observational", I mean that observations are directly involved. While a particular expert's opinion may be (largely) based on evidence, I think that what you have is the opinion, not the evidence. Trouble is, there's no clear way to ensure that it's reliable.

Regarding your hypothetical scenario, it sounds a bit like Sherlocke Holmes to me. Holmes' powers of observation were so finely tuned that he was able to observe what others couldn't. The game's afoot!

12:48 AM, May 06, 2006  
Anonymous Mohammed-TA said...

"That's correct. When I say "observational", I mean that observations are directly involved. While a particular expert's opinion may be (largely) based on evidence, I think that what you have is the opinion, not the evidence. Trouble is, there's no clear way to ensure that it's reliable"

I think one point that I wanted to make, but was not explicit about, is that an individual opinion also, and almost always, stems from reproducible observations by the individual (over time that we call experience), regardless of associated bias. The pre-supposed fear that it may not be reproducible by others in the community (because in the past, at times, evidence from individual observation could not be reproduced by others) makes it unreliable and unverifiable. Then we have no choice but to limit individual opinion to hypothesis generation.

However, the point I did make was that there could be an alternative explanation for the lack of reproducibility and confirmation of evidence by other members of the community at least in some circumstances -- collective absence of some important and unique tools of observation in the community that were present in the unique individual.

Further, reproducibility means reliability to us because we have this natural instinct to observe personally. But reproducibility is not always established in practice – in fact research funding practices discourage it. However, when we realise that variability does exist in observational potentials within us, then we begin to believe in observations of others and we start trusting them (regardless of whether they should or should not be trusted). In the end, for people evidence ultimately does become individual (coming from a select group of researchers for example) since it is not possible to generate it every time by everyone. Faith in evidence generated by others is then built upon assumed trust, since many of us do not have the patience for it to get established and the ability to recognise it when it has.

Its all very confusing to me, and I hope this discussion will take away some of it.

7:43 AM, May 06, 2006  
Anonymous Mohammed-TA said...

Just now I read this last post from me again -- simply poor!

So I am going to attempt again to express my question.

Almost every person continually observes and inquires. In drawing inferences, however, most err.

But it is possible for an individual to be superior to others in observation, be more systematic and cautious in the process, come to witness its reproducibility, and draw guarded inferences from it.

So whatever the mysterious reality is, and whatever is its true evidence, isn't this individual experience worthy of consideration as possible evidence, or is it still nothing more than "opinion"?

10:03 PM, May 06, 2006  
Blogger Nick Barrowman said...

Mohammed,

I agree with the points you make, but I think I interpret their implications a bit differently.

So whatever the mysterious reality is, and whatever is its true evidence, isn't this individual experience worthy of consideration as possible evidence, or is it still nothing more than "opinion"?

I certainly agree that it's worthy of consideration, but not as evidence. As I see it, the key difference is that one can (at least attempt to) evaluate evidence in a systematic, objective fashion. Opinion, on the other hand, can only be evaluated in a subjective way. As the saying goes, beauty is in the eye of the beholder. I think this applies equally to opinion.

Then we have no choice but to limit individual opinion to hypothesis generation.

Yes, I think so. But hypothesis generation is no small thing. Hypotheses guide the questions we ask, the studies we conduct, and the models we use to evaluate the data we observe.

12:43 AM, May 07, 2006  
Anonymous Mohammed-TA said...

In the end, I am back to where I was -- I can't reasonably disagree with you.

Individual experience is best considered as "opinion" or claim unless the individual can clearly demonstrate that his/her experience is based upon systematic, scientific, methodologies that reduce bias. In other words, that his/her experience is scientific research, and inferences drawn beyond conjecture.

Did James Lind base his findings on experience or research?

For some people, perhaps like Lind, both are one and the same thing.

11:47 AM, May 07, 2006  
Blogger Nick Barrowman said...

Not sure I know enough about James Lind to answer your question, but I believe his conclusions were based on studies he conducted. (For the benefit of other readers, James Lind (1716—1794) was a pioneer in the development of clinical trials. The James Lind Library, named after him, is well worth checking out.)

10:49 AM, May 08, 2006  

Post a Comment

<< Home