Thursday, April 06, 2006

Evidence-based ambiguity


In the last 15 years, evidence-based medicine has taken the world by storm. According to a famous definition, evidence-based medicine is
"the conscientious, explicit, and judicious use of current best evidence in making decisions about the care of individual patients."
In this spirit, the evidence supporting many time-honoured practices in medicine has been examined, and in a number of cases found wanting. (For an informative and entertaining look at this, see this slide presentation by former British Medical Journal editor Richard Smith.) The exalted status that expert opinion once enjoyed is waning. Today the cry is "Show me the evidence!" (cf. Jerry Maguire.)

If you'll pardon the pun, the success of evidence-based medicine has been infectious. The prefix "evidence-based" is popping up not just in connection with healthcare: today there is evidence-based education, evidence-based software-engineering, evidence-based librarianship, and the list goes on. The "evidence-based movement" seems victorious.

But there are stirrings of discontent. From the beginning, evidence-based medicine has had its critics. (See this editorial for a balanced account of their objections.) A key issue relates to the ambiguity in the word "evidence". If it means empirical evidence, it would seem that clinical experience, pathophysiological theory, patient values, and expert opinion have no role to play. Alternatively, evidence can be defined broadly: "evidence is anything that establishes a fact or gives reason for believing something" (The Oxford American Dictionary, via this report). But this "colloquial" definition opens the doors so wide as to be useless here. For example, a religious argument might be compelling for the believer, but surely would not constitute "evidence" for the present purposes. For some other interesting perspectives on the definition of evidence in evidence-based medicine, see this essay by Amanda Fullan (an undergraduate student at the time).

In recent years the evidence-based movement has expanded to areas such as public health and policy. In a 2004 essay titled What is Evidence and What is the Problem?, the Acting Executive Director of the American Psychological Association writes
"These days, you can hear the terms “good science”, “evidence”, and “data” a lot in Washington. One of the catch phrases around policy-making circles is “evidence-based”, applied to a host of contents including education, policy, practice, medicine, even architecture. You would think that this would make us all quite happy – at least those who advocate that decisions about policy, social interventions, and future directions be based on data. But, ironically, the new emphasis on evidence-based this and that has been simultaneously welcomed and greeted with raised anxiety levels and red flags of concern."
And the ambiguity of the word "evidence" is even more problematic in this context:
"It is clear that discussions of definitions of evidence, distinctions among kinds of evidence (including scientific data, expert judgment, observation, and theory), and consensus on when to use what, will occupy us for some time."
The Canadian Health Services Research Foundation (CHSRF) has recently grappled with these issues, issuing a report, and holding a workshop. One of the "key messages" from the workshop was that
"Although the literature shows that decision makers work with a colloquial understanding of evidence (often alongside a scientific understanding), some participants felt strongly that the information classified as colloquial evidence should not be called evidence. They acknowledged the importance of this information but suggested finding a substitute term, such as “colloquial knowledge” or “colloquial factors.”"
Finally, the CHSRF adopted the following (rather extended) definition:
"Evidence is information that comes closest to the facts of a matter. The form it takes depends on context. The findings of high-quality, methodologically appropriate research are the most accurate evidence. Because research is often incomplete and sometimes contradictory or unavailable, other kinds of information are necessary supplements to or stand-ins for research. The evidence base for a decision is the multiple forms of evidence combined to balance rigour with expedience—while privileging the former over the latter."
Hmmm ... not entirely convincing, but I see what they're getting at. But where did they get that stuff about coming "closest to the facts of the matter"? I'd say it's either begging the question or using a circular argument.

Epilogue: In their latest newsletter, the CHSRF announce that they've decided to abandon the term "evidence-based":
"Following feedback and discussions at the “Weighing Up the Evidence” workshop in September 2005, the mission of the foundation has been changed to better reflect the emerging realization that research is justifiably only one, albeit very important, input to decision-making."
The new term? Evidence-informed.
Bookmark and Share

3 Comments:

Blogger Raywat Deonandan said...

My clientele --community based youth mental health agencies-- is quite pleased with the introduction of "evidence informed" as the nomenclature of record, since it seems to suggest that empirical evidence is but one factor in determining what constitutes a best practice; personal wisdom and experience are given some weight in this model.

I have yet to decide how I feel about this.

4:02 PM, April 07, 2006  
Blogger mamegann said...

The new motto of the Halifax Regional School Board: Data Driven Decisions. Very inspiring. Sure makes me want to bounce out of bed Monday morning to teach music.

11:32 PM, April 07, 2006  
Anonymous Mohammed-TA said...

Greetings,

Very interesting and difficult topic. Before I even begin to write, I must acknowledge that I am confused -- and very much.

It would be great to begin with the word "Evidence". To dissect its linguistics. What is its literal meaning? Origin or root?

My working definition:

Evidence is testimony that something is either true or false. If this is established actually in truth and correctness, then assumed evidence is real evidence; otherwise it was an illusion/delusion of it. Since knowing what actually is real and true is most of the time beyond human endeavour, most evidence is in essence "assumed evidence". I hope this does not beg the question.

I posit:

In the absence of trust, evidence is meaningless. Before one could trust evidence, there should first be evidence of trust.

To me evidence-based medicine is at best, evidence-seeking medicine.

I can’t think of a scenario where evidence may be accepted without establishing trust on those who provide it, and those who somehow influence it.

That my dad is actually my dad, might be established (without reasonable doubt) by DNA match. However, unless it is I myself running the test, I’ll have to trust the technologist and/or the laboratory that carried it out.

Most of the time such trust is "assumed" and not established.

That evidence must be sought without bias and systematically is important, but equally important (if not more) is the seeking of trustworthiness of individuals associated with that evidence.

If knowledge serves me right, in certain times of old and in certain regions of the world, trust between a Master and a student HAD to be established before learning could even begin.

Trust and evidence, the wise ones knew, couldn’t possibly be separated.

In our times, there is much writing. Writing, writing and more writing. Emphasis is on appearing right in writing, as opposed to being right in conduct.

Theoretically, personal observations, experiences, and understandings of one with extraordinary vision can hold more weight than empiric observations of many with ordinary observations.

Many important scientific theories upheld today, are mere personal understandings that were perceived by extraordinary minds.

However, if one chooses not to consider trust in the science of evidence, then of course individual observations fall short tremendously compared with empiric evidence.

"Contact" is one movie that always come to mind when people ask of my favourite movie – starring Jodie Foster based on a book by late Carl Sagan.

Again it seems I have lost focus. Time to listen before any more rubbish.

7:38 PM, April 08, 2006  

Post a Comment

<< Home